Three vendors responded to our RFP: Endeavor, Ex Libris, and Sirsi. In addition to the traditional elements of evaluation--review of the vendor responses to the RFP, verification of the responses through visits and calls to existing customers, demos by the vendors, and follow-up questions to the vendors--the Evaluation Team worked with the Functional Sponsors to add patron input to the process. We invited faculty and students to attend vendor demos but found, as expected, that few were able to attend. Clearly, we needed to pursue other methods for gathering faculty and student assessment.
Our patron partners pointed out that some of our survey questions were 'rather obscure.'
Another good source of input was queries by faculty and graduate students to their colleagues at other universities where the three vendor systems were used. Talking to other librarians is a time-honored and effective way to gather information about the strengths and weaknesses of a system, since colleagues will often speak candidly about their experiences. The same is true when faculty ask faculty, or grad students ask grad students. As is often the case with other librarians, faculty and student perceptions of a new system are colored by what system was in use before. Allowing for that bias, the perceptions of patrons who use a particular system are invaluable--direct, unfiltered, and based on their daily needs.
Our final method for gathering user input was holding a series of focus groups to discuss trade-offs among the three systems. There is no one vendor system that has only strengths, so every system selection involves weighing the relative importance of strengths and weaknesses. The Evaluation Team distilled the major trade-offs and reviewed them with the library staff and patrons. For the user review, we hosted a faculty and student focus group where we demonstrated the differences that we believed would have the most impact on them.
What Would We Do Again?
All of these consultation and outreach methods helped us raise campus awareness of the ILS selection process and gave faculty and students a way to feel engaged in the decision. We would probably repeat all of them for that purpose alone. Some of the methods, though, yielded input that had particular impact on our decisions and approach.
• The Functional Sponsors were excellent partners, and we would recommend such a faculty and student advisory group to any university library that's evaluating and selecting a system. To be effective, the members of such a group should be active library users who care about the outcome, should reflect a mix of disciplines and areas of expertise, and should be sufficiently experienced to bring perspective and credibility to the group.
• Tapping faculty expertise on technology and database design was very helpful and we would recruit such advice again. To be most effective, such an expert should have a mix of scholarly and real-world perspective. We were fortunate to have a faculty member who had extensive consulting experience with corporate clients as well as impressive scholarly credentials.
• Having faculty and students test drive the systems deployed at other institutions was a wonderful way to involve patrons in concrete evaluations. To be more effective, if we did this again we might structure the feedback survey more tightly, and we might be more aggressive in recruiting test drivers.
• Having people conduct "reference checks" among their colleagues at other universities was very helpful. We asked only the Functional Sponsors to participate in this process. To be more effective, we would want to recruit more faculty and graduate students to get a broader set of comments.
Understanding Patrons' Views Influenced Our Selection
This assessment process reaffirmed for us that involving patrons in system selection is both crucial and challenging. By focusing on the real differences among the systems and asking faculty and students questions that reflected their approach to the catalog, we were able to elicit thoughtful feedback. With a committed, talented user advisory group and a willingness to see the system selection process through their eyes, you, too, can turn your patrons into valuable partners in the system evaluation and selection process.
No comments:
Post a Comment