College and Research Libraries Research Notes Archimedes: Analysis of a HyperCard Reference Tool J. S. Ottaviani This article describes Archimedes, a set of HyperCard stacks running on a network in the Engineering Library at the University of Michigan. Archimedes works as a reference alternative for library users when a librarian is unavailable. This article discusses the design philosophy of the stacks and presents descrip- tive statistics and analysis of the first year of use by the public. The data demonstrate the ease of anonymously and unobtrusively monitoring system use, support the design philosophy of the multistack networked approach, and indicate that library users consult Archimedes for reference information. The data afford a seldom-seen glimpse of the information-searching process library users employ. By recording the sequence of steps they take when left to their own devices and noting where that search process ended, the data give some indication of what people want from a reference interview and when they think they have received if. • ith the introduction of Hyper- Card in 1987 a new group of Macintosh users began to de- sign applications for them- selves and their clients. HyperCard provides a programming (scripting) envi- ronment that bypasses the complexities of coding the Macintosh user interface. It al- lows scripters to conceive, develop, and test ideas in a relatively short time. The resulting stack(s) can employ text, graph- ics, sound, animation, and hypertextual features. HyperCard has obvious attractions for library applications. Librarians with modest programming expertise can de- velop custom applications for their us- ers and themselves. Regular features on HyperTalk programming, critical reviews of HyperCard products, and reports of experience with home-grown applications are now common in the J.S. Ottaviani is head of Reference at the Engineering Library , The University of Michigan, Ann Arbor, MI 48109-2136. Readers can get copies of the Archimedes stacks and related materials from the UM Engineering Library. Contact us at: engin.lib.ref@umich.edu. Visit us via the URL http:/ /www.en- gin.umich.edu/library /ELHP.html. Thanks go to all of the other members of the reference staff at the Engineering Library (but especially to Bob Schwarzwalder, Gene Alloway, Fred Gilmore, and Jill Holman) for their vision, hard work, comments, and critiques of my contributions to the Archimedes system throughout the process of designing and writing about it. 171 172 College & Research Libraries library literature. Monica Ertel and Jane Oros provide a concise overview of what HyperCard can do and how those in an academic library setting can use it. 1 Carl Franklin compiled a bibliography on hypertext and hypermedia sources that provide an overview of the field, and with L. Bessmer provides pointers to Hy- perCard-specific applications.2.3 The staff at the University of Michi- gan's (UM) Engineering Library (EL) be- gan the Archimedes project in 1989. Funding for the project came from the University of Michigan Library System, and development began in January 1990. The EL reference staff, which includes both professionals and students, did all of the design, programming, and imple- mentation of the stacks. The Archimedes system has been used by the public since March 22, 1991. It consists of two Macintosh Plus comput- ers linked by a PhoneNet network to a Macintosh SE/30. Currently seven Hy- perCard stacks comprise Archimedes: March 1995 ELSystem,ELLocations,Mirlyn,Special Collections, Other Libraries, Reference Help, and Services. Each stack consists of a number of cards. These cards con- tain text, graphics, animation, and buttons used for navigating through Archimedes and uncovering more infor- mation. The HyperCard metaphor, in which a stack equals a pile of index cards, indicates only part of the program's power. The links between each card and the conditional branching capabilities programmed into these links move Hy- perCard beyond a pile of cards and be- yond a flat-file database as well. For example, when users select an area on the floor map, they move directly to a description of that area, which may re- side on another card, or even in another stack. After viewing the new informa- tion, · they can return to the map or move 'on to more details without a no- ticeable transition and without having to remember where they were in the hierarchy. --- ~~ ~------~ ==---=-------::::::::..::..~~ ARCHIMEDES ..IlL --, .... UNIVERSITY OF MICHIGAN ENGINEERING LIBRARY Click on the topics below for more information ~ Reference Help 'V Servires ~ Library Locations ~ FloorMap W Special Collections FIGURE 1 Archimedes' Main Menu Card A sample card is presented in figure 1. There are six opening cards; when shown in succession, they simulate a spin- ning globe. The user clicks on the globe to begin using Archimedes. Figure 1 shows the main menu that appears after the ani- mation. By clicking on one of the icons/la- bels, the user moves to a stack containing information on the Chosen topic. Archimedes covers both general fea- tures of the UM library system and de- tailed information tailored to EL users. 4 It provides quick reference help to li- brary users when reference librarians are not available-as such it runs whenever the library is open. DESIGN PHILOSOPHY Reference is one of the most visible services our library provides. However, finite resources translate directly into fi- nite staffing and hours, so personal serv- ice from 8 a.m. to 2 a.m. (EL library hours during fall and winter semesters) is im- possible. The complexity of the univer- sity library system, and the EL in particular, led us to consider alternatives to personal service during nonreference hours. The requirements for this alterna- tive help system include: • Multiple sites/many access points, • An interactive system allowing users to search for the information of their choosing, in the order they choose, • A reference tool that has something to offer to users at any level of library literacy, • A visually interesting system, and • A system we can modify and update in-house as needs change. These and other more mundane con- siderations (e.g., time and money con- straints) produced a set of guidelines and a direction for the reference aid that became the Archimedes system. We selected topics for inclusion in the system based on an analysis of the types of questions and needs EL's users have. Not intended as a substitute for a refer- ence librarian, Archimedes serves in- stead as a reference aid capable of answering simple, factual questions and providing information about library services. When possible, solutions to Archimedes 173 common problems handled by librarians (e.g., how to find a missing book, how best to search for conference proceed- ings) also appear. As a by-product of developing the system, the designers learned a great deal about how they solved problems. Occasionally writing procedures down resulted in streamlin- ing them to produce better results. The EL staff grouped related topics in sepa- rate stacks to allow easy updates. This modular approach has allowed us to eas- ily revise our Mirlyn stack to reflect changes in search screens resulting from implementing new versions of NOTIS system software and adding database files, without bringing the whole system down during the revision. As noted in the introduction, we chose HyperCard as a development environ- ment because of its flexibility, power, and ease of use. Most of the development team had little or no programming expe- rience when the project began. Those who wished to avoid scripting in Hyper- Card made significant contributions without delving into the HyperTalk lan- guage. Most quickly learned some scripting, however, and added advanced features to the stacks they designed. Workshops available at UM through its Information Technology Division also brought those interested in more ad- vanced techniques quickly up to speed. Design of a human-computer inter- face, especially in the hands of a mixed group of programmers and nonpro- grammers charged with diverse tasks, will probably not succeed without some kind of guidelines. Apple Computer provides some that address general is- sues involving a hypertext environ- ment.5 But because the EL and other libraries at UM continue to modify and run versions of the stacks, we drafted a set of guidelines written specifically for Archimedes. It covers cards, buttons, fields, and sounds, and also provides references and sample scripts. The guidelines serve two purposes. Stacks designed according to specifica- tions have a consistent look and feel. When designers use typefaces, special ef- fects, and especially navigation buttons 174 College & Research Libraries consistently, the user does not constantly struggle to learn about a new stack and can concentrate on its content. Guide- lines also free stack designers from too many choices. While this may seem unnec- essarily confining, a free-form environ- ment like HyperCard's can easily move the focus away from providing informa- tion in a clear and interesting way. A network environment enabled us to create an inexpensive but easily expand- able automated reference aid. Also, us- ing Macintosh Plus computers without hard disks allowed us to place two inex- pensive workstations on the floor in- stead of one dedicated and expensive machine. Further, designing the system introduced the EL staff to the basics of networking. Because they did every- thing from installing the software to run- ning the wire, many are now familiar enough with the· network to act as sys- tem operators. THE DATA ANALYSIS When a user initiates a session by clicking the mouse, a file opens to cap- ture the sequence of stacks and cards they visit. The time of day and time spent March 1995 in each stack also get recorded. We make no attempt to identify the user, and re- cording the session transcript occurs un- obtrusively. At the end of each month we analyze the data. The analysis program, also written in HyperCard, integrates the current session information (cards seen, total time spent in the session) to a run- ning total. Figure 2 below shows a typi- cal session transcript. Breakdowns by time of day (used to determine if the session occurred during a period when a reference librarian was available), month, and school term are all easily produced. Figure 3 shows a sample sta- tistics card for an individual stack. Note that on the cumulative card(s) the analysis program has distinguished between in-depth and scanning uses . This distinction acknowledges that dif- ferent types of library use occur: using a photocopier or asking where the bath- room is are quite different from conduct- ing a multiple database search on Mirlyn (UM's OPAC) or asking a librarian to find data on a topic. Early observations of the use of Archimedes showed that it sees varying levels of use as well. For 0 session17 gr into stack "e/t 1 1 ocat ions" at Wednesday~ May 5~ {} r-- 1 9 9 3- 1 2:0 3 PM J open i n g J north map 1 2: 1 1 PM out * * i n to stack "Services" at 5/20/93-1 :55 PM~ services~ pat~ doc de 1 i very~ NEW B K ~ I L L ~ s e rv i c e s ~ t a 11 y ~ 0 N L I N ESE RV I C E s ~ RESERVES~ HOLDS~ BUHR~ ILL~ BUHR~ HOLDS~ RESERVES~ 0 N L I N ESE RV ICES~ t a 11 y J services 1 :57 PM out 1 :57 PM out ** into stack "E/TL System" at Sunday~ May 30 ~ 1993-12:55 PM~ contentscro11 1:23 PM out *1 ' {} 0 FIGURE 2 A Typical Session Transcript Created Automatically by Archimedes Archimedes 175 Throug : October 31, 1 99 1 number of entries into stack 11489 1 1805 I floor map aver:.ge number of cards visited per stack use 1 800 I ,1 .7674221 campus map ( "mappen ") most cards visited per stack use -- session number 1652 1 7 I north map average time in stack 1274 I 0.851526 1 longest time in stack -- session number FIGURE3 Statistics for the EL Locations Stack convenience these levels were split into two categories. We define a scanning use as a session where the user: • Did not get beyond the opening ani- · mation (i.e., clicked on the spinning globe, perhaps watched the anima:- tion, and then left), or • Only saw the menu cards of three or fewer stacks. This type of use, while it presumably gives some information about what Archimedes does, almost certainly does not answer a reference question. If you liken the first point to picking up a book with an attractive jacket and immediately putting it down without looking inside, this kind of scanning use seems similar to briefly looking at that book's table of contents. The program considers any other use in-depth. These definitions of scanning use are obviously arbitrary. We chose them after observing some of the initial session transcripts. The definitions strike a bal- ance between differentiating between sessions where Archimedes provides real information and ease/ speed of ana- lyzing the data from a large number of transcripts. The data analysis program runs in the background on sufficiently powerful Macintoshes (SE/30 or higher), which frees the machine for other uses when a large number of sessions are being ana- lyzed. Parts of this article were written while the data analysis ran on the same Macintosh. USE STATISTICS The data in table 1 are cumulative from the first day of Archimedes opera- tion through March 31, 1992 (one year plus the initial week). 6 We collected much more detail than this article can present. Breakdowns by term/ month, stack/ card, and individual Archimedes station have also been computed but are 176 College & Research Libraries TABLEt DATA FROM THE FIRST YEAR (MARCH 1991 THROUGH MARCH 1992) OF ARCHIMEDES USE Number of sessions from March 1991 through March 1992 In-depth uses Scanning uses In-depth sessions During reference hours (April1991 through April1992} During nonreference hours The most frequently visited topics are: Ellocations Reference help Services Special collections Average number of cards visited per session Average time spent at Archimedes 5,130 3,480 (68%) 1,650 (32%} 1,730 (47%) 1,950 (53%) 2,900 times 2,800 times 950 times 890 times 9.4 cards 2.4 minutes not presented here. The most frequently visited stacks are described in more de- tail below. As the total number of sessions shows, Archimedes received quite a bit of use in its first year, averaging over fourteen sessions a day. Of these, two-thirds fell into the in-depth category, indicating that the system most likely provided some level of reference help to the major- ity of people who approached it. In the first full year there was no sig- nificant difference between the number of people who used Archimedes during hours a reference librarian was available and those who used it after hours . But when we shifted the window of data to look at information collected between March 1991 and May 1992, a statistically significant difference appeared (53 per- cent of uses occurring when no reference librarian was available). 7 Since these hours comprise only 47 percent of the hours the EL is open, and roughly half as many people visit the library during these nonreference hours as during regular hours, the result is quite encour- March 1995 aging. In addition, we found that users spent slightly more time with Ar- chimedes during nonreference hours and apparently explored the stacks in greater depth. The most frequently vis- ited stacks category lists the major groupings of information that people se- lected most often from the main menu (figure 1). TheEL Locations stack gives locations of the libraries on the UM main campus. It also has a floor map of the EL that links to many other stacks. The most frequently visited cards in EL Locations are: • Floor map (1,270 visits) gives a floor map of the EL. If the user enters this card from another stack (e.g., high- lights the appropriate area of Patents from Reference Help), the area of inter- est gets highlighted automatically. • Campus map (1,050 visits) gives a map of the UM central campus. When a user clicks on a library name, the loca- tion flashes and the hours of operation and telephone number are given. • North map (450 visits) is similar to Cam- pus map, but shows UM' s north campus. Reference Help answers some of the most common questions we encounter at the reference desk. It consists of cards that embody some of the knowledge of an engineering reference librarian. The most frequently visited cards in Reference ·Help are: • Missing book (270 visits) describes how to find a book missing from the shelf. It is a self-contained card (i.e., it links only to broader menu cards). • Conferences (240 visits) addresses one of the most difficult tasks in the EL - tracking down conference proceed- ings. It gives Mirlyn search hints and links to Floor map (see below). • Patents (170 visits) explains the re- sources available for patent searching in the EL. Simple instruction and a referral to a reference librarian also appear. The EL offers a variety of services to university faculty, staff, and students (and, of course, to the community at large). Among these are services unique to our library. This stack highlights both these unique items and those available in the UM library system as a whole: • Online services (290 visits) describes the various databases available for en- gineering faculty, staff, and students on both a drop-in basis and by ap- pointment. • Patents (160 visits) gives more detail on the mechanics of patent searching at the EL. • New books (130 visits) discusses the process of requesting, ordering, and obtaining a new book for the library collection. Special Collections describes some of the more difficult to access and more popular subsets of EL' s general collec- tion. The most frequently visited cards in Special Collections are: • Patents (170 visits) explains the re- sources available for patent searching in the EL. Simple instruction and a referral to a reference librarian also appear. • Technical Reports (140 visits) gives gen- eral tips on searching for technical re- ports in the online catalog. Like Patents, it also refers the user to a ref- erence librarian. • Company Information (130 visits) pro- vides information on where to find and how to use the company and in- dustry directories collected by the EL-information useful to both re- searchers and job searchers. Note that a Patents card appears in three of the four most heavily used stacks. While the individual cards in these stacks differ, if the user chooses to see more information, they all point to the single area of Archimedes that de- scribes patents in detail. Multiple access points to an area of interest is a feature that rewards both the user, who doesn't need to learn a hierarchy (as with Go- pher system) to find information, and the creator, who only needs to make sim- ple connections rather than repeat infor- mation in many spots. DISCUSSION The discussion below (Time Spent, Types of Questions) addresses only those measurements of live reference that eas- ily compare with the data analysis statis- tics. We have not considered many other interesting and useful subjective meas- Archimedes 177 ures of service. By comparing interac- tions with computers and those with live users, librarians can give some idea of how people use Archimedes. Examining a few quantitative aspects of use (the time spent on reference, types of ques- tions asked, paths people take, and where they end their search) and making comparisons with other library Hyper- Card applications will give a feel of whether Archimedes fulfills its mission as a reference aid. Additional analyses of the sequence of information users seek in answering their own questions (The Steps People Take) and where their information search ends in Archimedes (Ending the Search) are also discussed. The ability to capture these two data sets is unique to an inter- active reference aid-a literature search turned up no other study of hypertext system that presented quantitative meas- ures of service delivered. Time Spent The data on time spent by librarians to answer reference questions are sparse. Collecting such data is both time-con- suming and potentially very intrusive. Only a few studies done in academic libraries give an idea of how long a ref- erence encounter between user and li- brarian lasts. Jo Bell Whitlatch observed that 31 percent of academic reference questions were answered in less than two minutes, and 86 percent within five minutes. 8 Edward C. Jestes and W. David Laird found that the average academic reference interview lasted two min- utes.9·10 So, time spent with Archimedes (2.4 minutes, from table 1) is comparable to time spent with real librarians. (In addition we have recorded some ex- tremely long sessions, with users visit- ing more than one hundred cards and spending up to an hour.) Type of Questions The time librarians spend with a user is, of course, not the only indication of the service provided. The amount and kind of information given are also cru- cial to user satisfaction. If an automated system does not address the kinds of 178 College & Research Libraries questions typically encountered by real reference librarians, it can never provide a good supplement to their service. Rec- ognizing this, we made a comparison between the types of questions we hear at the reference desk and the questions Archimedes has answered. Again, the literature is relatively sparse on question types in academic li- braries. Jeffrey W. St. Clair and Rao Aluri found that 44 percent of reference ques- tions are directional, 18 percent instruc- tional, 32 percent reference, and 6 percent extended reference.U Jestes and Laird found that 19 percent are directionalP TheEL has collected data of this kind for a number of years, however, and these data provide the best basis of compari- son between live reference and Ar- chimedes use. The data we collected at the EL refer- ence desk break down into six categories for four types of requesters (UM/non-UM affiliation, in-house/telephone). Answers to directional questions send users to a physical location. Quick and quick/in- structional questions require fewer than five minutes to answer and use two or fewer reference sources. When we spend more than five minutes and use two or more reference sources and special refer- ence know ledge to answer a question, we mark it as research or research/in- structional. A referral directs a user to another library or information source. Refere~ce desk staff record the type of question asked after completing each en- counter at the desk. The data are very consistent-our historical data match the ·data from the period in which Ar- chimedes has operated (see table 2). TABLE2 NUMBERS OF REFERENCE QUESTIONS AT THEEL, MARCH 1991 THROUGH MARCH 1992 Directional Quick + Quick/ Instructional Research+ Research/ Instructional ( < 5 minutes, ~ 2 information sources) 19% of questions asked (1,760) 70% of questions asked (6,680) 11% of questions asked (1,000) - - - -----------~------..., March 1995 The information on an Archimedes session is merely statistical and cannot truly differentiate between quick and re- search questions without spending an unreasonable amount of programming and processing time. Therefore we have chosen to define only two session cate- gories: reference or directional. When broken down this way, 62 percent (5,510 subsessions) of the uses are reference, while 38 percent (3,420) are directional. We use subsessions-defined as visits to separate topic stacks-since a user can, and often does, see many types of infor- mation in a given session. These data demonstrate that on a (very) coarse scale, Archimedes and ref- erence librarians address similar kinds of questions. The large number of direc- tional requests most likely results from the focus of the stacks and the inability of the system to conduct a reference in- terview. The rank order between floor map, campus map, and north campu~ card visits is similar to what we experi- ence at the reference desk. This heavy directional use indicates that a button level analysis of the data (similar to that reported in Virginia Tschanz, below) would prove useful in determining where we should direct our efforts on these cards. 13 The Steps People Take When people approach the reference desk with a question, a librarian usually spends a small amount of time (two to five minutes) and sends them on their way. Where their information-seeking process goes from there is difficult-or at least annoying and obtrusive-to de- termine. Archimedes' ability to cap- ture completely an outline of this process provides an otherwise inaccessi- ble glimpse at how people attempt to find what they need. A session transcript (as shown in figure 2) gives information on not only the individual cards users visit but also on the order in which they visit them. With this, we can determine the most likely paths users take to reach the information they want. One of the major advantages (and pit- falls) of a hypertext environment is the almost limitless number of paths users can take to reach the information they need. Even in a simple hypertext system such as Archimedes, the user has a large number of possible paths. As you would expect in a hypertext environment, Ar- chimedes has a total of 161 different links between its 713 cards. During the first year, users explored 123 of them (76 per- cent). The average number of times a card was exited/ entered was 131 (21,100 total transitions), with a median of 24. Very few (2 of 73) of the cards were not visited. More interesting, however, are the ac- tual paths taken by users . Figure 4 gives a visual description of the sequence of first steps users take · through Ar- chimedes. Attempting to present more than these quickly leads to an (even standards menu menu Archimedes 179 more) incomprehensible mess. By far the most common path began at the table of contents and proceeded to the list of available campus maps-24 percent of the users chose this opening move. Given this opening move, 45 percent of the users then selected the map of UM's central campus. From there, people either moved back to the list of maps or to the map of north campus libraries (50 percent of the time), or back to the table of contents (47 percent). The other alter- natives chosen after the opening move were proceeding to the north campus map (19 percent of the users-at the time, a branch of the Engineering Li- brary was located there), returning to the table of contents (45 percent), or choos- ing the list of other local libraries (16 percent) . central campus map menu FIGURE4 The Paths through the First Three Cards Users Chose 180 College & Research Libraries The next most common opening move from the table of contents began in the reference help stack (21 percent of the users went to this stack first). From there, people proceeded to the Mirlyn stack (33 percent of the time) and then began to learn about tips on finding books by key- word (31 percent). Whether people think they are getting a tutorial or actually going to be able to search the OPAC is an open question. As this section demonstrates, tracing even a few steps through a hypertext interface gets complex and confusing very quickly. This should not surprise us much, since a hypertext interface is mul- tidimensional. Describing it in two di- mensions is bound to lead to a loss of either detail or resolution. In the case of figure 4, both may be true. Future work in the area of exploring and describing paths through hypertext is essential. Ending the Search Just as we find it difficult to determine where people go when they leave the reference desk with information we have provided them, it is also difficult to find out where they stop seeking informa- tion. The session data Archimedes col- lects do reveal these stopping points. Examining these in conjunction with type of use (in-depth/ scanning) and the paths that lead users to the end of their search affords us further insights into the information-seeking process. 14 If the last card seen is information-bearing, this gives us another, more conservative esti- mate for counting in-depth use. By this definition, 30 percent of the sessions ended on an information-bearing card and were therefore in-depth. The origi- nal definition of in-depth use coupled with the data from table 1 classify 68 percent of the sessions as in-depth. Comparison to Other HyperCard Projects Although there are many descriptions of HyperCard applications in library set- tings (see Ertel and Oros, Welsh, Kessel- man, and Tschanz), none but Tschanz has reported any quantitative data on use.15 Tschanz's article describes the Pen- March 1995 rose Tutorial and Guides stack at the Uni- versity of Denver and use statistics re- corded for its map and twelve menu cards over a seventeen-day period. The article presents no data for the presum- ably more content-bearing destination cards, but data were collected for the menu button cards, which gives some indication of what topics users consulted. The data collection also incorporated an electronic Suggestion Box to solicit comments on the stack. (We have not implemented this in Archimedes, since the stations do not have keyboards.) Users logged approximately five ses- sions per day on the Penrose Tutorial and . Guides stack (as opposed to over 14 per day on Archimedes). The button usage data show high use of what appears to be gen- eral subject area cards, with use decreas- ing as the topics covered become more specific. People reported generally favor- able comments in the Suggestion Box. 16 The data collected at the University of Denver do not directly compare to the data reported here. Our data, collected continuously from the first week of use, cover long periods of both very frequent use (the first few weeks, when Ar- chimedes was a novelty) and less frequent use (during the spring and summer half terms). The Archimedes data are also more detailed-and unwieldy, no doubt-giv- ing breakdowns by month, time of day, total session time, individual card use, etc. While we can do much additional work (as noted in the conclusions and future directions), this paper gives the first comprehensive, quantitative de- scription of a HyperCard-based refer- ence instruction program. CONCLUSIONS AND FUTURE DIRECTIONS The data collected and the results of the analysis of that data are encouraging. In particular: • Users spend on average over two min- utes on each session, visiting over nine cards, and consult Archimedes ap- proximately fourteen times a day. Ar- chimedes has been used the way we intended it. The time users spend with Archimedes is comparable to the time they spend with a reference librarian, and users are exposed to a number of the EL' s services. • There have been roughly twice as many in-depth uses as scanning uses. Users are more than just curious about Archimedes-the evidence indicates they are learning from it. • The Archimedes system sees more and longer use during hours when refer- ence librarians are not available. It seems to fill some need for reference help during these hours. As such, it shows promise as a cost-effective way to provide reference service--espe- cially at sites and times where refer- ence questions taper off in both difficulty and frequency. 17 Archimedes is not a static system; we continuously update it to include the lat- est information about the UM library system as a whole and theELin particular. Updating its contents to reflect changes in the OPAC already has been mentioned. We plan other enhancements as well, and have converted the stacks to HyperCard 2.1 (which has improved system response times slightly); at the card level, an analy- sis of what buttons are clicked may be used to learn more about the information viewed on each card. The structure of Archimedes also lends itself to testing interface ideas not Archimedes 181 directly related to reference work. These issues have received much debate but little research attention-the availability of a "live" data set produced by real users provides an opportunity for study. And as the section above on paths dem- onstrates (The Steps People Take), we have much work ahead of us before we can present easy-to-understand depictions of what people do when they use hy- pertext systems. The stacks users visited and cards they consulted within those stacks indicate that Archimedes fields questions typical of those we hear at the reference desk. Archimedes also aids our reference librarians, who consult it for other li- braries' hours, locations, and telephone numbers, etc. Because of the large num- ber of uses of the Mirlyn stack, our recent efforts have focused on updating this and related stacks to reflect the constant changes in the OPAC and related library services. Archimedes continues to grow into a more interactive tool. Using a larger workstation, we have begun work on plans to allow live links to the online catalog, to the reference desk (perhaps using real-time video technology) and to remote sites.18 As it evolves, the possibili- ties for enhanced service increase, as do the possibilities for better quantitative research into electronic library tools. REFERENCES AND NOTES 1. Monica Ertel and Jane Oros, "A Tour of the Stacks-HyperCard for Libraries," Online 13 (1989): 45-53. 2. Carl Franklin, "A Bibliography on Hypertext and Hypermedia with Selected Annotations, Database 13 (1990): 24-45. 3. Carl Franklin and Linda Bessmer, "A Bibliography on Hypertext Continues ... HyperCard Stacks and Tutorials," Database 13 (1990): 38-45. 4. TheEL staff also created an abridged version of the OPAC (Mirlyn) tutorial which comes loaded onto the hard drive of any Macintosh computer purchased from UM's annual student/staff computer sale. 5. Apple Computer, Inc., HyperCard Stack Design Guidelines (Reading, Mass.: Addison- Wesley, 1989). 6. The second year's data, straddling the Engineering Library's move to a new location (and consequently its unavailability for roughly 2 months), are as yet unanalyzed. Further, the changes to the stacks resulting from the move have made these new data difficult to compare with the old. Also note that all whole numbers are rounded to the nearest 10, and fractions to one decimal place. Changes made to Archimedes and the analysis program in the first months of operation resulted in session transcripts that the program could not accurately tally. These sessions represent only a small percentage (approximately 50 out of the first 182 College & Research Libraries March 1995 5,130, or less than 1 percent) and are not included in the statistics reported here. Further, other uncertainties (e.g:, in determining in-depth versus scanning use) make reporting more precision inappropriate. 7. Jill Holman, "HyperCard Reference Tool Use Data and Reference Services at the Engineering Library," Associates' Project Report, University of Michigan, Summer 1993, 11. 8. Jo Bell Whitlatch, "The Role of the Academic Reference Librarian," in New Directions in Information Management Number 22, ed. Michael Buckland (Westport, Conn.: Greenwood, 1990), 88. 9. Edward C. Jestes and W. David Laird, "A Time Study of General Reference Work in a University Library," Research in Librarianship 2 (1968): 9-16. 10. Researchers have performed most of these types of studies in a public library setting. Reference interviews typically last from 3.1 to 4.4 minutes (Patricia H. Dewdney, "The Effects of Training Reference Librarians in Interview Skills: A Field Experiment" (Ph.D. diss., London, Ontario: Univ. of Western Ontario, 1986), 75. Goldhor found an average interview duration of 3.6 minutes. (Herbert Goldhor, "The Patron's Side of Public Library Reference Question," Public Library Quarterly 1 (1979): 35-49.) Although it appears that reference interviews may last longer in a public library setting, the data reported do support an assertion that the difference is significant. 11. Jeffrey W. St. Clair and Rao Aluri, "Staffing the Reference Desk: Professionals or Non-pro- fessionals?" Journal of Academic Librarianship 3 (1977): 149-53. 12. Jestes and Laird, "A Time Study of General Reference Work in a University Library," 15. 13. Virginia Tschanz, "Assessment of a HyperCard Program at Penrose Library, The University of Denver," Reference Services Review 19 (1991): 39-47. 14. Thanks to Jayne Campbell at Johns Hopkins for suggesting this. 15. Ertel and Oros, "A Tour of the Stacks"; Erwin K. Welsch, "Using Research Assistant for Library Instruction," Computers in Libraries 10 (1990): 10-13; Martin Kesselman, 11LSM Infomaster: A HyperCard CAl Program on a Macintosh Network," College & Research Libraries News 49 (1988): 437-40; and Tschanz, 11 Assessment of a HyperCard Program at Penrose Library, The University of Denver." 16. Tschanz, II Assessment of a HyperCard Program," 43. 17. We have estimated the time required to develop Archimedes at about four hundred librarian hours. Using the number of in-depth uses during nonreference hours from table 1 (1950) coupled with the average time spent (2.4 minutes), we get an average of eighty hours of additional reference provided by Archimedes each year, for a payback in about five years. Though these figures do not reflect the (rather modest) equipment costs to run the system, they also do not include all-or even half-of the total use of Archimedes. So, based on present use, they give a reasonable and perhaps even conservative estimate of the cost-ef- fectiveness of Archimedes. 18. The World Wide Web (WWW) environment, as accessed through browsers such as Mosaic or Netscape, provides ~n ideal platform for developing these enhancements. We have begun creating WWW versions of our Archimedes stacks, and now have a number of information tools that go further than Archimedes ever could in terms of information provided and ease of access. The universal resource locator (URL) for the Engineering Library is http:/ /www.engin.umich.edu/library /ELHP.html.