College and Research Libraries EDWARD S. WARNER A Tentative Analytical Approach to the Determination of Interlibrary Loan Network Effectiveness Because increasing amounts of public resources are being committed to network development, analytical frameworks within which to conduct empirically based studies are needed to facili-tate scientific assessments of network ·effectiveness. A tentative approach is outlined wherein comparative unit-system measurements are employed. The data utilized are those r·esulting from a state interlibrary loan network study, but the resultant descriptive framework is applicable to net- works with similar unit-system relationships. INCREASING AMOUN TS of public resourc- es are being committed to network de- velopment so that rational decision-mak- ing with respect to the application of these resources must ultimately come to depend on scientific assessments of existing networks. Although there are numerous interlibrary loan n etworks in the United States, there are few em- pirically based general statements-de- scriptive, explanatory, or evaluative- relative to their effectiveness (i.e., the degree to which they realize their goals, normally measured in terms of output). This paucity of generalizations may be, in part, a function of the absence of suitable analytical frameworks within which to conduct empirical studies. The tentative analytical approach to the determination of interlibrary loan Mr. Warner is the library planner with the Baltimore Regional Planning Council, Baltirrwre, Maryland. The writer is indebted to Edwin E. Ol- son, associate professor, School of Libt·ary and Information Services, University of Maryland, for his critical review of an earlier draft of this paper. network effectiveness outlined herein, and its application in one statewide study, yield an essentially comparative descriptive-analytical framework. At the outset of 1969, the present writer was engaged as principal investi- gator to analyze Maryland Interlibrary Loan Network transactions in an effort to assess the network's effectiveness. Some months later the study culminated in a report issued by the appropriate state unit.l At the time of the study, the Mary- land Interlibrary Loan Network consist- ed of twenty-three county libraries, a regional subsystem of seven county li- braries, several small college and special libraries, a primary library, and a sec- ondary library. The Enoch Pratt Free Library in Baltimore has functioned as the network's transaction clearinghouse as well as its primary lending library. Generally, the county, college, and spe- cial libraries initiate the requests into the network by teletyping them directly to Pratt. As an exception to this, how- ever, the county libraries comprising the Eastern Shore Area Arrangement submit their requests to the Area Li - / 217 218 j College & Research Libraries • May 1971 brary. Unfilled requests are then for- warded to Pratt. Only the Pratt Library may send still unfilled requests on to the McKeldin Library at the University of Maryland in College Park, the secondary or "back-up" library (see Figure 1). The data base utilized h erein de- rives from a random sample of loan transactions ( evidenced by the teletype records of loan requests) introduced in- to the network, and the responses there- to through the twenty-eighth day after receipt. The sample consists of 1,148 re- quests received by the Pra tt Library 111111111111111111111111111 College 1 L'i't·~·a:·~re:s· 111111111111111111111111111 11111111111!1111111111111111111111111111111111111 McKeldin Library 1111111111111111111111111111111111111111111111111 i 11111111111111111111111111111111111111 Pratt Library 11111111111111111111111111111111111111 1111111111111111111111 Area 1111111111111111111111 Library 1111111111111111111111 11111111111111111111111111111111 111111111111111111111111 County Lib rar ies 11111111111111111111111111111111 111111111111111111111111 Figure 1 111111111111111111111111111 Snecial 11111 111 n~lllllllllllllllll Libraries 111111111111111111111111111 Maryland Interlibrary Loan Network, Showing Flow of Requests Interlibrary Loan Network Effectiveness/ 219 (including those received by the East- ern Shore Area Library, recorded si- multaneously by the Pratt Library and the Area Library TWX terminals) dur- ing a five-day period in January 1969.2 An overwhelming majority of the re- quest5 both originated with the twenty- three county libraries ( 93.1 percent) and were requests for monographic ma- terials ( 87.0 percent). The units of analysis, then, are loan requests, where- as output consists of the filled requests. Analysis of the 1,148 requests in the Maryland Study-using a definition of "requests filled" which allowed two weeks for the disposition of the requests -revealed, in total system terms, that 677 of the requests were filled; that is, network effectiveness could be charac- terized as 59.0 percent (system output over system input). However, in a sys- tem with unit relationships approximat- ing those found in the Maryland net- work, requests are subjected to a se- quential screening or eliminative process which must be accounted for before at- tempting to further describe or to ex- plain and evaluate effectiveness (see Figure 1). In the Maryland case, there- fore, it was necessary to measure the relative contributions of the Area, Pratt, and McKeldin libraries as component units in the system. Of the 1,148 requests introduced into the network (system input), 110 ( 9.6 percent ) were filled by the Eastern Shore Area Library prior to being proc- essed by the Pratt Library, so that 1,038 still active requests remained. Of these, 492 ( 42.9 percent) of the original 1,148 requests introduced into the network were filled by the Pratt Library. 3 In turn, the Pratt Library elected to for- ward 149 still active requests to the Mc- Keldin Library wherein 75 ( 6.5 per- cent) of the original 1,148 requests in- troduced into the network were filled. Using system input as a base, total sys- tem effectiveness of 59.0 percent can be seen as the sum of the unit outputs (see Table 1). On the other hand, of the 232 requests in the sample actually received by the Area Library (unit input), 110 ( 47.4 per- cent) were filled by the Area Library. Of the 1,038 requests actually received by the Pratt Library, 492 ( 47.4 percent) were filled by the Pratt Library; and of the 149 requests actually received by the McKeldin Library, 75 ( 50.3 percent) were filled by the M cKeldin Library ( see Table 1). In systems approximating the Mary- land network, then, meaningful assess- ment must comprehend unit contribu- tions to the system in terms of both the input to each unit as well as the in- put to the system. That is, unit output should be measured in unit input terms (i.e., unit effectiveness) as well as sys- tem input terms, whereas system out- put can be measured only in system input terms (i.e., system effectiveness). In the Maryland case, network effec- TABLE 1 UNIT OuTPUT SHOWING REQUESTS FILLED As A PROPORTION OF THOSE INTRODUCED INTO THE SYSTEM AND AS A PROPORTION OF THOSE ACTUALLY RECEIVED Introduced Received Area 110 1,148 llO 232 ( 9.6%) ( 47.4%) BY THE UNITS Pratt 492 1,148 492 1,038 ( 42.9%) ( 47.4%) McKeldin 75 1,148 75 149 ( 6.5%) Total = 59.0% ( 50.3%) Mean = 48.4% 220/ College & Research Libraries • May 1971 tiveness was shown to be greater than either the mean effectiveness of the com- ponent units or the effectiveness of any one of the component units. On its face, of course, a finding such as this would lend support to the existence and main- tenance of a system of the type assessed. The screening or eliminative aspect of this type of system may be illustrated in another manner. By considering the requests first submitted to the Area Li- brary and those not first submitted to the Area Library as two subsamples, it can be seen that the success rate of the Area Library subsample, when weight- ed by that subsample's proportion of to- tal requests , has a disproportionately positive effect on the total system's ef- fectiveness. That is , the requests first routed through the Area Library com- pris e only 20.2 percent of those intro- duced into the system, but 77.1 percent of them were filled , whereas the re- quests which were not routed through the Area Library first comprise 79.8 per- cent of the requests introduced into the system, of which only 54.4 p ercent were filled (see Table 2). When the success rate of the Area Library subsample is weighted by the proportion of the total requests that the subsample comprises and is then taken as a proportion of the system's success rate, the resultant contribution of that subsample to net- work effectiveness is revealed to be 26.4 percent and thereby greater than its ab- solute participation ( 20.2 percent of the requests). That the Maryland Interlibrary Loan Network has b een characterized herein as 59.0 percent effective does not neces- sarily mean that an output (requests filled) of 100 percent is to be taken as an ideal goal. One authority has sug- gested, in fact, that "the goal model may not supply the best possible frame of reference for effectiveness" in that "it compares the ideal with the real, as a result of which most levels of perform- ance look alike-quite low."4 The percentage or proportional char- acterization may however be conveni- ently utilized for intrasystem or internal analytical purposes and has been so used herein. That is , any total system effec- tiveness measurement must somehow be comprised of unit contributions ( al- though not, as has been shown, neces- sarily on a sum-of-the-parts basis) , so that to internally analyze a system's ef- fectiveness, it is necessary only that a quantitative total system effectiveness statement be made against which unit contributions can be gauged. Moreover, if a total system's effectiveness at any given time is to be measured against that system's effectiveness at another point in time, a quantitative total system effectiveness statement will again suffice in that each effectiveness measurement, if comparably derived, will becom e a relative measurement in a time series. 5 TABLE 2 UNIT AND SYSTEM OuTPUT, SHOWING AREA LIBRARY SuBsAMPLE, LIBRARY SuBSAMPLE, AND NETWORK EFFECTIVENESS Area Nonarea Total Area llO 232 llO 1,148 ( 47.4%) ( 9.6%) Pratt 61 232 (26.3%) 431 916 (47.1 %) 492 1,143 ( 42.9%) McKeldin 8 179 232 ( 3.4%) 232 ( 77.1%) 67 498 916 (7.2%) 916 ( 54.4%) 75 677 1,145 ( 6.5%) 1,148 ( 59.0%) ~ I Interlibrary Loan Network Effectiveness/ 221 There has emerged, then, an elemen- tary descriptive-analytical framework re- flecting comparative unit and system input and output and within which fur- ther analysis of effectiveness might pro- ceed. That is, additional statements de- scribing, as well as explaining and eval- uating effectiveness can be couched in terms of the comparative unit-system framework herein developed. REFERENCES 1. See Edward S. Warner, "Maryland Inter- library Loan Network Loan Requests: Sys- tem and Units Analysis" (Baltimore: Mary- land State Department of Education, Divi- sion of Library Development and Services, 1969). 2. The original study also utilized a smaller sample consisting of 612 requests received by the Pratt Library during a two-day peri- od in February 1969. 3. Includes, under the two-week definition re- ferred to above, requests filled as a result of placing reserves on Pratt Library mate- rials, an option which the Pratt Library may (and does) exercise. Forty requests were in this way filled in the sample and are in- cluded with the Pratt Library figures though they materialized after sending 149 still ac- tive requests on to the McKeldin Library. 4. Amitai Etzioni, "Two Approaches to Orga- nizational Analysis: A Critique and a Sug- gestion," Administrative Science Quarterly 5:260 ( Sept. 1969). 5. Intersystem or external comparisons will also produce relative effectiveness measuremeRts, assuming comparably derived measure- ments.