id sid tid token lemma pos crl-14518 1 1 college college NOUN crl-14518 1 2 and and CCONJ crl-14518 1 3 research research NOUN crl-14518 1 4 libraries librarie VERB crl-14518 1 5 letters letter NOUN crl-14518 1 6 to to ADP crl-14518 1 7 the the DET crl-14518 1 8 editor editor NOUN crl-14518 1 9 : : PUNCT crl-14518 1 10 recent recent ADJ crl-14518 1 11 articles article NOUN crl-14518 1 12 such such ADJ crl-14518 1 13 as as ADP crl-14518 1 14 those those PRON crl-14518 1 15 by by ADP crl-14518 1 16 whitlatch whitlatch NOUN crl-14518 1 17 1 1 NUM crl-14518 1 18 and and CCONJ crl-14518 1 19 douglas2 douglas2 ADJ crl-14518 1 20 have have AUX crl-14518 1 21 discussed discuss VERB crl-14518 1 22 the the DET crl-14518 1 23 use use NOUN crl-14518 1 24 and and CCONJ crl-14518 1 25 appropriateness appropriateness NOUN crl-14518 1 26 of of ADP crl-14518 1 27 unobtrusive unobtrusive ADJ crl-14518 1 28 testing testing NOUN crl-14518 1 29 of of ADP crl-14518 1 30 reference reference NOUN crl-14518 1 31 services service NOUN crl-14518 1 32 . . PUNCT crl-14518 2 1 unobtrusive unobtrusive ADJ crl-14518 2 2 testing testing NOUN crl-14518 2 3 , , PUNCT crl-14518 2 4 like like ADP crl-14518 2 5 any any DET crl-14518 2 6 other other ADJ crl-14518 2 7 methodology methodology NOUN crl-14518 2 8 , , PUNCT crl-14518 2 9 merits merit VERB crl-14518 2 10 rigorous rigorous ADJ crl-14518 2 11 review review NOUN crl-14518 2 12 and and CCONJ crl-14518 2 13 assessment assessment NOUN crl-14518 2 14 . . PUNCT crl-14518 3 1 the the DET crl-14518 3 2 purpose purpose NOUN crl-14518 3 3 is be AUX crl-14518 3 4 to to PART crl-14518 3 5 better well ADV crl-14518 3 6 understand understand VERB crl-14518 3 7 situations situation NOUN crl-14518 3 8 in in ADP crl-14518 3 9 which which PRON crl-14518 3 10 a a DET crl-14518 3 11 particular particular ADJ crl-14518 3 12 methodology methodology NOUN crl-14518 3 13 is be AUX crl-14518 3 14 most most ADV crl-14518 3 15 appropriate appropriate ADJ crl-14518 3 16 , , PUNCT crl-14518 3 17 while while SCONJ crl-14518 3 18 at at ADP crl-14518 3 19 the the DET crl-14518 3 20 same same ADJ crl-14518 3 21 time time NOUN crl-14518 3 22 identifying identify VERB crl-14518 3 23 and and CCONJ crl-14518 3 24 addressing address VERB crl-14518 3 25 , , PUNCT crl-14518 3 26 where where SCONJ crl-14518 3 27 possible possible ADJ crl-14518 3 28 , , PUNCT crl-14518 3 29 methodological methodological ADJ crl-14518 3 30 weaknesses weakness NOUN crl-14518 3 31 . . PUNCT crl-14518 4 1 unfortunately unfortunately ADV crl-14518 4 2 , , PUNCT crl-14518 4 3 misconceptions misconception NOUN crl-14518 4 4 about about ADP crl-14518 4 5 , , PUNCT crl-14518 4 6 and and CCONJ crl-14518 4 7 incorrect incorrect ADJ crl-14518 4 8 attributions attribution NOUN crl-14518 4 9 to to PART crl-14518 4 10 , , PUNCT crl-14518 4 11 unobtrusive unobtrusive ADJ crl-14518 4 12 research research NOUN crl-14518 4 13 persist persist NOUN crl-14518 4 14 . . PUNCT crl-14518 5 1 unobtrusive unobtrusive ADJ crl-14518 5 2 testing testing NOUN crl-14518 5 3 is be AUX crl-14518 5 4 only only ADV crl-14518 5 5 one one NUM crl-14518 5 6 of of ADP crl-14518 5 7 many many ADJ crl-14518 5 8 possible possible ADJ crl-14518 5 9 research research NOUN crl-14518 5 10 / / SYM crl-14518 5 11 evaluation evaluation NOUN crl-14518 5 12 methodologies methodology NOUN crl-14518 5 13 . . PUNCT crl-14518 6 1 articles article NOUN crl-14518 6 2 suggesting suggest VERB crl-14518 6 3 that that SCONJ crl-14518 6 4 unobtrusive unobtrusive ADJ crl-14518 6 5 testing testing NOUN crl-14518 6 6 is be AUX crl-14518 6 7 an an DET crl-14518 6 8 inferior inferior ADJ crl-14518 6 9 technique technique NOUN crl-14518 6 10 for for ADP crl-14518 6 11 evaluating evaluate VERB crl-14518 6 12 '' '' PUNCT crl-14518 6 13 reference reference NOUN crl-14518 6 14 services service NOUN crl-14518 6 15 " " PUNCT crl-14518 6 16 distort distort VERB crl-14518 6 17 the the DET crl-14518 6 18 research research NOUN crl-14518 6 19 record record NOUN crl-14518 6 20 . . PUNCT crl-14518 7 1 first first ADV crl-14518 7 2 , , PUNCT crl-14518 7 3 unobtrusive unobtrusive ADJ crl-14518 7 4 testing testing NOUN crl-14518 7 5 only only ADV crl-14518 7 6 examines examine VERB crl-14518 7 7 specific specific ADJ crl-14518 7 8 aspects aspect NOUN crl-14518 7 9 of of ADP crl-14518 7 10 reference reference NOUN crl-14518 7 11 service service NOUN crl-14518 7 12 . . PUNCT crl-14518 8 1 although although SCONJ crl-14518 8 2 data datum NOUN crl-14518 8 3 resulting result VERB crl-14518 8 4 from from ADP crl-14518 8 5 unobtrusive unobtrusive ADJ crl-14518 8 6 testing testing NOUN crl-14518 8 7 can can AUX crl-14518 8 8 provide provide VERB crl-14518 8 9 useful useful ADJ crl-14518 8 10 diagnostic diagnostic ADJ crl-14518 8 11 information information NOUN crl-14518 8 12 regarding regard VERB crl-14518 8 13 a a DET crl-14518 8 14 range range NOUN crl-14518 8 15 of of ADP crl-14518 8 16 other other ADJ crl-14518 8 17 reference reference NOUN crl-14518 8 18 service service NOUN crl-14518 8 19 activities activity NOUN crl-14518 8 20 , , PUNCT crl-14518 8 21 the the DET crl-14518 8 22 method method NOUN crl-14518 8 23 focuses focus VERB crl-14518 8 24 attention attention NOUN crl-14518 8 25 on on ADP crl-14518 8 26 reference reference NOUN crl-14518 8 27 staff staff NOUN crl-14518 8 28 responses response NOUN crl-14518 8 29 to to ADP crl-14518 8 30 factual factual ADJ crl-14518 8 31 and and CCONJ crl-14518 8 32 bibliographic bibliographic ADJ crl-14518 8 33 questions question NOUN crl-14518 8 34 . . PUNCT crl-14518 9 1 second second ADJ crl-14518 9 2 , , PUNCT crl-14518 9 3 murfin murfin NOUN crl-14518 9 4 and and CCONJ crl-14518 9 5 gugelchuk3 gugelchuk3 NOUN crl-14518 9 6 and and CCONJ crl-14518 9 7 benham benham PROPN crl-14518 9 8 and and CCONJ crl-14518 9 9 powell powell PROPN crl-14518 9 10 , , PUNCT crl-14518 9 11 4 4 NUM crl-14518 9 12 among among ADP crl-14518 9 13 others other NOUN crl-14518 9 14 , , PUNCT crl-14518 9 15 have have AUX crl-14518 9 16 used use VERB crl-14518 9 17 other other ADJ crl-14518 9 18 methodologies methodology NOUN crl-14518 9 19 to to PART crl-14518 9 20 assess assess VERB crl-14518 9 21 staff staff NOUN crl-14518 9 22 responses response NOUN crl-14518 9 23 to to ADP crl-14518 9 24 reference reference NOUN crl-14518 9 25 questions question NOUN crl-14518 9 26 . . PUNCT crl-14518 10 1 the the DET crl-14518 10 2 findings finding NOUN crl-14518 10 3 of of ADP crl-14518 10 4 these these DET crl-14518 10 5 studies study NOUN crl-14518 10 6 have have AUX crl-14518 10 7 supported support VERB crl-14518 10 8 those those PRON crl-14518 10 9 resulting result VERB crl-14518 10 10 from from ADP crl-14518 10 11 unobtrusive unobtrusive ADJ crl-14518 10 12 research research NOUN crl-14518 10 13 . . PUNCT crl-14518 11 1 studies study NOUN crl-14518 11 2 that that PRON crl-14518 11 3 apply apply VERB crl-14518 11 4 unobtrusive unobtrusive ADJ crl-14518 11 5 testing testing NOUN crl-14518 11 6 or or CCONJ crl-14518 11 7 any any DET crl-14518 11 8 other other ADJ crl-14518 11 9 methodology methodology NOUN crl-14518 11 10 must must AUX crl-14518 11 11 incorporate incorporate VERB crl-14518 11 12 standard standard ADJ crl-14518 11 13 research research NOUN crl-14518 11 14 practices practice NOUN crl-14518 11 15 . . PUNCT crl-14518 12 1 the the DET crl-14518 12 2 hernon hernon PROPN crl-14518 12 3 and and CCONJ crl-14518 12 4 mcclure mcclure NOUN crl-14518 12 5 study5 study5 NOUN crl-14518 12 6 as as ADV crl-14518 12 7 well well ADV crl-14518 12 8 as as ADP crl-14518 12 9 the the DET crl-14518 12 10 work work NOUN crl-14518 12 11 of of ADP crl-14518 12 12 crowley crowley NOUN crl-14518 12 13 and and CCONJ crl-14518 12 14 childers6 childers6 NOUN crl-14518 12 15 provide provide VERB crl-14518 12 16 specific specific ADJ crl-14518 12 17 information information NOUN crl-14518 12 18 about about ADP crl-14518 12 19 the the DET crl-14518 12 20 criteria criterion NOUN crl-14518 12 21 guiding guide VERB crl-14518 12 22 the the DET crl-14518 12 23 development development NOUN crl-14518 12 24 of of ADP crl-14518 12 25 test test NOUN crl-14518 12 26 questions question NOUN crl-14518 12 27 and and CCONJ crl-14518 12 28 about about ADP crl-14518 12 29 the the DET crl-14518 12 30 steps step NOUN crl-14518 12 31 taken take VERB crl-14518 12 32 to to PART crl-14518 12 33 ensure ensure VERB crl-14518 12 34 the the DET crl-14518 12 35 collection collection NOUN crl-14518 12 36 and and CCONJ crl-14518 12 37 analysis analysis NOUN crl-14518 12 38 of of ADP crl-14518 12 39 quality quality NOUN crl-14518 12 40 ( ( PUNCT crl-14518 12 41 reliable reliable ADJ crl-14518 12 42 and and CCONJ crl-14518 12 43 valid valid ADJ crl-14518 12 44 ) ) PUNCT crl-14518 12 45 data datum NOUN crl-14518 12 46 . . PUNCT crl-14518 13 1 these these DET crl-14518 13 2 reliability reliability NOUN crl-14518 13 3 and and CCONJ crl-14518 13 4 validity validity NOUN crl-14518 13 5 controls control NOUN crl-14518 13 6 were be AUX crl-14518 13 7 carefully carefully ADV crl-14518 13 8 implemented implement VERB crl-14518 13 9 and and CCONJ crl-14518 13 10 reported report VERB crl-14518 13 11 in in ADP crl-14518 13 12 the the DET crl-14518 13 13 studies study NOUN crl-14518 13 14 . . PUNCT crl-14518 14 1 in in ADP crl-14518 14 2 fact fact NOUN crl-14518 14 3 , , PUNCT crl-14518 14 4 hernon hernon NOUN crl-14518 14 5 and and CCONJ crl-14518 14 6 mcclure mcclure VERB crl-14518 14 7 summarizes summarize NOUN crl-14518 14 8 these these DET crl-14518 14 9 methodological methodological ADJ crl-14518 14 10 refinements refinement NOUN crl-14518 14 11 . . PUNCT crl-14518 15 1 7 7 NUM crl-14518 15 2 in in ADP crl-14518 15 3 contrast contrast NOUN crl-14518 15 4 , , PUNCT crl-14518 15 5 the the DET crl-14518 15 6 douglas douglas PROPN crl-14518 15 7 study study NOUN crl-14518 15 8 provides provide VERB crl-14518 15 9 no no DET crl-14518 15 10 information information NOUN crl-14518 15 11 about about ADP crl-14518 15 12 the the DET crl-14518 15 13 reliability reliability NOUN crl-14518 15 14 and and CCONJ crl-14518 15 15 validity validity NOUN crl-14518 15 16 of of ADP crl-14518 15 17 the the DET crl-14518 15 18 data datum NOUN crl-14518 15 19 he he PRON crl-14518 15 20 reports report VERB crl-14518 15 21 . . PUNCT crl-14518 16 1 similarly similarly ADV crl-14518 16 2 , , PUNCT crl-14518 16 3 whitlatch whitlatch NOUN crl-14518 16 4 gives give VERB crl-14518 16 5 no no DET crl-14518 16 6 information information NOUN crl-14518 16 7 on on ADP crl-14518 16 8 the the DET crl-14518 16 9 reliability reliability NOUN crl-14518 16 10 and and CCONJ crl-14518 16 11 validity validity NOUN crl-14518 16 12 of of ADP crl-14518 16 13 her her PRON crl-14518 16 14 obtrusive obtrusive ADJ crl-14518 16 15 data datum NOUN crl-14518 16 16 . . PUNCT crl-14518 17 1 rather rather ADV crl-14518 17 2 , , PUNCT crl-14518 17 3 the the DET crl-14518 17 4 reader reader NOUN crl-14518 17 5 must must AUX crl-14518 17 6 assume assume VERB crl-14518 17 7 that that SCONJ crl-14518 17 8 there there PRON crl-14518 17 9 is be VERB crl-14518 17 10 reliability reliability NOUN crl-14518 17 11 and and CCONJ crl-14518 17 12 that that SCONJ crl-14518 17 13 the the DET crl-14518 17 14 research research NOUN crl-14518 17 15 design design NOUN crl-14518 17 16 permits permit VERB crl-14518 17 17 the the DET crl-14518 17 18 drawing drawing NOUN crl-14518 17 19 of of ADP crl-14518 17 20 conclusive conclusive ADJ crl-14518 17 21 and and CCONJ crl-14518 17 22 widely widely ADV crl-14518 17 23 generalizable generalizable ADJ crl-14518 17 24 findings finding NOUN crl-14518 17 25 . . PUNCT crl-14518 18 1 whitlatch whitlatch NOUN crl-14518 18 2 's 's PART crl-14518 18 3 discussion discussion NOUN crl-14518 18 4 of of ADP crl-14518 18 5 content content NOUN crl-14518 18 6 validity validity NOUN crl-14518 18 7 misses miss VERB crl-14518 18 8 the the DET crl-14518 18 9 point point NOUN crl-14518 18 10 . . PUNCT crl-14518 19 1 obviously obviously ADV crl-14518 19 2 , , PUNCT crl-14518 19 3 factual factual ADJ crl-14518 19 4 and and CCONJ crl-14518 19 5 bibliographic bibliographic ADJ crl-14518 19 6 reference reference NOUN crl-14518 19 7 questions question NOUN crl-14518 19 8 do do AUX crl-14518 19 9 not not PART crl-14518 19 10 represent represent VERB crl-14518 19 11 all all DET crl-14518 19 12 reference reference NOUN crl-14518 19 13 questions question NOUN crl-14518 19 14 received receive VERB crl-14518 19 15 at at ADP crl-14518 19 16 the the DET crl-14518 19 17 reference reference NOUN crl-14518 19 18 desk desk NOUN crl-14518 19 19 . . PUNCT crl-14518 20 1 it it PRON crl-14518 20 2 might might AUX crl-14518 20 3 be be AUX crl-14518 20 4 noted note VERB crl-14518 20 5 that that SCONJ crl-14518 20 6 content content NOUN crl-14518 20 7 validity validity NOUN crl-14518 20 8 can can AUX crl-14518 20 9 be be AUX crl-14518 20 10 viewed view VERB crl-14518 20 11 from from ADP crl-14518 20 12 other other ADJ crl-14518 20 13 perspectives perspective NOUN crl-14518 20 14 . . PUNCT crl-14518 20 15 . . PUNCT crl-14518 21 1 an an DET crl-14518 21 2 informed informed ADJ crl-14518 21 3 discussion discussion NOUN crl-14518 21 4 and and CCONJ crl-14518 21 5 assessment assessment NOUN crl-14518 21 6 of of ADP crl-14518 21 7 unobtrusive unobtrusive ADJ crl-14518 21 8 testing testing NOUN crl-14518 21 9 necessitates necessitate VERB crl-14518 21 10 careful careful ADJ crl-14518 21 11 attention attention NOUN crl-14518 21 12 to to ADP crl-14518 21 13 methodological methodological ADJ crl-14518 21 14 issues issue NOUN crl-14518 21 15 and and CCONJ crl-14518 21 16 questions question NOUN crl-14518 21 17 centering center VERB crl-14518 21 18 around around ADP crl-14518 21 19 reliability reliability NOUN crl-14518 21 20 , , PUNCT crl-14518 21 21 validity validity NOUN crl-14518 21 22 , , PUNCT crl-14518 21 23 and and CCONJ crl-14518 21 24 ( ( PUNCT crl-14518 21 25 perhaps perhaps ADV crl-14518 21 26 ) ) PUNCT crl-14518 21 27 utility utility NOUN crl-14518 21 28 of of ADP crl-14518 21 29 the the DET crl-14518 21 30 data datum NOUN crl-14518 21 31 and and CCONJ crl-14518 21 32 findings finding NOUN crl-14518 21 33 . . PUNCT crl-14518 22 1 neither neither CCONJ crl-14518 22 2 douglas douglas PROPN crl-14518 22 3 nor nor CCONJ crl-14518 22 4 whitlatch whitlatch NOUN crl-14518 22 5 referred refer VERB crl-14518 22 6 to to ADP crl-14518 22 7 the the DET crl-14518 22 8 article article NOUN crl-14518 22 9 '' '' PUNCT crl-14518 22 10 quality quality NOUN crl-14518 22 11 of of ADP crl-14518 22 12 data data NOUN crl-14518 22 13 issues issue NOUN crl-14518 22 14 in in ADP crl-14518 22 15 unobtrusive unobtrusive ADJ crl-14518 22 16 testing testing NOUN crl-14518 22 17 of of ADP crl-14518 22 18 library library NOUN crl-14518 22 19 reference reference NOUN crl-14518 22 20 service service NOUN crl-14518 22 21 '' '' PUNCT crl-14518 22 22 by by ADP crl-14518 22 23 hernon hernon PROPN crl-14518 22 24 and and CCONJ crl-14518 22 25 mcclure mcclure NOUN crl-14518 22 26 . . PUNCT crl-14518 23 1 in in ADP crl-14518 23 2 fact fact NOUN crl-14518 23 3 , , PUNCT crl-14518 23 4 that that DET crl-14518 23 5 article article NOUN crl-14518 23 6 discusses discuss VERB crl-14518 23 7 a a DET crl-14518 23 8 number number NOUN crl-14518 23 9 of of ADP crl-14518 23 10 their their PRON crl-14518 23 11 concerns concern NOUN crl-14518 23 12 . . PUNCT crl-14518 24 1 thus thus ADV crl-14518 24 2 , , PUNCT crl-14518 24 3 recent recent ADJ crl-14518 24 4 criticism criticism NOUN crl-14518 24 5 of of ADP crl-14518 24 6 unobtrusive unobtrusive ADJ crl-14518 24 7 methods method NOUN crl-14518 24 8 and and CCONJ crl-14518 24 9 quality quality NOUN crl-14518 24 10 of of ADP crl-14518 24 11 data data NOUN crl-14518 24 12 issues issue NOUN crl-14518 24 13 tends tend VERB crl-14518 24 14 to to PART crl-14518 24 15 be be AUX crl-14518 24 16 simplistic simplistic ADJ crl-14518 24 17 and and CCONJ crl-14518 24 18 repetitive repetitive ADJ crl-14518 24 19 and and CCONJ crl-14518 24 20 ignores ignore VERB crl-14518 24 21 existing exist VERB crl-14518 24 22 research research NOUN crl-14518 24 23 on on ADP crl-14518 24 24 the the DET crl-14518 24 25 topics topic NOUN crl-14518 24 26 . . PUNCT crl-14518 25 1 additional additional ADJ crl-14518 25 2 examples example NOUN crl-14518 25 3 of of ADP crl-14518 25 4 misunderstandings misunderstanding NOUN crl-14518 25 5 about about ADP crl-14518 25 6 , , PUNCT crl-14518 25 7 and and CCONJ crl-14518 25 8 incorrect incorrect ADJ crl-14518 25 9 representations representation NOUN crl-14518 25 10 of of ADP crl-14518 25 11 , , PUNCT crl-14518 25 12 unobtrusive unobtrusive ADJ crl-14518 25 13 research research NOUN crl-14518 25 14 dot dot VERB crl-14518 25 15 the the DET crl-14518 25 16 literature literature NOUN crl-14518 25 17 . . PUNCT crl-14518 26 1 and and CCONJ crl-14518 26 2 , , PUNCT crl-14518 26 3 despite despite SCONJ crl-14518 26 4 some some DET crl-14518 26 5 problems problem NOUN crl-14518 26 6 , , PUNCT crl-14518 26 7 whitlatch whitlatch NOUN crl-14518 26 8 's 's PART crl-14518 26 9 piece piece NOUN crl-14518 26 10 is be AUX crl-14518 26 11 a a DET crl-14518 26 12 701 701 NUM crl-14518 26 13 702 702 NUM crl-14518 26 14 college college NOUN crl-14518 26 15 & & CCONJ crl-14518 26 16 research research PROPN crl-14518 26 17 libraries library NOUN crl-14518 26 18 november november PROPN crl-14518 26 19 1989 1989 NUM crl-14518 26 20 good good ADJ crl-14518 26 21 example example NOUN crl-14518 26 22 of of ADP crl-14518 26 23 exploring explore VERB crl-14518 26 24 some some PRON crl-14518 26 25 of of ADP crl-14518 26 26 these these DET crl-14518 26 27 issues issue NOUN crl-14518 26 28 in in ADP crl-14518 26 29 a a DET crl-14518 26 30 constructive constructive ADJ crl-14518 26 31 manner manner NOUN crl-14518 26 32 . . PUNCT crl-14518 27 1 however however ADV crl-14518 27 2 , , PUNCT crl-14518 27 3 contrary contrary ADV crl-14518 27 4 to to ADP crl-14518 27 5 her her PRON crl-14518 27 6 assertion assertion NOUN crl-14518 27 7 , , PUNCT crl-14518 27 8 the the DET crl-14518 27 9 literature literature NOUN crl-14518 27 10 already already ADV crl-14518 27 11 contains contain VERB crl-14518 27 12 examples example NOUN crl-14518 27 13 of of ADP crl-14518 27 14 constructive constructive ADJ crl-14518 27 15 uses use NOUN crl-14518 27 16 of of ADP crl-14518 27 17 unobtrusive unobtrusive ADJ crl-14518 27 18 testing testing NOUN crl-14518 27 19 in in ADP crl-14518 27 20 specific specific ADJ crl-14518 27 21 libraries library NOUN crl-14518 27 22 , , PUNCT crl-14518 27 23 for for ADP crl-14518 27 24 example example NOUN crl-14518 27 25 , , PUNCT crl-14518 27 26 stephan stephan NOUN crl-14518 27 27 and and CCONJ crl-14518 27 28 others8 others8 NOUN crl-14518 27 29 and and CCONJ crl-14518 27 30 williams williams PROPN crl-14518 27 31 and and CCONJ crl-14518 27 32 wedig.9 wedig.9 PROPN crl-14518 27 33 the the DET crl-14518 27 34 key key ADJ crl-14518 27 35 issues issue NOUN crl-14518 27 36 with with ADP crl-14518 27 37 unobtrusive unobtrusive ADJ crl-14518 27 38 testing testing NOUN crl-14518 27 39 are be AUX crl-14518 27 40 to to PART crl-14518 27 41 ( ( PUNCT crl-14518 27 42 1 1 X crl-14518 27 43 ) ) PUNCT crl-14518 27 44 advance advance VERB crl-14518 27 45 knowledge knowledge NOUN crl-14518 27 46 and and CCONJ crl-14518 27 47 the the DET crl-14518 27 48 discourse discourse NOUN crl-14518 27 49 regarding regard VERB crl-14518 27 50 the the DET crl-14518 27 51 evaluation evaluation NOUN crl-14518 27 52 of of ADP crl-14518 27 53 various various ADJ crl-14518 27 54 types type NOUN crl-14518 27 55 of of ADP crl-14518 27 56 reference reference NOUN crl-14518 27 57 services service NOUN crl-14518 27 58 , , PUNCT crl-14518 27 59 ( ( PUNCT crl-14518 27 60 2 2 X crl-14518 27 61 ) ) PUNCT crl-14518 27 62 increase increase VERB crl-14518 27 63 the the DET crl-14518 27 64 profession profession NOUN crl-14518 27 65 's 's PART crl-14518 27 66 awareness awareness NOUN crl-14518 27 67 of of ADP crl-14518 27 68 evaluation evaluation NOUN crl-14518 27 69 overall overall ADV crl-14518 27 70 and and CCONJ crl-14518 27 71 within within ADP crl-14518 27 72 specific specific ADJ crl-14518 27 73 library library NOUN crl-14518 27 74 organizations organization NOUN crl-14518 27 75 , , PUNCT crl-14518 27 76 and and CCONJ crl-14518 27 77 ( ( PUNCT crl-14518 27 78 3 3 X crl-14518 27 79 ) ) PUNCT crl-14518 27 80 develop develop VERB crl-14518 27 81 strategies strategy NOUN crl-14518 27 82 for for ADP crl-14518 27 83 improving improve VERB crl-14518 27 84 the the DET crl-14518 27 85 quality quality NOUN crl-14518 27 86 of of ADP crl-14518 27 87 reference reference NOUN crl-14518 27 88 services service NOUN crl-14518 27 89 . . PUNCT crl-14518 28 1 our our PRON crl-14518 28 2 experience experience NOUN crl-14518 28 3 suggests suggest VERB crl-14518 28 4 that that SCONJ crl-14518 28 5 while while SCONJ crl-14518 28 6 there there PRON crl-14518 28 7 is be VERB crl-14518 28 8 much much ADJ crl-14518 28 9 discussion discussion NOUN crl-14518 28 10 about about ADP crl-14518 28 11 the the DET crl-14518 28 12 evaluation evaluation NOUN crl-14518 28 13 of of ADP crl-14518 28 14 reference reference NOUN crl-14518 28 15 services service NOUN crl-14518 28 16 , , PUNCT crl-14518 28 17 actual actual ADJ crl-14518 28 18 formal formal ADJ crl-14518 28 19 evaluation evaluation NOUN crl-14518 28 20 ( ( PUNCT crl-14518 28 21 regardless regardless ADV crl-14518 28 22 of of ADP crl-14518 28 23 type type NOUN crl-14518 28 24 of of ADP crl-14518 28 25 evaluation evaluation NOUN crl-14518 28 26 ) ) PUNCT crl-14518 28 27 , , PUNCT crl-14518 28 28 overall overall ADV crl-14518 28 29 , , PUNCT crl-14518 28 30 occurs occur VERB crl-14518 28 31 much much ADV crl-14518 28 32 less less ADV crl-14518 28 33 frequently frequently ADV crl-14518 28 34 . . PUNCT crl-14518 29 1 thus thus ADV crl-14518 29 2 , , PUNCT crl-14518 29 3 we we PRON crl-14518 29 4 are be AUX crl-14518 29 5 pleased pleased ADJ crl-14518 29 6 that that SCONJ crl-14518 29 7 unobtrusive unobtrusive ADJ crl-14518 29 8 testing testing NOUN crl-14518 29 9 has have AUX crl-14518 29 10 increased increase VERB crl-14518 29 11 professional professional ADJ crl-14518 29 12 attention attention NOUN crl-14518 29 13 on on ADP crl-14518 29 14 issues issue NOUN crl-14518 29 15 related relate VERB crl-14518 29 16 to to ADP crl-14518 29 17 the the DET crl-14518 29 18 quality quality NOUN crl-14518 29 19 of of ADP crl-14518 29 20 reference reference NOUN crl-14518 29 21 service service NOUN crl-14518 29 22 , , PUNCT crl-14518 29 23 " " PUNCT crl-14518 29 24 correct correct ADJ crl-14518 29 25 answer answer NOUN crl-14518 29 26 fill fill NOUN crl-14518 29 27 rates rate NOUN crl-14518 29 28 , , PUNCT crl-14518 29 29 " " PUNCT crl-14518 29 30 and and CCONJ crl-14518 29 31 techniques technique NOUN crl-14518 29 32 for for ADP crl-14518 29 33 assessing assess VERB crl-14518 29 34 the the DET crl-14518 29 35 quality quality NOUN crl-14518 29 36 of of ADP crl-14518 29 37 various various ADJ crl-14518 29 38 aspects aspect NOUN crl-14518 29 39 of of ADP crl-14518 29 40 reference reference NOUN crl-14518 29 41 services service NOUN crl-14518 29 42 . . PUNCT crl-14518 30 1 such such ADJ crl-14518 30 2 discussions discussion NOUN crl-14518 30 3 , , PUNCT crl-14518 30 4 however however ADV crl-14518 30 5 , , PUNCT crl-14518 30 6 are be AUX crl-14518 30 7 better well ADV crl-14518 30 8 served serve VERB crl-14518 30 9 when when SCONJ crl-14518 30 10 the the DET crl-14518 30 11 participants participant NOUN crl-14518 30 12 are be AUX crl-14518 30 13 fully fully ADV crl-14518 30 14 informed inform VERB crl-14518 30 15 on on ADP crl-14518 30 16 the the DET crl-14518 30 17 issues issue NOUN crl-14518 30 18 ; ; PUNCT crl-14518 30 19 when when SCONJ crl-14518 30 20 they they PRON crl-14518 30 21 carefully carefully ADV crl-14518 30 22 craft craft VERB crl-14518 30 23 their their PRON crl-14518 30 24 studies study NOUN crl-14518 30 25 to to PART crl-14518 30 26 address address VERB crl-14518 30 27 specific specific ADJ crl-14518 30 28 research research NOUN crl-14518 30 29 questions question NOUN crl-14518 30 30 , , PUNCT crl-14518 30 31 especially especially ADV crl-14518 30 32 if if SCONJ crl-14518 30 33 they they PRON crl-14518 30 34 attempt attempt VERB crl-14518 30 35 to to PART crl-14518 30 36 prove prove VERB crl-14518 30 37 / / SYM crl-14518 30 38 disprove disprove VERB crl-14518 30 39 a a DET crl-14518 30 40 particular particular ADJ crl-14518 30 41 hypothesis hypothesis NOUN crl-14518 30 42 ; ; PUNCT crl-14518 30 43 and and CCONJ crl-14518 30 44 when when SCONJ crl-14518 30 45 they they PRON crl-14518 30 46 describe describe VERB crl-14518 30 47 indicators indicator NOUN crl-14518 30 48 of of ADP crl-14518 30 49 reliability reliability NOUN crl-14518 30 50 and and CCONJ crl-14518 30 51 validity validity NOUN crl-14518 30 52 for for ADP crl-14518 30 53 the the DET crl-14518 30 54 data datum NOUN crl-14518 30 55 they they PRON crl-14518 30 56 report report VERB crl-14518 30 57 . . PUNCT crl-14518 31 1 peter peter PROPN crl-14518 31 2 hernon hernon PROPN crl-14518 31 3 , , PUNCT crl-14518 31 4 simmons simmons PROPN crl-14518 31 5 college college PROPN crl-14518 31 6 charles charles PROPN crl-14518 31 7 r. r. PROPN crl-14518 31 8 mcclure mcclure PROPN crl-14518 31 9 , , PUNCT crl-14518 31 10 syracuse syracuse PROPN crl-14518 31 11 university university NOUN crl-14518 31 12 references reference NOUN crl-14518 31 13 1 1 NUM crl-14518 31 14 . . PUNCT crl-14518 31 15 jo jo PROPN crl-14518 31 16 bell bell PROPN crl-14518 31 17 whitlatch whitlatch NOUN crl-14518 31 18 , , PUNCT crl-14518 31 19 " " PUNCT crl-14518 31 20 unobtrusive unobtrusive ADJ crl-14518 31 21 studies study NOUN crl-14518 31 22 and and CCONJ crl-14518 31 23 the the DET crl-14518 31 24 quality quality NOUN crl-14518 31 25 of of ADP crl-14518 31 26 academic academic ADJ crl-14518 31 27 library library NOUN crl-14518 31 28 reference reference NOUN crl-14518 31 29 services service NOUN crl-14518 31 30 , , PUNCT crl-14518 31 31 " " PUNCT crl-14518 31 32 college college NOUN crl-14518 31 33 & & CCONJ crl-14518 31 34 research research PROPN crl-14518 31 35 libraries library NOUN crl-14518 31 36 50:181 50:181 NUM crl-14518 31 37 - - SYM crl-14518 31 38 94 94 NUM crl-14518 31 39 ( ( PUNCT crl-14518 31 40 mar mar PROPN crl-14518 31 41 . . PROPN crl-14518 31 42 1989 1989 NUM crl-14518 31 43 ) ) PUNCT crl-14518 31 44 . . PUNCT crl-14518 32 1 2 2 X crl-14518 32 2 . . X crl-14518 32 3 ian ian PROPN crl-14518 32 4 douglas douglas PROPN crl-14518 32 5 , , PUNCT crl-14518 32 6 " " PUNCT crl-14518 32 7 reducing reduce VERB crl-14518 32 8 failures failure NOUN crl-14518 32 9 in in ADP crl-14518 32 10 reference reference NOUN crl-14518 32 11 services service NOUN crl-14518 32 12 , , PUNCT crl-14518 32 13 " " PUNCT crl-14518 32 14 rq rq X crl-14518 32 15 28:94 28:94 NUM crl-14518 32 16 - - SYM crl-14518 32 17 101 101 NUM crl-14518 32 18 ( ( PUNCT crl-14518 32 19 fall1988 fall1988 PROPN crl-14518 32 20 ) ) PUNCT crl-14518 32 21 . . PUNCT crl-14518 33 1 3 3 X crl-14518 33 2 . . PROPN crl-14518 33 3 marjorie marjorie PROPN crl-14518 33 4 e. e. PROPN crl-14518 33 5 murfin murfin PROPN crl-14518 33 6 and and CCONJ crl-14518 33 7 gary gary PROPN crl-14518 33 8 m. m. PROPN crl-14518 33 9 gugelchuk gugelchuk PROPN crl-14518 33 10 , , PUNCT crl-14518 33 11 " " PUNCT crl-14518 33 12 development development NOUN crl-14518 33 13 and and CCONJ crl-14518 33 14 testing testing NOUN crl-14518 33 15 of of ADP crl-14518 33 16 a a DET crl-14518 33 17 reference reference NOUN crl-14518 33 18 transaction transaction NOUN crl-14518 33 19 assessment assessment NOUN crl-14518 33 20 instrument instrument NOUN crl-14518 33 21 , , PUNCT crl-14518 33 22 " " PUNCT crl-14518 33 23 college college NOUN crl-14518 33 24 & & CCONJ crl-14518 33 25 research research PROPN crl-14518 33 26 libraries librarie VERB crl-14518 33 27 48:314 48:314 NUM crl-14518 33 28 - - SYM crl-14518 33 29 38 38 NUM crl-14518 33 30 ouly ouly ADV crl-14518 33 31 1987 1987 NUM crl-14518 33 32 ) ) PUNCT crl-14518 33 33 . . PUNCT crl-14518 34 1 4 4 X crl-14518 34 2 . . X crl-14518 34 3 frances france VERB crl-14518 34 4 benham benham PROPN crl-14518 34 5 and and CCONJ crl-14518 34 6 ronald ronald PROPN crl-14518 34 7 r. r. PROPN crl-14518 34 8 powell powell PROPN crl-14518 34 9 , , PUNCT crl-14518 34 10 success success NOUN crl-14518 34 11 in in ADP crl-14518 34 12 answering answer VERB crl-14518 34 13 reference reference NOUN crl-14518 34 14 questions question NOUN crl-14518 34 15 ( ( PUNCT crl-14518 34 16 metuchen metuchen PROPN crl-14518 34 17 , , PUNCT crl-14518 34 18 n.j n.j PROPN crl-14518 34 19 . . PROPN crl-14518 34 20 : : PUNCT crl-14518 34 21 scarecrow scarecrow PROPN crl-14518 34 22 , , PUNCT crl-14518 34 23 1986 1986 NUM crl-14518 34 24 ) ) PUNCT crl-14518 34 25 . . PUNCT crl-14518 35 1 5 5 X crl-14518 35 2 . . X crl-14518 35 3 peter peter PROPN crl-14518 35 4 hernon hernon PROPN crl-14518 35 5 and and CCONJ crl-14518 35 6 charles charles PROPN crl-14518 35 7 r. r. PROPN crl-14518 35 8 mcclure mcclure PROPN crl-14518 35 9 , , PUNCT crl-14518 35 10 unobtrusive unobtrusive ADJ crl-14518 35 11 testing testing NOUN crl-14518 35 12 and and CCONJ crl-14518 35 13 library library NOUN crl-14518 35 14 reference reference NOUN crl-14518 35 15 services service NOUN crl-14518 35 16 ( ( PUNCT crl-14518 35 17 norwood norwood PROPN crl-14518 35 18 , , PUNCT crl-14518 35 19 n.j n.j PROPN crl-14518 35 20 . . PROPN crl-14518 35 21 : : PUNCT crl-14518 35 22 ablex ablex PROPN crl-14518 35 23 publishing publishing PROPN crl-14518 35 24 corp corp PROPN crl-14518 35 25 . . PROPN crl-14518 35 26 , , PUNCT crl-14518 35 27 1987 1987 NUM crl-14518 35 28 ) ) PUNCT crl-14518 35 29 . . PUNCT crl-14518 36 1 6 6 X crl-14518 36 2 . . PROPN crl-14518 36 3 terrence terrence PROPN crl-14518 36 4 crowley crowley PROPN crl-14518 36 5 and and CCONJ crl-14518 36 6 thomas thomas PROPN crl-14518 36 7 childers childer NOUN crl-14518 36 8 , , PUNCT crl-14518 36 9 information information NOUN crl-14518 36 10 service service NOUN crl-14518 36 11 in in ADP crl-14518 36 12 public public ADJ crl-14518 36 13 libraries library NOUN crl-14518 36 14 ( ( PUNCT crl-14518 36 15 metuchen metuchen ADJ crl-14518 36 16 , , PUNCT crl-14518 36 17 n n ADV crl-14518 36 18 .j .j PROPN crl-14518 36 19 . . PUNCT crl-14518 36 20 : : PUNCT crl-14518 36 21 scarecrow scarecrow PROPN crl-14518 36 22 , , PUNCT crl-14518 36 23 1971 1971 NUM crl-14518 36 24 ) ) PUNCT crl-14518 36 25 . . PUNCT crl-14518 37 1 7 7 X crl-14518 37 2 . . X crl-14518 37 3 peter peter PROPN crl-14518 37 4 hernon hernon PROPN crl-14518 37 5 and and CCONJ crl-14518 37 6 charles charles PROPN crl-14518 37 7 r. r. PROPN crl-14518 37 8 mcclure mcclure PROPN crl-14518 37 9 , , PUNCT crl-14518 37 10 " " PUNCT crl-14518 37 11 quality quality NOUN crl-14518 37 12 of of ADP crl-14518 37 13 data datum NOUN crl-14518 37 14 issues issue NOUN crl-14518 37 15 in in ADP crl-14518 37 16 unobtrusive unobtrusive ADJ crl-14518 37 17 testing testing NOUN crl-14518 37 18 of of ADP crl-14518 37 19 library library NOUN crl-14518 37 20 reference reference PROPN crl-14518 37 21 service service NOUN crl-14518 37 22 : : PUNCT crl-14518 37 23 recommendations recommendation NOUN crl-14518 37 24 and and CCONJ crl-14518 37 25 strategies strategy NOUN crl-14518 37 26 , , PUNCT crl-14518 37 27 " " PUNCT crl-14518 37 28 library library NOUN crl-14518 37 29 and and CCONJ crl-14518 37 30 information information NOUN crl-14518 37 31 science science NOUN crl-14518 37 32 research research NOUN crl-14518 37 33 , , PUNCT crl-14518 37 34 9:77 9:77 NUM crl-14518 37 35 - - SYM crl-14518 37 36 94 94 NUM crl-14518 37 37 ( ( PUNCT crl-14518 37 38 apr.-june apr.-june ADP crl-14518 37 39 1987 1987 NUM crl-14518 37 40 ) ) PUNCT crl-14518 37 41 . . PUNCT crl-14518 38 1 8 8 X crl-14518 38 2 . . X crl-14518 38 3 sandy sandy ADJ crl-14518 38 4 stephan stephan PROPN crl-14518 38 5 and and CCONJ crl-14518 38 6 others other NOUN crl-14518 38 7 , , PUNCT crl-14518 38 8 " " PUNCT crl-14518 38 9 reference reference NOUN crl-14518 38 10 breakthrough breakthrough NOUN crl-14518 38 11 in in ADP crl-14518 38 12 maryland maryland NOUN crl-14518 38 13 , , PUNCT crl-14518 38 14 " " PUNCT crl-14518 38 15 public public ADJ crl-14518 38 16 libraries library NOUN crl-14518 38 17 27:202 27:202 NUM crl-14518 38 18 - - SYM crl-14518 38 19 3 3 NUM crl-14518 38 20 ( ( PUNCT crl-14518 38 21 winter winter NOUN crl-14518 38 22 1988 1988 NUM crl-14518 38 23 ) ) PUNCT crl-14518 38 24 . . PUNCT crl-14518 38 25 . . PUNCT crl-14518 39 1 9 9 X crl-14518 39 2 . . X crl-14518 39 3 saundra saundra PROPN crl-14518 39 4 williams williams PROPN crl-14518 39 5 and and CCONJ crl-14518 39 6 eric eric PROPN crl-14518 39 7 wedig wedig PROPN crl-14518 39 8 , , PUNCT crl-14518 39 9 '' '' PUNCT crl-14518 39 10 improving improve VERB crl-14518 39 11 government government NOUN crl-14518 39 12 information information NOUN crl-14518 39 13 and and CCONJ crl-14518 39 14 documents document NOUN crl-14518 39 15 reference reference NOUN crl-14518 39 16 skills skill NOUN crl-14518 39 17 through through ADP crl-14518 39 18 a a DET crl-14518 39 19 staff staff NOUN crl-14518 39 20 development development NOUN crl-14518 39 21 program program NOUN crl-14518 39 22 , , PUNCT crl-14518 39 23 " " PUNCT crl-14518 39 24 rq rq X crl-14518 39 25 24:143 24:143 NUM crl-14518 39 26 - - SYM crl-14518 39 27 45 45 NUM crl-14518 39 28 ( ( PUNCT crl-14518 39 29 winter winter NOUN crl-14518 39 30 1984 1984 NUM crl-14518 39 31 ) ) PUNCT crl-14518 39 32 . . PUNCT