m S m S KBBBSSaSSSk LIBRARY OF THE UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN 510.84 U1 -P -P ^H C CO H CD CD rti E 3 -H e tr-p O CD M U >-i fO •p ,c w to CD C -H P -H H CD C M W M Xi CD CO £ C -H co -H <~\ c CT fl C w o -p CD CO C O CO CD -P CO CD CD TJ •H P O CD -P m o c o -H -P •H U CO CD P rd U CD cm CD P cn •H 12 boundaries. The concept of partial understanding allows the system a better interaction with the user. This is represented on the flow diagram by the feedback going from the translator to the user. The nature of a comment is a function of the distance of the rejected request from the accepted set and tells the user what to do in order to rephrase his sentence in a better way. This "partial understanding" feature makes the system cooperate with the user to achieve understanding. 2. 2 The Domain of Discourse We have described in the first chapter the purpose of the system. Its objective is to guide students through the library of computer sciences lessons, according to their goals and past performance. Users get selective access to the information they need by means of a request typed in English. These requests fall broadly into several categories: - Requests for lesson names given specifications like keywords, authors, date, ... "Give me a lesson about Fortran and numerical analysis." "What PL/1 lessons should I take before October 10?" - Requests for intrinsic information attached to each lesson, besides their names, like abstract, author, length, level of difficulty, ... "Who wrote lesson Somaga?" "Tell me about my next lesson." 13 - Requests for performance achieved by the student. "What grade did I get in the exam Fortxl?" "How much time did I spend in Fortif?" - Requests for requirements imposed by an instructor for students in a course. "When should I take my next test on Fortran?" "What grade should I get in Fortxl?" - Requests for counseling. "What lessons should I review before my next exam?" "Is Fortdo a better lesson than Fortif?" We have illustrated these categories by listing a few requests. Many more will be listed in the following sections (sections 2.5, 3.4 and 4.1 in particular). A short inspection reveals that the domain is quite large and that sentences can be phrased using past, present or future tenses. When we initially fixed the boundaries of the set of "acceptable" requests, we voluntarily discarded the last category because it appeared that in most cases the system could not have enough information to process them and algorithms would be too time consuming. However, we did decide to implement counseling routines to be activated periodically or at some critical points. These routines verify that a student properly follows the class schedule, meets the requirements of a course in which he is registered, and occasionally advises him which lessons to review for his own sake. In short, these 14 routines allow the Guide to assist a student in following a satisfactory program of study. 2. 3 The Data Bases When a request has been fully interpreted and accepted by the translator, its intermediate representation is handed out to another component, the request processor, which uses the information stored in several data bases to process the request and give an answer back to the user. Four data bases and one inverted file have been found necessary for this purpose Let us now describe the need for these data bases by means of examples. The concept space Let us first look at the following two requests and answers : student: "Show me what you have on bubble sort." Guide: "The following lessons may be relevant: lesson name: sorting abstract : Introduction to sorting theory and algorithms. lesson name: Files abstract: Survey of file organization and file processing techniques. " student: "Do you have a lesson about recursion using Fortran?" Guide: "I do not have such a lesson. Recursion is not a feature of Fortran." 15 In the first request, the Guide did not find a lesson having "bubble sort" as a keyword; however, it has expanded the request searching then for lessons having keywords closely related to "bubble sort." In the second request, the request processor has searched the library of lessons looking for one having recursion and Fortran as keywords. None was found. However, the system has also given an additional comment: "Recursion is not a feature of Fortran. " For these two requests, the underlying operations, which have allowed the system to expand the first request and give additional comment to the second one, have been made possible by having stored in memory an arrangement of concepts specific to computer science, which cover all the topics treated in any of the courses. This arrangement consists of a classification scheme which links various concepts together by means of semantic relations. Different models for the classification of concepts have been proposed. At one extreme there is Quillian's model [9] used by several deductive systems and which represents semantic information by general recursive graph structures. Its semantic map contains loops and different kinds of linkages. Simpler models can be found in the thesaurus of document retrieving systems. A thesaurus groups words together according to the subject concept to which they are related and concepts are organized on a hierarchical basis. This organization is 16 similar to the Dewey Decimal Classification ( DDC ) which is one of the best known and most frequently used library classifica- tion systems in the United States. The arrangement which seemed the most suitable for the Guide lies between these two extremes. It consists of a graph where the nodes are sets of intrinsically similar concepts, and the links between the nodes are semantic relations like: "x is a predecessor of y," "x is an important concept of y," "x is not a concept of y ..." To each node is attached a level indicating the position of the concept in the underlying hierarchy. The portion of the concept space in Figure 3 illustrates how the Guide has been able to expand the first request. The concepts are represented in this data base by identifiers which have the same length and occupy 10 characters (60 bits). The full length word is stored in another data base which is used by the translator to recognize English words from a request. The correspondence between the identifier and the full length word is best illustrated by a few examples: keyword up to 9 characters: Identifier (10 characters) FORTRAN FORTRAN (1_) keyword up to 19 characters : NUMERICAL IaNALJYSIS NUMEANAL_ (18) 4 10 14 C o •H -P rfl O -H 17 w CD & ■H o CD •P rfl •H U O -P rfl C •H XI 6 O u •H M-l C •H +J M O co -p o co m CD A .H 3 CD to l-i rfl rfl W to •H •H X c o ■H ■P rc H d) Sh IH O to CD a > EH >1 U rfl a w ■p ft CD O e o u CD P m o p c cd e rfl M fa CD U 2 •H fa 18 keyword up to 29 characters FIXED POINT REPRESENTATION 3 10 13 20 23 FIXT RTAT (26) The binary value of the last character indicates the length of the keyword. The catalog This data base represents the library of computer sciences lessons. It can be seen as a graph where - a node represents a lesson of the library and some inherent information related to it - arcs between nodes express relations between lessons . Let us look at the following requests to see the nature of information and relation we need to store. R: "What is the abstract of lesson Fortif?" R: "Give me an advanced Fortran lesson written by Smith. " R R: "Do you have any lessons on Fortran and numerical analysis?" "Can I take lesson Fortdo before fortio?" The information stored in each node indicates the name of a lesson, its type (lesson, exam, test), its length, author, level of difficulty, abstract, and a set of weighted keywords which allow the retrieval of lessons with high precision and relevance. The relations between lessons are set by the manager of the library of lessons and indicate such relations as : 19 "x should be taken before y," "x can be used to practice for y," ... These relations impose an order on the library of lessons which is useful primarily for casual users and for students who are not registered in a course and who learn by themselves. A second level of similar relations is set in another data base which we will describe later and which reflects the educational strategy of the instructors of courses. The organization of the catalog influences strongly the performance of the system since the retrieving of lesson information, given some of its attributes, is a frequent operation. A sequential file of lessons ordered alphabetically according to their names, where the attributes of each record indicate information inherent to the corresponding lesson, allows access to these records by a simple binary search when- ever the name of a lesson is given. However, this structure would be prohibitive to retrieve lessons , given a list of keywords , since the number of operations involved would be considerable. It then becomes necessary to include an inverted file where the attributes of each record are a computer science keyword and a list of all the lessons having such a keyword. Several inverted files could be built, as many as the number of attributes of the records of our first file. But these files would be used less frequently and would use much memory space, which is our main constraint. Figure 4 describes these two files. 20 CD CD CD CD e e e e ro ro f0 03 C C C C 1 C C C C 1 O O O 1 CO to S CO CO 1 CO CO o CO CO 1 Pi CD CD H CD CD 1 W H H En rH rH W 1 1 S 1 1 1 En W S Eh H H o p H l-H 0L| c o CO CO CD o •H 4-> g O m •H U ■H CO C •H >H -p c •H U CD £ 4-> O O -P CO >H CD P C -H O a e o •H -P rfl ^ e c • £1 CJ U o CD ■P 1 >i U $ 3 ,Q CD o A CD U c •H rH CD M ■H C "W CO ■P 4-1 O O •H -P rC T3 CO CD a) CD 3 o u T3 P o c c ^ H ■H CD O CD > 3 3 & > CD & CJ 1 > c >H CD CD CD •H ft CO co M C O CD e CO CO •H C CD C U O C CO CD CO 3 CD tf-H CD CO CD 4: 1 -P CD •rH CD m -P 3 H ^ ro ■H •H u P p c 4-> QJ rt 3 er CD OJ C CO O a p as U CD A P m o e o •H P CO N •H C (0 C7 M O CD U 3 •H fa 21 The student record In order to be able to answer requests like: "What grade did I get in my last test?" "How much time did I spend in Fortxl?" "How many lessons have I taken so far?" "What is the number of times I took lesson Fortdo?" another data base is needed which indicates the path followed by each student through the library of lessons and the per- performance that has been achieved. Part of this information is stored automatically by the PLATO System when a student is taking a lesson. The organization of this data base can be seen as a linear list of nodes, one list for each student, where a node indicates the name of a lesson taken, the student's performance, when the lesson has been taken, and the length of time spent on it. The information stored in this data base can be retrieved at the demand of the student or of the instructor of the course in which the student is registered. This information is very useful for providing statistics to evaluate lessons. The course outline This last data base is used to allow the instructor of a course to specify a schedule and various requirements. The type of information which is needed is best seen by looking at a few requests : 22 "What grade should I get in the exam Fortxl?" "What lessons should I learn before September 10?" "Is Fortdo a prerequisite of Fortif?" "When is my next exam?" "How much time should I spend in Fortdo?" "How many lessons are required in the course cs 101?" The number of lessons used in a course is usually small, typically less than 20. Hence the organization of the data has little influence on the overall performance. It consists of a directed graph where the nodes are the lessons used in the course along with the performance expected in the lessons , and proper paths through the graph correspond to the sequence in which lessons should be taken. It is well known that many topics can be taught in different ways. One instructor may prefer to start teaching Fortran by discussing I/O statements first, for example, while another one adopts a different strategy. It is, therefore, essential to allow an instructor to arrange the set of lessons corresponding to his course, i.e. , to allow him to specify his own teaching strategy. The relations which the instructor enters into the course outline are intended to complement those in the Catalog. They may even contradict them if an instructor feels that a specific part of the Catalog was not designed properly . The main difference is that the course outline is intended for a specific audience, while the Catalog is intended for all users. The course outline might be updated at the beginning of each semester by an instructor 23 who modifies his strategy by looking at statistics obtained from the student record data base, or by a new instructor. Figure 5 illustrates part of the structure of this data base. 2. 4 Method for the "Understanding" of Requests We started the search for a method by first analyzing the properties of the domain of discourse, in order to tailor the method to the real needs. We have outlined in section 1.3 the underlying philosophy of the method. Let us now describe it in more detail. One major feature is that the sentences we deal with are requests; and requests, independently of their domain, have in general a simple syntax. If we denote by W what a person wants to know and by S the specifications limiting the domain of interest , we see that a number of requests have the general structure W.S: "Let me see the abstract of lesson Fortdo." "What is the location of the ship sea wolf?" [5] "What is the departure time of flight AA-57 from Chicago?" [14] The last sentence, for example, breaks into two parts: "What is the departure time" "flight AA-57 from Chicago" The syntax, of course, is not always as simple as that, but it is essential to note that the number of different syntactic structures of requests is small. The parsing of a sentence, as described previously, is not defined in a very precise manner. The boundary between the W and S parts for c o to to tO 0> 10 •H U U Q) X ^ S > the word "leave" provides additional information for the W part, which could otherwise have been the same if the person had asked for an arrival time. It is a common observation that any word in a sentence is revealed by its context and is itself part of the context. All words within a given context interact with one another. These trivial observations lead us to our method which is based on the interaction of words and simulates the thought process of a person reading a context dependent sentence. Let us then describe this thought process. When we read a sentence, we anticipate naturally the meaning of context dependent words and we memorize automatically the place in the sentence where it occurred. We take the most plausible interpretation of the word and we keep it as lon<-* an the succeeding sequence of words does not undermine our confi- dence in our previous guessing. When the evidence comes that 26 the interpretation is wrong, we substitute for it another one by changing the meaning of previous context dependent words. The process is in some respect similar to the traversal of a graph where one node can lead to several others with different probabilities. We simulate this thought process by means of a non-deterministic automaton which identifies the role of each element of a sentence and produces a formal representation reflecting the grammatical structure of the sentence. We found this model appropriate for our application but it would certainly be unsufficient for more general situations. As any automaton, it has input and output symbols and states : The input symbols are English words. They represent a rational classification of words according to the meaning they have in particular contexts. The output symbols are functions. A request is mapped into a nested composition of these functions. The composition reflects the original structure of the sentence. The states represent a partial degree of understanding and context , which allows the system to generate appropriate comments to an unnacceptable request by looking at the sequences of states and output symbols which have been produced. The state diagram represents the syntax of acceptable requests. It 27 results that syntactic and semantic analysis are not two separate phases but are merged together. This adds a great deal of flexibility and power to the method. Since this automaton is non-deterministic, each state can lead to several others and each transition has a level of confidence. This level indicates how much confidence the automaton gives to a transition. We will describe in more detail this automaton and its operation in Chapter 3. 2. 5 The Intermediate Representation of Requests The intermediate representation is an interface between the users and the retrieving components. Its nature is particularly important since it influences strongly the per- formance of the system. There are as many different inter- mediate languages as there are such systems. At one extreme they are based on extensions of the predicate calculus; at the other extreme, they take the form of an n- tuple. When we designed the intermediate language, we wanted it to have the following properties: - It should be able to represent the largest possible number of relevant requests. One way to achieve this goal is to make the language complete. However, no program has so far been able to map all relevant requests into a corresponding formal representation; 28 completeness in a practical sense cannot be achieved. To remedy this limitation, the request processor in general does not answer strictly the formal request, but expands it to encompass a broader one. - The translation process should be as simple as possible. - The formal representation of the request should be such that it permits fast and efficient processing by the retrieving components. We have chosen to represent the original request of a user by a nesting of functions: f(g(h(x))), where the functions are the output symbols of the automaton. To each function corresponds a set of processing routines. Once the sentence is translated, the request processor processes it from the innermost function to the outermost by calling the appropriate retrieval routines. We considered the library of lessons as a hyperspace whose dimensions depend on the maximal number of characteristics we attached to any lessons. Some functions return a set of elements from the hyperspace while others return specified characteristics of the elements of this set. The functions may be partitioned into two groups: W' and S'. The W and S parts of an English sentence (see 2.4) are translated into a composition of functions of W' and S'. 29 Functions of W The twelve functions of W have as argument a set of lessons. The argument set is defined inside the hyperspace by a function of S 1 or a composition of functions of W and S'. TF : tests if the set is empty or not AB : returns abstracts of its elements AN : returns names of author of its elements NB : returns the size of the set GR : returns the grade required by the instructor GO : returns the grade obtained by the student TS : returns the schedule to achieve relative to the elements of the set TT : returns the schedule achieved by the student PQ : returns the set of lessons which are prerequisite to the argument set SQ : returns the set of lessons which are sequels to the argument set SI : returns the set of lessons which are similar to the argument set BE : has two arguments a lesson and a set. Tests if the lesson belongs to the set. Functions of S * : S' has 2 functions which return a set of lessons depending on the value of their variables. LN LS has two arguments , a course name and a lesson name. It returns the lesson defined by these. returns the set of lessons which are defined by characteristics other than their names. These characteristics might be a Boolean list of keywords, a type (lesson, exam), a course to which they belong, an author name, a level of difficulty, a sequence specification, a time period, or other specifications, such as whether the lessons have already been taken or not. Any of these characteristics can be specified or negated. Example of compositions of functions : - Request - Translation "Do you have a lesson on Fortan and numerical analysis?" TF (LS (lesson, Fortran & numerical analysis)) 30 - R : "How many prerequisites are required for lesson Fortif in course CS 101?" - T : NB (PQ(LN( Fortif , CS 101))) - R : "Should I take Fortdo after Fortif?" - T : BE (LN( Fortdo ) ; SQ (LN( Fortif )) ) - R : "Give me a lesson similar to Fortif." - T : SI(LN(Fortif ) ) Looking at these few examples, we see that the inter- mediate language has the desired properties mentioned previously. The formal representation reflects the structure of the original sentence and makes the translation process easier. The English request is understood as a statement which defines the subset of a set or asks about its characteristics. Hence it prepares and facilitates the task of the request processor. 2. 6 The Request Processor As we mentioned earlier, the implementation of this module has been left as a subsequent task. Let us give here a brief description of its basic functions: It is clear from what precedes that the complexity of the processing varies from one request to another. This processing is, of course, considerably facilitated by the intermediate representation, and let us outline here that when we chose the representation for the formalization of the sentences, we tried to balance the difficulties of the translation process into this representation versus the efficiency of the retrieving process. This efficiency is certainly essential for the overall per- formance of our system. 31 The first step taken by this component is to analyze the interpreted request and decide which actions to perform and in which order. This decision process is based on the information handed out by the translator (i.e., the nesting of functions and the value of variables). The following logical steps are then performed at a lower level and depend to some extent on partial results. A few examples will best describe the functions of this component : - The request: "What is lesson Fortif about?" is interpreted as AB(LN( Fortif)). It necessitates one access to the record of lesson Fortif in the "Catalog" and the retrieval of its abstract. - The request : "What are the lessons recommended next after Fortif in course CS 101?" is represented as: SQ(LN( Fortif, CS 101)). The processing is of similar complexity. It consists of an examination of the various links in the "course outline" relating lesson Fortif to the other lessons of course CS 101. A set of lessons (the sequels of lesson Fortif in CS 101) is then determined from this analysis. - The request: "How many Fortran lessons are there in CS 101?" is interpreted as: NB(LS ( lesson , Fortran, CS 101)). The request processor first determines from the "course outline" the set of lessons of 32 course CS 101, then it determines from the "Catalog" which elements of this set have "Fortran" as a keyword. The ordering in which the searching of the data bases is done is important in regard to the performance achieved. For this request, the "course outline" is accessed first because a course consists typically of less than 20 lessons while the "Catalog" will contain a larger number of lessons having "Fortran" as a keyword. The processing of requests, however, will not always be as simple as described above. The section (2.3) on the "concept space," shows in particular the ability of the request processor to expand the user's original request to a slightly different field of discourse, for which it has enough information to provide an answer. 33 3. THE TRANSLATOR 3. 1 General Description To understand adequately the features and capabilities of this system it is necessary first to describe how a request is processed. The interpretation of a sentence consists essentially of finding an acceptable computation in the space of all the computations which can be performed by the non- deterministic automaton. Let R : w n w_ ... w. ... w be the request and L. 1 2 l n ^ l denote the number of characters in the substring w. ... w . * l n w. and w. , are words separated by one or more blank characters. l l+l ^ 2 The translator processes the request from left to right, with possibly several backups by the following algorithm. step 1 Initialize i to and the current state of the automaton to its initial state. step 2 Increment i by 1. step 3 if L. =0 and the automaton is an accepting state, the request is processed. if L. =0 and the automaton is not in an l accepting state, go to step 6. if L. ^ try to match a substring starting with w. with words in the vocabulary data bases. step 4 If match unsuccessful go to step 2, otherwise get next transition from the automaton. (This transition is determined by the current state and the input symbol associated to the substring just matched. ) 34 step 5 If the output symbol of the transition indicates that a comment should be generated go to step 6 Otherwise take the action indicated by the transition, modify accordingly the value of i and go to step 3. step 6 Generate appropriate comment. The treatment of a request, as described by this algorithm, is visualized on the screen by the underlining of words as they are processed. When a transition in step 5 leads to the dead state, the value of i is decremented and the processing starts again from the last previous word where there was nondeterminism. The rightmost part of the line is erased accordingly. This visualization has the main advantage of keeping the attention of the user and showing him at any time the stage of the processing of his request (although this processing is fast: 3 seconds on the average). We will describe steps 4, 5 and 6 in more detail in the following pages. 3. 2 The Vocabulary Data Bases The words w. , which are recognized during the processing of a request, are of different types: - "grammatical" words (nouns, pronouns, verbs, auxiliary, adjectives, ...) - keywords specific to computer sciences - lesson names - author names - course names 35 When a word of the request is matched against one of these words, it serves as an input symbol to the automaton. The correspondence between the word and the input symbol depends on the type of the word: - For grammatical words, it has been found useful to have state-dependent input symbols (i.e., the symbol representing a word depends on the current state of the automation). This scheme makes it necessary to store with each grammatical word, a list of input symbols (as many as the number of states). This feature permits the reduction of the size of the automaton and provides more flexibility in its design. (See Figure 6.) - For the other types of words , the input symbol representing a word is independent of the current state of the automaton. Each type of word is always associated to the same input symbol. This allows a substantial saving in memory space since it is not necessary to store additional information with each word. The matching of the words of the requests against the words of the dictionary is done by binary search. A hashing technique would have been unfeasible because it is often necessary to match phrases (consisting of several words) in the request in one comparison (for example "numerical 5 [Q 36 co u o CO cu -p -p co -p c CD U U 3 U u •H +J e 6 CD <£> i — 0^ / ) ) ) H H I f • • • • • • j ) H H H \ H + H ( •H ■H \ £ 5 - | O O 1 H H . . — i CU u en •H 37 analysis," "numerical integration," or "programming," "programming language," "programming technique" ...). The dictionary of words consists of several data bases in which the words are ordered alphabetically. A word is assigned to a data base according to its type and its length. The data bases having the longest "words" are searched first. This allows to distinguish phrases of different lengths where one is the beginning of another: ("information," "information theory, " . . . ) . The dictionary consists of the following data bases : - 2 data bases for "grammatical words" holding words respectively up to 9 and 19 characters long - 4 data bases for computer science keywords holding words respectively up to 9 , 19, 29, 39 characters long - 1 data base for lesson names - 1 data base for author names - 1 data base for course names A user can consult any of these last seven data bases before formulating a request. It is important that he uses keywords or names stored in the dictionary since otherwise they are ignored during the processing of the request. The area of memory reserved for the dictionary is large enough to store 1400 keywords of 29 characters each, 200 lessons and author names. 38 Editing facilities are provided to update easily and rapidly the dictionary. This operation occurs whenever an instructor adds a new lesson to the library or when a new course is created, It is similar to adding a new node to a graph. First the node is defined, then relations with other nodes are set. 3 . 3 The Automaton The heart of the translator is a nondeterministic automaton where : - The i nput symbols represent categories of words , partitioned according to a rational classification scheme which allows a substantial saving in memory space at the expense of a small increase in computational complexity. Another interesting feature of this scheme is its inherent flexibility: grammatical words can be added to the vocabulary data bases without modifications of the automaton transitions. Let us outline the basic characteristic of the classification: Words of the same category belong to the same grammatical class (i.e., nouns, verbs, prepositions, ...) and are: - either context dependent synonyms ("abstract," "content," "topic," ...) ("from," "since," ...) - or words which according to a particular context, have different meaning, but 39 CD N co •H TJ 03 O TJ C a) 05 u CO c o 73 LD M •H CM O -P 5 Ol » >i C) to i ffl <-\ ■H CO CO CD -p -P Iti CD 6 05 CO e 6 as CO T3 •H 03 oJ C CD P M c C -p x: rd Q) CO rt U g +J e M CD u 03 g P o CO •H a) 03 & CO XJ M n u s CO +J 3 c m XT' o CD 3 O -H z CJ H 05 r CO 13 O to -P •H XI I O <& o CM ro >1 CO S-i CO CD CD CO X! 4-> CO 03 -P CJ 03 PQ o) ^ CD m CD U 03 Sh M •H 0) -P < a C Q -P 03 03 \ d) CD O -P f— 1 ^ Vf VL< CD CO Q CJ X) 3 03 & 03 u •rH 03 -P O CQ •H e H p -P .p 03 •H Pti P 03 03 C 03 rij -p ■P >i > U 03 CO Q < Q IH >i M 03 P X! 03 O o > CD XJ -p m e o ■H co ■P CD 03 W N rd •H CQ C 03 ro cn-p n rC O Q CD M P C7 ■H CD CO TS O CO -p •H Xl I o 40 serve the same purpose. This is the case of words which correspond to the different values of a function variable. Let us give some examples: ("lesson," "exam," "test") define the type of a library "lesson" ("elementary," "advanced," ...) define a level of difficulty, ("January," "Monday," "today," ...) define a date. The semantic distinction between these words is made by a routine which also sets the value of the corresponding function variable or flags according to the case. The automaton has 26 input symbols. - The states reflect a partial degree of understanding and the state diagram (see Figure 8) reflects the syntax of the accepted requests. The automaton has 11 states including a dead state and 6 accepting states. Let us give a brief description of the state diagram: s is the initial state; s, , s or s_ is the current state when the part of the request telling what the user wants to know has been . processed. Transitions occur from s to one of these states depending on what the user asks for and the syntax of the beginning of the request : "grade required for ..." leads from s to s. "my grade in ..." leads from s n to s^ "is p^lif a sequel ..." leads from s n to s~ 41 W CD P ft +J W CD -P t? (ti c -P •H W -P ft T3 0) 03 o CD o T3 rt u •H P CO -H c •H e M CD -P CD I C o CD x: p m o 6 XT rfl •H Q (D -P C o fO -P e o -p < CO CD S-l XT 0© 42 "grade I got in ..." leads first for s n to s - a dead state changes then the transition from s to s 3 A transition occurs to one of the remaining states when a specification defining the domain of interest of the user is processed. Several backups may then occur. 11 states have been found sufficient to process a large number of requests. By adding more states, request with unusual syntax like: "about lesson p^lif, what is its abstract?" could have been accepted. We did not do it because adding more states would have doubled the size of the automaton and the concept of partial under- standing allows the system to detect such structures and tell the user how to phrase his sentence. A state can lead to up to four states which are tried in a certain order. Moreover each transition has a level of confidence. When this level is low, the partial interpretation which has been produced is scanned and accordingly either the processing goes on or some comments are generated. An o utput symbol indicates : - either a function which will appear in the intermediate representation of the request. Different functions can be attached to an 43 English word, depending on the current state of the automaton and on the next state chosen. This is a basic feature of the approach we have taken. If this function has variables, the input symbols tell which variable is to be set (15 output symbols are used for this purpose ) . - or a flag to be set . Several flags have been found useful in memorizing a few contexts and permitting less and faster computations (6 output symbols). - The type of a comment to be generated when a transition reveals that the request is out of range or too complicated syntactically or semantically (6 output symbols). 3.4 The Generation of Comments The concept of partial understanding, inherent to our approach, allows the generation of specific comments whenever a request is outside the accepted set. Appropriate comments prevent the user from becoming frustrated and help him to use the system in a better way. All the comments are generated by a single routine which receives as input the type of the comment to be displayed and assembles the resulting comment from a number of stored messages which can be used in different circumstances. During this operation, the routine may use the sequence of output 44 symbols or states which has been produced to output a specific statement. It also keeps track of the number of previous comments and uses it to reinforce the comment generated. Its operation is summarized by the schema in Figure 9. Example of Comments R: What do I request? C: Please look at the purpose of the Guide and examples of requests. R: What grades did I get? C: I understand you want to know your performance. Your request is not specific enough. R: What grades should I get? C: Your request is not specific enough. R: What should I do until the fifth of January? C: Sorry, I cannot answer this request. Do not use the word which is underlined. R: Show me the abstract of pildo and the name of its author. C: Please phrase simpler requests. R: What lessons did I study since the 10? C: When you specify a date by a number, it must be preceded by the name of a day or of a month. R: Give me a lesson about Fortran or pil and numerical analysis. C: Indicate only a conjunction or a disjunction of computer science keywords. R: Is pil do a better lesson than Fortdo? C: Sorry, I cannot answer this request. I don't make such comparisons. When successive requests are refused, additional comments are given each time: 45 co -p c •H T3 CO CD CO -P CD rO U U U CD 3 C co CD cy rt CO •H p co CO CD CD CO -p U g CD c ft CJ CD Sh -0 e 0) CD 3 e P >H CO o C o u ■H p 4-1 CO *w rH rfi 4-1 M •H o CD 0) p A ft M p g >1 (TJ CD 3 p ft CO C CT> CD U •H Ph 46 Rl : Lesson Somaga , who wrote it? C: Please phrase simpler request. R2 : Somaga, who wrote it? C: Your request is not yet acceptable. Phrase your request differently. Indicate first what you want to know then specify which lesson. R3 : Who are the authors of lessons Somaga and pilif? C: Again! You really give me a hard time Please phrase simpler requests ! You have indicated too many lessons. 3 . 5 Theoretical Considerations We have given in the previous sections the description of a cybernetic system which processes information by means of a non-deterministic automaton. We present in this section a formal description of this automaton and justify the choice of this model rather than a deterministic or stochastic one. We can characterize our model by: ND = [I, 0, S, h] where I: denotes the set of input symbols (i). For reason of simplicity we can identify here I with the set of English words or phrases stored in the dictionary, 0: denotes the set of output symbols (o). The inter- mediate representation of a request consists of a composition of elements of this set. S: denotes the set of states (s). h(i,s): denotes the nonempty set of ordered pairs (o,s' ) from which the output symbol (o) and the next state (s 1 ) can be chosen in the situation (i,s). 47 The need for such a model has been explained previously. We can justify it again shortly by establishing a parallel between the multiple interpretations of certain words or phrases in a sentence in regard to its overall meaning and the multiple possibilities of ordered pairs (o,s') which can result from a situation (i,s). We implemented a non-deterministic Mealy automaton (see Figure 10) rather than a Moore model. Let us now justify the choice of this model rather than a deterministic or stochastic one. Deterministic automaton . In a deterministic machine, the set h(i,s) contains only one pair (o,s'), that is, the output symbol (o) and the next state (s 1 ) are uniquely determined in a given situation (i,s). The fact that words or phrases in a sentence may have different interpretations excludes this model. Stochastic automaton . This category of machines has been found useful for modelling certain forms of behavior and in particular to clarify the notion of intelligent behavior. With this model, a pair (o,s' ) is chosen from the set h(i,s) with a probability which depends only on the situation (i,s). This property makes the model unfeasible for our application since we must choose the pair (o,s r ) according to the past history of the situation (i,s). In our non-deterministic model the pairs (o,s') are taken from the ordered set h(i,s) according to the order of the occurrence of the situation (i,s). 48 •H H M C "o "o "o \ \ \ H CN C CO CO ••CO • CO CN CO CN "o c o ■p fd E o -p CD X •H -P CO •H c •H 6 U CD ■P i -P CO to c o •H -P rfl -P ■H fa P to 3 u 56 We will now show how our method could have been used and take a representative sample of requests (listed in [14] ) processed by this system as the domain of discourse. 4.4.1 Intermediate Language Using the same terminology as before, a request will be mapped into a composition of functions belonging to W' and S' . Functions of W : AL returns Airline Company name DT returns a Departure Time AT returns an Arrival Time NB returns the size of the set defined by the argument TF Test if the set defined by the argument is empty or not. (True False) Functions of S ' : S' has only one function of seven variables: FS (VI, V2, V3, V4, V5, V6 , V7) which consist of flight specifications and can be negated. VI : flight quantifier V2 : flight name V3 : Airline name V4 : Town of departure V5 : Town of arrival V7 : departure time DT V8 : Arrival time AT 57 with VI number comp. number comp < | > | < I > V7 date prep. date V8 date prep. date prep before after FS return the set of flights which verify the specifications. 4.4.2 State Di agram The state diagram (see Figure 12) is simpler than the one of the Guide because the meaning of words of the requests is not so context dependent. However, the number of states and functions of W or S ' would increase if we want to process more complex requests than the ones given as a sample by Woods in [14]. 4.4.3 Translation of a Sample of Requests The sample of requests listed in [14] would be interpreted as follows: - Doesn't American operate flight AA-57? TF (FS(1, AA-57, American, _, _, _, _) ) - Isn't AA-57 an American Airlines flight? TF (FS(1, AA-57, American, _, _, _, _) ) - Does American have a flight which goes from Boston to Chicago? TF (FS(1, _, American, Boston, Chicago, _, _) ) - Does American have a flight which doesn't go from Boston to Chicago? TF (FS(1, _, American, -iBoston, Chicago, _, _) ) - What American Airlines flight goes from Boston to Chicago? FS (1, _, American, Boston, Chicago, _, _) - What American Airlines flights go from Boston to Chicago? FS (_, _, American, Boston, Chicago, _, _) 58 _ - W (what the user wants ) (flight specification ) Figure 12. State Diagram for Airline Guide [14] 59 What is the departure time of AA-57 from Boston? DT (FS(_, AA-57, _, Boston, _, _, _) ) What is the departure time from Boston of every American Airlines flight that goes from Boston to Chicago? DT (FS(_, _, American, Boston, Chicago, _, _) ) What American Airlines flights arrive in Chicago from Boston before 1.00 p.m.? FS (_, _, American, Boston, Chicago, _, < 1.00 p.m. How many flights that go from Boston to Chicago does American Airlines operate? NB (FS(_, _, American, Boston, Chicago, _, _) ) What is the number of flights from Boston to Chicago? NB (FS(_, _, _, Boston, Chicago, _, )) 60 LIST OF REFERENCES [ I] Alpert, D. , Bitzer, D. L. , "Advances in Computer Based Education," Science , 167 (1970), pp. 1582-1590. [2] Bobrow , D. G. , "Natural Communications with Computers," Bolt-Beranek Newman (1967). [3] , "Problems in Natural Language Communications with Computers," Bolt-Beranek Newman (1967). [4] Codd, E. F. , "A Relational Model of Data for Large Shared Data Banks," Comm. ACM , 13, (1970), pp. 377-387. [5] Craig, J. A., Berezner, S. C. , Carney, H. C. and Longyear, C. R. , "DEACON: Direct English Access and Control," AFIPS Conference Proceedings , Vol. 29, The Thompson Book Company, pp. 365-380. [6] Engles , R. W. , "A Tutorial on Data- base Organization," Annual Review in Automatic Programming, 7 (1972), Pergamon Press, pp. 1-64. [7] Green, B. F. , Wolf, A. K. , Chomsky, C. and Laughery, K. , "BASEBALL: An Automatic Question Answerer," in E. Feigenbaum and J. Feldman (Eds. ) Computers and Thought , New York: McGraw-Hill (1963). [8] Nievergelt, J., Reingold, E. M. , Wilcox, T. R. , Watanabe , D. S. , Friedman, H. G. , Montanelli , R. G. and Pradels , J. f "ACSES : An Automated Computer Science Education System," to appear in Proc. 1974 National Computer Conference, AFIPS Conference Proceedings, Vol. 43, AFIPS Prsss. [9] Quillian, M. R. , "The Teachable Language Comprehender " Comm. ACM , 12 (1969), pp. 459-476. [10] Salton, G. , "The SMART Retrieval System," Prentice-Hall, Inc. , (1971). [II] Sherwood, B. , et al., "aids," PLATO IV Lesson, Computer- based Education Research Laboratory, University of Illinois at Urbana- Champaign, Urbana , Illinois. [12] Simmons, R. F. , "Natural Language Question-Answering Systems," Comm. ACM , 13 (1970), pp. 15-30. 61 [13] Winograd, T. , "Understanding Natural Language," Cognitive Psychology , 3 (1972), pp. 1-191. [14] Woods, W. A., "Procedural Semantics for a Question Answering Machine," AFIPS Conference Proceedings , Vol. 33, Part 1, The Thompson Book Company (1968), pp. 457-471. [15] , "The Lunar Sciences Natural Language Information System," Bolt-Beranek Newman (1972). [16] , "Transition Network Grammars for Natural Language Analysis," Comm. ACM , 13 (1970), pp. 591-606. 62 APPENDIX A DETAILED STRUCTURE OF THE TRANSLATOR This appendix contains a more detailed description of the translator. First we list the grammatical words which can be recognized during the processing of a request, then we describe which automaton states are involved in the interpreta- tion of various requests, and give a sample of words which cause a transition from one state to another. 1. List of Grammatical Words We called "grammatical words" the words which need to be recognized in the sentence and which are not specific to the domain of discourse, like computer science keywords, lesson names, author names or course names. These grammatical words are the most important ones for determining the interpretation of the request, and most state transitions are caused by them. The dictionary contains about 250 of them. Before listing these words, let us point out that the dictionary of English words has some idiosyncrasies due to our concern of saving memory space and having fast interpreta- tions. The dictionary contains a few expressions consisting of two or more words which are considered as individual entities and hence recognized in "one operation." Let us 63 take an example to show why this has been done. A sentence beginning by "what is" will probably yield a different interpretation than a sentence beginning by "is." If every single word is recognized, then it is necessary to have a large number of states and assign different transitions to each word. This approach complicates unnecessarily the design of the automaton and is moreover costly in terms of memory space required for both the automaton and the dictionary. It would also go against our design principles of having each state representing a partial degree of understanding. Instead we included in this dictionary expressions of words like: "what is," "how do," "next after," "last time," "my grade," ... . This solution is inevitably pragmatic, but contributes greatly to a substantial saving in memory space and to a faster interpretation of requests. The following words may partially indicate facts a user is interested in, depending on where each individual word occurs in a sentence and also on the preceding and succeeding parts of the sentence: 64 (abstract, description, outline, subject, summary, topic, information) ( author , who ) (how many, how much, number) (grade, level, performance, standing) (hour, how long, schedule, time, when) The following words may define a set of lessons in terms of another one : (prerequisite, requirement; continuation, sequel) (like, same, similar, related to) The following words may define some attributes of a " lesson" : - its type: (lesson, exam, test) - its level of difficulty: (advanced, difficult, easy, elementary, hard, introductory, simple) - its purpose: (exercise, practice, practicing, theoretical, theory, to program) - a date or a period: (January — ^ December, Monday — f Sunday, yesterday, now, today, tomorrow) The following words indicate the verbs and their tenses , which are recognized : (are, do, give, go through, have, is, learn, list, need, show, study, take, teach, tell, want, will) (did, had, gone through, learned, saw, seen, shown, studied, taken, took, was, went through, were) Prepositions : (about, after, and, before, between, but, by, during, except, first, from, following, in, last, next, not, no, on, or, previous, preceding, since, succeeding, to, until, without) 65 The groups of words considered as single entities : (how are, how bad, how can, how do, how good, how have, how is, how well) (my grade, my level, my performance, my standing) (what are, what did, what do, what have, what is, what will, what should, what would, what else) Finally there are the words which indicate independently of the state of the automaton that the request has a high probability to be out of range, or words which cannot be interpreted by the translator, due to its limitations: (why, reason, advice, improve / this, these, those its, their / week, month, day, semester, year / one, two, three, ... / second, third, ... / 'comparatives and superlatives') When these words are recognized, comments are displayed on the screen of the user, expressing the limitation of the system. Besides the words listed previously, numbers are also recognized and accepted only in the context of a date or period specification. Otherwise, appropriate comments are generated. As we mentioned earlier, this classification of words is superficial. Words within one group can be interpreted differently depending on the context, and they can cause different state transitions. However, this partition is useful to give an idea of the words stored in the dictionary and will be helpful to give an overview of the mechanism of the automaton, 2. The State Transitions The sentences accepted by the translator can be divided into three major categories which correspond to a partition of the states of the automaton into three subsets: 66 - The first category consists of requests which ask for information specific to some lessons or for information about the performance or the schedule to achieve by a student among the various lessons of a course he is registered in. - The second category consists of requests which ask for the performance or schedule achieved by a student in the various lessons he has already taken. - The third category consists of requests about the existence of a set of lessons or about the inclusion of a lesson defined by its name into another set of lessons. 2. 1 Interpretations of the Requests of the First Category The interpretation is done by the following subset of states : state s : initial state state s : The current state is s when the automaton has made a guess about what the user wants to know. state s^: The current state is s^ when the automaton has recognized or is waiting for words which define a set of lessons by some attributes different from a name. It is in this state and during transitions from another state that the variables of function (LS) are set. 67 Figure 13. Subset of States 68 state s • The current state is s. when a lesson is 4 4 defined by its name in the request and when there has been an interpretation about what the user wants to know about it. (The variables of function ( LN ) are set. ) dead state: A transition to the dead state occurs from either s, , s or s. when some word in the sentence indicate that the interpretation of previous word is incorrect. We list below a sample of words which cause transitions. F. . and S. . denote the sets of words for which T. . is the first ID ID ID transition or a subsequent one (second, third or fourth), respectively. ("Abstract," "author," "time" "prerequisite," "similar," "number," "grade" ) ("what is," "want," "lesson," "take") ("lesson," "take") no words (words which can indicate a value for the variables of function LS , "on," "by," "about," ) no words (a lesson name) (a course name) words which indicate that the request belongs to the second category: ("obtained," ...) or which modify a previous interpretation: example of interpretations : " abstract of a lesson on numerical analysis . " V S l S 5 S 5 S s T 01* F oi : s oi : T : 05 F 05 : S 0s : T : 15 F : 15 S 15 : T : 14 F 14 : S 14 : T Id' T 4d' sd wore t-n t- AB (LSdesson, numerical analysis) 69 " What is the abstract of lesson pllif ?" V S : S l S 5 D S * S l S l S 4 AB(LN(p£Lif ) ) " What is pllif about ?" S 0= S D S D s 1 44 AB(LN(pMif ) ) "What lessons should I take before pllif in cs 101 ?" U D D D D U 3 3D 5 14 4 4 PQ(LN(pilif , cs 101) ) 2. 2 Interpretations of the Requests of the Second Category The interpretation is done by the following subset of states : - The significance of the states S_, S r , S_ is the same as the one of states S. , S„ , S,. described 1 ' 4' 5 previously. However, the set of words causing a transition to the dead state is different. T 02 : F 02 : (" m Y grade," "my performance," ...) S-. ("grade," "time") "0 Figure 14. Subset of States 71 T_,, T__: Words which cause transitions from S„ to S, or S 7 are respectively the same as those which cause a transition from S. to S. or S r . 14 5 T 9 , , T , , , T_, : These transitions do not occur for words — — — — like ("obtained," "got") as they did previously from states S S S . example of interpretations : " Tell me my grade in the last exam . " S Q : S Q S 2 S ? S ? GO(LS(exam, last) ) " Tell me the grade I got in my last exam , " V S S l D S S 2 S 2 S 7 S 7 S 7 GO(LS(exam, last) ) " When did I take pjlXl ?" S Q : S x D S 2 S 5 D S 2 S 2 S 6 TT(LN(pilXl) ) 2. 3 Interpretations of the Requests of the Third Category The interpretation is done by the following subset of states: These requests begin typically by words like ("is," "any," "have"). The first guess by the automaton is that the request is about the existence of a set of lessons defined by some 72 \ T \ T 03! \ 09 Figure 15. Subset of States 73 attributes. Hence the first transition which occurs in T-... where S n has the same significance uy y as Sj- or S 7# If subsequent words indicate that this partial interpre- tation is wrong then the transition T__ is attempted. The interpretation anticipated is that the request is about the inclusion of a lesson uniquely defined into a set of lessons. The significance of states S_ and S q is the same as the one of states S„ and S,_ or S^ and S_. 09 03 39' 38 09 09 03 03 39 39 38 38 ("is," "do," "any," . . . ) no words no words ("is," "do," "any," . . . ) (lesson names, "prerequisite," "sequel," "before," "after," . . . ) no words no words ("prerequisite," "sequel," "before," "after," ...) example of interpretations : " Have you a lesson on numerical analysis ?" V S 9 S 9 TF(LS(lesson, numerical analysis)) " Have you the abstract of lesson p Hi i f ? " V D D S l S 5 S S 1 1 AB(LN(p^lif ) ) D S 74 " Is p t'li f a preregui si te of p lido ?" S 9 D S 3 S 9 S 3 S 3 D S 3 S 3 D S, 8 8 BE(LN(pnif), PQ(LN(pildo) ) ) V Is p ■": 1 i f a lesson on conditional statements ?" D S, S 9 S 9 3 9 9 9 9 BE ( LN (piilif ) , LSdesson, conditional statement) 75 APPENDIX B THE DATA BASE ORGANIZATION We have given in section 2.3 a description of the logical structure of the Concept Space, the Catalog, and the Course Outline. We present here a scheme for the implementation of these data bases which allows efficient memory allocation and search operations. Estimates have shown that, when the system is used, the three data bases can be kept entirely in an area of restricted size (4186 60-bits words) of the Extended Core Storage (ECS), a level of the computer memories hierarchy for which transfers of data with the main memory occur at a fast rate (ECS access time = 5 microseconds, Transfer rate = 600 millions bits/second), These estimates are based on the following figures: 1024 computer science keywords, 200 concepts, 128 lessons and 16 courses. The implementation scheme is illustrated in Figure 16. Each data base has a directory, relating a record identifier to the storage location of the corresponding record. A hashing function is used to determine the entry in the directory which corresponds to a record identifier. The records are of variable length and have a field called the record descriptor, which P CD U E ffl o a p in M-l •p ts a p a) o u u c ry> > p TS o ^* P >H O 03 U -P a; fO p u 76 d) £. CD -P C ■H E rH o +j P D 4-1 O T3 0) P U f"P ■H P l Q ^' ; o I psj: f ' pq~ -p j u w a) Q r a 1 p o u CD 1 g ! * I • i/ i In- ^ / * •H >-< ! « ! LI -m i r R P •: l a i i •H u j H 1 — a) ■ — I 1 Q i V \ u 1 o . ! °L * i » : ' \/\ \\ \ >1 p qj a; c p w •H u ^H 0) a -P p o P •H u o Q c o •H p rd N ■H c (3 C7> P O a) w rrj PQ rd P ro Q (D P •H 77 describes the internal organization, i.e., the location and the length of the other fields. Organization of the Concept Space The directory . A record identifier is a computer science keyword which has been mapped into a fixed length key (see section 2.3). This directory is the largest one since it will contain approximately 1000 entries. Some record identifiers are linked by a ring; they correspond to computer science keywords which represent the same concept. Only one of them gives the location of the record. This scheme implies a linear search over a few elements to access a record but makes the updating of the data bases more efficient, as we will explain later. This ring is also used by the request processor to obtain the list of all the keywords which characterize a concept. The records . Each record represents a computer science concept and has a field called the record descriptor which describes its internal organization. The information contained in these records has been described in section 2.3. The list of lessons which are relevant to a concept is also added to the corresponding record. Since each entry in the directories has a fixed location, it is a list of pointers rather than symbolic names. These pointers indicate the address of some entries in the Catalog directory. This representation contributes to a substantial saving in memory space (8 bits instead of 60 bits to represent a lesson in the list). Hence 78 by adding this list to each record, the Concept Space serves also the purpose of the inverted file described in section 2.3. Organization of the Catalog and Course Outline The organization is essentially the same as described previously. A record of Catalog has a lesson name as an identifier and contains an information which characterizes a particular lesson. The list of computer science keywords attached to each lesson is represented in a record by a list of pointers indicating some entries in the Concept Space directory. A record of Course Outline has a course name as an identifier and contains an information relative to the requirements and schedule of a course. Lessons are represented also by pointers to some entries in the Catalog directory rather than by symbolic names. Maintenance Deletions of records create holes of various sizes. Insertions of new records could then be performed by using the "first fit" or the "best fit" algorithms as described by Knuth, but whatever is the method, holes remain and give to the area reserved for storing the data bases , the appearance of being "checkerboarded. " The restricted size of this area prohibits these insertion methods. In our system, updating operations do not occur frequently, but mainly at the end of a semester. Hence the "compacting of the memory," i.e., moving the records until the holes have been coalesced into a single 79 hole, whenever one or several records have been deleted, is an appropriate solution which allows an efficient use of the area. In each record a pointer is added which indicates the location of the record identifier in the directory. Whenever records are moved this pointer allows to update directly the corresponding entries in the directories. The use of this pointer makes the compacting operation much more efficient than if the updating of the directories was done afterwards, particularly if we consider that the number of entries in the Concept Space directory is about five times the number of the records of this data base. 80 VITA The author, Jean Louis Pradels, was born in Chavagnac , France on February 5, 1946. He received a "diplome d' ingenieur 1 from the Ecole Superieure d' Electricite' in Paris in June 1970 and his Master of Science degree in Computer Science in June 1971 from the University of Illinois. Since September 1972 he has been a research assistant in the Department of Computer Science of the University of Illinois at Urbana-Champaign. He is a member of Sigma XI. BIBLIOGRAPHIC DATA SHEET 1. Report No. UIUCDCS-R-7^-626 4. Title and Subtitle . THE GUIDE - AN INFORMATION SYSTEM 3. Recipient's Accession No. 5. Report Date March 197*+ 7. Author(s) Jean Louis Pradels 8> Performing Organization Rept. No. 9. Performing Organization Name and Address University of Illinois Department of Computer Science Urbana, Illinois 6l801 10. Project/Task/Work Unit No. 11. Contract /Grant No. GJ 31222 12. Sponsoring Organization Name and Address National Science Foundation Washington, D.C. 13. Type of Report & Period Covered 14. 15. Supplementary Notes 16. Abstracts This thesis describes the design and implementation of a conventional information system, called the Guide „ The environment in which it must work is sufficiently different from that of other systems which have similar goals, that new techniques had to be devised to make its realization possible. This thesis also shows that these technqieus are applicable in contexts other than ours, and would probably lead to improved performance in a number of information systems based on other approaches . 7. Key Words and Document Analysis. 17a. Descriptors information system natural language computer-assisted-instruction 7b. Identifiers /Open-Ended Terms 7c. COSATI Field/Group J. Availability Statement >RM NTIS-35 ( 10-70) 19. Security Class (This Report) UNCLASSIFIED 20. Security Class (This Page UNCLASSIFIED 21. No. of Pages 22. Price USCOMM-DC 40329-P7I *v v> Sv tttf 5 1977 EraBB MB nW ncnsCVS nEwnHnff #5 HB n RSSfl BB v'?C r £^ TfwTlitflli Mi ■ Wm Wffl JjDHJXwIaBlKBiHlUnillgBtl SIS si H SI HIImTOIm m m m ■Mb m WWBW H BB $8 khbrShqs HtsosfinSra MafflKfiEBBjaHB 111