LIBRARY OF THE
UNIVERSITY OF ILLINOIS
AT URBANA-CHAMPAIGN
510. $4-
T*
p
RETURN failure
No
n n
. Yes
RETURN success
DEFINE program NP
^"PARSE oV.
No
RETURN failure
< s rjETERMINER/ > '
|Yes
^^PARSE o\
No
it
< \^ NOUN^^
jYes
RETURN success
DEFINE program VP
^^PARSE a\.
No
k
OCTI If,
->*
(_:■.'.-_
^^VERB^X^
nt lUnn i ui iui e
^■^s^
i
i
TVes
•^TRANSIT IV E^>
res ^
— -xf
'^PARSE\ s Yes
a NP J>
JNo
_jNo
s' is if ^s.
omtrawqitiufS
No
v^iiaj i r\MiMoi i ivtL^
jYes
41
RETURN success
Figure 2.4 Flow diagrams for a simple program
grammar (from Winograd (1974))
30
approach are the inability to handle ungrammatical inputs, and the amount
of processing time required for the parse.
2.2.3 Hybrid Approaches
It seems only reasonable to try to combine some of the best
aspects of both pattern matching and linguistic approaches into a single
parser. One such attempt is the dialog understanding process of the
SOPHIE CAI system discussed in Section 2.1. This parser is based on the
idea of a "semantic grammar" (Burton (1974)), one in which the syntactic
categories usually found in a grammar are replaced by concepts which have
semantic meaning in the domain of discourse (see Figure 2.5 for an example
taken from another application: NLS-SCHOLAR (Grignetti, et al . (1974)).
Thus the grammar rules express how each semantic concept may be expressed
in terms of constituent concepts. As Figure 2.5 shows, the terminals of
such a grammar are patterns of English words. In actual use, the grammar
of Figure 2.5 would be hand coded into LISP procedures. Execution of
these procedures is controlled by the input string just as Winograd's
PROGRAMMAR procedures are.
Developing such a grammar involves the same analysis of the
problem domain and probable inputs as the pattern matching approach does.
A semantic grammar can also be made to ignore some words in the input
stream or handle ungrammatical inputs. However, since the grammar, once
developed, is expressed as LISP procedures, it allows the same ease of
interfacing with procedures to query a world knowledge base as Winograd's
PROGRAMMAR grammar. This provides a capability to determine pronoun
referents and handle ellipses similar to the linguistic approach.
Disadvantages are that it is as difficult to extend to other problem
3]
:=
:=
DEFINE
WHAT DOES MEAN
WHAT DOES STAND FOR
WHAT DOES DO
:= WHAT IS THE PURPOSE OF
WHAT IS THE CONTENT OF
WHAT IS THE LEVEL OF
WHAT IS THE ADDRESS OF
WHAT ARE EXAMPLES OF
WHAT IS THE DEFINITION OF
WHAT IS
WHAT IS
WHAT ARE
WHAT ARE AT
WHAT ARE IN
WHAT IS
**ALSO 'TELL\ME, GIVE\ME, TELL\ME\ABOUT ' IN
PLACE OF 'WHAT IS'
:= WHAT CONTAINS
:= WHAT ARE IN
:= WHAT ARE AT
:= HOW\DO\I
TELL\ME\HOW\TO
TELL\ME\ABOUT
:= WHAT [NLS\COMMANDj
:= WHERE AM I
WHERE IS THE CM
WHERE IS
:=
DO IT
DO
Figure 2.5 BNF description of part
of a semantic grammar
32
areas as the traditional pattern matching approach, yet it requires more
processing time during execution.
33
3. GETTING TO KNOW PATTIE
3.1 Introduction
In this chapter we will describe the structure of PATTIE in some
detail. In particular, we discuss how various features of the design
relate to one another, and how they're involved in the interaction with
the student. Some of these features have been influenced by the particular
problem PATTIE is concerned with, or by the general area of programming.
Others are quite general.
The three most significant aspects of PATTIE 's design are:
(1) the use of an AND-OR graph to represent knowledge
of both the possible solutions and the processes
used to develop those solutions;
(2) the interaction control routine which traverses this
graph, conducts an interactive dialog with the
student, and maintains a screen display of the
student's developing solution;
and (3) the existence of a student model, intimately tied
in with the AND-OR graph and based on a list of
concepts relevant to the problem area.
The discussion in the remainder of this chapter is centered on the use of
these features to teach top-down programming techniques. However, it must
be realized that the techniques used in PATTIE may be adapted to teaching
top-down problem solving in any field in which that technique is applicable,
34
Thus, while the particular graph and concept list used by
PATTIE are highly problem specific, such graphs and concept lists may
be developed for problems in many different subjects. The interaction
control routines are totally independent of the specific problem area
PATTIE deals with, namely programming and differentiation. Even portions
of the vocabulary used in the natural language understanding routines may
be adapted to other subject areas (see Section 3.3.2).
3.2 Relation to Other Tutors
PATTIE is only one part of a large system to teach introductory
programming skills to students. Her specific task is to illustrate the
concepts involved in top-down program development by assisting the student
through a dynamic example of such a process, much as a private tutor would.
To perform this task properly, PATTIE must be able to understand student-
suggested refinements described in English, provide hints based on the
current state of the student's proposed solution, and extend that solution
if possible and necessary. How does the design of such a tutor relate to
those other problem solving tutors with similar abilities discussed in
Chapter 2?
Kimball's and Goldberg's tutors emphasized the need to give the
student a wide range of experience, and were capable of handling a
correspondingly wide range of both prestored and student-suggested problems,
To accomplish this, heuristic problem solving routines, with capabilities
similar to those of the students being tutored, were integral parts of the
system. Such an approach was possible due to the quantitative nature of
the subject areas. Solving problems in integration or proving simple logic
theorems requires using only a small number of rules applicable to many
35
problems (Kimball's tutor used only 12 rules to solve over 70 problems;
Goldberg's used 15 rules for over 25 proofs).
Unfortunately, in top-down programming there is no small set of
general rules which can be applied in many situations. There is, instead,
a very large number of distinct refinements which are applicable in only a
small number of instances. (For example, PATTIE recognizes 61 different
refinements applicable to a single sizeable problem. And those 61
refinements may be described in English in over 450 ways!) Using this
"problem solver" approach in PATTIE would require developing an automatic
programming routine which itself used the stepwise refinement method and
which could understand and apply such a large number of refinements. Such
a routine is vastly beyond current capabilities (see Green, et al . (1974)
for a status report on automatic programming systems).
If the problem solver approach is not viable, perhaps an approach
similar to that of Koffman and Blount is possible: develop a collection
of specialist routines which can determine the correctness of a portion of
a program, and combine their abilities to examine the student's proposed
solution to one of a number of predefined problems. For our purposes,
there are two flaws in this method. First, this approach is primarily
concerned with judging the solution program, and not at all concerned with
how the solution is developed. Koffman and Blount, in fact, require the
student to go directly from the problem statement to producing code, exactly
the procedure PATTIE is trying to combat. Secondly, it would be difficult
to implement, as the ability to mechanically judge the correctness of
programs written in a high-level language (and correct them when wrong)
is very limited (see Sussman (1973) and Goldstein (1974) for discussions
of the problems in debugging simple LISP and LOGO programs).
36
Finally, the problems of trying to simulate the execution of a
program consisting of PL/1 statements intermixed with actions described
(with varying degrees of precision) in English rule out using the
approach taken with SOPHIE. It seems that the very nature of the
successive refinement process forbids using the designs used in previous
tutors.
Hence, one of the primary problems faced in developing PATTIE
was to provide her with a knowledge structure which exactly described
acceptable ways of developing solutions, as well as the acceptable solutions,
while simultaneously satisfying the design criteria expected in a good
instructional program (good modularity, separation of knowledge and control
functions, relative ease of development of the data base, and effective
interaction with the student). As with SOPHIE, the detail of knowledge
required has forced us to limit PATTIE' s domain of expertise to a single
problem. This does not mean, of course, that she is limited to dealing
with only one problem forever. But adding additional problems will require
a sizeable amount of work to develop the necessary data bases, even though
the control program may be used as is.
The particular problem PATTIE is concerned with is not of vital
import, but the problem must meet certain criteria to achieve the desired
instructional objectives. First, the problem area should either be one
with which most students are familiar, or one which can be easily and
clearly explained. Otherwise, students make many errors and waste a good
deal of time trying to understand the problem rather than intelligently
pursuing a solution. Secondly, the problem must be large enough, relative
to the student's problem solving ability, that the student can't grasp the
1/
solution immediately. Otherwise, the value of using the top-down solution
method to reduce complexity to a manageable degree is not apparent. But it
is still necessary that the basic approach to the problem be fairly straight-
forward, so that successive refinement is an acceptable solution method.
Short but "tricky" problems can easily be found whose solution is not
immediately apparent, but then the essence of the solution lies in seeing
the trick, rather than in applying a problem solving method. Since a
majority of the problems beginning programmers encounter have fairly
straightforward solutions, eliminating tricky problems from those PATTIE
may be concerned with is not a significant restriction.
The actual problem PATTIE has been concerned with during the
course of this research is that of writing a PL/1 program for taking the
symbolic derivative of a sum of terms of a polynomial of two variables.
3.3 What PATTIE Knows — The Data Bases
The information PATTIE needs to guide a student to the solution
i
of the problem can be divided into three separate areas:
(1) knowledge of what the valid solutions to the
problem actually are;
(2) enough knowledge of human discourse in the
problem area to understand proposed refinements;
and (3) information about the student and his performance
during the course of solution development.
Let's look at each of these in turn.
3.3.1 The Solution Graph
3.3.1.1 AND-OR Graphs
PATTIE represents knowledge about acceptable solutions to the
example problem as an AND-OR graph. Such graphs (like that illustrated in
38
Figure 3.1(c)) have been used in a number of areas of artificial
intelligence research (Nilsson (1971)).
The basic idea behind an AND-OR graph is reducing a problem
to a series of subproblems, just as in stepwise refinement. In such a graph,
each node represents a problem. Solving the problem represented by an AND
node (Figure 3.1(a)) can be accomplished by solving all the subproblems
represented by the successor nodes. Solving the problem represented by an
OR node (Figure 3.1(b)) can be accomplished by solving any one of the
subproblems represented by the successor nodes. The solution to the
initially given problem (represented by the root of the graph) is
successively reduced to the solution of sets of subproblems, some of which
might be immediately recognized as being solved (LEAF nodes) , others of
which might need further reduction. Thus in Figure 3.1(c), solving the
complete problem (node R) involves solving node M or_ node N. Solving node
M involves solving nodes A and B, and so on.
In practice, AND-OR graphs are dynamically grown from the root
node, using a small number of "problem reduction operators." Clearly,
growing such a graph is a trial and error process, complete with deadends
and backtracking, which could halt as soon as any solution is discovered.
A full graph, such as represented by Figure 3.1(c), would be developed only
if all possible solutions were desired.
It is obvious how such a graph could represent the refinement
process. The root represents the initial problem to be solved, leaves
represent statements in the target programming language, and each intermediate
node represents a subtask on one of the levels of refinement. Thus the
process of developing a solution by stepwise refinement is equivalent to
tracing a path from the root to some subset of the leaves which represent a
39
Figure 3.1 Sample AND-OR graph
40
program to solve the problem. The immediate descendants of any given
node represent tasks which are refinements of the task represented by
that node.
Figure 3.2 illustrates a simple example of such task relationships.
Intuitively, an OR node corresponds to a point in the development of a
solution where a choice must be made between several (equally correct)
approaches. An AND node represents a point at which refinement involves
several tasks which must all be done to solve the problem.
For the same reasons that an automatic programming approach was
rejected for PATTIE (primarily the large number of possible refinements
and the qualitative nature of the successive refinement process), it would
be difficult for PATTIE to grow an AND-OR graph as is commonly done.
Instead, a "problem expert" supplies PATTIE with a complete AND-OR graph
representing all good (well-structured and correct) solutions to the
example problem, as well as a few anticipated bad approaches beginning
students might attempt. Figure 3.3 shows a small portion of PATTIE' s
refinement graph. The complete graph for PATTIE 's problem contains 150
nodes and 339 branches.
The claim of being able to represent all reasonable solutions to
a problem, as well as a few wrong approaches, as an AND-OR graph clearly
requires some justification. Intuitively, one feels that the number of
possible solutions must increase enormously as the complexity of the
problem increases. Undoubtedly, this would be true if we were considering
every correct solution; however, we are considering only "good" solutions,
which makes an important difference. Many of the possible solutions
(especially the solutions proposed by beginning students) may be immediately
rejected because they violate some obvious criterion of goodness. Thus
41
>-l
01
00
e
OJ
o
4J
u
c
u-t
•H
-C
u
S-J
II
*
«
o>
2
oi
oc
01
OJ
01
o
3
D
>-i
■u
4J
^H
iH
c
«
Cfl
u.
•H
r-l
>
>
I
2
cu
03 2
3
>>
u
ja
iH
CO
>^
U
i-H
I Ol
•H
i en
■U
•H
.H
5
3
U
6
0)
J=
T3
■u
C
o
CO
e
O!
X!
H
4J
II
r— 1
II
Ol
2
3
Figure 3.2 Refinement as an AND-0R graph
42
Figure 3.3 Partial solution graph
4 3
paths representing inefficient or poorly structured solutions need not
be included in the graph, even though such solutions may be correct in
a strict sense. At most, such solutions will be one of the plausible
wrong approaches. Further, the use of a graph form, as opposed to a
tree, allows maximum common utilization of subportions of the graph.
These factors significantly reduce the number of nodes the solution
graph must contain, and enable us to use such a representation.
3.3.1.2 Special Features
In order to use an AND-OR graph as the basis for a tutor of
successive refinement, several features were added. As Figure 3.3 shows,
branches between nodes are tagged with English phrases ("transition
phrases") which are usually descriptions of the tasks represented by the
node each branch leads to. Thus, nodes represent subproblems to be solved,
and branches are tagged with English descriptions of the subproblems they
lead to.
PATTIE uses these transition phrases to determine the path the
student is taking through the graph. Section 3.4 contains a detailed
description of how this is done, but the basic idea is simple: at any
time during development of the solution, one node in the graph represents
the task the student is currently refining. PATTIE matches the refinements
suggested by the student against the transition phrases associated with
branches leaving that node, and can thus determine the correctness of each
refinement and its effect on the solution.
Other branches (leading to LEAF nodes) are tagged with PL/1
statements and represent the final step in refinement of a particular task,
namely translation into the target programming language.
44
Some nodes and branches in Figure 3.3 are tagged with underlined
English phrases. These phrases represent one of (currently) 16 programming
or problem solving "concepts" which the problem expert may associate with
any node or branch. Concepts available are listed in Figure 3.4. The
programming concepts indicate major groupings of PL/1 language features,
thus specifying the general area of the target language to which a given
node or branch is most closely related. The groupings are primarily based
on those used in recent introductory programming texts using PL/1 (Conway
and Gries (1973), Weinberg, et al . (1973)). The problem solving concepts
are similar in intent, but indicate approaches to solving problems in
general. They represent the essence of the problem solving techniques
recommended in Polya's (1957) classic work, and of suggestions for finding
refinements in relation to a top-down approach to program development
(Conway and Gries (1973), Wirth (1973)).
While PATTIE doesn't require that every node and branch be tagged
with a concept, these concepts form the basis for her model of the student.
Thus the greater the amount of concept tagging done by the problem expert,
the greater PATTIE' s ability to aid the student and adapt her expectations
to his previous performance. And a small list of concepts is applicable
to a wide range of situations; in PATTIE 's refinement graph, 94% of the
nodes and 80% of the branches are tagged with one of the 16 concepts.
The remaining features added to the basic AND-OR graph formalism
are special branch or node types, illustrated in Figure 3.5. Special
ERROR branches (Figure 3.5(a)) allow the problem expert to provide specific
hints to be given to the student only in certain contexts. These ERROR
branches may run from either AND or OR nodes, and lead to LEAF nodes which
have error messages attached. ERROR branches tagged with transition phrases
45
programming
arrays
assignment statements
blocks
built-in functions
conditional statements
declarations
I/O
loops
recursion
subprocedures
problem solving
draw a picture
solve part of the problem
simplify the original data
find several distinct subcases
find a series of sequential actions
use additional variables
Figure 3.4 Programming/problem solving concepts
46
ERROR
Maybe it will help
to divide T into
pieces?
ERROR
divide T into
separate pieces
2 pieces.
pieces
How many pieces
will you use?
4 pieces
ERROR branches
(a)
Figure 3.5 Refinement graph special features
47
Mixing PL/1 and English
(b)
TERM: PROC(T,Z)
END TERM;
PROC node
(c)
Figure 3.5 Refinement graph special features
48
("expected" ERROR branches) correspond to bad approaches the problem
expert felt students were likely to attempt at that point, and the error
message can explain why that approach is not good. Untagged ("universal")
ERROR branches lead to error messages which simply suggest explicit actions
which are probably needed at that point.
A second special branch type was needed to handle the common
practice in top-down program development of intermixing partial programming
language statements with English descriptions of refinements. For instance,
once the need for a DO group has been established, the descriptions of
the actions to be done are surrounded by DO and END statements. Thus PATTIE
needed a way to both display PL/1 statements and accept refinements at a
single node. The solution (illustrated in Figure 3.5(b)) was simply to
allow branches to be tagged with PL/1 statements and marked so as to be
displayed as soon as the node is encountered, but not traversed. In
Figure 3.5(b), then, the DO is displayed as soon as the node is encountered,
the refinements are accepted as at any AND node, and the END is then
displayed.
The final special feature is the PROC node shown in Figure 3.5(c),
which allows invocation of a subroutine before it is programmed in detail.
When encountered in the refinement of one procedure, the PROC node acts
like a LEAF, but also causes the interaction control program to stack the
PROC as the root of the new procedure's subgraph, for later detailed
development. At that time, the branches leaving the PROC node are
displayed to provide the initial screen environment for programming the
new procedure.
To summarize, then, there are four types of nodes: ANDs, ORs,
LEAFs, and PROCs. A node may be tagged with a concept, associated with a
49
display message (for example, an error message), or not associated with
anything but the branches leaving it. There are three types of branches:
regular branches, which are tagged with a transition phrase and perhaps with
a concept; ERROR branches, which may or may not be associated with a
transition phrase; and immediate display branches, which are tagged with
PL/1 statements and are displayed as soon as they are encountered in the
traversal .
3.3.1.3 Advantages
There are several advantages of representing knowledge of the
solution as an AND-OR graph. For one thing, it is compact. PATTIE's
refinement graph of 150 nodes requires only 226 60-bit words for storage,
an average of less than 1.8 words per node. Error messages and displayable
transition phrases require another 700 words of memory, but the overall
average is still only 6.4 words per node.
Another advantage is that it allows separation of specific
problem knowledge and knowledge of how to interact with students, which
reduces the difficulty of adding new problems to PATTIE's repertoire.
For someone who understands and practices top-down programming, developing
a refinement graph is not a difficult matter. Essentially, it requires
solving the problem and maintaining a trace of the refinements and decisions
made. Reexamining that trace for decision points where other approaches
are equally valid identifies OR nodes, and additional subgraphs leading
from these nodes must be developed. Finally, ERROR branches (and perhaps
additional portions of the graph) can be added as a result of observing
students using the initial graph. Entering or modifying a graph in PATTIE's
data base is a simple (if somewhat time consuming) process using an editor
developed for the purpose.
50
A third advantage is that a single representation can provide
knowledge of the student's solution process as well as acceptable
solution programs. The solution process is described by a path through
the graph, and knowledge of the student's current position on such a
path simplifies PATTIE's control program. If the student gets stuck
at any point in the refinement process, PATTIE can immediately extend
his solution by simply traversing branches leaving the node representing
the current task. When a task is ready to be translated into the
programming language, a branch leaving the current node will be tagged
with a PL/1 statement which performs the required action. If the student
is truly lost, PATTIE can let him back up in the graph to a previous
decision point and try a different approach. And any acceptable solution
program is represented by some subset of the LEAF nodes of the solution
graph.
3.3.2 Natural Language Understanding Features
Any system which conducts a natural language dialog with a
person must be concerned with methods of understanding natural language
inputs. In Chapter 4 we discuss aspects of language understanding which
determine which technique is best suited to a system such as PATTIE.
However, natural language processing was not of prime interest in this
thesis, beyond the need to provide PATTIE with enough understanding to
allow for fairly natural description of refinements. The PLATO IV author
language, TUTOR, provides a facility for natural language understanding
(Tenczar and Golden (1972)) which PATTIE uses and which has proven adequate
for her needs.
Essentially, this facility is a keyword recognition, pattern
matching scheme. An author specifies a vocabulary, consisting of a number
i
51
of disjoint classes of synonymous words (groups of "content" words) and
a list of words which are allowed in a student's inputs but which carry
no meaning ("ignorable" words). Elements of a synonym class may be
single words or "phrases," which are a series of two or more words which
must appear contiguously. Phrases provide a simple means of handling
common idioms, and may consist of ignorable words, content words appearing
elsewhere in the vocabulary, or completely new words.
The purpose of a language understanding system is to assign a
meaning to the typed inputs. TUTOR'S facility attempts to assign a meaning
by matching the input against a series of stored patterns. Each pattern
consists of representatives from one or more of the classes of synonymous
words in the vocabulary. Since there are usually many ways of expressing
an idea in natural language, it is frequently necessary to attach more than
one keyword pattern to a single "meaning list." For example, since keyword
order and number of keywords are important in a pattern match, if it was
desired to assign the same meaning to the inputs "a brown cat," "a cat
that is brown," and "a cat" (assuming "cat" and "brown" are content words
and other words are ignorable), the meaning list must include. the patterns
"brown cat," "cat brown," and "cat."
Figure 3.6 illustrates this v/ith a small portion of PATTIE's
vocabulary and a few sample meaning lists. PATTIE's vocabulary contains
484 root words and phrases, and over 700 words if different endings are
considered. There are 61 separate meaning lists containing a total of
452 patterns.
Analyzing a student input is then a straightforward process,
performed by the PLATO IV system. Keywords are extracted, the list of
patterns searched, and the first matched pattern (if any) is identified.
52
vocaoulary
* ignorable words
* content words
(al locate, create, DCL, declare, del .define)
(i f , IF, 1 f *then, IF*THEN, 1 f*ther.*else, IF*THEN*ELSE)
(var 1 ab 1 e , var , temporary , storage)
(asterisk, mult ipl icat ion*sign,star)
(aster i sks , doub 1 e#star , exponent i at i on*s i gn)
(coe f f i c i ent , coe f , coe f f i c i ent * st r i ng)
(equa 1 , equa 1 s , equ i va 1 ent , same* as)
(exponent , exp, exponent *of*z ,exp*of*z)
(expression, string, term, T,character#str ing#T)
(decrease, decrement , deduct , reduce, subtract)
(delete, el iminate, get.*r id*of , remove, separate, spl it)
( f i nd*product , mu 1 1 i p 1 y , product , t i mes , t ake*product )
(one, 1)
* meaning lists
meaning del
dc 1 var
meaning coef term
remove coe f
remove coef term
mean i ng mu 1 1 1 p 1 y coe f exp
mu 1 1 i p 1 y coe f -t i mes exp
mu 1 1 i p 1 y exp coe f
mu 1 1 i p 1 y exp 1 1 me 3 coe f
coe f equa 1 coe f t i mes evp
coef equal evp times coef
coef product coef e>p
coe f product coe f t i mes exp
coef equal product coef exp
coe f equa 1 product coe f t i mes exp
coef product exp coef
coe f product exp t i mes coe f
coef equal product exp coef
coef equal product exp times coef
meaning reduce exp
subtract 1 exp
reduce exp 1
Figure 3.6 Sample vocabulary and meaning lists
53
In cases where there is no match found, the student answer is marked up
to indicate broken phrases, words not in the vocabulary, and suspected
misspellings, and the input must be retyped.
This understanding system is incorporated into PATTIE as follows.
The problem expert develops a vocabulary containing words he anticipates
students will use in the dialog. The transition phrases associated with
branches in the graph (the refinements) are determined, patterns are
established to accept each phrase or its equivalent (synonymous equivalents
as a result of the vocabulary, other equivalents through multiple patterns
for the same meaning), and the number of that meaning list is attached to
the corresponding branch in the graph. Vocabulary and patterns can be
improved by iteratively examining protocols of student interaction with
the system and modifying both patterns and vocabulary to correct misunder-
standings.
PATTIE 's actual implementation of these features uses only a
single vocabulary for the entire problem domain, but a separate set of
meaning lists for each procedure required in the solution program, similar
to an approach taken in later versions of ELIZA (Weizenbaum (1967)). This
reduces the difficulty of insuring that two patterns associated with
separate meaning lists don't match the same input.
One of the biggest drawbacks of a synonym-class approach such as
this is that a word can have several different meanings in different
contexts. Since no word can be in two classes (except as part of a
phrase), classes which contain the same words must be coalesced. This
introduces a certain amount of ambiguity into the meaning attached to some
inputs. Fortunately, a node in the refinement graph provides a well-defined
context which helps reduce this ambiguity caused by merged classes. The most
54
likely student inputs at a node are exactly those which correspond to
transition phrases tagged to branches leaving that node, or nodes
slightly lower in the graph. Therefore, if an input matches one of these
branches, there is a high probability that the intended meanings are the
same .
As might be expected, a difficult part of adding a problem to
PATTIE's repertoire is developing the vocabulary and meaning lists needed
to understand student inputs. As mentioned above, this is a trial and
error process involving real students, although the effort involved may
be decreased depending on how closely related the new problem is to the old.
For example, about 27% of PATTIE's present vocabulary is concerned with
differentiation, another 23% with programming and PL/1, and another 11%
is concerned with polynomials.
3.3.3 The Student Model
One of the prime arguments which proponents of CAI make for
using computers in education is that CAI systems can adapt themselves to
provide instruction tailored to each individual student. Such
individualized instruction, however, is seldom realized in practice.
PATTIE makes no claim to provide individualized tutoring for every
student, but she does make an attempt to adapt her presentation based on
the student's past performance. This adaptation is controlled by a simple
student model which is based on the programming and problem solving concepts
discussed in Section 3.3.1.2 and listed in Figure 3.4.
The typical student model employed by CAI systems is a vector
representing the student's performance in some fixed number of areas, which
are commonly the system's educational objectives. The student is presented
55
information or asked questions in one particular area until his performance
exceeds some threshold value, at which point the next topic is introduced.
Only a few attempts have been made to develop and use a strong,
effective model of the student. One of these is that of Kimball, in his
tutor for methods of integration. His model is probabilistic, holding
that an attempted solution can be modeled as a sequence of problem states,
and that the student can be modeled by a set of transition probabilities
from one state to any other. The possible problem states are dependent on
the subject being tutored, and the proper identification of these states
is a critical step in the development of such a tutor.
PATTIE's educational goal is to expose the student to a dynamic
example of program development by successive refinement, and thus show
him how he can use such a process in developing any program. The degree
to which this is achieved is not easily measured by a threshold level.
However, PATTIE's entire refinement graph may be viewed as a sequence of
problem states in the sense used by Kimball. Each node is a state,
characterized by the concepts attached to it and to the branches which
leave it. Probabilities are established for each concept, and in turn the
concepts characterizing a node provide a probability of the student
correctly describing the refinements required at that node.
Say that a student correctly "employs" a concept whenever he
(1) correctly describes all refinements needed at a
node tagged with that concept,
or (2) correctly describes a single refinement associated
with a branch tagged with that concept.
The heart of the student model is then a pair of counters for each of the
16 concepts, one counting the number of times the student has correctly
56
employed that concept, the other counting the number of opportunities he
had to do so. This establishes a probability of correct employment
for each concept. As will be described in Section 3.4.3, the interaction
control routine can use this correct employment probability to suggest
which branches the student should follow at choice points, or to decide
to simply display the necessary refinements at a node instead of asking
for them.
There are two other components of the model beyond the correct
employment probabilities described above. PATTIE keeps track of the
average number of levels in the graph spanned by each student input, and
uses this information to control how far down in the graph to search in
trying to match a student input (see Section 3.4.3.1). She also keeps
track of the average number of refinement attempts made for all nodes,
and uses this information in the interaction control routine as well.
3.4 How PATTIE Behaves — The Interaction
As has been mentioned, PATTIE' s basic model of the problem
solving process is the method of stepwise refinement, similar to that of
Wirth (1971). In relation to the AND-OR graph, the successive refinement
process corresponds to tracing a path through the graph from a root to
some subset of the LEAF nodes which represent a solution program. The
exact path taken is determined by suggested refinements input by the
student. At any given time, the student is actively refining only one
task, the "current task.' 1 This current task is represented in the
refinement graph by a single node, which PATTIE considers the "current
node." PATTIE determines the correctness of student-suggested refinements
by matching them against branches in the graph at and below the current
node. Once the student has described all the actions needed to refine the
57
current task, a new current task is selected by simply traversing one of
the described branches from the current node to a new current node. The
order in which these branches are traversed is determined by the control
program using a depth-first traversal algorithm.
It is interesting to note that, besides being easy to implement
as a control algorithm, there is some evidence that such a depth-first
design procedure is used in the development of actual software packages.
Ells and Freeman (1973) reviewed (after the fact) the logical and chronological
relationships in the design decisions made during development of three
separate BASIC translators. Figure 3.7 shows design trees they developed
for each translator by ordering the design decisions so that no decision was
logically dependent on other decisions beneath it in the tree. These
structures are clearly AND-OR trees with only one branch from any OR node
fully developed. The order in which the decisions were made is indicated
by the numbers at each node of the tree. As Figure 3.7 shows, this numbering
is clearly depth-first in two of the trees, somewhat less so in the other.
It is also interesting that these, crees contain more OR nodes than ANDs, while
PATTIE's solution graph shows the opposite mixture. Possibly this is because
the design trees of Figure 3.7 reflect higher level decisions of large
projects, while PATTIE's graph indicates detailed decisions of a considerably
smaller program.
The remaining subsections of this chapter describe bow PATTIE
handles interaction with the student. The reader may find it helpful to
refer back to the sample dialog of Section 1.3 for examples of use of
various features.
58
/\
'X
-^ /\
*
A Ar xa, m a
A / V\X | VVX X s
Figure 3.7 Design trees (from Ells
and Freeman (1973))
59
3.4.1 What the Student Sees — The Screen Display
The only medium through which PATTIE may talk to the student is
the display screen of the PLATO IV terminal, a 64 character by 32 line
plasma panel. Within this area PATTIE must conduct a dialog and maintain
a copy of the student's developing solution. Figure 3.8 is a copy of the
screen as the student sees it. The upper 20 lines are the "program area"
and contain the developing solution program. The lower part of the screen is
the "scratchpad," the area where the dialog is conducted.
On the left-hand side of the program area are a series of "task
names" indicating the relationship of each task to others in the solution,
exactly as the relationships between sections of this thesis are described
by the section numbers. Task names are assigned when the refinement task
is initially described, based on the task name of the current node and the
current node type. Each refinement at an AND node receives a task name
composed of the task name of the AND and a suffix indicating the number of
the branch matched by the refinement. At OR nodes, on the other hand, the
task name of the refinement is simply that of the OR itself, since only one
branch leaving the node is ever traversed.
Traditionally, indentation has been used to indicate the relationship
between various tasks described in the successive refinement process (Gries
(1974)). On a screen with limited horizontal and vertical extent, use of
task names performs the same function, saves room on the screen, and provides
a handy way to refer to specific tasks and describe their interrelationships.
Task names also describe the order of refinement of the tasks (and hence of
traversal of the refinement graph), namely in increasing order of the task
name s .
Note also that the limited extent of the program area forces a
great degree of modularity in the structure of the solution program. The
60
DIFTEPM: PPOC'CT.Z) ;
DCL(T,Z,BFOR ) CHHRj
1.1 DCL (NDXZ.LSTHR ) FIXED;
NDXZ = INDEX(T,Z) ;
1.2 IF NDXZ =
THEN RETURN ( ' ' ) ;
2.1.2 BFOR =SUDSTRH
«
5
tu
K
H
M
rn
n
t^
CJ>
M
W
w
n
S
«
Figure A.l High-level control flow
105
»
H
3
Q
a.
O
CO
M
Q
S5
g
S
U
M
<
IX
o
CO
s
Figure A. 2 Data usage
106
When in use by students, PATTIE uses from 2 to 3 milliseconds
of CPU time per second per student. Response times vary from essentially
instantaneously to as much as 5 seconds, depending on total system load
and the amount of real time since the student's previous input.
The remainder of this Appendix describes the major data
structures used in PATTIE, which may be divided into two groups: those
which are part of the refinement graph, and various operational structures.
PLATO's memory word length is 60 bits; however, TUTOR allows an author
to define addressable entities, called "segments," less than 60 bits long,
which are then packed several segments to a word, and addressed as elements
in a one-dimensional array which may span many words. For example, defining
a segment TEST as 12 bits results in five TEST segments being packed in each
word. Individual segments are referenced as TEST(l), TEST(2), ...,
TEST(24),
The graph is stored in an array of 30-bit segments called NODTBL.
Each node is stored as a 30-bit node header followed by one 30-bit branch
header for each branch leaving the node. Individual nodes are accessed by
using their number as an index into a table of 12-bit pointers (NODACES)
which indicate the index of the corresponding node header in NODTBL
(Figure A. 3). Each node and branch header is further subdivided.
The node header contains four fields. To access these fields,
the header must be removed from NODTBL and inserted in another 30-bit
segment, NHDR30. The four fields are then (Figure A. 4)
N0DHDR5(1) — 5 bits indicating node type: 1 = AND,
2 m OR, 3 = LEAF, 4 = PROC. If this
field is negative, all regular (i.e., non-
ERROR) branches leaving the node are tagged
with PL/1 statements
107
node number
NODACES
Figure A. 3 Node accessing scheme
NHDR30
node
type
// reg
branches
# error
branches
ptr to node
display msg
5 10 15
N0DHDR5(1) NODHDR5(2) NODHDR5(3)
NDHDR15(2)
29
Figure A. 4 Node header format
BHDR
meaning list
number
pointer to
display copy
next node number
10
20
29
BRCHDR(l)
BRCHDR(2)
BRCHDR(3)
Figure. A. 5 Branch header format
108
NODHDR5(2) — 5 bits telling number of regular branches
leaving this node
NODHDR5(3) — 5 bits telling number of ERROR branches
leaving this node
NDHDR15(2) — 15-bit pointer to any display message attached
to this node (pointer into MSGTBL) . If
> — English message
— no message
< — PL/1 statement
The branch header contains three fields, and must be removed from
NODTBL and inserted in a 30-bit BHDR before they may be accessed. The
fields are (Figure A. 5):
BRCHDR(l) — meaning list number corresponding to branch
transition phrase. If
> — English phrase
— universal ERROR branch,
no phrase
< — PL/1 statement for display
BRCHDR(2) — pointer to display copy of transition phrase
(pointer into MSGTBL). If
> — English phrase
— expected ERROR branch,
no display
< — PL/1 statement
BRCHDR(3) — number of node branch leads to
109
The programming/problem solving concepts associated with nodes
and branches are kept in a separate CONCTBL of 12-bit segments. There is
one CONCTBL entry for each node and branch, in the same order as the node
and branch headers in NODTBL. Hence NODACES also gives the correct index
into CONCTBL.
Display messages are coded as a series of indices into a list of
display words (DISPLST) to save space. The coded messages are kept in a
table of 60-bit words called MSGTBL, and for decoding must be moved to a
field called MHDR, which is subdivided into 10-bit segments:
MSGHDR(l) — no longer used
MSGHDR(2) — number of words in display message; indicates
number of 10-bit segments to be decoded
MSGHDR(3) — number of pointers to this message; allows
sharing of one message copy by several
branches or nodes
MSGHDR(4) - MSGHDR(6) — indices into DISPLST for single
words in the message.
MSGHDR(7) - MSGHDR(12)
etc.
One of the primary operational structures is the active AND stack
(ANDSTK) which forms the basis for the depth-first search algorithm. ANDSTK
is an array of 30-bit segments, each of which contains status information
on a single partially completed AND node. The index of the top entry in
ANDSTK is indicated by ANDPTR. Status information is accessed by removing
a 30-bit segment to AHD30, which has the following subfields:
AHD12(1) — 12-bit segment containing the node number of
the AND node whose status is described
110
AHD2(7) — 2-bit segments indicating the traversal status
through
AHD2(15)
of each regular branch leaving that node. If
-1 — branch not yet described
— branch described but not
yet traversed
1 — branch traversed
The PROC stack (PRCSTK) is an array of 12-bit segments containing
the numbers of PROC nodes which must be traversed. The index of the top
entry on the stack is contained in PSTKPTR.
Additional stacks are needed for the depth-first lookahead search
for branches matching a student input. The two stacks are NSTK, consisting
of 12-bit segments which contain the numbers of nodes on the lookahead path
from the current node, and BSTK, consisting of 6-bit segments containing
the number of the last branch traversed from the corresponding NSTK node during
lookahead. The index of the top entry for these two stacks is contained in
PTR.
OR nodes which are determined to have been satisfied during
lookahead are placed on a list so that they are simply traversed when they
become the current node. This list is SATOR20, consisting of 20-bit
segments, all but the first of which are divided into two subfields
SAT0R20(1) — points to the first open element in
the list of satisfied OR nodes
SATORlO(odd) — 10-bit segment containing the number of
an OR node satisfied during lookahead
SATORlO(even) — 10-bit segment indicating which branch
to follow out of the OR node pointed at
by the immediately preceding SATORlO(odd)
Ill
The student model variables consist of three arrays associated
with the concepts, plus three other counters. The three arrays are:
CNUMUSE — 12-bit counters, one for each concept,
indicating the number of times that concept
has been correctly employed
CNUMCH — 12-bit counters, one for each concept,
indicating the number of opportunities the
student has had to correctly employ that
concept
INCR — 3-bit segments, one for each concept,
containing an increment to add to the
corresponding CNUMCH the next time that
concept forces display of a node. CNUMUSE
is always increased by one.
The three (60-bit) counters are:
TOTLEVS — total number of levels spanned by all
student inputs
NUMTRIS — total number of attempts at all nodes
NUMNODS — total number of nodes the student has been
asked to submit refinements at
The final set of data structures are those concerned with
maintaining the screen display. The primary structures are:
SCRNTBL — one 5-word element for each line in the program
area. Holds 50 characters to allow immediate
regeneration of screen display
STMTNUM — one 60-bit word for each line in the program
area. Holds task name for each line
112
TOP — line number of uppermost blank line in the
program area (just below current task)
BOTTOM — line number of lowest blank line in the
program area (just above the program area
stack)
CURSTMT — task name of current task
SCRNPRG — one 6-bit segment for each line in the program
area, contains the total number of characters
of PL/1 code currently contained in that line
SCRNCOD — one 2-bit flag for each line of the program
area. If
1 — PL/1 code currently on the
corresponding line
— no PL/1 code on that line
DCLIST — one 6-bit segment for each line in the program
area. Each segment contains two subfields:
DCLHDR3(1) — type of variable declared on that
line
— no variable
1 — CHAR variable
2 — FIXED variable
3 — FLOAT variable
DCLHDR3(2) — number of variables declared on
that line
The final data structure is a stack (PRGSTK) containing one 30-bit
entry for each active AND node, indicating the number of lines on the
screen which will be consumed by display of branches leaving that node.
113
Each 30-bit segment has several subfields:
ANDPRG6(1) — 6-bit segment containing total number
of lines for that node. DCL statements
are not included in the count
ANDPRG3(3) — 3-bit segment indicating number of lines
through juv u • i l.
on screen consumed by branch i-2, where
ANDPRG3(10)
i is the index. Z ANDPRG3(i) = ANDPRG6(1)
The top entry in PRGSTK is indicated by ANDPTR, the stack pointer for the
active AND stack.
114
VITA
The author, Ronald Lee Danielson, was born In Duluth, Minnesota,
on October 5, 1945. He was graduated from the University of Minnesota,
Duluth, with a Bachelor of Arts degree (summa cum laude) in mathematics
in June, 1967. He received a Master of Science degree in mathematics from
Northwestern University in August, 1968. Since September, 1971, he has
been a teaching and research assistant in the Department of Computer Science
of the University of Illinois. He is a member of the Association for
Computing Machinery.
BLIOGRAPHIC DATA
IEET
I. Report No.
UIUCDCS-R-75-753
3. Re< tpient a Accession N<
Title uiJ Sunt K If
PATTIE: AN AUTOMATED TUTOR
FOR TOP-DOWN PROGRAMMING
5. Repon Date
October, 1975
; Author(s)
Ronald Lee Danielson
8. Performing Organization Rept.
No.
I Pcrfoniung Organization Name and Address
Department of Computer Science
University of Illinois at Urbana-Champaign
Urbana, Illinois 61801
10. Project/Task/Work Unit No.
11. Contract /Grant No.
NSF EC-41511
nsoring Organization Name and Address
National Science Foundation
Washington, D.C.
13. Type of Report & Period
Covered
14.
mentary Notes
^h>tr.ict s
The Department of Computer Science at the University of Illinois has under-
taken a project to automate a substantial portion of its introductory computer
science courses on the PLATO computer-assisted instruction (CAI) system. Experience
with teaching programming skills has indicated the importance of teaching beginning
students how to develop a program, as well as specific programming languages. This
thesis describes a CAI tutor for top-down programming which provides a detailed
example of the stepwise refinement process. The tutor guides the student to a
solution of a specific problem by judging the correctness of suggested refinements
input by the student in natural language. An AND-OR graph is used as the model of
the top-down programming process. Techniques used in the tutor are sufficiently
general that the design may be used for tutors in other subject areas.
1 Kc\ Words and Document Analysis. 17a. Descriptors
computer -assisted instruction
CAI
structured programming
teaching programming
computer science education
problem solving
artificial intelligence applications
lentifiers Open-Ended Terms
V < OSAT1 Field/Group
R\\ liability State
19. Security Class (This
Report )
UNCLASSIFIED
20. Security Class (Thi
Page
UNCLASSIFIED
21. No. of Pages
22. Price
?l|W N HS-35 (10-70/
USCOMM-DC 40329-P7I
INSTRUCTIONS FOR COMPLETING FORM NTIS-35 (10-70) (Bibliographic Data Sheet based on COSATI
Guidelines to Format Standards for Sc tentific and Technical Reports Prepared by or for the Federal Government,
FH-180 600).
1. Report Number. Each report shall carry a unique alphanumeric designation. Select one of the following types: (a) alpha-
nunn [i. designation provided by the sponsoring agency, e.g., F AA-RD-68-09; or, if none has been assigned, (b) alphanu-
meric designation established by the performing organization e.g., FASEB-NS-87; or, if none has been established, (c i
alphanumeric designation derived from contract or grant number, e.g., PH-43-64-932-4.
2- i cave hlank.
3. Recipient's Accession Number. Reserved for use by each report recipient.
4- Title and Subtitle. Title should indicate clearly and briefly the subject coverage of the report, and be displayed promi-
ncntlv. x ' i subtitle, if used, in smaller r > pe or otherwise subordinate it to main title. When a report is prepared in more
i ban one volume, repeat the primary title, add volume number and include subtitle for the specific volume.
5- Repcr* Date. I a< h '<■ port shall carry a date indicating at least month and year. Indicate the basis on which it was select
. . itc of issue-, date of approval, date 01 preparation.
6- Performing Organization Code. Leave blank.
7. Author(s). Give narne(s) in conventional order (e.g., John R. Doe, or J.Robert Doe). List author's affiliation if it d if ft
I rforming organization.
8- Performing Organi zotion Report Number. Insert il performing organization wishes to assign this number.
9. Performing Orgoni zotion Name and Address. Give name, street, city, state, and zip code. List no more than two level' ol
mizationa'l hierarchy. Display the name of the organization exactly as it should appear in Government indexes such
a USGRDR-I.
10. Project Task Work Unit Number, ''sc the proiect, task and work unit numbers under which the report was prepared.
11. Contract Grant Number. Insert contract or grant number under which report was prepared.
12. Sponsoring Agency Name and Address. Include zip code.
1 '< Tyc e of Report ond Period Covered. Indicate interim, final, etc., and, if applicable, dates covered.
14. Sponsoring Agency Code. Leave blank.
15. Supplementary Notes. Rnter information not included elsewhere but useful, such as: Prepared in cooperation with...
I ran ilation of . . . Presented at conference of . . . To be published in . . . Supersedes ... • Supplements
16- Abstroct. inc lude a brief (200 words or less) factual summary of the most significant information contained in the repofi
1: •!.• report contains a significant bibliography or literature survey, mention it here.
17. Key Words and Document Analysis, (a). Descriptors. Select from the Thesaurus of Lngineering and Scientific Terms tin
luthorizc d terms that identify the major concept of the research and are sufficiently spec if ic and precise to be use
lex enir •■ for i it aloging.
(b). Identifiers and Open-Ended Terms. Use identifiers for project names, code names, equipment designators, etc. I
,'pen-< tided terms written in descriptor form for those subjects for which no descriptor exists.
fc). COSATI Field Group. Field and Group assignments are to be taken from the 1965 COSATI Subject Category I.i v :
it y of documents are mu It id isc iplinary in nature, the primary Field/Group ass ignment (s ) will be the sp<
. I human endeavor, or type of physical object. The applicat ion(s ) will be cross-referenced with secondar
ignments that will follow tin primary posting(s).
18. Distribution Stotement. Denote te Icasabi 1 ity to the public or limitation for reasons other than security for example "
'" iti "■ ' ite any availability to the public, with address and price.
19 & 20. Security Classification. Do not submit classified reports to the National Technical Information Service.
21. Number of Page'.. Insert the total number of pa^es, including this one and unnumbered pages, but excluding distribui
1*1.. i/ any.
22. Price. Insert the price set by the National Technical Information Service or the Government Printing Office, if known.
I
FGBM NTIS-33 (10-70) USCOMM-uC 40329-'
OCT 1 71175
^
■&