i Gum LIBRARY OF THE UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN 510.64 Lt6r no. 715-721 cop. Z Digitized by the Internet Archive in 2013 http://archive.org/details/multiprocessorne717iway UIUCDCS-R-75-717 no. 7J7 December, 1975 A D MULTIPROCESSOR NETWORK SYSTEM DESIGN BY PROCESS CONCEPT by Akihiro Iwaya J>. f UIUCDCS-R-75-717 MULTIPROCESSOR NETWORK SYSTEM DESIGN BY PROCESS CONCEPT by Akihiro Iwaya December, 1975 Department of Computer Science University of Illinois at Urbana-Champaign Urbana, Illinois This work was supported by the Department of Computer Science and was submitted in partial fulfillment for the Master of Science degree in Computer Science, 1975 . ' Ill ACKNOWLEDGMENT The author would like to express his gratitude to his advisor, Professor Jane Liu, for her guidance during the preparation of this thesis and also her careful reading and valuable suggestions for the improvement of the original manuscript. In addition the author owes a debt of gratitude to his wife, Masayo, whose understanding made this thesis possible in a limited period of time. This paper is dedicated to his two children, Miki and Masaaki. IV TABLE OF CONTENTS Page 1. INTRODUCTION ! 2 . AIRLINE RESERVATION SYSTEM 3 2.1 System Requirements 1^ 2 . 2 Terminal Operation If. 2 . 3 Central Computer Operations 8 2.3.2 File Structure n 2 . 4 Summary jq 3. FILE SYSTEM DESIGN 15 3.1 File System Organization jc 3.2 Logical File System (LFS) j£ 3 . 3 Operation of File System 20 3. k External Search Routine 2k 3 . 5 Command Handling Routine ^1 3.6 Summary 32 k. CENTRAL COMPUTER DESIGN 38 h. 1 Summary of Operation 38 4.2 System Design ^c 4.2.1 System Decomposition Rules 1+5 4.2.2 Decomposition of Reservation System into Processes 1^ V Page 4.2.3 Operation Flow by Process Cooperation kQ 4.3 Reservation System Processes i±g 4.3.1 Line Input Process \±n 4.3.2 Interactive Job Handler Process 51 4.3.3 Line Output Process 5^ 4.3.4 Scheduler Process 5^ 4.3.5 Command Handler Process 57 4.3.6 Edit Process en ?9 4.3.7 File Management Process g-i 4.3.8. External Search Process g^ 4.3.9 Probe-delete Process 57 4.3.10 File I/O Process ?0 4.4 Hardware Architecture and Operating System 70 4.4. 1 The Rules of Organizing Cluster Network 70 4.4.2 Cluster Network Organization y 4. 4.3 Local Operating System 75 4.4.4 Node Processor Organization 77 4.4. 5 Implementation of Kernel Routines 78 J i.5 Summary o n 5. CONCLUSION o ol LIST OF REFERENCES D „ ' o3 1. INTRODUCTION During recent years, a great deal of efforts have been made in the development of computer network systems leading to the design and implementation of several centralized computer network systems [1,2,3], Typically, such a network contains several miniprocessors (or micro- processors) connected together to form a computing system whose performance is equivalent to a large scale computer system while offering such advantages as increase of reliability, expandability and cost reduction. This report describes the design of a dedicated multiprocessor cluster network [h] which functions as an airline reservation system. The design approach used is as follows: we decompose the system into cooperative processes. A dedicated processor is assigned to each process and processors are connected via suitable inter-process communication links to form a cluster network. The airline reservation system is selected to demonstrate these design approaches since the functions performed in the reservation system are fixed. Hence the problem of process decomposition is relatively straightforward. Moreover, it being an interactive system, we need to be concerned with many interesting aspects in operating system design, such as protection mechanisms, file structures, etc. In Chapter 2, a hypothetical airline reservation system is described. The terminal operations in the system and the central computer operations in an interactive job mode are also discussed. Chapter 3 describes the file system, which is the most crucial part of such an interactive system. The design of the central computer based on process cooperation concept is presented in Chapter k. The structure of the cluster network and the required characteristics of the system are also described. 2. AIRLINE RESERVATION SYSTEM In an airline reservation system, customers' requests are sent through communication lines from the terminals at sales offices to the central computer in the central reservation office. The sales offices are usually scattered over a wide area geographically. Hence, messages from these sales offices are often gathered in a message switching center before being sent to the central office. A typical airline reservation ' system is organized as a tree shaped network with the root representing the central office. Such systems are now used by most of the major airline companies. Many functions required in an airline reservation system are executed interactively between the computer and the operators at sales offices in a foreground mode. For example, an inquiry asking available seats on a given flight must be answered in 2 to 3 seconds. The airline reservation system must offer the capability of meeting this response time requirement. We believe that such a design goal can be achieved cost-effectively in a multiprocessor cluster network. To determine the hardware organization of a cluster network and the requirements of an operating system, a simplified model of an airline reservation system is considered. The system requirement is presented in section 2.1. The terminal operations and the central computer operations are discussed in sections 2.2 and 2.3, respectively. 1+ 2.1 System Requirements Throughout this paper, we consider a system designed to suit the need of a medium-sized airline company. The relevant characteristics of this company are as follows: 1. The central office has a central computer with a large file containing information on all schedule flights. 2. There are kO offices with teletype-like terminals, 3. There are 1000 flights per day; each flight has a capacity of 200. k. Reservations can be made 1 year in advance. 2.2 Terminal Operation References to the data base are made whenever a customer arrives at or calls the sales office. The operator keys in and sends a terminal command which refers the data base, and in reply, he receives a response from the central computer. The terminal command types which can be sent from a sales office are as follows : 1. Start operation (CMl) : conversation starts (job control command); Terminal input: Command; Response: Acknowledgment; 2. Available seats inquiry (CM2): ask for information about the available seat; Terminal: Command, Flight number, Flight date; Response: Acknowledgment with the number of seats available; 3. New reservation (CM3): reserve the seats of particular flight; Terminal input: Command, Flight number, Flight date, Number of seats, Passenger's name, Address, Phone number; Response: Acknowledgment with the number of seats reserved; h. Cancellation: cancel the reservation; Terminal input: Command, Flight number, Flight date, Number of seats, Passenger's name; Response: Acknowledgment with the number of seats remaining; 5. Confirmation (CM5): confirm the reservation and issue the ticket; Terminal input: Command, Flight number, Flight date,. Passenger's name; Response: Acknowledgment with the number of seats reserved; 6. Passenger's information inquiry (CM6): request the passenger's information; Terminal input : Command, Subcommand'' (Address, Phone number), Flight number, Flight date, Passenger's name; Response: (Address, Phone number), Reservation state; * This shows what kind of information items the operator wants. 7. Flight information inquiry (CM7): Request the information by origin and destination; Terminal input : Command, Subcommand (Time, Fare, Operation date, Meal, Equipment), Origin, Destination; Response: Flight number (Departure time, Arrival time, Fare, Operation date, Equipment); 8. Stop operation (CM8): Conversation ends (Job control command); Terminal input: Command; Response: Acknowledgment; Normally one transaction consists of few terminal commands including CM1 (start) command and the CM8 (stop) command. A typical terminal command is structured in the following way: Month - Flight number Command (CMl) i r Day Year avail, 302, oct 27 7h All terminal commands are typed in lower case letters. Response from the central computer are typed in upper case letters , as in conventional conversation systems. *This shows what kind of information items the operator wants. 7 Examples of terminal command format of different command types are shown as follows : CM1: Input start; Response START OK CM2: Input avail, 302, oct 27 7k; Response AVAIL NUM 020 CM3: Input reserve, 302, oct 27 7k, 00^, akihiro iwaya, 2023c orchard-st. urbana ill. 217-38^-1919; Response RESERVED, SEAT 00^ Points to INFOLIST Flight* Month* Day* Count of Year* free seats Pointer 10l+ 007 NOV JAN 23 25 7^ 75 030 055 r FLTLIST (Directory file.) FLTLIST name or address of PSRLIST Semaphore Pointer xxxx •1 Points to waiting process LFT (in main memory) Name* Address Reserved'' - seats Number Street l Phone City State number Res eve state ) A.IWAYA OOU 2023-C ORCHARD ST. URBANA ILL. 217_3b4- 1919 CONFIRMED B.TURNER 002 1107 W.GREEN URBANA ILL. 3030 RESVED PSRLIST (Data file) * It ems used as a key Figure 3.1 File System Data Structure (l) (Passenger Information File) 19 O o of CD o o o !w • • £ VD VD vo OJ H £1 OJ H ^9- *> -ee- ft t>- r- i>- •H 3 P-3 PL £ £ pq 1 pq i pq >H ! >H ! ft 0) P P : * «• -^ H H te ft o3 ^ <^ ! >\ j O Tf Q OS | 1 I > CD p^ ft ft •H O i i i •5 ft CD • O ft cd H ft ft o o o «lt CD CD w w a CD -p 1 AFD(*) CTL 2 FILENAME CHAR (10), 2 CHAIN PTR; 1 LFT(*) CTL 2 LEA FIXED BIN (32), 2 SEMAPHORE FIXED BIN (8), 2 CHAIN PTR; 1 FLTLIST BASED } 2 FLIGHT^") CTL, 3 NUM CHAR(3), 3 DATE, k MONTH CHAR(3), k DAY CHAR(3), k YEAR CHAR(2), 3 FRSEAT CHAR(3), 3 CHAIN PTR; 1 PSRLIST BASED 2 PSR (200), } 3 NAME CHAR(20), 3 RVSEAT CHAR(3), 3 ADDRESS, k NUM CHAR(6), k STREET CHAR (15), k CITY CHAR(15), k STATE CHAR(U), 3 RVSTATE CHAR(U); 1 INFO LIST BASED, 2 FLIGHT (1000), 3 ORIGIN; k PLACE CHAR(3), k TIME CHAR(5), 3 DESTIN, k PLACE CHAR(3), U TIME CHAR(5), 3 NUM CHAR(3), 3 OPDATE CHAR(5), 3 EQUIP CHAR(5), 3 FARE CHAR(6); 3 . 3 Operation of File System 20 /* ACTIVE FILE DIRECTORY */ /*. POINTS OPENED FILE */ /* LOCKED FILE TABLE */ /* LOCKED FILE ADDRESS */ /* SEMAPHORE FOR FILE LOCK */ /* FLIGHT DIRECTORY FILE */ /* FLIGHT NUMBER /* FLIGHT DATE /* COUNT OF FREE SEATS /* POINTER TO PSRLIST /* PASSENGER DATA FILE /* PASSENGER /* RESERVED SEAT */ V */ /* reserved/confirmed */ /* flight information file */ /* FLIGHT NUMBER /* OPERATION DATE /* FX 'B-727* /* IN $ * The file operations for the file structure presented in Section 3.2 are: read, multi-read, rewrite, and delte. Each operation is accompanied All records with the same key are read. 21 by a record search operation with keys. The statement which calls the file operation may look like the following: READ () INTO () KEY () Lock; The operations performed to obtain a desired record in a file system are described in Figs. 3.3 and 3.k. 1. The file name is searched for in the AFD to obtain the address of the file. 2. If the name is not found, the SFS routine and the BES routine are further called. 3. If the address of the file is given to the file system, step 2 and 3 are skipped. h. If this file is a type of file which needs a lock operation, the LET checks whether this file is already locked. If this file is locked, the operation is kept waiting until It is unlocked. If this file is not locked, the lock - operation is performed and proceeds to step 6. 5. However, if this file is not a type of file which needs a lock operation, step k is skipped. 6. External search operation is performed. 7. If specified operation is a read or multi-read operation, the record searched for is moved to working area. 8. If the specified operation is a new-write or rewrite operation, the record in working area is moved to file output buffer and then written into the area in a secondary memory. Search Active File Directory NO Call Symbolic File system, Basic • File svstem routines YES ><- Wait until unlocked Lock file 22 Figure 3.3 The Flowchart of File Management Routine (l) 23 Call External .Search routine Move data to working area YES READ OUT from working area UNLOCK \._N0 Figure 3.4 The Flowchart of File Management Routine (2) 2k 9. If the specified operation is a delete operation, step 7 and 8 are skipped. 10. If the unlock is specified then the unlock operation is performed. 3.1+ External Search Routine The external search routine designed for this system is a modified version of the open scatter hash addressing search algorithm. The environment in which this routine is executed is schematically shown in Fig. 3.5. Let 'M' be the total number of buckets, 'b' be the number of entries in a bucket and assume each entry consists of 'n' bytes. In order to access a particular record in a table (logical file), a hash value 'i' is calculated from the given key, then the logical file address (LFA) of the bucket is : LFA = i X b x n The physical block address (FBA) which includes this bucket is: EBA = FHA + [LFA/kJ Where FHA is the file head block address which can be obtained from the AFD (active file directory) and k is the number of bytes in a physical block. The bucket address (BA) within a physical block is: BA = LFA mod k The PBA (physical block address) is used as a secondary memory location to move the physical block in a secondary memory to the buffer area and the BA (bucket address) is used to move a bucket from the buffer area to the bucket number FilA I t LFA f < backet 25 i-1 i i+1 i+2 i+3 i+4 i+5 i+6 V M-2 M-l • t^BA Physical block J X File (Secondary memory) b-1 -> bucket File input buffer (in main memory) b-1 Search area (in main memory) Figure 3.5 Data Structure in Hash External Search Method Environment 26 search area. After the bucket is moved to the search area, linear search technique is used to find the required record. The flowchart of the external search routine is shown in Fig. 3.6. When the external search routine is called, argument lists include keys and operation code (access or insert or delete) as parameters. The steps in which the external search routine is performed are as follows: 1. Key-items 'X, Y, Z* are processed by some arithmetic-logical function to produce a single key 'K'. 2. The hash address value !i f is calculated using a hash function h(K). 3. The bucket i is read from the secondary memory into the search area. h. The linear probing begins within this search area from the bottom entry of the bucket. 5. If the entry is empty or the key in the entry matches, the probing ends. 6. Otherwise, the probing repeats within a loop until one of the two conditions described is satisfied. While in this probing loop if all entries within a bucket are searched unsuccessfully, another bucket is read-in from the secondary memory. Probing proceeds in this way in a descending address order. 7. If the key-match is the condition to leave the probing loop, next two steps are performed. (tatry ~) 27 —j— J K i h(K) -i: area fltesci-tn Bu£F£0 * into searching Entry) Figure 3.6 External Search Routine 28 8. If the specified operation is a delte operation, the number of occupied entries 'N' is decreased by one and the delete routine is invoked. 9. The routine returns with the bucket number 'i', the address of entry within the bucket 'j', andthe result code 'entry is found'. 10. However, if 'an empty entry is found' is the condition to leave the probing loop, then one of the next two steps is followed. 11. If the overflow condition is encountered (i.e., N = b X M - l), the result code 'overflow' is returned and the routine returns. 12. Otherwise, if the specified operation is an. insert-operation, 'N' is increased by one and the entry is marked 'occupied', the key is inserted into the entry and the inserted address is returned. 13. The result code 'entry is not found 1 is returned and the routine returns. The delete routine is invoked by the external search routine when delete operation is required. This routine is considered to be an extension of external search routine. Fig. 3.7 shows the flowchart of the delete routine. The steps of the delete routine are as follows: 1. The entry located by the external search routine i s marked ' empty ' . 2. The routine proceeds to examine entries in a descending order. First, the adjacent descending address entry CEZ3 <- o.upiy zxz l«-i,ia^i» INI'I <— TttUiiJ YE< h(KEYti,j3) Two U READ FILE (PSRLIST) INTO (PSRDATA) KEY (INDATA. NAME) NO / Entry Is found ?„ YES Address OUTDATA * — . PSRDATA. ADDRESS, PSRDATA. RVSTATE OUTDATA ««— 'PASSENGER NOT IN FILE* OUTDATA «— PSRDATA. PHONE, PSRDATA. RVSTAT: Figure 3.11. Flow of Command Handling Routine (k) (CM5- Confirmation, CM6-Passenger's Information) 37 d E H CO n w o p ^< «aj <: §1: 3g +0 „ external interrupt UJJ.CU s~ ^^ -' Edit operation 1 tf Figure k.l Sequence of Events for the Reservation System NO Entry Allocate input buffer 3 *ead-in terminal command Error check YES Return iH Figure k.2 Line Input Operation Entry k2 Check system status Check System status Figure k.3 Scheduler Operation ^3 Entry- Find out the location of required MAF Read-in to Input area from MAF Merge two commands Read-out to MAF from Input area Return Figure h.k Edit Operation hk Select line for reply Reading out reply to the line YES Send Error check bit Figure k.-5 Line Output Operation 45 4.2 System Design To design a cluster network of functional dedicated processors, the system is decomposed into cooperative processes. We discuss here the different processes and the format of messages for process communication. Then the structure of the network with dedicated nodes and required communication path between them are determined. 4.2.1 System Decomposition Rules To decompose the system into processes, the following rules are observed. The operations performed in a process must consist of self- contained functions. Such block of operations must be logically deep enough so as to decrease an overhead loss. The decomposition must be done in such a way that the length of inter-process message be as short as possible. Moreover, the system is decomposed into a family tree of processes and only the parent process has the right to create and destroy a child process. Within the family tree of processes a process can only communicate with the parent process, the child processes, and the brother processes. 4.2.2 Decomposition of Reservation System into Processes A family tree of processes for the reservation system is generated where each node corresponds to a process and each link corresponds to a create and destruct relation between processes. A process is a computation in which operations are carried out strictly one at a time [9]. Hence, a process usually runs with other processes concurrently. k6 Fig. k.6 shows the family tree of processes for the reservation system. The root of the tree consists of the initializer process which is loaded by the Initial Program Loader (IPL). The other processes are: 1. Line input process: Read in the terminal command into the input buffer area. 2. Interactive Job handler process: Control the flow of interactive processing job stream. 3. Line output process: Send out the answer from the output buffer area. h. Scheduler process: Monitor the overall system status. 5. Command handling process: Execute the command job. 6. Edit process: Edit the command. 7. File management process: Handle the file management system. 8. External search process: Search for the logical record in a file. 9. Probe delete process: Auxiliary process for record search. 10. File input process: Read in from secondary memory to file buffer area. 11. File output process: Read out from file buffer area to secondary memory. The initializer process creates four types of child, processes: the line input process, the line output process, the interactive job handler process and the scheduler process. (Former three types of processes are created corresponding to the terminal of this reservation system. ) u n o 10 r-\ 0) M c> •a o Xi H o P< w hi \ « i l/i -p 'i o ;-J o i; t < o .■-i ') , j ■ < CO H * C.*» r> V i-I. Oi Ilil 1 r— ' !' :., O t! rv vi t It, O P^-.H C'T 10 H » ft> ro o :i O 1 r-; P 4 .-. r J ••- 1 vi H • fa ,H P( f{ ^ n O lA -P C> 01 (/> c ^-' O 1 rH tea P H 1 P. Ph-H (0 -P to J"J (.) fa pi p. O 1 f 1 -P o f-> •r-< n ( ■'. -i CO 10 -p o fa 03 :1 o i r . fi. o M H | K P i |x, •h Ph.. I P. p f o H ri P. -P CD O 0) ,3 -P a •H CO W 0) o o u fa

rocess (2) 6k Then if file-lock is specified, it performs a V-operation on the file semaphore. Finally it sends a "completion" message to the command handler process and then again enters the message waiting state. k, 3. 8 EKternal Search Process The flow of the external search process is shown in Figs. k.±6 and U.17. This process performs the external search operation discussed in section 3.^ (see Fig. 3.6). When it is created, it creates the probe-delete process, the file input process, and the file output process. These processes are all used in the external search operation. This process waits for a message from the file management process. This message includes argument lists and control codes. First it computes the hash function which is in turn used to locate a bucket in the file. After that, it sends a message to the probe-delete process and waits for an answer message from the same process. The probe-delte process performs the actual search operation. The message from the probe-delete process is that an empty entry has been found, when the key entry is not in the file. If a write operation is required, the key insert routine is entered and then sends a message to the file output process to read-out new entry into the secondary memory. However, if the process is informed of a key match, and if a delete operation is required, it again sends a message to the probe-delete Create & Start Pro be -delete process Create & Start) File input process Create &~ Start File output process 65 1 Receive message from File management process Compute hash function A. Send message to Probe-delete process ±. deceive message from Pro be - delete process Figure k.l6 Flow of External Search Process (l) YES 66 Message ^OVERFLOW Insert key routine I Allocate j I File } output I buffer y Move data to File output buffer „__i Deallocate working area !Send ! message [to File i output process .1 Receive message from File output process E Message j —'EMPTY I NOT FOUND* ! Allocate File output buffer DELETE IffiftB li , Allocate | working j area Wove data to File output I buffer I J , jk_ Send message to Probe- delete : process j Move | data to | working .'area ^Deallocate working area I Receive message from Probe- delete i process i Send i message to File I output j process I Receive I message I from file ! output I process Deallocate search area Send message to File manage- iment [process © FI{jure ' 1 !.17 Flow of External Search Process 67 process to perform the deletion. If a read operation is required this process moves the data item to the working area, and a rewrite operation is required and process requires a read-out operation by sending a message to file output process. Finally, this process sends a "completion" message to the file management process. Although the process flow is designed based on the external search routine and the delete routine presented in section 3.k, the minor change was made to remove redundant operations. As a result, the probe-delete process is introduced. 4.3.9 Probe-delete Process The flow of the probe-delete process is shown in Figs. I4.I8 and Fig. 4.19. This process is introduced by merging probe and searching delete operations. This process performs actual search and delete operations depending on the content of message received. After receiving a "run" message from the external search process, if the probe operation is required, this process performs the bucket read-in where the key entry is supposed to be included. Then if a key entry or an empty entry is found, it immediately sends a message to the external search process. This message includes the resulting information. However, if neither entry is found in the bucket, this process enters into a probing loop until it finds a key entry or an empty entry. In this case, if the delete operation is specified, the loop includes a delete routine which moves items of entries in turn until this process finds an empty entry (see Fig. 3.7). 68 Receive message < from external search process I DELETE YES \ Yno (probe) Send message •READ IN' to File input process ..j .#. ! Receive message /COMPLETION' | from File ! input process „ jL_ - Allocate search area _Jc Move data to search area I Deallocate File input buffer £ Initial search © Figure k.lQ Flow of Probe-delte Process (l) 69 y. _ Probing routine Send message to [External search process © Figure U.19 Flow of Probe-delete Process (2) 70 if. 3. 10 File I/O Process The file input process performs a file read-in operation while the file output process performs a file read-out operation. The operation of both processes is based on a block transfer operation. Hence, the operations of blocking and deblocking must be included in these processes. k.k Hardware Architecture and Operating System The approach employed in the design of the reservation system is to assign a dedicated processor or subnetwork to each of these processes described in section U.3. U.U.I The Rules of Organizing Cluster Network The rules of organizing cluster network are as follows. Each process which was decomposed from the system runs on the dedicated node processor of cluster network configuration. One method of realizing inter-process communication is via messages which are transmitted on - high speed communication lines between processors. The other method for realizing inter-process communication is to use a common data-base. In this case, the data-base is considered to consist of global parameters which are accessed by more than one process. The nodes in the cluster network may be further organized as multi-processor subnetworks. Each node has a local operating system and hence, functions an autonomous system. Each node can run asynchronously with respect to other nodes. k.k.2 Cluster Network Organization Fig. U.20 shows the common data base in shared main memory used for the reservation system. In this figure the processors related to each n o -p (0 i « a co g -p o •H 3 o |^o ft h 1 -P o rfS c « w | CD 3 CD o UL1 jj 7T? CD W ca O O ftj I QJ cO Jh> H CO ■P-PP G O GO o cd o M c o o -&- 71 TT o o co 1 -p CO *x o o a> o £* N, ! h O 1 co i -p CO 4 0) G d) i H P< U •H G O fc -H u 1 Ot TT o +> co a> a CO a) iH •p u •H G o Ix, o 1 o rH (6 72 memory area are also shown. These processors allocate the corresponding memory area and work on them. The operations on these memory areas are as follows : 1. The line input data (terminal command) is read into the line input buffer by the line input processor. 2. The system job queue and the system resource table are updated by the interactive job processor and referenced by the scheduler processor. 3. The line input data in the line input buffer is moved to the input area by the interactive job handler processor. k. If editing is 'required, the file data in the file input buffer is moved to the input area by the file management processor. 5. If the input data needs to be stored, the input data in the input area is moved to the file output buffer by the file management processor. 6. The input data is moved to the working area from the input area by command handler processor. 7. If an external search is required, the file data in the file input buffer is moved to the search area by the probe-delete processor. 8. The results of the search are moved to the working area from the search area by the external search processor. 9. If a rewrite operation (entry update) is required, the data in the working area is moved to the file output buffer by the external search processor. 73 10. The results of command handling operation are created in the output area or moved to the output area from the working area by the command handler process. 11. The output data in the output area is moved to the line output buffer by the interactive job handler processor. 12. The output data in the line output buffer is sent out to the communication line by the line output processor. Fig. 4.21 shows the resultant cluster network configuration with shared main memory for this system. The shared main memory is connected with all processors by memory lines, whereas processors are connected to each other by inter-processor communi cation lines. Each processor has its own local memory where the local operating system and process routine reside. The shared main memory has memory mapping facilities and an allocation mechanism. Four processors: the line input processor, the line output processor, the file input processor, and the file output processor are considered to be I/O processors and connected to I/O devices. Note that 'port' is special hardware to connect each processor node to connection lines. All connections are implemented using dedicated lines. Fig. 4.22 shows a cluster network configuration without shared main memory for the reservation system. In this case, the volume of message between processes is sometimes very large, because the message includes data information as well as control information. However, it offers completely homogeneous physical network configuration. 7^ communication line Shared main memory manage - ment pro- cessor External search pro- cessor Heal- time job handler yrro- censor Command handler pro- cessor O Probe delete pro- | cessor ! port memory line inter-processor communication line Figure 4.21 Processor Network Configuration with Shared Main Memory 75 communication line I ?che- uler pro- cessor Keal-time 70b handler processor Line output pro- cessor Command" handler processor O Port _ Inter-processor communication line manage- ment processor Ddit pro- cessor File output processor External* Search orocessor * includes pro be -delete processor File input processor Secon- dary memory Figure 4.22 Processor Configuration Without Shared Main Memory 76 U.1+.3 Local Operating System Node processors run asynchronously with each other. In order to make the node processor autonomous, each node processor has its own local operating system. The local operating system includes a processor management mechanism and part of a memory management mechanism. These mechanisms are sometimes called a kernel. A kernel consists of two parts: a fixed part (program) and a variable part (data base) stored in Read Only Memory (ROM) and Random Access Memory (RAM), respectively. Also stored in ROM is, of course, the routine for the specified process. The program routines stored in the ROM are as follows: 1. Process dispatcher (traffic controller): This mechanism assigns a processor to a 'READY' process in a process descriptor queue in a multi-processor environment within the node processor. 2. Supervisor call (SVC) interrupt handler: This mechanism handles an SVC interrupt. It includes a copy of registers and a control transfer. 3. External interrupt handler: This handles an external interrupt. It may change the status of a process from "BLOCKED to "RUNNING. : ' k. SVC routines: P-operation, V-operation, create process, destroy process, start process, stop process, send message, receive message, allocate memory, deallocate memory, etc. 77 5. The routine for assigned process: A process routine discussed in section 4.3 is implemented. This process routine uses SVC routines. The data bases implemented in the RAM are as follows: 1. Process descriptor: A processor is represented on the data base called Process Control Block (PCB) with a unique process name. These PCBs are connected together to form a double linked list to ease insertions and deletions. 2. Message buffer: A message sent to a process is inserted in a queue with a header in the PCB of the receiving process. The header in the PCE includes a pointer to the first block of the queue and a message receiver semaphore. 4.4J-I Node Processor Organization The hardware organization of node processors assumed is that of components connected on a bus. These components are as follows: 1. Micro-processor: The micro-processor runs on micro-codes in the ROM. The number of micro-processors attached can be increased depending on the traffic volume of the node. 2 . ROM : Kernel routines and the routine for the assigned process reside here. 78 3. RAM: The data base on which the kernel rcaitines are implemented, is contained in the RAM. k. PORT: This is a special hardware unit to connect a node processor with other node processors or I/O devices by high-speed communication lines. It consists of multiplexer gates and the controller which is also a micro-processor. k.k.5 Implementation of Kernel Routines The implementations of the kernel routines are almost the same as those of conventional multi -programming systems. However, additional features are still necessary with the help of hardware. As an example, the flow of the kernel routine 'send message' is shown in Fig. k.23. In this figure, it is assumed that some process in sending processor calls the SVC routine 'send message'. On entry to the SVC routine the sending processor sends a receiving process-name to the receiving processor with an external interrupt signal. The receiver side is entered by the external interrupt signal, and the receiving processor tries to find the process name in the process descriptor. If there is no such process the receiver side sends a 'NO' code with external interrupt to sending processor and then returns to the calling process. However, if the process name is. found, it allocates a message buffer and sends an 'OK' code with the external interrupt to sending processor. 79 (Entry SVC interrupt Entry external interrupt V . \L. Find the n?.me of receiving process Send a process name with external interrupt Walt external interrupt f--. NO ' ' Error routine Allocate message buffer ±L Send •NO' with external interrupt Send message text Send 'OK* with external interrupt Receive message text- He turn from SVC interrupt Sender processor side JL Lock message buffer queue Insert message into message buffei queue Unlock message buffer queue V-oporatJon on message recej.yci° semaphore j±. Kcturn from external interrupt Receiver processor cidc Figure 4.23 Flow of Kernel Routine 'Send Message 8o The sender side which is waiting the external interrupt is awakened, and if the returned code shows that the process corresponding to the name has been found, it begins to send the message text. However, if the name has not been found the error routine is entered. Then the sender side routine returns to the calling process. The receiver side, after receiving the message text, locks the access to the messages buffer queue and inserts the message into the message buffer queue of the receiver process. Then this routine unlocks (these lock and unlock mechanisms are implemented by P and V-operation) the access to the message buffer queue. Then it performs a V-operation on the message receiver semaphore of the receiver process. Hence, if this process is waiting, its state is changed from "BLOCKED" to "READY. " After that it returns to the calling process. U.5 Summary In this chapter, the summary of operation for the reservation- system was discussed. Then the total system design based on the process cooperation concept was made and the precise flow of each process used in this system was presented. Finally, the cluster mult i -process or organization and the distributed operating system which are the final design results for this system were presented. 81 5. CONCLUSION The design approach utilizing a multi-processor cluster network system based on the process cooperation concept was presented with an airline reservation system as an example. Since the actual implementation of the design was not done, the complete validity of the design cannot be guaranteed. However, the feasibility of using a multi-processor cluster network system for the application was success- fully established in this paper. The design approach itself is very general and is applicable to every other system as well. For example, a time-sharing system (TSS) can be easily realized by changing the command handler process (see Fig. k.12) according to the specification of TSS. A batch system can also be realized by adding the batch job handler process which is analogous to a batch monitor in a conventional batch system. In both cases additional processes such as a user process and a compiler process are required and corresponding processor-nodes must be added to the multi-processor cluster network. Then the user process would always be run in a general-purpose processor-node. Future research in this area is still required for the efficient and elegant system. This future research must include the following: 1. Establishment of a complete and consistent method to decompose the system into processes: 82 In this paper the process decomposition was partly done in an ad hoc manner even though the rules for the decomposition were specified. Further development of structured programming seems to be in the direction of a solution to this problem. This is a problem inherent to every kind of system design. 2. Establishment of inter- connection network design method : In this design, processors, memory, and I/O devices were connected in one-to-one fashion as needed. However, the amount of inter-connection hardware sometimes becomes a bottleneck of hardware cost. The topology of minimum hardware network including a use of buses must be sought. Research work in this area seems to be advancing [k] . 3. Development of micro-processor architecture: Present micro-processors can be used as node-processors. However, it is desirable to develop further a micro- processor which has an ability to send and receive a vast amount of inter-process messages, i.e., a micro-processor with multiple ports. 83 LIST OF REFERENCES [1] Davis, R. L., S. Zucker and C. M. Campbell, "A Building Block Approach to Multiprocessing, " Proceedings, AFIPS, 1972, SJCC, Vol. kO, pp. 685-703. [2] Baskin, H. B., B. P. Borgerson and R. Roberts, "PRIME-A Modular Architecture for Terminal-Oriented Systems, " Proceedings, AFIPS, 1972, SJCC, Vol. kO, pp. ^31-^37. [3] Gillies, D. B., et al., 'MESH Project First Annual Report," Quarterly Technical Progress Report, UIUCDCS-OPR-73-4, Department of Computer Science, University of Illinois, Nov., 1973, pp. 102-127. [h] Chesson, G. L. , "Communication and Control in a Cluster Network," Unofficial report, Department of Computer Science, University of Illinois, I97U. [5] Madnick, S. E. and J. W Alsop, "A Modular Approach to File System Design, " Proceedings, AFIPS, I969, SJCC, Vol. 3I+, pp. 1-11+. [6] Knuth, D. E., The Art of Computer Programming: Sorting and Searching , Vol. 3, Addi son-Wesley, Reading, Mass., 1973^ pp. 5OU-569. [7] Hansen, P. B., "The Nucleus of a Multiprogramming System," Comm. ACM, Vol. 13, No. h, April, 1970, pp. 238-250. f 8 J > Operating System Principles , Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1973, p. 385. [9] Dijkstra, E. W., "The Structure of the "THE "-Multiprogramming System," Comm. ACM, Vol. 11, No. 5, May, 1968, pp. 3kl-3k6. BIBLIOGRAPHIC DATA SHEET i. Title and Subtitle 1. Report No. UIUCDCS-R-75-717 Multiprocessor Network System Design by Process Concept '. Authors') Akihiro Iwaya '. Performing Organization Name and Address Department of Computer Science University of Illinois Urbana, Illinois 2. Sponsoring Organization Name and Address Department of Computer Science University of Illinois Urbana, Illinois 5, Supplementary Notes 3. Recipient's Accession No. 5. Report Date December, 197$ 8. Performing Organization Rept. No. 10. Project/Task/Work Unit No. 11. Contract/Grant No. 13. Type of Report & Period Covered 14. 5. Abstracts This report describes the design of a dedicated multiprocessor network which functions as an airline reservation system. The design approach used is as follows: The system is decomposed into cooperative processes. A dedicated processor is assigned to each process and processors are connected via suitable inter-process communication links to form a cluster network. Many aspects in operating system design, such as protection mechanisms, file management, etc. are discussed as well. Key Words and Document Analysis. 17a. Descriptors multiprocessors information systems interprocessor communications Identifiers/Open-Ended Tei 1 OSATI Field/Group '^■nUbility Statement H NTIS-3S (10-70) 19. Security < lass (This Report ) UNCLASSIFIED 20. Security ( lass (This Page UN( LASSIFIF.P 21. No. of Pages 22. USCOMM-DC 40329-P7I UNIVERSITY OF ILLINOI9-URBANA 3 0112 047424251