* * "AN EVALUATION AND SELECTION METHODOLOGY FOR EXPERT SYSTEM SHELLS" by Louis LE BLANC and Tawfik JELASSI** N° 90/39/TM Associate Professor of Policy and Administration, School of Public and Environmental Affairs, Indiana University, Bloomington, Indiana 47405 U.S.A. Associate Professor of Information Systems, INSEAD, Boulevard de Constance, Fontainebleau 77305 Cedex, France Printed at INSEAD, Fontainebleau, France AN EVALUATION AND SELECTION METHODOLOGY FOR EXPERT SYSTEM SHELLS * Louis A. Le Blanc Policy and Administration Faculty School of Public and Environmental Affairs Indiana University Bloomington, Indiana 47405 U.S.A. and Tawfik Jelassi Technology Management Area INSEAD Boulevard de Constance 77305 Fontainebleau France May, 1990 * Forthcoming in Expert Systems with Applications: The International Journal, 1991. AN EVALUATION AND SELECTION METHODOLOGY FOR EXPERT SYSTEM SHELLS ABSTRACT This paper illustrates an evaluation and selection methodology for expert system (ES) shells. The methodology incorporates three stages: 1) ES shell screening; 2) shell evaluation; and, 3) assurance of final ES shell selection. Initially, developing a short list through screening of commercial shell products determines whether appropriate software exists and narrows the field of available expert system software products for detailed consideration. The second stage determines which of the remaining ES shells (the finalists) best meets the needs of the organization, from both functional and technical perspectives. The final stage compares user requirements with the features of the selected ES software by defining how these requirements will be satisfied by building expert system applications with the selected product. The methodology also considers the possibility that, at any stage of the process, no expert system shell is suitable and that a system must be developed with programming languages such as LISP, PROLOG, or some conventional programming language. AN EVALUATION AND SELECTION METHODOLOGY FOR EXPERT SYSTEM SHELLS TABLE OF CONTENTS 1.0 Introduction 1.1 Overview of Expert System Shells 1.2 ES Shell Definition 1.3 The ES Software Selection Problem 1.4 Structure of the Paper 2.0 A Decision Methodology for ES Software Selection 2.1 ES Shell Screening 2.1.1 Identify Candidate Software 2.1.2 Screening Criteria 2.1.3 Pick Finalists 2.2 ES Shell Evaluation 2.2.1 Expand Evaluation Criteria 2.2.2 Obtain Package Information 2.2.3 Evaluate ES Shells 2.2.4 Additional Selection Requirements 2.3 Specific ES Design 2.3.1 Alter Functional Requirements 2.3.2 ES Software Modifications and Supporting Programs 2.3.3 Finalize ES Shell Selection 3.0 Summary and Conclusion Page 1 AN EVALUATION AND SELECTION METHODOLOGY FOR EXPERT SYSTEM SHELLS 1.0 INTRODUCTION 1.1 Overview of Expert System Shells An expert system (ES) is a computer program for capturing human knowledge of a specific problem domain and delivering for decision making purposes. This expertise is often represented in the form of if-then rules, though it may take other forms of knowledge representation, such as "semantic networks" or "frames" (see, e.g., Harmon and King, 1985). Many organizations are developing ES applications with only minimal assistance from their information systems department by using the relatively inexpensive and widely available "off-the-shelf" ES shells. Historically, rather than develop a custom-built ES application, it was possible at times to borrow extensively from a previously constructed ES. This practice has resulted in the development of software that is known as ES shells or skeletal systems. The shells are ES with their knowledge component (which contains the facts and rules of a specific problem domain) removed and leaving only the explanation, inference, and user interface components. Currently, these commercial software packages include a ready- built inference engine, user interface, and framework for the knowledge base. All the system developer has to do is fill in the rules and facts of the knowledge base, state the goals of the system, and make the right connections with the user interface module. This module can provide a variety of techniques to interact with an end user. Page 2 Increasingly, it also includes the ability to integrate with existing databases and transaction processing systems of an organization. The knowledge base and inference engine are the heart of the ES shell. The inference engine, which is the reasoning component of the ES, examines the knowledge base and determines what to do in order to achieve the goal which it has been instructed to perform. 1.2 SS Shell Definition An ra Shell is "the dialog structure and inference engine which, when linked to a knowledge base, functions as a fully operational expert system" (Liebowitz, 1988). It offers several advantages over building ES from scratch. In addition to saving significant resources, using an ES shell allows the knowledge engineer to concentrate on the development of the knowledge base, from which the expert system derives its power. An ES shell constitutes one of the three technological levels of software that comprise the ES development environment (Henderson, 1987; Sprague, 1980; and Turban, 1990). Shells are essentially integrated packages of software that provide a set of capabilities to build specific ES quickly, easily, and at low cost. The other two levels are: Specific Ea, which are operational systems that actually support and/or advise managers/end-users on a specific issue; and Ea tools, which are software elements that provide developers with the facility to construct both specific ES and ES shells. Examples of such ES tools include procedural programming languages, such as C and PASCAL, and artificial intelligence languages like LISP, PROLOG, and FORTH. (Insert Figure 1 About Here) Page 3 Figure 1 (adapted from Sprague, 1980) illustrates the three ES technological levels defined by Turban (1990) and the relationships between them. Specific ES applications can be developed either directly from ES tools or by adapting the ES shell to satisfy the application requirements. In the latter situation, the ES builder could employ the iterative design approach to rapidly select capabilities from the ones available in the ES shell (or delete unnecessary features) as required by the specific ES (represented by the iterative cycling between the ES shell and the specific ES in Figure 1). A "prototyping" methodology could be employed between the "low" technological level of ES tools and a specific ES application. Given the ES development capabilities of a shell, ES applications of all dimensions can be rapidly prototyped. Expert system shells are available for a number of hardware platforms, including mainframes, mini and microcomputers, as well as special-purpose machines such as parallel processors. According to Newquist (1988), a new generation of ES vendors is emerging and placing ES software (i.e., ES shells) into the information systems mainstream. As companies like Apple Computer, IBM, Cullinet Software, and McCormack & Dodge become involved with ES technology, the choices for organi- zations will multiply. The options will range from the choice of hardware platform to the choice of computer language and software interfaces. Page 4 1.3 The ES Software Selection Problem Publications which have addressed the issue of evaluating and selecting ES software (Waterman and Hayes-Roth, 1982; Hayes-Roth, Waterman and Lenat, 1983; Liebowitz, 1985; Harmon and King, 1985; Forman and Nagy, 1985 and 1987; Harmon, et al., 1989; and Vetter, 1989) identified criteria, especially user-related ones, which are critical in selecting a suitable ES shell. However, these authors did not suggest how to incorporate multiple user criteria, as well as technical attributes, into a complete and thorough evaluation and selection process. Furthermore, Lynch (1984 and 1985) and Meador and Mezger (1984) suggest that inadequate examination of prospective software packages leads to serious difficulties if not failures when implementing information systems. Although a number of approaches to selecting application software for transaction processing systems (TPS) and management information systems (MIS) have been proposed (Breslin, 1986; Curry and Bonner, 1983; Gray, 1987; Martin and McClure, 1983), some very critical factors were omitted. These factors include assuring that the selected software package is superior to a custom alternative, or that a screening process is provided to reduce the number of packages subjected to detailed evaluation. To advance the importance of evaluation and selection of ES software (as compared to that of other information systems), it should be noted that ES shells are used to develop multiple management support applications. From the same ES shell, an organization may develop specific ES applications ranging from consultative and training to task execution in functional areas as diverse as marketing, manufacturing and finance. However, other types of software is usually employed only Page 5 for a single application, for example a general ledger or a material requirements planning system which respectively only support a single functional area. To efficiently develop specific ES applications using the iterative design approach (i.e., prototyping), an ES shell need be available (Naumann and Jenkins, 1982; Leary, 1987). The basic objectives of the ES shell are: 1) to permit quick and easy development of a wide variety of specific ES; and, 2) facilitate the iterative development process by which specific ES can respond quickly to changes. In a top-down fashion in order to satisfy these overall objectives, an ES shell must satisfy a number of general criteria or requirements. These criteria can be categorized according to ES components, such as user interface, inference engine and knowledge base. Once the ES shell is established, an enterprise can implement specific ES rapidly. Therefore, the very critical software evaluation and selection process for ES shells should take place prior to any systems analysis and design (iterative development) efforts for specific ES applications (Turban, 1990). 1.4 Structure of the Paper This paper illustrates a method to select the most appropriate ES shell where multiple criteria exist not only from functional requirements but also from technical and vendor-support perspectives. As an essential part of the methodology, an initial stage determines whether an ES software product is even suitable for a particular application, or should an ES be developed from programming languages or other ES tools. Page 6 [Insert Figure 2 About Here] The proposed selection process in this paper also ensures that, at each successive stage of the methodology, an ES shell is superior to a custom-built ES application (see Figure 2). It continually reduces the number of ES software products under consideration until a final selection of a shell is made or constructing an ES application from tools is chosen as the best alternative. This paper is primarily addressed to academics interested in software selection methodologies as well as practitioners faced with ES-related problems. Section Two suggests a multiple criteria method- ology for ES shell selection. The three stages of this process - ES software screening, ES shell evaluation, and ES software assurance - are described. Section Three concludes the paper with some final remarks. 2.0 A DECISION METHODOLOGY FOR ES SOFTWARE SELECTION There are three principal stages in the proposed ES shell evaluation and selection methodology: 1) screening of prospective candidates and development of a short list of ES software packages; 2) selecting an ES shell, if any, which best suits the application requirements; and, 3) matching user requirements to the features of the selected shell and describing how these requirements will be satisfied through the building of prototypes for specific ES. The detailed procedures involved in each stage of the selection process are described in the following sections. Page 7 The complete methodology provides a project outline for the evaluation and selection of ES software. At the end of each stage, the methodology produces a deliverable - a well-defined output or result (i.e., the short list of software packages at the conclusion of the initial stage). Given the project nature of this software evaluation process, scheduling (i.e., determination of start and finish dates) for each stage, as well as the entire project, is possible. (A Gantt chart or other similar device can be used for this purpose.) Staffing requirements can also be determined, including both the number of staff members and their required skills. To evaluate and select ES software, a project team should be assembled with personnel from user departments as well as information systems professionals. 2.1 ES Software Screening During this first stage of the evaluation and selection methodology, three key issues must be addressed: 1) Is there an ES shell that can be used or should a specific ES be developed from tools?; 2) What ES shells are available?; and, 3) Which ES software packages should be seriously considered and evaluated in detail? Examples of commercially available mainframe and microcomputer-based ES shells are given in Table 1. For more examples, see Computerworld (1988). [Insert Table 1 About Here] Page 8 The purpose of developing a short list of shell products is to narrow the field of available ES software for consideration during Shell Evaluation. A short list of candidate ES software (two or three) eliminates any unnecessary effort or confusion which might result because too many alternative ES shells are evaluated. 2.1.1 Identify Candidate ES Software The project team must first identify available ES shells that operate within the enterprise's specific computer hardware and are compatible with its operating system. An enterprise may have dedicated hardware for running ES applications, such as a LISP machine. This would certainly be a technical constraint for ES shells under consideration. To initially identify appropriate ES shells for a particular application, there are several publishers (e.g., catalogs published from Datapro and ICP, as well as Expert Systems Strategies by Harmon and Associates) who provide profiles of ES software vendors and the products they offer. 2.1.2 Screening Criteria The list of screening criteria will contain relatively few items and should concentrate on functional requirements not commonly provided by ES shells and which are very specific to the organization evaluating ES software. Most commercial software packages share many standard functions and capabilities. At the same time, organizations have many identical requirements for software. However, software packages have unique capabilities and features which can readily distinguish one package from another. Similarly, organizations have unique requirements for any software package to satisfy. Page 9 [Insert Figure 3 About Here] The screening process matches the unique capabilities of commercial software packages with the unique requirements of the organization. Figure 3 illustrates the overlap between unique capabilities of software with the unique requirements of an organi- zation. The commercial software packages (e.g., ES shell) with the greatest overlap of unique capabilities with unique organizational requirements would be retained for more detailed scrutiny in the next stage of the evaluation and selection methodology (i.e., ES Shell Evaluation). At this point in the process, the concern is not how well the respective software packages meet the screening criteria, but only if the screening criteria (i.e. unique capabilities) are offered by the software. In the next stage of the methodology, all criteria (including both standard and unique capabilities) are evaluated as to how well they are performed by each respective software package on the short list. The screening criteria can be categorized into four major types: 1) technical requirements; 2) functional requirements; 3) documentation and training; and 4) vendor information. Technical Requirements - An organization's hardware and software strategy will likely dictate the screening criteria in the technical area. To be considered, a shell must fit the framework of the proposed system; it must be compatible with the hardware and software direction already identified. For example, the Social Security Administration required ES shell software to operate on their current hardware and operating system platform (Leary, 1989). The operating system is Page 10 clearly a strict technical requirement. Others could include programming languages, peripherals, memory needs, or data communication capabilities. Functional Requirements - The functional requirements of an ES shell can be classified according to the following system components: 1) User Interface; 2) Inference Engine; and 3) Knowledge Base. The functional requirements associated with each of these three components readily distinguish ES shell evaluation and selection from other software appraisal efforts (Harmon and King, 1985; Harmon, et al., 1989). The user interface component of an ES provides the dialog structure through which the user can access the ES. Normally, the user interacts with the ES via a consultative mode; and many ES interface components include an explanation module. The inference engine component allows hypotheses to be generated from information contained in the knowledge base; while the latter is comprised of facts and rules of thumb based upon experience. [Insert Table 2 About Here] Table 2 lists several examples of functional criteria to conduct the first-cut screening according to the ES components. (Forman and Nagy (1987) offer a more exhaustive list of ES shell criteria.) Possible screening criteria for the user interface component would be that the ES shell offers certain necessary features for the knowledge engineer (e.g., knowledge base editor and rule trace) as well as needed features for the user (e.g., prompted-menu display and explanations and justifications). The inference engine component could necessitate generating new facts by modus ponens and decision tress, while control Page 11 strategies should include forward chaining and procedural control. The knowledge base criteria might require representing facts by object- attribute-value (O-A-V) triplets and frames, handle uncertainty, and model relationships with variable rules. Documentation and Training - ES software packages normally include the documentation required to install and support the ES shell. It should be detailed, complete, and easy to understand. The availability of vendor-developed training sessions and materials may be very important, especially when the organization's personnel are inexperienced in implementing software. Vendor Information - A vendor's ability to support its package through training, consultation, installation, and maintenance assistance is an important consideration in evaluating ES software packages. The vendor should also be able to refer an evaluation team to a user who is willing to talk to them about the ES package and the accompanying support. Some organizations may wish to acquire a run-time version of an ES shell. This shell is modified to incorporate a specific knowledge base and to deactivate certain programming features. Not every vendor may offer such a product. The financial stability of a vendor can also be an important consideration. Financially successful vendors that have been in existence for more than a few years are more likely to adequately support their packages initially and in the future because such vendors attract and retain competent personnel. Page 12 But financial success alone does not ensure adequate and continued support. Vendor image, package reputation, the unit price, and the number of installations are also important considerations. Either the vendors themselves or the users to whom they directed the prospective buyer should be able to provide the needed information in these areas. 2.1.3 Pick Finalists The matching of the screening criteria against the list of ES software and their capabilities will cause the elimination of many (but hopefully not all) shells. The following are typical reasons to eliminate potential ES software candidates: 1) a vendor has only a few employees and has been in business less than a year; 2) operating systems software and hardware is not supported by a vendor; and, 3) system documentation is inadequate. By reducing the number of ES software packages under consider- ation from as many as twenty to two or three, a project team can more effectively devote its attention to the critical details that can make the difference between selecting an adequate ES shell and selecting a superior one. Moreover, by determining which ES software packages are available for the application, the screening process also determines whether an ES shell can be used or if a specific expert system should be constructed from ES tools (see Figure 2). Page 13 2.2 ES Shell Evaluation This second stage focuses on the two or three ES shells that were identified in the screening of ES software. The objective is to evaluate in detail the ES shell finalists and select the one software product that best meets the needs of the organization. The primary tasks of ES software evaluation are: 1) to further define the detailed evaluation criteria; 2) obtain shell product information; and, 3) evaluate the ES software finalists and pick one as the best alternative. 2.2.1 Expand Evaluation Criteria The screening criteria are expanded in more detail and fall into the same four categories: 1) technical requirements; 2) functional requirements; 3) documentation and training; and 4) vendor information. Although all categories are expanded during shell evaluation, the functional requirements receive the main attention and are related to the interface, inference and knowledge components of the ES shell. The purpose of this task is to develop a rather comprehensive functional view of the proposed system and to summarize the requirements that must be satisfied by the ES. As the project team defines the functional criteria, they should also document the levels of importance and need to the user. The following functional requirements for ES software, identified in Harmon and King (1985) are those for a hypothetical enterprise: 1) user interface features, including those for knowledge engineering and data extraction from external sources; 2) inference engine features, including generating new facts and control strategies; and 3) knowledge base features for handling facts, relationships, and Page 14 uncertainty. Table 3 exhibits an expansion of the above summary list of functional shell requirements. [Insert Table 3 About Here] 2.2.2 Obtain Package Information Once the system requirements have been established and the criteria reviewed, the capability of each ES shell to satisfy the requirements must be measured. Several techniques may be used to gather enough information to determine how well each package meets the requirements. In many cases, the project team can meet directly with the vendor sales and support personnel and discuss each requirement. But if requirements are so comprehensive and detailed that a more formal procedure should be followed, a request for proposal (RFP) can be submitted to vendors. It should be noted that preparing an RFP can be time-consuming and costly. In situations where the requirements are not so detailed and complex, it can be replaced by a less formal and more direct procedure such as a basic letter of request. 2.2.3 Evaluate ES Shells Once the vendors' responses to requirements have been received, the actual evaluation process can begin. The review is very detailed at this point, since the project team is looking for specific strengths and weaknesses of each package. Page 15 The project team is searching for deciding factors - not only what ES software packages have and how well they provide it, but also what they don't have. Detailed information is desired on the functions of the ES software and its related processing, including if and how functions that are not included in the ES shell could be implemented. Shovel and Lugasi (1987) compared three models for evaluating and selecting a computer system from four alternatives in a manufacturing case study. The three scoring models were: 1) the additive weight model; 2) the eigen-vector model; and, 3) the multi-attribute utility model. The authors reported that these models provided identical rankings of combinations of computer hardware, software availability and system support. Given a short list of software alternatives, any scoring model will likely yield identical rankings. In evaluating ES shells, Forman and Nagy (1987) applied the Analytic Hierarchy Process (Saaty, 1980) to select a shell from among just three alternative software packages. However, this study did not indicate how the short list was developed nor were any comparisons made with different scoring or ranking techniques. Generating the short list is more critical than the method of ranking the software packages. Criticism levied against weighting schemes for software selection decisions (Klein and Beck, 1987) can be minimized through the screening process and development of a short list. Naumann and Pelvis (1982) successfully applied weighting and scoring measures to select a systems development tool from a relatively short list (four) of candidate techniques. By weighting criteria for only two or three packages rather than for a dozen (in which case the aforementioned criticism is probably valid), the proposed evaluation Page 16 process allows for a very detailed and focused inspection of just the few best alternative ES software products. 2.2.4 Additional Selection Requirements The ranking scores are not necessarily the determining factor for selecting a particular ES shell. They should be used as a decision tool - a means for organizing and summarizing the significant quantity of information that the project team has collected. The highest score may not always indicate the best ES shell. The scores may not accurately reflect certain intangible factors such as the cosmetic appearance of reports and screens, how easy it will be to use the ES software, etc. Tailoring - The scores may not indicate how much time or the level of technical expertise needed to "tailor" the ES shell. Tailoring can be either costly if it is relatively extensive, or difficult if the internal structure of the software is complex. The importance of the technical processing architecture will depend on how much tailoring is anticipated. Furthermore, the architecture of the ES shell also determines how much modification is even possible. Documentation - A decision to use a particular ES shell should not be made on the basis of functional requirements alone. The ES software's documentation is a very important non-functional factor. Its accuracy and level of detail can affect the time it will take to evaluate and modify the package. Comparing documentation is sometimes very difficult at this stage of the evaluation process due to differences in format, style, etc. Still, it is important to review the vendors' documentation and to Page 17 reconfirm that the information collected on maintenance and support, for instance, is accurate and correct. The availability of and easy access to vendor-developed training sessions is very important. Training materials, such as user tutorials or video-based learning aids, may be especially critical when company personnel are inexperienced in developing ES applications. At this stage, the vendor should be able to refer the project team to current users of his software. The comments of these customers should prove invaluable. Site visits and demonstrations of the ES software in operation may be helpful. 2.3 Specific ES Design Assuming that ES software which is anticipated to provide satisfactory performance has been selected (see Figure 2), the project team is ready to confirm the selection by developing some specific ES prototypes based on the chosen shell. The primary reasons for this stage are to ensure that the ES package can be used effectively and to provide one last chance to reconsider the ES software decision. It is often difficult to determine the degree of user satisfaction until the design process has begun for specific applications utilizing the selected ES software. Therefore, this stage involves the design of demonstration prototypes of specific ES built from the ES shell. Such prototypes can provide significant benefits before finalizing the selection decision (Alavi, 1984; Meador and Mezger, 1984; and Janson, 1986). Page 18 2.3.1 Alter Functional Requirements Based on the capabilities of the selected shell as experienced in the prototyping exercise of specific ES, the definition of user requirements might be altered to include package features not previously considered, or to change or eliminate others. The modified requirements should be reviewed with the users. The effect of ES software deficiencies perhaps can be minimized by altering user procedures or postponing the implementation of some requirements until shell enhancements could be made. 2.3.2 ES Software Modifications and Supporting Programs Typically, the specific ES being developed requires certain functions and/or interfaces not provided by the software. If an ES shell does not meet all the functional requirements of a system, the following alternatives should be considered: 1) persuade the vendor to include additional features; 2) develop supplemental software; and, 3) modify the vendor's software. The chosen alternative will depend on the extent of the ES shell's deficiencies, the potential costs and benefits of altering the software, and the size and technical skills of the programming staff. Vendor-Supplied Enhancements - If possible, the vendor should be persuaded to do the modification for the purchaser. This is often the best alternative, since the vendor will usually update and maintain the software on a routine basis. Page 19 Supporting Programs - Developing software to supplement the vendor's ES package is often the most practical alternative. The vendor will normally continue to service the ES shell; but if this alternative is selected, the supplemental software should conform to the standards used by the vendor in developing the ES shell. Alter Code - Modifying an ES shell is usually not recommended. If the software is modified, the vendor may be reluctant or may even refuse to service the package. Updates to the software may not be compatible with the modifications effected. In some cases, this may not even be an option, since the purchaser of the ES shell does not have (or cannot get at any price) a copy of the source code. In this instance, all that the purchaser can do is to build a front-end or back-end to the software package. 2.3.3 Finalize ES Shell Selection It is not unusual for an organization to complete the last stage of the evaluation and selection process for ES software, only to realize that the selected shell is not the best choice. Perhaps too many compromises have been made and users are no longer satisfied. Possibly, the tailoring effort has become so extensive that a custom ES (i.e., specific ES application built from tools) would be a better choice (see Figure 2). Therefore, a final commitment to using a particular shell should be avoided until the design of specific ES using the potential software package has progressed to the point where user satisfaction is ensured. Page 20 3.0 SUMMARY AND CONCLUSION Using ES shells for the development of specific ES will reduce personnel requirements and development costs. Conducting the evaluation of ES software increases the effort necessary for developing specific ES, but this undertaking is offset by the aforementioned advantages of using a shell package. Despite the promises offered by ES software, the performance of some ES shells is much less than expected. Weak or non-existent selection procedures may explain much of this poor implementation record. The methodology proposed in this paper will hopefully reduce the risks associated with ES software and facilitate success in developing specific ES from shells (Alavi, 1984; Janson, 1986). The most critical phases of the methodology are determining the criteria for an ES shell as incorporated in the first phase (the development of a short list) and the confirmation element of the third phase (design of specific ES with the selected shell). Initially, the screening process determines whether a shell is feasible and reduces the number of ES software packages to be evaluated in detail. Finally, the development of specific ES with the selected shell ensures that the ES software can be used effectively and provides a last chance to consider building specific ES from tools. Page 21 REFERENCES Alavi, M., "An Assessment of the Prototyping Approach to Information Systems Development," Communications gf the gO, 1984, 27 (6), 556-563. Breslin, J. Selecting and Installing Software packages. Westport, CN: Quorom Books, 1986. Domputerworld, "Expert System Shells," October 17, 1988, 90-96. Curry, J. W., and Bonner, D. M. Mow tg Find and Bu y good Software: A Guide for Business Anal Professional people. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1983. Forman, E. H., and Nagy, T. J. "A Multicriteria Model to Select an Expert System Generator." In proceedings g the Expert Systems in Government Conference, Washington, DC: IEEE/MITRE, 1985. Forman, E. H., and Nagy, T. J. "EXSYS vs. TOPSI/OPS5 vs. MICRO-PS: A Multicriteria Model to Select an Expert System Generator." Telematics And Informatics, 1987, 4 (1), 37-54. Gray, C. D. The bight Choice: A Complete Guide IQ Evaluating, Selecting, and Installing MEEII Software. Essex Junction, VT: Oliver Wight Limited Publications, Inc., 1987. Harmon, P., and King, D. Expert Systems: Artificial Intelligence in business. New York, NY: John Wiley & Sons, Inc., 1985. Harmon, P., Maus, R., and Morrissey, W. Expert Systems: Tools & Applications. New York, NY: John Wiley & Sons, Inc., 1985. Hayes-Roth, F., Waterman, D. A., and Lenat, D. B. Building Expert Systems. Reading, MA: Addison-Wesley, 1983. Henderson, J. C., "Finding Synergy Between Decision Support Systems and Expert Systems Research," Decision Sciences, 1987, 18 (3), 333-349. Janson, M., "Applying a Pilot System and Prototyping Approach to Systems Development and Implementation," Information And Management, 1986, 10 (4), 209-216. Klein, G., and Beck, P. O., "A Decision Aid for Selecting among Information System Alternatives," MIS Quarterl y , 1987, 11 (2) 177-185. Leary, E., "Collecting Shells for Rapid Prototyping," Computerworla, November 23, 1987, p. S2. Leary, E., "Expert Systems at the Social Security Administration," Journal QL policy Analysis and Management, 1989, 8 (2), 200-203. Page 22 Liebowitz, J., "Determining Functional Requirements for NASA Goddard's Command Management System Software Design Using Expert Systems," D. Sc. dissertation, George Washington University, Washington, DC, 1985. Liebowitz, J. introduction t2 Expert Systems. Watsonville, CA: Mitchell Publishing, 1988. Lynch, R. K., "Implementing Packaged Application Software: Hidden Costs and New Challenges," Systems, Objectives, Solutions, 1984, 4 (4), 227-234. Lynch, R. K., "Nine Pitfalls in Implementing Packaged Applications Software," Journal gf Information Systems Management, 1985, 2 (2), 88-92. Martin, J., and McClure, C., "Buying Software off the Rack," Harvard Business Review, 1983, 61 (6), 32-47. Meador, G. L., and Mezger, R. A., "Selecting An End User Programming Language For DSS Development," ma Quarterly, 1984, 8 (4), 267-281. Naumann, J. D., and Jenkins, A. M., "Prototyping: The New Paradigm For Systems Development," MIS Ouarterly, 1982, 6 (3), 29-44. Naumann, J. D., and Palvia, S., "A Selection Model for Systems Development Tools," MI$ Ouarterly, 1982, 6 (1), 39-48. Newquist, H., "AI Adapts to Use of the Vernacular," Computerwor14, October 17, 1988, pp. 82-83. Saaty, T. L., The Analytic Hierarchy Process, York, NY; McGraw-Hill, 1980. Shoval, P. and Y. Lugasi, "Models for Computer System Evaluation and Selection," Information k Management, 1987, 12 (3), 117-129. Sprague, R. H., Jr., "A Framework for the Development of Decision Support Systems," MIS Quarterly, 1980, 4 (4), 1-26. Turban, E. Decision Support And Expert Systems (second Edition). New York, NY: Macmillan Company, 1990. Vedder, R. G., "PC-Based Expert System Shells: Some Desirable and Less Desirable Characteristics," Expert Systems, 1989, 6 (1), 28-42. Waterman, D. A. A Guide tga £pert Systems. Reading, MA: Addison-Wesley, 1986. Waterman, D. A., and Hayes-Roth, F. An investigation gf Tools for Building Expert Systems. Santa Monica, CA: Rand Corporation, 1982. FIGURE 1. LEVELS OF ES TECHNOLOGY Sisecific ES Applicanons MoralIv• Deveicio►ent IS Soo* No guild Swine ES horn Tools Evaluate U Shells Yes Confirm U Shell Selection FIGURE 2. A MULTIPLE CRITERIA DECISION METHODOLOGY FOR ES SHELL SELECTION Develop Mon list IS Software Purchase ES Software Standard R•quinments Standard N CapablIttios I U U' N U/ / FIGURE 3. MATCHING OF SOFTWARE SCREENING CRITERIA Software Organization * TABLE 1. REPRESENTATIVE ES SHELL PRODUCTS Large Machine Shells** Product KEE ART ESE EMYCIN S.1 Level 5 KES Knowledge Craft IBM Knowledgetool Rulemaster 2 KBMS ADS Microcomputer Shells Product 1st-Class EXSYS M.1 KES-PC GURU Personal Consultant VP-Expert Knowledgepro Nexpert Vendor Intellicorp Inference Corp. IBM Stanford University Teknowledge, Inc. Information Builders Software A & E Carnegie Group, Inc. IBM Radian Corp. AI Corporation AION Corporation Vendor lst-Class Expert Systems EXSYS, Inc. Teknowledge, Inc. Software A & E Micro Database Systems Texas Instruments Paperback Software Knowledge Garden, Inc. Neuron Data * Numerous shells are available in both large machine and microcomputer versions. ** Includes minicomputers, mainframes and specialized LISP machines. TABLE 2. POTENTIAL SCREENING CRITERIA OF ES COMPONENTS User Inference Knowledge Interface Engine Base Knowledge Base Modus Ponens Editor Decision Tree Prompted-Menu Algorithm Display Forward Explanation Chaining Justification Backward Chaining Graphics Mixed Chaining Procedural Control Database/ Spreadsheet Links Hypertext O-A-V Triplets Frames Variable Rules Certainty Factors TABLE 3. DETAILED FUNCTIONAL CRITERIA FOR ES SOFTWARE Interface Component Knowledge Engineering Facilities Knowledge Base Editor Traces and Probes Graphic Display (Windows) End-User Elements Explanations and Justifications Line-Oriented Display Prompted-Menu Display Sources of Data Instruments Data Bases Other Languages and Procedures Inference Component Generating New Facts Modus Ponens Resolution Decision Tree Algorithm Control Strategies Backward Chaining Forward Chaining Depth-First Search Breadth-First Search Procedural Control Knowledge Component Representing Facts A-V Pairs O-A-V Triplets Frames Relationships If-Then Rules Variable Rules Examples Uncertainty Inheritance Certainty Factors Probabilities 1986 86/01 Arnoud DE MEYER 86/02 Philippe A. NAERT Marcel VEVERBERGN and Guido VERSVIJVEL 86/03 Michael BRIMN 86/04 Spyros MAKRIDAKIS and Michele HIBON 86/05 Charles A. VYPLOSZ 86/06 Francesco GIAVAllI, Jeff R. SHEEN and Charles A. VYPLOSZ 86/07 Douglas L. MacLACHLAN and Spyros MAKRIDAKIS 86/08 Jose de la TORRE and David H. NECKAR 86/09 Philippe C. HASPESLACH 86/10 R. MOENART, Arnoud DE MEYER, J. BARBE and D. DESCHOOLMEESTER. 86/11 Philippe A. NAERT and Alain BULTEZ 86/12 Roger BETANCOURT and David GAUTSCHI 86/13 S.P. ANDERSON and Damien J. NEVEN "The R i D/Production interface". "Subjective estimation in integrating communication budget and allocation decisions: a case study", January 1986. "Sponsorship and the diffusion of organizational innovation: a preliminary view". "Confidence intervals: an empirical investigation for the series in the M- Competition" . "A note on the reduction of the workweek", July 1985. "The real exchange rate and the fiscal aspects of a natural resource discovery", Revised version: February 1986. "Judgmental biases in sales forecasting", February 1986. "Forecasting political risks for international operations", Second Draft: March 3, 1986. "Conceptualizing the strategic process in diversified firms: the role and nature of the corporate influence process", February 1986. "Analysing the issues concerning technological de-maturity". "From "Lydiametry" to "Pinkhamization": misspecifying advertising dynamics rarely affects profitability". "The economics of retail firms", Revised April 1986. "Spatial competition a la Cournot". 86/16 B. Espen ECKBO and Hervig M. LANGOHR 86/17 David B. JEMISON 86/18 James TEBOUL and V. MALLERET 86/19 Rob R. VEITZ 86/20 Albert CORHAY, Gabriel HAVAVINI and Pierre A. MICHEL 86/21 Albert CORHAY, Gabriel A. HAVAVINI and Pierre A. MICHEL 86/22 Albert CORNAY, Gabriel A. HAVAVINI and Pierre A. MICHEL 86/23 Arnoud DE MEYER 86/24 David GAUTSCNI and Vithala R. RAO 86/25 H. Peter GRAY and Ingo WALTER 86/26 Barry EICHENGREEN and Charles VYPLOSZ 86/27 Karel COOL and Ingemar DIERICKX 86/28 Manfred KEYS DE VRIES and Danny MILLER 86/29 Manfred KETS DE VRIES 86/30 Manfred KETS DE VRIES 86/31 Arnoud DE MEYER "Les primes des offres publiques, la note d'information et le marche des transferts de contr8le des societie. "Strategic capability transfer in acquisition integration", May 1986. "Towards an operational definition of services", 1986. "Nostradamus: a knowledge-based forecasting advisor". "The pricing of equity on the London stock exchange: seasonality and size premium", June 1986. "Risk-premia seasonality in U.S. and European equity markets", February 1986. "Seasonality in the risk-return relationships some international evidence", July 1986. "An exploratory study on the integration of information systems in manufacturing", July 1986. "A methodology for specification and aggregation in product concept testing", July 1986. "Protection", August 1986. "The economic consequences of the Franc Poincare", September 1986. "Negative risk-return relationships in business strategy: paradox or truism?", October 1986. "Interpreting organizational texts. "Why follow the leader?". "The succession game: the real story. "Flexibility: the next competitive battle", October 1986. INSEAD WORKING PAPERS SERIES "Comparaison Internationale des marges brutes du commerce", June 1985. "Nov the managerial attitudes of firms vith FMS differ from other manufacturing firms: survey results", June 1986. 86/31 Arnoud DE MEYER, Jinichiro NAKANE, Jeffrey G. MILLER and Kasra FERDOVS 86/32 Karel COOL and Dan SCHENDEL "Flexibility: the next competitive battle", Revised Version: March 1987 Performance differences among strategic group members", October 1986. 86/14 Charles WALDMAN 86/15 Mihkel TOMBAK and Arhoud DE MEYER 87/06 Arun K. JAIN, "Customer loyalty as a construct in the Christian PINSON and marketing of banking services", July 1986. Naresh K. MALHOTRA 86/33 Ernst BALTENSPERGER and Jean DERMINE 86/34 Philippe HASPESLAGH and David JEMISON 86/35 Jean DERMINE 86/36 Albert CORHAY and Gabriel HAVAVINI 86/37 David GAUTSCHI and Roger BETANCOURT 86/38 Gabriel HAVAVINI 86/39 Gabriel HAVAVINI Pierre MICHEL and Albert CORHAY 86/40 Charles VYPLOSZ 86/41 Kasra FERDOWS and Wickham SKINNER 86/42 Kasra FERDOWS and Per LINDBERG 86/43 Damien NEVEN 86/44 Ingemar DIERICKX Carmen MATUTES and Damien NEVEN 1987 87/01 Manfred KETS DE VRIES 87/02 Claude VIALLET 87/03 David GAUTSCHI and Vithala RAO 87/04 Sumantra CHOSHAL and Christopher BARTLETT 87/05 Arnoud DE MEYER and Kasra FERDOWS "The role of public policy in insuring financial stability: a cross-country, comparative perspective", August 1986, Revised November 1986. "Acquisitions: myths and reality", July 1986. "Measuring the market value of a bank, a primer", November 1986. "Seasonality in the risk-return relationship: some international evidence", July 1986. "The evolution of retailing: a suggested economic interpretation". "Financial innovation and recent developments in the French capital markets", Updated: September 1986. "The pricing of common stocks on the Brussels stock exchange: a re-examination of the evidence", November 1986. "Capital flovs liberalization and the EMS, a French perspective", December 1986. "Manufacturing in a new perspective", July 1986. "FMS as indicator of manufacturing strategy", December 1986. "On the existence of equilibrium in hotelling's model", November 1986. "Value added tax and competition", December 1986. "Prisoners of leadership". "An empirical investigation of international asset pricing", November 1986. "A methodology for specification and aggregation in product concept testing", Revised Version: January 1987. "Organizing for innovations: case of the multinational corporation", February 1987. "Managerial focal points in manufacturing strategy", February 1987. "Equity pricing and stock market anomalies", February 1987. "Leaders who can't manage", February 1987. "Entrepreneurial activities of European MBAs", March 1987. "A cultural view of organizational change", March 1987 "Forecasting and loss functions", March 1987. "The Janus Bead: learning frost the superior and subordinate faces of the manager's job", April 1987. "Multinational corporations as differentiated networks", April 1987. "Product Standards and Competitive Strategy: An Analysis of the Principles", May 1987. "KETAFORECASTING: Vmys of improving Forecasting. Accuracy and Usefulness", May 1987. "Takeover attempts: what does the language tell us?, June 1987. "Managers' cognitive maps for upward and dovnvard relationships", June 1987. "Patents and the European biotechnology lag: a study of large European pharmaceutical firms", June 1987. "Why the EMS? Dynamic games and the equilibrium policy regime, May 1987. "A new approach to statistical forecasting", June 1987. "Strategy formulation: the impact of national culture", Revised: July 1987. "Conflicting ideologies: structural and motivational consequences", August 1987. "The demand for retail products and the household production model: nev views on complementarity and substitutability". 87/07 Rolf BANZ and Gabriel HAVAVINI 87/08 Manfred KETS DE VRIES 87/09 Lister VICKERY, Mark PILKINGTON and Paul READ 87/10 Andre LAURENT 87/11 Robert FILDES and Spyros MAKRIDAKIS 87/12 Fernando BARTOLOME and Andre LAURENT 87/13 Sumantra GHOSHAL and Nitin NOHRIA 87/14 Landis GABEL 87/15 Spyros MAKRIDAKIS 87/16 Susan SCHNEIDER and Roger DUNBAR 87/17 Andre LAURENT and Fernando BARTOLOME 87/18 Reinhard ANGELMAR and Christoph LIEBSCHER 87/19 David BEGG and Charles VYPLOSZ 87/20 Spyros MAKRIDAKIS 87/21 Susan SCHNEIDER 87/22 Susan SCHNEIDER 87/23 Roger BETANCOURT David GAUTSCHI 87/29 Susan SCHNEIDER and Paul SHRIVASTAVA "The internal and external careers: a theoretical and cross-cultural perspective", Spring 1987. "The robustness of MDS configurations in the face of incomplete data", March 1987, Revised: July 1987. "Demand complementarities, household production and retail assortments", July 1987. "Is there a capital shortage in Europe?", August 1987. "Controlling the interest-rate risk of bonds: an introduction to duration analysis and immunization strategies", September 1987. "Interpreting strategic behavior: basic assumptions themes in organizations", September 1987 87/41 Cavriel HAVAVINI and Claude VIALLET 87/42 Damien NEVEN and Jacques-F. THISSE 87/43 Jean CABSZEVICZ and Jacques-F. THISSE 87/44 Jonathan HAMILTON, Jacques-F. THISSE and Anita WESKAMP 87/45 Karel COOL, David JEMISON and Ingemar DIERICKX 87/46 Ingemar DIERICKX and Karel COOL "Seasonality, size premium and the relationship between the risk and the return of French common stocks", November 1987 'Combining horizontal and vertical differentiation: the principle of max-min differentiation", December 1987 "Location", December 1987 "Spatial discrimination: Bertrand vs. Cournot in a model of location choice", December 1987 "Business strategy, market structure and risk- return relationships: a causal interpretation", December 1987. "Asset stock accumulation and sustainability of competitive advantage", December 1987. 87/24 C.B. DERR and Andre LAURENT 87/25 A. K. JAIN, N. K. MALHOTRA and Christian PINSON 87/26 Roger BETANCOURT and David GAUTSCHI 87/27 Michael BURDA 87/28 Gabriel HAVAVINI 87/30 Jonathan HAMILTON "Spatial competition and the Core", August W. Bentley MACLEOD 1987. 1988 and J. F. THISSE 87/31 Martine OUINZII and J. F. THISSE 87/32 Arnoud DE MEYER 87/33 Yves DOZ and Amy SHUEN 87/34 Kasra FERDOWS and Arnoud DE MEYER 87/35 P. J. LEDERER and J. F. THISSE 87/36 Manfred KETS DE VRIES 87/37 Landis LABEL 87/38 Susan SCHNEIDER 87/39 Manfred KETS DE VRIES 1987 87/40 Carmen MATUTES and Pierre REGIBEAU "On the optimality of central places", September 1987. "German, French and British manufacturing strategies less different than one thinks", September 1987. "A process framework for analyzing cooperation between firms", September 1987. "European manufacturers: the dangers of complacency. Insights from the 1987 European manufacturing futures survey, October 1987. "Competitive location on networks under discriminatory pricing", September 1987. "Prisoners of leadership", Revised version October 1987. "Privatization: its motives and likely consequences", October 1987. "Strategy formulation: the impact of national culture", October 1987. 'The dark side of CEO succession", November "Product compatibility and the scope of entry", November 1987 88/01 Michael LAWRENCE and Spyros MAKRIDAKIS 88/02 Spyros MAKRIDAKIS 88/03 James TEBOUL 88/04 Susan SCHNEIDER 88/05 Charles WYPLOSZ 88/06 Reinhard ANGELMAR 88/07 Ingemar DIERICKX and Karel COOL 88/08 Reinhard ANGELMAR and Susan SCHNEIDER 88/09 Bernard SINCLAIR- DESGAGNe 88/10 Bernard SINCLAIR- DESGAGN4 88/11 Bernard SINCLAIR- DESGAGNe "Factors affecting judgemental forecasts and confidence intervals", January 1988. "Predicting recessions and other turning points", January 1988. "De-industrialize service for quality", January 1988. "National vs. corporate culture: implications for human resource management", January 1988. "The svinging dollar: is Europe out of step?", January 1988. "Les conflits dans les canaux de distribution", January 1988. "Competitive advantage: a resource based perspective", January 1988. "Issues in the study of organizational cognition", February 1988. "Price formation and product design through bidding", February 1988. "The robustness of some standard auction game forms", February 1988. "When stationary strategies are equilibrium bidding strategy: The single-crossing property", February 1988. 88/12 Spyros MAKRIDAKIS "Business firms and managers in the 21st century", February 1988 88/13 Manfred KETS DE VRIES "Alezithymia in organizational life: the organization man revisited", February 1988. 88/14 Alain NOEL "The interpretation of strategies: a study of the impact of CEOs on the corporation", March 1988. 88/15 Anil DEOLALIKAR and Lars-Hendrik ROLLER 88/16 Gabriel HAWAWINI 88/17 Michael BURDA 88/18 Michael BURDA 88/19 M.J. LAWRENCE and Spyros MAKRIDAKIS 88/20 Jean DERMINE, Damien NEVEN and J.F. THISSE 88/21 James TEBOUL 88/22 Lars-Hendrik ROLLER 88/23 Sjur Didrik FLAM and Georges ZACCOUR 88/24 B. Espen ECKBO and Hervig LANGOHR 88/25 Everette S. GARDNER and Spyros MAKRIDAKIS "The production of and returns from industrial innovation: an econometric analysis for a developing country", December 1987. "Market efficiency and equity pricing: international evidence and implications for global investing", March 1988. "Monopolistic competition, costs of adjustment and the behavior of European employment", September 1987. "Reflections on "Wait Unemployment" in Europe", November 1987, revised February 1988. "Individual bias in judgements of confidence", March 1988. "Portfolio selection by mutual funds, an equilibrium model", March 1988. "De-industrialize service for quality", March 1988 (88/03 Revised). "Proper Quadratic Functions with an Application to AT&T", May 1987 (Revised March 1988). "Equilibres de Nash-Cournot dans le marche europeen du gaz: un cas oil les solutions en boucle ouverte et en feedback coincident", Mars 1988 "Information disclosure, means of payment, and takeover premia. Public and Private tender offers in France", July 1985, Sixth revision, April 1988. "The future of forecasting", April 1988. 88/26 Sjur Didrik FLAM and Georges ZACCOUR 88/27 Murugappa KRISHNAN Lars-Hendrik ROLLER 88/28 Sumantra GHOSHAL and C. A. BARTLETT "Semi-competitive Cournot equilibrium in multistage oligopolies", April 1988. "Entry game with resalable capacity", April 1988. "The multinational corporation as a network: perspectives from interorganizational theory" May 1988. 88/29 Naresh K. MALHOTRA, Christian PINSON and Arun K. JAIN 88/30 Catherine C. ECKEL and Theo VERMAELEN 88/31 Sumantra CHOSHAL and Christopher BARTLETT 88/32 Kasra FERDOVS and David SACKRIDER 88/33 Mihkel M. TOMBAK 88/34 Mihkel M. TOMBAK 88/35 Mihkel M. TOMBAK 88/36 Vikas TIBREVALA and Bruce BUCHANAN 88/37 Murugappa KRISHNAN Lars-Hendrik ROLLER 88/38 Manfred KETS DE VRIES 88/39 Manfred KETS DE VRIES 88/40 Josef LAKONISHOK and Theo VERMAELEN 88/41 Charles WYPLOSZ 88/42 Paul EVANS 88/43 B. SINCLAIR-DESGAGNE 88/44 Essam MAHMOUD and Spyros MAKRIDAKIS 88/45 Robert KORAJCZYK and Claude VIALLET 88/46 Yves DOZ and Amy SHUEN "Consumer cognitive complexity and the dimensionality of multidimensional scaling configurations", May 1988. "The financial fallout from Chernobyl: risk perceptions and regulatory response", May 1988. "Creation, adoption, and diffusion of innovations by subsidiaries of multinational corporations", June 1988. "International manufacturing: positioning plants for success", June'1988. "The importance of flexibility in manufacturing", June 1988. "Flexibility: an important dimension in manufacturing", June 1988. "A strategic analysis of investment in flexible manufacturing systems", July 1988. "A Predictive Test of the NBD Model that Controls for Non-stationarity", June 1988. "Regulating Price-Liability Competition To Improve Welfare", July 1988. "The Motivating Netale of Envy : A Forgotten Factor in Management, April 88. "The Leader as Mirror : Clinical Reflections", July 1988. "Anomalous price behavior around repurchase tender offers", August 1988. "Assymetry in the EMS: intentional or systemic?", August 1988. "Organizational development in the transnational enterprise", June 1988. "Croup decision support systems implement Bayesian rationality", September 1988. "The state of the art and future directions in combining forecasts", September 1988. "An empirical investigation of international asset pricing", November 1986, revised August 1988. "From intent to outcome: a process framework for partnerships", August 1988. 88/47 Alain BULTEZ, Els GIJSBRECHTS, Philippe NAERT and Piet VANDEN ABEELE 88/48 Michael BURDA 88/49 Nathalie DIERKENS 88/50 Rob WEITZ and Arnoud DE MEYER 88/51 Rob VEITZ 88/52 Susan SCHNEIDER and Reinhard ANCELMAR 88/53 Manfred KETS DE VRIES 88/54 Lars-Hendrik ROLLER and Mihkel M. TOMBAK 88/55 Peter BOSSAERTS and Pierre HILLION 88/56 Pierre HILLION 88/57 Wilfricd VANHONACKER and Lydia PRICE 88/58 B. SINCLAIR-DESGAGNE and Mihkel M. TOMBAK "Asymmetric cannibalism between substitute items listed by retailers", September 1988. "Reflections on 'Wait unemployment' in Europe, II", April 1988 revised September 1988. "Information asymmetry and equity issues", September 1988. "Managing expert systems: from inception through updating", October 1987. "Technology, work, and the organization: the impact of expert systems", July 1988. "Cognition and organizational analysis: vho's minding the store?", September 1988. "Whatever happened to the philosopher king: the leader's addiction to power, September 1988. "Strategic choice of flexible production technologies and welfare implications", October 1988 "Method of moments tests of contingent claims asset pricing models", October 1988. "Size-sorted portfolios and the violation of the random walk hypothesis: Additional empirical evidence and implication for tests of asset pricing models", June 1988. "Data transferability: estimating the response effect of future events based on historical analogy", October 1988. "Assessing economic inequality", November 1988. 88/63 Fernando NASCIHENTO and Vilfried R. VANHONACKER 88/64 Kasra FERDOWS 88/65 Arnoud DE MEYER and Kasra FERDOVS 88/66 Nathalie DIERKENS 88/67 Paul S. ADLER and Kasra FERDOWS 1989 89/01 Joyce K. BYRER and Tavfik JELASSI 89/02 Louis A. LE BLANC and Tavfik JELASSI 89/03 Beth H. JONES and Tavfik JELASSI 89/04 Kasra FERDOWS and Arnoud DE MEYER 89/05 Martin KILDUFF and Reinhard ANCELHAR 89/06 Mihkel M. TOMBAK and B. SINCLAIR-DESGACNE "Strategic pricing of differentiated consumer durables in a dynamic duopoly: a numerical analysis", October 1988. "Charting strategic roles for international factories", December 1988. "Quality up, technology down", October 1988. "A discussion of exact measures of information assymetry: the example of Myers and Majluf model or the importance of the asset structure of the firm", December 1988. "The chief technology officer", December 1988. "The impact of language theories on DSS dialog", January 1989. "DSS software selection: a multiple criteria decision methodology", January 1989. "Negotiation support: the effects of computer intervention and conflict level on bargaining outcome", January 1989. "Lasting improvement in manufacturing performance: In search of a new theory", January 1989. "Shared history or shared culture? The effects of time, culture, and performance on institutionalization in simulated organizations", January 1989. "Coordinating manufacturing and business strategies: I", February 1989. 88/59 Martin KILDUFF "The interpersonal structure of decision 89/07 Damien J. NEVEN "Structural adjustment in European retail making: a social comparison approach to banking. Some view from industrial organizational choice", November 1988. organisation", January 1989. 88/60 Michael BURDA 88/61 Lars-Hendrik ROLLER 88/62 Cynthia VAN HULLE, Theo VERMAELEN and Paul DE WOUTERS "Is mismatch really the problem? Some estimates of the Chelvood Cate I/ model vith US data", September 1988. "Modelling cost structure: the Bell System revisited", November 1988. "Regulation, taxes and the market for corporate control in Belgium", September 1988. 89/08 Arnoud DE MEYER and Hellmut SCHUTTE 89/09 Damien NEVEN, Carmen MATUTES and Marcel CORSTJENS 89/10 Nathalie DIERKENS, Bruno GERARD and Pierre MILLION "Trends in the development of technology and their effects on the production structure in the European Community", January 1989. "Brand proliferation and entry deterrence", February 1989. "A market based approach to the valuation of the assets in place and the growth opportunities of the firm", December 1988. 89/11 Manfred KETS DE VRIES and Alain NOEL 89/12 Vilfried VANHONACKER 89/13 Manfred KETS DE VRIES 89/14 Reinhard ANGELMAR 89/15 Reinhard ANGELMAR 89/16 Vilfried VANHONACKER, Donald LEHMANN and Fareena SULTAN 89/17 Gilles AMADO, Claude FAUCHEUX and Andre LAURENT 89/18 Srinivasan BALAK- RISHNAN and Mitchell KOZA 89/19 Vilfried VANHONACKER, Donald LEHMANN and Fareena SULTAN 89/20 Vilfried VANHONACKER and Russell VINER 89/21 Arnoud de MEYER and Kasra FERDOWS 89/22 Manfred KETS DE VRIES and Sydney PERZOW 89/23 Robert KORAJCZYK and Claude VIALLET 89/24 Martin KILDUFF and Mitchel ABOLAFIA 89/25 Roger BETANCOURT and David GAUTSCHI 89/26 Charles BEAN, Edmond MALINVAUD, Peter BERNHOLZ, Francesco GIAVAllI and Charles VYPLOSZ "Understanding the leader-strategy interface: application of the strategic relationship interview method", February 1989. "Estimating dynamic response models when the data are subject to different temporal aggregation", January 1989. "The impostor syndrome: a disquieting phenomenon in organizational life", February 1989. "Product innovation: a tool for competitive advantage", March 1989. "Evaluating a firm's product innovation performance", March 1989. "Combining related and sparse data in linear regression models", February 1989. "Changement organisationnel et rialitis culturelles: contrastes franco-americains", March 1989. "Information asymmetry, market failure and joint-ventures: theory and evidence", March 1989 "Combining related and sparse data in linear regression models", Revised March 1989 "A rational random behavior model of choice", Revised March 1989 "Influence of manufacturing improvement programmes on performance", April 1989 "What is the role of character in psychoanalysis? April 1989 "Equity risk premia and the pricing of foreign exchange risk" April 1989 "The social destruction of reality: Organisational conflict as social drama" April 1989 "Two essential characteristics of retail markets and their economic consequences" March 1989 "Macroeconomic policies for 1992: the transition and after", April 1989 89/27 David KRACKHARDT and Martin KILDUFF 89/28 Martin KILDUFF 89/29 Robert GOGEL and Jean-Claude LARRECHE 89/30 Lars-Hendrik ROLLER and Mihkel M. TOMBAK 89/31 Michael C. BURDA and Stefan GERLACH 89/32 Peter HAUG and Tavfik JELASSI 89/33 Bernard SINCLAIR- DESGAGNE 89/34 Sumantra CHOSHAL and Nittin NOHRIA 89/35 Jean DERMINE and Pierre BILLION 89/36 Martin KILDUFF 89/37 Manfred KETS DE VRIES 89/38 Manfrd KETS DE VRIES 89/39 Robert KORAJCZYK and Claude VIALLET 89/40 Balaji CHAKRAVARTHY 89/41 B. SINCLAIR-DESGAGNE and Nathalie DIERKENS 89/42 Robert ANSON and Tavfik JELASSI 89/43 Michael BURDA 89/44 Balaji CHAKRAVARTHY and Peter LORANGF. 89/45 Rob WEITZ and Arnoud DE MEYER "Friendship patterns and cultural attributions: the control of organizational diversity", April 1989 "The interpersonal structure of decision making: a social comparison approach to organizational choice", Revised April 1989 "The battlefield for 1992: product strength and geographic coverage", May 1989 "Competition and Investment in Flexible Technologies", May 1989 "Intertemporal prices and the US trade balance in durable goods", July 1989 "Application and evaluation of a multi-criteria decision support system for the dynamic selection of U.S. manufacturing locations", May 1989 "Design flexibility in monopsonistic industries", May 1989 "Requisite variety versus shared values: managing corporate-division relationships in the it-Form organisation", May 1989 "Deposit rate ceiliess and the market value of banks: The case of France 1971-1981", May 1989 "A dispositional approach to social networks: the case of organizational choice", May 1989 "The organisational fool: balancing a leader's hubris", May 1989 "The CEO blues", June 1989 "An. empirical investigation of international asset pricing", (Revised June 1989) "Management systems for innovation and productivity", June 1989 "The strategic supply of precisions", June 1989 "A development fraaevork for computer supported conflict resolution", July 1989 "A note on firing costs and severance benefits in equilibrium unemployment", June 1989 "Strategic adaptation in multi-business firms", June 1989 "Managing expert systems: a framevork and case study", June 1989 89/46 Marcel CORSTJENS, Carmen MATUTES and Damien NEVEN 89/47 Manfred KETS DE VRIES and Christine MEAD "Entry Encouragement", July 1989 "The global dimension in leadership and organization: issues and controversies", April 1989 89/64 Enver YUCESAN and "Complexity of simulation models: A graph (TM) Lee SCHRUBEN theoretic approach", November 1989 89/65 Soumitra DUTTA and "MARS: A mergers and acquisitions reasoning (TM, Piero BONISSONE system", November 1989 AC, PIN) 89/48 Damien NEVEN and Lars-Hendrik ROLLER 89/49 Jean DERMINE 89/50 Jean DERMINE 89/51 Spyros MAKRIDAKIS 89/52 Arnoud DE MEYER 89/53 Spyros MAKRIDAKIS 89/54 S. BALAKRISHNAN and Mitchell KOZA 89/55 H. SCHUTTE 89/56 Vilfried VANHONACKER and Lydia PRICE 89/57 Taekvon KIM, Lars-Hendrik ROLLER and Mihkel TOMBAK 89/58 Lars-Hendrik ROLLER (EP,TM) and Mihkel TOMBAK 89/59 Manfred KETS DE VRIES, (08) Daphna ZEVADI, Alain NOEL and Mihkel TOMBAK 89/60 Enver YUCESAN and (TM) Lee SCHRUBEN 89/61 Susan SCHNEIDER and (All) Arnoud DE MEYER "European integration and trade flows", August 1989 "Home country control and mutual recognition", July 1989 "The specialization of financial institutions, the EEC model", August 1989 "Sliding simulation: a new approach to time series forecasting", July 1989 "Shortening development cycle times: a manufacturer's perspective", August 1989 "Why combining works?", July 1989 "Organisation costs and a theory of joint ventures", September 1989 "Buro-Japanese cooperation in information technology", September 1989 "On the practical usefulness of meta-analysis results", September 1989 "Market growth and the diffusion of multiproduct technologies", September 1989 "Strategic aspects of flexible production technologies", October 1989 "Locus of control and entrepreneurship: a three-country comparative study", October 1989 "Simulation graphs for design and analysis of discrete event simulation models", October 1989 "Interpreting and responding to strategic issues: The impact of national culture", October 1989 "On the regulation of procurement bids", November 1989 "Market microstructure effects of government intervention in the foreign exchange market", December 1989 89/66 B. SINCLAIR-DESGAGNE (TM,RP) 89/67 Peter BOSSAERTS and (PIN) Pierre HILLION 89/62 Arnoud DE MEYER "Technology strategy and international B 6 D (TM) operations", October 1989 89/63 Enver YUCESAN and "Equivalence of simulations: A graph theoretic (TM) Lee SCHRUBEN approach", November 1989 90/16 Richard LEVICH and FIN Ingo WALTER 90/17 Nathalie DIERKENS PIN 90/18 Wilfried VANHONACKER MKT 90/19 Beth JONES and TM Tavfik JELASSI 90/20 Tavfik JELASSI, TM Gregory KERSTEN and Stanley ZIONTS 90/21 Roy SMITH and FIN Ingo WALTER 90/22 Ingo WALTER FIN 90/23 Damien NEVEN 90/24 Lars Tyge NIELSEN 90/25 Lars Tyge NIELSEN 90/26 Charles KADUSHIN and OB/BP Michael BRIBE 90/27 Abbas FOROUGHI and TM Tavfik JELASSI 90/28 Arnoud DE MEYER TM 90/29 Nathalie DIERKENS FIN/AC 90/30 Lars Tyge NIELSEN FIN/EP 90/31 David GAUTSCHI and MKT/EP Roger BETANCOURT 90/32 Srinivasan BALAK- SM RISHNAN and Mitchell KOZA "Tax-Driven Regulatory Drags European Financial Centers in the 1990's", January 1990 "Information Asymmetry and Equity Issues", Revised January 1990 "Managerial Decision Rules and the Estimation of Dynamic Sales Response Models", Revised January 1990 "The Effect of Computer Intervention and Task Structure on Bargaining Outcome", February 1990 "An Introduction to Group Decision and Negotiation Support", February 1990 "Reconfiguration of the Global Securities Industry in the 1990's", February 1990 "European Financial Integration and Its Implications for the United States", February 1990 "EEC Integration towards 1992: Some Distributional Aspects", Revised December 1989 "Positive Prices in CAPE", January 1990 "Existence of Equilibrium in CAPM", January 1990 "Why netvorking Fails: Double Binds and the Limitations of Shadow Networks", February 1990 "NSS Solutions to Major Negotiation Stumbling Blocks", February 1990 • "The Manufacturing Contribution to Innovation", February 1990 "A Discussion of Correct Measures of Information Asymmetry", January 1990 "The Expected Utility of Portfolios of Assets", March 1990 "What Determines U.S. Retail Margins?", February 1990 "Information Asymmetry, Adverse Selection and Joint-Ventures: Theory and Evidence", Revised, January 1990 1990 90/01 B. SINCLAIR-DESGAGNE "Unavoidable Mechanisms", January 1990 TM/EP/AC 90/02 Michael BURDA "Monopolistic Competition, Costs of EP Adjustment, and the Behaviour of European Manufacturing Employment", January 1990 90/03 Arnoud DE MEYER "Management of Communication in International TM Research and Development", January 1990 90/04 Gabriel HAVAVINI and "The Transformation of the European Financial FIN/EP Eric RAJENDRA Services Industry: From Fragmentation to Integration", January 1990 90/05 Gabriel HAVAVINI and, "European Equity Markets: Toward 1992 and FIN/EP Bertrand JACOUILLAT Beyond", January 1990 90/06 Gabriel HAVAVINI and "Integration of European Equity Markets: FIN/EP Eric RAJENDRA Implications of Structural Change for Key Market Participants to and Beyond 1992", January 1990 90/07 Gabriel HAVAVINI "Stock Market Anomalies and the Pricing of FIN/EP Equity on the Tokyo Stock Exchange", January 1990 90/08 Tavfik JELASSI and "Modelling vith MCDSS: What about Ethics'', TM/EP B. SINCLAIR-DESGACNE January 1990 90/09 Alberto CIOVANNINI "Capital Controls and International Trade EP/FIN and Jae VON PARK Finance", January 1990 90/10 Joyce BRYER and "The Impact of Language Theories on 055 TM Tavfik JELASSI Dialog", January 1990 90/11 Enver YUCESAN "An Overview of Frequency Domain Methodology TM for Simulation Sensitivity Analysis', January 1990 90/12 Michael BURDA "Structural Change, Unemployment Benefits and El nigh Unemployment: A U.S.-European Conoarison", January 1990 90/13 Soumitra DUTTA and "Approximate Reasoning about Temporal TM Shashi SHEKHAR Constraints in Real Time Planning and Search", January 1990 90/14 Albert ANGEHRN and "Visual Interactive Modelling and Intelligent TM Bans-Jakob LOTHI DSS: Putting Theory Into Practice", January 1990 90/15 Arnoud DE MEYER, "The Internal Technological Renewal of a TM Dirk DESCHOOLMEESTER, Business Unit with a Mature Technology", Rudy MOENAERT and January 1990 Jan BARBE EP/SM FIN/EP FIN/EP 90/33 Caren SIEHL, 08 David BOWEN and Christine PEARSON "The Role of Rites of Integration in Service Delivery", March 1990 90/34 Jean DERMINE "The Gains fro. European Banking Integration, PIN/EP a Call for a Pro-Active Competition Policy", April 1990 90/35 Jae Von PARK "Changing Uncertainty and the Time-Varying EP Risk Premia in the Term Structure of Nominal Interest Rates", December 1988, Revised March 1990 90/36 Arnoud DE MEYER "An Empirical Investigation of Manufacturing TM Strategies in European Industry", April 1990 90/37 William CATS-BARIL "Executive Information Systems: Developing an TM/08/SM Approach to Open the Possibles", April 1990 90/38 Wilfried VANHONACKER "Managerial Decision Behaviour and the MKT Estimation of Dynamic Sales Response Models", (Revised February 1990) Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Page 8 Page 9 Page 10 Page 11 Page 12 Page 13 Page 14 Page 15 Page 16 Page 17 Page 18 Page 19 Page 20 Page 21 Page 22 Page 23 Page 24 Page 25 Page 26 Page 27 Page 28 Page 29 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Page 36 Page 37 Page 38 Page 39 Page 40 Page 41