Publications

Papers on EXPECT

Abstract: This paper describes an integrated acquisition interface that includes several techniques previously developed to support users in various ways as they add new knowledge to an intelligent system. As a result of this integration, the individual techniques can take better advantage of the context in which they are invoked and provide stronger guidance to users. We describe the current implementation using examples from a travel planning domain, and demonstrate how users can add complex knowledge to the system.

Abstract: The EXPECT project has focused on acquiring problem-solving knowledge for users for the last decade, using an expressive language that is open to inspection. Our aim has been to alleviate the bottleneck in creating knowledge-based systems by providing support for both knowledge engineers and end users to specify problem-solving knowledge. In this paper we summarize selected areas of current research where we focus on end users, and highlight some of the research questions that arise from them. We also summarize some of the results from experiments with end users using our tools.

Abstract: In order to support a wide range of planning-related activities, we argue that plan and action representations must move to a more expressive language for goals and capabilities than is found in most current systems. A structured representation for capabilities can make explicit a hierarchy of capabilities based on subsumption, resulting in benefits for reasoning, representing, and acquiring operators and plans. By making capabilities more easily understandable to humans, such a representation can also benefit mixed-initiative approaches. We present a structured representation of capabilities and a subsumption-based matcher for it. We then describe three existing systems that use this approach in different kinds of planning tasks and tools. We finish with a discussion of how plan generation systems can benefit from using this representation.

Abstract: Modifying knowledge-based systems is a complex activity. One of its difficulties is that several related portions of the system might have to be changed in order to maintain the coherence of the system. However, it is difficult for users to figure out what has to be changed and how. This paper presents a novel approach for building knowledge acquisition tools that overcomes some of the limitations of current approaches. In this approach, knowledge of prototypical procedures for modifying knowledge-based systems is used to guide users in changing all related portions of a system. These procedures, which we call knowledge acquisition scripts (or KA Scripts), capture how related portions of a knowledge-based system can be changed coordinately. By using KA scripts, a knowledge acquisition tool would be able to relate individual changes in different parts of a system, enabling the analysis of each individual change from the perspective of the overall modification. The paper also describes the implementation of ETM: a script-based tool that builds on the EXPECT framework for building knowledge-based systems, discusses how we have addressed some important issues of this approach, and describes a preliminary empirical evaluation of ETM that shows that KA Scripts allow users to perform knowledge-based systems modification tasks more efficiently.

Abstract: Developing tools that allow non-programmers to enter knowledge has been an ongoing challenge for AI. In recent years researchers have investigated a variety of promising approaches to knowledge acquisition (KA), but they have often been driven by the needs of knowledge engineers rather than by end users. This paper reports on a series of experiments that we conducted in order to understand how far a particular KA tool that we are developing is from meeting the needs of end users, and to collect valuable feedback to motivate our future research. This KA tool, called EMeD, exploits Interdependency Models derived from a knowledge base in order to guide users in specifying problem-solving knowledge. One of the challenges of this work is to devise a methodology and experimental procedure for conducting user studies of knowledge acquisition tools. We describe how our experiments helped us address several questions and hypotheses regarding the acquisition of problem-solving knowledge from end users and the benefits of Interdependency Models, and discuss what we learned in terms of improving not only our KA tools but also about KA research.

Abstract: This paper describes a series of experiments with a range of users to evaluate an intelligent interface for acquiring problem-solving knowledge to describe how to accomplish a task. The tool derives the interdependencies between different pieces of knowledge in the system and uses them to guide the user in completing the acquisition task. The paper describes results obtained when the tool was tested with a wide range of users, including end users. The studies show that our acquisition interface saves users an average of 32% of the time it takes to add new knowledge, and highlight some interesting differences across user groups. The paper also describes what are the areas that need to be addressed in future research in order to make these tools usable by end users.

Abstract: Modifying knowledge-based systems (KBSs) is a complex activity. One of its difficulties is that several related portions of the KBS might have to be changed in order to maintain the coherence of the system. However, it is difficult for users to figure out what has to be changed and how. This paper presents a novel approach for building knowledge acquisition (KA) tools that overcomes some of the limitations of current approaches. In this approach, knowledge of prototypical procedures for modifying KBSs is used to guide users in changing all related portions of a KBS. These procedures, which we call knowledge acquisition scripts (or KA Scripts), capture how related portions of a KBS can be changed coordinately. By using KA scripts, a KA tool would be able to relate individual changes in different parts of a KBS, enabling the analysis of each individual change from the perspective of the overall modification. The paper also describes the implementation of ETM: a script-based tool that builds on the EXPECT framework for building KBSs (Gil, 1994), discusses how we have addressed some important issues of this approach, and describes a preliminary empirical evaluation of ETM that shows that KA Scripts allow users to perform KBSs modification tasks more efficiently.

Abstract: Our goal is to build knowledge acquisition tools that support users in modifying knowledge-based systems. These modifications may require several individual changes to various components of the knowledge base, which need to be carefully coordinated to prevent users from leaving the knowledge-based system in an unusable state. This paper describes an approach to building knowledge acquisition tools which capture knowledge about commonly occurring modification sequences and support users in completing the modifications they start. These sequences, which we call KA Scripts, relate individual changes and the effects that they have on the knowledge base. We discuss our experience in designing and compiling a library of KA Scripts. We also describe the implementation of a tool that uses them and our preliminary evaluations that demonstrate their usability.

Abstract: Role-limiting approaches support knowledge acquisition (KA) by centering knowledge base construction on common types of tasks or domain-independent problem-solving strategies. Within a particular problem-solving strategy, domain-dependent knowledge plays specific roles. A KA tool then helps a user to fill these roles. Although role-limiting approaches are useful for guiding KA, they are limited because they only support users in filling knowledge roles that have been built in by the designers of the KA system. EXPECT takes a different approach to KA by representing problem-solving knowledge explicitly, and deriving from the current knowledge base the knowledge gaps that must be resolved by the user during KA. This paper contrasts role-limiting approaches and EXPECT's approach, using the propose-and-revise strategy as an example. EXPECT not only supports users in filling knowledge roles, but also provides support in 1) adapting the problem-solving strategy, 2) changing the types of information to be acquired about a knowledge role, 3) adding new knowledge roles, and 4) acquiring additional background information about the domain needed by the knowledge-based system. EXPECT's guidance changes as the knowledge base changes, providing a more flexible approach to knowledge acquisition. This work provides evidence supporting the need for explicit representations in building knowledge-based systems.

Abstract: EXPECT provides an environment for developing knowledge-based systems that allows end-users to add new knowledge without needing to understand the details of system organization and implementation. The key to EXPECT's approach is that it understands the structure of the knowledge-based system being built: how it solves problems and what knowledge it needs to support problem-solving. EXPECT uses this information to guide users in maintaining the knowledge-based system. We have used EXPECT to develop a tool for evaluating transportation plans.

Abstract: A system that acquires knowledge from a user should be able to reflect upon the knowledge that it has - at each moment - and understand what kinds of new knowledge it needs to learn. For the past two decades, research in the area of knowledge acquisition has been moving towards systems that have access to richer representations of knowledge about their task. This paper reviews some well-known knowledge acquisition tools representative of this trend. It also describes our recent work in EXPECT, a system with explicit representations of knowledge about the task and the domain that supports knowledge acquisition for a wider range of tasks and applications than its predecessors. We hope our observations will be useful to researchers in user interfaces and in machine learning concerned with acquiring information from users.

Abstract: Our goal is to build knowledge acquisition tools that support users in making a broad range of changes to a knowledge base, including both factual and problem-solving knowledge. These changes may require several individual modifications to various parts of the knowledge base, that need to be carefully coordinated to prevent users from introducing errors in the knowledge base. Thus, it becomes essential that our KA tools understand the consequences of each kind of change that the user may initiate, detect any harmful side-effects that can be introduced in the system, and guide the user in resolving them. To address this issue, we have developed a transaction-based approach to knowledge acquisition that can support users in making complex modifications to a knowledge base. A transaction is a sequence of changes that together modify some aspect of the behavior of a knowledge-based system, and that when only partially carried out may leave the knowledge base in an undesirable state. If a user executes a transaction partially, the knowledge acquisition tool must provide guidance to finish it and support the user in achieving the desired modification. This paper also describes our work in extending EXPECT's knowledge acquisition tool to support transaction-based mechanisms. EXPECT tracks the possible problems that arise as a consequence of each individual change to the knowledge base, keeps information about the context of each change, and uses this context to resolve the problems detected and to request the user's intervention if additional information is needed.

Abstract: To create more powerful knowledge acquisition systems, we not only need better acquisition tools, but we need to change the architecture of the knowledge based systems we create so that their structure will provide better support for acquisition. Current acquisition tools permit users to modify factual knowledge but they provide limited support for modifying problem solving knowledge. In this paper, we argue that this limitation (and others) stem from the use of incomplete models of problem solving knowledge and inflexible specification of the interdependencies between problem solving and factual knowledge. We describe the EXPECT architecture which addresses these problems by providing an explicit representation for problem solving knowledge and intent. Using this more explicit representation, EXPECT can automatically derive the interdependencies between problem solving and factual knowledge. By deriving these interdependencies from the structure of the system itself EXPECT supports more flexible and powerful knowledge acquisition.

Abstract: Rapid prototyping and tool reusability have pushed knowledge acquisition research to investigate method-specific knowledge acquisition tools appropriate for predetermined problem-solving methods. We believe that method-dependent knowledge acquisition is not the only approach. The aim of our research is to develop powerful yet versatile machine learning mechanisms that can be incorporated into general-purpose but practical knowledge acquisition tools. This paper shows through examples the practical advantages of this approach. In particular, we illustrate how existing knowledge can be used to facilitate knowledge acquisition through analogy mechanisms within a domain and across domains. Our sample knowledge acquisition dialogues with a domain expert illustrate which parts of the process are addressed by the human and which parts are automated by the tool, in a synergistic cooperation for knowledge-base extension and refinement. The paper also describes briefly the EXPECT problem-solving architecture that facilitates this approach to knowledge acquisition.

Abstract: A knowledge acquisition tool should provide a user with maximum guidance in extending and debugging a knowledge base, by preventing inconsistencies and knowledge gaps that may arise inadvertently. Most current acquisition tools are not very flexible in that they are built for a predetermined inference structure or problem-solving mechanism, and the guidance they provide is specific to that inference structure and hard-coded by their designer. This paper focuses on EXPECT, a reflective architecture that supports knowledge acquisition based on an explicit analysis of the structure of a knowledge-based system, rather than on a fixed set of acquisition guidelines. EXPECT's problem solver is tightly integrated with LOOM, a state-of-the-art knowledge representation system. Domain facts and goals are represented declaratively, and the problem solver keeps records of their functionality within the task domain. When the user corrects the system's knowledge, EXPECT tracks any possible implications of this change in the overall system and cooperates with the user to correct any potential problems that may arise. The key to the flexibility of this knowledge acquisition tool is that it adapts its guidance as the knowledge bases evolve in response to changes introduced by the user.

About EXPECT | Research | Projects | Project Members | Publications | Movie |
| Intelligent Systems Division | Information Sciences Institute | USC |