Publications


Papers on EXPECT

Papers on Planning

Papers on Knowledge Acquisition

Papers on Intelligent Agents

Papers on Ontologies and Problem-Solving Methods

Papers on Proactive Dialogue

Papers on Machine Learning


Papers on EXPECT


Jim Blythe, Jihie Kim, Surya Ramachandran and Yolanda Gil. "An Integrated Environment for Knowledge Acquisition". Best Paper, International Conference on Intelligent User Interfaces 2001 (
PDF file)

Abstract: This paper describes an integrated acquisition interface that includes several techniques previously developed to support users in various ways as they add new knowledge to an intelligent system. As a result of this integration, the individual techniques can take better advantage of the context in which they are invoked and provide stronger guidance to users. We describe the current implementation using examples from a travel planning domain, and demonstrate how users can add complex knowledge to the system.


Yolanda Gil, Jim Blythe, Jihie Kim and Surya Ramachandran. "Acquiring Procedural Knowledge in EXPECT". In AAAI 2000 fall symposium on Learning How to Do Things. ( PDF file)

Abstract: The EXPECT project has focused on acquiring problem-solving knowledge for users for the last decade, using an expressive language that is open to inspection. Our aim has been to alleviate the bottleneck in creating knowledge-based systems by providing support for both knowledge engineers and end users to specify problem-solving knowledge. In this paper we summarize selected areas of current research where we focus on end users, and highlight some of the research questions that arise from them. We also summarize some of the results from experiments with end users using our tools.


Yolanda Gil and Jim Blythe. "How Can a Structured Representation of Capabilities Help in Planning?". In AAAI 2000 workshop on Representational Issues for Real-world Planning Systems ( PDF file)

Abstract: In order to support a wide range of planning-related activities, we argue that plan and action representations must move to a more expressive language for goals and capabilities than is found in most current systems. A structured representation for capabilities can make explicit a hierarchy of capabilities based on subsumption, resulting in benefits for reasoning, representing, and acquiring operators and plans. By making capabilities more easily understandable to humans, such a representation can also benefit mixed-initiative approaches. We present a structured representation of capabilities and a subsumption-based matcher for it. We then describe three existing systems that use this approach in different kinds of planning tasks and tools. We finish with a discussion of how plan generation systems can benefit from using this representation.


Marcelo Tallis. "A Script-Based Approach to Modifying Knowledge-Based Systems". To appear in the International Journal of Human-Computer Studies (PDF file )

Abstract: Modifying knowledge-based systems is a complex activity. One of its difficulties is that several related portions of the system might have to be changed in order to maintain the coherence of the system. However, it is difficult for users to figure out what has to be changed and how. This paper presents a novel approach for building knowledge acquisition tools that overcomes some of the limitations of current approaches. In this approach, knowledge of prototypical procedures for modifying knowledge-based systems is used to guide users in changing all related portions of a system. These procedures, which we call knowledge acquisition scripts (or KA Scripts), capture how related portions of a knowledge-based system can be changed coordinately. By using KA scripts, a knowledge acquisition tool would be able to relate individual changes in different parts of a system, enabling the analysis of each individual change from the perspective of the overall modification. The paper also describes the implementation of ETM: a script-based tool that builds on the EXPECT framework for building knowledge-based systems, discusses how we have addressed some important issues of this approach, and describes a preliminary empirical evaluation of ETM that shows that KA Scripts allow users to perform knowledge-based systems modification tasks more efficiently.


Jihie Kim and Yolanda Gil. "Acquiring Problem-Solving Knowledge from End Users: Putting Interdependency Models to the Test". Proceedings of AAAI-2000, pp. 223-229. (PDF file )

Abstract: Developing tools that allow non-programmers to enter knowledge has been an ongoing challenge for AI. In recent years researchers have investigated a variety of promising approaches to knowledge acquisition (KA), but they have often been driven by the needs of knowledge engineers rather than by end users. This paper reports on a series of experiments that we conducted in order to understand how far a particular KA tool that we are developing is from meeting the needs of end users, and to collect valuable feedback to motivate our future research. This KA tool, called EMeD, exploits Interdependency Models derived from a knowledge base in order to guide users in specifying problem-solving knowledge. One of the challenges of this work is to devise a methodology and experimental procedure for conducting user studies of knowledge acquisition tools. We describe how our experiments helped us address several questions and hypotheses regarding the acquisition of problem-solving knowledge from end users and the benefits of Interdependency Models, and discuss what we learned in terms of improving not only our KA tools but also about KA research.


Jihie Kim and Yolanda Gil. "User Studies of an Interdependency-Based Interface for Acquiring Problem-Solving Knowledge". Proceedings of the Intelligent User Interface Conference, pp. 165-168. (IUI-2000) (PDF file )

Abstract: This paper describes a series of experiments with a range of users to evaluate an intelligent interface for acquiring problem-solving knowledge to describe how to accomplish a task. The tool derives the interdependencies between different pieces of knowledge in the system and uses them to guide the user in completing the acquisition task. The paper describes results obtained when the tool was tested with a wide range of users, including end users. The studies show that our acquisition interface saves users an average of 32% of the time it takes to add new knowledge, and highlight some interesting differences across user groups. The paper also describes what are the areas that need to be addressed in future research in order to make these tools usable by end users.


William Swartout, Yolanda Gil, and Andre Valente. "Representing Capabilities of Problem-Solving Methods". Proceedings of 1999 IJCAI Workshop on Ontologies and Problem-Solving Methods.

(PDF file )


Marcelo Tallis and Yolanda Gil. "Designing Scripts to Guide Users in Modifying Knowledge-Based Systems". Proceedings of AAAI-99.(PDF file )

Abstract: Knowledge Acquisition (KA) Scripts capture typical modification sequences that users follow when they modify knowledge bases. KA tools can use these Scripts to guide users in making these modifications, ensuring that they follow all the ramifications of the change until it is completed. This paper describes our approach to design, develop, and organize a library of KA Scripts. We report the results of three different analysis to develop this library, including a detailed study of actual modification scenarios in two knowledge bases. In addition to identifying a good number of KA Scripts, we found a set of useful attributes to describe and organize the KA Scripts. These attributes allow us to analyze the size of the library and generate new KA Scripts in a systematic way. We have implemented a portion of this library and conducted two different studies to evaluate it. The result of this evaluation showed a 15 to 52 percent time savings in modifying knowledge bases and that the library included relevant and useful KA Scripts to assist users in realistic settings.


Jihie Kim and Yolanda Gil. "Deriving Expectations to Guide Knowledge Base Creation". Proceedings of AAAI-99, pp. 235-241. (PDF file)

Abstract: Successful approaches to developing knowledge acquisition tools use expectations of what the user has to add or may want to add, based on how new knowledge fits within a knowledge base that already exists. When a knowledge base is first created or undergoes significant extensions and changes, these tools cannot provide much support. This paper presents an approach to creating expectations when a new knowledge base is built, and describes a knowledge acquisition tool that we implemented using this approach that supports users in creating problem-solving knowledge. As the knowledge base grows, the knowledge acquisition tool derives more frequent and more reliable expectations that result from enforcing constraints in the knowledge representation system, looking for missing pieces of knowledge in the knowledge base, and working out incrementally the inter-dependencies among the different components of the knowledge base. Our preliminary evaluations show a thirty percent time savings during knowledge acquisition. Moreover, by providing tools to support the initial phases of knowledge base development, many mistakes are detected early on and even avoided altogether. We believe that our approach contributes to improving the quality of the knowledge acquisition process and of the resulting knowledge-based systems as well.


Jim Blythe and Surya Ramachandran. "Knowledge Acquisition using an English-Based Method Editor". Proceedings of the Tenth Banff Knowledge Acquisition for Knowledge-Based Systems Workshop (KAW-99). (PDF file)

Abstract: We describe an editor for problem-solving knowledge that communicates with the user through English paraphrases of the knowledge. Although it does not support the full range of modifications one might want to make, the value of the tool lies in the fact that the user need not understand the syntax of the expert system to make modifications. By analyzing the problem-solving knowledge, the tool can allow the user to select semantically coherent chunks of the knowledge. It then presents English paraphrases of possible substitutions that would result in new problem-solving knowledge that is syntactically correct. In this way the tool expands the range of modifications that a naive user can make to problem-solving knowledge in an expert system.


Marcelo Tallis. "A Script-Based Approach to Modifying Knowledge-Based Systems". Procueedings of the Ninth Banff Knowledge Acquisition for Knowledge-Based Systems Workshop (KAW-98), Banff, Alberta, Canada, April 1998. (PDF file or Html )

Abstract: Modifying knowledge-based systems (KBSs) is a complex activity. One of its difficulties is that several related portions of the KBS might have to be changed in order to maintain the coherence of the system. However, it is difficult for users to figure out what has to be changed and how. This paper presents a novel approach for building knowledge acquisition (KA) tools that overcomes some of the limitations of current approaches. In this approach, knowledge of prototypical procedures for modifying KBSs is used to guide users in changing all related portions of a KBS. These procedures, which we call knowledge acquisition scripts (or KA Scripts), capture how related portions of a KBS can be changed coordinately. By using KA scripts, a KA tool would be able to relate individual changes in different parts of a KBS, enabling the analysis of each individual change from the perspective of the overall modification. The paper also describes the implementation of ETM: a script-based tool that builds on the EXPECT framework for building KBSs (Gil, 1994), discusses how we have addressed some important issues of this approach, and describes a preliminary empirical evaluation of ETM that shows that KA Scripts allow users to perform KBSs modification tasks more efficiently.


Yolanda Gil and Marcelo Tallis. "A Script-Based Approach to Modifying Knowledge Bases". Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), Providence, RI, July 27-31, 1997. (PDF file )

Abstract: Our goal is to build knowledge acquisition tools that support users in modifying knowledge-based systems. These modifications may require several individual changes to various components of the knowledge base, which need to be carefully coordinated to prevent users from leaving the knowledge-based system in an unusable state. This paper describes an approach to building knowledge acquisition tools which capture knowledge about commonly occurring modification sequences and support users in completing the modifications they start. These sequences, which we call KA Scripts, relate individual changes and the effects that they have on the knowledge base. We discuss our experience in designing and compiling a library of KA Scripts. We also describe the implementation of a tool that uses them and our preliminary evaluations that demonstrate their usability.


Yolanda Gil and Eric Melz. "Explicit Representations of Problem-Solving Strategies to Support Knowledge Acquisition". Proceedings of the Thirteen National Conference on Artificial Intelligence (AAAI-96), Portland, OR, August 4-8, 1996. (PDF file )

Abstract: Role-limiting approaches support knowledge acquisition (KA) by centering knowledge base construction on common types of tasks or domain-independent problem-solving strategies. Within a particular problem-solving strategy, domain-dependent knowledge plays specific roles. A KA tool then helps a user to fill these roles. Although role-limiting approaches are useful for guiding KA, they are limited because they only support users in filling knowledge roles that have been built in by the designers of the KA system. EXPECT takes a different approach to KA by representing problem-solving knowledge explicitly, and deriving from the current knowledge base the knowledge gaps that must be resolved by the user during KA. This paper contrasts role-limiting approaches and EXPECT's approach, using the propose-and-revise strategy as an example. EXPECT not only supports users in filling knowledge roles, but also provides support in 1) adapting the problem-solving strategy, 2) changing the types of information to be acquired about a knowledge role, 3) adding new knowledge roles, and 4) acquiring additional background information about the domain needed by the knowledge-based system. EXPECT's guidance changes as the knowledge base changes, providing a more flexible approach to knowledge acquisition. This work provides evidence supporting the need for explicit representations in building knowledge-based systems.


William R. Swartout and Yolanda Gil. "EXPECT: A User-Centered Environment for the Development and Adaptation of Knowledge-Based Planning Aids". In Advanced Planning Technology: Technological Achievements of the ARPA/Rome Laboratory Planning Initiative, ed. Austin Tate. Menlo Park, Calif.: AAAI Press, 1996. (PDF file)

Abstract: EXPECT provides an environment for developing knowledge-based systems that allows end-users to add new knowledge without needing to understand the details of system organization and implementation. The key to EXPECT's approach is that it understands the structure of the knowledge-based system being built: how it solves problems and what knowledge it needs to support problem-solving. EXPECT uses this information to guide users in maintaining the knowledge-based system. We have used EXPECT to develop a tool for evaluating transportation plans.


Bill Swartout and Yolanda Gil. "Flexible Knowledge Acquisition Through Explicit Representation of Knowledge Roles". 1996 AAAI Spring Symposium on Acquisition, Learning, and Demonstration: Automating Tasks for Users, Stanford, CA, March 1996. (PDF file )

Abstract: A system that acquires knowledge from a user should be able to reflect upon the knowledge that it has - at each moment - and understand what kinds of new knowledge it needs to learn. For the past two decades, research in the area of knowledge acquisition has been moving towards systems that have access to richer representations of knowledge about their task. This paper reviews some well-known knowledge acquisition tools representative of this trend. It also describes our recent work in EXPECT, a system with explicit representations of knowledge about the task and the domain that supports knowledge acquisition for a wider range of tasks and applications than its predecessors. We hope our observations will be useful to researchers in user interfaces and in machine learning concerned with acquiring information from users.


Yolanda Gil and Marcelo Tallis. "Transaction-Based Knowledge Acquisition: Complex Modifications Made Easier". In Proceedings of the Ninth Knowledge Acquisition for Knowledge-Based Systems Workshop, February 26-March 3, 1995. Banff, Alberta, Canada. (PDF file)

Abstract: Our goal is to build knowledge acquisition tools that support users in making a broad range of changes to a knowledge base, including both factual and problem-solving knowledge. These changes may require several individual modifications to various parts of the knowledge base, that need to be carefully coordinated to prevent users from introducing errors in the knowledge base. Thus, it becomes essential that our KA tools understand the consequences of each kind of change that the user may initiate, detect any harmful side-effects that can be introduced in the system, and guide the user in resolving them. To address this issue, we have developed a transaction-based approach to knowledge acquisition that can support users in making complex modifications to a knowledge base. A transaction is a sequence of changes that together modify some aspect of the behavior of a knowledge-based system, and that when only partially carried out may leave the knowledge base in an undesirable state. If a user executes a transaction partially, the knowledge acquisition tool must provide guidance to finish it and support the user in achieving the desired modification. This paper also describes our work in extending EXPECT's knowledge acquisition tool to support transaction-based mechanisms. EXPECT tracks the possible problems that arise as a consequence of each individual change to the knowledge base, keeps information about the context of each change, and uses this context to resolve the problems detected and to request the user's intervention if additional information is needed.


Bill Swartout and Yolanda Gil. "EXPECT: Explicit Representations for Flexible Acquisition". In Proceedings of the Ninth Knowledge Acquisition for Knowledge-Based Systems Workshop, February 26-March 3, 1995. Banff, Alberta, Canada. (PDF file)

Abstract: To create more powerful knowledge acquisition systems, we not only need better acquisition tools, but we need to change the architecture of the knowledge based systems we create so that their structure will provide better support for acquisition. Current acquisition tools permit users to modify factual knowledge but they provide limited support for modifying problem solving knowledge. In this paper, we argue that this limitation (and others) stem from the use of incomplete models of problem solving knowledge and inflexible specification of the interdependencies between problem solving and factual knowledge. We describe the EXPECT architecture which addresses these problems by providing an explicit representation for problem solving knowledge and intent. Using this more explicit representation, EXPECT can automatically derive the interdependencies between problem solving and factual knowledge. By deriving these interdependencies from the structure of the system itself EXPECT supports more flexible and powerful knowledge acquisition.


Yolanda Gil and Cecile Paris. "Towards Method-Independent Knowledge Acquisition". Knowledge Acquisition, Special issue on Machine Learning and Knowledge Acquisition, Volume 6 Number 2, June 1994. (PDF file)

Abstract: Rapid prototyping and tool reusability have pushed knowledge acquisition research to investigate method-specific knowledge acquisition tools appropriate for predetermined problem-solving methods. We believe that method-dependent knowledge acquisition is not the only approach. The aim of our research is to develop powerful yet versatile machine learning mechanisms that can be incorporated into general-purpose but practical knowledge acquisition tools. This paper shows through examples the practical advantages of this approach. In particular, we illustrate how existing knowledge can be used to facilitate knowledge acquisition through analogy mechanisms within a domain and across domains. Our sample knowledge acquisition dialogues with a domain expert illustrate which parts of the process are addressed by the human and which parts are automated by the tool, in a synergistic cooperation for knowledge-base extension and refinement. The paper also describes briefly the EXPECT problem-solving architecture that facilitates this approach to knowledge acquisition.


Yolanda Gil. "Knowledge Refinement in a Reflective Architecture". Proceedings of the Twelfth National Conference of Artificial Intelligence (AAAI-94), Seattle, WA, August 1994. (PDF file)

Abstract: A knowledge acquisition tool should provide a user with maximum guidance in extending and debugging a knowledge base, by preventing inconsistencies and knowledge gaps that may arise inadvertently. Most current acquisition tools are not very flexible in that they are built for a predetermined inference structure or problem-solving mechanism, and the guidance they provide is specific to that inference structure and hard-coded by their designer. This paper focuses on EXPECT, a reflective architecture that supports knowledge acquisition based on an explicit analysis of the structure of a knowledge-based system, rather than on a fixed set of acquisition guidelines. EXPECT's problem solver is tightly integrated with LOOM, a state-of-the-art knowledge representation system. Domain facts and goals are represented declaratively, and the problem solver keeps records of their functionality within the task domain. When the user corrects the system's knowledge, EXPECT tracks any possible implications of this change in the overall system and cooperates with the user to correct any potential problems that may arise. The key to the flexibility of this knowledge acquisition tool is that it adapts its guidance as the knowledge bases evolve in response to changes introduced by the user.


About EXPECT | Research | Projects | Project Members | Publications | Movie |
| Intelligent Systems Division | Information Sciences Institute | USC |