This file is part of:

	AT&T Bell Laboratories and University of Pittsburgh
	CLASSIC Knowledge Representation System Tutorial

	Copyright AT&T and University of Pittsburgh 1994


			  ASG5.TXT

			PRELIMINARIES

Open a new log file:
>>> (dribble "your-file5.log")

The knowledge base that we will work with is essentially the one at
the start of assignment 4.  In order to make it easier to peruse what
was loaded in, this has all been merged into one file.  This new file
is basically just "Recover/after-just-wines.lisp", expanded to include
all the files it loaded.  We also added the rules in
"foods-wines.lisp".  The entire file is now in
"expanded-wines-kb.lisp".  (There are some slight modifications in
this file, so be sure to use the file "expanded-wines-kb.lisp" for
this exercise, rather than "after-just-wines.lisp".)  Load this file:

>>> (load "<topdir>/Customize.lisp")
>>> (lload "Recover/expanded-wines-kb.lisp")


REVIEW OF asg1.txt through asg4.txt

In the first module of this assignment we covered:

 1. How to create "disjoint-primitive" concepts.  Using this construct
    we created the concepts WINE-PROPERTY, CONSUMABLE-THING,
    WINERY, and WINE-REGION. We also constructed the concepts
    EDIBLE-THING, POTABLE-LIQUID, WINE-COLOR, WINE-BODY,
    WINE-SUGAR, WINE-FLAVOR, WINE and MEAL-FOOD.

 2. How to get verbose or laconic debugging information from Classic when
    concepts and individuals are defined.

 3. Commands for inspecting and navigating the KB.

 4. Definition of roles and attributes.

 5. Creation of individuals and the effect of role restrictions
    on individuals.

In the second module of this assignment we covered:

 6. Adding concepts to the KB in such a way that existing concepts and
    instances get reclassified under them (we illustrated this with
    REGIONAL-WINERY and its children).

 7. Use of the inverse of a role.

In the third module of this assignment we covered:

 8. Using rules in Classic: adding and removing rules.
    The main use of rules, i.e. for adding non-definitional info to
    instances that get classified under a concept.

In the fourth module of this assignment we covered:

 9. More use of rules: using rules to create disjunctive concepts,
    query templates, and circular definitions.

10. Using tests.

11. "Same-as" restrictions.

You may want to review some of these topics.

The purpose of this assignment is to investigate some of the issues
involved with debugging and maintaining knowledge bases.  While we are
primarily interested in your learning the material, we are also
interested in how easy or hard it is for you to learn.  There are 5
times where you will be asked to time yourself while doing a task.
You will not be graded on your times.  Please be accurate on times.
Please do make sure to include times.  If by some chance you forget to
time something, make an estimate and note that it was an estimate.
You will also be asked to write notes in your dribble files at places.
You can either precede each line with a ; so that LISP does not try to
evaluate your statements, or you can surround a block of text with 

	#| lots of text |# 

There is a 3 part question on one step explanations, a 4 part question
on complete explanations of error situations, a 4 part question on
pruning settings, and an extra credit question.

		USING, DEBUGGING and MAINTAINING KNOWLEDGE BASES
		------------------------------------------------

In order to effectively develop, use, and extend knowledge bases, we
need to be able predict the results of additions and queries.  Thus,
we should understand the inferences that the system makes.  We will
first explore the use of explanation functions to aid in this process.

The last assignment (asg4) suggested that you create My-Meal-Food and
explored what was true of the associated wine.  We will now go into a bit more
detail about some of the inferences CLASSIC makes.  Let's investigate
how some of the foods in the knowledge base got their restrictions on
their appropriate-drink role.  

In asg4, you looked at the restrictions on spicy-red-meat foods.
There are two ways to do this: you could just pick one of the
SPICY-RED-MEAT foods in the knowledge base like Beef-curry and examine
that or you could create a new SPICY-RED-MEAT food and look at it.  Let's
define the same SPICY-RED-MEAT food again, and examine it through the
explanation functions.

Define @My-meal-food again as:

>>> (cl-create-ind 'My-Meal-Food
                   '(AND SPICY-RED-MEAT
			(FILLS appropriate-drink Some-Wine)))

Let's investigate Some-Wine again.  First print it:

>>>(cl-print-ind @some-wine)

If we are interested in recommending wines with foods, we would want
to figure out why some wine has certain fillers and restrictions for the
sugar, flavor, body, and color roles.  We could ask for a general
explanation of the food as follows: 

>>>(cl-exp @some-wine) 

However, since the four roles, which we are interested in, have specific
fillers, we can ask specifically how the fillers got there by typing:

>>>(cl-exp @some-wine :aspect 'fills)

We see that almost everything of interest on Some-Wine came from a
propagation on My-Meal-Food because of restrictions on its
appropriate-drink role.  The restrictions on My-Meal-Food's
appropriate-drink role influence Some-Wine because Some-Wine is known
to fill My-Meal-Food's appropriate-drink role, and thus it must
satisfy all the accompanying restrictions.  The next question is how
those restrictions got there at all.  To use explanation to determine
that, ask

>>> (cl-exp @my-meal-food :role-path @appropriate-drink)

Now we see that rule firings account for all the fills restrictions on color, body,
flavor, and sugar.  So, any spicy red meat food, such as Beef-Curry,
should have a Red-Wine as its appropriate-drink because of the
red-meat-wine-color rule, a Full-Wine because of the
spicy-red-meat-wine-body rule, a Moderate-Wine because of the
spicy-red-meat-wine-flavor rule, and a Dry-Wine because of the
meat-wine-flavor rule.

Above, we went through a two step explanation, first explaining
Some-Wine and then explaining a particular role on My-Meal-Food.  In
this case, we could have actually simplified this process by just
choosing a food, like Beef-Curry, and explaining the restrictions on
its appropriate-drink role.

>>> (cl-exp @beef-curry :role-path @appropriate-drink)

Now, let's look at another class of food in the knowledge base.
We will consider things that fall under Non-Oyster-Shellfish.
Consider Lobster, and suppose that we are interested in the
appropriate-drink's color, body, flavor, and sugar roles.  Don't
bother to create a new food and associate a wine with it, just use
Lobster.

Now we will time a task.

 Explanation 1.
***START TIMING HERE.  
Put a ; in your dribble file and note the time.  Now use cl-exp on
Lobster to answer why its appropriate-drink role has each restriction
on the following roles: color, body, flavor, and sugar.

***STOP TIMING HERE.  
Note the time that it took you to do this task in your dribble file
(preceded by a ;)

We are going to time another task.  This time please DO NOT use the
explanation commands.  On another food, Fettuccine-Alfredo, please
explain the restrictions on its appropriate-drink's color, body,
flavor, and sugar roles.  Make sure you time this task.  You can use
whatever CLASSIC commands you like.  (You might consider just printing
the food and then looking at the file you loaded to discover what
rules, propagations, etc.  could have applied.)

Explanation 2.
**** PLEASE START TIMING NOW
Put a ; and the time in your dribble file.  Put your answer in the
dribble file (starting each line with a ; or surrounding the whole
answer by #| and |# )

*** STOP TIMING HERE.  
Please note your time in the dribble file.

>>> Explanation 3.
Make a note in your dribble file about if and when you would rather
use the explanation commands and if and when you would rather use some
other technique to discover why things happened.

==========================ERROR HANDLING============================

Now we will turn to working with errors.  CLASSIC needs to maintain a
consistent state at all times.  Thus, when someone enters something
that is inconsistent with the deductive closure of what CLASSIC knows,
CLASSIC notes the inconsistency, provides an error message, and
retracts the new information that was inconsistent.  That means that
if you try to add something that is not legal to a knowledge base, the
new object will not be part of the updated knowledge base.  

However, CLASSIC does maintain error objects.  Thus, if you try to add
an object A that will cause an inconsistency somewhere, A will be
retracted, but CLASSIC will keep the error object for A around.  You
can access the error object for A after A caused an error by typing

	(cl-error-object 'A).  

So, for instance, you can print the error object.

Let's create a food and state that a wine is appropriate with it.  We
can see if CLASSIC agrees that this wine is appropriate.  Suppose that
we make a meal food that is SPICY-RED-MEAT and say that we want to
drink Forman-Chardonnay with it.

>>>  (cl-create-ind 'meal-food-1 
	'(and spicy-red-meat 
	      (fills appropriate-drink forman-chardonnay)))

We see that CLASSIC will not add Meal-food-1 to the knowledge base,
since this creates a contradiction.  We can examine the problem more
closely by printing the error object that is referred to in the error
message.

>>>  (cl-print-ind (cl-error-object 'forman-chardonnay))

We look at the color role since the error said that there were
problems on that role.  We see two fillers for color:  white and red.
(And we also see that the color role can have at most 1 filler.)
So, we might want to find out how it got each of those fillers.
Use cl-exp to find out why the error object has both those fillers for
its color role by typing:

>>> (cl-exp (cl-error-object 'forman-chardonnay) 
	:role-path @color :aspect 'fills)

At this point, we may want to ask followup questions:  
  1) Why did the chardonnay-color rule fire?
  2) Why did Meal-Food-1 get its restriction on its appropriate-drink role?
Fortunately, there is an explanation function, cl-complete-exp, that
will ask ALL outstanding questions for you until the final answer is
either that a rule fired, or that some information was told.  (NOTE:
you only want to do a cl-complete-exp when you have a very specific
question in mind.  Otherwise, too many followup questions will be
asked automatically.)

Now ask the cl-complete-exp question
>>> (cl-complete-exp (cl-error-object 'forman-chardonnay) 
	:role-path @color :aspect 'fills)


So, Forman-Chardonnay is incoherent because it should have at most 1
filler for its color role.  It has two fillers: White and Red.  White
is a filler because of the chardonnay-color rule. (This rule fired
because Forman-Chardonnay is an instance of Chardonnay).  Its other
filler, Red, is there since Forman-Chardonnay fills the
appropriate-drink role on Meal-Food-1.  Meal-Food-1 has a restriction
that requires anything that fills its color role to be Red because of
the red-meat-wine-color rule.  (This rule fired because Meal-Food-1 is
an instance of RED-MEAT.)

Now you will do this for another individual.  This will be timed
again.  Create another meal-food (This time make the food
NON-OYSTER-SHELLFISH.) and fill the appropriate-drink role with
Mt-Adam-Riesling. 

>>>  (cl-create-ind 'meal-food-2
	'(and non-oyster-shellfish
	      (fills appropriate-drink mt-adam-riesling)))

This creates another error.

Error handling 1.
 Note the time in your dribble file (preceded by a ;)
Now use the explanation functions to figure out why CLASSIC reported
an error.  Note the time you finished.

Create another meal-food:

(cl-create-ind 'meal-food-3 
	'(and dark-meat-fowl 
		(fills appropriate-drink taylor-port)))

We will time this again, and now PLEASE DO NOT use the
explanation functions to discover why CLASSIC complains about this
addition.   You can use any functions other than the explanation
functions and you can look through the knowledge base.  You might
consider printing the error object, looking for incoherencies, and
then looking through the Knowledge base to figure out how the
incoherencies occurred.

Error handling 2.
Please make sure to note the time in the dribble file along with your
answer.

Error handling 3.
Also, please comment on when you would rather use the explanation
functions and when you would rather track things down without them.
Put your answer in the dribble file.

Error handling 4.
You should now be able to extend this exercise to the scenario in
assignment 4 where you created an error when you said that Beef-Curry
was an instance of FOOD-GOOD-WITH-ANY-RED-WINE.
Set up that scenario again.  Load the following lines:

>>> (cl-define-primitive-concept
  'food-good-with-any-red-wine 'meal-food)
 
>>> (defun find-red-wines ()
  (cl-concept-instances @red-wine))
 
>>> (cl-add-filler-rule 'add-red-wines @food-good-with-any-red-wine
                    @appropriate-drink 'find-red-wines)

Now, state that Beef-Curry is an instance of a
Food-Good-With-Any-Red-Wine. 

>>> (cl-ind-add @beef-curry 'food-good-with-any-red-wine)

Note the error condition on a previously existing wine in the
knowledge base.  This is another timed task.  Begin timing after you
tried to say that Beef-Curry is a FOOD-GOOD-WITH-ANY-RED-WINE.  Use
the explanation functions to explain why the error situation occurred.
Note how long this took you to do in the dribble file.

We will move on to pruning but it would be wise to read the section 
called "Explanation Functions" in the Classic Manual (section 7.1 in 
version 2.2).  Two particularly useful functions are cl-exp-rule-firing,
which explains why a rule fired, and cl-exp-not-subsumes-[concept or ind],
which explains why an object is not subsumed by a concept.


=======PRUNING===============

Using the explanation commands wisely usually means asking specific
questions when using a powerful command like cl-complete-exp.  It also
means using pruning settings.  CLASSIC has a set of pruning variables.
You should read about these in the section entitled "Pruning Function
and Variables" in the Classic Manual (section D.10.3 in version 2.2).
You can see their default settings (along with other defaults) by
typing:

>>> (cl-print-global-settings) 
	
(Look for the explanation pruning variable settings.)

Now we will investigate pruning settings that are appropriate for you.
Sometimes, for example, you will not want to see the explanation for
the at-least restrictions.  In our previous example with Meal-Food-1, if you
printed the error object, you would see that it had an at-least 2
restriction (and an at-most 1 restriction) on the role that turned out
to be incoherent.  The at-most 1 restriction comes from the fact that
the role was declared to be an attribute, and attributes have at most
one filler.  The at-least 2 restriction comes from the fact that the
role had two known fillers; the number of known fillers has been set
to the at-least restriction on the role.  (These inferences,
attribute-implies-at-most-1 and fillers-implies-at-least, are
documented in the section entitled "Regular Inferences" in the Classic
manual (section 7.4.1 in version 2.2)  We didn't
really need to know all this information for debugging.  We really
wanted to find out how the 2 fillers got there.  Thus, it was good
to have the default values of the pruning variables cause Classic to 
ignore the at-least and at-most restrictions.

You can get a feeling of how the default pruning settings cut down
the output in explanation by trying the following:

>>>  (cl-exp-set-pruning-variables nil)
>>>  (cl-exp @lobster)
>>>  (cl-exp-set-pruning-variables t)
>>>  (cl-exp @lobster)

Pruning 1.
Please use a situation that is natural to you where you want to use an
explanation function.  This could be one that we have set up in this
homework or one that you define.  Show a pruning selection that is
best for that setting and the explanation tasks.  Either defend why
the default selection is the best or change one or more of the pruning
variable settings and justify your selection.  You can set the
individual pruning variables using setq as follows:

(setq *cl-exp-ignore-at-least* nil)

Your dribble file should show:

1) What was loaded to create the situation,
2) What the pruning variables are set to - ie. do a
	cl-print-global-settings to show all the settings 
3) The explanation question (or questions) that you want to ask.
   Show the output of the explanation with 
	a) the default pruning settings (cl-exp-set-pruning-variables)
	b) your settings (if they are different)
        c) pruning setting to nil (cl-exp-set-pruning-variables nil)
4) A justification of why your settings are better in your chosen situation.
	Would your settings work for other "typical" questions in the 
	same domain? (I.e., could you set the pruning settings once and 
	basically forget about them?)

Pruning extra credit
Finally, for extra credit, suggest any new pruning variables
that could be added that would be useful.

Also, if you come up with additional pruning suggestions because of
later work on your project, please send them to dlm@research.att.com.
We are in the process of adding extra pruning functionality.
Also, version 2.3 uses meta-information to provide additional pruning
capabilities.


Type: (dribble) to close your log file now.

---------------------------------------------------------------------------
END OF asg5.txt

YOU ARE DONE WITH THIS ASSIGNMENT