File: lab-asg5.txt

PRELIMINARIES

Open a new log file:
>>> (dribble "lab-asg5.log")

We will be working with the knowledge base basically as it was when you were
starting assignment 4.  In order to make it easier to peruse what was
loaded in, this has all been merged into one file.  This new file is
basically "Recover/after-just-wines.lisp expanded to include all the
files it loaded.  We also added the rules in foods-wines.lisp.
The entire file is now in expanded-wines-kb.lisp.
(Also, there are some slight modifications in this file so use this
one and do not use after-just-wines.lisp.)
Load this file:

>>> (load "<topdir>/Customize.lisp")
>>> (lload "Recover/expanded-wines-kb.lisp")


REVIEW OF lab-asg1.txt through lab-asg4.txt

In the previous modules of this assignment we covered:

 1. How to create "DISJOINT-PRIMITIVE" concepts.  Using this construct
    we created the concepts 'WINE-PROPERTY', 'CONSUMABLE-THING',
    'WINERY', and 'WINE-REGION'. We also constructed the concepts
    'EDIBLE-THING', 'POTABLE-LIQUID', 'WINE-COLOR', 'WINE-BODY',
    'WINE-SUGAR', 'WINE-FLAVOR', 'WINE' and 'MEAL-FOOD'.

 2. How to get verbose or laconic debugging information from Classic when
    concepts and individuals are defined.

 3. Commands for inspecting and navigating the KB.

 4. Definition of Roles and  Attributes.

 5. Creation of individuals and the effect of role restrictions
    on individuals.

 6. Adding concepts to the KB in such a way that existing concepts and
    instances get reclassified under them (we illustrated this with
    'REGIONAL-WINERY' and its children).

 7. Use of the inverse of a role.

 8. Using rules in Classic: attaching rules, adding and removing rules.
    The main use of rules, i.e. for adding non-definitional info to
    instances that get classified under a concept.

 9. More use of rules: using rules to create disjunctive concepts,
    query templates, and circular definitions.

10. explanation functions, particularly cl-exp

11. Using tests

12. Same-as restrictions

You may want to review some of these.



The purpose of this assignment is to investigate some of the issues
involved with debugging and maintaining knowledge bases.
While we are primarily interested in your learning the material, we
are also interested in how easy or hard it is for you to learn.  
There are 5 times where you will be asked to time yourself while doing
a task.  You will not be graded on your times.  Please be accurate on
times.  Please do make sure to include times.  If by some chance you
forget to time something, make an estimate and note that it was an estimate.
You will also be asked to write notes in your dribble files at 
places.  You can either precede each line with a ; so that lisp does
not try to evaluate your statements, or you can surround a block of 
text with
#|
lots of text
|#
There is a 3 part question on one step explanations, a 4 part
question on complete explanations of error situations, a 4 part
question on pruning settings, and an extra credit question. 

		USING, DEBUGGING and MAINTAINING KNOWLEDGE BASES
		------------------------------------------------

In order to effectively develop, use, and extend knowledge bases, we
need to be able predict the results of additions and queries.  Thus,
we should understand the inferences that the system makes.
We will first explore the use of explanation functions to aid in this
process. 

The last assignment (lab-asg4) suggested that you use an explanation
function to see why CLASSIC believed certain things about @Some-wine
after you defined @My-meal-food.  We will now go into a bit more
detail about some of the inferences CLASSIC makes.
Lets investigate how some of the foods in the knowledge base got
their restrictions on their appropriate-drink role.
In lab-asg4, you looked at the restrictions on spicy-red-meat foods.
One could just pick one of the SPICY-RED-MEAT foods in the knowledge
base like Beef-curry and examine that or we could make a new
SPICY-RED-MEAT food and look at it.
Lets define the same SPICY-RED-MEAT food again, and examine it
through the explanation functions.

Define @My-meal-food again as:
>>> (cl-create-ind 'My-meal-food
                   '(AND SPICY-RED-MEAT
			(FILLS appropriate-drink Some-Wine)))

lets investigate Some-Wine again.
First print it:
>>>(cl-print-ind @some-wine)

If we are interested in recommending wines with foods, we would want
to figure out why it has specified fillers and restrictions for the
sugar, flavor, body, and color roles.  
We could ask for a general explanation of the food by asking:
>>>(cl-exp @some-wine)
but since the 4 roles we are interested in have specific fillers, we
can ask how those fillers alone got there by asking

>>>(cl-exp @some-wine :aspect 'fills)

We see that almost everything of interest on Some-wine came from a
propagation on My-meal-food because of restrictions on it's
appropriate-drink role.
The reason that the restrictions on My-meal-food's appropriate-drink
role influence Some-wine is because Some-wine is known to fill
My-meal-food's appropriate-drink role, and thus it must satisfy all
those restrictions.
The next question is how those restrictions got there at all.
To use explanation to determine that, ask

>>> (cl-exp @my-meal-food :role-path @appropriate-drink)

Now we see that color, body, flavor, and sugar all got fills 
restrictions because of rule firings.   So, any spicy red meat food,
such as Beef-curry, should have a
 red-wine because of the red-meat-wine-color rule,
 full wine because of the spicy-red-meat-wine-body rule,
 moderate wine because of the spicy-red-meat-wine-flavor rule, and 
 dry wine because of the meat-wine-flavor rule.

Above we went through a 2 step explanation - starting with explaining
Some-wine and then explaining a particular role on My-meal-food.
In the case above, we could have actually simplified this process by
just choosing a food, like Beef-curry and explaining the restrictions
on its appropriate-drink role. 
>>> (cl-exp @beef-curry :role-path @appropriate-drink)

Now, lets look at another food class of food in the knowledge base.
We will consider things that fall under NON-OYSTER-SHELLFISH.
Consider, Lobster.  Don't bother to create a new food and associate a
wine with it - just use Lobster.  We are interested in the
appropriate-drink's color, body, flavor, and sugar roles.

Now we will time a task.

>>> Explanation 1.
***START TIMING HERE.  Put a ; in your dribble file and note the time.
Now use cl-exp on Lobster to answer why it's appropriate-drink role 
has each restriction on the roles - color, body, flavor, and sugar.

***STOP TIMING HERE.  Note the time that it took you to do this task
in your dribble file (preceded by a ;)

We are going to time another task.  This time please DO NOT use the
explanation commands.  On another food, Fettuccine-alfredo, please
explain the restrictions on its appropriate-drink's color, body,
flavor, and sugar roles.  Make sure you time this task.
You can use whatever CLASSIC commands you like.
(you might consider just printing the food and then looking at the
file you loaded to discover what rules, propagations, etc.
could have applied.)

>>> Explanation 2.
**** PLEASE START TIMING NOW - put a ; and the time in your 
dribble file.  Put your answer in the dribble file (starting 
each line with a ; or surrounding the whole answer  by #|  and  |#  )


*** STOP TIMING HERE.  Please note your time in the dribble file.

>>> Explanation 3.
Make a note in your dribble file about if and when you would rather
use the explanation commands and if and when you would rather use some
other technique to discover why things happened.

==========================ERROR HANDLING============================

Now we will turn to working with errors.
CLASSIC needs to maintain a consistent state at all times - thus when
someone enters something that is inconsistent with the deductive
closure of what CLASSIC knows, CLASSIC notes the inconsistency,
provides an error message, and retracts the new information that was
inconsistent.  That means if you try to add something to a knowledge
base that is not legal, the new object will not be part of the
consistent knowledge base.
CLASSIC does maintain error objects however.  Thus if you try to add
an object A that will cause an inconsistency somewhere, A will be
retracted, but CLASSIC will keep the error object for A around.
You can access the error object for A after A caused an error by
typing (cl-error-object 'A).
You can print the error object.

Lets create some food and state that some wine is appropriate with it.
We can see if CLASSIC agrees that this wine is appropriate.
Lets make a meal food that is SPICY-RED-MEAT and say that we want to
drink Forman-chardonnay with it.

>>>
(cl-create-ind 'meal-food-1 
	'(and spicy-red-meat 
	      (fills appropriate-drink forman-chardonnay)))

We see that CLASSIC will not add Meal-food-1 to the knowledge base
since this creates a contradiction.  Lets look at the problem by
printing the error object that is referred to in the error message.

>>>
(cl-print-ind (cl-error-object 'forman-chardonnay))

We look at the color role since the error said that there were
problems on that role.  We see two fillers for color - white and red.
(and we also see that the color role can have at most 1 filler.)
So, we might want to find out how it got each of those fillers.
Use cl-exp to find out why the error object has both those fillers for
its color role by typing:

>>>
(cl-exp (cl-error-object 'forman-chardonnay) 
	:role-path @color :aspect 'fills)

Actually, we now want to ask followup questions - why did the
chardonnay-color rule fire and why did Meal-food-1 get its 
restriction on its appropriate-drink role.
Fortunately, there is an explanation function - cl-complete-exp -
that will ask ALL outstanding questions for you until the final
answer is a rule fired or some information was told.
NOTE - you only want to do a cl-complete-exp when you have
a very specific question in mind, otherwise, you will get too many
followup questions asked automatically.

Now ask the cl-complete-exp question
>>>
(cl-complete-exp (cl-error-object 'forman-chardonnay) 
	:role-path @color :aspect 'fills)


So, Forman-chardonnay is incoherent because it should have at most 1
filler for its color role.  It has two fillers - White because the
chardonnay-color rule. (This rule fired because Forman-chardonnay is
an instance of chardonnay).  Its other filler Red is there since
Forman-chardonnay fills the appropriate-drink role on Meal-food-1.
Meal-food-1 has a restriction that requires anything that fills its
color role to be Red because of the red-meat-wine-color rule.
(This rule fired because Meal-food-1 is an instance of RED-MEAT.)

Now you will do this for another individual.  This will be timed
again.
create another meal-food-2 - this time make the food
NON-OYSTER-SHELLFISH and fill the appropriate-drink role with
Mt-adam-riesling.
(cl-create-ind 'meal-food-2 
	'(and non-oyster-shellfish
	      (fills appropriate-drink mt-adam-riesling)))

This creates another error.

>>> Error handling 1.
Now note the time in your dribble file (preceded by a ;)
Now use the explanation functions to figure out why classic reported
an error.
Note the time you finished.

Create another meal - this time
(cl-create-ind 'meal-food-3 
	'(and dark-meat-fowl 
		(fills appropriate-drink taylor-port)))

We will time this again and this time PLEASE DO NOT use the
explanation functions to discover why CLASSIC complains about this
addition.   You can use any functions other than the explanation
functions and you can look through the knowledge base.  You might
consider printing the error object, looking for incoherencies, and
then looking through to Knowledge base to figure out how the
incoherencies occurred.

>>> Error handling 2.
Please make sure to note the time in the dribble file along with your
answer.

>>> Error handling 3.
Also, please comment on when you would rather use the explanation
functions and when you would rather track things done without them.
Put your answer in the dribble file.

>>> Error handling 4.
You should now be able to extend this exercise to the scenario in
assignment 4 where you created an error when you said that beef-curry
was an instance of FOOD-GOOD-WITH-ANY-RED-WINE.
Set up that scenario again.  Load the following lines:

>>>

(cl-define-primitive-concept
  'food-good-with-any-red-wine 'meal-food)
 
(defun find-red-wines ()
  (cl-concept-instances @red-wine))
 
(cl-add-filler-rule 'add-red-wines @food-good-with-any-red-wine
                    @appropriate-drink 'find-red-wines)

Now, state that beef-curry is an instance of a
food-good-with-any-red-wine. 
>>> (cl-ind-add @beef-curry 'food-good-with-any-red-wine)

Note the error condition on a previously existing wine in the
knowledge base.
This is another timed task.  Begin timing after you tried to say that
Beef-curry is a FOOD-GOOD-WITH-ANY-RED-WINE.
Use the explanation functions to explain why the error situation
occurred. 
Note how long this took you to do in the dribble file.


=======PRUNING===============

Using the explanation commands wisely usually means asking specific
questions when using a powerful command like cl-complete-exp. 
It also means using pruning settings.
CLASSIC has a set of previously defined pruning variables set.
You should read about these in section 7.2 - Pruning Explanation 
Output of the manual.
You can also see their default settings (along with
other defaults) by typing:

>>>
(cl-print-global-settings) 
	(look for the explanation pruning variable settings).

Now we will investigate pruning settings that are appropriate for you.
Sometimes it will be the case that you do not want to see the
explanation for the at-least value, for example.  In our previous
example with Meal-food-1, if you printed the error object,
you would see that it had an at-least 2 restriction (and an at-most 1
restriction) on the role that turned out to be incoherent.
The at-most 1 was there because the role was an attribute and
attributes have at most one filler.  The at-least 2 was there because
the role had two known fillers and the number of known fillers gets
set to be the at-least restriction on that role.  (these inferences -
attribute-implies-at-most-1 and fillers-implies-at-least  - are
documented in section 7.4.1 on inferences in the manual.)
We didn't really need to see that information for debugging though -
what we really wanted to find out was why the 2 fillers got there.
Thus, it was good to have the default setting of ignoring the 
at-least and at-most settings.

You can get a feeling of how the default pruning settings cut down
the output in explanation by doing:

>>>
(cl-exp-set-pruning-variables nil)
(cl-exp @lobster)
(cl-exp-set-pruning-variables t)
(cl-exp @lobster)

>>> Pruning 1.
Please use a situation that is natural to you where you want to use 
an explanation function.   This could be one that
we have set up in this homework or one that you define.
Show a pruning selection that is best for that setting and
the explanation tasks.  Either defend why the default selection is 
the best or change one or more of the pruning variable settings 
and justify your selection.
You can set the individual pruning variables using setq as follows:

(setq *cl-exp-ignore-at-least* nil)

Your dribble file should show:
1) what was loaded to create the situation,
2) what the pruning variables are set to - ie. do a
	cl-print-global-settings to show all the settings 
3) the explanation question (or questions) that you want to ask.
   show the output of the explanation with 
	a) the default pruning settings (cl-exp-set-pruning-variables)
	b) your settings (if they are different)
        c) pruning setting to nil (cl-exp-set-pruning-variables nil)
4) a justification of why your settings are better in your chosen situation.
	Would your settings work for other "typical" questions in the 
	same domain? (ie. could you set the pruning settings once and 
	basically forget about them?)

>>> Pruning extra credit
Finally, for extra credit, suggest any new pruning variables
that could be added that would be useful.

Also, if you come up with additional pruning suggestions because of
later work on your project, please send them to dlm@research.att.com.
We are looking in the process of adding extra pruning functionality.