 |
UMBC CMSC 671 Fall 2009 Principles of Artificial Intelligence |
|
HOMEWORK ONE
out 9/2, due 9/14
This is a simple exercise to help you think about the question of how one could decide if a machine was "intelligent" or not.
Turing Test
The Turing test is a behavioral approach to determining whether or not a system is intelligent. It was originally proposed by mathematician Alan Turing, one of the founding figures in computing. Turing argued in a 1950 paper that conversation was the key to judging intelligence. In the Turing test, a judge has conversations (via teletype) with two systems, one human, the other a machine. The conversations can be about anything, and proceed for a set period of time (e.g., an hour). If, at the end of this time, the judge cannot distinguish the machine from the human on the basis of the conversation, then Turing argued that we would have to say that the machine was intelligent.There are a number of different views about the utility of the Turing test in cognitive science. Some researchers argue that it is the benchmark test of what Searle calls strong AI, and as a result is crucial to defining intelligence. Other researchers take the position that the Turing test is too weak to be useful in this way, because many different systems can generate correct behaviors for incorrect (i.e., unintelligent) reasons. Famous examples of this are Weizenbaum's ELIZA program and Colby's PARRY program. Indeed, the general acceptance of ELIZA as being "intelligent" so appalled Weizenbaum that he withdrew from mainstream AI research, which he attacked in his landmark 1976 book. (From The University of Alberta's Cognitive Science Dictionary )
Try it out
Experiment with at least two different programs that offer an interface which appears to understand unrestricted natural language input. Here are three available to you. Some of these will only work under Linux.- Jaberwacky is a web based version of the 2005 Loebner prize winner.
- Emacs Doctor -- in Emacs , enter $x doctor (escape-X doctor). Terminate all input with two carriage returns. You can find the source code here.
- Prolog Eliza -- On one of the GL SGI's, cd to ~finin/pub/471/,
sicstus then enter [eliza].
. You can find the Prolog source code here. - MegaHal -- take a look at the open source megahal system.
- Hex -- On the UMBC gl system, cd to ~finin/pub/471/hex and type hex. Source code is in the directory.
- Chatbot::Eliza -- This is a perl module and is thus easy to incorporate into a web application. CD to ~finin/pub/471/Chatbot-Eliza/ and try simple or twobots.
- Experiment with an online version of Verbot or down load a PC demo version by VirtualPersonalities, Inc.
- You can find many more implementations of such programs on the web. See, for example, Implementations of Eliza from the CMU Artificial Intelligence Repository
What to do
- Start by reading Turing's original paper (Turing, A.M. (1950). Computing machinery and intelligence. Mind, 59, 433-560 (pdf) and John McCarthy's essay, What is Artificial Intelligence. Then explore the Loebner Prize web site. (Note -- Hugh Loebner was director of UMBC's academic computing services department in the 80's).
- The Turing Test allows an unlimited range of questioning and is thus a very difficult proposition. The Loebner prize version of the Turing test is much more limited -- in topic and in duration and offers computer programs a much better chance of passing. Speculate on whether such a "partial" Turing Test makes sense. Explain your ideas in an essay of 400-600 words. Would it be a worthwhile effort? What does it tell us about the nature of understanding natural language conversations? What possible problems could arise?
- Read Sheiber's paper, Lessons from a Restricted Turing Test, and Hayes Ford, Turing Test Considered Harmful. Write an additional 200-400 words addressing the potential usefulness of the simple kinds of pattern matching techniques used in these program. Does the kind of technology it encourages developers to use have a use or is it a technological blind alley?
Your work should be submitted as a hardcopy document in class on Monday 14 September.
Background reading
- The Wikipedia Turing Test article
- The Alan Turing Home Page and the sub page on the Turing test
- Turing, A.M. (1950). Computing machinery and intelligence. Mind, 59, 433-560.
- J. Weizenbaum, ELIZA - A Computer Program For the Study of Natural Language Communication Between Man And Machine, Communications of the ACM, Vol 9, No 1, January 1966
- Colby, K.M. et al. (1971) Artificial paranoia. Artificial Intelligence, 2, 1-26.
- Weizenbaum, J. (1976). Computer power and human reason. San Francisco, CA: W.H. Freeman.
- Pat Hayes and Ken Ford, Turing Test Considered Harmful, 1995.
- Introducing Megahal by Jason L. Hutchens and Michael D. Alder of the University of Western Australia.
- Hutchens, Jason L. 1996. How to pass the Turing test by cheating.
- Stuart Shieber. Lessons from a Restricted Turing Test. Communications of the Association for Computing Machinery, volume 37, number 6, pages 70-78, 1994.
- The Loebner Prize web site.
- Chatterbots, Tinymuds, And The Turing Test: Entering The Loebner Prize Competition, Michael L. Mauldin, AAAI-94.
- Saygin, A.P., Cicekli, I. and Akman, V. (2000), 'Turing Test: 50 Years Later', Minds and Machines 10(4): 463-518.
- John R. Searle, Is the Brain a Digital Computer?.