Shortcomings of expert systems
A commenter on a previous entry asked for some background on why expert systems have "failed." I'm not exactly sure of his context, but there are two schools of thought here. One is that expert systems haven't really failed, business has found the appropriate place for rules-based systems that capture the heuristics experts use to problem solve or answer questions. The other school is that the traditional expert system could never fully capture the expertise used in difficult situations, as this required understanding of the context in a much deeper way than could be embedded in traditional rules. This suggests expert systems have "failed" because they could never capture this deeper, tacit, knowledge.
A quick Google for expert system shortcomings produced a number of useful results. I particularly appreciate the first link, as it is a researcher's blog on their expert system project: The Use of Expert Systems in Conservation, and the author provides links to her research and some bibliographical references.
4 Comment(s)
said:
I suspect one aspect of failure or sucess here is how exciting they are. Expert systems were never as sexy as, for example, neural networks. As you say, they may well have found their niche in commerce/research and haven't failed at all.
said:
My thesis work was on expert systems and I know what you mean about the "sexy." I looked at a sub-genre of expert systems that were model-based, rather than the traditional rule-based. In my work, I had modeling frameworks which I used in combination with heuristics (rules). Models were qualitative, based on the work of Ben Kuipers's qualitative simulation tool. My grad school colleagues also looked at Ken Forbus' qualitative reasoning work. My advisor, Lyle Ungar, has been interested in machine learning (neural nets, genetic algorithms, etc) for some time and has moved completely into computer science.
For me the big failure of ES was their inability to adapt and learn. Sure you can cycle between inductive and deductive reasoning, but the brittle nature comes from not understanding the world, from a lack of sense-making and from working within a very small fixed domain.
True insights come from mashing, hashing, satisficing, ripping, mixing and from failures.
I wonder how CyC is helping with adding some common sense to the ES scene these days?
http://en.wikipedia.org/wiki/Cyc
It was the little, inconsequential, trivial things that made ES look dumb. So my vote goes to a lack of robustness as the chief failing of ES.
Dorothy Leonard and Walter C. Swap describe how expert knowledge is developed in their book, "Deep Smarts: How to Cultivate and Transfer Enduring Business Wisdom" which could explain why expert systems failed. In describing how experts become experts, the authors write that job experience can be thought of as a Gaussian (Bell) Curve in which a new employee starts in the middle of the curve where the most common job experience is. As they continue in the job, their ranger of experience grows out toward the tails of the curve where the least probable (but usually most impact) experiences occur. Thus, expertise is the gradual accumulation of experience that is codified into tacit knowledge.
Because an expert system can only capture the codification of the experience and not the actual chain of thinking resulting from grappling with the new experience, there are numerous hidden assumptions that are not reflected in the codification. The cookbook is the chef, basically.