Without such a study, it is argued that knowledge engineering will remain very much an art. KBS succeed in solving problems where solutions are derived from the application of a substantial body of knowledge, rather than by the application of an imperative algorithm. In the s, KBS technology was widely applied to solve stand-alone problems. In the s, many organisations have identified their collective knowledge as their most important resource, and are applying KBS technology to capture and exploit these "knowledge assets" in a systematic manner Liebowitz and Wilcox, The characteristic feature of problem domains where KBS technology is suitable is that the problems are ill-defined: Often, the domain is such that there can be no guarantee that a solution will be found, or that found solutions will be optimal.
Many KBS offer a "best effort" solution, which is good enough when the application requirements permit this that is, the system is not safety or mission-critical.
The literature on KBS requirements specification recommends that the requirements be divided into minimum and desired functionality Rushby, In practice, desired requirements will be difficult to specify, due to the ill-defined nature of the problem to be solved for example, in the vehicle loading application, it may be very difficult to determine what constitutes an "optimal solution" for the desired requirements Batarekh, Preece, Bennett and Grogono, This is unsurprising; from a software engineering point-of-view, given the fact that the problem is ill-defined, it follows that the user requirements will be ill-defined also.
Knowledge engineering can be viewed as a special instance of software engineering, where the overall development strategy typically must employ exploratory prototyping: The knowledge engineer will have the hardest task when the domain knowledge itself is not well-understood; for example, when the knowledge is locked up in the heads of human experts who are not able to articulate it clearly.
It is not unusual for a knowledge engineer to face a situation in which the users will be unable to say what they really want, experts will be unable to say what they really know, and somehow a KBS must be built! Building KBS is something of an art. There is much confusion about the distinction between validation and verification, but the conventional view is that verification is the process of checking whether the software system meets the specified requirements of the users, while validation is the process of checking whether the software system meets the actual requirements of the users.
Boehm memorably characterised the difference as follows Boehm, Verification is building the system right. Validation is building the right system. Verification can be viewed as a part of validation: However, verification is unlikely to be the whole of validation, due to the difficulty of capturing specifying user requirements. As noted above, this is a particularly important distinction in knowledge engineering.
In software engineering, efforts have been made to formalise the development process so that user requirements may be stated as a fully-formal specification, from which it can be proven that the implemented software system meets the requirements.
While formal methods are desirable - even essential - in some cases notably safety and mission-critical systems , these methods are unsuitable in large classes of software applications: Where requirements are amenable to formal specification, it may be too difficult to create the specification within project time and budgetary constraints. There are many kinds of requirement that are not amenable to formal specification for example, the "usability" of a graphical user interface.
Given that knowledge engineering is an inexact art, the most fundamental measures of the success of a KBS project would seem to be: Did we get it right? That is, does it meet the users' actual requirements. Can we keep it right? That is, is it sufficiently maintainable for anticipated future changes. Can we do it again?
That is, is the process repeatable to ensure success with future projects. The final point refers to the capability of the knowledge engineers, and reflects the modern view of software quality being determined primarily by the quality of the development process Preece, While verification and validation are only part of the overall development process, they are extremely important because they are the only way to produce an answer to the first of the three questions above "Did we get it right?
Obviously, if the techniques are incomplete or unsound, then they cannot be trusted to provide measurement of software quality and project success. Knowledge engineering and software engineering can be seen as a combination of methods and measurement: The following artifacts are of particular importance in the KBS development process: Requirements Specification The requirements specification document states the minimum and desired user requirements as described in Section 1 , typically in natural language or, less usually, in some restricted or semi-structured natural language subset.
Conceptual Model The conceptual model describes the knowledge content of the KBS in terms of real-world entities and relations. This description is entirely independent of the ways in which the KBS may be designed or implemented: The best-known framework for defining KBS conceptual models is KADS Wielinga, Schreiber and Breuker, , in which models may be initially defined using a semi-formal, largely diagrammatic representation, from which a refined, formal model can be derived.
The conceptual model forms the basis of the design model. Design Model The design model serves to "operationalise" the conceptial model into an executable KBS; it describes the required system in terms of computational entities: For example, the design model may specify that a particular conceptual task is to be performed by a backward-chaining search, or that a concept taxonomy is to be represented using a frame heirarchy.
The design model dictates the form of the implemented system. Implemented System This is the final product of the development process: Once the design issues have been explored in the design model, the system may be implemented in any programming language, although typically a special-purpose KBS language is used.
Five of the most common approaches are listed below. Arguably, it is also the least reliable, as it essentially involves nothing more than human proof-reading the text of the various artifacts. Typically, a domain expert is asked to check the statements in the knowledge base; since the formal languages used in the design model and implemented system will be unfamiliar to domain experts, this technique is better-suited to use with the semi-formal conceptual model which will typically use a more "reader-friendly" graphical representation.
Static Verification Static verification consists of checking the knowledge base of the KBS for logical anomalies. Frameworks for anomalies in rule-based KBS have been well-explored, and software tools exist to detect them Preece, Shinghal and Batarekh, The most commonly-identified anomalies - and the ones detected by most of the available tools - are redundancy and conflict.
Redundancy occurs when a knowledge base contains logical statements that play no purpose in the problem-solving behaviour of the system; this typically indicates that the system is incomplete in some way. Conflict occurs when there are logical statements that are mutually inconsistent, and would therefore cause the system to exhibit erroneous behaviour. Anomalies may exist in any of the formal artifacts: Formal Proof Formal proof is a more thorough form of logical analysis of the formal artifacts in the development process than that provided by static verification.
As described in Section 1, where requirements are amenable to formal specification, proof techniques can be employed to verify that the formal artifact meets the specified requirements. A review of opportunities to use formal methods in knowledge engineering is provided by Meseguer and Preece Meseguer and Preece, In practice, however, while there are many formal specification languages for KBS, there are few documented examples of the use of proof techniques to very user requirements.
Cross-Reference Verification When there exists descriptions of the KBS at different "levels", it is desirable to perform cross-checking between these, to ensure consistency and completeness. For example, we would expect the concepts that are specified as being required at the conceptual level to be realised in terms of concrete entities at the design level, and in terms of concrete data structures in the implemented system.
Therefore, the most appropriate uses of cross-reference verification are to check correspondence between: Empirical Testing All software testing involves running the system with test cases, and analysing the results. The software testing literature distinguishes between function-based testing and structure-based testing. Function-based testing bases the selection of test cases upon the functional requirements of the system, without regard for how the system is implemented. The success of function-based testing is dependent upon the existence of a "representative" set of test cases.
In structure-based testing, test cases are selected on the basis of which structural components of the system they are expected to exercise; the objective is to show that the system produces acceptable results for a set of test cases that exercise all structural components of the system. Testing can be applied only to the executable artifacts: