Marketing Gate's Knowledge
Professor John M. McCannFuqua School of Business
Duke University
The theme of this series of hypertext documents is that the computer should take a stronger role in the thought processes that are usually reserved for people. In fact, computers today have not been very good in this area.
To fill this void, new ways must be found for increasing the ability of the computer to think.
Where do the thoughts that the computer thinks come from? Does the computer make them up, or are there other sources for the thoughts that it thinks. The approach described in this book involves a recycling philosophy: old thoughts are not discarded, they are saved and recycled as needed. This approach was adopted based upon a simple examination of how managers do some of their thinking.
First, it was recognized that a lot of analysis is simply a re-analysis in which existing methods are applied to new data. The standard practice is to design and/or customize an information system so that a few dozen standard reports can be run upon demand. For instance, a brand topline report gives a summary view of how a brand is performing in the market. Managers tend to select one of these common report formats, have the computer fill in the data, and then apply one or a few thought processes to the data. The experienced manager is usually going to simply apply the same thought processes to the new data that have been applied to similar data in the past. This is a form of re-analysis in which an old analysis method is applied to new data.
One can then pose the question: If analysis is re-analysis, is thinking really rethinking?
Managers and professionals think thoughts on an on-going basis. The two thoughts shown on the left about a competitor and a retailer are typical of the hundreds of thoughts that managers think every day. These thoughts are placed into the human memory system and thus form part of the manager's knowledge and beliefs. And, they form a resource for future thinking. When it is time to think new thoughts, they can draw upon these "old thoughts". They can simply remember them, or they can use them as a source for thinking new thoughts. These old thoughts can serve as "templates" for the new thoughts.
A thought template is a guide to new thoughts. A manager can use it to assist in the formulation of new thoughts.
Such a template can be formed by selecting past thoughts, decomposing them into sentences, and then determining which of the words in the sentence are fixed and which are likely to be changed in a rethinking of the sentence. (These past thoughts can be found in a firm's file cabinets in the form of reports and memos. Marketing managers and analysts have devoted many man-years to the analysis of marketing data, and thus have built a large reservoir of thoughts that can be used as the bases for thought templates. The latter sections of this chapter describe this knowledge engineering process). The final step is to write a computer program, an analyzer, that fills in these blanks and determines if the sentence should be written.
New thoughts can be, but do not have to be, the recombination of old thoughts. The human mind makes extensive use of its memory in the thought process, and thus can make use of past thoughts. The manager can say to herself: "The last time I faced this situation, I looked to see who was taking space away from whom, and I looked to see how I got more space in a particular account. I think I will do the same thing this time." This manager is effectively saying: "I think I will rethink some old thoughts". In such a case, thinking is actually rethinking.
A manager does not have to be restricted to only thinking his/her own past thoughts. S/he can also rethink the thoughts of other people, perhaps professors, peers, managers, or even the thoughts that s/he read in books or reports. There are many sources for thought templates, and a process is needed to get them into a form so that a computer can go through the rethinking process. In insight generators, these thought templates are put into analyzers in the form of sentences. An analyzer will perform the necessary analysis to allow it to "fill in the blanks" in a thought template. In addition, it determines which of several thoughts it should rethink.
This use of thought templates is the foundation of Marketing Gate's knowledge and application structure. The remainder of this chapter describes this knowledge and introduces the notion of reverse knowledge engineering as a means of building insight generators.
Marketing Gate is a knowledge-based system running in Goldworks, a hybrid AI tool which runs on 80386-based personal computers and workstations. It offers multiple knowledge representation methods: rules, frames, inheritance, and object oriented programming. It allows for multiple reasoning methods: inheritance, forward chaining, and backward chaining, and it allows for access to other languages and packages. A unique feature of Goldworks permits Marketing Gate to operate in a manner similar to a multitasking environment in which one can easily jump from one computing task or program to another. Goldworks uses a special kernel on top of the PC DOS operating system to allow it to run in memory above the PC DOS's standard 640k limit. By running in this non-standard memory, it makes the other memory (the lower 640k) available to regular DOS programs such as Lotus 1-2-3, Word, and mainframe communications software. In fact, a PC-based MMIS program could operate in this lower memory, with the MMKS running in the upper memory. While the two programs cannot operate simultaneously, one can be suspended while the other is executed.
Later sections will discuss how the various features of Goldworks are used to produce Marketing Gate; this section catalogs the types of knowledge possessed by Marketing Gate. Marketing Gate's knowledge is two types:
This section describes the former, and the next section discusses the methodology used to capture and display Marketing Gate's accumulated market knowledge.
To review, Marketing Gate has a report oriented structure in which the manager does analysis by writing reports which are composed of paragraphs. The manager selects the paragraph and provides the context data, and Marketing Gate writes the paragraph. Once a context has been set by the manager, Marketing Gate remembers and uses it in subsequent paragraphs unless the manager enters a new context. A paragraph is a structured document containing three main sections: context, data, and a statement about the data. The statement section is further divided into observations and conclusions. In order to accomplish its work, Marketing Gate must have knowledge about paragraph generation, knowledge disposition, analysis, planning, and monitoring.
PARAGRAPH GENERATION: The knowledge required to write a paragraph is contained in distinct units within Marketing Gate called ANALYZERS, DESIGNERS, and MONITORS. To apply an ANALYZER and thus write a paragraph, Marketing Gate must know:
KNOWLEDGE DISPOSITION: After a paragraph has been written, the manager controls its disposition. Marketing Gate must know how to alter its internal memory either to keep the knowledge or to erase it so that Marketing Gate will "forget" about the paragraph's statement according to what the manager decides about the importance of the knowledge in the paragraph, . Marketing Gate must also know how to transport the paragraph to the desktop publishing software. In the current version, it knows how to merge the paragraph into Microsoft Word.
ANALYSIS KNOWLEDGE: The analysis process involves a series of linked data views and interpretations. An analyst must be able to select the next data view according to what has been learned from earlier ones and/or knowledge stored in memory. Marketing Gate must possess this same type of knowledge. In order to recommend the next analysis step, it must know about the structure of markets and categories. For instance, it knows that if a market has been identified as a problem, then the next step would be to examine chains within the market. It also knows that an account problem could be caused by the lack of merchandising support by the chain. Once the next step has been selected by the manager, Marketing Gate must know how to transfer its current context into the next phase, i.e., into the context in the ANALYZER, DESIGNER, or MONITOR that is used in the next phase. Finally, Marketing Gate knows when sufficient evidence has been gathered to warrant the transition from analysis to planning.
PLANNING KNOWLEDGE: Marketing Gate knows how to transfer its analysis results into the planning phase; it knows what it has been working on and suggests the appropriate context. It also has knowledge about the appropriate marketing programs and events to solve the problems identified in the analysis phase. Once the manager has selected the appropriate type of event, Marketing Gate has knowledge about the design of that event.
MONITORING KNOWLEDGE: Marketing Gate knows how to transfer its accumulated knowledge into the monitoring phase. It knows the how to select the appropriate monitor, how to set the context, the type of event being planned, and the key reason why the event was necessary. It knows how to monitor the event before, during, and after its implementation.
With this knowledge base, Marketing Gate could work all the time. It could monitor events, troll for problems and opportunities, detect competitive moves, and write standard presentations such as a trade presentation. It would never be "turned off"; no holidays, no evenings off, no spare time -- it would not be an 8-to-5 employee.
Marketing Gate is unique among marketing systems for several reasons; one key reason is the concept of doing analysis by directing the system to rewrite existing paragraphs. This approach exists alongside another key aspect of Marketing Gate: the ability to learn and store knowledge for future use. These two aspects required the development of a knowledge representation methodology with a transparent mapping between the way humans store and process knowledge and the corresponding methods employed by a computer program. The system has to be able to display what it knows to the user so that they can both know the same things.
For design purposes, we assumed that managers reason with sentences, like "For Folgers, the situation in Boston requires much attention." This string of nine words has meaning to a marketing manager. He can interpret it and incorporate it with other information and knowledge. Most computer programs cannot interpret and read such a sentence.
The Goldworks software tool uses assertions to store knowledge and information. The sentence about Folgers could be placed in the following Goldworks assertion:
Marketing Gate must have some way to translate one form into another, and a "sentence" structure was developed in Marketing Gate for such mapping. The study of a number of sentences led to the recognition that a typical sentence can be decomposed into permanent text and blanks, with the blanks instantiated every time the sentence is written for a new context.
Consider the following sentence:
If one wanted to structure this sentence so that it could be rewritten for different brands, then one could partition the sentence into segments, with the partitioning based on the identification of sentence parts which would remain permanently with the sentence and sentence parts which could vary in other situations. For instance, the following sentence segments seem appropriate:
The segments in capital letters are permanent and the ones in lower case are instantiatable.
The sentence could be rewritten for Ragu, an older brand of spaghetti sauce, by making the following substitutions: "Ragu" for "Newman's Own", and "established" for "relatively new".
The sentence would then read:
Or, it could be rewritten for Maxwell House coffee as:
It is easy to see that the instantiatable parts of this sentence can take on different values depending upon the age of the brand and its product category. These segments which can be instantiated are termed "blanks" and these blanks are interspersed among the permanent segments to produce the following sentence structure:
Since the blanks are filled in by the system, it is important that the system know the types of values each blank can take, i.e., the blank's legal values.
Since Marketing Gate writes the sentence by filling in the blanks, the system requires knowledge of the brands within a category and of the categories in order to fill in blank1 and blank3. It gets the brand value from the context section of the paragraph and then uses its internal knowledge of the brand-to-category relationships to infer the product category value. Marketing Gate must use other knowledge to infer the value of blank2, the brand's age. Rules can be used for such inference by coupling a blank with a parameter, in this case the BRAND-AGE parameter. The following rules could be used to infer the value for BRAND-AGE:
IF TIME-SINCE-INTRO = NONE
THEN BRAND-AGE = NEW
IF TIME-SINCE-INTRO = LONG
THEN BRAND-AGE = ESTABLISHED
Additional rules would be needed to establish a value for TIME-SINCE-INTRO, and these rules would be based upon attributes of the category such as inter-purchase time and age.
This process has produced three structures: a sentence structure which partitions a sentence into permanent and blank segments, an assertion structure in which the sentence is represented in a form usable by Goldworks, and a blank structure which specifies the values each blank can take. An additional example illustrates this method of structuring a sentence.
This type of structuring permits a new approach to the knowledge engineering process.
The previous section introduced Marketing Gate's sentence concept and the three sentence structures: sentence, assertion, and blanks. Marketing Gate reasons with the assertions and presents the results via the sentences. The blanks provide a way to customize each sentence to the particular context, and the blanks structure provides linkages between a sentence and the knowledge necessary to write the sentence given a context. The last example in the previous section illustrated how an existing statement could be structured in terms of blanks and permanent text, and how the statement would appear once it had been written in a different context. This section extends this example by discussing it in relation to the knowledge engineering process.
Knowledge engineering is the process of extracting and representing knowledge. When applied in the development of an expert system, it is the process of extracting expertise from a single expert and representing this knowledge so that an expert system tool can reason with it. It is a very labor intensive process involving continual interaction between the expert and the knowledge engineer. It is very time-consuming because most experts do not consciously know, and therefore cannot explain, the knowledge they are applying at any point in time. Such knowledge is thought to have a major heuristic component which accumulates with experience, and it must be extracted from the expert with various mechanisms.
During the first two years of the Marketing Workbench Laboratory, a number of sponsoring companies indicated that knowledge engineering is a very difficult process in consumer packaged goods marketing for several reasons:
The first reason seems to be a very common one: marketers are unable and/or unwilling to give their time to the knowledge engineering process. Therefore, it is very difficult to computerize what market managers know.
Marketing Gate provides a different approach to knowledge engineering: reverse engineering. Look at what they say, not what they know. Reverse engineer the knowledge from its applications. A significant portion of a marketing manager's time involves the decision-influencing role of marketing management. In this role, marketers must influence the decisions of others, and this influencing process usually involves the written word in the form of reports and/or presentations. By examining this material, the knowledge engineer can deduce marketing knowledge. What would a person have to know and which facts would they have to examine to arrive at the statements in the report or presentation?
This resembles the case analysis approach to knowledge engineering popularized in the early expert system days in the development of medical expert systems. In this approach, the expert, a physician, is given a patient work-up (the case study) and asked for a diagnosis. Then the expert is asked to explain how s/he arrived at the diagnosis. In the reverse engineering process, the "case study" is the statement from a report. Instead of asking busy marketing managers how they arrived at this statement, a marketing research analyst can play this role. By using the MMIS, the analyst can reproduce the numerical results in the report or generate the numbers if they are not present in the statement. Given the numbers, the analyst can infer the knowledge the manager used in arriving at the statement.
At this point, the analyst would possess sufficient knowledge to generate the exact statement appearing in the report. The analyst has (1) used the MMIS to produce the numbers which underlay the statement, and (2) inferred the knowledge necessary to use the numbers to generate the statement. Next, the analyst must do original thinking to determine all possible similar sentences which can be written, given different numerical values. The existing sentence is one instantiation of a generalized sentence, and this generalized sentence must be developed via the process introduced in the previous section. The remainder of this section expands this process by describing its application to another example.
The first step in reverse engineering this statement is to apply the methods of the previous section to produce a sentence structure and the corresponding assertion and blank structures. Rather than repeat these steps in this section, we focus on the next step, which is the determination of the knowledge necessary to reach the key conclusions in the statement and to fill in the appropriate supporting evidence. The purpose of this statement is to classify the type of approach being employed by the brand. The conclusion is the first sentence in the statement. The second sentence contains the observations necessary to support the conclusion. The following knowledge could lead to the conclusion that Newman's Own has taken a very low risk approach in its attempt to penetrate the category.
This knowledge can be represented via the following rule:
With this rule and the appropriate sentence structuring, the statement could be written as shown above.
The next step is to generalize this statement so that it could be written for brands following other strategies. The first step in this generalization process is to identify other approaches. Then, one would determine the evidence that implies the existence of each approach. One such approach is opposite of Newman's Own's low risk approach -- a high risk approach. If a new brand were concentrating its brand items in declining segments and not in growing segments, then it could be said to be using a high risk approach. The following rule captures this knowledge:
This example has illustrated that
This same process could be applied to the other parts of the report. It should be easy for the reader to see how to generalize the following sentences:
After reverse engineering these statements, the knowledge engineer would have added a new store of knowledge to Marketing Gate. It would know
A good analyst and knowledge engineer could determine the methods for making this store of knowledge operational so that it could be applied when needed.
A firm's library of annual plans and other reports could serve as the source of this type of knowledge. The high turnover in marketing management assures that these reports would have been prepared by many different managers, thus providing a wide variety of analyses and statements. If a firm has 10 brands, then 10 years of reports would produce 100 sources of knowledge. Large firms such as P&G may have 100 brands and thus 1,000 reports based upon 10 planning efforts. Ad hoc reports could greatly increase the size of this library of knowledge engineering material.
This reverse engineering approach could be augmented by a direct approach. Instead of examining the written record, the firm could assign a good analyst or manager to work with a knowledge engineer in the data analysis process. The analyst could use the MMIS to examine a brand and/or market. S/he would use the MMIS to obtain and interpret one data view; s/he would make statements about the data view. The knowledge engineer would record these statements and perhaps direct the flow of the analyses so as to maximize the knowledge extraction process.
For instance, the manager could be asked to use the MMIS to run a Brand Topline report for one market. After the manager had made a number of statements about this report, s/he could be directed to run the same report in a different market. By cycling through a series of Brand Topline reports for different markets, different categories, and different time periods, the knowledge engineer would learn the types of statements one could make from this report. Additional statements could be obtained by working with other managers and/or analysts. This entire process could be repeated for all of the data views used by the marketing managers.
Marketing Gate represents an evolution of marketing systems from the MMIS to a combination of MMIS and MMKS. Marketing managers in a MMIS-only environment have to use their cognitive skills to reason with data. This reasoning process involves the application of knowledge to the information to reach conclusions.
The MMIS brings the information to the game, but the knowledge and reasoning processes are the contribution of the marketing manager. The data explosion makes this an increasingly untenable model. When the firm's knowledge is placed in a MMKS, the marketing manager's cognitive skills are augmented by an assistant or agent. This is the promise of the Marketing Gate, which improves on this model by making the knowledge dynamic. The conclusion of one reasoning process is retained in Marketing Gate; it is added to the knowledge base. The system remembers what it knows and can thus reason with its knowledge at a later date. Conclusions are not only formed as sentences, they are stored as knowledge within the system.
For instance, when Marketing Gate makes an observation using a data view, the observation, in the form of an assertion, is placed in the knowledge base and monitored in the future to insure that it is still true. Marketing Gate's reasoning is goal directed, as are most knowledge-based systems. Reasoning is directed by a goal. The goal determines the rules, frames, and other knowledge representation schemes that are brought to bear on the information and knowledge. For instance, when a marketing manager instructs Marketing Gate to write a particular paragraph, the construction of the paragraph becomes Marketing Gate's goal.
When this task is completed, Marketing Gate is ready for another goal. Such a goal can be dynamic in the sense that the result of one reasoning process can give the system a new goal. A conclusion serves a new goal to the system.
For instance, when Marketing Gate makes an observation which must be monitored, it gets a new goal: monitoring the observation. Because it may accumulate several such goals, Marketing Gate must have a facility for controlling the goal-directed reasoning processes, for deciding which goal to pursue at any point in time. In the present version, Marketing Gate presents a list of goals to the user and waits for directions. A future version could incorporate an Agenda which is a set of current goals, with each goal having a priority for being satisfied. After Marketing Gate has satisfied a goal, it could look at its Agenda to find the goal with the highest priority. During the process of satisfying this goal, it may add other goals to the Agenda, or even remove some goals. Therefore, the Agenda could become a dynamic facility for managing the Marketing Gate system.
Marketing managers working with an MMIS are involved in a data focused activity. Most of their energies are directed at understanding the data. Marketing Gate is a bridge to a different focus, a marketing focus where the emphasis is on both understanding and influencing markets. The Marketing Gate's philosophy is to leverage all the firm's marketing knowledge and make it available to marketing managers to increase their efficiency and effectiveness. Marketing Gate is built around several themes:
Our experiences in building Marketing Gate yielded the following observations:
Marketing Gate is presently a prototype of a larger system which could best be described as a shell for building knowledge-based systems based upon the principles stated in this manuscript. The term shell is used to denote a system which already knows a lot about marketing and the MMIS -- it has general knowledge and some specific knowledge into which managers and/or knowledge engineers can place additional knowledge. In this way, Marketing Gate could grow over time.