|Publication number||US7621748 B2|
|Application number||US 11/749,473|
|Publication date||Nov 24, 2009|
|Priority date||Aug 31, 1999|
|Also published as||US6736642, US6962495, US7226292, US20020072040, US20040175681, US20060073463, US20070264622|
|Publication number||11749473, 749473, US 7621748 B2, US 7621748B2, US-B2-7621748, US7621748 B2, US7621748B2|
|Inventors||Javier Bajer, Timothy Gbedemah|
|Original Assignee||Accenture Global Services Gmbh|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (91), Non-Patent Citations (84), Referenced by (1), Classifications (17), Legal Events (2)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is a continuation of pending U.S. application Ser. No. 11/258,746 filed on Oct. 26, 2005 which is a continuation of U.S. application Ser. No. 10/806,983 filed on Mar. 23, 2004 and granted as U.S. Pat. No. 6,962,495. U.S. application Ser. No. 10/806,983 is a continuation of U.S. application Ser. No. 09/935,348 filed on Aug. 22, 2001 and granted as U.S. Pat. No. 6,736,642. U.S. application Ser. No. 09/935,348 is a continuation of U.S. application Ser. No. 09/387,618 filed on Aug. 31, 1999, now abandoned. U.S. application Ser. No. 09/387,618 is a continuation of U.S. application Ser. No. 09/386,895, filed on Aug. 31, 1999, now abandoned. U.S. application Ser. No. 09/386,895 is a continuation of U.S. application Ser. No. 09/387,193, filed on Aug. 31, 1999, now abandoned. U.S. application Ser. No. 09/387,193 is a continuation of U.S. application Ser. No. 09/387,748, filed Aug. 31, 1999, now abandoned. The entire disclosures and contents of these applications are incorporated by reference herein.
The present invention relates to training software, and more particularly to training a user to validate assumptions on which his or her decisions are based.
The behaviors, assumptions and attitudes of police officers are strongly influenced by the overall ‘culture’ of the service. The police service culture has been established and reinforced over many years. Police officers naturally tend to tolerate and conform to the behaviors, assumptions and attitudes common to the culture. They may find they are most effective as police officers if they behave in the way the culture demands, or they may feel they have to do so in order to ‘fit in’ and feel comfortable around fellow officers. They may not believe themselves to be acting or thinking in accord with any culture, and the actions in question may not be overt, conscious or deliberate. That is the nature and influence of an organization's culture and the way it is shared amongst the organization's members.
This is not unique to police service. Most public institutions, and likewise private sector firms, develop a culture of their own—with prevailing behaviors, assumptions, attitudes and vocabulary amongst their employees. Such a culture can be a positive strength. It can attract people to join and stay with an organization, and it can help people work well together in performing the organization's business.
Efforts to change the weaknesses in a culture must not destroy the strengths that help give an organization its effectiveness. But a culture has a problem when some of its consequences come to be judged unacceptable. For the police service, that public judgment has been reached, and is acknowledged by senior officers and officials.
The common thread of unacceptability is a police service culture with behaviors, assumptions and attitudes that fail to accommodate the differences between individuals, or worse, that are based on stereotypes of race, sex, age, or dress. It has been stated that it was not enough for police officers to be fair and to avoid acting on stereotypes by treating everyone the same. Instead, the challenge is to treat everybody as individuals.
Whilst the nature of policing is unique, this challenge of individualizing the treatment of all members of the public is not unique to the police service. It is one that is shared by many other public servants—by tax inspectors, employment and benefits advisers, fraud investigators, and all others who must offer public services to people with individual needs and circumstances, and must do so with equity.
For police and other public services, changing behavior towards individuals requires awareness of diversity amongst members of the public. But awareness of diversity does not produce change without the breaking down of old habits of behavior and action, and without the acquisition of new skills, including:
These skills and behaviors are not easy for anyone to acquire, and it must also become a new habit for officers to apply them in both routine and exceptional circumstances on duty. Achieving this re-skilling and breaking down of old habits is at the heart of the challenge at hand.
It is a challenge similar to the one faced by many private sector companies seeking to win and retain consumer business. The nature of the diversity that needs to be recognized by a business may be different, and simpler, than that which the police service must recognize. The opportunity may exist for businesses to decide to ignore some apparently ‘unprofitable’ individuals; a moral luxury police service cannot have. And for the private sector, the consequences of failure to individualize treatment are seldom matters of life, death or basic human dignity for the individual concerned. At the center of individualized treatment is the awareness and validation of assumptions which may tend to cause the treatment of people in a generalized manner.
A system, method and article of manufacture are disclosed which implement a training session for training a user to validate assumptions. First, a plurality stimuli are presented to a user in the context of a simulation. The user is then prompted to enter responses to the stimuli which are subsequently recorded. The user is then allowed to enter assumptions that form a basis of the responses. Next, the user is prompted to enter information on how to validate the assumptions which is subsequently received. Feedback is then outputted based on the stimuli and responses. Such feedback relates to appropriate assumptions and information on how to validate the appropriate assumptions. As an option, the feedback may be further based on a number of assumptions entered.
In one embodiment, the stimuli may include a prerecorded audio clip, a prerecorded video clip, an animation, a graphic, a text presentation, a real-time audio stream, and/or a real-time video stream. Further, the user may be automatically prompted to enter the assumptions that form the basis of the responses and the information on how to validate the assumptions.
In another embodiment of the present invention, the simulation may include a police call regarding a situation with the stimuli including images of at least one person in the situation. Further, the responses may include questions for the person in order to collect evidence pertinent to the situation. The user may be prompted to enter the responses to the stimuli by listing a plurality of responses, and allowing the user to select between the listed responses.
As one aspect of the present invention, the stimuli and the responses are displayed after the receipt thereof in the form of a transcript during the course of the simulation. Similarly, the assumptions and the information on how to validate the assumptions may be displayed after the receipt thereof during the course of the simulation.
The invention will be better understood when consideration is given to the following detailed description thereof. Such description makes reference to the annexed drawings wherein:
In accordance with at least one embodiment of the present invention, a system is provided for making a user, or student, aware of assumptions being made in various environments. In addition, the user may be trained on validating such assumptions of which he or she is aware. Such assumption awareness and validation is then reinforced by reversing the role of the user from a trainee to a trainer. This is all accomplished via a unique method of interaction that is facilitated by multiple user 10 interfaces.
A representative hardware environment of a preferred embodiment of the present invention is depicted in
Object oriented programming (OOP) has become increasingly used to develop complex applications. As OOP moves toward the mainstream of software design and development, various software solutions require adaptation to make use of the benefits of OOP. A need exists for the principles of OOP to be applied to a messaging interface of an electronic messaging system such that a set of OOP classes and objects for the messaging interface can be provided.
OOP is a process of developing computer software using objects, including the steps of analyzing the problem, designing the system, and constructing the program. An object is a software package that contains both data and a collection of related structures and procedures. Since it contains both data and a collection of structures and procedures, it can be visualized as a self-sufficient component that does not require other additional structures, procedures or data to perform its specific task. OOP, therefore, views a computer program as a collection of largely autonomous components, called objects, each of which is responsible for a specific task. This concept of packaging data, structures, and procedures together in one component or module is called encapsulation.
In general, OOP components are reusable software modules which present an interface that conforms to an object model and which are accessed at run-time through a component integration architecture. A component integration architecture is a set of architecture mechanisms which allow software modules in different process spaces to utilize each other's capabilities or functions. This is generally done by assuming a common component object model on which to build the architecture. It is worthwhile to differentiate between an object and a class of objects at this point. An object is a single instance of the class of objects, which is often just called a class. A class of objects can be viewed as a blueprint, from which many objects can be formed.
OOP allows the programmer to create an object that is a part of another object. For example, the object representing a piston engine is said to have a composition relationship with the object representing a piston. In reality, a piston engine comprises a piston, valves and many other components; the fact that a piston is an element of a piston engine can be logically and semantically represented in OOP by two objects.
OOP also allows creation of an object that “depends from” another object. If there are two objects, one representing a piston engine and the other representing a piston engine wherein the piston is made of ceramic, then the relationship between the two objects is not that of composition. A ceramic piston engine does not make up a piston engine. Rather it is merely one kind of piston engine that has one more limitation than the piston engine; its piston is made of ceramic. In this case, the object representing the ceramic piston engine is called a derived object, and it inherits all of the aspects of the object representing the piston engine and adds further limitation or detail to it. The object representing the ceramic piston engine “depends from” the object representing the piston engine. The relationship between these objects is called inheritance.
When the object or class representing the ceramic piston engine inherits all of the aspects of the objects representing the piston engine, it inherits the thermal characteristics of a standard piston defined in the piston engine class. However, the ceramic piston engine object overrides these ceramic specific thermal characteristics, which are typically different from those associated with a metal piston. It skips over the original and uses new functions related to ceramic pistons. Different kinds of piston engines have different characteristics, but may have the same underlying functions associated with them (e.g., how many pistons in the engine, ignition sequences, lubrication, etc.). To access each of these functions in any piston engine object, a programmer would call the same functions with the same names, but each type of piston engine may have different/overriding implementations of functions behind the same name. This ability to hide different implementations of a function behind the same name is called polymorphism and it greatly simplifies communication among objects.
With the concepts of composition-relationship, encapsulation, inheritance and polymorphism, an object can represent just about anything in the real world. In fact, the logical perception of the reality is the only limit on determining the kinds of things that can become objects in object-oriented software. Some typical categories are as follows:
With this enormous capability of an object to represent just about any logically separable matters, OOP allows the software developer to design and implement a computer program that is a model of some aspects of reality, whether that reality is a physical entity, a process, a system, or a composition of matter. Since the object can represent anything, the software developer can create an object which can be used as a component in a larger software project in the future.
If 90% of a new OOP software program consists of proven, existing components made from preexisting reusable objects, then only the remaining 10% of the new software project has to be written and tested from scratch. Since 90% already came from an inventory of extensively tested reusable objects, the potential domain from which an error could originate is 10% of the program. As a result, OOP enables software developers to build objects out of other, previously built objects.
This process closely resembles complex machinery being built out of assemblies and sub-assemblies. OOP technology, therefore, makes software engineering more like hardware engineering in that software is built from existing components, which are available to the developer as objects. All this adds up to an improved quality of the software as well as an increase in the speed of its development.
Programming languages are beginning to fully support the OOP principles, such as encapsulation, inheritance, polymorphism, and composition-relationship. With the advent of the C++ language, many commercial software developers have embraced OOP. C++ is an OOP language that offers a fast, machine-executable code. Furthermore, C++ is suitable for both commercial-application and systems programming projects. For now, C++ appears to be the most popular choice among many OOP programmers, but there is a host of other OOP languages, such as Smalltalk, Common Lisp Object System (CLOS), and Eiffel. Additionally, OOP capabilities are being added to more traditional popular computer programming languages such as Pascal.
The benefits of object classes can be summarized, as follows:
Class libraries are very flexible. As programs grow more complex, more programmers are forced to reinvent basic solutions to basic problems over and over again. A relatively new extension of the class library concept is to have a framework of class libraries. This framework is more complex and consists of significant collections of collaborating classes that capture both the small scale patterns and major mechanisms that implement the common requirements and design in a specific application domain. They were first developed to free application programmers from the chores involved in displaying menus, windows, dialog boxes, and other standard user interface elements for personal computers.
Frameworks also represent a change in the way programmers think about the interaction between the code they write and code written by others. In the early days of procedural programming, the programmer called libraries provided by the operating system to perform certain tasks, but basically the program executed down the page from start to finish, and the programmer was solely responsible for the flow of control. This was appropriate for printing out paychecks, calculating a mathematical table, or solving other problems with a program that executed in just one way.
The development of graphical user interfaces began to turn this procedural programming arrangement inside out. These interfaces allow the user, rather than program logic, to drive the program and decide when certain actions should be performed. Today, most personal computer software accomplishes this by means of an event loop which monitors the mouse, keyboard, and other sources of external events and calls the appropriate parts of the programmer's code according to actions that the user performs. The programmer no longer determines the order in which events occur. Instead, a program is divided into separate pieces that are called at unpredictable times and in an unpredictable order. By relinquishing control in this way to users, the developer creates a program that is much easier to use.
Nevertheless, individual pieces of the program written by the developer still call libraries provided by the operating system to accomplish certain tasks, and the programmer must still determine the flow of control within each piece after it's called by the event loop. Application code still “sits on top of’ the system.
Even event loop programs require programmers to write a lot of code that should not need to be written separately for every application. The concept of an application framework carries the event loop concept further. Instead of dealing with all the nuts and bolts of constructing basic menus, windows, and dialog boxes and then making all these things work together, programmers using application frameworks start with working application code and basic user interface elements in place. Subsequently, they build from there by replacing some of the generic capabilities of the framework with the specific capabilities of the intended application.
Application frameworks reduce the total amount of code that a programmer has to write from scratch. However, because the framework is really a generic application that displays windows, supports copy and paste, and so on, the programmer can also relinquish control to a greater degree than event loop programs permit. The framework code takes care of almost all event handling and flow of control, and the programmer's code is called only when the framework needs it (e.g., to create or manipulate a proprietary data structure).
A programmer writing a framework program not only relinquishes control to the user (as is also true for event loop programs), but also relinquishes the detailed flow of control within the program to the framework. This approach allows the creation of more complex systems that work together in interesting ways, as opposed to isolated programs, having custom code, being created over and over again for similar problems.
Thus, as is explained above, a framework basically is a collection of cooperating classes that make up a reusable design solution for a given problem domain. It typically includes objects that provide default behavior (e.g., for menus and windows), and programmers use it by inheriting some of that default behavior and overriding other behavior so that the framework calls application code at the appropriate times.
There are three main differences between frameworks and class libraries:
Thus, through the development of frameworks for solutions to various problems and programming tasks, significant reductions in the design and development effort for software can be achieved. A preferred embodiment of the invention utilizes HyperText Markup Language (HTML) to implement documents on the Internet together with a general-purpose secure communication protocol for a transport medium between the client and a company. HTTP or other protocols could be readily substituted for HTML without undue experimentation. Information on these products is available in T. Bemers-Lee, D. Connoly, “RFC 1866: Hypertext Markup Language—2.0” (November 1995); and R. Fielding, H, Frystyk, T. Berners-Lee, J. Gettys and J. C. Mogul, “Hypertext Transfer Protocol—HTTP/1.1: HTTP Working Group Internet Draft” (May 2, 1996). HTML is a simple data format used to create hypertext documents that are portable from one platform to another. HTML documents are SGML documents with generic semantics that are appropriate for representing information from a wide range of domains. HTML has been in use by the World-Wide Web global information initiative since 1990. HTML is an application of ISO Standard 8879; 1986 Information Processing Text and Office Systems; Standard Generalized Markup Language (SGML).
To date, Web development tools have been limited in their ability to create dynamic Web applications which span from client to server and interoperate with existing computing resources. Until recently, HTML has been the dominant technology used in development of Web-based solutions. However, HTML has proven to be inadequate in the following areas:
Sun Microsystem's Java language solves many of the client-side problems by:
With Java, developers can create robust User Interface (UI) components. Custom “widgets” (e.g., real-time stock tickers, animated icons, etc.) can be created, and client-side performance is improved. Unlike HTML, Java supports the notion of client-side validation, offloading appropriate processing onto the client for improved performance. Dynamic, real-time Web pages can be created. Using the abovementioned custom UI components, dynamic Web pages can also be created.
Sun's Java language has emerged as an industry-recognized language for “programming the Internet.” Sun defines Java as “a simple, object-oriented, distributed, interpreted, robust, secure, architecture-neutral, portable, high performance, multithreaded, dynamic, buzzword-compliant, general-purpose programming language. Java supports programming for the Internet in the form of platform-independent Java applets.” Java applets are small, specialized applications that comply with Sun's Java Application Programming Interface (API) allowing developers to add “interactive content” to Web documents (e.g., simple animations, page adornments, basic games, etc.). Applets execute within a Java-compatible browser (e.g., Netscape Navigator) by copying code from the server to client. From a language standpoint, Java's core feature set is based on C++. Sun's Java literature states that Java is basically, “C++ with extensions from Objective C for more dynamic method resolution.”
Another technology that provides similar function to JAVA is provided by Microsoft and ActiveX Technologies, to give developers and Web designers wherewithal to build dynamic content for the Internet and personal computers. ActiveX includes tools for developing animation, 3-D virtual reality, video and other multimedia content. The tools use Internet standards, work on multiple platforms, and are being supported by over 100 companies. The group's building blocks are called ActiveX Controls, which are fast components that enable developers to embed parts of software in hypertext markup language (HTML) pages. ActiveX Controls work with a variety of programming languages including Microsoft Visual C++, Borland Delphi, Microsoft Visual Basic programming system and, in the future, Microsoft's development tool for Java, code named “Jakarta.” ActiveX Technologies also includes ActiveX Server Framework, allowing developers to create server applications. One of ordinary skill in the art readily recognizes that ActiveX could be substituted for JAVA without undue experimentation to practice the invention.
Upon execution, a start-up screen is displayed. As shown in
In one embodiment, a ‘virtual’ coach playing the role of a tutor constable may introduce the learner to the training, setting their goals and expectations for the training in an audible or visual manner. In accordance with the three main buttons set forth hereinabove, the training afforded by the present invention is divided into three sections: Awareness, Validation, and Action.
This section makes the learner cognizant of their behavior, in this case the practice of making assumptions. In the first stage of the training, the user learns that humans make assumptions all the time. The awareness section of the training is used to train the learner to become aware of the assumptions they are making
The second phase of the training shows the learner that it is important not only to be aware that they are making assumptions, but that they must validate those assumptions.
The final stage in the training is comprised of 2 parts, first of all the learner swaps roles and plays the role of a ‘tutor constable’ coaching a ‘virtual’ learner about assumption making and validation. The reason to do this is that if one refers to Bloom's Taxonomy of Educational Objectives, the ultimate way to learn is to teach someone else what one have already learned.
The second stage of the action module is involved in learning alternative new behaviors for dealing with people when existing stereotypes or assumptions no longer fit.
The learner would begin the training with the first phase, “Awareness.”
Upon selection of the Awareness button 200 of
In one embodiment, prior to presenting the second portion of the stimulus, the user is further prompted to enter an assumption that forms a basis of the user's reaction to the stimulus. Then, based on the assumption, feedback may be provided which helps the user become aware of the affects of the assumption on the response.
The user's reaction to the stimulus may include a text entry and/or a choice of presented options. Further, the form of the stimulus may range from a prerecorded audio clip, a prerecorded video clip, an animation, a graphic, a text presentation, a real-time audio stream, and/or a real-time video stream.
In the awareness portion of the present invention, the screen first changes to show a screen shown in
The purpose of the virtual coach is to welcome the learner and explain how to use the application. The coach also explains that the learner is about to take part in a series of “What happens next” scenarios. To continue the user should press the ‘continue’ button and watch the first video. Then they ask to decide what happens next.
The screen of
The user may then click on the continue button 306 of
Upon selection of one of the scenarios 402 of
Automatically, a virtual psychologist 502 appears on the screen and a voice over starts to explain “assumption making is a natural process . . . ” for identifying how the user's reaction was based on an assumption. In particular, the virtual psychologist appears to point out to that assumption making is a natural and acceptable behavior that is in use in everyday life.
Should the learner have chosen the correct assumption, the virtual psychologist's feedback would have changed to start “well done you chose the correct option, but on the whole most people would have assumed that . . . . ” The training application always provides tailored feedback to the learner, driven by their actions. It is important to at this point to assure the learner that they are doing nothing wrong by making assumptions. The learning point here is that people make assumptions but should also learn to confirm the assumptions that are made.
The learner is then prompted to try another scenario by clicking on a continue button 10 506.
By click on the continue button 506, additional stimuli may be presented in a manner similar to that set forth hereinabove in addition to providing guidance by a virtual consultant. For example, a picture of an elderly man on a beach may appear along with a title “What is an assumption?” At this time, the virtual psychologist 502 would point out at this time what constitutes an assumption, i.e. the way a person is dressed, their surroundings and personal experiences in a similar situation leads one to draw certain conclusions about a person. For example, in the present picture with the elderly man on the beach, it might be assumed that the elderly gentleman does not constitute any threat to us. One might have made that assumption because he is smiling and appears to be happily strolling along the beach. He may also have a walking stick so as not to be physically threatening.
By clicking on another continue button, the picture may change to show a younger man on the beach. Again, the virtual psychologist 502 may state that assumptions about the situation may now have changed because the person on the beach has changed, but it is still not a threatening situation.
By clicking again on another continue button, the picture may change to show the same image of the young man in a different environment, namely a riot scene. The virtual psychologist 502 may then state that the assumptions that one is making about the person are completely different. One might now assume that the scene is highly dangerous and that the young man is threatening and poses a significant danger.
The foregoing process thus aids in determining what an assumption constitutes. With that, the first portion of the video of
Again, the learner can choose whichever option for ‘what happens next?’ that they believe is most appropriate. At this point the learner is probably automatically reflecting on what assumptions they are making, and they will be aiming not be ‘tricked’ into making a false assumption again.
To further enhance learning, the language “you expected” and the chosen option then appear on screen. Thereafter, the virtual coach appears and a text box “based on what assumptions” appears, prompting the learner to type in the assumptions that they made in order to choose the option they did. In this section of the training, the purpose is to start to elevate assumption making to the meta-cognitive level, i.e. learner has started to think about the assumptions they are making.
Upon entry of the assumptions by the user, the text “Based on the assumption that” and the free text typed appears on screen. In summary, beneath the display while depicting the second portion of the video, the text “You expected (selected option) based on (entered assumption), and this happened: (description of second portion of video)” is depicted. With this textual description, the user is made aware of the existence of assumptions in his or her decision making. It should be noted that the present embodiment is not limited to a video having two portions, but rather may be a video with multiple breaks which are handled in a similar manner.
Next, if the example of the youth that saves the man is used, the psychologist 502 appears again to explain that most people would have assumed that because the youth looked aggressive—because of his clothing/haircut and actions—they would have expected him to attack the man in the suit.
Evidently this wasn't the case, and this is an important learning point—this is what can happen when assumptions are made but not validated, i.e. the wrong conclusions are drawn. It is critical that the learner learns to validate the assumptions that they make.
During the course of the training application, the user is expected to complete more ‘What happens next?” scenarios, each time inputting the assumptions that they are making about the characters and situations in the videos. This encourages the learner to think about assumption making, the assumptions that they themselves make and how those can be both correct and incorrect.
At the end of the ‘Awareness’ phase a ‘point of reflection’ may be added. ‘Points of Reflection’ are a critical part of ‘learning by practice’ since they give the user an opportunity to think about their actions—what they have learned and what they might change going forward.
The virtual coach may give the user feedback on the options that they chose throughout the scenarios and prompt the user of the necessity to validate the assumptions that were made. The virtual coach then prompts the learner to practice validating their assumptions in part two of the training, namely ‘Validation’.
When desired, the validation portion of the present invention may be accessed by clicking on the home button on a current screen and returning to the main screen of
The toolbar along the left-hand side of the screen is populated with the foregoing buttons, and may or may not be available to the user. The functions associated with the foregoing buttons will now be set forth.
As shown in
Multiple areas 604 may be highlighted on the map each corresponding to a different scenario. A toolbar 606 along the left hand side of the screen has been populated with multiple buttons 608 the details for which have been set forth hereinabove.
While the audio sample may relate to any type of stimulus, a domestic incident may be chosen as the example scenario. Once listening to the audio sample, the appropriate area on the briefing screen 602 may be selected.
Upon selection of the appropriate area on the briefing screen 602, a street scene screen 700 appears which includes a pair of doors 702 and 704, as shown in
After selecting one of the doors, a close up of a person answering the door may appear.
The interface 801 further includes a first frame 805 for displaying an image or video presentation. Also on the interface is a first text box 806, for displaying multiple choice questions relating to the presentation. A second frame 807 is included for displaying a transcript of the training session. Such transcript includes the questions selected from the first text box and answers generated based on the questions.
For illustrating comments entered by the user during the training session, a second text box 808 is provided. As will be set forth hereinafter, comments entered by the user during the training session may be entered via a pop-up window. As an option, the second text box 808 includes at least two overlaid pages in the form of a folder.
The first text box 806 includes an ‘Ask’ list box. The ‘Ask’ list box contains a number of questions that the learner can use to interview the neighbor with more questions appearing as the interview progresses. During this phase of the training, the learner asks questions and is asked to recognize when they are making assumptions, and then to validate those assumptions.
A continue button 802 is also included on the screen of
Using the foregoing interface, a method of teaching validation of assumptions is implemented.
The user is then prompted to enter responses to the stimuli in operation 882. Such responses are subsequently received and recorded in operation 884. The responses may include questions for the person in order to collect evidence pertinent to the situation. Further, the user may be prompted to enter the responses to the stimuli by listing a plurality of responses in the first text box 806 of
The user is then allowed to enter assumptions that form a basis of the responses, as indicated in operation 886. Next, the user is prompted to enter information on how to validate the assumptions which is subsequently received in operation 888. The user may be automatically prompted to enter the assumptions that form the basis of the responses and the information on how to validate the assumptions. Similar to the responses and stimuli, the assumptions and the information on how to validate the assumptions may be displayed after the receipt thereof during the course of the simulation in the second text box 808 of
Feedback is then outputted based on the stimuli and responses in operation 890. Such feedback relates to appropriate assumptions and information on how to validate the appropriate assumptions. As an option, the feedback may be further based on a number of assumptions entered.
In the present example shown in
In the case of the question selected in
The text that is entered on the pop-up window of
At the end of the scenario, the virtual coach returns and dependent upon the number of assumptions the learner identified (and optionally on the text they typed in), the user is provided feedback on their performance. The coach point outs that the learner identified some key points where assumptions and validations would need to be made.
The virtual coach also explains that the transcript of the interview along with the assumptions and validations that the learner made have now been submitted to a ‘live’ coach, who may discuss them further.
There is particular value in having a live coach or group discussion at this point, not only for the reflective value, but also as a gauge of what the learner is actually learning. The learner further has the opportunity to view a video/audio clip including an anecdote from an experienced officer relating his or her experience in a similar situation, and the mistakes they made and how they learned from the situation.
‘Lessons learned’ are an important learning tool as points of reflection for knowledge sharing; reassurance and validation of the learner's behavior. And from the trainee's point of view, if the war story is delivered by a recognized and respected Officer, they also add credence to the training.
In one example, the officer had been in a similar position to the user; the witness was a young West Indian male, and throughout the interview he would not meet the Officer's eye. However, because of the prior cultural awareness training that the officer had had, he did not make the assumption that the witness was uneasy.
Instead he concluded that the witness would not meet his eye because in West Indian culture it is a sign of respect to not directly meet the eye of a senior.
When desired, the action portion of the present invention may be accessed by clicking on the home button on a current screen and returning to the main screen of
The Action portion of the present invention provides a computer implemented training session for training a user on assumptions by reversing a role of the user from a trainee to a trainer. An outline of such session is shown in
Then, in operation 904, the virtual trainee is showed giving responses to the stimuli during the simulation. At this point, the user is allowed to enter assumptions that form a basis of the responses of the virtual trainee, as indicated in operation 906. The user is then prompted to enter information on how to validate the assumptions in operation 908.
The assumptions and the information on how to validate the assumptions may be displayed after the receipt thereof during the course of the simulation. As an option, the user may be automatically prompted to enter the assumptions that form the basis of the responses and the information on how to validate the assumptions.
Finally, feedback is outputted in operation 910. During the course of the simulation, the user may be given access to a knowledge base including information on assumptions and validation. A sample screen 950 for the action portion of the present invention is shown in
The learner now has the opportunity to remediate a virtual learner on their performance in a scenario similar to that of the validation portion of the invention and practice any new behavior they have learned. In the previous situations in the validation portion of the present invention, the coach explains the learner's role and goal to them. In this case, however, the user now change roles and plays the part of a coach to the new ‘virtual’ user. The new virtual user walks through a similar scenario to the one that the learner has just completed. However, this time it is up to the learner to point out to the virtual officer the points in the interview when they are probably making assumptions and the likely avenues for validation that they might try. Learning by teaching is the only more effective way of learning other than ‘learning by practice’.
And at the end of the present section, the coach returns and encourages the learner to reflect on what they have learned. Now that the learner has become aware of assumption making and the necessity for validating their assumptions, they must also learn the appropriate new behaviors for when they find they have made erroneous assumptions. For example, having taught the learner that the way they were behaving with an elderly Muslim gentleman was inappropriate they must relearn what is acceptable behavior.
The end of the session would be an ideal time for the user to analyze real scenarios from their early training, offline with their ‘Tutor Constable’.
Observations for the Police Service on Changing Behavior and Culture in Business
An organization's culture is not very susceptible to change through conventional training. Where new behaviors are taught, they do not survive well outside the classroom. Where new policies and procedures are taught they do not make an impact on the underlying behaviors, assumptions or attitudes. This is for several reasons:
Each of these techniques is helpful to the process of changing culture and behavior, but the most successful organizations are those which are also applying principles from cognitive psychology to their training and personal development programs. These principles lead to programs that are vastly more effective than conventional training at changing behaviors, assumptions, and beliefs amongst staff.
5 Principles from Cognitive Psychology
It has been found that the most effective and successful programs for adult training and personal development meet five design principles from cognitive psychology. In layman's terms:
People learn best when they attempt to defend the stances they take, justify the decisions they make, and understand how these relate to the consequences they find. Making errors is as instructive as getting it right, if the objective is to avoid those errors in future. All this is what happens when people experience a task for themselves, rather than being told about doing it, or watching someone else do it.
Conventional training styles (such as lectures, audio-visuals, demonstration and discussion) typically result in around 20% retention of information. Up to 80% of the information originally taught is not retained or retrievable by a trainee when needed. By contrast, ‘learning by doing’ typically results in 75% to 80% retention.
This principle is already recognized by most organizations, including the police service. Learning ‘on the job’ provides the experience by which many people learn the 80% of their job, trade or profession that was not taught by their conventional training.
Role plays are an attempt to follow this design principle to enrich conventional training—but too many role play courses involve many of their trainees only as observers for much of the time, or are too heavily scripted, for those trainees actually to ‘learn by doing’. Such role plays become little more than structured opportunities to ‘observe and discuss’.
First-hand experience of doing and behaving in the new way, and of the adverse consequences of behaving in the old way, is essential in instilling new behaviors.
Feedback, Support and Coaching
Roger Schank at the Institute of Learning Sciences has identified that people learn best when they can draw on expert advice and ‘war stories’ to help them to understand the problem being posed and to explore possible responses.
Positive feedback (‘reinforcement’) should be given to the learner when desired behaviors are used. Constructive feedback (‘criticism’), including an explanation of why adverse consequences have followed, should be given when the wrong behaviors are used. Both should be given in a manner that is acceptable to the learner. This level of support requires sophisticated and responsive feedback mechanisms, as well as expert coaches to deliver the feedback well.
An expert should follow the progress of someone who is being trained to ensure that they are experiencing the right behaviors, not the wrong ones, and to give immediate advice and redirection if not. This is how apprenticeships “at the feet of a master” have worked so well at passing skills down the centuries. It is also why organizations in which new people work closely with ‘old hands’ can have persistent cultures, for good and bad.
“Practice Makes Perfect”
Michael Howe, professor of psychology at Exeter University, has looked at outstanding performances in the arts and sport to establish what produces excellence. He concluded that opportunity, encouragement, motivation, self-confidence but—most of all—practice, determine excellence. The more a learner repeats an exercise, the more skilled and proficient he or she is likely to become. Practice is fundamental to the development of proficiency, in mental as well as physical disciplines, and therefore in behaviors.
Practice aids memory through repetition. If one says the same twenty numbers repeatedly he or she will eventually learn them by heart. Many people learned their “tables” and the alphabet this way. To do so, teachers probably broke the numbers down into groups whose rhythm or pattern helped people remember the sequence.
Practicing can boil much more complex tasks than knowing multiplication tables down into manageable patterns that aid memory.
A police officer who has to decide what to do in complex situations will, during his or her years of service, have developed patterns to simplify those situations. These patterns need to contain the relevant information to trigger the desired new behaviors. Achieving this takes repetition—i.e. extensive practice—in which the new patterns and behaviors are practiced, and the old ones deliberately ignored. Imagine learning new tables where “5 times 9” is no longer “45”, etc. Then take a test on them in the dark and in potential danger.
Context is Critical
Insufficiently relevant practice is useless. Similarity between the context in which the practice takes place and that where the real-life behavior must occur is essential. Information is stored in the brain with the context in which it was learnt. This context is used when trying to retrieve the information from memory into order to use it. It is part of the brain's index, regardless of whether the information stored is a new acquaintance's name, a chronology of the kings and queens of England, or the skill of dealing with an individual as an individual.
Research into the memory of elderly people has recently been carried out using scents to recreate the context in which information was stored in memory years ago. The scent of Blitz bombing was so effective in helping one man recall information that he could reel off the serial number of the rifle he used in the war, which he had not seen since.
Retrieving learned skills when they are needed in a different environment to that where they were learned is far more difficult. Ballet dancers may practice their individual movements in a classroom but to practice expert performance they take part in dress rehearsals, in full costume and on stage under the theatre lights, in front of an audience of colleagues. This simulates the context of the real performance and they practice performing. On opening night, they can reproduce this practice because the context is similar.
The context relevant to practice includes internal factors, not just similarity of the physical environment. Internal factors can be more important than the physical environment. Sportsmen visualize competition to create the mental context in which they can reproduce winning performances. Hypnotherapy helps people to recreate past contexts that can allow them to remember information that is otherwise inaccessible.
Conventional training seldom creates any relevant similarity between the context of the training—either the physical class—or lecture—room, or the mental state of the trainee—and the environments and circumstances in which the lessons learned have to be retrieved and applied. Analyzing these environments and designing a relevant context in which lessons should be learned is essential.
Motivation and Goals
People learn more readily if they are interested in something:
If one has a child who plays computer games, it may be considered how much more quickly proficiency is achieved in the more interesting games and how much longer those games are played. The most commercially successful computer games present goals that interest and engage, to the point of addiction. Training is a far more purposeful activity than computer gaming, and the skills learned in computer games are usually of little transferable value, but the principle of motivation is similar.
A ‘goal-based exercise’ is a problem presented in way that motivates a learner, by giving him or her a goal to achieve within a learning exercise. The goal encourages persistence in the exercise, whilst exploring and solving the problem within the exercise provides experience, practice if repeated, and therefore learning.
5 Practical Constraints on Training within the Police Service
Adult training and personal development programs that aim to change behavior are always subject to constraints that arise from the organization within which the program delivered, and from the nature of the job that trainees are doing.
For the police service:
The 5 principles and 5 constraints above should be recognized in building any training and personal development program that aims to change behaviors within the police service. To do so requires considerable professional expertise to be applied to the analysis and design of the coaching, the context, and the goals. The outcome of that expert design process could be constructed as training and development material in one of three possible forms:
Materials in these forms have been successfully employed on programs to change behavior and culture, in the business and voluntary sectors. The program aims have been as varied as combating alcohol dependency, increasing managerial decisiveness, converting instructors to effective coaches, introducing team-working, and effective negotiating. Each form of material has strengths and weaknesses when measured against the 5 principles and the practical constraints within the police service. But the generic design process by which material should be developed is established, proven and in use. This process, and the professional expertise that is required to perform it, can be shared with the public sector.
‘On the Job’ Training
‘On the job training’ would involve an officer being coached in new behaviors for police service by an expert, whilst on duty in real-life situations and surrounded by fellow officers and the public. Against each of the 5 principles:
For the police service, ‘on the job training’ would be highly dependent on the subjective interventions of the coaches who were available, and on enough opportunity for practice occurring during the routine duty of the training period. As a result, it is likely to deliver variable messages to trainees, and these messages are unlikely to have a lasting impact on them. To deliver in a reasonable period, ‘on the job training’ would also require more expert coaches and role models than are likely to be available. Without them, it would be unlikely to deliver much of a message in the first place.
Staged simulation involves role playing in a setting that mimics real-life incidents, using actors to create scenarios in which the trainee plays his or her own role. It differs from simple role playing in that the balance of scripting and the ability of the actors to improvise must allow the trainee a lot of freedom to alter the course of events by his or her behavior.
Against the 5 Principles
Staged simulation is an effective way of delivering personal development training to compact audiences. However, for the police service it has very substantial logistical difficulties, particularly in providing enough capacity for every trainee to obtain adequate practice. Staged simulation also depends heavily upon the availability of sufficient numbers of capable actors. And the actors can subjectively influence the messages delivered since they must be allowed the latitude for improvisation yet they control the scope and content of the trainees' experience. Staged simulation is likely to be scalable for the police service.
Practice-Based Development Packages
Practice-based development packages provide trainees with multi-media computer software to perform exercises in a ‘virtual world’ simulation—rather like a flight simulator with less hardware and more psychology. They combine repeatability of each exercise, with the mass reproduction of the best expertise for all trainees, with the ability of software to respond from a repertoire rather than either from an overly simple script or by free improvisation.
Against the 5 Principles
Practice-based development packages on multi-media are an effective way of delivering personal development training, and are scalable to the extent needed by the police service. They can provide every trainee with first-hand experience, freedom to practice, and coaching ‘on demand’.
Such packages also eliminate subjectivity from the delivery—once material has been developed, it reaches every trainee precisely as “signed off.” And once released its logistics allow the program to be completed by everybody in a short period.
Practice-based development packages should be the core material in any training and personal development program that aims to change behaviors and culture within the police service. The full content of a program with this at its core might include:
The design of a program in accord with the principles and constraints discussed in this submission is a professional discipline. So is the creation of multi-media software. An effective program cannot be designed without expert use of the know-how and methods of both, together with professional knowledge of the nature of policing and consensus amongst policy-makers on the specific behaviors to be trained.
None of the proposals advanced in this submission is intended to address so-called “bad apples”—the community of those who exploit and extrapolate the current culture, as opposed to simply tolerating and conforming to it. But if the culture is changed for all around them, then those around them and their leaders can more readily isolate and deal with this community.
While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4622013||May 21, 1984||Nov 11, 1986||Interactive Research Corporation||Interactive software training system|
|US4874784||Feb 17, 1989||Oct 17, 1989||Ethyl Corporation||Quaternary ammonium compounds|
|US4931950||Jul 25, 1988||Jun 5, 1990||Electric Power Research Institute||Multimedia interface and method for computer system|
|US4964077||Oct 6, 1987||Oct 16, 1990||International Business Machines Corporation||Method for automatically adjusting help information displayed in an online interactive system|
|US4977529||Feb 23, 1973||Dec 11, 1990||Westinghouse Electric Corp.||Training simulator for a nuclear power plant|
|US4981766||Sep 6, 1989||Jan 1, 1991||Canon Kabushiki Kaisha||Light receiving member having a multilayered light receiving layer composed of a lower layer made of aluminum-containing inorganic material and an upper layer made of a non-single-crystal silicon material|
|US5002491||Apr 28, 1989||Mar 26, 1991||Comtek||Electronic classroom system enabling interactive self-paced learning|
|US5170464||Nov 15, 1991||Dec 8, 1992||International Business Machines Corporation||Method for rolling back an expert system|
|US5189402||May 24, 1989||Feb 23, 1993||Advanced Interaction, Inc.||Content addressable video system for image display|
|US5208745||Jun 4, 1990||May 4, 1993||Electric Power Research Institute||Multimedia interface and method for computer system|
|US5208898||May 30, 1989||May 4, 1993||Hitachi, Ltd.||Adaptive knowledge inference method and system|
|US5239617||Jan 5, 1990||Aug 24, 1993||International Business Machines Corporation||Method and apparatus providing an intelligent help explanation paradigm paralleling computer user activity|
|US5259755||Jul 31, 1992||Nov 9, 1993||Hauck Manufacturing Company||Combination burner with boost gas injection|
|US5267865||Feb 11, 1992||Dec 7, 1993||John R. Lee||Interactive computer aided natural learning method and apparatus|
|US5310349||Apr 30, 1992||May 10, 1994||Jostens Learning Corporation||Instructional management system|
|US5311422||Jun 28, 1990||May 10, 1994||The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration||General purpose architecture for intelligent computer-aided training|
|US5317688||Feb 26, 1993||May 31, 1994||Hewlett-Packard Company||Software agent used to provide instruction to a user for a plurality of computer applications|
|US5326270||Aug 29, 1991||Jul 5, 1994||Introspect Technologies, Inc.||System and method for assessing an individual's task-processing style|
|US5327507||Apr 4, 1991||Jul 5, 1994||Sharp Kabushiki Kaisha||Headphone apparatus|
|US5359701||Feb 17, 1993||Oct 25, 1994||Hitachi, Ltd.||Expert system development support system and expert system environment utilizing a frame processing tecnique|
|US5395243||Jun 20, 1994||Mar 7, 1995||National Education Training Group||Interactive learning system|
|US5441415||Dec 6, 1993||Aug 15, 1995||John R. Lee||Interactive computer aided natural learning method and apparatus|
|US5491743||May 24, 1994||Feb 13, 1996||International Business Machines Corporation||Virtual conference system and terminal apparatus therefor|
|US5533903||Jun 6, 1994||Jul 9, 1996||Kennedy; Stephen E.||Method and system for music training|
|US5535422||Oct 20, 1994||Jul 9, 1996||International Business Machines Corporation||Interactive online tutorial system for software products|
|US5537141||Apr 15, 1994||Jul 16, 1996||Actv, Inc.||Distance learning system providing individual television participation, audio responses and memory for every student|
|US5539869||Sep 28, 1992||Jul 23, 1996||Ford Motor Company||Method and system for processing and presenting on-line, multimedia information in a tree structure|
|US5566291||Jun 6, 1995||Oct 15, 1996||Diacom Technologies, Inc.||Method and apparatus for implementing user feedback|
|US5576844||Sep 6, 1994||Nov 19, 1996||Unilearn, Inc.||Computer controlled video interactive learning system|
|US5577186||Oct 12, 1995||Nov 19, 1996||Mann, Ii; S. Edward||Apparatus and method for providing a generic computerized multimedia tutorial interface for training a user on multiple applications|
|US5597312||May 4, 1994||Jan 28, 1997||U S West Technologies, Inc.||Intelligent tutoring method and system|
|US5616033||Aug 3, 1994||Apr 1, 1997||Kerwin; Patrick A.||Speed learning system computer based training|
|US5644686||Apr 29, 1994||Jul 1, 1997||International Business Machines Corporation||Expert system and method employing hierarchical knowledge base, and interactive multimedia/hypermedia applications|
|US5644727||Dec 6, 1994||Jul 1, 1997||Proprietary Financial Products, Inc.||System for the operation and management of one or more financial accounts through the use of a digital communication and computation system for exchange, investment and borrowing|
|US5673369||Mar 2, 1995||Sep 30, 1997||International Business Machines Corporation||Authoring knowledge-based systems using interactive directed graphs|
|US5690496||Aug 8, 1996||Nov 25, 1997||Red Ant, Inc.||Multimedia product for use in a computer for music instruction and use|
|US5696885||Dec 27, 1996||Dec 9, 1997||International Business Machines Corporation||Expert system and method employing hierarchical knowledge base, and interactive multimedia/hypermedia applications|
|US5701400||Mar 8, 1995||Dec 23, 1997||Amado; Carlos Armando||Method and apparatus for applying if-then-else rules to data sets in a relational data base and generating from the results of application of said rules a database of diagnostics linked to said data sets to aid executive analysis of financial data|
|US5710007||Sep 29, 1995||Jan 20, 1998||Luderer; Albert A.||Methods for diagnosing prostatic adenocarcinoma|
|US5727161||Sep 16, 1994||Mar 10, 1998||Planscan, Llc||Method and apparatus for graphic analysis of variation of economic plans|
|US5727950||May 22, 1996||Mar 17, 1998||Netsage Corporation||Agent based instruction system and method|
|US5745652||May 31, 1995||Apr 28, 1998||International Business Machines Corporation||Adaptive resource allocation using neural networks|
|US5772446||Sep 19, 1995||Jun 30, 1998||Rosen; Leonard J.||Interactive learning system|
|US5779468||Apr 4, 1996||Jul 14, 1998||Bayer Aktiengesellschaft||Device for supplying gas to rotary kilns|
|US5788508||Jun 7, 1995||Aug 4, 1998||John R. Lee||Interactive computer aided natural learning method and apparatus|
|US5791907||Mar 8, 1996||Aug 11, 1998||Ramshaw; Bruce J.||Interactive medical training system|
|US5799151||Jul 24, 1995||Aug 25, 1998||Hoffer; Steven M.||Interactive electronic trade network and user interface|
|US5799292||Sep 25, 1995||Aug 25, 1998||International Business Machines Corporation||Adaptive hypermedia presentation method and system|
|US5806056||Dec 27, 1996||Sep 8, 1998||International Business Machines Corporation||Expert system and method employing hierarchical knowledge base, and interactive multimedia/hypermedia applications|
|US5810747||Aug 21, 1996||Sep 22, 1998||Interactive Remote Site Technology, Inc.||Remote site medical intervention system|
|US5822745||Dec 27, 1996||Oct 13, 1998||International Business Machines Corporation|
|US5823781||Jul 29, 1996||Oct 20, 1998||Electronic Data Systems Coporation||Electronic mentor training system and method|
|US5823788||Nov 13, 1995||Oct 20, 1998||Lemelson; Jerome H.||Interactive educational system and method|
|US5835683||Jan 12, 1995||Nov 10, 1998||International Business Machines Corporation||System and method for authoring an expert system|
|US5868575||May 20, 1996||Feb 9, 1999||Kuczewski; Robert M.||Cooperative/interactive learning system for logic instruction|
|US5870768||Jan 10, 1997||Feb 9, 1999||International Business Machines Corporation|
|US5875437||Apr 15, 1997||Feb 23, 1999||Proprietary Financial Products, Inc.||System for the operation and management of one or more financial accounts through the use of a digital communication and computation system for exchange, investment and borrowing|
|US5889845||Nov 1, 1996||Mar 30, 1999||Data Race, Inc.||System and method for providing a remote user with a virtual presence to an office|
|US5893123||Jun 22, 1995||Apr 6, 1999||Tuinenga; Paul W.||System and method of integrating a spreadsheet and external program having output data calculated automatically in response to input data from the spreadsheet|
|US5911581||Feb 27, 1997||Jun 15, 1999||Braintainment Resources, Inc.||Interactive computer program for measuring and analyzing mental ability|
|US5974446||Oct 24, 1996||Oct 26, 1999||Academy Of Applied Science||Internet based distance learning system for communicating between server and clients wherein clients communicate with each other or with teacher using different communication techniques via common user interface|
|US5987443||Dec 22, 1998||Nov 16, 1999||Ac Properties B. V.||System, method and article of manufacture for a goal based educational system|
|US6003021||Dec 22, 1998||Dec 14, 1999||Ac Properties B.V.||System, method and article of manufacture for a simulation system for goal based education|
|US6015348||Oct 18, 1996||Jan 18, 2000||Starwave Corporation||Scalable game server architecture|
|US6015486||May 13, 1998||Jan 18, 2000||Sanden Corporation||Dispensing apparatus capable of avoiding variation in dispensed amount and improving a sanitary condition|
|US6018730||Dec 22, 1998||Jan 25, 2000||Ac Properties B.V.||System, method and article of manufacture for a simulation engine with a help website and processing engine|
|US6018731||Dec 22, 1998||Jan 25, 2000||Ac Properties B.V.||System, method and article of manufacture for a goal based system utilizing a spreadsheet and table based architecture|
|US6018732||Dec 22, 1998||Jan 25, 2000||Ac Properties B.V.||System, method and article of manufacture for a runtime program regression analysis tool for a simulation engine|
|US6023691||Dec 22, 1998||Feb 8, 2000||Ac Properties B.V.||Goal based stimulator utilizing a spreadsheet architecture|
|US6023692||Dec 22, 1998||Feb 8, 2000||Ac Properties B.V.||Goal based tutoring system with behavior to control flow of presentation|
|US6026386||Dec 22, 1998||Feb 15, 2000||Ac Properties B.V.||System, method and article of manufacture for a goal based educational system with support for dynamic media control|
|US6029156||Dec 22, 1998||Feb 22, 2000||Ac Properties B.V.||Goal based tutoring system with behavior to tailor to characteristics of a particular user|
|US6029158||Dec 22, 1998||Feb 22, 2000||Ac Properties B.V.||System, method and article of manufacture for a simulation enabled feedback system|
|US6029159||Dec 22, 1998||Feb 22, 2000||Ac Properties B.V.||System, method and article of manufacture for a simulation enabled accounting tutorial system|
|US6032141||Dec 22, 1998||Feb 29, 2000||Ac Properties B.V.||System, method and article of manufacture for a goal based educational system with support for dynamic tailored feedback|
|US6064998||Dec 22, 1998||May 16, 2000||Ac Properties, B.V.||System, method and article of manufacture for a simulation engine with an expert system example processing engine|
|US6067537||Dec 22, 1998||May 23, 2000||Ac Properties B.V.||System, method and article of manufacture for a goal based educational system with support for dynamic personality feedback|
|US6067538||Dec 22, 1998||May 23, 2000||Ac Properties B.V.||System, method and article of manufacture for a simulation enabled focused feedback tutorial system|
|US6073127||Dec 22, 1998||Jun 6, 2000||Ac Properties B.V.||System, method and article of manufacture for a goal based system with dynamic feedback information|
|US6085184||Dec 22, 1998||Jul 4, 2000||Ac Properties B.V.||System, method and article of manufacture for a dynamic toolbar in a tutorial system|
|US6101489||Dec 22, 1998||Aug 8, 2000||Ac Properties, B.V.||System, method and article of manufacture for a goal based system utilizing a time based model|
|US6125358||Dec 22, 1998||Sep 26, 2000||Ac Properties B.V.||System, method and article of manufacture for a simulation system for goal based education of a plurality of students|
|US6134539||Dec 22, 1998||Oct 17, 2000||Ac Properties B.V.||System, method and article of manufacture for a goal based education and reporting system|
|EP0689132A2||May 23, 1995||Dec 27, 1995||International Business Machines Corporation||Visualizing object-oriented software|
|EP0710942A2||Oct 27, 1995||May 8, 1996||AT&T GLOBAL INFORMATION SOLUTIONS INTERNATIONAL INC.||A computer-assisted system for instructing students|
|EP0798655A2||Mar 13, 1997||Oct 1, 1997||Sun Microsystems, Inc.||Internet-enabled graphical user interface with toolbar icons having built-in links to world-wide web documents and an integrated web browser|
|WO1997044766A1||May 22, 1997||Nov 27, 1997||Agent Based Curricula, Inc.||Agent based instruction system and method|
|WO1998003953A2||Jul 23, 1997||Jan 29, 1998||Avalon Information Technologies Inc.||Method of interactive computer based instruction|
|WO1998025251A1||Dec 1, 1997||Jun 11, 1998||Chi Fai Ho||A learning method and system based on questioning|
|WO1998032109A1||Jan 21, 1998||Jul 23, 1998||B.V. Uitgeverij En Boekhandel W.J. Thieme & Cie.||Self-tuition apparatus|
|WO2000004478A2||Jul 16, 1999||Jan 27, 2000||The Franklin Institute||A system containing a multi-user virtual learning environment|
|1||"A Browser-Based System to Support & Deliver DE," 28th Annual Frontiers in Education Conference, Conference Proceedings, vol. 1, Nov. 4-7, 1998.|
|2||"A Fuzzy Logic-Based Intelligent Tutoring System," Information Processing 92, Vol. II, pp. 66-72, Dec. 1992.|
|3||"A Goal-Centered Architecture for Intelligent Tutoring Systems," Proc. Of 7th World Conf. on Artificial Intelligence in Education, pp. 307-314, Aug. 1995.|
|4||"A Role for AI in Education: Using Technology to Reshape Education", Northwestern University, The Institute for the Learning Sciences, Journal of Artificial Intelligence in Education, Winter 1990, Jan. 1990, pp. 1-24 and 2 pgs. of references.|
|5||"A Special Section-Goal Based Scenarios: A New Approach to Professional Education: Reengineering Education At Andersen Consulting," Educational Technology, Nov.-Dec. 1994.|
|6||"ACM Transactions on Information Systems," 1998 at www.acm.org/pubs/toc/Abstracts/tois/59298.html.|
|7||"An Electronic Infrastructure for a Virtual University," Communications of the ACM, vol. 40, No. 9, Sep. 1997.|
|8||"An Object-Oriented Architecture for Evolutional Development of Interactive Learning Environment with Coached Problem-Solving," Proc. Of 1997 World Conf. On Artificial Intelligence in Education, pp. 592-594, Dec. 1997.|
|9||"Applications of BrainMaker Neural Network to stocks, business, medical, manufacturing" pp. 1-63, online at http://www.calsci.com/Applications.html, retrieved online on Jan. 17, 2001.|
|10||"Architecture of an Intelligent Tutoring System on the WWW," Proc. Of 1997 World Conf. on Artificial Intelligence in Education, pp. 39-46 Dec. 1997.|
|11||"Automate Your Business Plan" 1996-2001, pp. 1-4, retrieved online at http://www.business-plan.com/screen2.html, retrieved online on Jul. 18, 2001.|
|12||"Automated Exercise Progression in Simulation-Based Training," IEEE Trans. On Systems, Man, and Cybernetrics, vol. 24(6), pp. 863-874, Jun. 1994.|
|13||"Award-Winning Kiplinger TaxCut Now Shipping," 1998, pp. 1-2, retrieved online at http://www.taxcut.com/taxcut/98press-releases/pr98-nowshipping.html, retrieved online on Jan. 17, 2001.|
|14||"Brainmaker (Dos, MAC, Win) Cat. #051-S04-32" retrieved online at http://www.npiec.on.ca/~echoscan/28-04.htm, retrieved online on Jul. 18, 2001.|
|15||"Bridging the Virtual and the Physical: The InterSim as a Collaborative Support Interface," Proc. of 1997 World Conf. on Artificial Intelligence in Education, pp. 556-558, Dec. 1997.|
|16||"Conducting and Supporting a Goal-Based Scenario Learning Environment," Educational Technology, Nov.-Dec. 1994.|
|17||"DDD-A Free Graphical Front-End for UNIX Debuggers," Jan. 1996, ACM Sigplan Notices, vol. 31, No. 1, pp. 22-27.|
|18||"Decision Pro3.0" 1999, pp. 1-10, retrieved online at http://www.vanguardsw.com/, retrieved online on Jan. 17, 2001.|
|19||"Developing a Design System into an Intelligent Tutoring System," Int'l J. Engr. Eud., vol. 13(5), pp. 341-346, Dec. 1997.|
|20||"Development of a Simulation-Based Intelligent Tutoring System for Assisting PID Control Learning," Jan. 1994, IEICE Transactions on Information and Systems,E77-D, No. 1, Tokyo, Japan, pp. 108-117.|
|21||"Development of an integrated Simulator and Real Time Plant Information System," Advances in Operational Safety of Nuclear Power Plants, Proceedings of an International Symposium 1996.|
|22||"Distributed Intelligent Tutoring on the Web," Proc. of 1997 World Conf. On Artificial Intelligence in Education, pp. 482-489, Dec. 1997.|
|23||"Eight Goal-Based Scenario Tools", Technical Report #67, Northwestern University, The Institute for the Learning Sciences, Jan. 1996, pp. 1-37.|
|24||"Embedding an Intelligent Tutoring System in a Business Gaming-Simulation Environment," Proc. Of the 1994 Winter Simulation Conference, pp. 1399-1406, Dec. 1994.|
|25||"Engines for Education" URL: http://www.ils.nwu.edu/~e-for-e/nodes/I-M-INTRO-ZOOMER-pg.html; viewed Feb. 15, 1999.|
|26||"Enhancing Simulation Eeducation with Intelligent Tutoring Systems," Proc. of the 1996 Winter Simulation Conf., pp. 675-680, Dec. 1996.|
|27||"From Computer-Assisted Instruction to Intelligent Tutoring Systems," J. Artificial Intelligence in Education, V. 2(3), pp. 39-50, Dec. 1997.|
|28||"Goal-Based Scenarios and Business Training: A Conversation with Roger C. Schank," Educational Technology, Nov.-Dec. 1994.|
|29||"Goal-Based Scenarios and the Problem of Situated Learning: A Commentary on Andersen Consulting's Design of Goal-Based Scenarios," Education Technology, Nov.-Dec. 1994.|
|30||"Goal-Based Scenarios", Technical Report #36, Northwestern University, The Institute for the Learning Sciences, Dec. 1992, pp. 1-30.|
|31||"Intelligent Computer-Aided Instruction: A Survey Organized Around System Components," Jan. 1989, IEEE Inc., New York, vol. 49, No. 1, pp. 40-57.|
|32||"Interval Spreadsheet for Financial Planning," Projects: FinPlan System, retrieved online at http://www.rriai.org.ru/FinPlan/, retrieved online on Jan. 17, 2001.|
|33||"KBLPS Overview" pp. 1-13, online at http://www.cgi.com/CGIWEB/KBLPS/overindex4.html, retrieved online on Aug. 12, 1999.|
|34||"Learning with Computers," May 1994, Accountancy vol. 113, No. 1209, pp. 60-64.|
|35||"MUSE Patents," retrieved online at http://www.muser.com/html/patents.html, retrieved on Jan. 17, 2001.|
|36||"News at ESAP" retrieved online at http;//www.hops.wharton.upenn.edu/~esap/news.html, retrieved online on Aug. 9, 1999.|
|37||"Pedagogical, natural language and knowledge engineering techniques in Sophie I, II, and III," in Intelligent Tutoring Systems, D. Sleeman & J.S. Brown eds., pp. 227-282, Dec. 1982.|
|38||"Persistent issues in the Application of Virtual Environment Systems to Training," Aug. 1996, Proceedings, Third Annual Symposium on Human Interaction with Complex Systems, IEEE, pp. 124-132.|
|39||"RadTutor: The Theoretical and Empirical Basis for the Design of a Mammography Interpretation Tutor," Proc. of 1997 World Conf. on Artificial Intelligence in Education, pp. 386-393, Dec. 1997.|
|40||"Smart Avatars in JackMOO," Proceedings of the 1999 IEEE Conference on Virtual Reality, pp. 156-163.|
|41||"SMART Evaluation: Cognitive Diagnosis, Mastery Learning & Remediation," Proc.Of 7th World Conf. on Artificial Intelligence in Education, pp. 123-130, Aug. 1995.|
|42||"Socialized Collaborative Learning in Multimedia Virtual Worlds" URL: http://www.iscs.nus.edu.sg/labs/learning/lels/VRML.html; viewed Feb. 16, 1999.|
|43||"Task-Oriented Learning on the Web", Innovations in Education and Training International, vol. 36, No. 1, Feb. 1999.|
|44||"Teaching Through Case-Based Reasoning: An ITS Engine Applied to Business Communication," Proc. of 1997 World Conf. On Artificial Intelligence in Education, pp. 111-118 Dec. 1997.|
|45||"Teaching with the internet" 1998, JAI Press Inc., USA vol. No. 3, pp. 217-222.|
|46||"The Lisp Tutor," Byte, pp. 159-175, Apr. 1985.|
|47||"The SimQuest Authoring System for Simulation-Based Discovery Learning," Proc. of 1997 World Conf. On Artificial intelligence in Education, pp. 79-86, Dec. 1997.|
|48||"The Virtual Learning Environment System," 28th Annual Frontiers in Education Conference, Conference Proceedings, vol. 2, Nov. 4-7, 1998.|
|49||"Train with Less Pain," Oct. 13, 1997, Informationweek No. 652, pp. 150-154.|
|50||"TurboTax Deluxe Product Information" 1999-2000, pp. 1-2, retrieved online at http://www.intuit.com/turbotax/prodinfo/ttdlx.html, retrieved online on Jan. 17, 2001.|
|51||"User-Sensitive Multimedia Presentation System," Mar. 1, 1996, vol. 39, No. 3, pp. 93-94.|
|52||"Using Planning Techniques to Provide Feedback in Interactive Learning Environments," Proc. Sixth Int'l Conf. on Tools with Artificial Intelligence, pp. 700-703, Nov. 1994.|
|53||"Using the Wizard of Oz Technique to Prototype a Scenario-Based Simulation Tutor," Proc. of 1997 World Conf. On Artificial Intelligence in Education, pp. 458-465, Dec. 1997.|
|54||"Virtual Learning: A Revolutionary Approach to Building a Highly Skilled Workforce," Autumn 1998, Personnel Psychology vol. 51, No. 3, pp. 767-771.|
|55||"Why Should The Teens Have All the Best Games? Management Skill with Oil, Health, Housing Games," Computergram Int'l, Jun. 176, 1996.|
|56||Carroll, John M. et al., "Interface design issue for advicing giving expert systems", Communications of the ACM, 1987, pp. 14-31, vol. 30, No. 1.|
|57||Cleary, Chip et al., "Practical methods for automatically generating typed links", 1996, pp. 31-41, Hypertext.|
|58||Computer Dictionary, 1997, pp. 264, 276, 383, 446, 462, 507.|
|59||Cooper, Thomas et al., "Rule-Based Programming with OPS5", retrieved online at http://www.mkp.com/books-catalog/O-934613-51-6.asp, retrieved online on Aug. 9, 1999.|
|60||Evaluating Intelligent Tutoring with Gaming Simulations, Proc. Of the 1995 Winter Simulation Conf., pp. 1376-1383, Dec. 1995.|
|61||Flexible Lerning, Feb. 1998, Credit Union Management vol. 21, No. 2, pp. 32-33+.|
|62||Foster, David A., "Teaching Real-World Analysis Skills for Goal-Based Scenario," pp. 68-74, retrieved on Jul. 13, 2001.|
|63||Hafner, Carole D., "Smartlaw: Adapting "Classic" Expert System Techniques for the Legal Research Domain," 1993, pp. 133-141, Association for Computing Machinery.|
|64||Interactive Multimedia Distance Learning (IMDL): The Prototype of the Virtual Classroom, 1994.|
|65||Keys, J. Bernard et al., "Microworlds and Simuworlds: Practice Fields for the Learning Organization," 1996, Organizational Dynamics, pp. 36-49.|
|66||Manzoni, Jean-Francois, "Understanding Organizational Dynamics of IT-Enabled Change: A Multipedia Simulation Approach," Journal of Management Information Systems, Winter 1997/1998, pp. 109-140, vol. 14, No. 3.|
|67||McArthur, David et al. "The Roles of Artificial Intelligence in Education: Current Progress and Future Prospects" retrieved online at http://www.rand.org/education/mcarthur/Papers/role.html, retrieved online on Jul. 18, 2001.|
|68||McArthur, David, "Artificial Intelligence and Mathematics Education" retrieved online at http://www.rand.org/hot/ macarthur/Papers/aied.html, retrieved online on Jan. 17, 2001.|
|69||Min, Rik "Computer Simulation as an Educational Took," Simulation Technology and Parallelism in Learning Environments, pp. 1-26, retrieved online at http://www.to.utwente.nl.prj/min/Book/chapter1.htm, retrieved online on Jul. 18, 2001.|
|70||Mitrovic, Antonija et al., "Evaluating the effectiveness of feedback in SQL-tutor", 2000, pp. 143-144, IEEE.|
|71||Multimedia Training "Get Lemonade, Not a Lemon!" Jun. 1997, Journal for Quality and Participation vol. 20, No. 3, pp. 22-26.|
|72||Muntjewerff, Antoinette, "Automated Training of Legal Reasoning", 9th Bileta Conference, The Changing Legal Information Environment, Apr. 11-12, 1994, pp. 1-7, online at http://www.bileta.ac.uk/94papers/muntjew.html, retrieved online on Jul. 18, 2001.|
|73||Murray, Tom, "Authoring Intelligent Tutoring Systems: An Analysis of the State of the Art" International Journal of Artificial Intelligence in Education, 1999, pp. 1-35, vol. 10.|
|74||No More Boring CPE,: Jul. 1997, Accounting Technology, pp. 27-35.|
|75||Prabhu, Suresh S. et al., "Computer Aided Instruction for Statistics: A Knowledge-Based Systems Approach," Computers in Adult Education and Training, 1995, pp. 3-14, vol. 5, No. 1.|
|76||Shreiner, Curt, "CAPTOR a model for delivering web based intelligent tutoring system technology", 2000, pp. 5. C.4.1-5.C.4-5, vol. 2, IEEE.|
|77||Suthers, Daniel D., "Technical Report: Computer Aided Education and Training Initiative", Jan. 12, 1998, pp. 1-51, online at http://advleran.lrdc.pitt.edu/advlearn/papers/FINALREP.html, retrieved online on Jul. 18, 2001.|
|78||Tams, E.K. et al., "WITS: A Reusable Architecture for a VR-Based ITS", Jun. 10, 1996, retrieved online at http://advlearn.irdc.pitt.edu/its-arch/papers/tam.html, retrieved online on Jul. 18, 2001.|
|79||Template Software Business Simulator, Chapter 8, Aug. 1997.|
|80||Template Software Workflow Simulator, Chapter 8, Jun. 1998.|
|81||The Design of Goal-Based Scenarios, Technical Report #39, Northwestern University, The Institute for the Learning Sciences, Mar. 1993, pp. 1-58.|
|82||The Virtual Classroom: Great Expectations. Delivering Graduate Education by Computer: A Success Story, 1996.|
|83||What are Intelligent Coaching Systems and Why are they (in)evitable? IEEE Colloquium on Artificial Intelligence in Educational Software, pp. 2/1-2/5, Jun. 1998.|
|84||Yazdani, Masoud, "Intelligent Tutoring Systems: An Overview," Expert Systems, Jul. 1986, pp. 154-162, vol. 3, No. 3.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US20120237915 *||Aug 15, 2011||Sep 20, 2012||Eric Krohner||System and method for assessment testing|
|U.S. Classification||434/219, 434/118, 434/307.00R|
|International Classification||G09B5/00, G09B5/06, G09B19/00, G09B11/00|
|Cooperative Classification||G09B5/065, G09B19/00, G09B5/00, G09B7/02, G09B7/06|
|European Classification||G09B5/00, G09B19/00, G09B5/06C, G09B7/06, G09B7/02|
|Jan 26, 2011||AS||Assignment|
Owner name: ACCENTURE GLOBAL SERVICES LIMITED, IRELAND
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ACCENTURE GLOBAL SERVICES GMBH;REEL/FRAME:025700/0287
Effective date: 20100901
|Mar 4, 2013||FPAY||Fee payment|
Year of fee payment: 4