CA2100510A1 - Method and system for processing and presenting on-line multimedia information in a tree structure - Google Patents
Method and system for processing and presenting on-line multimedia information in a tree structureInfo
- Publication number
- CA2100510A1 CA2100510A1 CA002100510A CA2100510A CA2100510A1 CA 2100510 A1 CA2100510 A1 CA 2100510A1 CA 002100510 A CA002100510 A CA 002100510A CA 2100510 A CA2100510 A CA 2100510A CA 2100510 A1 CA2100510 A1 CA 2100510A1
- Authority
- CA
- Canada
- Prior art keywords
- multimedia
- information
- display means
- tree
- tree structure
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2257—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/34—Graphical or visual programming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S715/00—Data processing: presentation processing of document, operator interface processing, and screen saver display processing
- Y10S715/961—Operator interface with visual structure or function dictated by intended use
- Y10S715/965—Operator interface with visual structure or function dictated by intended use for process control and configuration
- Y10S715/966—Computer process, e.g. operation of computer
- Y10S715/967—Visual or iconic programming
Abstract
Abstract Method and system are provided for processing and presenting on-line, multimedia information such as diagnostic information for a machine tool. The system uses multimedia information tools (i.e. graphics, documentation, drawings, photographs, full motion video with audio, PLC ladder, etc.) and techniques to assist maintenance personnel.
Diagnostic development and multimedia information assignment are accomplished through graphical visual programming, which requires no conventional software programming efforts. The system uses diagnostic trees having multimedia graphic icons which define the diagnostic components. In developing a visual diagnostic tree, the multimedia information can be assigned or mapped to each node of the tree by selecting an appropriate icon to represent the multimedia from an icon library. Then, an appropriate media information file or record and comments for that icon are further added to define that node assignments. The system is user friendly, uses conventional hardware and can be used on the plant floor. The system is immediately executable and is generic. The method and system can also be used for training, process planning, marketing, sales, help desk applications, other machine diagnostics, and engineering design.
Diagnostic development and multimedia information assignment are accomplished through graphical visual programming, which requires no conventional software programming efforts. The system uses diagnostic trees having multimedia graphic icons which define the diagnostic components. In developing a visual diagnostic tree, the multimedia information can be assigned or mapped to each node of the tree by selecting an appropriate icon to represent the multimedia from an icon library. Then, an appropriate media information file or record and comments for that icon are further added to define that node assignments. The system is user friendly, uses conventional hardware and can be used on the plant floor. The system is immediately executable and is generic. The method and system can also be used for training, process planning, marketing, sales, help desk applications, other machine diagnostics, and engineering design.
Description
~ ' 21~0510 .
METHOD AND SYSTEM FOR PROCESSING AND PRESENTING
ON-LINE, MULTIMEDIA INFORMATION IN A TREE STRUCTURE
Cross-Reference To Related A~lication This application relates to an application entitled "Method And System For Diagnosing Machines"
filed on the same day as this application, having the ~;
same assignee, and which is hereby expressly incorporated in its entirety by reference. ~ -Technical Field This invention relates to methods and systems for processing and presenting multimedia information andj in particular, to methods and systems for processing and presenting on-line, multimedia information such as diagnostic information for a machine tool in a tree structure.
Backqround Art There exists a need for timely delivery of diagnostic information for machine tools such as stamping presses. As illustrated in Figure l, the need is the greatest when the press is first delivered. As time passes, the plant personnel become more experienced and less dependent on a diagnostic - system. The key is continually to have the proper tools present to minimize press down time, regardless of the need.
Based on these observations, the machine tool manufacturer must develop the diagnostic aid as .
,, :
i". ' i .,:,: , ' ', '.' , ' ' ' : ' '' , '.. ', . ,' ''. ' ' .' ,' :
21~0~
,' ' the press is being designed if this goal is going to be achieved.
Current expert systems and/or artificial intelligence technology tools can not meet the above -direction, goals and design needs. The deficiency is in the ability to allow end users to build and maintain diagnostics, especially as the plant machinery goes through modification. Based on the current state of software/hardware technology (i.e.
object oriented), there exists a need for a new method and system which uses this technology in a novel way -~
for capturing, preserving and delivering machine tool - diagnostic and support information.
The U.S. patent to Isle et al. (4,931,950) ~ -discloses a multimedia interface method for a knowledge-based diagnostic system. The knowledge-based system stores multimedia commands to control multimedia outputs for presenting information to --users.
The U.S. patent to Tsumura (4,954,969) discloses a system for processing multimedia information in a easily usable form.
The U.S. patent to Singh (4,954,964) discloses an apparatus and method for expert analysis of metal failure with automated visual aid. The invention combines an expert system with a video or photographic display system.
The U.S. patent to Rutherfoord et al.
4,893,256) discloses an interactive multimedia ~` 30 presentation system and a method for developing the presentation.
The U.S. patent to Bodick et al. (4,945,476) `~
discloses a knowledge-base system with stored ~ -' .: ' . . ,.,.. - .. : .. , . , .,., ~ :,., .: : , .:,, ., . .. .. . ... . ,. , . . . .. : . .. , .,: . . . . . .
METHOD AND SYSTEM FOR PROCESSING AND PRESENTING
ON-LINE, MULTIMEDIA INFORMATION IN A TREE STRUCTURE
Cross-Reference To Related A~lication This application relates to an application entitled "Method And System For Diagnosing Machines"
filed on the same day as this application, having the ~;
same assignee, and which is hereby expressly incorporated in its entirety by reference. ~ -Technical Field This invention relates to methods and systems for processing and presenting multimedia information andj in particular, to methods and systems for processing and presenting on-line, multimedia information such as diagnostic information for a machine tool in a tree structure.
Backqround Art There exists a need for timely delivery of diagnostic information for machine tools such as stamping presses. As illustrated in Figure l, the need is the greatest when the press is first delivered. As time passes, the plant personnel become more experienced and less dependent on a diagnostic - system. The key is continually to have the proper tools present to minimize press down time, regardless of the need.
Based on these observations, the machine tool manufacturer must develop the diagnostic aid as .
,, :
i". ' i .,:,: , ' ', '.' , ' ' ' : ' '' , '.. ', . ,' ''. ' ' .' ,' :
21~0~
,' ' the press is being designed if this goal is going to be achieved.
Current expert systems and/or artificial intelligence technology tools can not meet the above -direction, goals and design needs. The deficiency is in the ability to allow end users to build and maintain diagnostics, especially as the plant machinery goes through modification. Based on the current state of software/hardware technology (i.e.
object oriented), there exists a need for a new method and system which uses this technology in a novel way -~
for capturing, preserving and delivering machine tool - diagnostic and support information.
The U.S. patent to Isle et al. (4,931,950) ~ -discloses a multimedia interface method for a knowledge-based diagnostic system. The knowledge-based system stores multimedia commands to control multimedia outputs for presenting information to --users.
The U.S. patent to Tsumura (4,954,969) discloses a system for processing multimedia information in a easily usable form.
The U.S. patent to Singh (4,954,964) discloses an apparatus and method for expert analysis of metal failure with automated visual aid. The invention combines an expert system with a video or photographic display system.
The U.S. patent to Rutherfoord et al.
4,893,256) discloses an interactive multimedia ~` 30 presentation system and a method for developing the presentation.
The U.S. patent to Bodick et al. (4,945,476) `~
discloses a knowledge-base system with stored ~ -' .: ' . . ,.,.. - .. : .. , . , .,., ~ :,., .: : , .:,, ., . .. .. . ... . ,. , . . . .. : . .. , .,: . . . . . .
2 ~ 5 ~ ~
pictorial images for use in medical diagnostic environments.
The prior art discloses the use of icons in various systems. For example, the U.S. patent to Nose et al. (5,025,395) discloses a data processing system which employs icons to represent various functions of the system.
The U.S. patent to Wexelblat et al.
(5,021,976) discloses a method and system for generating dynamic, interactive visual representations of information structures. The patent teaches the use of iconic representations in an information system.
The U.S. patents to Dunn (4,656,603 and 4,813,013) disclose interactive rule based systems employing icons. The system generates and stores a~ -specific set of rules pertaining to the use of each icon. -The paper entitled "The Relationship Between Multimedia And Expert Systems" by John Coyne, discloses a philosophy to using an expert system and multimedia but fails to provide many details.
The U.S. patent to Franke et al. (4,710,763) discloses a method of constructing and developing a tree structure. An operator is capable of performing editing and evaluating functions on the tree.
The U.S. patent to Watson et al. (4,902,469) discloses apparatus for producing a discrete state display including status trees and status tree node operations.
2100~10 :-Summary Of The Invention An object of the present invention is to provide a method and system for processing and -presenting on-line, multimedia information in a tree structure.
Another object of the present invention is to provide a relatively simple and flexible method and system for processing and presenting on-line, multimedia information such as diagnostic information for a machine tool wherein initial diagnostic ~ knowledge and documentation (such as drawings) are electronically captured as designers are designing the machine tool, which eliminates any possible loss or re-engineering of thi~ information at a later time.
In carrying out the above objects and other objects of the invention, a method for processing and presenting multimedia information in a tree structure including branches having a plurality of nodes and node-connecting links in a computer system having a tree display for displaying nodes of the tree structure is disclosed. The method includes the step of providing a command display for displaying -multimedia commands representative of various types of multimedia information. The method also includes the -~
step of providing a multimedia output which provides -~
multimedia information to a user of the system. The method further includes the step of providing a knowledge base for storing a multiplicity of ; ` information records mapped into the tree structure.
The information records include means for denoting the text strings that can be displayed on the tree display, means for denoting the multimedia commands that can be displayed on the command display, and 21005~0 means for denoting the multimedia information that can be presented by the multimedia output. In response to a received initial command, at least one selected node of the tree structure is displayed including at least one text string on the tree display. A plurality of multimedia commands corresponding tQ the at least one selected node is displayed on the command display.
After a user command related to one of the displayed multimedia commands is received, a selected amount of the multimedia information is delivered to the multimedia output in response thereto for prosentation to the user of the system.
A system is also provided~for carrying-out each of the above method steps.
Preferably, the multimedia commands are - ;displayed on the command display in the form of graphic icons.
The above objects and other objects, features, and advantages of the present invention are readily apparent from the following detailed description oS the best mode for carrying out the invention when taken in connection With the accompanying drawings.
- Brief Descri~tion Of The Drawinas Figure 1 is a graph illustrating the interrelationship of the need for diagnostic information to press development time and press operational time;
Figure 2 is a schematic diagram illustrating the architecture of the system of the present invention including a diagnostic system coupled to a -- ~ :
:: :
.
:
21~510 i -database of machine-specific knowledge and support information;
Figure 3 is a schematic diagram of a diagnostic tree and multimedia mapping of the present ~.
S invention;
Figure 4 is a schematic view of the hardware of the system;
Figure 5 is a schematic view of the software of the system;
Figure 6 is a schematic block diagram of a diagnostic tree and presentation server structure of the present invention;
Figure 7 is a schematic view of an example ~uestion and answer (Q and A) screen in text format; :. :
Figure 8 .~s a schematic view of an example question and answer screen in graphics-with-text format; . -Figure 9 is a schematic view of a presentation server screen; . ``
Figure 10 is a schematic view of a diagnostic branch (edit) screen; :
Figure 11 is a graphical view of a presentation server edit screen or multimedia definition screen;
Figure 12 is a graphical view of a screen :
illustrating diagnostic tree development with multimedia icon edit; ~
Figure 13 is a graphical view of a media -selection server;
0 Figure 14 is a graphical view of an initial application screen;
Figure 15 is a graphical view of a standard Q and A screen similar to Figure 8; -: : ' '.
. . : . . .
2100~10 Figure 16 is a graphical view of a multimedia view screen similar to Figure 9 with video selected;
Figure 17 is a graphical view of a S documentation view screen;
Figure 18 is a graphical view of a screen for a drawing;
Figure 19 is a graphical view of a screen for still video;
Figure 20 is a graphical view of a screen for full motion values; and Figure 21 is a graphical view of a solution screen.
Best Mode For Carrying Out The Inventlon Referring now to the drawing figures, there is schematically illustrated in Figure 2 a system including a generic core of software and hardware, which is referred to below as a diagnostic system 20 -for use with a machine tool such as a press. To give life to the diagnostic system 20 for a unique machine tool diagnostic solution requires specific machine tool diagnostic knowledge and support information (referred to below as a database 22).
~he architecture of the system illustrated in Figure 2, shows three primary components of the -system: the diagnostic system 20, a knowledge acquisition tool 24, and the database 22.
The diagnostic system 20 is dependent on the data to enable it as an application. The data in the database 22 includes mach1ne tool specific diagnostic knowledge and support information (i.e. drawings and video) which is stored in the form of information ~: .
, ' 21~0~10 ~ -91-809/810 ~-records. The machine tool support information provides alternative methods of communication and -documentation necessary to properly inform the end user while doing his or her job. Another important `
a~pect of this architecture is that the machine -specific support information can be stored in its - -original format (for example, Wordperfect). There is no need to translate or re-engineer the information, thus reducing development time, expense and effort.
The knowledge acquisition tool 24 of the present invention is relatively simple and intuitive to use; provides graphical representation of diagnostics knowledge; and provides mapping of machine tool related support information. - -The tool 24 represents the knowledge in graphical trees (or diagnostic trees). Within a~tree ;~
there are several questions and-answers, and solution nodes as illustrated in Figure 3. Nodes can be grouped together into branches, making the diagnostic tree more modular and reusable. There is no software programming involved in building the trees or mapping the support knowledge - it is performed through graphical visual programming. Training to use of the diagnostic tool can be accomplished relatively quickly. - :
The procedure is similar to using a computer drawing package, by graphically selecting and placing objects (denoted by boxes), asserting text (questions & answers), linking boxes and assigning support -.
~` 30 information to respective nodes as illustrated in Figure 3. Training to build and maintain diagnostic trees, and assign or edit machine specific support information can also be accomplished relatively quickly.
2100~10 The method and system of the present invention preferably utilizes an IBM PC platform that operates under DOS 5.0 and Windows 3Ø This platform provides a rich design environment and option flexibility.
The target system preferably includes, at a minimum, an IBM 386/486 or compatible computer with a 200Mb hard drive, 8 Mb of RAM, VGA display, Video card, a computer controlled VCR, optional speakers, and a direct connection to a press PLC network as illustrated in Figure 4.
There is illustrated in Figure 5, a software portion of the system. The software includes commercially available software and one custom piece ; 15 of software called a presentation server 26. The Windows 3.0 environment allows multiple software package to run concurrently-and communicate between themselves. Communication between the software packages leverages the-software packages $unctionality into a integrated solution and minimizes software development.
The diagnostic system 20, the knowledge acquisition tool 24 and the database 22 are preferably incorporated in a software package called "Advisor"
provided by Emerald Intelligence of Ann Arbor, Michigan. Advisor provides the environment to build ~-and execute diagnostics. To assign (i.e. map) and present (i.e. display) multimedia information for each ~ ~ of the diagnostic tree nodes as illustrated in Figure 1 30 6, Advisor communicates and interacts with the presentation server 26. The presentation server 26 then communicates with the media specific software ~ ~ packages (i.e. Microsoft Word for documentation media) I ~ to select and view.
. .
~100~1~ '. .
With this architecture, it is possible for each diagnostic node to have multimedia information assigned or mapped thereto thus improving the communication between the operator and the computer. -The standard forms of multimedia utilized are:
drawings, documentations, graphics, photographs, full motion video and audio, animation, sound, etc. For example, this approach can provide a better and easier ;
means of on-line interactive repair and training procedures (i.e. by using full motion video) to the operator when a cause of the problem is determined.
Also, because the architecture allows for the multimedia information to be used in its original -format, there is no need to re-engineer the media infor~ation-for it to be deliverable.
Based on this software architecture, there are two system utilities: the runtime solution or utility referred to as RUNTIME and the diagnostic tree building solution or utility referred to as~BUILDER, as also described hereinbelow.
.
RUNTIME
Within the RUNTIME software there are two forms of executing the diagnostics: PLC directed or operator step-by-step operation. The PLC directed ~ ~ 25 approach involves the computer receiving a predefined 5~ ~ : PLC fault code relative to a press malfunction. Once the PLC ~ault is recognized, the software jumps to a specific assigned location in the diagnostic tree (the same diagnostic process can be completed using the operator step-by-step approa~h, discussed below~. The benefit of this approach is to minimize operator-to-~ ~ computer diagnostic dialogue and get to the cause of .
, .
. , . ,; . , .. . ., . . .. , , . . ,.... . - ., . . . ~
the problem as guickly as possible. The key - -limitation of this approach is that it only applies to sensory faults. Non-sensory faults will need to be diagnosed in the step-by-step approach.
In either of the approaches (PLC directed or step-by-step), the operator is asked a series of questions until he reaches the cause of the press problem. In the step-by-step approach, the diagnostics always starts from the top of the tree.
Answers for questions are presented in one of two formats: text or graphics with text. The text type answer is a traditional format illustrated in Figure 7, and is the easiest to generate and least time-consuming. The graphics with text, as illustrated in Figure 8, is a format that conveys the message faster ~ -and with less effort for the operator, but does take more development time because of the graphic development. Both of the answer formats can be intermixed in the diagnostics, but are defined and Sixed in the diagnostic at the time of tree development.
Another important feature in both the Question & Answer formats is the placeholders. The placeholder reminds the operator of the path that has 2 been chosen and allows the operator to "Back Up" to a previous diagnostic screen.
Also~ at any time during the diagnostic session, if related machine specific support information has been assigned to a particular questiOn & answer or solution screen, an "i" button is displayed in the upper left hand corner as illustrated in Figures 7 and 8. By selecting the "i" button, the presentation server 26 is activated as illustrated in Figure 9, which allows the operator to view the - . .. . . ~ . : , ,: : . ~ :. .. . .. .
2100~
:.. , different assigned media relative to that node function in the form of multimedia commands. More detail on the use of the runtime solution is described hereinbelow. ~ -BUILDER
BUILDER allows one to define and/or maintain machine diagnostic trees (a graphical representation of a diagnostic decision step or ~low) and assign multimedia machine information to the respective tree node.
To briefly demonstrate the ease and -; - -simplicity of creating a diagnostic tree, reference may be made to Figure 10. Diagnostic trees are made up o~ branches. Branches are made up of four primary components:
* Question & Answer (Q&A) nodes * Solutions nodes * Branch jump nodes * links or lines.
The Q&A node states a question with an unlimited number of answers. The solution node states the cause of the problem and needed repair for the cause of the problem. To assign a Q&A node (or solution node), a point mark is first located on the screen (click mouse) where one would like to place the node. Second, the type of node is selected from the above screen menu select: OBJECT-CREATE-SYMPTOM for Q&A node, or OBJECT-CREATE-SOLUTION for a solution node. This procedure places the object node on the screen as previously marked. To fill in the node name, the Q&A, or solution text information, the node is selected with the mouse pointer and the text is , .
.
' ,. ~ ' ~' ' , ' ' .. . ~. '' .-. :
: . ~: ' . .,'. ' :' .~ ', , . . ~ , . . . . .
2100~ 0 entered. Graphical answers with text can alternatively be used to replace the standard textual answers. To add more answers to a Q&A node, the node is selected and then OBJECT-ADD VALUE/TEST is selected from the menu screen. To link an answer to the next -question or solution, a right arrow icon is selected in the preferred answer box then the mouse pointer is dragged to the respective question, branch or solution and the mouse button is released. This places a link or line between the two object nodes.
The above steps are repeated until the desired tree is built. Objects within a screen can freely be moved around, primarily for appearance and readability, without effecting the linking. Anytime while editing a tree, one can run the diagnostics by selecting from the screen menu, FILE-RUN TEST. This will bring one into the RUNTIME (test mode) with the current tree information.
To define machine specific multimedia information for any of the Q&A and solution nodes, one needs to access the presentation server screen.
First, one double click on "?" in the Q&A box or just on the solution box. This brings up the question or solution editor screen and then one enters CTRL and -~
keys, which brings up the presentation server (screen illustrated in Figure 11). The keys do this by ~ -executing a recorded Windows 3.0 macro. To assign the desired media (documentation, drawing, photograph or video), the button on the left is selected. A file directory box (not shown) pops up to allow one to look up the hard disk for the desired file. Once the file is selected, there is an opportunity to preview the material, define required additional information (i.e.
page number within a documentation) and assign comment 2100510 .
information before mapping. When the media is selected, it is listed in the appropriate media box as illustrated in Figure 11. There are, preferably, five different choices of material for each of the forms of media. When all the media is assigned, OK is entered to return to the builder screen.
, Building Diagnostic Trees And Assigning Multimedia Information Usinq Gra~hic Icons .
The process here is similar to the above BUILDER process but is simpler to generate trees and assign multimedia information. There is also the use of more graphic icons (i.e. see the solution block of Figur~ 12) instead of menu selections.
STEP 1:
To an assign symptom block (Q&A block) or solution block, an icon on the left in Figure 12 is first selected, and located on the right side or work area of Figure 12.
To fill in the Q&A or solution block text information name, the block field with the mouse pointer is selected and the text is entered. To link an answer to the next question or solution, the link icon is selected and the respective blocks are connected. This places a link or line between the two blocks.
The above steps are repeated until the desired tree is built. Blocks within a screen can freely be moved around, primarily for appearance and readability, without effecting the linking.
.
:
. . . . ~ : ~ : .. ..
. . . .- ., . . . :
- 2100~0 :
The use of icons is a unique method for building diagnostic trees and defining multimedia information. Prior solutions required~menu selections or involved accessing other screens.
. .
STEP 2:
To define machine specific multimedia information for any of the Q&A and solution blocks, a media icon of choice is selected on the left side of the screen of Figure 12.
This media icon is placed near a Q&A or~
solution block of choice and is then linked to the desired block. By reselecting an -assigned media icon on the right side, a ~ --media selection screen pops up as illustrated in Figure 13. Once the file is ~ -~
selected, there is the opportunity~to~
preview the material, define required additional information (i.e.~ page number within a documentation) and assign comment information before mapping. ~ ~ -This step is unique, because of the use of direct visual aids (icons) and ` programming to assign~the media to its respective block. This process is simpler than~prior solutions because everything is accomplished on one screen. The media remains in~its natural format and does not require any re-engineering. ;~ -' ' ' '; ~ ` ' ' ' . '` ' ' ~ ` : ' , ' i ;
2100~
STARTUP ~;
First one turns on the computer power or reboots the system. After a few minutes (software applications are being loaded), the first screen of the system as illustrated in Figure 14 is displayed.
This initial Question & Answer node is displayed on a tree display portion of the screen. This initial node requests one to select what transfer press subsystem (Feeder, Press or Destacker) has a problem. After selection in response to an initial command, additional Question & Answer screens are displayed until one reaches a solution screen or node. Details of each of the different types of screens and their operations are described hereinbelow.
OUESTION AND ANSWER SCREEN
The diagnostic screen of Figure 15 is an example of a Question & Answer (Q&A) screen. The standard Q&A screen is broken down into three major components: the question, the answer, and the placeholder. Also, in some of the Q&A screens, the Info Button is displayed, as highlighted in Figure 15, which means that there is additional multimedia information defined with the question. ~-.
OUESTION BOX
-For each diagnostic screen, there is a question posted and thus m~st be answered. The question is typically stated in text format only.
: ' ' , - ~
.
' ,: . .: .: ,.:,: . . ,, , , . :.:~ ,, ,, . ,.. -, . , , ,. - -. .: .: . : . , .. :..... : . .. .; . : ~ . .
-: ~ ...... ... : : . .. . ..
21~5~0 ' ` ~'. ':
ANSWER AREA
In the answer area, there can be two types of answers: text or graphics with text as shown in Figure 15. The next type is just a sentence as shown in Figure 7. The graphics with text type shows a picture that represents the answer. To answer a question, the correct answer (line of text or picture) is selected.
~ACEHOLDER
: ~ .
Once an answer is selected, a box with a -picture or text, is placed in the placeholder area or branch display portion of~the tree display~portlon of the screen. These placeholder~boxes represent the diagnostic path that has been taken. Selecting~a placeholder box allows one to "BArKUP" to a~previous diagnostic screen.
l~ORMATION
.
By selecting the Info Button, "i", the multimedia presentation screen pops up, as shown in ~Figure 9. ~This;screen allows one to select~desired ; media information (documentation, photographsj prints/drawings~and full motion~video) in~response to the received user command.~ ~The~multimedia in$ormation is there for the~operator~to help answer any questions 25 ~ or to explain testing or repair procedures~about the current dlagnostic screen.
~.: - . - ~ .
view screen For Text Document The screen of Figure 17 displays a text document, which can have embedded graphics, for a respective diagnostic screen. The whole page is not displayed. To view the other portions of the page, the arrow buttons ~highlighted) are selected to move about the page. To look at the previous or next page within the document, the page symbol at the right hand side of the screen is selected (as highlighted).
Finally, to exit and return back to the diagnostic screen "File" is selected from the menu and then "Done" is chosen from the "File Menu."
YleyL~L-~een For Dràwinq ~ ~
The screen of Figure 18 displays an AutoCad drawing file (also called a print) and predefined views within an AutoCad file. These drawings can be an electrical, pneumatic, hydraulic or mechanical - drawing. To exit the drawing, the screen is touched.
View Screen For Still Video ~ 20 By selecting any of the still video (same as a photograph) buttons (the camera icon), the image is display~d in the top left side of the screen. An associated comment about the still video may ~e displayed above the still buttons as illustrated in Figures 16 and 19.
~. ~
:~, .~, ' ^
.
~, :2 ~ 0~
View Screen For Full Motion Video By selecting any of the full motion video -buttons (the video tape icon), the video image is displayed in the top left side of the screen as illustrated in Figure 20. Prior to displaying the video, a comment is displayed at the top of the box.
This gives the operator the chance to determine if it is the right video to view. If not, another video button is selected. If it is the right video, the VIEW button is selected and the VCR will start searching for the tape segment under control of the computer. Once the video is found, it is displayed in the upper left side of the screen. Volume (if speakers are attached) and brightness can be adjusted ~ -while monitoring video tape. Two other video control buttons allow the audio to be muted and the video to be frozen (the VCR is still playing at this time and only the image on the screen is frozen). To exit once the video is completed, the DONE button is selected in the video control box.
Solution Screen ' ;: .
When the last node, or the Solution, has ~--been reached, the screen of Figure 21 is displayed.
If there is any`help information assigned at this ~25 node, the help information is displayed first. At this point, one can still back up if needed or save this particu~lar diagnostic session in a session file ~ (*.SES). ~ -: ~ ~ ~ - .' ' :
.
-.. .. .,~..
:~
~g ~
To save session To save current session from the top to current node, File-Save Session menu is selected. The appropriate file name is typed in. The default file extension is (*.SES).
To restore session To restore a previously saved diagnostic session, File-Restore Session menu is ~--selected. Then, the session file name is selected. By restoring session, one~can trace the previous path or immediately return to the solution node.
The system as described herein below diagnoses three primary components of a machine tool such as a transfer press: the press, the feeder, and the destacker. The press and the feeder diagnostic trees and related information are included. The press and feeder trees may address as many as 143 primary symptoms that include approximately 800 nodes.
Building these two trees and accumulating and ;~ assigning the multimedia information is relatively straight forward after the paper, drawings and documentation~were converted into electronic format.
The method and system can be used to visually locate components on a large and complex transfer press. This has a significant impact in reducing the repair time because plant electricians can immediately locate defective component(s).
The method and system is not limited to stamping press~diagnostics, but may be used for other forms of diagnostics (i.e. other plant machinery and :
2100~10 office applications) and for use on non-machine diagnostics (i.e. process diagnostics or as engineering design aid).
The method and system provide the following advantages for use with a press: machine diagnostics when the machine is delivered; better transfer of technology and documentation from the vendor to the customer; reduced press down time; reduced diagnostic and repair time; better informed repair personnel, especially the apprentice; on-line training; better utilization of vendor/machine information; better service to the machine; the opportunity to capture new and refined diagnostic knowledge from plant personnel;
reusability of information, that can be applied to other similar machines; and actual machine diagnostic flow and cause information can be captured to provide good feedback to the supplier with the hope of making better equipment and diagnostic models in the future. -The method and system can be used to provide a multi-lingual solution. Using a stereo VCR, one audio track may contain information in a first language and the other audio track may contain information in a second language. Further, a multi-lingual solution is made possible through the selection of alternate keyboard assignments using the windows operating environment.
The present system requires well disciplined and structured information management techniques to allow for ease of upgradeability,~maintenance and - -adjustments. This process is also invisible and seamless to the people making these changes.
The presented media information demands high storage space but can be used for other similar and identical machines. To optimize storage and decrease ~; ' 21~0~1~
~1-809/810 redundancy of information, there may be a centralized network/file server solution. While there is currently no clean way to store, retrieve and transmit .
analog information (video and audio) over a data ~ : .
network, a complete centralized library of machine information such as 2 and 3 dimensional animation, multi-language media, audible answers to diagnostic questions, speech, 3-D drawings, and a point-to-point distribution system can be accommodated when such is possible.
While the best mode for carrying out the invention has been described in detail, those familiar `
with the art to which this invention relates will recognize various alternative designs and embodiments : .
for practicing the invention as defined by the ~oll~w ~g cl~i~s.
, , . , .,. , .. .. . , ,. : .~ , :. . . . .. , . .: : -. :
pictorial images for use in medical diagnostic environments.
The prior art discloses the use of icons in various systems. For example, the U.S. patent to Nose et al. (5,025,395) discloses a data processing system which employs icons to represent various functions of the system.
The U.S. patent to Wexelblat et al.
(5,021,976) discloses a method and system for generating dynamic, interactive visual representations of information structures. The patent teaches the use of iconic representations in an information system.
The U.S. patents to Dunn (4,656,603 and 4,813,013) disclose interactive rule based systems employing icons. The system generates and stores a~ -specific set of rules pertaining to the use of each icon. -The paper entitled "The Relationship Between Multimedia And Expert Systems" by John Coyne, discloses a philosophy to using an expert system and multimedia but fails to provide many details.
The U.S. patent to Franke et al. (4,710,763) discloses a method of constructing and developing a tree structure. An operator is capable of performing editing and evaluating functions on the tree.
The U.S. patent to Watson et al. (4,902,469) discloses apparatus for producing a discrete state display including status trees and status tree node operations.
2100~10 :-Summary Of The Invention An object of the present invention is to provide a method and system for processing and -presenting on-line, multimedia information in a tree structure.
Another object of the present invention is to provide a relatively simple and flexible method and system for processing and presenting on-line, multimedia information such as diagnostic information for a machine tool wherein initial diagnostic ~ knowledge and documentation (such as drawings) are electronically captured as designers are designing the machine tool, which eliminates any possible loss or re-engineering of thi~ information at a later time.
In carrying out the above objects and other objects of the invention, a method for processing and presenting multimedia information in a tree structure including branches having a plurality of nodes and node-connecting links in a computer system having a tree display for displaying nodes of the tree structure is disclosed. The method includes the step of providing a command display for displaying -multimedia commands representative of various types of multimedia information. The method also includes the -~
step of providing a multimedia output which provides -~
multimedia information to a user of the system. The method further includes the step of providing a knowledge base for storing a multiplicity of ; ` information records mapped into the tree structure.
The information records include means for denoting the text strings that can be displayed on the tree display, means for denoting the multimedia commands that can be displayed on the command display, and 21005~0 means for denoting the multimedia information that can be presented by the multimedia output. In response to a received initial command, at least one selected node of the tree structure is displayed including at least one text string on the tree display. A plurality of multimedia commands corresponding tQ the at least one selected node is displayed on the command display.
After a user command related to one of the displayed multimedia commands is received, a selected amount of the multimedia information is delivered to the multimedia output in response thereto for prosentation to the user of the system.
A system is also provided~for carrying-out each of the above method steps.
Preferably, the multimedia commands are - ;displayed on the command display in the form of graphic icons.
The above objects and other objects, features, and advantages of the present invention are readily apparent from the following detailed description oS the best mode for carrying out the invention when taken in connection With the accompanying drawings.
- Brief Descri~tion Of The Drawinas Figure 1 is a graph illustrating the interrelationship of the need for diagnostic information to press development time and press operational time;
Figure 2 is a schematic diagram illustrating the architecture of the system of the present invention including a diagnostic system coupled to a -- ~ :
:: :
.
:
21~510 i -database of machine-specific knowledge and support information;
Figure 3 is a schematic diagram of a diagnostic tree and multimedia mapping of the present ~.
S invention;
Figure 4 is a schematic view of the hardware of the system;
Figure 5 is a schematic view of the software of the system;
Figure 6 is a schematic block diagram of a diagnostic tree and presentation server structure of the present invention;
Figure 7 is a schematic view of an example ~uestion and answer (Q and A) screen in text format; :. :
Figure 8 .~s a schematic view of an example question and answer screen in graphics-with-text format; . -Figure 9 is a schematic view of a presentation server screen; . ``
Figure 10 is a schematic view of a diagnostic branch (edit) screen; :
Figure 11 is a graphical view of a presentation server edit screen or multimedia definition screen;
Figure 12 is a graphical view of a screen :
illustrating diagnostic tree development with multimedia icon edit; ~
Figure 13 is a graphical view of a media -selection server;
0 Figure 14 is a graphical view of an initial application screen;
Figure 15 is a graphical view of a standard Q and A screen similar to Figure 8; -: : ' '.
. . : . . .
2100~10 Figure 16 is a graphical view of a multimedia view screen similar to Figure 9 with video selected;
Figure 17 is a graphical view of a S documentation view screen;
Figure 18 is a graphical view of a screen for a drawing;
Figure 19 is a graphical view of a screen for still video;
Figure 20 is a graphical view of a screen for full motion values; and Figure 21 is a graphical view of a solution screen.
Best Mode For Carrying Out The Inventlon Referring now to the drawing figures, there is schematically illustrated in Figure 2 a system including a generic core of software and hardware, which is referred to below as a diagnostic system 20 -for use with a machine tool such as a press. To give life to the diagnostic system 20 for a unique machine tool diagnostic solution requires specific machine tool diagnostic knowledge and support information (referred to below as a database 22).
~he architecture of the system illustrated in Figure 2, shows three primary components of the -system: the diagnostic system 20, a knowledge acquisition tool 24, and the database 22.
The diagnostic system 20 is dependent on the data to enable it as an application. The data in the database 22 includes mach1ne tool specific diagnostic knowledge and support information (i.e. drawings and video) which is stored in the form of information ~: .
, ' 21~0~10 ~ -91-809/810 ~-records. The machine tool support information provides alternative methods of communication and -documentation necessary to properly inform the end user while doing his or her job. Another important `
a~pect of this architecture is that the machine -specific support information can be stored in its - -original format (for example, Wordperfect). There is no need to translate or re-engineer the information, thus reducing development time, expense and effort.
The knowledge acquisition tool 24 of the present invention is relatively simple and intuitive to use; provides graphical representation of diagnostics knowledge; and provides mapping of machine tool related support information. - -The tool 24 represents the knowledge in graphical trees (or diagnostic trees). Within a~tree ;~
there are several questions and-answers, and solution nodes as illustrated in Figure 3. Nodes can be grouped together into branches, making the diagnostic tree more modular and reusable. There is no software programming involved in building the trees or mapping the support knowledge - it is performed through graphical visual programming. Training to use of the diagnostic tool can be accomplished relatively quickly. - :
The procedure is similar to using a computer drawing package, by graphically selecting and placing objects (denoted by boxes), asserting text (questions & answers), linking boxes and assigning support -.
~` 30 information to respective nodes as illustrated in Figure 3. Training to build and maintain diagnostic trees, and assign or edit machine specific support information can also be accomplished relatively quickly.
2100~10 The method and system of the present invention preferably utilizes an IBM PC platform that operates under DOS 5.0 and Windows 3Ø This platform provides a rich design environment and option flexibility.
The target system preferably includes, at a minimum, an IBM 386/486 or compatible computer with a 200Mb hard drive, 8 Mb of RAM, VGA display, Video card, a computer controlled VCR, optional speakers, and a direct connection to a press PLC network as illustrated in Figure 4.
There is illustrated in Figure 5, a software portion of the system. The software includes commercially available software and one custom piece ; 15 of software called a presentation server 26. The Windows 3.0 environment allows multiple software package to run concurrently-and communicate between themselves. Communication between the software packages leverages the-software packages $unctionality into a integrated solution and minimizes software development.
The diagnostic system 20, the knowledge acquisition tool 24 and the database 22 are preferably incorporated in a software package called "Advisor"
provided by Emerald Intelligence of Ann Arbor, Michigan. Advisor provides the environment to build ~-and execute diagnostics. To assign (i.e. map) and present (i.e. display) multimedia information for each ~ ~ of the diagnostic tree nodes as illustrated in Figure 1 30 6, Advisor communicates and interacts with the presentation server 26. The presentation server 26 then communicates with the media specific software ~ ~ packages (i.e. Microsoft Word for documentation media) I ~ to select and view.
. .
~100~1~ '. .
With this architecture, it is possible for each diagnostic node to have multimedia information assigned or mapped thereto thus improving the communication between the operator and the computer. -The standard forms of multimedia utilized are:
drawings, documentations, graphics, photographs, full motion video and audio, animation, sound, etc. For example, this approach can provide a better and easier ;
means of on-line interactive repair and training procedures (i.e. by using full motion video) to the operator when a cause of the problem is determined.
Also, because the architecture allows for the multimedia information to be used in its original -format, there is no need to re-engineer the media infor~ation-for it to be deliverable.
Based on this software architecture, there are two system utilities: the runtime solution or utility referred to as RUNTIME and the diagnostic tree building solution or utility referred to as~BUILDER, as also described hereinbelow.
.
RUNTIME
Within the RUNTIME software there are two forms of executing the diagnostics: PLC directed or operator step-by-step operation. The PLC directed ~ ~ 25 approach involves the computer receiving a predefined 5~ ~ : PLC fault code relative to a press malfunction. Once the PLC ~ault is recognized, the software jumps to a specific assigned location in the diagnostic tree (the same diagnostic process can be completed using the operator step-by-step approa~h, discussed below~. The benefit of this approach is to minimize operator-to-~ ~ computer diagnostic dialogue and get to the cause of .
, .
. , . ,; . , .. . ., . . .. , , . . ,.... . - ., . . . ~
the problem as guickly as possible. The key - -limitation of this approach is that it only applies to sensory faults. Non-sensory faults will need to be diagnosed in the step-by-step approach.
In either of the approaches (PLC directed or step-by-step), the operator is asked a series of questions until he reaches the cause of the press problem. In the step-by-step approach, the diagnostics always starts from the top of the tree.
Answers for questions are presented in one of two formats: text or graphics with text. The text type answer is a traditional format illustrated in Figure 7, and is the easiest to generate and least time-consuming. The graphics with text, as illustrated in Figure 8, is a format that conveys the message faster ~ -and with less effort for the operator, but does take more development time because of the graphic development. Both of the answer formats can be intermixed in the diagnostics, but are defined and Sixed in the diagnostic at the time of tree development.
Another important feature in both the Question & Answer formats is the placeholders. The placeholder reminds the operator of the path that has 2 been chosen and allows the operator to "Back Up" to a previous diagnostic screen.
Also~ at any time during the diagnostic session, if related machine specific support information has been assigned to a particular questiOn & answer or solution screen, an "i" button is displayed in the upper left hand corner as illustrated in Figures 7 and 8. By selecting the "i" button, the presentation server 26 is activated as illustrated in Figure 9, which allows the operator to view the - . .. . . ~ . : , ,: : . ~ :. .. . .. .
2100~
:.. , different assigned media relative to that node function in the form of multimedia commands. More detail on the use of the runtime solution is described hereinbelow. ~ -BUILDER
BUILDER allows one to define and/or maintain machine diagnostic trees (a graphical representation of a diagnostic decision step or ~low) and assign multimedia machine information to the respective tree node.
To briefly demonstrate the ease and -; - -simplicity of creating a diagnostic tree, reference may be made to Figure 10. Diagnostic trees are made up o~ branches. Branches are made up of four primary components:
* Question & Answer (Q&A) nodes * Solutions nodes * Branch jump nodes * links or lines.
The Q&A node states a question with an unlimited number of answers. The solution node states the cause of the problem and needed repair for the cause of the problem. To assign a Q&A node (or solution node), a point mark is first located on the screen (click mouse) where one would like to place the node. Second, the type of node is selected from the above screen menu select: OBJECT-CREATE-SYMPTOM for Q&A node, or OBJECT-CREATE-SOLUTION for a solution node. This procedure places the object node on the screen as previously marked. To fill in the node name, the Q&A, or solution text information, the node is selected with the mouse pointer and the text is , .
.
' ,. ~ ' ~' ' , ' ' .. . ~. '' .-. :
: . ~: ' . .,'. ' :' .~ ', , . . ~ , . . . . .
2100~ 0 entered. Graphical answers with text can alternatively be used to replace the standard textual answers. To add more answers to a Q&A node, the node is selected and then OBJECT-ADD VALUE/TEST is selected from the menu screen. To link an answer to the next -question or solution, a right arrow icon is selected in the preferred answer box then the mouse pointer is dragged to the respective question, branch or solution and the mouse button is released. This places a link or line between the two object nodes.
The above steps are repeated until the desired tree is built. Objects within a screen can freely be moved around, primarily for appearance and readability, without effecting the linking. Anytime while editing a tree, one can run the diagnostics by selecting from the screen menu, FILE-RUN TEST. This will bring one into the RUNTIME (test mode) with the current tree information.
To define machine specific multimedia information for any of the Q&A and solution nodes, one needs to access the presentation server screen.
First, one double click on "?" in the Q&A box or just on the solution box. This brings up the question or solution editor screen and then one enters CTRL and -~
keys, which brings up the presentation server (screen illustrated in Figure 11). The keys do this by ~ -executing a recorded Windows 3.0 macro. To assign the desired media (documentation, drawing, photograph or video), the button on the left is selected. A file directory box (not shown) pops up to allow one to look up the hard disk for the desired file. Once the file is selected, there is an opportunity to preview the material, define required additional information (i.e.
page number within a documentation) and assign comment 2100510 .
information before mapping. When the media is selected, it is listed in the appropriate media box as illustrated in Figure 11. There are, preferably, five different choices of material for each of the forms of media. When all the media is assigned, OK is entered to return to the builder screen.
, Building Diagnostic Trees And Assigning Multimedia Information Usinq Gra~hic Icons .
The process here is similar to the above BUILDER process but is simpler to generate trees and assign multimedia information. There is also the use of more graphic icons (i.e. see the solution block of Figur~ 12) instead of menu selections.
STEP 1:
To an assign symptom block (Q&A block) or solution block, an icon on the left in Figure 12 is first selected, and located on the right side or work area of Figure 12.
To fill in the Q&A or solution block text information name, the block field with the mouse pointer is selected and the text is entered. To link an answer to the next question or solution, the link icon is selected and the respective blocks are connected. This places a link or line between the two blocks.
The above steps are repeated until the desired tree is built. Blocks within a screen can freely be moved around, primarily for appearance and readability, without effecting the linking.
.
:
. . . . ~ : ~ : .. ..
. . . .- ., . . . :
- 2100~0 :
The use of icons is a unique method for building diagnostic trees and defining multimedia information. Prior solutions required~menu selections or involved accessing other screens.
. .
STEP 2:
To define machine specific multimedia information for any of the Q&A and solution blocks, a media icon of choice is selected on the left side of the screen of Figure 12.
This media icon is placed near a Q&A or~
solution block of choice and is then linked to the desired block. By reselecting an -assigned media icon on the right side, a ~ --media selection screen pops up as illustrated in Figure 13. Once the file is ~ -~
selected, there is the opportunity~to~
preview the material, define required additional information (i.e.~ page number within a documentation) and assign comment information before mapping. ~ ~ -This step is unique, because of the use of direct visual aids (icons) and ` programming to assign~the media to its respective block. This process is simpler than~prior solutions because everything is accomplished on one screen. The media remains in~its natural format and does not require any re-engineering. ;~ -' ' ' '; ~ ` ' ' ' . '` ' ' ~ ` : ' , ' i ;
2100~
STARTUP ~;
First one turns on the computer power or reboots the system. After a few minutes (software applications are being loaded), the first screen of the system as illustrated in Figure 14 is displayed.
This initial Question & Answer node is displayed on a tree display portion of the screen. This initial node requests one to select what transfer press subsystem (Feeder, Press or Destacker) has a problem. After selection in response to an initial command, additional Question & Answer screens are displayed until one reaches a solution screen or node. Details of each of the different types of screens and their operations are described hereinbelow.
OUESTION AND ANSWER SCREEN
The diagnostic screen of Figure 15 is an example of a Question & Answer (Q&A) screen. The standard Q&A screen is broken down into three major components: the question, the answer, and the placeholder. Also, in some of the Q&A screens, the Info Button is displayed, as highlighted in Figure 15, which means that there is additional multimedia information defined with the question. ~-.
OUESTION BOX
-For each diagnostic screen, there is a question posted and thus m~st be answered. The question is typically stated in text format only.
: ' ' , - ~
.
' ,: . .: .: ,.:,: . . ,, , , . :.:~ ,, ,, . ,.. -, . , , ,. - -. .: .: . : . , .. :..... : . .. .; . : ~ . .
-: ~ ...... ... : : . .. . ..
21~5~0 ' ` ~'. ':
ANSWER AREA
In the answer area, there can be two types of answers: text or graphics with text as shown in Figure 15. The next type is just a sentence as shown in Figure 7. The graphics with text type shows a picture that represents the answer. To answer a question, the correct answer (line of text or picture) is selected.
~ACEHOLDER
: ~ .
Once an answer is selected, a box with a -picture or text, is placed in the placeholder area or branch display portion of~the tree display~portlon of the screen. These placeholder~boxes represent the diagnostic path that has been taken. Selecting~a placeholder box allows one to "BArKUP" to a~previous diagnostic screen.
l~ORMATION
.
By selecting the Info Button, "i", the multimedia presentation screen pops up, as shown in ~Figure 9. ~This;screen allows one to select~desired ; media information (documentation, photographsj prints/drawings~and full motion~video) in~response to the received user command.~ ~The~multimedia in$ormation is there for the~operator~to help answer any questions 25 ~ or to explain testing or repair procedures~about the current dlagnostic screen.
~.: - . - ~ .
view screen For Text Document The screen of Figure 17 displays a text document, which can have embedded graphics, for a respective diagnostic screen. The whole page is not displayed. To view the other portions of the page, the arrow buttons ~highlighted) are selected to move about the page. To look at the previous or next page within the document, the page symbol at the right hand side of the screen is selected (as highlighted).
Finally, to exit and return back to the diagnostic screen "File" is selected from the menu and then "Done" is chosen from the "File Menu."
YleyL~L-~een For Dràwinq ~ ~
The screen of Figure 18 displays an AutoCad drawing file (also called a print) and predefined views within an AutoCad file. These drawings can be an electrical, pneumatic, hydraulic or mechanical - drawing. To exit the drawing, the screen is touched.
View Screen For Still Video ~ 20 By selecting any of the still video (same as a photograph) buttons (the camera icon), the image is display~d in the top left side of the screen. An associated comment about the still video may ~e displayed above the still buttons as illustrated in Figures 16 and 19.
~. ~
:~, .~, ' ^
.
~, :2 ~ 0~
View Screen For Full Motion Video By selecting any of the full motion video -buttons (the video tape icon), the video image is displayed in the top left side of the screen as illustrated in Figure 20. Prior to displaying the video, a comment is displayed at the top of the box.
This gives the operator the chance to determine if it is the right video to view. If not, another video button is selected. If it is the right video, the VIEW button is selected and the VCR will start searching for the tape segment under control of the computer. Once the video is found, it is displayed in the upper left side of the screen. Volume (if speakers are attached) and brightness can be adjusted ~ -while monitoring video tape. Two other video control buttons allow the audio to be muted and the video to be frozen (the VCR is still playing at this time and only the image on the screen is frozen). To exit once the video is completed, the DONE button is selected in the video control box.
Solution Screen ' ;: .
When the last node, or the Solution, has ~--been reached, the screen of Figure 21 is displayed.
If there is any`help information assigned at this ~25 node, the help information is displayed first. At this point, one can still back up if needed or save this particu~lar diagnostic session in a session file ~ (*.SES). ~ -: ~ ~ ~ - .' ' :
.
-.. .. .,~..
:~
~g ~
To save session To save current session from the top to current node, File-Save Session menu is selected. The appropriate file name is typed in. The default file extension is (*.SES).
To restore session To restore a previously saved diagnostic session, File-Restore Session menu is ~--selected. Then, the session file name is selected. By restoring session, one~can trace the previous path or immediately return to the solution node.
The system as described herein below diagnoses three primary components of a machine tool such as a transfer press: the press, the feeder, and the destacker. The press and the feeder diagnostic trees and related information are included. The press and feeder trees may address as many as 143 primary symptoms that include approximately 800 nodes.
Building these two trees and accumulating and ;~ assigning the multimedia information is relatively straight forward after the paper, drawings and documentation~were converted into electronic format.
The method and system can be used to visually locate components on a large and complex transfer press. This has a significant impact in reducing the repair time because plant electricians can immediately locate defective component(s).
The method and system is not limited to stamping press~diagnostics, but may be used for other forms of diagnostics (i.e. other plant machinery and :
2100~10 office applications) and for use on non-machine diagnostics (i.e. process diagnostics or as engineering design aid).
The method and system provide the following advantages for use with a press: machine diagnostics when the machine is delivered; better transfer of technology and documentation from the vendor to the customer; reduced press down time; reduced diagnostic and repair time; better informed repair personnel, especially the apprentice; on-line training; better utilization of vendor/machine information; better service to the machine; the opportunity to capture new and refined diagnostic knowledge from plant personnel;
reusability of information, that can be applied to other similar machines; and actual machine diagnostic flow and cause information can be captured to provide good feedback to the supplier with the hope of making better equipment and diagnostic models in the future. -The method and system can be used to provide a multi-lingual solution. Using a stereo VCR, one audio track may contain information in a first language and the other audio track may contain information in a second language. Further, a multi-lingual solution is made possible through the selection of alternate keyboard assignments using the windows operating environment.
The present system requires well disciplined and structured information management techniques to allow for ease of upgradeability,~maintenance and - -adjustments. This process is also invisible and seamless to the people making these changes.
The presented media information demands high storage space but can be used for other similar and identical machines. To optimize storage and decrease ~; ' 21~0~1~
~1-809/810 redundancy of information, there may be a centralized network/file server solution. While there is currently no clean way to store, retrieve and transmit .
analog information (video and audio) over a data ~ : .
network, a complete centralized library of machine information such as 2 and 3 dimensional animation, multi-language media, audible answers to diagnostic questions, speech, 3-D drawings, and a point-to-point distribution system can be accommodated when such is possible.
While the best mode for carrying out the invention has been described in detail, those familiar `
with the art to which this invention relates will recognize various alternative designs and embodiments : .
for practicing the invention as defined by the ~oll~w ~g cl~i~s.
, , . , .,. , .. .. . , ,. : .~ , :. . . . .. , . .: : -. :
Claims (23)
1. A method for processing and presenting multimedia information in a tree structure including branches having a plurality of nodes and node-connecting links in a computer system having tree display means for displaying nodes of the tree structure, the method comprising the steps of:
providing command display means for displaying multimedia commands representative of various types of multimedia information;
providing multimedia output means for providing multimedia information to a user of the system;
providing a knowledge base for storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting the text strings that can be displayed on the tree display means;
means for denoting the multimedia commands that can be displayed on the command display means; and means for denoting the multimedia information that can be presented by said multimedia output means;
receiving an initial command;
displaying at least one selected node of the tree structure including at least one text string on the tree display means in response to the initial command;
displaying a plurality of multimedia commands corresponding to the at least one selected node on the command display means;
receiving a user command related to one of the displayed multimedia commands; and delivering a selected amount of the multimedia information to the multimedia output means to be presented thereby to the user of the system in response to the user command.
providing command display means for displaying multimedia commands representative of various types of multimedia information;
providing multimedia output means for providing multimedia information to a user of the system;
providing a knowledge base for storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting the text strings that can be displayed on the tree display means;
means for denoting the multimedia commands that can be displayed on the command display means; and means for denoting the multimedia information that can be presented by said multimedia output means;
receiving an initial command;
displaying at least one selected node of the tree structure including at least one text string on the tree display means in response to the initial command;
displaying a plurality of multimedia commands corresponding to the at least one selected node on the command display means;
receiving a user command related to one of the displayed multimedia commands; and delivering a selected amount of the multimedia information to the multimedia output means to be presented thereby to the user of the system in response to the user command.
2. A system for processing and presenting multimedia information in a tree structure including branches having a plurality of nodes and node-connecting links in a computer system having tree display means for displaying nodes of the tree structure, the system comprising:
command display means for displaying multimedia commands representative of various types of multimedia information;
multimedia output means for providing multimedia information to a user of the system;
a knowledge base for storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting the text strings that can be displayed on the tree display means;
means for denoting the multimedia commands that can be displayed on the command display means; and means for denoting the multimedia information that can be presented by said multimedia output means;
means for receiving an initial command, the tree display means displaying at least one selected node of the tree structure including at least one text string in response to the initial command, and the command display means displaying a plurality of multimedia commands corresponding to the at least one selected node on the command display means; and means for receiving a user command related to one of the displayed multimedia commands, the multimedia output means presenting a selected amount of the multimedia information to the user of the system in response to the user command.
command display means for displaying multimedia commands representative of various types of multimedia information;
multimedia output means for providing multimedia information to a user of the system;
a knowledge base for storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting the text strings that can be displayed on the tree display means;
means for denoting the multimedia commands that can be displayed on the command display means; and means for denoting the multimedia information that can be presented by said multimedia output means;
means for receiving an initial command, the tree display means displaying at least one selected node of the tree structure including at least one text string in response to the initial command, and the command display means displaying a plurality of multimedia commands corresponding to the at least one selected node on the command display means; and means for receiving a user command related to one of the displayed multimedia commands, the multimedia output means presenting a selected amount of the multimedia information to the user of the system in response to the user command.
3. The invention as claimed in claim 1 or claim 2 wherein the knowledge base-references at least a portion of the multimedia information stored in analog form.
4. The invention as claimed in claim 1 or claim 2 wherein the knowledge base stores at least a portion of the multimedia information in digital form.
5. The invention as claimed in claim 1 or claim 2 wherein the computer system is an expert computer system.
6. The invention as claimed in claim 1 or claim 2 wherein at least a portion of the multimedia information is diagnostic information.
7. The invention as claimed in claim 1 or claim 2 wherein the at least one selected node is a question and answer node.
8. The invention as claimed in claim 1 or claim 2 wherein the nodes of the tree structure include question and answer and branch jump nodes.
9. The invention as claimed in claim 1 or claim 2 wherein the nodes of the tree structure include question and answer, branch jump and solution nodes.
10. The invention as claimed in claim 1 or claim 2 wherein the tree structure is a diagnostic tree structure.
11. The invention as claimed in claim 1 or claim 2 wherein the multimedia output means includes video display means for displaying video images.
12. The invention as claimed in claim 1 or claim 2 wherein the multimedia output means includes sound generating means for generating sound messages.
13. The invention as claimed in claim 1 or claim 2 wherein the multimedia output means includes drawing display means for displaying drawing images.
14. The invention as claimed in claim 1 or claim 2 wherein the multimedia output means includes graphics display means for displaying graphical images.
15. The invention as claimed in claim 1 or claim 2 wherein the multimedia output means includes document display means for displaying images of documents.
16. The invention as claimed in claim 1 or claim 2 wherein the knowledge base stores a multiplicity of digitized video images, each of the stored video images corresponding to one or more pieces of equipment from a predefined set of pieces of equipment.
17. The invention as claimed in claim 1 or claim 2 wherein the multimedia commands are displayed on the command display means in the form of graphic icons.
18. The invention as claimed in claim 1 or claim 2 wherein the initial command is a user command.
19. The invention as claimed in claim 1 or claim 2 wherein the tree display means, the command display means and the multimedia output means define a video display of the computer system.
20. The invention as claimed in claim 1 or claim 2 wherein the tree display means includes branch display means for displaying the selected nodes of the tree structure.
21. The invention as claimed in claim 1 or claim 2 wherein the information records further include means for denoting graphical images that can be displayed on the tree display means.
22. A method for creating a tree structure for processing and presenting multimedia information, said tree structure including branches having a plurality of nodes and node-connecting links in a computer system having tree display means for displaying nodes of the tree structure, the method comprising the steps of:
providing a computer system, the computer system including:
command display means for displaying multimedia commands representative of various types of the multimedia information;
multimedia output means for providing the multimedia information to the system;
multimedia input means for providing the multimedia information to the system;
multimedia storage means for storing the multimedia information;
inputing the multimedia information to the computer system;
storing the multimedia information;
providing a knowledge base for storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting text strings that can be displayed on the tree display means;
means for denoting multimedia commands that can be displayed on the command display means;
and means for denoting the multimedia information that can be presented by said multimedia output means; and assigning the stored multimedia information to one or more of the information records.
providing a computer system, the computer system including:
command display means for displaying multimedia commands representative of various types of the multimedia information;
multimedia output means for providing the multimedia information to the system;
multimedia input means for providing the multimedia information to the system;
multimedia storage means for storing the multimedia information;
inputing the multimedia information to the computer system;
storing the multimedia information;
providing a knowledge base for storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting text strings that can be displayed on the tree display means;
means for denoting multimedia commands that can be displayed on the command display means;
and means for denoting the multimedia information that can be presented by said multimedia output means; and assigning the stored multimedia information to one or more of the information records.
23. A system for creating a tree structure for processing and presenting multimedia information, said tree structure including branches having a plurality of nodes and node-connecting links in a computer system having tree display means for displaying nodes of the tree structure, the system comprising:
a computer system, the computer system including:
command display means for displaying multimedia commands representative of various types of the multimedia information;
multimedia output means for providing the multimedia information to a user of the system;
multimedia input means for providing the multimedia information to the system;
multimedia storage means for storing the multimedia information in the form of a knowledge base, the knowledge base storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting text strings that can be displayed on the tree display means;
means for denoting multimedia commands that can be displayed on the command display means;
and means for denoting the multimedia information that can be presented by said multimedia output means; and means for assigning said stored multimedia information to at least one information record.
a computer system, the computer system including:
command display means for displaying multimedia commands representative of various types of the multimedia information;
multimedia output means for providing the multimedia information to a user of the system;
multimedia input means for providing the multimedia information to the system;
multimedia storage means for storing the multimedia information in the form of a knowledge base, the knowledge base storing a multiplicity of information records mapped into the tree structure, the information records including:
means for denoting text strings that can be displayed on the tree display means;
means for denoting multimedia commands that can be displayed on the command display means;
and means for denoting the multimedia information that can be presented by said multimedia output means; and means for assigning said stored multimedia information to at least one information record.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US07/953,034 US5539869A (en) | 1992-09-28 | 1992-09-28 | Method and system for processing and presenting on-line, multimedia information in a tree structure |
| US07/953,034 | 1992-09-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2100510A1 true CA2100510A1 (en) | 1994-03-29 |
Family
ID=25493487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002100510A Abandoned CA2100510A1 (en) | 1992-09-28 | 1993-09-23 | Method and system for processing and presenting on-line multimedia information in a tree structure |
Country Status (6)
| Country | Link |
|---|---|
| US (1) | US5539869A (en) |
| JP (1) | JPH06214738A (en) |
| KR (1) | KR940007705A (en) |
| CA (1) | CA2100510A1 (en) |
| DE (1) | DE4332193A1 (en) |
| GB (1) | GB2271005A (en) |
Families Citing this family (141)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5963931A (en) * | 1992-10-05 | 1999-10-05 | Expert Systems Publishing Co. | Computer-assisted decision management system |
| JP3634391B2 (en) * | 1993-09-30 | 2005-03-30 | 富士通株式会社 | Multimedia information addition system |
| EP0658855A1 (en) * | 1993-12-16 | 1995-06-21 | International Business Machines Corporation | Method and system for integration of multimedia within an object oriented user interface |
| EP0666542A3 (en) * | 1994-02-04 | 1996-05-15 | Fuji Facom Corp | Multimedia process monitor and control system. |
| US5608909A (en) * | 1994-04-15 | 1997-03-04 | Microsoft Corporation | Method and system for caching presentation data of a source object in a presentation cache |
| AUPM704494A0 (en) * | 1994-07-25 | 1994-08-18 | Canon Information Systems Research Australia Pty Ltd | Efficient methods for the interpretation of a graphical programming language |
| US5632007A (en) * | 1994-09-23 | 1997-05-20 | Actv, Inc. | Interactive system and method for offering expert based interactive programs |
| JPH08137647A (en) * | 1994-11-07 | 1996-05-31 | Fujitsu Ltd | Method and device for reproducing operation guide information reproduction, method and device for reproducing multimedia information |
| JP2985697B2 (en) * | 1994-12-16 | 1999-12-06 | 株式会社日立製作所 | Sound output method of image information |
| DE4445617A1 (en) * | 1994-12-21 | 1996-06-27 | Sel Alcatel Ag | Process, system component and program module for selecting a search term |
| US5623589A (en) * | 1995-03-31 | 1997-04-22 | Intel Corporation | Method and apparatus for incrementally browsing levels of stories |
| AU5442796A (en) * | 1995-04-06 | 1996-10-23 | Avid Technology, Inc. | Graphical multimedia authoring system |
| US5675753A (en) * | 1995-04-24 | 1997-10-07 | U.S. West Technologies, Inc. | Method and system for presenting an electronic user-interface specification |
| JPH0916609A (en) * | 1995-04-25 | 1997-01-17 | Matsushita Electric Ind Co Ltd | Device and method for retrieving document |
| US5850221A (en) * | 1995-10-20 | 1998-12-15 | Araxsys, Inc. | Apparatus and method for a graphic user interface in a medical protocol system |
| US5786816A (en) * | 1995-10-20 | 1998-07-28 | Araxsys, Inc. | Method and apparatus for graphical user interface-based and variable result healthcare plan |
| US5826237A (en) * | 1995-10-20 | 1998-10-20 | Araxsys, Inc. | Apparatus and method for merging medical protocols |
| US5886693A (en) * | 1995-10-20 | 1999-03-23 | Araxsys, Inc. | Method and apparatus for processing data across a computer network |
| US5737739A (en) * | 1995-12-19 | 1998-04-07 | Xerox Corporation | System that accesses a knowledge base by markup language tags |
| US5760771A (en) * | 1996-07-17 | 1998-06-02 | At & T Corp | System and method for providing structured tours of hypertext files |
| US5862223A (en) | 1996-07-24 | 1999-01-19 | Walker Asset Management Limited Partnership | Method and apparatus for a cryptographically-assisted commercial network system designed to facilitate and support expert-based commerce |
| US6628303B1 (en) * | 1996-07-29 | 2003-09-30 | Avid Technology, Inc. | Graphical user interface for a motion video planning and editing system for a computer |
| US6059494A (en) * | 1996-09-09 | 2000-05-09 | Thermwood Corporation | Tool bit monitoring system for machine tools |
| US5893082A (en) * | 1996-12-23 | 1999-04-06 | Mccormick; John M. | System for processing and presenting cost estimates in the construction industry |
| US6009355A (en) | 1997-01-28 | 1999-12-28 | American Calcar Inc. | Multimedia information and control system for automobiles |
| US6151683A (en) * | 1997-03-31 | 2000-11-21 | Sun Microsystems, Inc. | Rebuilding computer states remotely |
| US6028601A (en) * | 1997-04-01 | 2000-02-22 | Apple Computer, Inc. | FAQ link creation between user's questions and answers |
| US6677960B1 (en) * | 1997-05-07 | 2004-01-13 | Unisys Corporation | User assistance for heterogeneous data processing systems |
| US6182249B1 (en) | 1997-05-12 | 2001-01-30 | Sun Microsystems, Inc. | Remote alert monitoring and trend analysis |
| US6154128A (en) * | 1997-05-21 | 2000-11-28 | Sun Microsystems, Inc. | Automatic building and distribution of alerts in a remote monitoring system |
| US6148261A (en) | 1997-06-20 | 2000-11-14 | American Calcar, Inc. | Personal communication system to send and receive voice data positioning information |
| US5864337A (en) * | 1997-07-22 | 1999-01-26 | Microsoft Corporation | Mehtod for automatically associating multimedia features with map views displayed by a computer-implemented atlas program |
| US6275231B1 (en) * | 1997-08-01 | 2001-08-14 | American Calcar Inc. | Centralized control and management system for automobiles |
| GB2329042B (en) * | 1997-09-03 | 2002-08-21 | Ibm | Presentation of help information via a computer system user interface in response to user interaction |
| NZ503089A (en) * | 1997-09-30 | 2002-02-01 | Crown Equip Corp | Diagnosing for materials handling vehicles |
| US5995101A (en) * | 1997-10-29 | 1999-11-30 | Adobe Systems Incorporated | Multi-level tool tip |
| EP0915422B1 (en) * | 1997-11-06 | 2006-03-01 | Hewlett-Packard Company | Expert system |
| US6237114B1 (en) | 1998-05-13 | 2001-05-22 | Sun Microsystems, Inc. | System and method for evaluating monitored computer systems |
| US6259448B1 (en) | 1998-06-03 | 2001-07-10 | International Business Machines Corporation | Resource model configuration and deployment in a distributed computer network |
| DE19847894A1 (en) * | 1998-10-16 | 2000-04-20 | Focke & Co | Method and device for displaying operating instructions |
| US6260048B1 (en) * | 1998-10-27 | 2001-07-10 | Hewlett-Packard Company | Resolution of incidents which occur during the use of a product |
| FR2786014B1 (en) * | 1998-11-16 | 2001-01-05 | Clotures Le Brun Sarl | INCIDENT MANAGEMENT SYSTEM FOR INDUSTRIAL PORTALS |
| US6032141A (en) * | 1998-12-22 | 2000-02-29 | Ac Properties B.V. | System, method and article of manufacture for a goal based educational system with support for dynamic tailored feedback |
| US6018731A (en) * | 1998-12-22 | 2000-01-25 | Ac Properties B.V. | System, method and article of manufacture for a goal based system utilizing a spreadsheet and table based architecture |
| US6018732A (en) * | 1998-12-22 | 2000-01-25 | Ac Properties B.V. | System, method and article of manufacture for a runtime program regression analysis tool for a simulation engine |
| US6003021A (en) * | 1998-12-22 | 1999-12-14 | Ac Properties B.V. | System, method and article of manufacture for a simulation system for goal based education |
| US6018730A (en) | 1998-12-22 | 2000-01-25 | Ac Properties B.V. | System, method and article of manufacture for a simulation engine with a help website and processing engine |
| US6064998A (en) * | 1998-12-22 | 2000-05-16 | Ac Properties, B.V. | System, method and article of manufacture for a simulation engine with an expert system example processing engine |
| US6023691A (en) * | 1998-12-22 | 2000-02-08 | Ac Properties B.V. | Goal based stimulator utilizing a spreadsheet architecture |
| US6029159A (en) * | 1998-12-22 | 2000-02-22 | Ac Properties B.V. | System, method and article of manufacture for a simulation enabled accounting tutorial system |
| US6073127A (en) * | 1998-12-22 | 2000-06-06 | Ac Properties B.V. | System, method and article of manufacture for a goal based system with dynamic feedback information |
| US6029158A (en) | 1998-12-22 | 2000-02-22 | Ac Properties B.V. | System, method and article of manufacture for a simulation enabled feedback system |
| US6745170B2 (en) | 1999-02-08 | 2004-06-01 | Indeliq, Inc. | Goal based educational system with support for dynamic characteristic tuning |
| US6023692A (en) * | 1998-12-22 | 2000-02-08 | Ac Properties B.V. | Goal based tutoring system with behavior to control flow of presentation |
| US6535861B1 (en) | 1998-12-22 | 2003-03-18 | Accenture Properties (2) B.V. | Goal based educational system with support for dynamic characteristics tuning using a spread sheet object |
| US6067538A (en) * | 1998-12-22 | 2000-05-23 | Ac Properties B.V. | System, method and article of manufacture for a simulation enabled focused feedback tutorial system |
| US6782374B2 (en) | 1998-12-22 | 2004-08-24 | Accenture Global Services Gmbh | System, method and article of manufacturing for a runtime program analysis tool for a simulation engine |
| US6125358A (en) * | 1998-12-22 | 2000-09-26 | Ac Properties B.V. | System, method and article of manufacture for a simulation system for goal based education of a plurality of students |
| US6085184A (en) * | 1998-12-22 | 2000-07-04 | Ac Properties B.V. | System, method and article of manufacture for a dynamic toolbar in a tutorial system |
| US6016486A (en) * | 1998-12-22 | 2000-01-18 | Ac Properties B.V. | System method and article of manufacture for a goal based system utilizing an activity table |
| US6026386A (en) * | 1998-12-22 | 2000-02-15 | Ac Properties B.V. | System, method and article of manufacture for a goal based educational system with support for dynamic media control |
| US6101489A (en) | 1998-12-22 | 2000-08-08 | Ac Properties, B.V. | System, method and article of manufacture for a goal based system utilizing a time based model |
| US6067537A (en) * | 1998-12-22 | 2000-05-23 | Ac Properties B.V. | System, method and article of manufacture for a goal based educational system with support for dynamic personality feedback |
| US6134539A (en) * | 1998-12-22 | 2000-10-17 | Ac Properties B.V. | System, method and article of manufacture for a goal based education and reporting system |
| US6029156A (en) * | 1998-12-22 | 2000-02-22 | Ac Properties B.V. | Goal based tutoring system with behavior to tailor to characteristics of a particular user |
| US6542880B2 (en) * | 1998-12-22 | 2003-04-01 | Indeliq, Inc. | System, method and article of manufacture for a goal based system utilizing a table based architecture |
| US5987443A (en) | 1998-12-22 | 1999-11-16 | Ac Properties B. V. | System, method and article of manufacture for a goal based educational system |
| US6550055B1 (en) * | 1998-12-29 | 2003-04-15 | Intel Corp. | Method and apparatus for cheating an information report resulting from a diagnostic session |
| US6970858B1 (en) | 1999-02-08 | 2005-11-29 | Accenture, Llp | Goal based system utilizing an activity table |
| US7065513B1 (en) | 1999-02-08 | 2006-06-20 | Accenture, Llp | Simulation enabled feedback system |
| US7194444B1 (en) | 1999-02-08 | 2007-03-20 | Indeliq, Inc. | Goal based flow of a control presentation system |
| US7386524B2 (en) | 1999-02-08 | 2008-06-10 | Accenture Global Services Gmbh | Simulation enabled focused feedback tutorial system |
| US7065512B1 (en) | 1999-02-08 | 2006-06-20 | Accenture, Llp | Dynamic toolbar in a tutorial system |
| US7156665B1 (en) | 1999-02-08 | 2007-01-02 | Accenture, Llp | Goal based educational system with support for dynamic tailored feedback |
| US7089222B1 (en) | 1999-02-08 | 2006-08-08 | Accenture, Llp | Goal based system tailored to the characteristics of a particular user |
| US7054848B1 (en) | 1999-02-08 | 2006-05-30 | Accenture, Llp | Goal based system utilizing a time based model |
| DE19905630A1 (en) * | 1999-02-11 | 2000-08-17 | Deutsche Telekom Ag | Process for the graphic representation and / or processing of values of data types |
| DE19907328C2 (en) * | 1999-02-20 | 2002-10-24 | Johannes Reichardt | Visual programming method and system |
| US6341246B1 (en) * | 1999-03-26 | 2002-01-22 | Kuka Development Laboratories, Inc. | Object oriented motion system |
| US6611822B1 (en) | 1999-05-05 | 2003-08-26 | Ac Properties B.V. | System method and article of manufacture for creating collaborative application sharing |
| US20030023686A1 (en) * | 1999-05-05 | 2003-01-30 | Beams Brian R. | Virtual consultant |
| US7152092B2 (en) | 1999-05-05 | 2006-12-19 | Indeliq, Inc. | Creating chat rooms with multiple roles for multiple participants |
| US6993513B2 (en) * | 1999-05-05 | 2006-01-31 | Indeliq, Inc. | Interactive simulations utilizing a remote knowledge base |
| US20020138590A1 (en) * | 2000-05-05 | 2002-09-26 | Beams Brian R. | System method and article of manufacture for creating a virtual university experience |
| AU4826100A (en) | 1999-05-05 | 2000-11-17 | Accenture Properties (2) Bv | System, method and article of manufacture for creating collaborative simulationswith multiple roles for a single student |
| US6370683B1 (en) | 1999-05-07 | 2002-04-09 | Arnold Sobers | Computer software for generating flowchart images of a source program |
| AUPQ139099A0 (en) * | 1999-07-02 | 1999-07-29 | Link Resources (Qld) Pty Ltd | Interactive training system |
| US6553290B1 (en) * | 2000-02-09 | 2003-04-22 | Oshkosh Truck Corporation | Equipment service vehicle having on-board diagnostic system |
| US6736642B2 (en) * | 1999-08-31 | 2004-05-18 | Indeliq, Inc. | Computer enabled training of a user to validate assumptions |
| US7016056B2 (en) * | 1999-09-02 | 2006-03-21 | Hewlett-Packard Development Company, L.P. | Authoring tool for bayesian network diagnostic systems |
| US7385716B1 (en) | 1999-09-02 | 2008-06-10 | Hewlett-Packard Development Company, L.P. | Authoring tool for bayesian network troubleshooters |
| US7337159B2 (en) * | 1999-09-30 | 2008-02-26 | Topiary Communications, Inc. | System and method for sharing of expert knowledge |
| US6985889B1 (en) | 1999-09-30 | 2006-01-10 | Topiary Communications, Inc. | System and method for sharing of expert knowledge |
| US7176949B1 (en) * | 1999-11-17 | 2007-02-13 | Moser Albert N | System, method and article of manufacture for an incremental explanatory object in a learning application assembly framework |
| US6833847B1 (en) * | 1999-12-21 | 2004-12-21 | International Business Machines Corporation | Visual wizard launch pad |
| WO2001046859A1 (en) * | 1999-12-21 | 2001-06-28 | Seiko Epson Corporation | Contents providing device and contents using device |
| US6652283B1 (en) * | 1999-12-30 | 2003-11-25 | Cerego, Llc | System apparatus and method for maximizing effectiveness and efficiency of learning retaining and retrieving knowledge and skills |
| US7187947B1 (en) | 2000-03-28 | 2007-03-06 | Affinity Labs, Llc | System and method for communicating selected information to an electronic device |
| US7725812B1 (en) | 2000-03-31 | 2010-05-25 | Avid Technology, Inc. | Authoring system for combining temporal and nontemporal digital media |
| US7555557B2 (en) * | 2000-04-07 | 2009-06-30 | Avid Technology, Inc. | Review and approval system |
| US6877115B2 (en) * | 2000-06-30 | 2005-04-05 | Sinapse Graphic International | Interactive on-line diagnostics for printing |
| US8990367B2 (en) * | 2006-09-29 | 2015-03-24 | Dell Products L.P. | Consistent binding of shared storage across clustered servers |
| KR20070118707A (en) * | 2000-09-21 | 2007-12-17 | 아메리칸 캘카어 인코포레이티드 | Technique for operating a vehicle effectively and safely |
| US20020047857A1 (en) * | 2000-10-20 | 2002-04-25 | Srinivas Venkatram | Systems and methods for directed knowledge management using the DISHA platform |
| US20030011629A1 (en) * | 2000-12-28 | 2003-01-16 | Rouse Jennifer K. | System to provide instructional information |
| US6671655B2 (en) | 2000-12-28 | 2003-12-30 | John A Moccio | Strategy based diagnostic system |
| US6750879B2 (en) * | 2000-12-30 | 2004-06-15 | Intel Corporation | Method and apparatus for communicating cable modem problem demarcation through a graphical user interface |
| JP4719987B2 (en) * | 2001-02-09 | 2011-07-06 | ソニー株式会社 | Screen display control method, program, and screen display control device |
| US7043702B2 (en) * | 2001-03-15 | 2006-05-09 | Xerox Corporation | Method for visualizing user path through a web site and a path's associated information scent |
| US7930624B2 (en) * | 2001-04-20 | 2011-04-19 | Avid Technology, Inc. | Editing time-based media with enhanced content |
| US20020188628A1 (en) * | 2001-04-20 | 2002-12-12 | Brian Cooper | Editing interactive content with time-based media |
| US20020199132A1 (en) * | 2001-06-22 | 2002-12-26 | Lachimia Joseph N. | Systems and methods for a knowledge-based power assembly system component diagnostic tool |
| CA2463922C (en) | 2001-06-27 | 2013-07-16 | 4 Media, Inc. | Improved media delivery platform |
| WO2003042823A1 (en) * | 2001-11-14 | 2003-05-22 | Exegesys, Inc. | Method and system for software application development and customizable runtime environment |
| DE10161570A1 (en) * | 2001-12-14 | 2003-07-03 | Fette Wilhelm Gmbh | Method for instructing an operator in maintenance and repair work on a tablet press |
| US20030120631A1 (en) * | 2001-12-21 | 2003-06-26 | Eastman Kodak Company | Method and system for hierarchical data entry |
| DE10203776A1 (en) * | 2002-01-30 | 2003-08-07 | Siemens Ag | Method for industry-specific software library for creating customer applications in industrial companies (Industry Suite) |
| DE10204310A1 (en) * | 2002-02-01 | 2003-08-14 | Correct Software Gmbh | Computer control method e.g. for producing or manipulating data, involves input of acoustic command signal |
| JP2004070651A (en) * | 2002-08-06 | 2004-03-04 | Fujitsu Ten Ltd | Electrical component control system and graphical user interface processing software structure |
| US8523575B2 (en) * | 2002-09-02 | 2013-09-03 | Nextthinksoft Pty Ltd. | Recalling items of information |
| US20040138867A1 (en) * | 2003-01-14 | 2004-07-15 | Simkins David Judson | System and method for modeling multi-tier distributed workload processes in complex systems |
| US9818136B1 (en) | 2003-02-05 | 2017-11-14 | Steven M. Hoffberg | System and method for determining contingent relevance |
| US7463281B2 (en) * | 2003-08-06 | 2008-12-09 | Microsoft Corporation | Smart vehicle video management |
| US20050131595A1 (en) * | 2003-12-12 | 2005-06-16 | Eugene Luskin | Enhanced vehicle event information |
| US20060095230A1 (en) * | 2004-11-02 | 2006-05-04 | Jeff Grier | Method and system for enhancing machine diagnostics aids using statistical feedback |
| US20060148570A1 (en) * | 2004-12-30 | 2006-07-06 | Samuel Leopold | Portable data techniques for gaming devices |
| FR2883086B1 (en) * | 2005-03-11 | 2007-05-04 | Alcatel Sa | DEVICE FOR OPTIMIZING DIAGNOSTIC TREES OF A DIAGNOSTIC TOOL OF A COMMUNICATION NETWORK |
| US20060271380A1 (en) * | 2005-05-25 | 2006-11-30 | Chowdhury Seshadri S | Systems and methods for managing product satisfaction |
| FR2886792B1 (en) * | 2005-06-07 | 2007-08-10 | Evolium Sas Soc Par Actions Si | COMMUNICATION NETWORK DIAGNOSTIC TOOL, OPERATING RESULTS OF REAL TESTS AND / OR VALIDATION BY CONTROL DISPLAY MODE |
| US8874477B2 (en) | 2005-10-04 | 2014-10-28 | Steven Mark Hoffberg | Multifactorial optimization system and method |
| US7830547B2 (en) * | 2005-11-30 | 2010-11-09 | Xerox Corporation | User interface assistant |
| US9021041B2 (en) * | 2006-06-29 | 2015-04-28 | Nextpat Limited | System to provide integrated on-line support |
| US7793219B1 (en) * | 2006-12-07 | 2010-09-07 | Adobe Systems Inc. | Construction of multimedia compositions |
| KR20080073480A (en) * | 2007-02-06 | 2008-08-11 | 삼성전자주식회사 | Flat cable and electronic appliance having the same |
| US20080244398A1 (en) * | 2007-03-27 | 2008-10-02 | Lucinio Santos-Gomez | Direct Preview of Wizards, Dialogs, and Secondary Dialogs |
| JP5158635B2 (en) * | 2008-02-28 | 2013-03-06 | インターナショナル・ビジネス・マシーンズ・コーポレーション | Method, system, and apparatus for personal service support |
| US20110153039A1 (en) * | 2009-12-23 | 2011-06-23 | Viktor Gvelesiani | System and method for providing diagnostic information and graphical user interface therefor |
| USD665407S1 (en) * | 2011-01-24 | 2012-08-14 | Microsoft Corporation | Display screen with graphical user interface |
| US8559793B2 (en) | 2011-05-26 | 2013-10-15 | Avid Technology, Inc. | Synchronous data tracks in a media editing system |
| US20140279718A1 (en) * | 2013-03-12 | 2014-09-18 | Xerox Business Services, Llc | Machine Assisted Troubleshooting of a Customer Support Issue |
| EP3020020A4 (en) * | 2013-07-10 | 2016-12-21 | Ifthisthen Inc | Systems and methods for knowledge management |
Family Cites Families (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4813013A (en) * | 1984-03-01 | 1989-03-14 | The Cadware Group, Ltd. | Schematic diagram generating system using library of general purpose interactively selectable graphic primitives to create special applications icons |
| US4656603A (en) * | 1984-03-01 | 1987-04-07 | The Cadware Group, Ltd. | Schematic diagram generating system using library of general purpose interactively selectable graphic primitives to create special applications icons |
| US4710763A (en) * | 1984-10-19 | 1987-12-01 | Texas Instruments Incorporated | Method for generating and displaying tree structures in a limited display area |
| JPH0711779B2 (en) * | 1986-02-21 | 1995-02-08 | 株式会社日立製作所 | Processing target instruction pattern display device |
| US4893256A (en) * | 1986-04-04 | 1990-01-09 | International Business Machines Corporation | Interactive video composition and presentation systems |
| US4902469A (en) * | 1986-05-05 | 1990-02-20 | Westinghouse Electric Corp. | Status tree monitoring and display system |
| DE3735117C2 (en) * | 1986-10-17 | 1993-10-21 | Sharp Kk | Electronic information search facility |
| EP0272886A3 (en) * | 1986-12-19 | 1990-12-19 | Pfu Limited | Image information processing system |
| US4945476A (en) * | 1988-02-26 | 1990-07-31 | Elsevier Science Publishing Company, Inc. | Interactive system and method for creating and editing a knowledge base for use as a computerized aid to the cognitive process of diagnosis |
| US4931950A (en) * | 1988-07-25 | 1990-06-05 | Electric Power Research Institute | Multimedia interface and method for computer system |
| US4954964A (en) * | 1988-11-09 | 1990-09-04 | Singh Gurvinder P | Apparatus and method for expert analysis of metal failure with automated visual aide |
| US5021976A (en) * | 1988-11-14 | 1991-06-04 | Microelectronics And Computer Technology Corporation | Method and system for generating dynamic, interactive visual representations of information structures within a computer |
| US5159685A (en) * | 1989-12-06 | 1992-10-27 | Racal Data Communications Inc. | Expert system for communications network |
| US5257185A (en) * | 1990-05-21 | 1993-10-26 | Ann W. Farley | Interactive, cross-referenced knowledge system |
| JPH06266727A (en) * | 1990-10-24 | 1994-09-22 | Osaka Gas Co Ltd | Method and equipment for displaying diagnosis |
| US5317732A (en) * | 1991-04-26 | 1994-05-31 | Commodore Electronics Limited | System for relocating a multimedia presentation on a different platform by extracting a resource map in order to remap and relocate resources |
| US5267865A (en) * | 1992-02-11 | 1993-12-07 | John R. Lee | Interactive computer aided natural learning method and apparatus |
| US5297150A (en) * | 1992-06-17 | 1994-03-22 | International Business Machines Corporation | Rule-based method for testing of programming segments |
-
1992
- 1992-09-28 US US07/953,034 patent/US5539869A/en not_active Expired - Fee Related
-
1993
- 1993-09-07 GB GB9318554A patent/GB2271005A/en not_active Withdrawn
- 1993-09-21 JP JP5235057A patent/JPH06214738A/en active Pending
- 1993-09-22 DE DE4332193A patent/DE4332193A1/en not_active Withdrawn
- 1993-09-23 CA CA002100510A patent/CA2100510A1/en not_active Abandoned
- 1993-09-24 KR KR1019930019664A patent/KR940007705A/en not_active Application Discontinuation
Also Published As
| Publication number | Publication date |
|---|---|
| DE4332193A1 (en) | 1994-03-31 |
| US5539869A (en) | 1996-07-23 |
| JPH06214738A (en) | 1994-08-05 |
| KR940007705A (en) | 1994-04-27 |
| GB9318554D0 (en) | 1993-10-20 |
| GB2271005A (en) | 1994-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5539869A (en) | Method and system for processing and presenting on-line, multimedia information in a tree structure | |
| CN1069424C (en) | System development support | |
| US6205575B1 (en) | Scenario presentation tool | |
| US5726688A (en) | Predictive, adaptive computer interface | |
| US5471560A (en) | Method of construction of hierarchically organized procedural node information structure including a method for extracting procedural knowledge from an expert, and procedural node information structure constructed thereby | |
| US5428729A (en) | System and method for computer aided software engineering | |
| CN105144672B (en) | Dynamic demonstration prototype and generation | |
| WO2006003485A2 (en) | Automatic guide for data analysis | |
| JPH0713754A (en) | Work process modeling method, work process model execution tracking method and computer system for work process molding program system | |
| CN1973256A (en) | Automatic text generation | |
| Clayton et al. | Interpretation objects for multi-disciplinary design | |
| EP1065603A2 (en) | Method and apparatus for generating visual assembly instructions from text manuals | |
| Bødker et al. | A conceptual toolbox for designing CSCW applications | |
| JPH06208465A (en) | Method and system for trouble diagnaosis | |
| JP4629183B2 (en) | Requirement specification description support apparatus and method, and recording medium | |
| JP2608085B2 (en) | Production planning simulation device | |
| JPH0997277A (en) | Design supporting device | |
| Kontogiannis | Applying information technology to the presentation of emergency operating procedures: implications for usability criteria | |
| Kovesdi et al. | Demonstration and Evaluation of an Advanced Integrated Operations Concept for Hybrid Control Rooms | |
| Greenough et al. | Development of a digital manual for a manufacturing system–a case study | |
| EP0738964B1 (en) | Method for animating composite behavior of scenarios | |
| JP3608418B2 (en) | Multimedia application development upstream process support method and apparatus, and storage medium storing multimedia application development upstream process support program | |
| GB2412832A (en) | Status and progress monitoring system for computer based interviews | |
| KR20020090554A (en) | Integrated construction process and cost management system and method, and media for storing program source thereof | |
| van der Veer et al. | Teaching a spreadsheet application—visual-spatial metaphors in relation to spatial ability, and the effect on mental models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| FZDE | Discontinued |