US 7031907 B1
A voice recognition grammar (“VRG”) tool for facilitating the creation of speech grammars for speech recognition applications is described. The VRG tool 114 uses syntax diagrams to enable a developer to visually build vendor-independent speech recognition grammars. The grammars are built from visual components called blocks, which represent the elements of a grammar. Blocks are connected and assigned values to define a grammar. A grammar is built with the VRG tool within a “Construction Area” by selecting blocks from a “Tool Kit.” Every phrase must start with a start block and be completed with an end block, with the various grammar blocks being connected between them. Once the syntax for the grammar is constructed, using the aforementioned blocks labeled and defined appropriately, a grammar specific to the selected vendor may be generated and is then ready for use by a voice recognition application.
1. A method for generating a speech recognition grammar for a voice recognition system, comprising:
(a) providing a visual area on which to graphically construct a syntax diagram,
(b) placing a block in this area, wherein the first block represents an element of grammar of the syntax diagram,
(c) defining the element of grammar,
(d) placing another block in the area,
(e) visually connecting the blocks to define a grammatical relationship between the connected blocks,
(f) repeating steps (b) through (e) until the syntax diagram is complete, and
(g) recursively analyzing the grammatical relationships and the elements of grammar of the syntax diagram to generate the speech recognition grammar in a compatible format for use with the speech recognition system.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. The method of
10. A speech recognition grammar (“SRG”) tool for use with a speech recognition system, the SRG tool comprising:
means for providing a visual area on which to graphically construct a syntax diagram,
means for a plurality of blocks in the area, wherein some of the plurality of blocks represent an element of grammar of the syntax diagram,
means for defining the element of grammar,
means for placing another block in the area,
means for visually connecting the blocks to define a grammatical relationship between the connected blocks, and
means for recursively analyzing the grammatical relationships and the elements of grammar of the syntax diagram to generate the speech recognition grammar in a compatible format for use with the speech recognition system.
11. The SRG tool of
12. The SRG tool of
13. The SRG tool of
14. The SRG tool of
15. The SRG tool of
16. The SRG tool of
17. A computer program product for constructing a syntax diagram representing a speech recognition grammar for a speech recognition system, the computer program product comprising:
a computer program processable by a computer system, the computer program comprising a plurality of instructions, including:
instructions for enabling a user to select a block comprising a graphical display element to represent an element of the grammar;
instructions for enabling a user to connect the selected block to at least one other block;
instructions for enabling a user to define what is represented by the selected block; and
instructions for recursively analyzing grammatical relationships and the elements of grammar to generate the speech recognition grammar in a compatible format for use with the speech recognition system.
18. The computer program product of
The invention relates generally to speech recognition tools and, more particularly, to a speech recognition grammar (“SRG”) tool for facilitating the creation of grammars for speech recognition tools.
Speech recognition is a process of recognizing one or more words or phrases spoken by a user. A SRG provides a recognizer with information about the expected input. By knowing the structure of the expected input, a recognizer can narrow its choices and thus allow a wider range of voice quality and intonation to be correctly identified. The recognition is based on what the user says, as well as a vocabulary and, perhaps, a grammar. A grammar is a set of rules that limit the recognition tasks to a specific set of words and/or phrases. In an effort to improve recognition accuracy, a speech recognition tool, or “recognizer,” is typically advised as to which words and/or phrases (or word sequences) are valid in the application. Grammar rules vary between various recognizers; therefore, to develop a grammar for a particular recognizer, a developer is required to be familiar with the particular rules and syntax applied by the vendor of the particular recognizer. Accordingly, if a developer develops grammars for multiple recognizers, he or she must be familiar with multiple sets of rules and syntaxes.
Therefore, what is needed is a system and method for simplifying the development of grammars for multiple recognizers available from different vendors.
In one embodiment, the invention is a speech recognition grammar (“SRG”) tool for facilitating the creation of speech grammars for speech recognition applications. In particular, the SRG tool uses syntax diagrams to enable a developer to build vendor-independent speech recognition grammars. The grammars are built visually in construction areas from graphical components called blocks, which represent the elements of a grammar. Blocks are connected and assigned values to define a grammar. The various blocks implemented by the SRG tool include “word blocks,” which represent the words or letters that are acceptable as input at a specific location in the grammar; “phrase blocks,” which represent a new construction area where a subgrammar can be defined for a particular phrase; “external blocks,” which identify grammars defined outside of the SRG tool and include, for example, drop-in grammars, subgrammars, and runtime variables; “reference blocks,” which point to an existing phrase block to enable a phrase to be defined once and used multiple times; and “connector blocks,” the properties of which may contain important path information specific to the current vendor.
A grammar is built with the SRG tool within a “Construction Area” by selecting blocks from a “Tool Kit.” Every phrase must start with a start block and be completed with an end block, with the various grammar blocks being connected between them. Once the diagram for the grammar is constructed, using the aforementioned blocks labeled and defined appropriately, a vendor grammar may be generated and is then ready for use by a speech recognition system.
A technical advantage achieved with the invention is that it can be used to construct grammars in a vendor-independent manner for a number of different speech recognition systems.
In the following description, well-known elements are presented without detailed description in order not to obscure the present invention in unnecessary detail. For the most part, details unnecessary to obtain a complete understanding of the present invention have been omitted inasmuch as such details are within the skills of persons of ordinary skill in the relevant art.
The SRG tool 114 does not include grammar compilers, which are vendor specific. Table I below lists examples of vendors/products that may be supported by the SRG tool 114:
It will be recognized that the vendors listed in Table I are representative only and that use of the SRG tool 114 is not limited to those vendors listed in Table I, but may be used to support any vendor grammar format.
As will be explained in greater detail below, the SRG tool 114 provides a visual area or “construction” area on a display of computer system 100 on which to construct a syntax diagram.
Turning now to
After the first block has been defined and placed in the construction area, in step 126 another block may be placed in the construction area. In step 128, the appropriate element of grammar may also be defined for this block. In step 130, the blocks may be visually connected with a graphic element, such as a line. As will be explained in greater detail below, the line represents a relationship between the blocks. The ordered combination of blocks and lines creates a syntax diagram. The syntax diagram is, therefore, a visual representation of the elements of grammar and the grammatical relationships between the appropriate elements.
Turning back to
Once constructed, in step 132, the user may select a specific vendor. In step 134, the syntax diagram may then be converted into the specific grammar format for the chosen vendor. The SRG tool 114 converts the syntax diagram into a multitude of vendor specific formats.
The SRG tool 114 converts the syntax diagram into the appropriate grammar by recursively analyzing the grammatical elements and their “connections” or relationships to each other (which are visually represented as lines and connector elements). This recursive analysis allows the syntax diagram to be decompose into sequences, alternatives, and cycles. For instance, a block which has a single exiting connection is a participant in a sequence. A sequence is graphically illustrated in
Alternatives, sequences, and cycles have specific format representations in every current speech recognition vendor grammar. Thus, after a syntax diagram has been decomposed into these generic parts, the parts can in turn be used to generate a specifically formatted text-based grammar which has been formatted to meet the user-selected vendor's format requirements.
In order to describe present invention, an exemplary embodiment will be discussed below in reference to
Referring now to
Referring again to
Referring now to
Selection of the Global Properties menu pick may result in the display of a tabbed Global Properties dialog box 500, as shown in
Referring now to
Referring now to
Selection of the Phrase List menu pick enables the user to generate a phrase list. A phrase list can be generated any time after the grammar file has been saved. An exemplary phrase list 600 is illustrated in
Referring now to
Connector blocks can be used to contain “path” information that is useful to the recognizer to interpret what was input. Were this path information not available, the recognizer would have to parse the input. For example, the following four phrases all have the same meaning:
Clearly, it would be difficult for a recognizer to parse all of the above entries (and these are only a small sampling of the number of alternatives for expressing the same command) and determine therefrom the intent of the user. Using connector blocks, path information can be inserted into the input to enable the recognizer to determine what was spoken. Taking the same phases, but assuming the grammar contains path information (denoted by brackets), the phrases read as follows:
Now, along with the spoken input, the recognizer will receive the path information and in all of the above cases, the following path information will be returned:
An informative connector block (one which is yellow) may provide information regarding the progress of the recognition, as describe above. Recognition can be considered to follow one of the paths in the syntax diagram from the start block to the end block in a construction area. When an informative connector is in the path, it provides information to the recognizer that defines the path taken. For example, if the word “five” is in a word block and the word block is followed by an informative connector, the information defined in the connector block might append the digit “5” to the output generated by the recognizer, thus identifying the path taken, and by implication, the phrase recognized.
Blocks can be added to the Construction Area 200 in one of a number of methods. For instance, using the Blocks pull-down menu on the Construction Area, the user can select the desired block. This will place the block in the upper left corner of the Construction Area. From here, it can be dragged and dropped to the desired location. In another method, the desired block can be clicked on in the Tool Kit 202. Again, this will place the block in the upper left corner of the Construction Area, from where it can be dragged and dropped to the desired location. In yet another example, the desired block can be dragged and dropped from the Tool Kit 202 to the Construction Area 200, allowing the user immediately to place the block in the desired location.
In the illustrative embodiment, for sentences to be completed, blocks are be connected. This can be accomplished in one of a variety of methods. For instance, after a block is placed in the Construction Area 200, another block can be placed right after it. To connect these blocks, the first block is highlighted by clicking on it once and then the user right-clicks on the block to which it is to be connected. Next, the user selects Connect from the drop down menu. Blocks that are already connected can be disconnected in the same manner by selecting Disconnect from the drop down menu. In another method, after a block is placed in the Construction Area 200, another block can be placed after it. To connect these blocks, the first block is highlighted by clicking on it once. Using the middle mouse button, the user clicks on the block to which it is to be connected, which will automatically connect the two blocks. Blocks that are already connected can be disconnected in the same manner. In yet another method, after a block is placed in the Construction Area 200, the user highlights it by clicking on it once. Another block is added in any of the three manners described above. Using the pull-down menu or clicking on the corresponding button on the Tool Kit 202 will place the new block immediately after the existing block. Dragging and dropping a block will enable the user to place the block in any desired location. To disconnect the added block, the original block is highlighted by clicking on it once, at which point right-clicking on the connected block and selecting Disconnect from the menu results in the blocks being disconnected.
Blocks can be connected form specific sides, if the user so chooses. This can be useful when designing loops or alternatives to keep the Construction Area 200 orderly. The SRG tool 114 builds phrases from left to right within the Construction Area 200. When a loop is introduced, the flow is typically run from right to left before continuing. Constraining “from” and “to” specific points of the blocks can accomplish this. To constrain from or to a specific side of a block (i.e., top, bottom, left, or right), the user right-clicks on that edge of the block. From the menu, the user then selects “Constrain To” or “Constrain From”.
As previously indicated, a word block is a block that represents a single word within a phrase. Once placed, the block can be labeled and defined. Alternative pronunciations of a word can be defined as follows. Using the default dictionary, words will be defined by the vendor. Non-vendor words/terms can be defined phonetically for speech recognition purposes as follows. Using either the pull-down menu or the Tool Kit 202, a word block is added to the Construction Area 200. The user then double-clicks on the word block to open it. This results in the display of a word dialog box 900 (
It should be noted that most words are already defined in the SRG tool's or the vendor's dictionary. Words that need to be defined typically include, for example, proper names, company names, products, and the like. Such words may be phonetically spelled as the user would speak them. Several phonetic spellings can be made for a single word to account for differing accents or intonations of callers.
Properties can be assigned to a phrase block in a manner similar to that of assigning properties to a word block, for example, by choosing the Properties menu pick from the Edit pull-down menu or by clicking the phrase block (to highlight it) and then right-clicking and choosing properties. Choosing Properties after right-clicking on the phrase block results in the display of a dialog box 1000 (
An external block may allow grammars defined outside of the SRG tool 114 to be included in the current grammar. External blocks include Drop-In Grammars, Subgrammars, and Runtime Variables, each of which will be described in greater detail below. Opening an external block results in the display of a dialog box 1100, as illustrated in
A Drop-In Grammar may be predefined by the vendor; therefore, the pull-down list displayed will be specific to the current vendor. Only certain vendors have Drop-In Grammar capabilities. An exemplary pull-down list is illustrated in
A Subgrammar may be defined externally, typically by the vendor, to be used inside the current grammar. To use a Subgrammar, from the dialog box 1100, the user may select a Subgrammar tab 1112, as illustrated in
A Runtime Variable is a placeholder that dynamically changes the grammar. At runtime, acceptable responses may be inserted into the grammar and are then used in place of the Runtime Variable. These responses may be inserted and deleted based on the user's requirements. For example, at the beginning of a phone call (or upon acquiring the recognition resource), the responses are stored in a placeholder. At the end of the phone call (or freeing of the recognition resource), the responses are removed. The user inserts a Runtime Variable into the application (e.g., “dial Mom”). Speech recognition is accomplished using a vocabulary application. For example, if there are two phone numbers for “Mom” in the application for two different users, the application will, by voice intonation, be able to determine which “Mom” to insert into the grammar. To use a Runtime Variable, the user selects a Runtime Variable tab 1120 from the dialog box 1100, as illustrated in
A reference block executes an existing phrase block and may enable a phrase block to be used multiple times, having been defined only once. To use a reference block, after the phrase block to be referenced has been added to the grammar, the user right-clicks on the phrase block to display a menu 1200 (
A connector block typically connects two other blocks. Double-clicking on a connector block in the grammar enables a user to add path information. After path information has been added, the connector block may turn a different color, such as yellow. The type of path information that can be added is vendor-dependent. For example, using Nuance, a path can contain a return value or a parameter. When returned, the value may then be inserted into a variable defined for the current phrase. A parameter is a defined slot (“NL Slot”/“NL Return”) value that is returned to the grammar with the desired value. NL Slots and NL Returns are created in the grammar, which in turn can be included in a folder and sent to another application. NL Slots and NL Returns allow the grammar and the application to communicate.
For example, assume a caller is presented with two options from which to choose, such as “Checking Account Balance” and “Savings Account Balance.” Assume further that the caller selects Savings Account Balance and the value is stored as a variable in a slot. The recognizer returns this value to any application that uses the grammar. The SRG tool 114 ensures only valid syntax for Slot and Return values by enabling/disabling fields based on the current value.
To add path information for an NL Slot, the user may want to first ensure that NUANCE is selected in the Grammar tab 502 of the Global Properties dialog box 500 (
For some vendor's systems, such as BBN, a Label may be defined for a path. A Label may allow the user selectively to enable/disable parts of the grammar in the application. For example, there is a list from which to choose that includes “add a course,” “drop a course,” and “list courses.” For a new student, it would be desirable to disable the “drop” and “list” options and only allow the student to “add”. To add path information using a Label, the user first ensures that BBN is selected using the Grammar tab 502 of the Global Properties dialog box 500 (
When using some vendor's speech recognition systems, such as BBN, JSGF, and Philips, a Tag may be defined for a path. A Tag is the value returned, with the recognized grammar, as a valid response. A Tag is similar to a Slot, in that it acts as a placeholder. To add path information using a Tag, the user first ensures that the appropriate vendor is selected in the Grammars tab 502 of the Global Properties dialog box 500 (
The following section will illustrate the construction of a simple grammar using the SRG tool 114; specifically, the sentence “Today is a wonderful day,” with the words “beautiful” and “sunny” being used as alternative for “wonderful,” will be described. Referring to
At this point, a phrase block 1428 is added to the Construction Area 1400, connected to the “a” block 1426, and labeled “wonderful.” To add alternative acceptable words for “wonderful,” the user double-clicks on the “wonderful” block 1428 to open a new Construction Area 1430 entitled “wonderful,” as illustrated in
Opening the “Balance” phrase block 1516 results in the display of a Balance phrase 1530, as illustrated in
Referring again to
Referring again to
Similarly, opening the Day phrase block 1548 results in the display of a Day phrase 1552, as illustrated in
As illustrated herein, the SRG tool 114 enables a user to construct a grammar, such as those illustrated herein in
The above disclosure provides many different embodiments, or examples, for implementing different features of the invention. Also, specific examples of components and processes are described to help clarify the invention. These are, of course, merely examples and are not intended to limit the invention from that described in the claims.
While the invention has been particularly shown and described with reference to the preferred embodiment thereof, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention, as set forth in the following claims.