Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberUS20080184143 A1
Publication typeApplication
Application numberUS 11/957,076
Publication dateJul 31, 2008
Filing dateDec 14, 2007
Priority dateDec 14, 2006
Also published asUS8127238, US20080163084
Publication number11957076, 957076, US 2008/0184143 A1, US 2008/184143 A1, US 20080184143 A1, US 20080184143A1, US 2008184143 A1, US 2008184143A1, US-A1-20080184143, US-A1-2008184143, US2008/0184143A1, US2008/184143A1, US20080184143 A1, US20080184143A1, US2008184143 A1, US2008184143A1
InventorsHarry N. Gottlieb, Edward Lott, Evan Jacover, Gabriel Val, Steven Fram
Original AssigneeGottlieb Harry N, Edward Lott, Evan Jacover, Gabriel Val, Steven Fram
Export CitationBiBTeX, EndNote, RefMan
External Links: USPTO, USPTO Assignment, Espacenet
Methods for Identifying Actions in a Flowchart
US 20080184143 A1
Abstract
Tools, methods and systems for identifying coordinated actions in a flowchart representing a multimedia experience. These include displaying a graphical view of a flowchart representing a multimedia experience and defining, within the flowchart, a cell including a primary action configured to trigger playback of a primary media asset and secondary actions configured to trigger playback of secondary media assets, wherein each of the secondary actions has a timing coordinated with the primary action. The tools, methods and systems further include displaying a representation of the primary action, displaying representations of the secondary actions in chronological order according to their timing, and visually associating each of the representations of the secondary actions with the representation of the primary action.
Images(5)
Previous page
Next page
Claims(27)
1. A method for identifying coordinated actions, the method comprising:
displaying a graphical view of a flowchart representing a multimedia experience;
defining, within the flowchart, a cell including a primary action configured to trigger a playback of a primary media asset and secondary actions configured to trigger a playback of secondary media assets, wherein each of the secondary actions has a timing coordinated with the primary action;
displaying a representation of the primary action;
displaying representations of the secondary actions in chronological order according to their timing; and
visually associating each of the representations of the secondary actions with the representation of the primary action.
2. The method of claim 1 further comprising changing the timing of at least one of the secondary actions and updating the display of the representations of the secondary actions in chronological order according to their timing.
3. The method of claim 1, wherein at least one of the representations of the secondary actions includes a description of a secondary media asset triggered by a secondary action and a value for the timing of the secondary action.
4. The method of claim 1 further comprising selectively minimizing the display of the representations of the secondary actions.
5. The method of claim 1, wherein the representations of the secondary actions are visually linked with the representation of the primary action within a graphical representation of the cell.
6. The method of claim 5, wherein the representation of the primary action is displayed in a first display area of the graphical representation of the cell and the representations of the secondary actions are displayed in a second display area of the graphical representation of the cell.
7. The method of claim 6, wherein the representations of the secondary actions include descriptions of the secondary media assets triggered by the secondary action and values for the timing of the secondary actions.
8. The method of claim 5 further comprising editing the secondary actions directly in the graphical representation of the cell.
9. The method of claim 8, wherein editing the secondary actions directly in the graphical representation of the cell includes:
selecting a target secondary action in the graphical representation of the cell;
identifying the target secondary action by displaying a representation of the target secondary action differently from representations of other secondary actions; and
applying an input received in the cell to the target secondary action.
10. The method of claim 1, wherein the graphical view of the flowchart is displayed in a first display region, and further comprising displaying a textual view of the cell in a second display region.
11. The method of claim 1 further comprising:
selecting a first target secondary action;
displaying a first properties window, including a description of the secondary media asset triggered by the first target secondary action and a value for the timing of the first target secondary action;
selecting a second target secondary action;
displaying a second properties window, including a description of the secondary media asset triggered by the second target secondary action and a value for the timing of the second target secondary action;
applying an input received in the first properties windows to the first target secondary action; and
applying an input received in the second properties windows to the second target secondary action.
12. The method of claim 1 further comprising:
selecting a plurality of target secondary actions;
displaying descriptions of the secondary media assets triggered by the target secondary actions and values for the timing of the target secondary actions in a properties window; and
applying an input received in the properties window to at least one of the target secondary actions.
13. The method of claim 1, wherein at least one of the secondary actions is configured to trigger a presentation of text, and wherein displaying a representation of the at least one of the secondary actions includes displaying the text.
14. The method of claim 13 further comprising editing the text directly in the flowchart.
15. A method for identifying coordinated actions, the method comprising:
displaying a graphical view of a flowchart representing a multimedia experience, the flowchart including a cell representing a playback of coordinated media content;
defining, within the flowchart, a plurality of actions associated with the cell and configured to trigger the playback of coordinated media content, including a primary action and at least one secondary action having a timing coordinated with the primary action; and
differentiating within a graphical representation of the cell a display of the primary action and a display of the at least one secondary action.
16. The method of claim 15, wherein the at least one secondary action includes a plurality of secondary actions, and further comprising sorting the secondary actions in chronological order according to their timing.
17. The method of claim 15, wherein the display of the at least one secondary action includes displaying a description of media content associated with the at least one secondary action and a value for the timing of the at least one secondary action.
18. The method of claim 15, wherein differentiating within the graphical representation of the cell the display of the primary action and the display of the at least one secondary action includes displaying the primary action in a first display area of the cell and displaying the at least one secondary action in a second display area of the cell proximate the first display area.
19. The method of claim 18 further comprising selectively minimizing the display of the at least one secondary action.
20. The method of claim 15 further comprising editing the at least one secondary action directly in the cell.
21. The method of claim 15, wherein the least one secondary action is configured to trigger a presentation of text, and wherein the display of the at least one secondary action includes the text.
22. The method of claim 21 further comprising editing the text directly in the flowchart.
23. A method for identifying coordinated actions, the method comprising:
displaying a graphical view of a flowchart representing a multimedia experience;
defining, within the flowchart, a cell including a primary action configured to trigger a playback of a primary media asset and secondary actions configured to trigger a playback of secondary media assets,
associating each of the secondary actions with a timing coordinated with the primary action, wherein at least one secondary action is associated with a timing defined by a time relative to and prior to the end of the playback of the primary media asset triggered by the primary action;
displaying a representation of the primary action;
displaying representations of the secondary actions; and
visually associating each of the representations of the secondary actions with the representation of the primary action.
24. The method of claim 23, wherein at least one other secondary action is associated with a timing defined by a time relative to a start of the playback of the primary media asset.
25. The method of claim 23, wherein the representations of the secondary actions are displayed in chronological order according to their timing.
26. The method of claim 23 further comprising changing a timing of at least one of the secondary actions and updating the display of the representations of the secondary actions in chronological order according to their timing.
27. The method of claim 23, wherein a representation of a secondary action includes a description of the secondary media asset triggered by the secondary action and a value for the timing of the secondary action.
Description
RELATED APPLICATIONS

The present patent document claims the benefit of the filing date under 35 U.S.C. § 119(e) of Provisional U.S. Patent Application Ser. No. 60/875,179, filed Dec. 14, 2006, which is hereby incorporated by reference.

This patent is related to co-pending, Provisional U.S. Patent Application Ser. No. 60/875,071, titled, “System and Method for Controlling Actions Within a Programming Environment,” filed Dec. 14, 2006; and U.S. patent application Ser. No. 10/038,527, titled, “Method for Identifying Cells in a Path in a Flowchart and for Synchronizing Graphical and Textual Views of a Flowchart,” filed on Jan. 2, 2002. The entire contents of these related patent applications are incorporated herein by reference for all purposes.

BACKGROUND

A multimedia experience refers to the use of media in an interactive environment. The media generally include one or more types of information content, including for example, text, audio, graphics, animation, and video. During a multimedia experience, the media are presented to an end-user according to a logical sequence that can be affected by the end-user's actions.

A multimedia experience can be modeled as a flowchart that defines the logical sequence for playback of the multimedia experience. Such a flowchart generally consists of a sequence of linked cells that directly or indirectly reference media assets to be played by an application in a predefined order. The selection of which media referenced in the cells will be played can depend, in part, upon the choices made by end-users during each playback session of the multimedia experience. Flowcharts can be prepared by hand using pencil and paper or can be prepared electronically using a computer. Some software applications require a user to build a flowchart by drawing graphical shapes and then typing text into each graphical shape.

Often, during a playback session, the media referenced by several cells are presented to the end user in a coordinated manner, either simultaneously or in short succession of one another. In these situations, one cell is considered to represent playback of primary media off of which related cells representing secondary media are timed. In order to illustrate the dependencies between coordinated media, it is known to display the secondary cells on a side branch connecting these cells to the primary cell. The side branch may be visually distinct from other branches in the flowchart. This approach is presented in application Ser. No. 10/038,527 filed Jan. 2, 2002.

It is often desirable to determine the linear flow through a single path in the flowchart. However, if there are many branching structures and secondary cells in the flowchart, it can be difficult to identify the primary cells forming a single path through the flowchart. This can happen, for example, if a writer is creating a flowchart to structurally represent a multimedia experience since a reasonably sophisticated experience can generate a flowchart that is quite large and unwieldy, with hundreds or thousands of cells and complex branching between the cells. From the writer's standpoint, it is desirable to be able to visualize the linear flow through the flowchart as if it were a normal sequential storyline in the experience by viewing the graphical view of the flowchart.

There is a need, therefore, for a method that can be used to overcome the disadvantages discussed above.

BRIEF SUMMARY

The present invention provides tools, methods and systems for identifying coordinated actions in a flowchart representing a multimedia experience. The tools, methods and systems include displaying a graphical view of a flowchart representing a multimedia experience and defining, within the flowchart, a cell including a primary action configured to trigger playback of a primary media asset and secondary actions configured to trigger playback of secondary media assets, wherein each of the secondary actions has a timing coordinated with the primary action. The tools, methods and systems further include displaying a representation of the primary action, displaying representations of the secondary actions in chronological order according to their timing; and visually associating each of the representations of the secondary actions with the representation of the primary action.

Additional features and advantages of the present invention are described in, and will be apparent from, the following Detailed Description and the figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of an embodiment in which actions of a media cell are identified in the cell;

FIG. 2 is an illustration of an embodiment in which a media cell is selected to modify its actions;

FIG. 3 is an illustration of an embodiment in which an action of the media cell of FIG. 2 is edited in the cell.

FIG. 4 is an illustration of an embodiment in which the actions of the media cell of FIG. 2 are sorted in chronological order of their timing; and

FIG. 5 is an illustration of an embodiment in which actions of a media cell are displayed and edited in a separate display region.

DETAILED DESCRIPTION

The preferred embodiments described herein are preferably implemented using software and/or hardware components. For example, the preferred embodiments can be implemented with a software application (i.e., computer-readable program code) running on a processor of a general-purpose computer. Alternatively, some or all of the functionality of the application can be implemented with application-specific hardware components. For simplicity, the term “application” shall be used herein to refer generally to the entity (be it software and/or hardware) used to implement the preferred embodiments described below. The term “tool” shall be used interchangeably with the term “application.”

Turning now to the drawings, FIG. 1 is an illustration of a display output of an application of a preferred embodiment. Here, the application displays two display regions 100, 200. As used herein, the term “display region” refers to an area of display on one or more display devices (e.g., computer monitors). Each display region 100, 200 can be a separate window, or the display regions 100, 200 can be different areas in a single window. The first and second display regions 100, 200 can be fixed or movable and can be non-overlapping (as in FIG. 1) or can overlap each other. Additionally, the first and second display regions 100, 200 can be alternately shown (one then the other) in response to a command from the application and/or in response to a command from the user. The first display region 100 will sometimes be referred to herein as the “Structure Window,” and the second display region 200 will sometimes be referred to herein as the “Editor Pane.”

As shown in FIG. 1, the application displays a flowchart 150 representing a sample interactive multimedia experience in the first display region 100. As used herein, the term “flowchart” refers to any graphical representation of a branching structure. In this embodiment, the flowchart 150 consists of a sequence of linked cells 1-8. As described in greater detail further below, the cells represent media content and user interactivity and the links connecting the cells represent the flow of the multimedia experience through time. The first display region 100 is referred to as the “Structure Window” because it displays the graphical representation (i.e., the cells and the links connecting the cells) for the flowchart 150. In FIG. 1, the flowchart 150 is displayed in its entirety in the first display region 100. It should be noted that a flowchart is displayed in the first display region 100 even if only part of the flowchart is visible at one time in the first display region 100. For example, if the flowchart were larger than the first display region 100, only a portion of the flowchart would be visible in the first display region 100. In this situation, a scroll bar and/or other navigation devices can be provided in the first display region 100 to allow a user to select which part of the flowchart is visible.

The flowchart 150 in FIG. 1 comprises a plurality of cells 1-8. In this embodiment, the cells 1-8 are illustrated as graphical symbols, some of which are rectangular-shaped and others of which are diamond-shaped. It will be understood that the graphical symbols described herein are intended as illustrative rather than limiting and that, according to the present invention, other types of displays can be used to illustrate the cells in a flowchart. The rectangular-shaped cells are referred to herein as “simple cells,” and the diamond-shaped cells are referred to herein as “conditional cells.” A simple cell is a cell that contains a single branching link to a single cell. For example, in the flowchart 150 shown in FIG. 1, simple cell 4 branches to simple cell 5 and simple cell 5 branches to conditional cell 6. Unlike a simple cell, a conditional cell contains multiple branching links to multiple cells. The branching links are associated with conditions necessary for that branching link to be followed. For example, in the flowchart 150 shown in FIG. 1, conditional cell 6 branches to simple cells 7 and 8. The branch to simple cell 7 is followed if the condition “ready to order” is true, while the branch to simple cell 8 is followed if the condition is false.

It should be noted that a cell can contain two or more individual cells (simple cells or conditional cells). Such a cell is referred to herein as a “group cell” and can be used to simplify the display of a flowchart by grouping multiple cells together in a single cell. In addition to simple and conditional cells, a group cell can contain other group cells (i.e., groups cells can be nested) and other types of cells, such as “go to” cells, “alias” cells, and other types of cells described in the next paragraph. “Go to” cells can be used instead of a line to show the flow from that cell to another cell. In this way, “go to” cells can be used to keep a flowchart clean and readable. “Alias” cells can be used to mimic the function/operation of another cell.

Different cells can have different timing. For example, some cells can be played after a previous cell initiates its function, while other cells can be played after a previous cell finishes its function. Other timing mechanisms for playing cells can be used, for example, a timing mechanism based on a common clock.

The cells in a flowchart can contain any suitable content. For example, as in a traditional flowchart drawn on paper, a cell can contain text. A cell can also provide selection choices to a user and evaluate which choice was selected. For example, when conditional cell 6 in FIG. 1 is played, the user is prompted to input “yes” or “no” using a user interface device, such as a keyboard, mouse, microphone, remote control, or any other type of device. Conditional cell 6 also determines whether the user input is “yes” or “no.” If the user input is “yes,” the branch leading to cell 7 is followed, and the application runs the script “Great. Let's proceed.”

According to the present embodiment, a cell can also contain actions such as instructions or references to trigger playback of media, gather user input, generate visual or audio information, send/receive information or media to a database, process data, perform a calculation, or perform other functions, such as describing how and when media should be played. Examples of media assets include, but are not limited to, digital audio/video objects representing still art, animation, graphics, on-screen text, music, sound effects, or voiceover dialogue. It will be understood that any type of media playback from within a cell includes an action for executing the media playback associated with that cell.

The term “media cell” shall be used herein to refer generally to a cell that can reference at least one media asset or represent a reference to the at least one media asset to be played by the application. Thus, conceptually, a media cell acts as a container for actions, or references to actions, that trigger media playback. A media cell can contain multiple actions for executing media playback that are implemented by the application (or by another application) in a coordinated manner when the cell is played. In particular, each media cell includes at least one primary action and possibly one or more secondary action whose timing is coordinated with the primary action. The primary action preferably triggers playback of media that conveys a comprehensive message at a particular moment of the experience, such as audio or video of an actor speaking. A secondary action preferably triggers playback media that illustrates, reinforces and/or supports the comprehensive message being delivered through the primary action, such as text displaying all or a subset of the script performed by the actor and/or graphics that help illustrate the message of the actor's script. Preferably, the timing of a secondary action of a media cell is based on a time X before or after the beginning or end of the primary action of that media cell. Different actions in a cell can trigger playback of media that execute simultaneously with each other. It will be understood that the timing of a secondary action can be coordinated with the timing of a primary action using other timing mechanisms. For example, the primary and secondary actions could be timed off of a common clock, the secondary action could be timed off of another secondary action that is timed off of the primary action, etc. Thus, each media cell contains one or more actions whose timing is coordinated.

In this way, the application distinguishes between media cells generally representing application flow and the coordinated actions performed by a cell when it is played. In particular, the media cell is a compound cell where several discrete and timed events can occur when flow reaches the cell during playback. Thus, media cells may function as placeholders within the overall progression of the application at runtime. Coordinated actions, such as those corresponding to the presentation or modification of media content during runtime flow, can be modeled internally in a media cell as separate objects themselves.

When the flowchart 150 is played during a playback session, the cells 1-8 in the flowchart 150 of FIG. 1 provide an interactive multimedia experience for a user. It will be understood that playing the flowchart means either that the application itself is executing the flowchart or that another application is executing the flowchart. For example, the flowchart application could export its logic and data, including its media assets, to a runtime engine that plays back the multimedia experience using the exported logic and data. The interactive multimedia experience consists of spoken or verbal scripts accompanied by on-screen text and graphics, and proceeds according to the logical sequence defined by flowchart 150 that can be affected by the end-user's actions. The cells 1-8 represent distinct moments within the overall progression of the multimedia experience and are arranged in the flowchart 150 in the order in which they are traversed in operation. When playback reaches a particular cell, the application and/or runtime engine will generate a signal to invoke the cell's primary action, for example, for executing playback of a media asset referenced by the cell. The application and/or runtime engine will also generate signals to invoke any of the cell's secondary actions based on timing coordinated with the timing of the primary action.

In particular, when the flowchart is played from its beginning, the user first hears an audio file voicing the greeting “top of the morning” triggered by the primary action 20 in the media cell 2. The user then hears a sound effects triggered by secondary actions 22 and 24 in the media cell 2 timed off of the primary action. First, a “leaves falling” sound effect corresponding to the secondary action 22 begins to play 1.50 seconds before the end of playback of the phrase “top of the morning.” Then, a “birds chirping” sound effect the secondary action 24 begins to play 0.50 seconds before the end of playback of the phrase “top of the morning.” Thus, the “birds chirping” sound effect would be presented during the experience 1.00 second after the “leaves falling sound effect has started. Next, an audio file voicing the phrase “walking down the main path” is played at cell 3. The user then hears the greeting “Welcome” at cell 4, followed by the question “Are you ready to order?” at cell 5. Depending on the user's response, an audio file voicing the phrase “Great. Let's proceed.” is played at cell 7 or an audio file voicing the phrase “No problem. Take your time.” is played at cell 8.

The interactive conversation interface can be used to communicate ideas and information in an audio and/or visual environment, such as interactive computer games, commercials, guided tours, auctions, stories, and news, health, or financial services. The interface can be implemented with wired or wireless equipment that includes both audio and video output devices (such as a home computer or television) as well as with equipment that includes only an audio output device (such as a conventional telephone) or only a video output device. It is important to note that these preferred embodiments can be used in flowchart applications that are unrelated to an interactive multimedia experience. Accordingly, a specific type of cell should not be read into the claims unless explicitly recited therein.

In accordance with one embodiment of the present invention, the secondary actions 22 and 24 for the media cell 2 are identified by being displayed within the media cell 2 in a separate area, for example a column, next to the primary action 20, as shown in FIG. 1. Preferably, the secondary actions 22 and 24 are organized or stacked within the media cell 2 according to their chronological order of execution. Therefore, the media for the secondary actions 22 and 24 within the media cell 2 would begin execution in the same order in which the secondary actions 22 and 24 are displayed. The media cell also includes a minimize and restore button 26 that is clickable to selectively hide the display of the secondary actions 22 and 24, for example, by alternatively minimizing and restoring the display of the column having the secondary actions 22 and 24 therein.

The timing values for the secondary actions 22 and 24 are stored in the secondary actions and are displayed directly beneath the descriptions of the secondary actions in the media cell 2. In this example, the timing values for the secondary actions 22 and 24 of the media cell 2 are based on a time before the end of the primary action 20. Thus, the timing value for the secondary action 22 is E−1.50 seconds, which indicates that the media associated with this secondary action (“leaves falling”) would begin playing 1.50 before the end of playback of the primary action 20 (“top of the morning”). Likewise, the timing value of E−0.50 seconds for the secondary action 24 means that the media associated with this secondary action (“birds chirping”) would begin to play 0.50 seconds before the end of playback of the primary action 20 (“top of the morning”) or 1.00 seconds after the secondary action 22 begins. In addition, if the timing of a secondary action is based on a time X after the end of a primary action, the timing value can be represented, for example, as E+X; if the timing of the secondary action is based on a time X before the start of the primary action, the timing value can be represented, for example, as S−X; and if the timing of a secondary action is based on a time X after the start of a primary action, the timing value can be represented, for example, as S+X.

The display of related secondary actions 22 and 24 within the media cell 2 in this way correctly reflects their child-parent relationship to the primary action 20 and their membership in the same group, represented by the media cell 2. Also, the display of related secondary actions 22 and 24 within the media cell 2 is consistent with the flowchart metaphor used by the application. In particular, from a writer's standpoint, it may be desirable to be able to visualize the linear flow through the flowchart as if it were a normal sequential storyline in the experience. However, a flowchart may contain many different branching structures. A single path represents application flow along one of many possible paths in the flowchart during an actual runtime session. When a secondary action in media cell is selected, for example secondary action 22 in the media cell 2 of FIG. 1, the secondary action is highlighted within the media cell, while the single path continues beyond the media cell along the path that contains it. This is consistent with the notion that playback in the flowchart should proceed sequentially from one cell to the next.

It should be understood that the application may implement different methods to determine a single path. For example, the selection of a single cell can be used to isolate the cells above and below the selected cell to form a single path based on the history of the cell and the history of the cells above and below it in succession. Alternatively, a path can be determined based on the most-frequently selected path containing the selected cell or the application can choose the path that most recently contained the selected cell. Other methods can be used, for example, the application can randomly determine a path comprising the selected cell or can semi-randomly determine a path comprising the selected cell, such as when part of the path is selected based upon some form of logic (e.g., most frequently selected links three cells above and below the selected cell), but the rest of the path is selected at random (e.g., all other links are selected randomly). The application may also implement different techniques to identify the cells in a single path. For example, cells along a single path may be identified by displaying the lines linking the cells differently (e.g., in a different color, shading, or thickness) than the lines linking cells that are not along the path. Other techniques can be used to identify cells along a path, for example, the borders of the cells along a path and their branches may be displayed with thicker lines in the first display region 100.

Additionally, the content of the cells along a single path is displayed in the second display region 200 (the Editor Pane) to allow a user to read through the content of those cells in isolation from the cells in the other paths. As shown in FIG. 1, the second display region 200 contains a textual view 250 of the cells 1-7 in the order in which the cells appear in the flowchart 150. The textual view generally shows a “description” of the primary action of a cell (e.g., the text that is contained in the primary action, the line of dialogue that will be voiced by the primary action, the description of the animation or of the SFX that will be played by the primary action, etc.). The textual view also may include “descriptions” of the secondary actions of the cell. The content displayed in the textual view 250 can also contain other cell “properties,” such as the font of displayed text, the name of the actor who will provide the media content, the cell's timing relative to other cells, the volume at which audio content of the cell should be played, the frames per second of animation, and other information. It should be noted that the “textual” view can also contain graphics. For example, the text of the cell can also be supported with a graphics button or thumbnail image next to the text that indicates the type of media in the cell (e.g., a button with a little horn indicating a SFX) and is clickable to open a properties window of the cell, to playback the media, or to perform other functions.

In the present embodiment, editing of the secondary actions 22 and 24 can be performed in-place within each media cell. To edit a secondary action, a user individually selects a cell in the flowchart. In the flowchart 150 displayed in FIG. 2, a user uses a pointing device (such as a mouse or trackball) to move a pointer 30 over the media cell 2, and then selects that cell by pressing a selector switch (such as the mouse button). Alternatively, the user can select the media cell 2 using any other type of user interface device. For example, if the cells are numbered, the user can select the media cell 2 by typing in the cell number using a keyboard or by speaking the number of the cell into a microphone. Additionally, instead of the user selecting the media cell 2, the application can automatically select the media cell 2 (e.g., based on the output of some operation being run by the application or based on the output sent to the application by another application). A user can then select one of the secondary actions 20 and 24 shown in the media cell 2 using the pointing device or another user interface device as described above in order to edit their content and/or properties, including script description, timing properties, and visual display properties. For example, as shown in FIG. 3, a user selects the secondary action 22 and changes its content from “leaves falling” to “rain drops keep falling” in-place within the media cell 2. Then, as shown in FIG. 4, the user changes the timing value for this secondary action from E−1.50 seconds to E−0.20 seconds.

Alternatively, when the user selects a secondary action, a third display region can be provided for editing its content and/or properties. For example, FIG. 5 illustrates a media cell 9, including a primary action 90 and secondary actions 92 to 96, and a display region 300 or “properties window” that displays and allows a user to edit the content and/or properties of the actions 90 to 96. In the example, the display region 300 displays different properties 94 a and 94 b of the secondary action 94 selected by the user. The display region 300 receives user input, and the application can apply the input received to the secondary action 94 in the media cell 9.

Referring again to FIGS. 3 and 4, the application automatically rearranges secondary actions within a media cell to reflect their chronological order of execution. As shown in FIG. 4, when the timing of the secondary action 22 (“leaves falling”) is changed from E−1.50 seconds to E−0.20 seconds, the order of execution of the secondary action 22 and 24 changes because the secondary action 22 would now be executed 0.30 seconds after the secondary action 24 (“birds chirping”) begins. As a result, the application automatically shifts the secondary action 22 to fall below the secondary action 24 (“birds chirping”). Therefore, if the timing of a secondary action is updated, so is its order in the list of secondary actions. This is consistent with the notion that playback in the flowchart should proceed sequentially from one action to the next.

While according to the present embodiment the primary action 20 and the secondary actions 22 and 24 are displayed in separate areas within the media cell 2, it should be understood that the application may implement other methods to identify and/or to differentiate the secondary actions of a media cell from the primary action of the cell. For example, the secondary actions 22 and 24 can be displayed in the first display region 100 adjacent or proximate the media cell 2 according to their chronological order of execution. The positioning of the secondary actions 22 and 24 relative to the media cell 2 would identify their child-parent relationship with the primary action 20 of the media cell 2. Visual cues can also be provided, for example, links connecting each the secondary actions 22 and 24 to the primary action 20, so as to visually reinforce the child-parent relationship between the secondary action 22 and 24 and the primary action 20 and their membership in the same group. Alternatively, the textual view 250 in the second display region 200 can show both a description of the primary action 20 and the descriptions of the secondary action 22 and 24 in the order of their execution. For example, the textual view 250 can display the primary action 20 in a first column and the secondary action 22 and 24 next to the primary action in a second column. In another embodiment, the secondary actions 22 and 24 of the media cell 2 can be displayed in a fourth display region separate from the first and second display regions 100 and 200, the fourth display region being associated and/or visually linked with the media cell 2 to reinforce the child-parent relationship between the secondary action 22 and 24 and the primary action 20 and their membership in the same group.

It will be understood that the application can be equipped with various functionality to allow it to facilitate the construction of the media assets scripted by the writer and for providing the programming necessary to fully render the interactive multimedia experience on a given platform. As noted above, cells can contain actions to be implemented, including instructions to play a media asset such as an audio file or a video file. When a writer is scripting the content of the interactive multimedia experience, those media assets may not yet exist. The application can sort the various pieces of desired media based on the talent that is necessary to create the media or on other criteria for sorting. For example, the actions in a media cell can be divided into music tasks, animation tasks, art tasks, programming tasks, SFX tasks, writing tasks, video tasks, and performance tasks. In this way, artists used to create the media can be assigned a list of tasks that need to be performed. When each of the media assets is created by the artists and inserted into designated “slots” in a database, the application can assemble the completed media for playback. The slots in the database can be created before or after the media assets are created. Because an interactive multimedia experience can have thousands of assets and a non-linear structure, it is preferred that the application maintain a database to track the media assets.

It is intended that the foregoing detailed description be regarded as illustrative rather than limiting, and that it be understood that it is the following claims, including all equivalents, that are intended to define the spirit and scope of this invention.

Referenced by
Citing PatentFiling datePublication dateApplicantTitle
US7930329 *Feb 27, 2007Apr 19, 2011Samsung Electronics Co., Ltd.System, method and medium browsing media content using meta data
US8127238Dec 14, 2007Feb 28, 2012The Jellyvision Lab, Inc.System and method for controlling actions within a programming environment
US8276058Feb 8, 2008Sep 25, 2012The Jellyvision Lab, Inc.Method of automatically populating and generating flowerchart cells
US8464169Oct 19, 2007Jun 11, 2013The Jellyvision Lab, Inc.Methods for identifying cells in a path in a flowchart and for synchronizing graphical and textual views of a flowchart
US8521709Oct 26, 2007Aug 27, 2013The Jellyvision Lab, Inc.Methods for preloading media assets
US8970496 *Apr 15, 2008Mar 3, 2015Razer (Asia-Pacific) Pte. Ltd.Ergonomic slider-based selector
US20090172559 *Dec 28, 2007Jul 2, 2009Microsoft CorporationCreating and editing dynamic graphics via a web interface
US20100281359 *Apr 19, 2010Nov 4, 2010International Business Machines CorporationMethod, apparatus and system for processing graphic objects
Classifications
U.S. Classification715/764
International ClassificationG06F3/048
Cooperative ClassificationY10S715/967, G11B27/034
European ClassificationG11B27/034
Legal Events
DateCodeEventDescription
Oct 24, 2008ASAssignment
Owner name: THE JELLYVISION LAB, INC., ILLINOIS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOTTLIEB, HARRY N.;LOTT, EDWARD;JACOVER, EVAN;AND OTHERS;REEL/FRAME:021734/0658
Effective date: 20080324