|Publication number||US20100097315 A1|
|Application number||US 12/505,192|
|Publication date||Apr 22, 2010|
|Filing date||Jul 17, 2009|
|Priority date||Sep 6, 2006|
|Also published as||US7567233, US20080055239|
|Publication number||12505192, 505192, US 2010/0097315 A1, US 2010/097315 A1, US 20100097315 A1, US 20100097315A1, US 2010097315 A1, US 2010097315A1, US-A1-20100097315, US-A1-2010097315, US2010/0097315A1, US2010/097315A1, US20100097315 A1, US20100097315A1, US2010097315 A1, US2010097315A1|
|Inventors||Jeffrey M. Garibaldi, Walter M. Blume, Nathan Kastelein, Raju R. Viswanathan, Gery Tomassoni, Guy Besson|
|Original Assignee||Garibaldi Jeffrey M, Blume Walter M, Nathan Kastelein, Viswanathan Raju R, Gery Tomassoni, Guy Besson|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (3), Referenced by (12), Classifications (13), Legal Events (3)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is a continuation of U.S. patent application Ser. No. 11/670,930, filed Feb. 2, 2007, which is now U.S. Pat. No. 7,567,233, and issues Jul. 28, 2009, and claims priority to U.S. Provisional Patent Applications 60/871,774, filed Dec. 22, 2006, and 60/842,633, filed Sep. 6, 2006. The disclosures of the above-referenced applications are incorporated herein by reference.
This invention relates to a consolidated user interface systems and methods and in particular, to an interface for operating multiple medical systems.
Many of today's most advanced and effective diagnostic and therapeutic procedures involve the coordinated use of multiple separate medical systems. Each of these separate medical systems may include its own visual display and its own input device. As a result, the visual displays and input devices can take up valuable space in the procedure room, as they must be placed in position for convenient access and use. These multiple displays and input device also pose a risk of confusion among the various displays and input devices.
Generally, the present invention relates to the seamless, workflow-driven operation of multiple separate medical systems. Embodiments of the present invention provide systems and methods for the operation of two or more medical systems, each having a visual display and standard input devices.
Generally a preferred embodiment of a system for controlling multiple separate medical systems comprises a display with an active display portion and an inactive display portion, an input device, and a controller for selecting one of the medical systems whose visual display is displayed on the active region of the display and which is controlled by the input device. For the purpose of this disclosure, “active” means that the corresponding medical system or remote computer has been selected for both inputs and outputs. The controller selects the medical system at least in part according to a predetermined routine based upon the procedure being performed. Alternatively or additionally, the controller selects the medical system at least in part based upon information from at least one of the medical system. Alternatively or additionally, the controller selects the medical system at least in part based upon user selection.
Generally a preferred embodiment of a method for controlling multiple separate medical systems comprises displaying the visual display output of one of the medical systems on a dedicated active display area, and displaying the visual display output of at least one non-selected medical systems on an inactive display area; communicating commands from at least one input device to the medical system whose visual display output is being displayed on the active area, and actively switching the medical system whose visual display output is displayed on the active area. The medical system whose visual display output is displayed on the active display area is selected at least in part according to a predetermined order based upon the type of procedure being performed with the medical systems. Alternatively or additionally, the medical system selected to have its visual display output displayed on the active display is selected at least in part based upon input received from at least some of the medical systems. Alternatively or additionally the medical system selected to have its visual display output displayed on the active display is selected at least in part based upon user input.
In another preferred embodiment, multiple active displays of multiple medical systems are provided on the composite display, and the movement of a user input device such as a computer mouse is seamlessly tracked by a processor and actively assigned to the appropriate medical system based on the location of the mouse within the composite display.
Embodiments of this invention provide systems and methods facilitating the operation of multiple separate medical systems, simplifying the procedure control site and reducing the risk of confusion.
Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.
A preferred embodiment of a system for controlling multiple medical systems is indicated generally as 20 in
In this preferred embodiment the system 20 includes a display 28, having an active display area 30 and a passive display area 32. The active and passive display areas 30 and 32 can be dedicated portions of a single display device, or they can be dynamically determined portions of a single display device, or they can be portions of separate display devices, which are preferably integrated together.
In this preferred embodiment the system 20 also includes at least one input device, such as a keyboard 34 and/or a mouse 36. The system 20 can also include additional or different input devices such as track balls, joysticks, haptic devices, touch screens, etc.
In this preferred embodiment the system 20 also includes a controller 38, which in this preferred embodiment includes a computer, such as PC 40. The computer is programmed to display the visual display data from a selected one of the medical systems (e.g. one of systems 22, 24, and 26) on the active area 30 of the display 28. The computer is also programmed to provide control signals from the input devices (e.g. keyboard 34 and mouse 36) to the selected medical system, so that the input devices control the medical system whose visual display data is displayed on the active display area 30 of the display 28. This helps eliminate confusion in operating the separate medical systems in the course of a procedure.
In this preferred embodiment the computer is preferably programmed to intelligently select the medical system based at least in part upon a predetermined routine for the medical procedure being performed. Thus, for example, some identification of the medical procedure being performed is provided to the computer, and the computer then selects the medical system to display in the active display area 30 of the display 28. For example this selection can be made based on the time since the start of the procedure, based upon indications of completion of various phases of the procedure either obtained from the various medical systems or from the user. For example after a particular action or command using one medical system, the computer may be programmed to automatically select another medical system. Alternatively or in addition, the computer may select the medical system at least in part based upon information received from the various medical systems. For example if the computer receives an alarm from a particular system, it might automatically select that system to facilitate the user's response to the alarm. Alternatively or in addition, the computer may select the medical system at least in part based upon information received from the user, including an express instruction to switch to a different medical system.
In particular the computer can be programmed to select the medical system based at least in part on a predetermined routine for a given medical procedure, and can take into account signals, including emergency signals, from the various medical systems being used, as well as inputs from the user reflecting the starting or completion of various phases or steps of the procedure, and specific requests to select a specific medical system.
As shown in
The computer 40 can be programmed to prompt the user through a predetermined sequence of steps to conduct the procedure, the program automatically changing the selected medical system as the user responds to the on screen prompts, thereby automatically passing control of appropriate medical system to the keyboard 34 and mouse 36 (or other input devices) at the appropriate times. A portion of the display 28 can be dedicated to displaying prompts, or the prompts can be dynamically allocated.
For example a Clinical Workflow Manager program can be run on the computer 40 to direct the user to perform the steps necessary to perform the procedure, such as setting up an imaging, localization, and magnetic navigation systems for performing a procedure with magnetically navigated devices, and to make the systems required to perform the procedure accessible to the user via the system 20, and in particular the display 28 and the keyboard 34 and mouse 36.
The Clinical Workflow Manager Program might begin with a prompt on the display 28 to “Import Pre-Op image” or similar designation. When the user selects this, a Pre-Op window is opened, and a dialog box enables selection of the appropriate image. The image is loaded into the Pre-Op window. A prompt on the display 28 then prompts the user to move to the next step of acquiring a first X-ray image for registration.
During the acquisition of the first X-ray image for registration the user takes an X-ray image, and the active portion 30 of the display 28 displays the same image as the X-ray monitor. The x-ray system controls (zoom level, frame rate, window width, window level, etc.) are available on this “Live X-ray” display using the keyboard 34, mouse 36, or other input devices. As the user moves a cursor across this active portion 30 of the display 28 with the mouse 36, mouse movements are suitably scaled by the processor and fed to the x-ray system computer. Mouse clicks are also fed to the x-ray system computer. Thus X-ray system control settings may be changed if desired. When the user presses the X-ray foot switch to acquire an X-ray image, the user can directly observe the image on the active portion 30 of the display. The image is automatically stored to a separate “Reference X-ray” window when the mouse cursor is in the appropriate display area on the composite display corresponding to the X-ray system.
The Clinical Workflow Manager can then display a prompt on the display 28 to prompt the user to move to the next step of acquiring a second X-ray image for registration, the second x-ray image possibly being acquired at a different angle with respect to the patient. The x-ray system controls are accessible to the user from the “Live X-ray” window. When the second image is also selected, the Clinical Workflow Manager can then display a prompt on the display 28 to move to the next step of marking landmarks on the pair of X-ray images. A pair of x-ray images is displayed to the user on the active portion 30 of the display. After the user marks a set of corresponding points on the two images (thus defining a set of 3D points through epipolar geometry) using the keyboard 34 and mouse 36, the Clinical Workflow Manager can then display a prompt on the display 28 to select corresponding points on the Pre-Op image. The Clinical Workflow Manager can then display the Pre-Op image in the active portion 30 of the display 28. The user may rotate and translate the Pre-Op image as desired and select a set of landmarks. The landmarks can be matched with the corresponding set of X-ray landmarks from the previous step and the system effects a registration between the Pre-Op image and X-ray coordinates.
The Clinical Workflow Manager can then display a prompt on the display 28 to start Auto Map moves (apply preset field sequence, etc.) and switch the localization/mapping system display to the active portion 30 of the display 28. A sequence of moves is made and the user may move the mouse 36 to select and “freeze” points as one would typically do with a localization system in order to create an anatomical/ECG map. Mouse moves made on (or with reference to) the action portion 30 of the display 28 are suitably scaled and fed to the localization system computer, so that localization user interface tools can be accessed and used via system 20.
The mapping process continues until a complete or suitable anatomical map is obtained. The Pre-Op image may at any time be displayed as well upon user selection, so that anatomical targets can be selected from this image as locations to drive the catheter to (in order to further refine a map, for instance). This sequence outlines a mapping procedure that can be carried out with the system 20 coordinating separate imaging and localization and navigation systems. Although this sequence demonstrates an implementation with user selected screen displays on the system 20, other variations can consist entirely of an automated selection of displays (for instance: (i) as soon as an X-ray is transferred, the computer picks the next window/system to display, or (ii) instead of prompting the user to select the next display as in the above, the processor directly switches the display as appropriate), or a combination of automatically selected and user selected displays at various steps. Likewise, the displays of various systems such as Localization system, ECG system, blood pressure monitoring system, X-ray, Ultrasound or other imaging system, remote navigation system, and so on can be displayed on the Consolidated UI in various sequences as appropriate for the procedure.
In the preferred embodiment shown in
A software program on the controlling PC 40 monitors the keyboard 34 and mouse 36 and passes information on as necessary to the remotely switchable kvm 44 The remotely switchable kvm 44 routes keystrokes to the appropriate medical system computer as commanded by the controlling PC. An example of a remote KVM that could do this is the Black Box KV3108SA-R5, 8 port controllable KVM
The medical systems (e.g. systems 22, 24, and 26) can have their own mouse/keyboards that serve as backup controls. These could be attached via keyboard splitter and mouse splitter cables.
Of course the system 20 does not have to use a PC 40, and instead any device capable of producing a video image could be substituted. The computer would need to at least have a microprocessor with some software. Any clinical video producing device that could be controlled by a keyboard/mouse would be plugged into the KVM. It is possible that some medical systems will only produce video display data, and not accept inputs. These devices could nonetheless be integrated with the system 20, and displayed at appropriate times on the active display area 30 of the display 28, even though the input devices do not provide any control signals.
A second embodiment of a consolidated user interface in accordance with the principles of this invention is indicated generally as 100 in
Each of the separate systems 102, 104, and 106 are operatively connected to the CPU 108 of the system 100. This connection can be a dedicated wire or wireless connection, or connection via a network. The CPU 108 is programmed to display the associated displays of at least two of the separate medical systems on a composite display 110. The composite display 110 is preferably a single display, such an LCD flat panel display, a CRT display, a plasma display, or a projection display. However, the composite display could comprise a plurality of separate displays that are associated together in a manner for convenient viewing by users.
As shown in
The system 100 also includes input devices, such as a key board 120 and mouse 122 for operating the system and selectively controlling the separate medical systems. Other input devices can be provided in addition to, or instead of, the key board 120 and mouse 122.
In this preferred embodiment the CPU is also programmed to sequentially display prompts 124 for at least some of the steps of a multistep medical procedure on the composite display 110. These prompts 124 can be a detailed description of the step, or a more abbreviated message recognizable to the user. The prompts can include text or symbolic characters, and are preferably displayed on a dedicated portion or window 126 on the composite display.
The system 100 can display all of the prompts 124 simultaneously, or a subset of the prompts (
The CPU 108 is preferably programmed to automatically change the composite display 110 if completing the currently prompted step involves use of one of the separate medical systems. This change in the composite display 110 can take many forms. For example, it can be adding the display of the involved computer controlled medical system to the composite display as is shown
Alternatively the CPU can be programmed to highlight the display of the involved computer controlled medical system if it is already being displayed on the composite display 110. Highlighting the display of the involved computer controlled medical system can be accomplished by displaying the associated display of the involved computer controlled medical system in a prominent location, as shown in
Alternatively, highlighting the display of the involved computer controlled medical system can be accomplished by resizing the associated display of the involved medical system as shown in
Alternatively, highlighting the display of the involved computer controlled medical system can be accomplished by applying a border or other indicator to the associated display of the involved medical system as shown in
Alternatively, highlighting the display of the involved computer controlled medical system can be accomplished by changing the color or brightness of the involved medical system as shown in
Of course some other manner of highlighting can be used, the highlighting serving to quickly identify to the user, the display from the medical system that is involved in the current step, and/or facilitating the use of the display from the medical system that is involved in the current step.
The system 100 may be able to automatically detect the completion of the step associated with the prompt of the current step, and automatically advance the prompt. Alternatively, the user can operate a “completed” 130 or “next” 132 control (
In the preferred embodiment the composite display 110 is configured so that the input devices control a single displayed cursor, and wherever the displayed cursor is located, the input devices, such as the key board and mouse, can be used to operate whatever system is associated with the display over which the cursor is positioned. Thus, by manipulating the input devices the user can move the cursor over then entire composite display 110, and when the cursor is positioned over the portion displaying the associated display of one of the separate medical systems, the input devices can be used to operate that medical system. In this embodiment, more than one of the medical system displays on the composite system is active. The assignment of the input device or computer mouse position to the appropriate medical system is performed by the controller processor, which scales the mouse cursor movement over the composite display based on the current absolute mouse position so as to seamlessly tie the mouse to the appropriate medical system display on the composite display. The controller unit assigns, routes, and feeds a corresponding USB output to the appropriate medical system computer. In this manner a single display (the composite display 110), and a single set of input devices (the keyboard and mouse) can be used to control the navigation system, imaging system, localization/mapping system, and so on, independently of the separate displays and separate individual input devices of those systems.
Thus embodiments of the methods and apparatus of this invention provide a way for a user to operate a plurality of separate computer systems, by displaying the displays from each of a plurality of separate computer systems on a composite display; and operating a cursor control device to manipulate a cursor on the composite display (including on the displays of the separate computer systems), and operating the cursor control to operate the separate computer system whose display corresponds to the location of the cursor on the composite display. The cursor manipulated on the composite display may change appearance when it is positioned over at least a subset of the displays of the separate computer systems.
In accordance with another embodiment of this invention, a control can be provided for operating a plurality of separate computer systems. The control comprises a composite display; a video system for receiving display input from each of the plurality of separate computer systems and displaying it on the composite display; a processor; a distribution system connected to the processor and to each of the separate computer systems, and a cursor control device connected to the distribution system, the distribution system receiving input from the cursor control device and communicating the input to the processor, and communicating cursor control input generated by the processor to at least one of the separate computer systems when the cursor on the composite display is on the display corresponding to that separate computer system.
In accordance with another embodiment of this invention, a method of operating a plurality of separate computer systems is provided. The method comprises displaying the separate displays from each of the plurality of separate computer systems on a composite display; translating inputs generated by the operation of a cursor control device to control signals to an interface computer to move a cursor in the interstices between the separate displays on the composite screen and to control signals to the separate computer systems to operate the separate computer systems, so that the cursor appears to operate substantially continuously across the entire composite display, and when the cursor is positioned over the separate display of one of the separate computer systems, the cursor control device operates the separate computer system. In the preferred embodiment the step of translating inputs generated by the operation of a cursor control device to control signals to an interface computer and to control signals to the separate computer systems includes communicating the signals using the USB protocol.
In accordance with another embodiment of this invention, a method of operating a plurality of computer systems is provided. The method comprises translating inputs generated by the operation of a cursor control device to control signals to a first computer to operate the first computer, and generating and transmitting cursor control signals to operate at least one other computer system to operate the at least one other computer system in a corresponding manner. In the preferred embodiment the step of generating and transmitting cursor control signals to at least one other computer systems comprises generating control signals that are different from the control signals for the first computer. The step of generating and transmitting cursor control signals to at least one other computer systems comprises communicating signals via USB protocol.
In accordance with another embodiment of this invention, a method of controlling multiple computer systems running programs with complimentary actions is provided. The method comprises accepting inputs from the operation of a cursor control device and operating a first computer in accordance with the accepted inputs to achieve a desired action, and translating the inputs to a second set of inputs to cause at least one other computer to perform a complimentary action. In the preferred embodiment, the inputs are translated into USB inputs that cause the least one other computer to perform the complimentary action.
One method of accomplishing this is mapping one or more mouse actions on a first computer system (for example operating a particular feature) with a corresponding or complementary action (for example operating the same or a related feature) on at least one other computer system. When one of the mapped actions on the first computer system is performed, the system can automatically generate artificial mouse commands to at least one other computer system to cause that computer system to take a corresponding action. For example a medical navigation system can have a user interface in which the display can be oriented in any of several directions, by pointing and clicking on the control on the display for the medical navigation system. The corresponding controls for orienting the displays from the medical imaging system and the medical localization system can be mapped to the appropriate controls on the navigation system, so that when the user moves a cursor on the display for the navigation system and clicks, artificial mouse commands are generated for the imaging and localization systems to cause their respective cursors to point to a corresponding control and click, without requiring the user to move the cursor and click in each of these systems. In this way, by pointing and clicking in the display of just one of the systems, corresponding actions in the other systems can be taken so that all of the displays are oriented in the same manner. Depending upon the mapping, pointing and clicking on a display of one of the systems on the composite display can cause the same action or a related action to be taken by the other systems.
In another embodiment of this invention a method of performing a medical procedure using a plurality of medical systems controlled by separate computer systems, in which at least some of the computer systems have complementary actions, is provided. The method comprises operating a cursor control device to cause a first computer system to perform a desired action; and generating a cursor control signal to cause at least one other computer system to perform a complementary action. The step of generating a cursor control signal preferably includes using a look-up table to determine the cursor control signal for the at least one other computer system that causes a complementary action to the action performed by the first computer system. This complementary action can be a corresponding action, so that the systems perform the same action, or a coordinated action, so that the systems operate together when commands are made on one of the systems. In specific embodiments, the coordinated actions are at least in part dependent upon the selected intervention workflow.
In another embodiment of this invention, an integrating control and display system for operating a plurality of separate computer systems is provided. The integrating control comprises a master display integrating the separate displays from each of a plurality of separate computer systems; and a cursor control device for manipulating a cursor on the master display (including on the displays of each of the separate computer systems on the master display). The cursor control device operates the system corresponding to the separate display on which it appears on the master display. This allows the user to manipulate a cursor over a single composite display using a single control device, and to control any of a plurality of different computer systems whose displays are displayed on the composite display.
This is preferably achieved using USB protocol so that the systems can be connected without the need for any modification to the system or its programming. Thus, in another embodiment of this invention, a method of operating a plurality of separate computer systems without altering the hardware or software of the separate computer systems is provided. The method comprises: displaying the separate displays of each of the plurality of separate computer systems on a composite display of a master computer system; and operating the cursor control device of the master computer system to manipulate a cursor on the master display, and when the cursor on the master display overlies the separate display of one of the separate computer systems, generating cursor control signals to operate the separate computer system corresponding to the operation of the cursor control device of the master system. The generated cursor control signals are preferably USB cursor control signals for operating the separate computer system.
In another embodiment of this invention, a method of controlling a plurality of separate computer systems is provided. The method comprises displaying the display from each of the separate computer systems on an integrated display; accepting inputs form a cursor control device to move a cursor control device across the integrated display. The cursor control being operable to operate the separate computer system over which the cursor is displayed on the integrated display.
In another embodiment, a system for controlling a plurality of separate computer systems via a master computer is provided. The system comprises a master computer, including a processor, display for displaying a display output from the master computer and each of the separate computer systems, and a keyboard for entering instructions to the master computer system. The processor of the master computer system is programmed to generate control signals for each of the separate computer systems based on the keystrokes on the keyboard, at least some of the keystrokes generating control signals for less than all of the plurality of computer systems. The system is preferably configured so that at least one keystroke on the keyboard generates a control signal for only one of the plurality of separate computer signals. This allows a single keyboard to be used to control a plurality of computer systems, while preserving at least one key for dedicated control of a particular computer system, even while the remainder of the keys are controlling other computer systems.
In one embodiment of this invention, a software parser resides on the master computer and interprets command modifiers to decide onto which of the controlled medical system(s) or computer(s) the command is applicable. The command format is modified in accordance with the parser design to un-ambiguously interpret commands, including generic commands such as “rotate image.” In one embodiment, the command syntax is such that commands are preceded by a letter, or combination of letters, that uniquely designate control system(s) to be recipient for the issued command. For example, and in the context of
Alternatively or additionally, the parser functions as a “global keyboard.” Specific commands or command shortcuts direct active control to one of a plurality of controlled systems. That is, upon the user typing a short keyboard sequence, active control is passed to a specific controlled system as determined by the keyboard sequence contents. In specific implementations, the keyboard inputs also moves the master computer mouse cursor onto the active window as determined by the command parser.
In another embodiment of this invention, a touch screen is provided as part of the master control system user interface. Buttons on the touch screen enable selection of the active system among the plurality of controlled systems. Additionally, buttons on the touch screen are programmed to activate control command macros consisting of at least one command or operation; in specific embodiments, launching such a macro also redirects the master control system cursor to a location on the associated system display to press a button or select a menu item.
In another embodiment of this invention, a method of controlling at least one of a plurality of separate computer systems based upon the operation of a master computer system is provided. The method comprises generating synthesized USB control signals for at least one of the plurality of separate computer systems, in response to the operation of the master computer systems.
One possible implementation of several of the embodiments of the invention is illustrated as 200 in
The consolidated display preferably also includes a window 220 in which prompts 222, preferably generated by the system, guide the user according to a workflow plan. This workflow can be entirely predetermined based upon the type of procedure, or it can be dynamically determined based in part upon selections by the user and/or inputs from the various separate medical systems. In either case the system 200 is preferably set up to automatically reconfigure the composite display 204 for current step in the workflow. The configuration of the composite display 204 for each step can be one that was preset, or it can be one that was modified and saved by the user.
A single cursor is movable on the consolidated display 204 by using the keyboard 206 or mouse 308. The keyboard 206 and mouse 208 are functional to control the separate medical system associated with the portion (e.g. 212, 214, 216, and 218) over which the cursor is positioned. Thus the user can move the cursor from portion to portion by manipulating the keyboard 206 or mouse 208, and when the cursor is in a particular portion of the consolidated display, use the keyboard 206 and mouse 208 to control the associated system. Thus with a single keyboard and mouse, the user can control all of the systems, simply by moving the cursor on the consolidated display 204. A dedicated controller 210 can be provided with buttons or other controls that operate one or more of the separate medical systems regardless of the position of the cursor on the consolidated display. This allows the user to immediately input commands to a system irrespective of the position of the cursor. The dedicated controller can operate just one of the systems, such as the navigation system, or it can have dedicated controls for two or more of the systems. The controls can be dedicated to particular functions, or they can be reprogrammable by the system based upon the context, or by the user, based upon preference.
Alternatively or additionally, a parsing software running on the master controller enables direct use of the keyboard as a means to specify which subset of systems is addressed by a given command. This option for example is available to the user upon moving the master cursor into the master computer text output/input window(s). Alternatively the cursor is automatically moved to the text output/input window upon the user pressing a key or key combinations; such key or key combination being defined for example as a key override mode. Alternatively the cursor is automatically moved to the text output/input window upon the user pressing a combination of mouse buttons or sequence of mouse buttons. In implementations where the system includes a touch screen interface, specific buttons on the touch screen are mapped to activate control command macros; such macros can redirect the master control system cursor to a location on the associated system display to press a button or select a menu item.
The operation of the system 200 is illustrated in
A possible architecture of the system is shown in
Regardless of the configuration, the user can control any of the systems displayed simply by moving the cursor over the appropriate portion of the composite display, and operating the graphical user interface of the system with the mouse and keyboard.
The components of the system 200 are shown schematically in
This arrangement allows the user to connect separate systems together without modifying the individual systems, and control them from the composite display 204 using the keyboard 206 and mouse 208. The system 200 operates so that the mouse appears to move continuously across the composite display 204 across the portions that represent the displays of the individual systems, where the cursor is actually a cursor moving on the interface of the separate system, and across the portions of the composite display between those portions, where the cursor is the cursor of the interface of the system 200. By managing the cursor in this way, it appears to be a single cursor moving continuously over the display 200, even though it may actually be several cursors.
As shown in
One embodiment of a seamless multi-system keyboard and mouse controller uses a USB implementation.
Thus embodiments of this facilitate the control of multiple medical systems with a single, simple interface system.
The consolidated system as described above limits the number of simultaneous keyboard and mouse controls to one. It is possible to retain the native keyboard and mouse of each system connected to the consolidated system; however, these components do not operate similarly to the consolidated system global keyboard and mouse. Instead, these components are limited to control the native system they are directly connected to rather than being capable of operating any system connected to the consolidated system. In another embodiment of this invention, a consolidated user interface is disclosed that controls a multiplicity of medical systems or computers, and a plurality of keyboard and mice combinations can be connected to the consolidated system to support at least two simultaneous controls operating seamlessly with the consolidated display.
In a first embodiment, two or more input devices combinations such as keyboard and mouse are connected to the consolidated system. Each keyboard may initially be set to focus its input on a separate system controlled by the consolidated display (subject to the number of input device combinations being not more than the number of controlled systems). Each keyboard is associated with the system display it's controlling through a graphical indication; one example of such a graphical indication is a colored border around the window and a corresponding colored marker on the keyboard, or alternatively a keyboard of the corresponding color. Several design approaches enable simultaneous control of at least one of a plurality of controlled systems. In one embodiment, the position of each global cursor is indicated with a graphic such as a circle, halo, or similar graphic differentiator presented as an overlay constantly displayed over all windows of the consolidated display. In this manner, as any of the mice is moved, the corresponding global cursor graphic is moved. In a preferred embodiment, the graphic does not block the central area of the global cursor position, so as to allow the native cursor of each system to appear in its center. When the global cursor position of each mouse is located in a distinct associated system window, each mouse controls its associated system independently without interference from any of the other mice; the native cursor of each system appearing in the center of each global cursor position graphic. When the global cursor positions of two or more mice are in the same system window, careful design enables seamless operation without interference. For example, when a first global cursor is in a first system window and a second global cursor is in a second system window, each global cursor begins with the native cursor of the system in the center of the global cursor graphic. When a first mouse is moved to translate a first global cursor to a second system window, the native cursor for the second system disappears from the center of the second global cursor and immediately appears in the center of the first global cursor as soon as it crosses the border into the second system window. In this implementation, the locus of control switches to the latest entrant into the window of the particular system under control. Effectively, only one mouse will control the system at any time even though the global cursor graphics of the two (or more) mice appear in the same window. A list of control priority is maintained for each control window, in such a manner that the first priority is always with the latest entrant cursor, and prior cursor(s) that are still in the window are pushed down the ordered list. Once a cursor leaves a given window, it cease to be listed in the priority list for that window. Should the latest entrant cursor stop moving, the locus of control returns to the next cursor in the priority list that is moving, which then assumes the first priority in the priority list, and the former “priority” cursor (the cursor having control) moves to the second place on the priority list. Should a second cursor start moving while the priority cursor is still moving, priority remains with the current priority cursor. If none of the cursors is moving, priority remains with the last cursor having priority. Accordingly in such situations the native cursor will always react to the currently moving mouse and in the event two or more mice are concurrently moving, the mouse that initiated movement first maintains control of the native cursor. This priority order is shuffled only by a “new” cursor entering the window; or by the priority cursor ceasing to move and an other cursor continuing or resuming motion. The global priority cursor graphic appears with a matching shape or color corresponding to the controlling mouse, and is highlighted by use of a significantly higher intensity, larger shape, or similar differentiator that is not distracting to the users. With this approach, the operation appears nearly the same as having a consolidated system mouse operating in the same window as a native mouse on the same independent system, but enables multi-user workflows from a single large display. In addition, this embodiment of the present invention offers a new level of control for remote proctoring or procedure scenarios. In this manner, a remote proctor or physician can effectively work in tandem with a local user of the system.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US6784869 *||Nov 15, 2000||Aug 31, 2004||The Boeing Company||Cursor and display management system for multi-function control and display system|
|US20030023754 *||Jul 27, 2001||Jan 30, 2003||Matthias Eichstadt||Method and system for adding real-time, interactive functionality to a web-page|
|US20060230110 *||Apr 12, 2005||Oct 12, 2006||Vanharlingen Brian||Apparatus and system for managing multiple computers|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7961926||Jul 13, 2010||Jun 14, 2011||Stereotaxis, Inc.||Registration of three-dimensional image data to 2D-image-derived data|
|US8308628||May 15, 2012||Nov 13, 2012||Pulse Therapeutics, Inc.||Magnetic-based systems for treating occluded vessels|
|US8313422||May 15, 2012||Nov 20, 2012||Pulse Therapeutics, Inc.||Magnetic-based methods for treating vessel obstructions|
|US8369934||Jul 6, 2010||Feb 5, 2013||Stereotaxis, Inc.||Contact over-torque with three-dimensional anatomical data|
|US8529428||May 31, 2012||Sep 10, 2013||Pulse Therapeutics, Inc.||Methods of controlling magnetic nanoparticles to improve vascular flow|
|US8687019 *||Mar 25, 2011||Apr 1, 2014||Seiko Epson Corporation||Composite multi-image display device, composite multi-image information processing device, composite multi-image display system, composite multi-image display method, and composite multi-image formation processing method|
|US8715150||Nov 2, 2010||May 6, 2014||Pulse Therapeutics, Inc.||Devices for controlling magnetic nanoparticles to treat fluid obstructions|
|US8926491||Sep 6, 2013||Jan 6, 2015||Pulse Therapeutics, Inc.||Controlling magnetic nanoparticles to increase vascular flow|
|US20100083122 *||Apr 1, 2010||International Business Machines Corporation||Systems, methods and computer products for controlling multiple machines using a seamless user-interface to a multi-display|
|US20100295870 *||May 24, 2010||Nov 25, 2010||Amir Baghdadi||Multi-source medical imaging system|
|US20110234632 *||Sep 29, 2011||Seiko Epson Corporation||Image display device, image information processing device, image display system, image display method, and image information processing method|
|US20110291976 *||Mar 2, 2010||Dec 1, 2011||Ricoh Company, Ltd||Touch panel device, display device equipped with touch panel device, and control method of touch panel device|
|U.S. Classification||345/157, 345/1.1|
|Cooperative Classification||G09G5/08, G09G5/006, G09G2370/24, G06F3/0481, G06F19/3406, A61B19/56, G06F3/14|
|European Classification||G06F3/0481, G09G5/00T4, G06F3/14|
|Jan 19, 2010||AS||Assignment|
Owner name: STEREOTAXIS, INC.,MISSOURI
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GARIBALDI, JEFFREY M.;BLUME, WALTER M.;KASTELEIN, NATHAN;AND OTHERS;SIGNING DATES FROM 20070303 TO 20070328;REEL/FRAME:023805/0323
|Dec 6, 2011||AS||Assignment|
Owner name: SILICON VALLEY BANK, ILLINOIS
Free format text: SECURITY AGREEMENT;ASSIGNOR:STEREOTAXIS, INC.;REEL/FRAME:027332/0178
Effective date: 20111130
|Dec 8, 2011||AS||Assignment|
Owner name: COWEN HEALTHCARE ROYALTY PARTNERS II, L.P., AS LEN
Free format text: SECURITY AGREEMENT;ASSIGNOR:STEREOTAXIS, INC.;REEL/FRAME:027346/0001
Effective date: 20111205