|Publication number||USRE42210 E1|
|Application number||US 12/228,650|
|Publication date||Mar 8, 2011|
|Filing date||Aug 13, 2008|
|Priority date||Mar 22, 2002|
|Also published as||US7092735, US20040203608, USRE44732|
|Publication number||12228650, 228650, US RE42210 E1, US RE42210E1, US-E1-RE42210, USRE42210 E1, USRE42210E1|
|Inventors||Robert Osann, Jr.|
|Original Assignee||Osann Jr Robert|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (25), Referenced by (3), Classifications (30), Legal Events (2)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This invention relates to the operation of mobile phone communication systems such as those including cellular phones or any form of mobile wireless communication device capable of voice communication, and in particular, enhancements to conventional realtime voice communications and voicemail storage and retrieval systems for mobile phones allowing the integration of still images and video clips.
Today, voicemail for mobile phones is simply that, storing voice messages when the receiving party is not available, to be retrieved at a later time. Essentially, voice mail for mobile phones operates in a very similar manner to voicemail for conventional office phones. To date, there has been no attempt to integrate images or video with voicemail for mobile phones or conventional office phones.
Communications between two parties where both voice and video are utilized is well known and is commonly referred to as video conferencing or teleconferencing. Sometimes this capability is also known as a “Net Meeting”. Here, a group of individuals, each having a computing device including microphone, speaker, video camera, and a connection to the World Wide Web, are able to communicate in a real-time manner through both video and voice mediums. The ultimate goal of a video conferencing system is to transmit a continuous stream of audio and video to and from each participant in the meeting or conversation, and to emulate as much as possible the interaction that would occur if all participants were in the same room. To do this, there will be compromises and limitations for many years to come relative to the desired functionality for video conferencing due to bandwidth limitations. In contrast to this goal and these compromises, the invention described herein uses existing bandwidth capabilities to selectively integrate images and video with voice communication in order to solve very specific and valuable problems. The manner in which this integration occurs has not been offered before.
The term “videomail” is often used in the industry, but in contrast to voice mail, does not refer to the storage of messages for later retrieval. Instead, it refers to attaching video clips to e-mails and a similar manner to the common practice of attaching files containing digital photographs to e-mails.
Cameras have, at times, been incorporated into mobile phones for surveillance purposes. Instead of broadcasting digital video via a data link with the mobile/cellular service provider, these phones broadcast an RF signal to a viewing receiver in the same manner as any other covert surveillance camera system.
Digital cameras are available as attachments to some PDAs (Personal Digital Assistants), some of which also are available with wireless connection to a service provider allowing accessibility to the World Wide Web. Some PDAs with wireless Web-interface capability can also function as mobile or cellular phones. Some mobile phones have added PDA capabilities, again with wireless Web-interface capability. Also, some mobile phones now have digital cameras available as options to allow pictures to be captured and attached to emails. However, none such devices offer the integration of still images or video with cellular voice communication or with voicemail for cellular phones following the existing paradigm of phone calls and voicemail messages.
A very successful capability offered by one cellular service provider (the Nextel Direct Connect® digital two-way radio service), allows frequent communications among a group of individuals who work closely together to be more convenient and less costly. As a result, this capability has been adopted by the majority of businesses that require frequent communications with individuals working at different locations in a local area, for the most part, businesses in the construction industry. This allows a manager, foreman, or responsible person to more easily keep track of the progress at a variety of locations, and more readily communicate to affect swift problem resolution. Unfortunately, these communications rely on the ability of the individuals involved to clearly describe situations and problems they observe in terms that the responsible person can understand in order to best make decisions and guide the remote workers. The ability for the responsible person to see the subject or problem area would significantly enhance the value of these communications.
A solution is needed that, given the bandwidth limitations of current and next generation cellular data transmission capability, provides an easy way for persons to communicate image and video information, while maintaining a communication paradigm that is familiar, basically the paradigm of phone calls and voicemails. Such a system would allow high resolution images to be transmitted when a high level of detailed is required, and alternately, video clips (which may be at lower resolutions) where spatial relationships and or motion information is required.
An enhanced communication and voicemail solution for mobile phones is described where still images and/or video clips are injected into the voice stream creating a “video-voice” call. When a receiving party is not available to take a video-voice call, this combined stream of voice and image information is stored at the mobile service provider in a manner similar to voice mail today. Then, stored video-voicemails may be retrieved at a later time by the receiving party. While the sending party may use a normal size mobile phone containing a miniature digital camera, the receiving party may view video-voicemail images on a variety of devices including a wireless mobile phone or PDA, or alternately a conventional PC connected to the World Wide Web.
Compared with continuous, full motion video, occasionally injecting a still image or video clip into the voice stream allows much higher resolution images to be sent given bandwidth limitations, allowing the receiving party to view a subject or situation in much greater detail. For use in business applications, conveniently viewing this more detailed information, synchronized with voice explanations, enables better decisions thereby saving time and money.
The present invention is described with respect to particular exemplary embodiments thereof and reference is accordingly made to the drawings in which:
A better way to communicate between persons operating at remote locations and a manager, advisor, or person in authority, would include interspersing images and/or video clips within the voice communication stream, whether or not it is real time or a voicemail message left for future retrieval. Such a solution utilizes video clips where they are most effective (even with reduced resolution)—conveying motion information, or alternately conveying spatial information of the subject area by way of a “Pan” motion with the camera. In addition, high resolution still images can be injected into the voicestream to allow a very detailed view of a particular subject or problem area, synchronized with a verbal description and other related discussion.
Many applications will benefit from this new capability, including the construction industry, the medical and care industry, field service and repair, building inspection, insurance adjusters, or any application where people need a convenient way to document situations at remote locations and make this information available to others. Another specific application that will benefit from video-voice communications is that of emergency situations. When someone calling 911 to report an emergency can also provide video clips and high resolution still images, a dispatcher or paramedic receiving the call can much better understand the situation and even instruct the caller in a way that may save lives.
In general, integrating this capability with the familiar paradigm of the mobile phone call and voicemail is most convenient and useful. As discussed earlier, it is known to attach digital photos and digital video clips to emails. Emails are inherently digital, so this attachment is natural. Emails are also not a realtime communications medium. However, it is not known to attach digital photos and digital video clips to voice communications, whether realtime or as stored messages.
Note that, throughout this specification, the terms “mobile phone”, “cellular phone”, and “wireless phone” are synonymous and refer to any mobile communications device capable of bi-directional voice communication.
If the receiving party is not available for realtime communications, a message may be stored at service provider 2 in database 3 designed to store voice, image, and video clip information while retaining the time relationships between the three. Per scenario 6, the retrieving party accesses previously stored voice, image, and video clip information directly from the service provider via some form of wireless phone device. Alternately, per scenario 5, the receiving party may access stored voice, image, and video clip information via the World Wide Web through either wired or wireless Web access. Although
For cellular voice communications today, real-time conversations are never recorded. Only voicemail messages are recorded. As relates to video-voice conversations, it is however useful to record real-time communications that contain either still images or video clips injected into the voice stream. This may be desired in order to document the visual information being conveyed as part of the conversation for later retrieval. It may also be desired when the receiving party answers a call and, as part of the conversation, subsequently realizes that images or video clips are being transferred, but can't properly or safely view them.
With the ability to have voicemail or video-voicemail messages stored at the mobile service provider retrieved via the World Wide Web, users can archive conversations, voicemail messages, and video-voicemail conversations and messages on their personal or business computer system. Today, there is not a convenient way to archive mobile voicemail messages.
If the receiving party is not in a position to view these images real-time, such as when driving a car, having the conversation recorded including all images will allow them to review the visual and audio information by retrieving the recorded message at a later time. Alternately, if the receiving party is driving or otherwise in a situation where viewing is inconvenient or impossible, it may be useful to have the ability to transfer a real-time conversation into video-voicemail, if the conversation reaches a point where it is meaningless to continue real-time without the receiving party being able to view the images or video clips. This capability may be implemented by always recording realtime video-voice conversations at the mobile service provider, and discarding the information at the end of the conversation if the receiving (or calling) party has not taken some action (like pressing some button) to initiate the saving of the video-voice conversation. Alternately, the user could set the default mode to be that of automatically saving conversations, deleting them later if not needed or deleting them after they have been downloaded via the Web and archived. A variation on this these would include automatically saving conversations from a particular calling party, and deleting them later if not needed or deleting them after they have been downloaded via the Web and archived. The accumulation of information resulting from these recording scenarios would require a much larger amount of memory for storing messages at the service provider, but if this is valuable, it is a service that users would be willing to pay extra for.
The scenarios just described can also be applied to recording realtime voice conversations for later retrieval and archive. Even archiving voice alone can be a powerful tool within business applications. For recording realtime conversations, the issue of permissions and privacy arises. One easy to handle the granting of “permission to be recorded” would be that permission is deemed to be given to record conversations from or to a particular phone number by calling from that particular phone number and taking a prescribed action which could include entering a specified code. It may instead be desired to record only video-voice conversations and not voice-only conversations, again with permissions having been given.
Although the essence and value of a video-voice conversation as described herein is bi-directional for voice communication, but mostly or solely unidirectional with regard to the transmission of video clips and still images, the methods described may in certain circumstances, be valuable in a bi-directional manner. For instance, a worker at a jobsite may send images to a supervisor at a remote location, and the supervisor may, in return, send an image of a portion of a blueprint while pointing out some specific details to resolve certain issues. Of course, this requires that the worker have a device with a display capable of presenting the blueprint image with enough resolution to properly discriminate the necessary information.
Another purpose for recording video-voice messages involves a reverse scenario where the supervisor/foreman/responsible party may wish to record information and instructions concerning a particular job site, such that the information and instructions may be viewed by workers at a later time before commencing work or as part of problem resolution during the job.
If the integral camera and associated electronics are designed to allow either still images or video clips to be captured and transmitted, there are at least two methods of utilizing shutter button 8 to easily support both. In the first method, the button may be pressed briefly and released to record a still image, or pressed and held for a longer duration to enable a video clip to be transmitted during the duration of time for which the button is held. In the second method, button 8 may be partially depressed to capture a still image or fully depressed to capture a video clip. Alternately for each of the above methods, which button action captures still images and which captures video clips may be reversed. In addition to the methods just described for determining when video-clips are captured as opposed to still images, a more conventional method can always be utilized where the desired mode of capture is first selected through a key or combination of key presses on the phone's keypad, followed by pressing a “shutter” button.
Although this specification refers to transmitting both still images and video clips, an implementation may only deal with one of the two. If the bandwidth limitations are severe, occasionally injecting high resolution still images into the voicestream is probably more valuable than video clips. Eventually, when the available bandwidth is at a level where high resolution video clips can be easily sent via mobile communications, sending only video clips will be appropriate. In the interim, the combination of high resolution still images and lower resolution video clips may be the best overall compromise.
Since the user of the phone/camera will be holding the device in front of them, they will not be able to talk directly into a normal microphone. Hence, it is necessary to have either a speaker phone capability, or some form of wired or wireless headset to allow bi-directional voice communication while the camera function is being utilized.
Looking again at
Where, according to this invention, video clips or high-resolution still images are injected into the voice stream, the last image to be captured according to the action of shutter button 8 will typically be maintained on the display of the capturing phone for a predetermined time period after button 8 is released, or until otherwise terminated by a subsequent action of the sending party. Also, when button 8 is released, normal phone display information may optionally again be superimposed on the displayed image if desired.
Since it is desired that the party capturing and sending images can clearly and easily observe what images are being captured, it may be necessary to add some form of sun-shade to allow clear observation of an LCD “viewfinder” display on the phone/camera. Alternately, or in addition, it may be useful to add a polarizing filter over an LCD display for better viewing in bright sunlight. Other display technologies, more easily viewed in direct sunlight, may be utilized. It may also be useful to add an optical viewfinder such as those found in many conventional digital cameras.
Therefore, a methods and apparatus for implementing a combination video/voicemail system especially useful in the construction industry and other industries requiring remote viewing with guidance and supervision, has been described
It should be understood that the particular embodiments described above are only illustrative of the principles of the present invention, and various modifications could be made by those skilled in the art without departing from the scope and spirit of the invention. Thus, the scope of the present invention is limited only by the claims that follow.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5206721 *||Mar 8, 1990||Apr 27, 1993||Fujitsu Limited||Television conference system|
|US5550646 *||Sep 13, 1993||Aug 27, 1996||Lucent Technologies Inc.||Image communication system and method|
|US5734425 *||Jan 18, 1995||Mar 31, 1998||Eastman Kodak Company||Electronic still camera with replaceable digital processing program|
|US5754227 *||Sep 28, 1995||May 19, 1998||Ricoh Company, Ltd.||Digital electronic camera having an external input/output interface through which the camera is monitored and controlled|
|US5963245||Sep 24, 1997||Oct 5, 1999||Mcdonald; Arcaster||Video telephone|
|US6031526 *||Aug 8, 1996||Feb 29, 2000||Apollo Camera, Llc||Voice controlled medical text and image reporting system|
|US6069648||Aug 14, 1998||May 30, 2000||Hitachi, Ltd.||Information communication terminal device|
|US6084630 *||Apr 18, 1996||Jul 4, 2000||Canon Kabushiki Kaisha||Multimode and audio data compression|
|US6111604 *||Feb 21, 1996||Aug 29, 2000||Ricoh Company, Ltd.||Digital camera which detects a connection to an external device|
|US6188432 *||Jun 23, 1997||Feb 13, 2001||Nikon Corporation||Information processing method and apparatus for displaying and zooming an object image and a line drawing|
|US6373516 *||Nov 15, 1999||Apr 16, 2002||Ericsson, Inc.||Picture position indicator for picture phone|
|US6380975||Jul 5, 2000||Apr 30, 2002||Ricoh Company, Ltd.||Digital still video camera having voice data generation|
|US6424843||Apr 22, 1998||Jul 23, 2002||Nokia Oyj||Multi-function telecommunication device|
|US6751473||Feb 26, 1999||Jun 15, 2004||Lucent Technologies Inc.||Wireless communication terminal having a video image capability|
|US6812954||May 4, 1999||Nov 2, 2004||Orange Personal Communications Services Limited||Mobile communications|
|US6882864||Feb 4, 2002||Apr 19, 2005||Mitsubishi Denki Kabushiki Kaisha||Cellular phone with imaging device|
|US6906741||May 30, 2002||Jun 14, 2005||Palm, Inc.||System for and method of conferencing with a handheld computer using multiple media types|
|US20010032335||Mar 5, 2001||Oct 18, 2001||Jones Lawrence R.||Picture communications system and associated network services|
|US20010050977||Dec 31, 1997||Dec 13, 2001||Irwin Gerszber||Video phone multimedia announcement answering machine|
|US20020051181||Apr 26, 2001||May 2, 2002||Takanori Nishimura||Information processing apparatus and method, information processing system and medium|
|US20020130956||Feb 25, 2002||Sep 19, 2002||Ricoh Company, Ltd.||Digital still video camera, image data output system for digital still video camera, frame for data relay for digital still video camera, data transfer system for digital still video camera, and image regenerating apparatus|
|US20020147661 *||Sep 26, 2001||Oct 10, 2002||Fujitsu Limited||Method of ordering and delivering picture data|
|US20020171673 *||May 3, 2001||Nov 21, 2002||International Business Machines Corporation||Method, system, and program for mining data in a personal information manager database|
|US20040014456||Aug 20, 2001||Jan 22, 2004||Vaeaenaenen Mikko Kalervo||Instant video-and voicemail messaging method and means|
|US20050083642||Mar 8, 2002||Apr 21, 2005||Tsuyoshi Senpuku||Mobile communications device, and display-control method and program for mobile communications device|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8284233 *||Oct 9, 2012||Broadcom Corporation||Utilizing image sequences to perform video streaming during video conferencing|
|US20100201780 *||Feb 6, 2009||Aug 12, 2010||Bennett James D||Utilizing image sequences to perform video streaming during video conferencing|
|USRE44732 *||Jan 31, 2011||Jan 28, 2014||Foremost Technologies Llc||Video-voicemail solution for wireless communication devices|
|U.S. Classification||455/556.1, 345/699, 455/557, 348/14.02, 348/14.01, 715/201, 455/566, 348/240.99|
|International Classification||H04N7/14, H04M1/725, H04L12/28|
|Cooperative Classification||G09G2370/02, G06F3/1423, G09G2380/00, H04L65/1083, H04L65/1069, H04M1/656, H04M1/7255, H04N21/231, H04N21/4788, H04L12/2854, H04L65/607, H04N21/41407, H04M3/5315, H04M2201/50, H04N7/155, H04N7/147, H04M1/72555, H04M3/42221, H04M2203/253|
|Jan 18, 2012||AS||Assignment|
Owner name: FOREMOST TECHNOLOGIES LLC, DELAWARE
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OSANN, ROBERT, JR.;REEL/FRAME:027552/0872
Effective date: 20110215
|Dec 31, 2015||AS||Assignment|
Owner name: NYTELL SOFTWARE LLC, DELAWARE
Free format text: MERGER;ASSIGNOR:FOREMOST TECHNOLOGIES LLC;REEL/FRAME:037392/0895
Effective date: 20150826