|Publication number||USRE40688 E1|
|Application number||US 10/006,971|
|Publication date||Mar 31, 2009|
|Filing date||Dec 6, 2001|
|Priority date||Mar 6, 1995|
|Also published as||US6072933|
|Publication number||006971, 10006971, US RE40688 E1, US RE40688E1, US-E1-RE40688, USRE40688 E1, USRE40688E1|
|Original Assignee||Karlgar Limited|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (19), Non-Patent Citations (3), Referenced by (3), Classifications (17), Legal Events (7)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is a continuation-in-part of my application Ser. No. 08/399,013, filed Mar. 6, 1995, now abandoned.
This invention relates to the production of video sequences, in which users own video signal can be combined with a prerecorded video signal to provide a composite sequence, in which prerecorded images are integrated with the user's own material.
So called video karaoke systems are well known in which a video recording is provided, with the video display overlaid by text indicating the words of a song, usually with some form of marker to indicate the synchronization of the words of the song with a musical accompaniment recorded on the sound channel or channels of the video recording. This enables persons viewing the recording to “sing along” with the musical accompaniment.
It is also well known to produce composite video signals by overlaying one signal on another, utilizing one of several keying techniques of which those known as chroma-keying and luminance keying are the most common. In chroma-keying, essential elements of a foreground scene, typically a person or persons, are imaged against a background having a higher level of saturation of a particular colour than is likely to occur in the foreground objects. Typically an intense blue background is utilized, but other colours may be used provided that, in the particular application, they enable the foreground and background to be reliably differentiated by signal processing circuitry. An alternative approach is known as luminance keying, in which it is arranged that the luminance level of the background against which the foreground objects are imaged is consistently and detectably lower than that of the wanted foreground objects. During the combination process, boundaries between the foreground objects and the background are detected on the basis of the above-mentioned difference in colour content or luminance level, so as to produce a switching signal which switches a second video signal, synchronized with the first, into the background areas. These techniques are well known and understood in the art, but require good quality and hence expensive equipment to operate reliably.
In U.S. Pat. No. 5,099,337 (Cury) it is proposed to provide a selection of background audio and video recordings which can be selected from separate libraries and combined with foreground audio and video signals provided by a user, so as to enable the user to provide customized video recordings in which the users own foreground images and audio signals are superimposed upon selected background audio and video signals. This in effect provides system in which the user, as well as providing a foreground audio signal, also provides a foreground video signal, and is provided with means for recording the result. A performer performs in front of a blue screen, so that the performer's image may be chromakeyed into the background video signal, thus providing the illusion that the performer is performing in the selected background. During performance, prompt information is provided to a performer through a prompt monitor from a prerecorded prompt library.
A limitation of such systems is that, by their nature, they can only provide background for a user's performance, and may require the user,s signal to be keyed which, as mentioned above, may be difficult to achieve reliably with consumer quality equipment.
In its broadest aspect, the present invention relates to a system in which a prerecorded video signal is prekeyed to define background areas which, on playback by a user of a recording medium carrying the keyed signal on apparatus configured to recognize the prekeyed background areas, will generate a signal into which may be inserted, in those background areas, a local signal provided by the user, which need not itself by keyed.
The present invention further seeks to provide a system in which a keyed video prerecording is used to provide a prerecorded signal which is combined with a user provided background signal to provide a final combined signal, the prerecording including prompt channel, which can be suppressed in the final combined signal, to assist a user and/or a user's equipment to provide a background signal compatible with the prerecorded signal.
According to the invention, there is provided a system for the production of video signals, comprising a playback device for playing back prerecorded video and audio signals from a prerecorded storage medium, a source of user supplied video and audio signals, a video and audio mixer for combining the prerecorded and user supplied signal to provide combined video and audio outputs, a production monitor connected to the mixer to display to the user the mixed signals, and a storage or reproduction device receiving a mixed video signal output from the mixer, wherein the prerecorded storage medium stores, as well as a video channel and at least one audio channel, at least one prompting channel, the video signals stored on the prerecorded medium being prekeyed to indicate areas to be overlaid in the mixer by the user supplied video signals, and the mixer being operative to convert signals from the prompting channel into production control signals. Typically the production control signals include prompts displayed on the production monitor but absent from the combined video output.
The invention extends to a recording medium providing multiple channels of information, including a video channel, a least one audio channel, and at least one prompting channel, the video channel being recorded with a video signal prekeyed to indicate picture areas available for overlay by a user provided video signal, and the prompting channel including data translatable into instructions for control of the user provided video signal.
In order to permit production of prerecorded tapes (or discs or other media) which will perform reliably with low cost user equipment, I employ by preference a modified luminance keying system in producing the prerecorded tapes. In simple terms, the brightness level of at lease the lowlights of portions of images which are not to be overlaid by the user signal is artificially enhanced so that the “black level” of retained image portions is well above the normal black level of a recorded signal, thus enabling the keyed portion which is at or below the normal black level, to be readily distinguished. The user supplied signals are also brightness enhanced in a similar manner prior to mixing with the signal from the prerecorded tape, and the brightness level of the mixed signal is then returned to normal so as to restore the original black levels.
The word “video signal” as used in this specification, means a television type video signal consisting of a sequence of frames which when reproduced in radial succession are capable of providing a moving picture. It does not include bit-mapped or vector digital representations of a single static image.
In the drawings:
A controllable user video source, usually a camera 8 (or one of multiple selectable cameras) has a signal output to a mixer 12 which combines the video signals by inserting the user signal output 10 in those areas of the prerecorded signal 4 which are identified by the keying, or by mixing the user signal with the prerecorded signal, depending upon the effect desired. The superimposition provided by mixing may be useful for some applications for example training videos. The prompting signals on channel 6 are translated by the mixer 12 into signals displayed on a user monitor 14 so that a user may control the camera 8, or sing (or otherwise perform) along with a prerecorded artist; or the user monitor may implement a camera control function (for example a power zoom control) which is applied directly to the camera to control its input to the mixer 12, which replaces the keyed portions of the signal from the prerecording 2 with the signal 10 from the camera 8. It is however preferred that zooming be performed electronically within the mixer, because of the lack of standardization of camera controls. The signals 4 and 10 must of course be synchronized to a common set of scanning signals before combination. This is preferably achieved as discussed below with reference to
Referring now to
Referring now to
The mixer operates under control of a microcomputer 102 including appropriate working memory. An example of an suitable device is the 87C752 from Intel Corporation, and in general it controls the mixer in a manner similar to that of known video mixers. Accordingly, the mixer will be described primarily with a view to explaining how it differs from conventional digital video mixers, such as the MX-1 digital video mixer from Videonics (although it should be understood that many of the functions provided by such mixers are not essential to the present application and may be omitted to reduce costs), and so as to explain its relationship to the essential functions of the invention.
The video input 4 from the prerecorded source 2 may be either composite or S-video. In the latter case the video chroma and luminance signals are applied to separate analog to digital converters 104, 106 under control of a clock generator 108 which also controls a decoder 110 passing digital Y (luminance) and U and V (chrominance) signals to a frame memory 112 under control of a video memory write controller 114. If a composite video signal is provided, it is applied to the converter 106 and converter 104 is not used.
Similarly, the input 10 from the camera 8 is applied to A/D converters 124 and 126, or converter 124 only if a composite signal, under control of a clock generator 128 also controlling a decoder 130 passing signal to a memory 132 under control of controller 134.
A synchronized memory read controller 140 reads the contents of the memories 112 and 132 in synchronism under control of the microcomputer 102, the Y, U and V signals read from the memories being selected by a multiplexer 142, under control of a luminance or color keyer 144 itself controlled by signals read from the memory 112. When the selected keying signal, be it luminance or chroma, is present at a level denoting background, then the keyer causes the multiplexer 142 to pass at least part of the signals derived from input 10 and block at least part of the signals from input 4; otherwise it passes at least part of the signals derived from input 4 and blocks at least part of the signals derived from input 10. By this means signals from the camera input 10 are inserted into the keyed portions of the video input 4.
The Y (luminance) signal derived from the video input 4 is also passed to the closed caption decoder 100, which decodes the closed caption data in conventional manner to recover data contained therein and pass it to microcomputer 102 which interprets the data and either forwards it to a conventional overlay generator 146 and/or generates signals applied on a line 148 to control zooming (for example) of camera 8, and/or to the scalar 162 or interpolator 160 discussed below, if provided.
The Y, U and V signals from multiplexer 142 are passed to output encoders 150 and 152 under control of an output clock and synchronization separator 158, providing the video outputs 24 and 26 via amplifiers and filters 154 and 156, the V signal to encoder 152 being overlaid by text or video regenerated by the overlay generator 146 so that control instructions are passed to the user monitor 14.
The microcomputer 102 may control additional optional processing circuits between the multiplexer 142 and the encoders 150 and 152, these being of types known in digital video mixers; in this case there is shown a zoom interpolator 160, a scaler 162 controlled by a clock generator 164, and a lower bit switch 166. The zoom interpolator and scaler provide an electronic zoom effect which is preferred to optical zoom controlled by the line 48. The electronic zoom interpolator and scaler will act on the combined signal, and not just the local camera input as would an optical zoom. The lower bit switch 166 can be activated in known manner to provide a posterization effect. The interpolater 160 and scaler 162 may also be configured to be controlled manually by the user, since they act conjointly on the prerecorded and user provided signals.
Audio inputs 32 from local microphones are processed by potentiometer 168, preamplifiers 170, mixer 172, and a master potentiometer 174 before being applied to an A/D converter 176 while audio inputs associated with the prerecorded video signal are applied to A/D converters 278, the outputs from the A/D converters 176 and 178 being combined and optionally processed by a digital signal processor 180 in known manner under control of microcomputer 102. The processed digital audio signals are then passed through digital to analog converters 182 and preamplifiers 184 to the outputs 36 and 38.
The programming of microcomputer 102 has not been described, since except for any processing of the signals from the closed caption decoder 100, it is similar to that for known digital mixers. Processing by microcomputer 102 of closed caption data merely consist of intercepting data encoded on the closed option line 20 which provides control signals for passage by the microcomputer to the output 148 or the processing circuits 160, 162 and 166.
In use, the mixer 12 will operate much like a conventional digital video mixer, except that the keying function controlled by the keying signal in the input 4 is a default function, and control signals or messages in the closed caption field of the input 4 are decoded and output either as video overlays on the monitor 14 or as camera or mixer control signals.
It will therefore be appreciated that a users local generated video (and audio) signals may be combined with the video input from a prerecorded tape or disc to provide video and audio outputs in which user contributed images and sounds are combined with those on the prerecorded tape to provide a composite output in which prerecorded images are inserted into images provided by the user so that for example a famous singing star or animated cartoon character may appear to be performing together with a user in the user's own home or the user's choice of surroundings. The control or prompting channel, for example close captions appearing on the monitor 14, may provide on screen prompts to a user, which are invisible on the output passed to a recorder 16, either in the form of words, or indicators indicating how the user should place a locally generated image on the screen. This channel may also carry data which can be converted by the microcomputer 102 into data output on the line 148 in the form of camera control signals, for example to control of a zoom function of the camera 8. Unlike prior systems, the user does not need to perform against a blue screen or other means to generate keying of the user signal, since the keying is prerecorded into the prerecorded foreground signal. Such prerecorded, prekeyed signals have numerous potential applications of which those discussed above are merely exemplary.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4688105||May 10, 1985||Aug 18, 1987||Bloch Arthur R||Video recording system|
|US4947256||Apr 26, 1989||Aug 7, 1990||The Grass Valley Group, Inc.||Adaptive architecture for video effects|
|US4987552||Feb 8, 1988||Jan 22, 1991||Fumiko Nakamura||Automatic video editing system and method|
|US5016113||Sep 22, 1988||May 14, 1991||Pioneer Electronic Corporation||Apparatus for reproducing and processing picture information including graphic codes from a recording medium|
|US5077610||Jun 7, 1990||Dec 31, 1991||Quantel Limited||Previewing cuts and transformations in an electronic image composition system|
|US5099337||Oct 31, 1989||Mar 24, 1992||Cury Brian L||Method and apparatus for producing customized video recordings|
|US5151793||Aug 30, 1990||Sep 29, 1992||Pioneer Electronic Corporation||Recording medium playing apparatus|
|US5264933||Jan 28, 1992||Nov 23, 1993||Princeton Electronic Billboard, Inc.||Television displays having selected inserted indicia|
|US5416529 *||Jan 14, 1994||May 16, 1995||Immix||Method and system for digital video processing with combined downstream keyer and fade to black mixer|
|US5434678||Jan 11, 1993||Jul 18, 1995||Abecassis; Max||Seamless transmission of non-sequential video segments|
|US5566251||May 31, 1995||Oct 15, 1996||David Sarnoff Research Center, Inc||Video merging employing pattern-key insertion|
|US5625461 *||Jun 7, 1994||Apr 29, 1997||Matsushita Electric Industrial Co., Ltd.||Recording and reproducing apparatus|
|US5625570||Jun 7, 1994||Apr 29, 1997||Technicolor Videocassette, Inc.||Method and system for inserting individualized audio segments into prerecorded video media|
|US5655053||May 13, 1996||Aug 5, 1997||Renievision, Inc.||Personal video capture system including a video camera at a plurality of video locations|
|US5703995||May 17, 1996||Dec 30, 1997||Willbanks; George M.||Method and system for producing a personalized video recording|
|US5872887||Nov 14, 1996||Feb 16, 1999||Gte Laboratories Incorporated||Personal video, and system and method of making same|
|US5923791 *||Jul 29, 1996||Jul 13, 1999||Sarnoff Corporation||Video merging employing pattern-key insertion|
|US6061532||Jan 30, 1997||May 9, 2000||Eastman Kodak Company||Animated image presentations with personalized digitized images|
|US6072537||Jan 6, 1997||Jun 6, 2000||U-R Star Ltd.||Systems for producing personalized video clips|
|1||Mitsunaga, Tomoo, et al., "AutoKey: Human Assisted Key Extraction," Computer Graphics Proceedings, Annual Conference Series, SIGGRAPH 95 Conference Proceedings, ACM SIGGRAPH, Aug. 6-11, 1995, Los Angeles, CA, pp. 265-272.|
|2||Ultimatte According to Bob [online]. Bluescreen According to Bob Kertesz, Jan. 28, 1995 [retrieved on Jun. 1, 2001]. Retrieved from the Internet: <URL:http://www.gregssandbox.com/bs/kertesz.htm>. ( 3 pages).|
|3||Ultimatte Basics-faq's [online]. Ultimatte website [retrieved on Jun. 5, 2001]. Retrieved from the Internet: <URL: http://www.ultimatte.com/basics.html>. (6 pages).|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8077179||Jul 11, 2006||Dec 13, 2011||Pandoodle Corp.||System and method for creating animated video with personalized elements|
|US20070008322 *||Jul 11, 2006||Jan 11, 2007||Ludwigsen David M||System and method for creating animated video with personalized elements|
|US20080170131 *||Sep 10, 2007||Jul 17, 2008||Samsung Electronics Co., Ltd.||Display apparatus and video adjusting method thereof|
|U.S. Classification||386/296, 386/E05.034, 386/E05.069|
|International Classification||G11B27/028, H04N5/93, H04N5/77, G10H1/36, H04N5/91|
|Cooperative Classification||G10H1/368, H04N5/77, G11B27/028, G11B2220/90, H04N5/9305|
|European Classification||G10H1/36K7, H04N5/77, H04N5/93M, G11B27/028|
|Oct 20, 2004||AS||Assignment|
Owner name: COHN, ARTHUR, SWITZERLAND
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SBS INTERACTIVE, CO;REEL/FRAME:015906/0113
Effective date: 20040722
|Nov 16, 2004||AS||Assignment|
Owner name: ARTHUR COHN, SWITZERLAND
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SBS INTERACTIVE, CO.;SBS INTERACTIVE, INC.;REEL/FRAME:015992/0618
Effective date: 20040722
|Jan 16, 2009||AS||Assignment|
Owner name: KARLGAR LIMITED, VIRGIN ISLANDS, BRITISH
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SBS INTERACTIVE, INC.;REEL/FRAME:022117/0544
Effective date: 20090114
|Dec 15, 2009||AS||Assignment|
Owner name: KARLGAR, LTD., VIRGIN ISLANDS, BRITISH
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SBS INTERACTIVE, CO;SBS INTERACTIVE, INC.;REEL/FRAME:023657/0854
Effective date: 20080313
|Dec 6, 2011||FPAY||Fee payment|
Year of fee payment: 12
|Jan 30, 2014||AS||Assignment|
Owner name: AQTIV INTERNATIONAL LTD., VIRGIN ISLANDS, BRITISH
Free format text: CHANGE OF NAME;ASSIGNOR:KARLGAR LTD.;REEL/FRAME:032137/0247
Effective date: 20120904
|Feb 2, 2014||AS||Assignment|
Owner name: DUOMATTE INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AQTIV INTERNATIONAL LTD.;REEL/FRAME:032114/0870
Effective date: 20140131