|Publication number||US8199942 B2|
|Application number||US 12/099,022|
|Publication date||Jun 12, 2012|
|Filing date||Apr 7, 2008|
|Priority date||Apr 7, 2008|
|Also published as||US20090252355|
|Publication number||099022, 12099022, US 8199942 B2, US 8199942B2, US-B2-8199942, US8199942 B2, US8199942B2|
|Original Assignee||Sony Computer Entertainment Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (30), Referenced by (3), Classifications (11), Legal Events (5)|
|External Links: USPTO, USPTO Assignment, Espacenet|
Embodiments of this invention are related to computer gaming and more specifically to audio headsets used in computer gaming.
Many video game systems make use of a headset for audio communication between a person playing the game and others who can communicate with the player's gaming console over a computer network. Many such headsets can communicate wirelessly with a gaming console. Such headsets typically contain one or more audio speakers to play sounds generated by the game console. Such headsets may also contain a near-field microphone to record user speech for applications such as audio/video (A/V) chat.
A recent development in the field of audio headsets for video game systems is the use of multi-channel sound, e.g., surround sound, to enhance the audio portion of a user's gaming experience. Unfortunately, the massive sound field from the headset tends to cancel out environmental sounds, e.g., speech from others in the room, ringing phones, doorbells and the like. To attract attention, it is often necessary to tap the user on the shoulder or otherwise distract him from the game. The user may then have to remove the headset in order to engage in conversation.
It is within this context that embodiments of the present invention arise.
The teachings of the present invention can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:
Although the following detailed description contains many specific details for the purposes of illustration, anyone of ordinary skill in the art will appreciate that many variations and alterations to the following details are within the scope of the invention. Accordingly, examples of embodiments of the invention described below are set forth without any loss of generality to, and without imposing limitations upon, the claimed invention.
According to an embodiment of the present invention, the disadvantages associated with the prior art may be overcome through the use of targeted sound detection and generation in conjunction with an audio headset. By way of example, the solution to the problem may be understood by referring to the schematic diagram shown in
As used herein, the term “multi-channel audio” refers to a variety of techniques for expanding and enriching the sound of audio playback by recording additional sound channels that can be reproduced on additional speakers. As used herein, the term “surround sound” refers to the application of multi-channel audio to channels “surrounding” the audience (generally some combination of left surround, right surround, and back surround) as opposed to “screen channels” (center, [front] left, and [front] right). Surround sound technology is used in cinema and “home theater” systems, games consoles and PCs, and a growing number of other applications. Consumer surround sound formats include sound on videocassettes, Video DVDs, and HDTV broadcasts encoded as Dolby Pro Logic, Dolby Digital, or DTS. Other surround sound formats include the DVD-Audio (DVD-A) and Super Audio CD (SACD) formats; and MP3 Surround.
Surround sound hardware is mostly used by movie productions and sophisticated video games. However, some consumer camcorders (particularly DVD-R based models from Sony) have surround sound capability either built-in or available as an add-on. Some consumer electronic devices (AV receivers, stereos, and computer soundcards) have digital signal processors or digital audio processors built into them to simulate surround sound from stereo sources.
It is noted that there are many different possible microphone and speaker configurations that are consistent with the above teachings. For example, for a five channel audio signal, the headset may be configured with five speakers instead of two, with each speaker being dedicated to a different channel. The number of channels for sound need not be the same as the number of speakers in the headset. Any number of channels greater than one may be used depending on the particular multi-channel sound format being used.
Examples of suitable multi-channel sound formats include, but are not limited to, stereo, 3.0 Channel Surround (analog matrixed: Dolby Surround), 4.0 Channel Surround (analog matrixed/discrete: Quadraphonic), 4.0 Channel Surround (analog matrixed: Dolby Pro Logic), 5.1 Channel Surround (3-2 Stereo) (analog matrixed: Dolby Pro Logic II), 5.1 Channel Surround (3-2 Stereo) (digital discrete: Dolby Digital, DTS, SDDS), 6.1 Channel Surround (analog matrixed: Dolby Pro Logic IIx), 6.1 Channel Surround (digital partially discrete: Dolby Digital EX), 6.1 Channel Surround (digital discrete: DTS-ES), 7.1 Channel Surround (digital discrete: Dolby Digital Plus, DTS-HD, Dolby TrueHD), 10.2 Channel Surround, 22.2 Channel Surround and Infinite Channel Surround (Ambisonics).
In the multi-channel sound format notation used above, the number before the decimal point in a channel format indicates the number of full range channels and a 1 or 0 after the decimal indicates the presence or absence limited range low frequency effects (LFE) channel. By way of example, if a 5.1 channel surround sound format is used, there are five full range channels plus a limited range LFE channel. By contrast in a 3.0 channel format there are three full range channels and there is no LFE channel.
Each of the earphones includes one or more speakers 106A, 106B. The different signal channels in the multi-channel audio signal 101 are distributed among the speakers 106A, 106B to produce enhanced sound. Normally, this sound would overwhelm any environmental sound. As used herein, the term “environmental sound” refers to sounds, other than source media sounds, generated from sound sources in the environment in which the headset 102 is used. For example, if the headset 102 is used in a room, environment sounds include sounds generated within the room. By way of example, an environmental sound source 108 may be another person in the room or a ringing telephone.
To allow a user to realistically hear targeted sounds from the environmental source 108 the headset 102 includes one or more microphones. In particular, the headset may include far-field microphones 110A, 110B mounted to the earphones 104A, 104B. The microphones 110A, 110B are configured to detect environmental sound and produce microphone signals 111A, 111B in response thereto. By way of example, the microphones 110A, 110B may be positioned and oriented on the earphones 104A, 104B such that they primarily receive sounds originating outside the earphones, even if a user is wearing the headset. By contrast, prior art noise canceling headphones may include microphones within the earphones of a headset. However, in such cases, the microphones are positioned and oriented to detect sounds coming from the speakers within the headphones, particularly if a user is wearing the headset.
In certain embodiments of the invention, the microphones 110A, 110B may be far-field microphones. It is further noted that two or more microphones may be placed in close proximity to each other (e.g., within about two centimeters) in an array located on one of the earphones.
The microphone signals 111A, 111B may be coupled to an environment sound detector 112 that is configured to detect and record sounds originating from the environmental sound source 108. The environmental sound detector 112 may be implemented in hardware or software or some combination of hardware and software. The environmental sound detector 112 may include some sort of sound filtering to remove background noise or other undesired sound. The environmental sound detector produces an environmental sound signal 113.
Where two or more microphones are used, the environmental sound signal 113 may include environmental sound from the microphones 110A, 110B in both earphones. The environmental sound signal 113 may take into account differences in sound intensity arriving at the microphones 110A, 110B. For example, in
In some embodiments, the two microphones 110A, 110B may be mounted on each side of an earphone and structured as two-microphone array. Array beam-forming or maybe simple coherence based sound-detection technology (so called “music” algorithm) may be used to detect the sound and determine the direction from sound source origination to the array geometry center as well.
By way of example, and without loss of generality, the environmental sound signal 113 may be a discrete time domain input signal xm(t) produced from an array of two or more microphones. A listening direction may be determined for the microphone array. The listening direction may be used in a semi-blind source separation to select the finite impulse response filter coefficients b0, b1 . . . , bN to separate out different sound sources from input signal xm(t). One or more fractional delays may optionally be applied to selected input signals xm(t) other than an input signal x0(t) from a reference microphone M0. Each fractional delay may be selected to optimize a signal to noise ratio of a discrete time domain output signal y(t) from the microphone array. The fractional delays may be selected to such that a signal from the reference microphone M0 is first in time relative to signals from the other microphone(s) of the array. A fractional time delay Δ may optionally be introduced into an output signal y(t) so that: y(t+Δ)=x(t+Δ)*b0+x(t−1+Δ)*b1+x(t−2+Δ)*b2+ . . . +x(t−N+Δ)bN, where Δ is between zero and ±1 and b0, b1, b2 . . . bN are finite impulse response filter coefficients.
Fractional delays and semi-blind source separation and other techniques for generating an environmental sound signal to take into account differences in sound intensity due to the different locations of the microphones are described in detail in commonly-assigned US Patent Application publications 20060233389, 20060239471, 20070025562, and 20070260340, the entire contents of which are incorporated herein by reference for all purposes.
A multi-channel sound generator 114 receives the environmental sound signal 113 from the environmental sound detector 112 and generates a multi-channel signal environmental sound signal 115. The multi-channel environmental sound signal 115 is mixed with the source media sound signal 101 from the media device 103. The resulting mixed multi-channel signal 107 is played over the speakers in the headset 102. Thus, environmental sounds from the sound source 108 can be readily perceived by a person wearing the headset and listening to source media sound from the media device 103. The environmental sound reproduced in the headset can have a directional quality resulting from the use of multiple microphones and multi-channel sound generation. Consequently, the headset-wearer could perceive the sound coming from the speakers 106A, 106B as though it originated from the specific location of the sound source 108 in the room as opposed to originating from the media device 103.
It is noted that embodiments of the present invention include the possibility that the headset 102 may have a single far-field microphone. In such a case, the signal from the single microphone may be mixed to all of the channels of a multi-channel source media signal. Although, this may not provide the headset user with a full multi-channel sound experience for the environmental sounds, it does allow the headset user to perceive targeted environmental sounds while still enjoying a multi-channel sound experience for the source media sounds.
According to an alternative embodiment of the present invention, targeted sound detection and generation may be implemented an audio system 300 may be configured as shown in
The headset 301 may include speaker communication interfaces 308A, 308B that allow the speakers to receive source media signals from the source media device 330. The speaker communication interfaces 308A, 308B may be configured to receive signals in digital or analog form from the source media device 330 and convert them into a format that the speakers may convert into audible sounds. Similarly, the headset 301 may include microphone communication interfaces 310A, 310B coupled to the microphones 306A, 306B. The microphone communication interfaces 310A, 310B may be configured to receive digital or analog signals from the microphones 306A, 306B and convert them into a format that can be transmitted to the media device 330. By way of example, any or all of the interfaces 308A, 308B, 310A, 310B may be wireless interfaces, e.g., implemented according to a personal area network standard, such as the Bluetooth standard. Furthermore the functions of the speaker interfaces 308A, 308B and microphone interfaces 310A, 310B may be combined into one or more transceivers coupled to both the speakers and the microphones.
In some embodiments, the headset 301 may include an optional near-field microphone 312, e.g., mounted to the band 303 or one of the earphones 302A, 302B. The near-field microphone may be configured to detect speech from a user of the headset 300, when the user is wearing the headset 301. In some embodiments, the near-field microphone 312 may be mounted to the band 303 or one of the earphones 302B by a stem 313 that is configured to place the near-field microphone in close proximity to the user's mouth. The near-field microphone 312 may transmit signals to the media device 330 via an interface 314.
As used herein, the terms “far-field” and “near-field” generally refer to the sensitivity of microphone sensor, e.g., in terms of the capability of the microphone to generate a signal in response to sound at various sound wave pressures. In general, a near-field microphone is configured to sense average human speech originating in extremely close proximity to the microphone (e.g., within about one foot) but has limited sensitivity to ordinary human speech originating outside of close proximity. By way of example, the near-field microphone 312 may be a −46 dB electro-condenser microphone (ECM) sensor having a range of about 1 foot for average human voice level.
A far-field microphone, by contrast, is generally sensitive to sound wave pressures greater than about −42 dB. For example, the far-field microphones 306A, 306B may be ECM sensors capable of sensing −40 dB sound wave pressure. This corresponds to a range of about 20 feet for average human voice level.
It is noted, there are other types of microphone sensors that are potentially capable of sensing over both the “far-field” and “near-field” ranges. Any sensor may be “far-field” as long as it is capable of sensing small wave pressure, e.g., greater than about −42 db).
The definition of “near-field” is also meant to encompass technology which may use an different approaches to generating a signal in response to human speech generated in close proximity to the sensor. For example, a near-field microphone may use a material that only resonates if sound is incident on it within some narrow range of incident angles. Alternatively, a near-field microphone may detect movement of the bones of the middle ear during speech and re-synthesizes a sound signal from these movements.
The media device may be any suitable device that generates source media sounds. By way of example, the media device 330 may be a television system, home theater system, stereo system, digital video recorder, video cassette recorder, video game console, portable music or video player or handheld video game device. The media device 330 may include an interface 331 (e.g., a wireless transceiver) configured to communicate with the speakers 302A, 302B, the microphones 306A, 306B and 312 via the interfaces 308A, 308B, 310A, 310B and 314. The media device 330 may further include a computer processor 332, and a memory 334 which may both be coupled to the interface 331. The memory may contain software 320 that is executable by the processor 332. The software 320 may implement targeted sound source detection and generation in accordance with embodiments of the present invention as described above. Specifically, the software 320 may include instructions that are configured such that when executed by the processor, cause the system 300 to record environmental sound using one or both far-field microphones 310A, 310B; mix the environmental sound with source media sound from the media device 330 to produce a mixed sound; and play the mixed sound over one or more of the speakers 304A, 304B. The media device 330 may include a mass storage device 338, which may be coupled to the processor and memory. By way of example, the mass storage device may be a hard disk drive, CD-ROM drive, Digital Video Disk drive, Blu-Ray drive, flash memory drive, and the like that can receive media having data encoded therein formatted for generation of the source media sounds by the media device 330. By way of example, such media may include digital video disks, Blu-Ray disks, compact disks, or video game disks. In the particular case of video game disks, at least some of the source media sound signal may be generated as a result a user playing the video game. Video game play may be facilitated by a video game controller 340 and video monitor 342 having speakers 344. The video game controller 340 and video monitor 342 may be coupled to the processor 332 through input/output (I/O) functions 336.
While the above is a complete description of the preferred embodiment of the present invention, it is possible to use various alternatives, modifications and equivalents. Therefore, the scope of the present invention should be determined not with reference to the above description but should, instead, be determined with reference to the appended claims, along with their full scope of equivalents. Any feature described herein, whether preferred or not, may be combined with any other feature described herein, whether preferred or not. In the claims that follow, the indefinite article “A” or “An” refers to a quantity of one or more of the item following the article, except where expressly stated otherwise. The appended claims are not to be interpreted as including means-plus-function limitations, unless such a limitation is explicitly recited in a given claim using the phrase “means for”.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5448637 *||Mar 30, 1995||Sep 5, 1995||Pan Communications, Inc.||Two-way communications earset|
|US5715321 *||Oct 23, 1995||Feb 3, 1998||Andrea Electronics Coporation||Noise cancellation headset for use with stand or worn on ear|
|US5815582 *||Jul 23, 1997||Sep 29, 1998||Noise Cancellation Technologies, Inc.||Active plus selective headset|
|US6771780 *||Apr 22, 2002||Aug 3, 2004||Chi-Lin Hong||Tri-functional dual earphone device|
|US7430300 *||Nov 17, 2003||Sep 30, 2008||Digisenz Llc||Sound production systems and methods for providing sound inside a headgear unit|
|US7512245 *||Feb 4, 2004||Mar 31, 2009||Oticon A/S||Method for detection of own voice activity in a communication device|
|US7545926 *||May 4, 2006||Jun 9, 2009||Sony Computer Entertainment Inc.||Echo and noise cancellation|
|US7903826 *||Mar 8, 2006||Mar 8, 2011||Sony Ericsson Mobile Communications Ab||Headset with ambient sound|
|US20060013409 *||Jul 16, 2004||Jan 19, 2006||Sensimetrics Corporation||Microphone-array processing to generate directional cues in an audio signal|
|US20060083388 *||Oct 18, 2004||Apr 20, 2006||Trust Licensing, Inc.||System and method for selectively switching between a plurality of audio channels|
|US20060204016 *||Apr 28, 2004||Sep 14, 2006||Pham Hong C T||Headphone for spatial sound reproduction|
|US20060233389||May 4, 2006||Oct 19, 2006||Sony Computer Entertainment Inc.||Methods and apparatus for targeted sound detection and characterization|
|US20060239471||May 4, 2006||Oct 26, 2006||Sony Computer Entertainment Inc.||Methods and apparatus for targeted sound detection and characterization|
|US20070025562||May 4, 2006||Feb 1, 2007||Sony Computer Entertainment Inc.||Methods and apparatus for targeted sound detection|
|US20070260340||May 4, 2006||Nov 8, 2007||Sony Computer Entertainment Inc.||Ultra small microphone array|
|US20070274535||May 4, 2006||Nov 29, 2007||Sony Computer Entertainment Inc.||Echo and noise cancellation|
|US20080165988 *||Jan 5, 2007||Jul 10, 2008||Terlizzi Jeffrey J||Audio blending|
|US20080292111 *||Dec 21, 2005||Nov 27, 2008||Comtech, Inc.||Headset for Blocking Noise|
|US20090022343 *||May 29, 2008||Jan 22, 2009||Andy Van Schaack||Binaural Recording For Smart Pen Computing Systems|
|US20090196443 *||Jan 31, 2008||Aug 6, 2009||Merry Electronics Co., Ltd.||Wireless earphone system with hearing aid function|
|US20090196454 *||Jan 31, 2008||Aug 6, 2009||Merry Electronics Co., Ltd.||Earphone set|
|US20090252344 *||Apr 7, 2008||Oct 8, 2009||Sony Computer Entertainment Inc.||Gaming headset and charging method|
|US20090268931 *||Apr 24, 2009||Oct 29, 2009||Douglas Andrea||Headset with integrated stereo array microphone|
|US20100166204 *||Nov 9, 2009||Jul 1, 2010||Victor Company Of Japan, Ltd. A Corporation Of Japan||Headphone set|
|US20100215198 *||Feb 10, 2010||Aug 26, 2010||Ngia Lester S H||Headset assembly with ambient sound control|
|US20100316225 *||Jan 26, 2010||Dec 16, 2010||Kabushiki Kaisha Toshiba||Electro-acoustic conversion apparatus|
|US20110007927 *||Jul 9, 2010||Jan 13, 2011||Atlantic Signal, Llc||Bone conduction communications headset with hearing protection|
|US20110081036 *||Oct 7, 2010||Apr 7, 2011||Wayne Brown||Ballistic headset|
|US20110150248 *||Dec 16, 2010||Jun 23, 2011||Nxp B.V.||Automatic environmental acoustics identification|
|US20110206217 *||Feb 24, 2011||Aug 25, 2011||Gn Netcom A/S||Headset system with microphone for ambient sounds|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US20080165988 *||Jan 5, 2007||Jul 10, 2008||Terlizzi Jeffrey J||Audio blending|
|US20130322667 *||Jun 15, 2012||Dec 5, 2013||GN Store Nord A/S||Personal navigation system with a hearing device|
|US20150104033 *||Apr 28, 2014||Apr 16, 2015||Voyetra Turtle Beach, Inc.||Electronic Headset Accessory|
|U.S. Classification||381/309, 381/375, 381/370|
|International Classification||H04R5/02, H04R5/027, H04R5/033, H04R5/00|
|Cooperative Classification||H04R1/1083, H04R5/04, H04R1/1008|
|Jun 27, 2008||AS||Assignment|
Owner name: SONY COMPUTER ENTERTAINMENT INC., JAPAN
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MAO, XIADONG;REEL/FRAME:021164/0980
Effective date: 20080522
|Dec 26, 2011||AS||Assignment|
Owner name: SONY NETWORK ENTERTAINMENT PLATFORM INC., JAPAN
Free format text: CHANGE OF NAME;ASSIGNOR:SONY COMPUTER ENTERTAINMENT INC.;REEL/FRAME:027446/0001
Effective date: 20100401
|Dec 27, 2011||AS||Assignment|
Owner name: SONY COMPUTER ENTERTAINMENT INC., JAPAN
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SONY NETWORK ENTERTAINMENT PLATFORM INC.;REEL/FRAME:027557/0001
Effective date: 20100401
|Dec 14, 2015||FPAY||Fee payment|
Year of fee payment: 4
|Jul 1, 2016||AS||Assignment|
Owner name: SONY INTERACTIVE ENTERTAINMENT INC., JAPAN
Free format text: CHANGE OF NAME;ASSIGNOR:SONY COMPUTER ENTERTAINMENT INC.;REEL/FRAME:039239/0356
Effective date: 20160401