WO2013057438A1

WO2013057438A1 - Method for the sending and sound reproduction of audio information

Info

Publication number: WO2013057438A1
Application number: PCT/FR2012/052381
Authority: WO
Inventors: Joseph Zirah; Jean-Pierre Richard
Original assignee: Esii
Priority date: 2011-10-20
Filing date: 2012-10-18
Publication date: 2013-04-25
Also published as: FR2981782B1; FR2981782A1

Abstract

The present invention relates to a method for the sending and sound reproduction of audio information, in which: a source (I) emits (i) a sound signal containing audio information and (ii) transmits a video signal; the video signal is sent and reproduced on viewing means in the form of images; the sound signal is captured at a central beacon (2) which, via a wireless communications network within a predetermined area, sends a stream (3) of digital data corresponding to the sound signal; said stream (3) is received at at least one terminal (4) that is personal to a user located within the aforementioned area; the terminal (4) transcribes the stream (3) into at least one transcribed signal using an on-board software application; and the terminal (4) reproduces a sound emission (6) of the transcribed signal. The method is characterised in that: it consists in receiving the sound signal asynchronously before the video signal with a pre-determined time delay; and the user manually synchronises the reproduced sound emission and the images seen using the above-mentioned application. The aim of the invention is preferably to improve the listening and understanding ability of an hearing-impaired or foreign person.

Description

METHOD FOR SENDING SOUND RESTITUTION Ό 'AUDIO INFOIRMM''IONS

The present invention is in the field of communication and information transmission, in particular in sound sending and restoring d ^! information,

To do this, the subject of the invention is a method for sending and audio reproduction of audio information.

The invention will find use preferential application, but in no way limiting, in the sending and sound reproduction for hearing-impaired persons, as well as in the improvement of listening ergonomics during said sound reproduction, in particular through the use of sound. access to a choice of languages according to the user.

The inventio ¹ could advantageously be used in a place for public â receive as a movie theater or a museum.

In addition, the invention is portable, through a computer application embedded on a personal mobile terminal, including cell phone type, personal digital assistant or tablet.

In known manner, during the audio transmission of audio information, a listener may have difficulty understanding, particularly because of the sound context of the transmission and other perceived sounds. This results in a partial loss of information, which can lead to misunderstanding or even misinterpretation.

This phenomenon is increased for some listeners, in the case of a hearing-impaired person. Indeed, it has been shown that below three words on d x perceived, a hearing-impaired person, even when paired, will no longer understand the information transmitted.

This phenomenon can also apply to a foreign person, not speaking at all or very little in the language of the information transmitted.

At present, very few rights for receive from the public have acoustic characteristics to avoid this phenomenon. In the case of cinemas or museums, echoes form parasitic noises, as well as other people with; nearby. Moreover, in the particular case of a cinematographic projection, the musical soundtrack as well as the special effects further parasitize the perception of the essential information, namely the dialogues.

Moreover, in the particular case of people with hearing loss, it has been found that they are much more sensitive to a difference between the image and the perceived sound, especially when it comes to speech.

It is known that if the sound is ahead of the image by more than 20 milliseconds or is over 40 milliseconds behind, it will cause discomfort to a person.

Currently, many systems try to improve the synchronization between the voice and the image, in order to reduce or even eliminate the offset. These known systems use the evolution of the technologies, in particular the computing power, the increase of the flows of the networks, in particular wireless. These evolutions are applied to ensure a real-time reproduction of the audio data with respect to the projected image, which does not mean that they are synchronous with the image,

For example, the document EP 1 531 650 describes a synchronization system of ear-type devices worn by the public. These devices continuously receive a stream of data from a dedicated server for this purpose. The data can come from a soundtrack, in the case of a film, but also from a microphone-type receiver, which captures the voice of a person, such as a speaker. Thus, the sound can be restored in real time.

However, this type of system is always subject to minimal fluctuations in transmission delays, causing shifts it is. so difficult to compensate. A similar solution is described in WO 03/049315. allowing to send through a wireless network a digital data stream, corresponding to a subtitling, to terminals that users view. Once again, the synchronization takes place in real time, without however being free from the risks of lag. In particular, the subtitle poster does not take into account a perfect synchronization, because of their time to display. Another example of such a solution, described in the document WO 2007/046748, consists of a transcription of a given audio signal, like the subtitles, to improve the ergonomics of a worker. hearing impaired.

Another solution, described in document EP 2 015 603, uses a personal terminal for the user, such as his cellular telephone, to transmit and reproduce audio data, with amplification in the case of a hearing-impaired person, or with a translation for someone who does not speak the same language. Once again, a slight offset is observed during the processing of the sound, which may be greater than 40 milliseconds and, thus, cause discomfort when listening "

The present invention solves this problem by proposing a system for sending and sound reproduction of audio information, which has a listening ergonomics adapted to hearing-impaired persons, as well as to a person speaking a different language.

Advantageously, the invention lies precisely in the asynchronous sending of the sound data with respect to the images displayed. In particular, this shift is prev negative, namely that the sound data is transmitted before the projection of images. In short, images are sent with a calculated delay.

The invention has synchronization in two phases: an automatic phase of calculations performed in the terminal to approach a zero offset, and a phase that allows the tilisata, if he wishes, to perfect this adjustment manxielle ent according to his own perception. In short, the invention provides for allowing the user to manually adjust the offset of the transmitted data with respect to the image.

To this end, the subject of the present invention is a method for sending and sound reproduction of audio information, in which:

a source emits, on the one hand, a sound signal containing said audio information and, on the other hand, a video signal;

said video signal is sent and restored on display means in the form of images;

the said sound signal is picked up at a central beacon which sends, via a wireless communication network within a given zone, a digital data stream corresponding to the said sound signal;

said stream is received at at least one personal terminal to a user within said area;

said terminal performs, by means of an on-board software application, a transcription of said stream into at least one transcribed signal;

said terminal renders a sound emission of said transcribed signal, characterized in that it consists of;

asynchronously receiving said sound signal before said video signal with a determined time shift;

and in that

said user performs a manual synchronization between said restored sound emission and said images perceived by means of said application.

Therefore, with this interval of sound advance time, the invention overcomes the offset problems that initially exist between the projection of the image and the sound reproduction, in particular due to the processing of the signal. e moreover, this shift always allows the possibility for the user to make up for this delay between the image and the sound.

According to other additional features, said manual synchronization may consist of increasing or decreasing the emission offset of said sound signal with respect to the amplitude of said images.

Preferably, the restitution can be carried out through an apparatus belonging to each user connected to his terminal.

Advantageously, said transcription may consist of a processing of the sound frequencies of said stream.

According to an embodiment, said user can configure, through said application, processing variables of said stream.

In addition, the reception of said stream at the terminal may consist only of an offset synchronization of data previously recorded on said terminal with said

Thus, from his cell phone, a person can easily retrieve and listen to the sound information essential to his understanding, such as dialogues or a translation during a cinematographic projection, or subtitles.

In addition, this solution offers a simple and inexpensive implementation, through a software application © jR arched on the phone, while ensuring a quality of listening optimized and customizable by everyone.

D ¹ Other features and advantages of the invention rassortiront the detailed description that follows of non-limiting embodiments of the invention, with reference x appended figures, in which:

FIG. 1 diagrammatically represents the architecture implemented by the method according to an example of a preferred embodiment; and

- Figure 2 schematically shows the different steps of transmission of sound and visual data, with the shift according to the invention.

The present invention relates to sending, but above ut the sound reproduction of audio information, offset with visual data, in particular video. In particular, ^Dr. invention aims to improve the sound ergonomics are said émi information.

As preferred examples, which are in no way limitative, the notion of ergonomically sound reproduction can consist in making accessible or improving the quality of audio information for people who are hard of hearing, or translate this information into a different language for users who do not understand. the original language of the said information _*

It will be noted that said audio information will be processed with the aim of obtaining as clear language as possible, thus improving the comprehension of words, expressions and sentences. In addition, the rendering of audio data may also include viewing information in text form, such as subtitles.

On the other hand, a source 1 emits a sound signal 13 containing said audio information 10 and, on the other hand, said source 1 transmits a video signal 11,

The latter is sent and restored on display means 12 in the form of mages. Such means may be a screen, type television, or a surface on which said images are projected, as in the cinema. The transmission and the rendering of said video signal is done in a conventional manner, through known means,

Furthermore, da s a first case, said sound signal 13 can be emitted directly acoustically, output of suitable means, including speakers or loudspeakers. The origin of the signal 13 emitted by the source 1 can then be of many types, in particular the soundtrack of a film. In another case, said sound signal 13 can be transmitted by a communication network, wired or wireless, under fo ma of data files, analog or digital.

Advantageously, said sound signal 13 is picked up at the level of a central beacon 2. Depending on the transmission, acoustic or by a network, said beacon 2 comprises suitable pickup means, reciprocally acoustic sensors typa microphones or means connection to said network.

This beacon 2 sends, through a wireless communication network within a determined zone, a stream 3 of digital data corresponding to said sound signal 13.

Once again, in the case of a direct sound recording, the beacon 2 transcribes the sound signal picked up by its sensors to transcribe it into said stream 3. In the case of a sound signal 13 in the form of files, it may be formatted according to the communication protocols of said network.

Moreover, this capture can be carried out continuously, in particular in the case of a capture by acoustic sensors, but also discontinuously, In the latter case, the audio information is transmitted in advance to said tag 2, which keeps them stored in recording. At the time and periodically during transmission by the source, the tag 2 then synchronizes the information it has with the signal it captures to be said flow 3 <

According to the preferred embodiment, this sending of said stream 3 is carried out within a wireless communication network, in particular by radio wave, for example of the wireless type. Thus, it is possible to broadcast in a targeted manner terr orially said sending and, optionally, to pass it to several different areas, or even to expand the scope and the targeted area, including by means of relays adapted to said communication network . For example, as part of an implementation within a cinema, a tag 2 can equip each room and send said stream 3 inside thereof.

Preferably, said communication network used for this sending may be a wireless local area network, Said beacon 2 then comprises means of communication and wireless connection adapted and compatible with said network.

Once sent to the area, said stream 3 is received at at least one terminal 4, preferably several, Each terminal 4 is located within said area, to ensure the reception of the signal.

According to one embodiment, this reception is carried out continuously and in a manner synchronized with the transmission of said tag 2. The reception of the signal is therefore carried out in "streaming", namely in streaming, in streaming. , in streaming or streaming mode. Streaming is used here to send content in "live" (or slightly delayed).

According to another embodiment, the reception may only consist of a synchronization of data already previously recorded on said terminal. This "duplex" reception is carried out after the user has connected, wired or not, his terminal to a terminal which has transmitted the data, Such a terminal allows in particular to choose options, such as the language information thus downloaded in advance.

In addition, each terminal 4 may be personal to a user preference, said terminal 4 may be a cellular phone, a PDA or a t¾3 <¾5t ¾.o S £ X _*

Advantageously, said terminal 4 performs, by means of an embedded application, a transcription 6 of said stream 3 received in at least one transcribed signal. In other words, said application installed on the terminal 4 receives the stream 3 and performs a processing operation in order to restore it to the user in another form.

In particular, said terminal 4 renders a sound emission 5 of said transcribed signal. Therefore, said terminal 4 is provided with the self an acoustic speaker or loudspeaker. It may also include an electronic connector to connect such a speaker, including the form of earphone, headset or headphones.

Thus, the restitution can be carried out through a device specific to each user and connected to its terminal, It is then possible to connect listening means dedicated to a person niaientendeante, such as for example an induction loop of type collar, hook attached to a prosthesis or hearing implant "

According to a specific characteristic, the method provides for periodically synchronizing the emitted sound signal with the reproduction of the sound emission of the signal transcribed at said terminal 4.

This synchronization can take place via dedicated frames sent regularly to and from said beacon 2 to terminal 4.

Advantageously, said tx-anseription 6 consists of a processing of the sound frequencies of said stream 3.

In particular, said processing of said stream 3 may consist of at least one ergonomic sound adaptation according to said user, in the form of at least, on the one hand, an improvement of the sound quality or, on the other hand, to the less a translation into a different language "

In the first case, the treatment consists of a quality improvement that transforms the sound signal so that it is perceptible and comprehensible to the user, especially a person with hearing loss. It is then possible to intervene on different streams, in order to extract only the voice and the dialogue, to suppress or to attenuate all other parasitic and environmental sounds. In addition, it is possible to modify certain equation ranges, bass, midrange and / or treble, in order to improve the sound reception for the user "

As such, said embedded software application may then include means for selection and modification by the user of sound processing variables, in particular control of a amplification cation or increase or a decrease in the sound volume, and that of all or part of the signal globally or separately, especially for bass, midrange and treble.

XI is then possible to select simply via the application of the filters, in order to adapt the signal in a personal way, according to the listening of each. The user can then modify the sound himself until it suits him.

In the second case of translation, the user can also through said selection means configure the language and thus emit a sound signal at the terminal 4, so that he can understand or interpret the dialogues "Several simultaneous translations can be envisaged.

In particular, said tag 2 can transmit simultaneously and synchronized, an additional flow of translated information.

According to a particular embodiment, the selection means may also include an option for displaying written information, such as subtitles, which will appear, synchronously with said stream 3, on display means equipping said terminal 4. These subtitles are transmitted from said tag 2, synchronously with the sending of the stream difc 3.

In sum, said user configures reception parameters of said stream 3 and said beacon 2 or said application reciprocally sends to said terminal 4 or generates additional data according to said stream 3 and said parameters, said complementary data being displayed on said terminal 4. Depending on the case, the additional data can be directly included within the application or else transmitted from the beacon 2, with a synchronization step in order to avoid the shifts of sounds or sound with respect to these data »

A preferred application is to use the method according to the invention within a cinema, XI is then possible to a user, once his right, input paid and recovered, e¾ select from a terminal or automâtiquesseat to from his countermark, the data related to the film for which he has reserved his place,

The embedded application then retrieves the data or identifiers that will allow the reception of the sound signals emitted by the beacon 2 and the synchronization with the image made by the terminal, once the user in the

After the session, the data received or downloaded beforehand are destroyed by the said application, in a manner that is transparent to the user. Therefore, no legal problem arises in the registration of data protected by copyright.

According to an essential characteristic, said synchronization is effected in a shifted manner in the audio data and the visual data, namely between the sound and the images. For this purpose, the emission of the sound signal is performed asynchronously before said video signal with a given time shift.

It should be noted that the synchronization previously mentioned is understood at the global level, concerning all the audio and video transmissions, over the entire duration of sending the audio and video data, in particular during the entire duration of the film. This synchronization is effected autostatiqueraent. and concerns so the formatting, receiving frames by the terminal in a period of a few milliseconds.

The terminal will perform an automatic synchronization in a time less than that of the asynchronous transmission. The calculation of the synchronization can be modified by the manual adjustment made by the user,

Conversely, the asynchronous transmission is particularly relevant for a specific time period 20 and moindr _f of millisecond (iss), in particular several tens or hundreds of milliseconds, but not more than one second,

According to the preferred embodiment, the lag time 10 of the asynchronous emission shift is of the order of 300 w, preferably 290 me. This setpoint can vary from approximately plus or minus 100 to 500 ms depending on the protocols and the equipment used. "The precise offset is counted in number of images, in 1 / 24th of a second jumps for speed. most common scrolling.

Moreover, this value of 300 ms, preferably 290 ms, is calculated on the fact that, for most of the equipment and particularly according to the models of the terminals, the transmission, calculation and restitution times are much lower and will leave still the possibility of manual adjustment.

In this context, the reception of the sound signal takes place a little before that of the video signal, in particular at a time instant separated from said lapse of time with respect to the instant of transmission of said video stream.

Therefore, at the time of processing to restore the sound data, this lag time offset can compensate for all delays,

This offset therefore also allows the user to very precisely adjust the synchronization between the visualization of the images and the restitution. This adjustment is done manually directly at its terminal 4. Thus, the user can, according to his preferences, adapt himself the sound retransmission to his suitability.

In particular, according to the preferred embodiment, the manual synchronization by the user consists in increasing or decreasing the transmission offset of said sound signal with respect to the perception of said images. As a sum, the user increases or shortens the offset. it feels listening to (or reading) the audio data in relation to the images it displays.

According to the preferred embodiment, the increase or decrease is effected through suitable control means 21, in particular displayed on the terminal. Moreover, they can be provided very precisely, making it possible to increase and decrease the offset. per millisecond or tens of milliseconds.

Thus, the method according to the invention ensures the adaptation d ^! sound information according to the particularities of a user, including a person who is hard of hearing or foreign. Moreover, said user can himself intervene to modulate the sound transcribed and rendered that he hears, in order to obtain optimal personalized listening.

Claims

I »Method of sending and. sound reproduction of audio information, in which;

a source {1} emits, on the one hand, a sound signal containing the said audio information and, on the other hand, a video signal;

said video signal is sent and restored on visualization means in the form of images;

said sound signal is received at a central beacon {2} which sends, via a wireless communication network within a given zone, a stream {3} of digital data corresponding to said sound signal;

said stream {3} is received at at least one personal terminal {4} to a user within said ¾one;

~ said terminal {4} performs, by means of an on-board software application, a transcription of said stream (3) into at least one transcribed signal;

said terminal (4) restores a sound emission {6} of said transcribed signal;

characterized in that it consists of:

- asynchronously receive said sound signal before said video signal with a given time offset?

and in that

2. Method according to claim 1, characterized in that said manual synchronization consists of increasing or decreasing the emission offset of said sound signal with respect to the perception of said images.

3. Method according to any one of the preceding claims, characterized in that the restitution takes place through a device specific to each user connected to its terminal (4).

4. Process according to any one of the preceding claims, characterized in that said transcription {6} consists of a processing of the sound frequencies of said stream {3}.

Method according to claim 3, characterized in that said user configures, via said application, processing variables of said stream (3),

5. Method according to any one of claims 1 to 4, characterized in that the reception of said stream {3} at the terminal (4) consists only in a shifted syschrosinatio of data previously recorded on said terminal (4) with said stream.