US6765961B2 - Encoding device mounted on one chip for multiplexing video signal and audio signal - Google Patents
Encoding device mounted on one chip for multiplexing video signal and audio signal Download PDFInfo
- Publication number
- US6765961B2 US6765961B2 US09/814,092 US81409201A US6765961B2 US 6765961 B2 US6765961 B2 US 6765961B2 US 81409201 A US81409201 A US 81409201A US 6765961 B2 US6765961 B2 US 6765961B2
- Authority
- US
- United States
- Prior art keywords
- timing
- processor unit
- encoding
- audio
- encoding device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime, expires
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2368—Multiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
Definitions
- the present invention relates to encoding devices, and more particularly to encoding devices allowing multiplexing of a video signal and an audio signal according to a predetermined compression encoding standard.
- MPEG2 Moving Picture Experts Group
- MPEG2 standard defined in three layers, that is, video, audio and system layers, allows multiplexing of a video signal and an audio signal.
- audio data compression and a process relating to a system do not require an excessively enormous and complicated operation, whereas an operation for video data compression is complicated and the amount of operation is enormous.
- the process relating to the audio and the system is realized through a software process in a processor unit, whereas the process relating to video is realized through a dedicated hardware, which is controlled through a software process in the processor unit.
- Encoding devices performing such multiplexing are, therefore, implemented as multiplexing encoding systems formed as a combination of three devices, that is, an audio compression encoding device and a system multiplexing device both realized with a DSP (Digital Signal Processor), for example, in a software-like manner, and a video compression encoding device including an LSI (Large Scale Integrated Circuit) dedicated for image compression and a processor unit controlling the LSI.
- LSI Large Scale Integrated Circuit
- a multiplexing encoding system 300 of the conventional art includes a video compression encoding device 400 performing a video encoding process on a video signal, an audio compression encoding device 500 performing an audio encoding process on an audio signal and a system multiplexing device 600 performing a system process for multiplexing encoded video data and encoded audio data.
- Video compression encoding device 400 performs a compression encoding of original video data supplied to a video input terminal 402 to generate encoded video data.
- Audio compression encoding device 500 performs a compression encoding of original audio data supplied to an audio input terminal 502 to generate encoded audio data.
- System multiplexing device 600 receives encoded video data and encoded audio data from video compression encoding device 400 and audio compression encoding device 500 , respectively, performs a multiplexing process according to a format defined by a system layer of MPEG2 standard, for example, generates and outputs resulting compressed data via an output terminal 606 .
- the compression encoding process will also be referred to as an encoding process hereinbelow.
- video compression encoding device 400 includes a video encoding unit 420 encoding original video data supplied to video input terminal 402 , a processor unit 410 controlling the encoding process at video encoding unit 420 , an FIFO (First In First Out) buffer 455 receiving encoded video data generated in video encoding unit 420 , a video timing generation unit 430 generating a timing signal for controlling a timing of execution of the video encoding process and an internal bus 450 .
- the encoded video data supplied to FIFO buffer 455 is output via a video output terminal 406 .
- Processor unit 410 is connected to video encoding unit 420 via internal bus 450 .
- an image compression LSI which is a dedicated hardware, is employed, for example.
- Video encoding unit 420 includes a register 422 to hold a control parameter employed in the video encoding process.
- the control parameter held by register 422 is set through processor unit 410 via internal bus 450 .
- the timing signals generated from video timing generation unit 430 include a picture timing signal PSYNC indicating an activation timing of a picture-related process and a macro block timing signal MBSYNC indicating an activation timing of a macro block-related process (hereinafter referred to also as MB process) in MPEG2 standard, for example.
- FIG. 14 shows a timing chart referenced for describing an execution timing of the video encoding process.
- the picture-related process is executed in response to the generation of picture timing signal PSYNC and the MB process is executed in response to the generation of macro block timing signal MBSYNC.
- the number of frames n (n is a natural number) included in a second is determined and a cycle of the generation of picture timing signal PSYNC is 1/n second. For example, when thirty frames are included in a second, the cycle of picture timing signal PSYNC is 1/30 second.
- the video compression encoding is executed with a one-frame image divided into macro blocks (hereinafter referred to simply as MB) serving as a unit of processing, of sixteen pixels ⁇ sixteen lines. If an image of one frame is formed from 720 pixels in a vertical direction ⁇ 480 lines in a horizontal direction, one frame includes 1350 MB's. Then, video timing generation unit 430 generates 1350 macro block timing signals MBSYNC during one cycle of picture timing signal PSYNC.
- MB macro blocks
- Video encoding section 420 operates in response to timing signals PSYNC and MBSYNC.
- video encoding unit 420 executes the MB process for each MB in response to macro block timing signal MBSYNC and executes the picture-related process which is common to all MB included in one frame in response to picture timing signal PSYNC.
- a necessary control parameter for the picture-related process and the macro block-related process is set in register 422 through processor unit 410 .
- Processor unit 410 hence, must operate in synchronization with timing signals PSYNC and MBSYNC.
- processor unit 410 can execute an operation corresponding to the picture-related process and the macro block-related process as an interrupt process in synchronization with video encoding unit 420 .
- audio compression encoding device 500 includes an FIFO buffer 532 receiving original audio data supplied to audio input terminal 502 , a processor unit 510 including a software for executing an encoding process on the original audio data, an FIFO buffer 534 receiving encoded audio data supplied from processor unit 510 , an audio output terminal 505 supplying the encoded audio data supplied from FIFO buffer 534 , and an internal bus 550 .
- Processor unit 510 is connected to FIFO buffers 532 and 534 via internal bus 550 .
- Processor unit 510 reads the original audio data from FIFO buffer 532 via internal bus 550 .
- Processor unit 510 encodes the read original audio data according to an internally stored program to generate encoded audio data.
- the encoded audio data is written into FIFO buffer 534 via internal bus 550 .
- the encoded audio data supplied to FIFO buffer 534 is output from audio output terminal 505 .
- the activation of the program for executing the audio encoding process in processor unit 510 is controlled through an audio timing generation unit 520 .
- the audio encoding process in processor unit 510 is activated in response to generation of a timing signal FSYNC.
- a sampling frequency of 48 kHz according to a layer II of the MPEG1 audio standard is employed as a mode for the audio encoding process, a frame defined by 1152 samples will be processed as one unit.
- audio timing generation unit 520 generates timing signal FSYNC at a cycle of 24 msec and supplies the signal to the interrupt terminal of processor unit 510 .
- processor unit 510 activates a software for performing the audio compression encoding and executes the audio encoding process for each frame.
- system multiplexing device 600 includes an encoded video data input terminal 602 receiving an input of encoded video data, an FIFO buffer 652 provided corresponding to the encoded video data, an encoded audio data input terminal 604 receiving encoded audio data, an FIFO buffer 654 provided corresponding to the encoded audio data, a processor unit 610 controlling the multiplexing process, a header memory 640 holding header data of multiplexed compressed data stream and an internal bus 650 .
- System multiplexing device 600 further includes a multiplexing control unit 620 receiving header data output from header memory 640 , encoded video data supplied from FIFO buffer 652 and encoded audio data from FIFO buffer 654 , and selectively transferring one type of data to FIFO buffer 656 , a compressed data output terminal 606 supplying the compressed data stream supplied from FIFO buffer 656 , and a status generation unit 630 generating a timing signal BSOREQ for requesting a supply of new compressed data according to data amount in FIFO buffer 656 .
- a multiplexing control unit 620 receiving header data output from header memory 640 , encoded video data supplied from FIFO buffer 652 and encoded audio data from FIFO buffer 654 , and selectively transferring one type of data to FIFO buffer 656 , a compressed data output terminal 606 supplying the compressed data stream supplied from FIFO buffer 656 , and a status generation unit 630 generating a timing signal BSOREQ for requesting a supply of new compressed
- Processor unit 610 generates and supplies header data to header memory 640 via internal bus 650 .
- processor unit 610 sets via internal bus 650 a control parameter in a register 625 , which parameter is provided in multiplexing control unit 620 , for designating a selective transfer of data in multiplexing control unit 620 .
- TS transport stream
- data is sequentially transferred to an FIFO buffer 656 in the order of 4-byte header data, 184-byte compressed (encoded) video data, 4-byte header data, and 184-byte compressed (encoded) audio data, thus encoded video data and encoded audio data are multiplexed and compressed data thereof can be generated.
- compressed data output terminal 606 is connected to a transmission line.
- compressed data must be output at a constant rate.
- the amount of data held in FIFO buffer 656 is, therefore, monitored through status generation unit 630 , which generates and supplies timing signal BSOREQ to the interrupt terminal of processor unit 610 when the empty area of FIFO buffer 656 becomes larger than 188 bytes which corresponds to a constituent unit of the TS data, for example.
- processor unit 610 indicates to multiplexing control unit 620 to transfer new multiplexed compressed data to FIFO buffer 656 .
- An encoding system allowing the multiplexing of the audio signal and the video signal can be thus provided, with the video compression device employing the image compression LSI which is a dedicated hardware device, the audio compression encoding device and the system multiplexing device both executing a software process corresponding to a program stored in the processor unit.
- the video compression device employing the image compression LSI which is a dedicated hardware device
- the audio compression encoding device and the system multiplexing device both executing a software process corresponding to a program stored in the processor unit.
- the video encoding process, the audio encoding process and the system process must be executed independent from each other at a predetermined cycle as described above.
- the control of the order of process execution becomes important when activation requests for the plurality of processes are made such that their execution overlap.
- An object of the present invention is to provide an encoding device mountable on one LSI chip, for multiplexing a video signal and an audio signal while effectively sharing resources for different processes.
- the present invention is an encoding device multiplexing an input video signal and an input audio signal according to a predetermined compression encoding standard, which encoding device includes: a processor unit, a timing control unit, a first buffer, a video encoding process unit, a second buffer, a third buffer and a multiplex process unit.
- the processor unit has a function of executing an audio encoding process to convert the audio signal to encoded audio data, a function of executing a first control process for controlling a video encoding process and a function of executing a second control process for controlling a system process.
- the timing control unit generates at least one first timing signal for activating the video encoding process, a second timing signal for activating the audio encoding process and a third timing signal for activating the system process.
- the first buffer holds the encoded audio data temporarily.
- the video encoding process unit is activated in response to the first timing signal and executes the video encoding process to convert the video signal to encoded video data.
- the second buffer holds the encoded video data temporarily.
- the third buffer holds header data generated from the processor unit.
- the multiplex process unit executes the system process according to an indication from the processor unit and sequentially outputs the encoded video data, the encoded audio data and the header data supplied from first, second and third buffers, respectively, by an amount and in an order according to a system process parameter.
- the processor unit includes an interrupt control circuit receiving at least one of first, second and third timing signals to designate for an interrupt process one of the audio compression process, first and second control processes in response to generation of each timing signal supplied as an input.
- the processor unit preferentially executes the interrupt process.
- a main advantage of the present invention lies in that mounting of the encoding device on a single LSI chip is allowed as the processor unit is shared for the video encoding process, the audio encoding process and the system process.
- FIG. 1 is a block diagram showing an overall structure of an encoding device 100 according to the first embodiment of the present invention
- FIG. 2 is a timing chart referenced for describing an interrupt process in processor unit 10 ;
- FIGS. 3 A ⁇ 3 D are flow charts of programs of respective interrupt processes in processor unit 10 ;
- FIG. 4 is an overall block diagram showing a structure of an encoding device 110 according to the second embodiment of the present invention.
- FIG. 5 is a flow chart of a main process program in a processor unit 10 in an encoding device 110 ;
- FIG. 6 is an overall block diagram showing a structure of an encoding device 120 according to the third embodiment of the present invention.
- FIGS. 7A and 7B are flow charts of a video encoding process in an encoding device 120 ;
- FIG. 8 is a timing chart of a timing signal relating to an audio encoding process
- FIG. 9 is a flow chart of a main program in a processor unit 10 according to the fourth embodiment of the present invention.
- FIG. 10 is a block diagram showing a structure of an encoding device 130 according to the fifth embodiment of the present invention.
- FIG. 11 is a block diagram showing a structure of a timing control unit 140 ;
- FIG. 12 is a schematic block diagram showing an overall structure of a multiplexing encoding system 300 according to a conventional art
- FIG. 13 is a block diagram showing a structure of a video compression encoding device 400 ;
- FIG. 14 is a timing chart referenced for describing an execution timing of a video encoding process
- FIG. 15 is a block diagram showing a structure of an audio compression encoding device 500 ;
- FIG. 16 is a timing chart referenced for describing an execution timing of an audio encoding process.
- FIG. 17 is a block diagram showing a structure of a system multiplexing device 600 .
- FIG. 1 shows an overall structure of an encoding device 100 according to the first embodiment of the present invention.
- the encoding device of the present invention is not intended only for the compression encoding process in compliance with MPEG2 standard and can be applied widely to encoding devices for multiplexing a video signal and an audio signal.
- encoding device 100 which can be mounted on one semiconductor substrate CH, includes an audio input terminal 2 receiving an input of original audio data, a video input terminal 4 receiving an input of original video data and a compressed data output terminal 6 supplying compressed data generated by multiplexing encoded video data and encoded audio data.
- Encoding device 100 further includes an FIFO buffer 62 receiving original audio data, a processor unit 10 having a software for executing a compression encoding process on the original audio data (hereinafter also referred to simply as audio encoding process) and controlling an overall operation of encoding device 100 , an FIFO buffer 64 receiving encoded audio data generated by compression encoding in processor unit 10 , via an internal bus 50 , a video encoding unit 20 executing a compression encoding process of original video data (hereinafter also referred to simply as video encoding process), an FIFO buffer 66 receiving encoded video data and a header memory 70 holding header data of a compressed data stream.
- a processor unit 10 having a software for executing a compression encoding process on the original audio data (hereinafter also referred to simply as audio encoding process) and controlling an overall operation of encoding device 100
- an FIFO buffer 64 receiving encoded audio data generated by compression encoding in processor unit 10 , via an internal bus 50
- Encoding device 100 further includes a multiplexing control unit 30 receiving header data supplied from header memory 70 , encoded video data supplied from FIFO buffer 66 and encoded audio data from FIFO buffer 64 , and executing multiplexing process (hereinafter also referred to as system process) for supplying data of a type and amount according to a control parameter set in a register 32 , an FIFO buffer 68 receiving data supplied from multiplexing control unit 30 and supplying the data to compressed data output terminal 6 at a constant rate and a timing control unit 40 generating a timing signal according to data amount in FIFO buffer 68 , for activating the audio encoding process, the video encoding process and the system process.
- a multiplexing control unit 30 receiving header data supplied from header memory 70 , encoded video data supplied from FIFO buffer 66 and encoded audio data from FIFO buffer 64 , and executing multiplexing process (hereinafter also referred to as system process) for supplying data of a type and amount according to a control parameter set in
- Timing control unit 40 generates an audio timing signal FSYNC for activating the audio compression encoding process, picture timing signal PSYNC and macro block timing signal MBSYNC for activating the video compression encoding process and timing signal BSOREQ for requesting a transfer of a multiplex stream from multiplexing control unit 30 according to filled data amount of FIFO buffer 68 .
- original video data is compressed and encoded in video encoding unit 20 and supplied to multiplexing control unit 30 via FIFO buffer 66 .
- the original audio data is supplied to processor unit 10 via FIFO buffer 62 and internal bus 50 .
- Processor unit 10 performs compression encoding of the original audio data and supplies encoded audio data to FIFO buffer 64 .
- the encoded audio data is supplied to multiplexing control unit 30 from FIFO buffer 64 .
- Processor unit 10 generates and supplies header data to header memory 70 via internal bus 50 .
- Header memory 70 supplies header data to multiplexing control unit 30 .
- Multiplexing control unit 30 processes the encoded audio data from FIFO buffer 64 , the encoded video data from FIFO buffer 66 and header data from header memory 70 according to the control parameters held in register 32 and selectively transfers one type of data to FIFO buffer 68 .
- Multiplexed compressed data stream thus supplied to FIFO buffer 68 is then supplied to compressed data output terminal 6 at a constant rate.
- the video encoding process in video encoding unit 20 is executed in response to macro block timing signal MBSYNC and picture timing signal PSYNC generated from timing control unit 40 according to the cycles of macro block-related process and picture-related process, respectively.
- video encoding unit 20 executes the MB process for each MB in response to macro block timing signal MBSYNC and executes the picture-related process common to all MB included in the same frame in response to picture timing signal PSYNC.
- processor unit 10 Necessary control parameters for picture-related process and macro block-related process are set in a register 22 through processor unit 10 . Hence, the operation of processor unit 10 must be in synchronization with timing signals PSYNC and MBSYNC.
- the process at multiplexing control unit 30 is also controlled through processor unit 10 via internal bus 50 and register 32 .
- Each of a control process for controlling the audio encoding process and the video encoding process in video encoding unit 20 and a control process for controlling the system process in multiplexing control unit 30 is executed as an interrupt process through processor unit 10 .
- Processor unit 10 receives a timing signal generated from timing control unit 40 as an interrupt signal.
- Processor unit 10 includes an interrupt control circuit 15 .
- Interrupt control circuit 15 determines, based on a plurality of interrupt signals supplied from timing control unit 40 and according to a predetermined priority, which interrupt process is to be executed.
- Processor unit 10 suspends a process which has been executed before the assertion of the interrupt process and preferentially executes the determined interrupt process.
- encoding device 100 is characterized in that encoding device 100 employs a single processor unit 10 for the video encoding process, the audio encoding process and the system process and hence can be formed on single LSI chip CH.
- FIG. 2 is a timing chart referenced for describing an interrupt process in processor unit 10 .
- processes are preferentially performed in the order of the picture-related process, the macro block-related process, and a multiplex stream supplying process. Such priority may be determined in advance based on a necessary length of the cycles for these processes.
- the video signal it is assumed that thirty frames are processed in a second, which one frame is constituted from 1350 macro blocks.
- a sampling frequency of 48 kHz of MPEG1 audio standard layer II is employed and the process unit is defined as 1152 samples.
- the output rate of compressed data stream output from compressed data output terminal 6 is assumed to be 20 Mbps.
- the generation cycle Tps of picture timing signal PSYNC for activating the picture-related process can be given by
- Tmb Tps /1350 32 24.7 ⁇ sec.
- Tbs 1/(20 Mpbs/8 bit/188 byte) ⁇ 75 ⁇ sec.
- the priority of the interrupt by each process may be determined based on the generation cycles of these timing signals and so on.
- the priority of timing signals is in the order of MBSYNC, PSYNC, BSOREQ and FSYNC.
- Interrupt control circuit 15 does not accept an interrupt during an execution of a certain interrupt process if the generated interrupt is of a lower priority but accepts the interrupt if the priority of the generated interrupt process is higher than the one under way.
- the video picture-related process is activated in response to picture timing signal PSYNC generated at a time t 0 .
- timing signal BSOREQ is generated.
- the system process is not executed immediately and starts from time t 2 at which the picture-related is completed.
- a timing signal MBSYNC of a higher priority is generated at time t 3 . Accordingly, the system process is suspended and the MB process is preferentially executed. The system process is resumed at time t 4 at which the MB process completes.
- interrupt control circuit 15 determines which interrupt process is to be executed according to a predetermined priority.
- FIG. 3A is a flow chart of a picture-related interrupt process in processor unit 10 .
- the interrupt process for the picture-related process is started with the generation of picture timing signal PSYNC (step S 100 ).
- a context save is executed to hold an internal state of processor unit 10 at the point (step S 102 ).
- a video picture-related process is executed, such as setting of the control parameters for register 22 in video encoding unit 20 (step S 104 ).
- a context return is executed to recover the internal state before the interrupt process (step S 106 ).
- processor unit 10 returns to a main process and resumes the normal process (step S 150 ).
- FIG. 3B shows an execution flow chart of a macro block-related interrupt process in processor unit 10 .
- the interrupt process for the MB process is started with the generation of macro block timing signal MBSYNC (step S 110 ).
- a context save is executed to hold an internal state of processor unit 10 at the point (step S 112 ).
- the video MB process is executed, such as setting of the control parameters for register 22 in video encoding unit 20 (step S 114 ).
- a context return is executed to recover the internal state before the interrupt process (step S 116 ).
- processor unit 10 returns to a main process and resumes a normal process (step S 150 ).
- FIG. 3C is an execution flow chart of a system interrupt process in processor unit 10 .
- the interrupt process for the system process is started with the generation of timing signal BSOREQ (step S 120 ).
- a context save is executed to hold an internal state of processor unit 10 at the point (step S 122 ).
- the system process is executed, such as setting of the control parameters for register 32 in multiplexing control unit 30 (step S 124 ).
- a context return is executed to recover the internal state before the interrupt process (step S 126 ).
- processor unit 10 returns to a main process and resumes a normal process (step S 150 ).
- FIG. 3D is an execution flow chart of an audio encoding interrupt process in processor unit 10 .
- the interrupt process for the audio encoding process is started with the generation of timing signal FSYNC (step S 130 ).
- a context save is executed to hold an internal state of processor unit 10 at the point (step S 132 ).
- a software for performing the audio encoding process is activated and the audio process is executed (step S 134 ).
- a context return is executed to recover the internal state before the interrupt process (step S 136 ).
- processor unit 10 returns to a main process and resumes a normal process (step S 150 ).
- each interrupt process program is written such that an internal state of processor unit 10 is temporarily held through the context save at the execution of each interrupt process and the internal state is recovered after the completion of the interrupt process to return to the main process.
- the video process, the audio process and the system process thus can sequentially be activated as an interrupt process according to the priority.
- the audio process and the system process are activated as interrupt processes in processor unit 10 in encoding device 100 according to the first embodiment, a single processor unit can be shared for the video process, the audio process and the system process. Thus unnecessary cost can be reduced.
- interrupt control circuit 15 must be provided in processor unit 10 , which leads to a problem of increase in the hardware.
- all processes are executed as the interrupt process, during an execution of one interrupt process, another interrupt process which should be performed preferentially may be requested.
- the audio encoding can be interrupted by the system process, which can further be interrupted by the macro block-related process.
- the process being executed when the interrupt is requested must be suspended and the internal state of the processor unit must be held through the context save each time the interrupt is requested.
- a program for these operations is necessary which further leads to the increase in the number of process cycles.
- the second embodiment is characterized in that the audio process is not activated as an interrupt process by the timing signal but is performed as a part of the main process in processor unit 10 .
- an encoding device 110 according to the second embodiment of the present invention is different from encoding device 100 according to the first embodiment in that encoding device 110 includes a status register 45 in timing control unit 40 , for indicating the state of audio timing signal FSYNC and that processor unit 10 does not execute the interrupt process activated by audio timing signal FSYNC.
- processor unit 10 can read a value in status register 45 via internal bus 50 . By monitoring the state of status register 45 , processor unit 10 can execute the audio encoding process in the main program process at a timing requested from timing control unit 40 .
- FIG. 5 is a flow chart of a main process program of processor unit 10 in encoding device 110 .
- step S 200 data reading from status register 45 is executed (step S 210 ) to check whether a value of status register 45 is in a state indicating the execution of the audio encoding process (step S 220 ).
- an audio encoding process is activated (step S 230 ).
- reading of status register 45 is executed again (step S 210 ).
- the video process and the system process are executed as an interrupt to the main program, which interrupt is responsive to the generation of timing signals PSYNC, MBSYNC and BSOREQ.
- the number of interrupt levels can be reduced from three to two and the amount of hardware structure of interrupt control circuit 115 can be decreased. Further, with the reduction of the number of context process activation, process time can be reduced.
- the audio encoding process is activated through polling. Based on the condition of the encoding and so on, the system process instead of audio encoding can be activated through the polling.
- an encoding device 120 according to the third embodiment of the present invention is different from encoding device 110 according to the second embodiment in that timing control unit 40 further includes a status register 145 in addition to status register 45 and that timing signal BSOREQ is not supplied to processor unit 10 from the timing control unit, which timing signal is used for interrupt request relating to the system process.
- Status register 145 indicates a state of timing signal BSOREQ, for executing the activation of the system process also through polling.
- the content of status register 145 can be read through processor unit 10 via internal bus 50 .
- status register 45 indicates the state of audio timing signal FSYNC.
- encoding device 120 though the polling relating to the audio encoding process is executed in the main process of processor unit 10 as described with reference to FIG. 5, a polling relating to the system process must be executed in the interrupt process program relating to the video encoding process.
- FIG. 7A is an execution flow chart of a picture-related interrupt process in encoding device 120 .
- step S 160 when the interrupt process for the picture-related process is started with the generation of picture timing signal PSYNC (step S 160 ), a context save is executed to hold an internal state of the processor unit at the point (step S 162 ) followed by the video picture-related process (step S 164 ).
- status read of status register 145 is executed (step S 166 ) and a signal level of timing signal BSOREQ is checked to confirm whether the system process is requested or not (step S 168 ). If timing signal BSOREQ has been generated to request the system process, the system process is executed (step S 170 ). After the completion of the system process, a context return is executed to recover the internal state of the processor unit before the interrupt process (step S 172 ).
- timing signal BSOREQ has not been generated and the execution of the system process is not requested, a context return is executed without the execution of the system process (step S 172 ). Then, the internal state of the processor unit before the video picture-related is recovered through the context return and, the process returns to the main process and the processor 10 resumes the main process (step S 200 ).
- FIG. 7B is an execution flow chart of a macro block-related interrupt process in encoding device 120 .
- step S 180 when the interrupt process for the video MB process is started with the generation of macro block timing signal MBSYNC (step S 180 ), a context save is executed to hold an internal state of the processor unit at the point (step S 182 ) followed by the video MB process (step S 184 ).
- step S 186 status read of status register 145 is executed (step S 186 ) and a signal level of timing signal BSOREQ is checked to confirm whether the system process is requested or not (step S 188 ). If timing signal BSOREQ has been generated to request the system process, the system process is executed (step S 190 ). After the completion of the system process, a context return is executed to recover the internal state of the processor unit before the interrupt process (step S 192 ).
- timing signal BSOREQ has not been generated and the execution of the system process is not requested, a context return is executed without the execution of the system process (step S 192 ). Then, the internal state of the processor unit before the video MB process is recovered through the context return and, the process returns to the main process and the processor 10 resumes the main process (step S 200 ).
- the interrupt to processor unit 10 With both audio encoding process and the system process being activated through the polling, the interrupt to processor unit 10 becomes only one level interrupt, and it becomes unnecessary to provide a hardware structure corresponding to interrupt control circuits 15 and 115 determining the priority of the interrupt processes. In addition, because the interrupt is of only one level, it becomes relatively easy to reproduce a defective state when the defect is found in encoding device 120 . Then, it becomes easier to debug and the development time can be reduced.
- the polling relating to the audio encoding process is executed in the main process of processor unit 10 and, in the third embodiment, the polling relating to the system process is executed in the interrupt process of processor unit 10 . Yet, it is possible to execute polling relating to the system process and the audio encoding respectively during the main process and the interrupt process.
- a check signal FSYNCend additionally is provided for checking the secure ending in an audio cycle Tfs of the audio process corresponding to audio timing signal FSYNC generated to have audio cycle Tfs.
- Check signal FSYNCend is generated a time period Tf prior to the end of audio cycle Tfs.
- Time period Tf may be set corresponding to a time period required for the forced termination of the audio process.
- Secure operation of the encoding devices is intended by providing a structure, in which check signal FSYNCend is reflected to the status register in timing control unit 40 and checked in the main program.
- FIG. 9 is a flow chart of a main program of processor unit 10 according to the fourth embodiment.
- step S 210 while the main process is executed (step S 210 ), the reading of the status register is performed (step S 220 ), and whether the audio encoding process is requested or not is checked based on a state of audio timing signal FSYC (step S 230 ).
- the audio process is divided into a plurality of process units.
- an audio encoding process in one cycle is divided into two process units, that is, an audio process unit 1 and an audio process unit 2 .
- step S 240 In response to the activation of audio timing signal FSYNC, first divided unit, that is, audio process unit 1 is executed (step S 240 ). After the completion of audio process unit 1 , reading of the status register is executed again (step S 250 ), to check the state of check signal FSYNCend (step S 260 ).
- check signal FSYNCend is generated at this timing, the audio process is forced to terminate without the execution of remaining audio process 2 to prevent the failure of the operation of the encoding device as a whole (step S 280 ).
- check signal FSYNCend is not generated, the remaining audio process 2 is executed.
- the audio process is divided into two process units as an example in FIG. 9, the audio process can be divided into any number of process units.
- a similar check signal can be provided for timing signals other than the audio timing signal.
- the video encoding process, the audio encoding process and the system process can securely end in a predetermined cycle, thus the failure of the encoding device as a whole can be prevented.
- the video encoding process, the audio encoding process and the system process are activated in response to the generation of corresponding timing signals, respectively.
- the internal state of the video encoding unit after the activation needs to be checked in processor unit 10 , in a case, for example, where setting of a quantization parameter which is one of the control parameters for the video encoding process is desired to be performed based on an average data value of the macro block-related process results.
- a structure must be formed such that an interrupt process can be started through processor unit 10 after the elapse of a predetermined time period from the activation of each process using delay signals of respective timing signals.
- FIG. 10 is a block diagram showing a structure of an encoding device 130 according to the fifth embodiment of the present invention.
- a signal MBSYNCdel is generated from a timing control unit 140 as a delay signal corresponding to macro block timing signal MBSYNC.
- Timing control unit 140 relating to the macro block-related process generates macro block timing signal MBSYNC and a delay signal thereof MBSYNCdel. Timing control unit 140 sends timing signal MBSYNC to video encoding unit 22 and supplies delay signal MBSYNCdel of timing signal MBSYNC to processor unit 10 . Thus, processor unit 10 activates an interrupt operation relating to the video encoding process after the elapse of a predetermined time period from the activation of the macro block-related process to set an encoding parameter such as a quantization parameter in register 22 .
- timing control unit 140 further includes a delay circuit 142 generating a delay signal, in addition to the structure of timing control unit 40 .
- Delay circuit 142 generates a delay timing signal MBSYNCdel which is a delay signal of timing signal MBSYNC by a delay time set in a register 144 .
- processor unit 10 can set a delay time to be set in register 144 via internal bus 50 to any value.
- a delay timing signal with any delay time can be generated corresponding to each timing signal indicating an activation timing of one of the video process, the audio process and the system process.
- the interrupt process can be executed at any timing through processor unit 10 .
Abstract
Description
Claims (15)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2000-115573(P) | 2000-04-17 | ||
JP2001-115573 | 2000-04-17 | ||
JP2000115573A JP2001309372A (en) | 2000-04-17 | 2000-04-17 | Encoder |
Publications (2)
Publication Number | Publication Date |
---|---|
US20010031008A1 US20010031008A1 (en) | 2001-10-18 |
US6765961B2 true US6765961B2 (en) | 2004-07-20 |
Family
ID=18627186
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/814,092 Expired - Lifetime US6765961B2 (en) | 2000-04-17 | 2001-03-22 | Encoding device mounted on one chip for multiplexing video signal and audio signal |
Country Status (2)
Country | Link |
---|---|
US (1) | US6765961B2 (en) |
JP (1) | JP2001309372A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020126697A1 (en) * | 2001-03-07 | 2002-09-12 | Jost Arthur P. | Methods and apparatus for reconfiguring protocol data when reducing multiplexed data streams |
US20020126702A1 (en) * | 2001-03-07 | 2002-09-12 | Vince Lawrence D. | Methods and apparatus for constructing a program association table when reducing multiplexed data streams |
US20070046817A1 (en) * | 2005-08-24 | 2007-03-01 | Yoriko Yagi | Processor, processing method and processing program |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4962400B2 (en) * | 2008-04-30 | 2012-06-27 | ソニー株式会社 | Arithmetic decoding device |
US20120311070A1 (en) * | 2011-05-31 | 2012-12-06 | Fanhattan Llc | Intelligent application adapted to multiple devices |
US9276989B2 (en) | 2012-03-30 | 2016-03-01 | Adobe Systems Incorporated | Buffering in HTTP streaming client |
US9602827B2 (en) * | 2012-07-02 | 2017-03-21 | Qualcomm Incorporated | Video parameter set including an offset syntax element |
US9485508B2 (en) | 2013-04-08 | 2016-11-01 | Qualcomm Incorporated | Non-entropy encoded set of profile, tier, and level syntax structures |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5488385A (en) * | 1994-03-03 | 1996-01-30 | Trident Microsystems, Inc. | Multiple concurrent display system |
JPH11177436A (en) | 1997-12-12 | 1999-07-02 | Kokusai Electric Co Ltd | Data communication equipment |
JPH11225333A (en) | 1998-02-06 | 1999-08-17 | Toshiba Corp | Decode processing circuit |
US6449311B1 (en) * | 1997-03-13 | 2002-09-10 | Sony Corporation | Methods and apparatus for error concealment utilizing temporal domain motion vector estimation |
US6466592B1 (en) * | 1998-11-02 | 2002-10-15 | Cisco Technology, Inc. | MPEG frame add/drop multiplexer with bandwidth controller |
US6466619B1 (en) * | 1994-02-22 | 2002-10-15 | Victor Company Of Japan, Ltd. | Decoding apparatus for protection of data reproduction according to both protection data and apparatus protection data |
US6477185B1 (en) * | 1997-11-17 | 2002-11-05 | Hitachi, Ltd. | Demultiplexing and decoding apparatus for coded audio and video data |
-
2000
- 2000-04-17 JP JP2000115573A patent/JP2001309372A/en not_active Withdrawn
-
2001
- 2001-03-22 US US09/814,092 patent/US6765961B2/en not_active Expired - Lifetime
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6466619B1 (en) * | 1994-02-22 | 2002-10-15 | Victor Company Of Japan, Ltd. | Decoding apparatus for protection of data reproduction according to both protection data and apparatus protection data |
US5488385A (en) * | 1994-03-03 | 1996-01-30 | Trident Microsystems, Inc. | Multiple concurrent display system |
US6449311B1 (en) * | 1997-03-13 | 2002-09-10 | Sony Corporation | Methods and apparatus for error concealment utilizing temporal domain motion vector estimation |
US6477185B1 (en) * | 1997-11-17 | 2002-11-05 | Hitachi, Ltd. | Demultiplexing and decoding apparatus for coded audio and video data |
JPH11177436A (en) | 1997-12-12 | 1999-07-02 | Kokusai Electric Co Ltd | Data communication equipment |
JPH11225333A (en) | 1998-02-06 | 1999-08-17 | Toshiba Corp | Decode processing circuit |
US6466592B1 (en) * | 1998-11-02 | 2002-10-15 | Cisco Technology, Inc. | MPEG frame add/drop multiplexer with bandwidth controller |
Non-Patent Citations (2)
Title |
---|
"A Chip Set for Programmable Real-Time MPEG2 MP@ML Video Encoder", by Matsumura et al., IEICE Trans. Electron., vol. E81-C, No. 5 (May 1998), pp. 680-693. |
"A Single-Chip MPEG2 422@ML Video, Audio, and System Encoder with a 162-MHz Media-Processor and Dual Motion Estimation Cores", by Kumaki et al., IEEE 1999 Custom Integrated Circuits Conference, pp. 94-99. |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020126697A1 (en) * | 2001-03-07 | 2002-09-12 | Jost Arthur P. | Methods and apparatus for reconfiguring protocol data when reducing multiplexed data streams |
US20020126702A1 (en) * | 2001-03-07 | 2002-09-12 | Vince Lawrence D. | Methods and apparatus for constructing a program association table when reducing multiplexed data streams |
US6847659B2 (en) * | 2001-03-07 | 2005-01-25 | General Instrument Corporation | Methods and apparatus for reconfiguring protocol data when reducing multiplexed data streams |
US20050105564A1 (en) * | 2001-03-07 | 2005-05-19 | General Instrument Corporation | Methods and apparatus for reconfiguring protocol data when reducing multiplexed data streams |
US6934303B2 (en) * | 2001-03-07 | 2005-08-23 | General Instrument Corporation | Methods and apparatus for constructing a program association table when reducing multiplexed data streams |
US7653089B2 (en) * | 2001-03-07 | 2010-01-26 | General Instrument Corporation | Methods and apparatus for reconfiguring protocol data when reducing multiplexed data streams |
US20070046817A1 (en) * | 2005-08-24 | 2007-03-01 | Yoriko Yagi | Processor, processing method and processing program |
Also Published As
Publication number | Publication date |
---|---|
JP2001309372A (en) | 2001-11-02 |
US20010031008A1 (en) | 2001-10-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US5845152A (en) | Method for transmission of isochronous data with two cycle look ahead | |
US6490250B1 (en) | Elementary stream multiplexer | |
JPH08511384A (en) | Video peripherals for computers | |
US6765961B2 (en) | Encoding device mounted on one chip for multiplexing video signal and audio signal | |
EP1652054B1 (en) | Event time-stamping | |
JP3429652B2 (en) | Digital coding and multiplexing equipment | |
US20020131765A1 (en) | Digital video recorder | |
EP1576813B1 (en) | Video viewing system and method | |
JP3185863B2 (en) | Data multiplexing method and apparatus | |
US20080317139A1 (en) | Moving picture coding apparatus | |
EP2557787B1 (en) | Encoding apparatus and the method | |
JP4153817B2 (en) | Data encoding method and data encoding apparatus | |
US5341473A (en) | System of transferring data in a multi-CPU arrangement using address generators | |
JP4770023B2 (en) | Digital signal processing apparatus and method, and digital signal processing program | |
US20050235075A1 (en) | Method of and device for changing an output rate | |
JP2000083064A (en) | Transport stream processor | |
KR100962083B1 (en) | Method and system for converting a first data stream into a second data stream | |
JP2001103116A (en) | Stream generator | |
KR100250982B1 (en) | Mpeg-2 system multiplexer | |
JP2008135133A (en) | Bit stream reproducing device | |
JPH11261573A (en) | Multimedia data input/output interface device and fixed rate stream output method applied to the same | |
JP2001016140A (en) | Line switching device | |
JPH10150623A (en) | Data recording and reproducing device | |
JP2003234714A (en) | Multiplexer and bit rate converting circuit | |
JPH1175171A (en) | Multimedia data coding multiplexing method, multimedia data coding multiplexer and multimedia data coding multiplex program storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MITSUBISHI DENKI KABUSHIKI KAISHA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SEGAWA, HIROSHI;KUMAKI, SATOSHI;REEL/FRAME:011663/0483 Effective date: 20010123 |
|
AS | Assignment |
Owner name: RENESAS TECHNOLOGY CORP., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MITSUBISHI DENKI KABUSHIKI KAISHA;REEL/FRAME:014502/0289 Effective date: 20030908 |
|
AS | Assignment |
Owner name: RENESAS TECHNOLOGY CORP., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MITSUBISHI DENKI KABUSHIKI KAISHA;REEL/FRAME:015185/0122 Effective date: 20030908 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: RENESAS ELECTRONICS CORPORATION, JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:RENESAS TECHNOLOGY CORP.;REEL/FRAME:024973/0001 Effective date: 20100401 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FPAY | Fee payment |
Year of fee payment: 12 |
|
AS | Assignment |
Owner name: RENESAS ELECTRONICS CORPORATION, JAPAN Free format text: CHANGE OF ADDRESS;ASSIGNOR:RENESAS ELECTRONICS CORPORATION;REEL/FRAME:044928/0001 Effective date: 20150806 |