|Publication number||US5253071 A|
|Application number||US 07/811,089|
|Publication date||Oct 12, 1993|
|Filing date||Dec 20, 1991|
|Priority date||Dec 20, 1991|
|Publication number||07811089, 811089, US 5253071 A, US 5253071A, US-A-5253071, US5253071 A, US5253071A|
|Inventors||Michael T. MacKay|
|Original Assignee||Sony Corporation Of America|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (7), Non-Patent Citations (4), Referenced by (56), Classifications (15), Legal Events (5)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
The present invention relates to electronic stabilization of a video image, and more particularly, to a method and apparatus for reducing or eliminating jitter in a video image caused from hand shakes in a video camera.
2. Art Background
The reduction in size of video cameras and camcorders has resulted in the availability of hand-held video cameras or camcorders. The size of video cameras provides mobility for a camera-person allowing the video camera to be carried and subjects easily recorded. Although compact and to use, the hand-held video camera is often difficult to hold steady. As result of hand-held shakes from the camera-person, the video output is unstable and such instability results in poor quality video images.
In response to the hand-held video camera shake problem, there have been a number of inventions attempting to stabilize the output video The focus of these inventions has been on stabilizing the physical video camera in order to stabilize the video image. For example, Steadicam™one such system which employs a pair of interconnected spring loaded One end of the arm is pivotally supported by a carrying brace, while the end is connected to a handle which mounts the camera at the center of moment of inertia. The carrying brace is worn by the camera-person, and weight of the camera is counter balanced by action in the spring loaded The Steadicam JR., a portable system, is limited by the weight of video camera. The Steadicam JR will only support camcorders weighing up to approximately 3.5 pounds. Furthermore, the use of the Steady Cam requires skill in order to obtain optimum quality in the video output.
Solid state image sensors, light weight and compact in size, provide advantage over older tube-type imagers. Because of this advantage, solid imagers were developed for applications requiring lightweight and size image sensors such as hand-held video cameras. The two types of state image sensors used in video cameras today are charged coupled (CCD) image sensors and metal oxide semiconductor (MOS) image sensors. In both devices, the image sensor comprises of an array of photosensitive elements. Each element, known as a picture element or pixel, generates a value over an interval of time whereby the value is dependent upon the intensity of light incident upon that photosensitive element. For both and MOS image sensors, the value is a negative charge generated by the photosensitive element. For images sensors compatible with the National Television Standards Committee (NTSC) standard, there may be 768 horizontal pixels and 482 vertical pixels. For image sensors to be for high definition television (HDTV), the array of photosensitive elements be as large as 1920 horizontal and 1035 vertical pixels.
In CCD image sensors, transferring of charge from the photosensitive elements out of the array is accomplished by either frame transfer, transfer, or a combination of the two methods known as frame-interline transfer. In all types of transfer methods, the CCD image sensor uses shift registers. The transfer methods are dependent upon the arrangement the image sensor's vertical registers. The CCD shift register is comprised of p-type silicon material covered with a thin insulating oxide film. On top of oxide layer are electrodes which are connected to clocking control applying a positive voltage to the electrode, the CCD shift register potential well which can store negative charge. By generating a series controlled clocking signals to the electrodes, the charges may be through the shift register and eventually out of the array.
FIG. 1a illustrates a frame transfer CCD image sensor. As shown FIG. 1a, the CCD image sensory is divided into optically sensitive section 10, store section 12 and line readout section 14. After a charge has accumulated in the photosensitive cells in image section 10, charge is transferred into store section 12 very rapidly. Upon transfer charges from image section 10 to store section 12, the photosensitive in store section 12 are emptied for preparation for the next exposure The charge, now stored in store section 12, may be transferred line by line readout section 14 and then gated by output gate 16.
Referring to FIG. 1b, an interline transfer CCD array is In the interline transfer technique, after an initial exposure period of photodiodes 22, charge is transferred from photodiodes 22 to vertical registers 18. In the interline transfer method, the charges are shifted vertical registers 18, and the charges on the bottom of vertical shift registers 18 are transferred into horizontal shift register 20. Then, the charges horizontal shift register 20 are shifted out of the array. In this way, charges are transferred, one at a time, down vertical shift registers then transferred out of the array through horizontal shift line by line.
During high speed transfer of charges from either the image area to stored area in frame transfer, or from the vertical shift registers in interline transfer, the potential wells accumulate extra unwanted charge which generate an effect known as vertical smear. The vertical smear results unwanted line super imposed upon the image. In FIG. 1c, frame interline, a technique which combines the vertical shift registers and field store area, is illustrated. One advantage of the frame interline charge transfer technique is the reduction of vertical smear. The charges are photodiode 22 to vertical registers 26, and then immediately transferred high speed to CCD field memory 28. This high speed transfer reduces the time in which vertical registers 26 may be affected by extra charge accumulation or vertical smear. Once the charges have been transferred field memory section 28, the charges are then transferred out of the CCD image sensor through the horizontal registers in much the same matter as the frame transfer technique.
An electronic image stabilization method and apparatus is disclosed which has application for use in video cameras and camcorders, and in particular, hand-held video cameras. As a camera-person physically moves the video camera while video taping a subject image, the resultant video image exhibits jitters thereby degrading the quality of the video image. The electronic stabilization apparatus comprises a video camera with a high density television (HDTV) image sensor. The HDTV image sensor has a plurality of cells that generate a value representative of light incident upon that cell, and a transferring circuit to transfer the cell values out of the HDTV image sensor. A lens configuration on the video camera projects a subject image with a spatial resolution compatible with NTSC or PAL standard format onto the larger HDTV image sensor. Because the HDTV image sensor has a greater spatial resolution than the desired NTSC or PAL standard spatial resolution, only a portion of the HDTV image sensor is illuminated by the subject image. As the camera is physically jittered, a gimbaled ring assembly coupled to the lens configuration projects the subject image on a different portion of the HDTV image sensor. The movement of the gimbaled ring assembly is limited by stoppers such that the outer limit of the stopper projects the subject image on an outer edge of the HDTV image sensor.
A scanner circuit reads the cell values generated by the HDTV image sensor one cell at a time. A scanner counter circuit generates an address representative of the location of the cell which is being read by the scanner circuit. A threshold comparator circuit then compares the scanned cell value to a pre-determined threshold value to determine whether the corresponding cell is illuminated by the subject image. If it is, an output of the threshold comparator circuit disables the scanning counter, and the scanning counter then holds the address of a corner of the subject image. The address is input to a control circuit which alters the clocks signals transferring the values out of the HDTV image sensor array. In this way, only the image area illuminated by the subject image is transferred out of the HDTV image sensor. As a consequence, a video signal is generated which reduces or eliminates physical jitter from the video image.
The objects, features and advantages of the present invention will be apparent from the following detailed description in which:
FIG. 1a is an illustration of a frame transfer CCD image sensor.
FIG. 1b is an illustration of an interline transfer CCD image sensor.
FIG. 1c is an illustration of a frame interline transfer CCD image sensor.
FIGS. 2a-2b are illustrations of the concept of the present invention.
FIG. 3a is a front view of a gimbaled lens configuration of the present invention.
FIG. 3b is a right front perspective view of a gimbaled lens configuration and an HDTV image sensor of the present invention.
FIGS. 3c-3d are a top view a gimbaled lens configuration and an HDTV image sensor of the present invention.
FIG. 4 is a portion of an interline transfer CCD image sensor with control circuitry.
FIG. 5 is a high level block diagram illustrating circuitry of the present invention.
FIG. 6 is a block diagram illustrating the control circuit of the present invention.
FIG. 7 is an illustration of an HDTV image sensor of the present invention.
An method and apparatus for video image stabilization is disclosed. In the following description, for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that these specific details are not required in order to practise the present invention. In other instances, well known circuits and devices are shown in block diagram form in order not to obscure the present invention unnecessarily.
Referring to FIGS. 2a and 2b, the video image stabilization method of the present invention is illustrated. When camera 35 is held in a steady position, gimbaled lens configuration 34 will project subject 32 onto high definition television (HDTV) charged coupled device (CCD) image sensor 40 at subject image 36. Subject 32 may be focused upon HDTV CCD image sensor 40 so as to illuminate a number of pixels equal to any desired video format. For example, subject image 36 could be of a spatial resolution compatible with either the NTSC or PAL television standards. All that is required is that the projected image have a lower spatial resolution than the spatial resolution of the HDTV image sensor used. In the preferred embodiment, focused images 36 and 38 have a vertical resolution of 482 pixels and a horizontal resolution of 768 pixels, and HDTV image sensor 40 has a horizontal resolution of 1920 pixels and a vertical resolution of 1035 pixels.
Subject image 36 represents the location of subject 32 at an initial time, t(0), when camera 35 is held stable. When camera 35 is physically shaken, lens configuration 34 is displaced by the physical vibration, and as a result subject 32 is now projected onto HDTV image sensor 40 as subject image 38. At a predetermined time t(1), after the initial time t(0), camera 35 will transfer subject image 38 out of HDTV image sensor 40. The time interval between t(0) and t(1) is equal to an electronic shutter speed of camera 35. In video applications conforming to the NTSC standard format, shutter speeds are set to 1/60 of a second per video field. Because subject image 36 is centered on HDTV image sensor 40 when camera 35 is held in a steady position, subject 32 may be tracked in an equal distance in all directions. Consequently, at the end of each electronic shutter speed interval, subject 32 will be provided upon HDTV image sensor 40 representing the physical displacement, within limitations, occurring in that shutter speed time interval. At the end of each shutter speed interval, only the subject image portion of HDTV image sensor 40 will be transferred out of HDTV image sensor 40 as will be explained in detail below.
Referring to FIG. 3a, a front view of gimbaled lens configuration 51 is illustrated. Gimbaled lens configuration 51 has two mutually perpendicular and intersecting axes of rotation thereby giving free angular movement in two directions. In this way, lens configuration 51 is free to move in both X and Y directions with respect to a plane parallel to the face of CCD image sensor 40. Lens casing 56, a cylindrical shell, houses the gimbaled lens configuration. Outer gimbal ring 62 is mounted within lens casing 56, and outer gimbal ring 62 has a diameter less than lens casing 56. Outer gimbal pins 52 and 54 support outer gimbal ring 62 such that outer gimbal ring 62 is free to rotate in the Y-axis direction as shown in FIG. 3b. Vertical stoppers 53 and 57 provide limits in which outer gimbal ring 62 may extent within lens casing 56. As illustrated in FIG. 3b, if outer helical mount 62 is in the fully extended position as shown, then the subject image will be projected onto the top of HDTV image sensor 40. Therefore, outer gimbal ring 62 allows lens configuration 51 to project the subject image onto HDTV image sensor 40 in the Y-direction from the top to the bottom of CCD image sensor 40.
Helical mount 64, cylindrical in shape, is mounted within outer gimbal ring 62 by inner gimbal pins 58 and 60. Helical mount 64 may serve as a housing for any lens or lens combination desired. Inner gimbal pins 58 and 60 are mounted on the top and bottom of helical mount 64 so as to provide freedom of rotation in the X-axis with respect to the face of image sensor 40. Also, inner gimbal pins 58 and 60 are located in the front section of lens casing 56 so as to minimize the movement of the front portion of helical mount 64. As illustrated in the top view of lens configuration 51 in FIG. 3d, lens configuration 51 may only rotate as far as stoppers 63 and 65 will permit. When helical mount 64 is in this fully extended position, subject image 66 will be projected to the left side of HDTV image sensor as shown in FIG. 3c 40. As a result, when a video camera housing lens configuration 51 is physically displaced, the inner gimbaled mounting allows helical mount 64 to move in a horizontal direction with respect to the face of CCD image sensor 40. Similarly, the outer gimbaled mounting allows helical mount 64 to move in a vertical direction with respect to the face of HDTV image sensor 40.
In the preferred embodiment, standard interlaced video is employed. In interlaced video or field-integrated mode, the image sensor is divided into odd and even vertical lines. The scanning of the vertical lines are altered from one field to the next field and both the odd and even fields comprise a complete video frame. The odd field is generally scanned first followed by the even field. The alternate scanning of photosensitive elements results in proper color reproduction. The speed at which video images are exposed to the image sensor, and then subsequently clocked out of the array, is dependent upon the electronic shutter speed. Typically, for applications involving normal speeds, the shutter speed, or field period is 1/60 of a second. Correspondingly, one interlaced frame would occur at 1/30 of a second.
Now referring to FIG. 4, a portion of an interline transfer CCD image sensor, and part of the CCD image sensor transfer circuitry is illustrated. In the preferred embodiment, CCD solid state image sensor 40 is of the interline transfer type. Although the invention has been described in conjunction with a interline transfer CCD image sensor, numerous alternatives, variations, modifications to a CCD interline transfer image sensor will be apparent to those skilled in the art in light of the foregoing description. The electrodes and the potential wells beneath them on both CCD vertical and horizontal shift registers 18 and 20 are arranged in groups of four as shown. Because of this arrangement, the method used for transferring the charges out of CCD image sensor 40 is four-phase double clocking. Charges may be transferred within a group of four potential wells, as well as from one group of wells to an adjacent group of wells. For any one video field in this particular arrangement, there are two photosensitive elements for a group of four potential wells. In FIG. 4, the photosensitive elements are photodiodes 22. The method of transfer of charge from photodiodes 22 and out of CCD image sensor 40 is identical for both the odd and even fields. For purposes of illustration, only the transfer of the odd video field is shown in FIG. 4. The transfer of negative charges from photodiodes 22 to the first two potential wells in vertical shift register 18 occurs during the vertical blanking period. In the horizontal blanking interval, the charges are moved in vertical shift register 18 from potential wells one and two of one group, to potential wells one and two of the next group. The charge in wells one and two at the well group at the bottom of vertical shift register 18 will be transferred into wells one and two of horizontal shift register 20. In the last interval, the horizontal scan period, the charges are clocked out of horizontal shift register 20 and gated by output gate 21. Output Vo is then sent to a video processing circuit.
As described above, the CCD potential wells are constructed in groups of four for both vertical and horizontal shift registers 18 and 20 with each potential well having a corresponding electrode. The electrodes in vertical shift register 18 are connected to vertical shift register clocking signals V1, V2, V3 and V4. Vertical shift register clocking signals V1, V2, V3 and V4 are generated by clock generator 42 and the appropriate drive levels of the signals are generated in vertical drive 46. At a first time interval in the vertical blanking period, a V1 positive voltage is applied to the first electrode of all potential wells in vertical shift register 18. This positive voltage allows the negative charge of corresponding photodiode 22 to be transferred to the first potential well underneath the first electrode. At a second timing interval in the vertical blanking period, a V2 positive voltage is applied to the second electrode, and consequently the charge from the first potential well is shared with the second potential well. At a third timing interval in the vertical blanking period, V1 goes low, and the remaining charge is then transferred entirely to the second potential well.
Connected to each electrode on horizontal shift register 20 are horizontal shift register clocking signals H1, H2, H3 and H4. H1, H2, H3 and H4 clocking signals are generated by CCD clock generator 42, and buffered by buffer 48. During the horizontal blanking period, the negative charge in vertical shift register 18 is moved from wells one and two of one group to wells one and two to the adjacent group. Also, the group at the very bottom of vertical shift register 18 transfers its charge to wells one and two of horizontal shift register 20. To accomplish this, H1 and H2 are held at a high voltage level during the horizontal blanking period so as to allow the negative charge from vertical shift register 18 to enter into the first two potential wells in horizontal shift register 20. The charge is held in potential wells one and two of horizontal shift register 20 until the horizontal scan period.
During the horizontal scan period, the entire line of charges stored in horizontal shift register 20 is transferred out of CCD image sensor 40 by the horizontal clocking signals H1, H2, H3 and H4. In a first time interval of the horizontal scan period, H1 and H2 transition to become a low voltage value while H3 and H4 are a high voltage value resulting in the transfer of charges from wells one and two to wells three and four in horizontal shift register 20. During a second time interval, the polarity of the control signals is reversed, and charges are transferred from wells three and four of one well group to wells one and two of the next adjacent well group. Output gate 21 is coupled to horizontal shift register 20 and operation of the gate is controlled by horizontal gating signal (HG). CCD image sensor 40 uses the horizontal gating (HG) signal to control its internal output gate and only allows charges to exit when the gate is open. Accordingly, when HG is low, charges will not exit horizontal shift register 20 but when HG is high logic level, the charges will be gated and transferred to video signal processing circuits. The horizontal blanking and the horizontal scan period are repeated until all of the horizontal lines have been shifted out of CCD image sensor 40.
Referring now to FIG. 5, a block diagram of a preferred embodiment is illustrated. During an exposure time interval, photosensitive elements are charged dependent upon light intensity incident upon the photosensitive elements. During the vertical blanking period, the negative charges located in the photosensitive elements are transferred to vertical registers 18 of CCD image sensor 40. CCD clock generator 42 is enabled when a logic high voltage value is applied to enable 158 on FIG. 6. At the beginning of the vertical blanking period, VB 126, generated by clock counter 128, will supply a logic high voltage to OR gate 130. Although load signal 122 will be a logic low voltage level at the beginning of the vertical blanking period, VB 126 will enable CCD clock generator 42. Upon completion of the vertical blanking period, VB 126 will transition to a low voltage level, and therefore OR gate 130 will supply a logic low voltage value to enable pin 158. By disabling CCD clock generator 42, the horizontal blanking period will be suspended until load signal 122 becomes a logic high voltage level. At this time, OR gate 130 will again output a high logic level to enable pin 158 thereby causing CCD clock generator 142 to resume with the horizontal blanking period.
Referring to FIG. 6, VB will have a low logic level after the vertical blanking period is finished as described in the previous paragraph. Inverter 156 will output a signal VBBAR 126 which will have a high voltage state after the vertical blanking period. VBBAR 126 is input to XOR gate 124 and as a consequence horizontal counter 82 is enabled. Returning to FIG. 5, each potential well of vertical shift register 18 located in CCD image sensor 40 is connected to MUX 68 in scanning circuit 72. The charged values will be selected by MUX 68. Output registers 94 and 96 of horizontal and vertical counters 82 and 84 are connected to MUX 68. Output registers 94 and 96 are addresses for selecting the vertical and horizontal potential wells in CCD image sensor 40. The values contained in output registers 94 and 96 correspond to a particular vertical shift register location on CCD image sensor 40. The top portion of CCD image sensor 40 begins the first horizontal line of CCD potential wells, and the left portion of CCD image sensor 40 begins a first vertical line of vertical shift registers.
In the preferred embodiment, the scanning of CCD image sensor 40 begins with the first horizontal line at first vertical shift register 18. The horizontal line is scanned from left to right although any direction of scanning would suffice. Upon completion of scanning the first horizontal line, the next lower horizontal line is scanned also starting with the first vertical shift register. When the initial horizontal and vertical count are both 0, MUX 68 will receive the address (0,0) from horizontal and vertical counters 82 and 84. This address will select a line connected to a potential well located in the first vertical shift register in the first horizontal line. Scanning oscillator 80 is coupled to both latch register 70 in scanning circuit 72 and clock inputs 86 of vertical and horizontals counters 82 and 84. On a leading clock edge of scanning oscillator 80, the (0,0) address will be provided to MUX 68, and the charged value of the selected potential well will be provided to register 70. On a falling clock edge of scanning oscillator 80, the charged value will be latched in register 70, and horizontal counters 82 will be incremented. Similarly, on the rising edge of the next scanning oscillator clock cycle, MUX 68 will select the next potential well in CCD image sensor 40 corresponding to the (1,0) address. Horizontal counter 82 is cascaded to vertical counter 84 so that when horizontal counter 82 reaches its terminal count of 768, the vertical count will be incremented by one. In this way, scanning circuit 72 will read each charged value within the vertical shift registers one potential well at a time.
As illustrated in FIG. 5, the output of register 70 is connected to an input of comparator 76 of threshold circuit 78. A voltage is applied to each photosensitive cell in CCD image sensor 40 for operation of the photosensitive cells. As a consequence, even cells not exposed to light will exhibit a small negative charge. The amount of quiescent charge is referred to as black noise of the cell. The black noise level of a cell is a pre-determined voltage value dependent upon the particular type of photosensitive cell used. In the preferred embodiment, this pre-determined black noise level is threshold value 74 in threshold circuit 78. Threshold value 74 is input to comparator 76 of threshold circuit 76. Comparator circuit 76 compares the charged value in register 70 to threshold value 74. If the charged value contained within register 70 is greater than pre-determined threshold value 74, the output of comparator 76 generates a high voltage level.
If the output on comparator 76 indicates that the selected shift register contains a charge greater than threshold value 74, then the top left corner of the subject image on CCD image sensor 40 has been found. The output of comparator circuit 76 is buffered so as to be able to drive a plurality of inputs. When threshold circuit 78 generates a high output voltage signal, scanning circuit 72, vertical and horizontal counters 96 and 94 are all disabled. Enable 88 on horizontal counter 82 is connected to XOR gate 124. VBBAR 126, input to XOR gate 124, will be a high voltage value signifying the completion of the vertical blanking period. Therefore, a high voltage value input from threshold circuit 78 will result in a low voltage value to enable pin 88 and subsequently disable horizontal counter 82. As a consequence, vertical counter 84 will also be disabled because of horizontal counter will not reach it terminal count necessary to enable pin 92 of vertical counter 84. Decoder 98 will decode output register 94 so as to enable pin 92 on vertical counter 84 when horizontal counter 82 has reached the maximum horizontal count. In the preferred embodiment, when horizontal counter 82 reaches the value of 768, vertical counter 84 will be enabled. Contained in output registers 94 and 96 located in horizontal and vertical counters 82 and 84 are horizontal count and vertical count values respectively. X and Y count values correspond to the address of a potential well containing the charge corresponding to the upper left corner of the subject image on CCD image sensor 40. With the pre-determined knowledge of the spatial resolution size of the subject image and the determination of the address of the upper left corner of the subject image, the information to extract only the charged cells containing the subject image is obtained.
Adder circuit 100 calculates two Y preset values, two X preset values and a load signal for use in control circuit 102. Both the horizontal count and the vertical count values are used in these calculations. Upon finding an illuminated pixel on CCD image 40 corresponding to the top left corner of the subject image, threshold circuit 78 will latch horizontal count and vertical count in registers 104 and 106, respectively, in adder circuit 100. Also, the horizontal and vertical resolution values for the subject image are pre-stored within adder circuit 100 in registers 108 and 110. In the preferred embodiment, the pixel element resolution for the subject image is 768 horizontal pixels by 482 vertical pixels, but any horizontal or vertical resolution could be pre-stored within adder circuit 100. Also, horizontal and vertical resolution values could be connected to a user interface such that the values are selectable for a particular application. To calculate Y1 preset value 114, vertical resolution, stored in register 110, is added in computation circuit 112 to vertical count stored in register 104. The sum is then subtracted, also in computation circuit 112, from the HDTV vertical image resolution size. In the preferred embodiment, the HDTV vertical resolution size is 1035, and therefore Y1 preset is equal to 1035-(482+Vertical count). Y2 preset value is simply equal to vertical count.
Adder circuit 100 also calculates X1 and X2 preset values. X1 preset value is set equal to the horizontal count value. X2 preset value is calculated from adding horizontal resolution value in register 108 to horizontal count value in register 106, and then subtracting the sum from the horizontal resolution of the HDTV image sensor. In the preferred embodiment, the horizontal resolution of the HDTV CCD image sensor is 1920 pixels. Therefore, in this example, X2 preset is equal to 1920-(768+horizontal count). X1, X2, Y1 and Y2 preset values are stored in registers within computation circuit 112 and load signal 122 is generated. Referring to FIG. 6, a more detailed block diagram of control circuit 102 is illustrated. Upon generation of load signal 122, Y1 and Y2 presets are loaded into preset inputs of Y1 and Y2 counters 134 and 136. Charge transfer oscillator 44, used by CCD clock generator 42 to generate control clocking signals, is input to clock counter 128. Clock counter 128 divides the fundamental frequency of charge transfer oscillator 44 to represent timing intervals VB and HS/HB. VB is a clock with a period equal to the vertical blanking period in CCD clock generator 42. HS/HB is also a clock with a period equal to the horizontal blanking and scan periods. Because HS/HB clocks Y1 counter 134 and Y2 counter 136, Y1 counter 134 will be incremented after a horizontal blanking and horizontal scanning intervals have elapsed. Therefore, each increment in Y1 counter will occur when one horizontal line of charge has been extracted from CCD image sensor 40. TC 138 and 140 located on Y1 and Y2 counters respectively, are initially in a low logic level, but when Y1 and Y2 counters 134 and 136 reach a terminal count, TC 138 and 140 will be at a high logic level. TC 138 of Y1 counter is coupled to one input of 5 input AND gate 154.
Clock inputs 131 of X1 and X2 counters 142 and 144 are coupled to output clocking signal H3 on CCD clock generator 42. H3 and H4 clocking control signals are only active during the horizontal scan period, and therefore both X1 and X2 counters 142 and 144 will only be incremented during the horizontal scan period. X1 and X2 present values 118 and 114 are loaded into present registers 133 and 135 respectively. Loading occurs when the load signal is generated from adder circuit 100, which is the same time Y1 and Y2 present registers 114 and 116 are loaded. Tc 148 on X1 counter 142 is coupled to a second input on AND gate 154, and TC 146 on X2 counter 144 is coupled to a third input to AND gate 154 via inverter 150. As is the case in TC 138 in Y1 counter 134, the initial TC value before terminal count has occurred is a low voltage level. Because of the initial low voltage state of TC 148, HG will be disabled until the terminal count has been reached by X1 counter 142.
Upon completion of the loading of all preset counter registers, CCD clock generator 42 is enable by load signal 122 at enable pin 158. At this point, control clocking signals will be generated in CCD clock generator 42, and the horizontal blanking and scanning periods may proceed. TC 138 on Y1 counter 134 will initially possess a low voltage value, and therefore will generate a low voltage level at AND gate 154. As a consequence of the state of TC 138, HG will be a low value and no charges will be gated out of the horizontal shift register. As each horizontal blanking and scanning period is completed, clock cycle HS/HB input to Y1 and Y2 counters 134 and 136 will increment the counters. Terminal count is obtained in Y1 counter 134 when the value in Y1 counter registers is equal to preset Y1 value 114 plus the number of horizontal lines extracted from CCD image sensor 40. When the terminal count of Y1 counter 134 occurs, TC 138 becomes low and will no longer disable HG. The number of horizontal lines in which will not be gated because of Y1 counter 134 disabling of HG is represented in FIG. 7 by the area designated Y1. Therefore, at the terminal count of Y1 counter 134, all the charges in CCD image sensor 40 will be shifted the distance represented by the area Y1, and consequently no charges exiting the horizontal shift register will be gated. At this time, the charge representing the subject image illustrated in CCD image sensor 40 will now appear on the bottom of the vertical registers in CCD image sensor 40.
After Y1 counter 134 has reached its terminal count, TC 138 will not longer disable horizontal gating. Unlike Y1 counter 134, which is incremented only after horizontal scan and horizontal blanking periods have occurred, X1 and X2 counters 142 and 144 are incremented after each clock cycle of H3 within the horizontal scan period. Both X1 and X2 counters 142 and 144 will reach terminal count every horizontal scan period, and as a consequence X1 preset value 118 and X2 preset value 114 but be re-loaded every horizontal scan period. The re-loading of X1 and X2 preset values 142 and 144 occurs after the horizontal scan period and during the horizontal blanking period. In operation, at the beginning of each horizontal scan period, X1 counter 142 will disable HG because of the low voltage value of TC 148 at AND gate 154. The period in which X1 counter disables HG is illustrated in FIG. 7 by the area designated X1.
Upon the terminal count of X1 counter 142, HG will be enabled and the charges will be gated out of image sensor 40 until the terminal count of X2 counter 144 occurs. At this time, TC 146 becomes a high voltage value, and inverting the signal by inverter 150 will disable HG. As a consequence, HG will remain disable for the remainder of the horizontal scan period. The period in which X2 counter 144 disables HG is represented in FIG. 7 by the area designated as X2. After the horizontal scan period is completed, X1 and X2 present values 118 and 114 are re-loaded for preparation for the next horizontal scan period by the rising edge of HS/HB designating a new horizontal blanking period.
The last of control circuit 102 counter to be enabled is Y2 counter 136. Enable 160 on Y2 counter 136 is coupled to TC 138 on Y1 counter 134, and therefore Y2 counter 136 is not enabled until Y1 counter 134 has reached its terminal count. Y2 counter 136 is loaded with Y2 preset value in the same manner as Y1 preset value 114 is loaded into Y1 counter 134. TC 140 on Y2 counter 136 is also coupled to an input on AND gate 154. In operation, Y2 counter 136 will began counting horizontal blanking and horizontal scan periods when the subject image has begun to enter the horizontal shift registers. HG will be enabled during this period so as to clock out the charged registers representing the subject image. Upon completion of clocking out all the changes remaining in the vertical shift registers, TC 140 will become a high voltage level, and after inverting the signal by inverter 152, will disable AND gate 154. The disabled horizontal gating period as a result of Y2 counter 136 is indicated in FIG. 7 by the area designated Y2.
When the last charge has been shifted out of the vertical and horizontal shift registers, CCD clock generator 42 will reset itself, and the vertical blanking period will begin on the even field. Also, a third clock counter within clock counter 132, will generate reset signal 90 after the last charge has been transferred out of CCD image sensor 40. Reset signal 90 is coupled to adder circuit 100, horizontal counter 82 and vertical counter 84. Horizontal and vertical counters 82 and 84 are then reset to an initial state. In adder circuit 100, vertical and horizontal count registers 104 and 106 are cleared, as well as Y1, Y2, X1, X2 and load signal in computation circuit 112. As a consequence, all circuits are reset to an initial set to begin the next vertical blanking period.
The foregoing has described a method and apparatus for image stabilization. It is contemplated that changes and modifications may be made by one of ordinary skill in the art, to the materials and arrangements of elements of the present invention without departing from the spirit and scope of the invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4673276 *||Feb 22, 1985||Jun 16, 1987||Canon Kabushiki Kaisha||Blur detecting device for a camera|
|US4788596 *||Apr 25, 1986||Nov 29, 1988||Canon Kabushiki Kaisha||Image stabilizing device|
|US4855838 *||May 27, 1988||Aug 8, 1989||Cues, Inc.||Remotely controlled pan and tilt television camera|
|US4959725 *||Jul 13, 1988||Sep 25, 1990||Sony Corporation||Method and apparatus for processing camera an image produced by a video camera to correct for undesired motion of the video camera|
|US4965619 *||Jan 19, 1989||Oct 23, 1990||Canon Kabushiki Kaisha||Image stabilizing device|
|US4979034 *||Jul 13, 1989||Dec 18, 1990||Konica Corporation||Image memory apparatus|
|US5060074 *||Oct 10, 1990||Oct 22, 1991||Hitachi, Ltd.||Video imaging apparatus|
|1||*||Product Announcement for: Panasonic PV S770 Video Camcorder/HQ (1992).|
|2||Product Announcement for: Panasonic PV-S770 Video Camcorder/HQ (1992).|
|3||*||Uomori, Morimura and Ishii, Electronic Image Stabilization System for Video Cameras & VCRs, SMPTE Journal , vol. 101, No. 2, Feb. 1992, pp. 66 75.|
|4||Uomori, Morimura and Ishii, Electronic Image Stabilization System for Video Cameras & VCRs, SMPTE Journal, vol. 101, No. 2, Feb. 1992, pp. 66-75.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US5396286 *||Jun 4, 1993||Mar 7, 1995||Sony Corporation||Video signal processing apparatus|
|US5436658 *||Aug 17, 1994||Jul 25, 1995||Sony Corporation||Camera apparatus for forming a plurality of different kinds of pictures corresponding to different television standards and different aspect ratios|
|US5497192 *||Feb 28, 1995||Mar 5, 1996||Sony Corporation||Video signal processing apparatus for correcting for camera vibrations using CCD imager with a number of lines greater than the NTSC standard|
|US5587739 *||May 17, 1995||Dec 24, 1996||Nikon Corporation||Variable magnification image taking device|
|US5699112 *||Jun 1, 1995||Dec 16, 1997||Vision Iii Imaging, Inc.||Imaging stablizing apparatus for film and video cameras utilizing spurious camera motion compensating movements of a lens aperture|
|US5933664 *||Oct 10, 1997||Aug 3, 1999||Vision Iii Imaging, Inc.||Method of using a parallax scanning lens aperture in a range-finding application|
|US5963749 *||Feb 9, 1998||Oct 5, 1999||Nicholson; Lynn||Self-leveling invertible camera stabilizer|
|US6285399 *||Jul 9, 1997||Sep 4, 2001||Flashpoint, Technology, Inc.||System and method for generating timing signals in an electronic imaging device|
|US6636266||Dec 2, 1997||Oct 21, 2003||Canon Kabushiki Kaisha||Image recording device with structure for adjusting the device based on the video signal|
|US6658136 *||Dec 6, 1999||Dec 2, 2003||Microsoft Corporation||System and process for locating and tracking a person or object in a scene using a series of range images|
|US6760613||Jun 24, 2002||Jul 6, 2004||Medispectra, Inc.||Substantially monostatic, substantially confocal optical systems for examination of samples|
|US6768918||Jul 10, 2002||Jul 27, 2004||Medispectra, Inc.||Fluorescent fiberoptic probe for tissue health discrimination and method of use thereof|
|US6818903||Sep 13, 2002||Nov 16, 2004||Medispectra, Inc.||Method and apparatus for identifying spectral artifacts|
|US6826422||Jan 11, 2000||Nov 30, 2004||Medispectra, Inc.||Spectral volume microprobe arrays|
|US6839661||Dec 15, 2000||Jan 4, 2005||Medispectra, Inc.||System for normalizing spectra|
|US6847490||Jun 9, 2000||Jan 25, 2005||Medispectra, Inc.||Optical probe accessory device for use in vivo diagnostic procedures|
|US6902935||Dec 15, 2000||Jun 7, 2005||Medispectra, Inc.||Methods of monitoring effects of chemical agents on a sample|
|US6933154||Nov 15, 2002||Aug 23, 2005||Medispectra, Inc.||Optimal windows for obtaining optical data for characterization of tissue samples|
|US7103401||Jul 10, 2002||Sep 5, 2006||Medispectra, Inc.||Colonic polyp discrimination by tissue fluorescence and fiberoptic probe|
|US7114810||Jun 25, 2004||Oct 3, 2006||Hewlett-Packard Development Company, L.P.||Multimedia display device|
|US7127282||Apr 12, 2002||Oct 24, 2006||Medispectra, Inc.||Optical methods and systems for rapid screening of the cervix|
|US7136518||Apr 18, 2003||Nov 14, 2006||Medispectra, Inc.||Methods and apparatus for displaying diagnostic data|
|US7187810||Oct 18, 2002||Mar 6, 2007||Medispectra, Inc.||Methods and systems for correcting image misalignment|
|US7260248||Mar 15, 2002||Aug 21, 2007||Medispectra, Inc.||Image processing using measures of similarity|
|US7282723||Apr 18, 2003||Oct 16, 2007||Medispectra, Inc.||Methods and apparatus for processing spectral data for use in tissue characterization|
|US7309867||Apr 18, 2003||Dec 18, 2007||Medispectra, Inc.||Methods and apparatus for characterization of tissue samples|
|US7310547||Jul 19, 2004||Dec 18, 2007||Medispectra, Inc.||Fluorescent fiberoptic probe for tissue health discrimination|
|US7459696||Apr 18, 2003||Dec 2, 2008||Schomacker Kevin T||Methods and apparatus for calibrating spectral data|
|US7469160||Apr 18, 2003||Dec 23, 2008||Banks Perry S||Methods and apparatus for evaluating image focus|
|US7642741||Apr 22, 2006||Jan 5, 2010||Sidman Adam D||Handheld platform stabilization system employing distributed rotation sensors|
|US7705884||Jul 21, 2004||Apr 27, 2010||Zoran Corporation||Processing of video data to compensate for unintended camera motion between acquired image frames|
|US7791643 *||Sep 7, 2010||Hewlett-Packard Development Company, L.P.||Sequenced response image stabilization|
|US7830565||Jun 27, 2006||Nov 9, 2010||Motorola, Inc.||Image capture device with rolling band shutter|
|US8005527||Aug 23, 2011||Luma Imaging Corporation||Method of determining a condition of a tissue|
|US8102457||Jan 24, 2012||Flashpoint Technology, Inc.||Method and apparatus for correcting aspect ratio in a camera graphical user interface|
|US8120661||Jul 26, 2007||Feb 21, 2012||Human Monitoring Ltd||Image stabilizer|
|US8127232||Dec 21, 2007||Feb 28, 2012||Flashpoint Technology, Inc.||Method and apparatus for editing heterogeneous media objects in a digital imaging device|
|US8179078||Dec 6, 2009||May 15, 2012||Sidman Adam D||Handheld or vehicle-mounted platform stabilization system|
|US8223235||Dec 13, 2007||Jul 17, 2012||Motorola Mobility, Inc.||Digital imager with dual rolling shutters|
|US8289401||Oct 16, 2012||Csr Technology Inc.||Processing of video data to compensate for unintended camera motion between acquired image frames|
|US8970761||Nov 28, 2011||Mar 3, 2015||Flashpoint Technology, Inc.||Method and apparatus for correcting aspect ratio in a camera graphical user interface|
|US8972867||Jan 16, 2012||Mar 3, 2015||Flashpoint Technology, Inc.||Method and apparatus for editing heterogeneous media objects in a digital imaging device|
|US9224145||Aug 30, 2006||Dec 29, 2015||Qurio Holdings, Inc.||Venue based digital rights using capture device with digital watermarking capability|
|US20020107668 *||Dec 15, 2000||Aug 8, 2002||Costa Peter J.||System for normalizing spectra|
|US20040007674 *||Sep 13, 2002||Jan 15, 2004||Schomacker Kevin T.||Method and apparatus for identifying spectral artifacts|
|US20050286027 *||Jun 25, 2004||Dec 29, 2005||Bakkom Jeffrey S||Multimedia display device|
|US20060017814 *||Jul 21, 2004||Jan 26, 2006||Victor Pinto||Processing of video data to compensate for unintended camera motion between acquired image frames|
|US20060170782 *||Jan 28, 2005||Aug 3, 2006||Stavely Donald J||Sequenced response image stabilization|
|US20060170783 *||Jan 28, 2005||Aug 3, 2006||Stavely Donald J||Adaptive response image stabilization|
|US20070050139 *||Apr 22, 2006||Mar 1, 2007||Sidman Adam D||Handheld platform stabilization system employing distributed rotation sensors|
|US20070297026 *||Jun 27, 2006||Dec 27, 2007||John George C||Image Capture Device with Rolling Band Shutter|
|US20090153710 *||Dec 13, 2007||Jun 18, 2009||Motorola, Inc.||Digital imager with dual rolling shutters|
|US20090256918 *||Jul 26, 2007||Oct 15, 2009||Human Monitoring Ltd||Image stabilizer|
|US20100079101 *||Apr 1, 2010||Sidman Adam D||Handheld or vehicle-mounted platform stabilization system|
|EP0994619A1 *||Aug 5, 1999||Apr 19, 2000||Hewlett-Packard Company||Readout method for sub-area of digital camera image sensor|
|EP2052553A2 *||Jul 26, 2007||Apr 29, 2009||Human Monitoring Ltd||Image stabilizer|
|U.S. Classification||348/208.11, 348/352, 348/E05.046, 348/E03.02|
|International Classification||H04N5/3728, H04N5/361, H04N5/3725, H04N5/3722, H04N5/232|
|Cooperative Classification||H04N5/23248, H04N5/23287, H04N3/1562|
|European Classification||H04N5/232S2C2, H04N3/15E4, H04N5/232S|
|Dec 20, 1991||AS||Assignment|
Owner name: SONY CORPORATION OF AMERICA A CORPORATION OF DEL
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST.;ASSIGNOR:MAC KAY, MICHAEL P.;REEL/FRAME:005956/0889
Effective date: 19911217
|Jul 26, 1993||AS||Assignment|
Owner name: SONY ELECTRONICS, INC., NEW JERSEY
Free format text: CHANGE OF NAME;ASSIGNOR:SONY CORPORATION OF AMERICA;REEL/FRAME:006624/0714
Effective date: 19930601
|Apr 14, 1997||FPAY||Fee payment|
Year of fee payment: 4
|Apr 11, 2001||FPAY||Fee payment|
Year of fee payment: 8
|Apr 12, 2005||FPAY||Fee payment|
Year of fee payment: 12