CA2221683C - Direct digital airborne panoramic camera system and method - Google Patents

Abstract

The present invention relates to an im-proved airborne, direct digital panoramic camera system and method in which an in-line electro-optical sensor (80) operating in conjunction with a data handling unit (82), a controller unit (84), and real time archive unit (86), eliminates the need for photographic film and film transport appara-tus normally associated with prior art airborne reconnaissance cameras and yet still retains the very high image resolution quality which is so im-portant in intelligence operations and commercial geographic information systems (GIS), mapping and other remote sensing applications. Precise geographic data for the system is provided by the use of navigation aids which include the Global Positioning Satellite (GPS) System (14) and an airborne platform carried GPS receiver (85). The present invention provides a simpler, more effi-cient and less costly panoramic camera by utiliz-ing a simpler and less expensive line-focus type of lens in conjunction with an electro-optical line array sensor.

Description

WO 96~3~003 PC~JUS96/06771 Dl[IRECT DIGITAL AIRBORNE PANORAMIC CAMERA
XYSil~;~ AND METHOD
B~. l~. ~u..d of the Invention The present invention relates to airborne panoramic camera systems and more particularly .to a direct digital panoli~...ic camera system and mPthr rl in which an electro-optical digital sensor el;...i~ s the need for the film and film transport d~aldlus normally associated with prior art airborne l~,col~ iXX~nre C~,ldS.
Airborne camera r~co.~ x~nre or surveill~nre ~y~LIlls are nearly as old as the use 10 of military and civilian aircraft. The prior art camera or photo reco~ x~re ~y~L~lllS
generally involved camera photos taken from an aircraft flying over the area of interest, and the exposed film was l~l.. Pd to the ground after the flight where it was developed and processed before it could be delivered to the intelligence agencies or groups who could then tlPternnillP whether the photographs contain the desired intelligçn~e. A number of prior art reco~ ;c~ e systems, including ~hose disclosed in U.S. Patent No. 3,066,589, disclose an l,ol.le reco~ e system which includes an aill,.,ll.e f~ proceccing, se~nnin~ and~","~",i~icion of the data to associated ground stations. U.S. Patent No. 4,143,971 discloses an airlborne photo ~Co~ cs~nre system in which photo cells and color f:lter techniques are employed to identify specific targets of interest which have an unique optical or IR paKern.
U.S. Patent No. 4,442,453 discloses a combined film and electro-optical sensor for co,lv~lLing the exposed film to data suitable for relay over a radio link to a ground station where it may be ~lPmllltirlexed and displayed on television type monitors.
The above-described and other similar prior art photo reconn~icc~nce systems employ film as the light-sensing mPrlil-m and therefore have the ~tten-l~nt drawbacks of a bulky film system and film transport ayydldLus~ delays in developing the film and further generally include a more complex and subsldlllially more costly lens that utilize a square focal plane system which must focus in two dillle~ions.
It is the.er)le a ylillldly object of the present invention to prove an improved airborne ydllOl~ iC camera system and method which is less costly and more efficient in operation.
It is another object of the present invention to provide a direct digital airborne panoramic camera system and method in which an electro-optical sensor eli...i~ s the need for the film and film lld~yO~L apparatus of the prior art airborne camera systems.
It is yet another object of the present invention to provide an illlpn~vt:d airborne ~allol~llic camera system and method in which panoramic errors are easily correctable in 35 the digital image data.
It is yet another object of the present invention to provide an improved airborne panoramic camera system and method which employs a simplified, less costly optical system.

W O9~3~ PCTrUS96/06771 It is yet another object of the present invention to provide a direct digital, massive data rate airborne ~)dnOldllliC camera and system capable of efficiently supporting mapping and other remote sensing applications by producing Illassivc data bases which are efficiently o-~ li,~d and ~-o~-iale for use with geographic i-lrv~ ation systems.
It is yet another object of the present invention to provide a direct digital airborne panoramic camera system and method which el;...;.l~lPs the need for photographic film in airborne data collection process.
It is yet another object of the present invention to provide an improved airborne panoramic camera system and method having high image quality data particularly with 10 respect to contrast and dynamic range.
It is yet another object of the present invention to provide an improved direct digital airborne palloldll,ic camera system and method in which the images may be conveniently, accurately, and easily geo-,crclellced.
It is a further object of the present invention to provide an illl~Lvv~;d direct digital 15 airborne panoramic camera system and method in which the massive amounts of ~ligiti7P~
image data are rapidly and t:rr~ciel,Lly available to the user in co"ll!uLel friendly fOll,lal~.
These and other advantages of the present invention will become more a~pd~llL from ehe following ~lPt, iled description taken in conju~ ion with the illu~LIdlive embodiments shown in the accom~allyillg drawings.
Su~ (d~ ~ of the Il-v~lll;o,l The present invention relates to an improved airborne, direct digital ~anor~ ic camera system and method in which an in-line, electro-optical sensor eli...i..~lr~ the need for photographic film and film transport appaldLus normally associated with prior art airborne 25 reconn~ Anre cameras and yet still retains the very high image resolution quality which is so important in intelligence operations and in commercial geographic information systems (GIS), mapping and other remote sensing applications. The present invention provides a simpler, more ef~lcient and less costly panoramic camera by lltili7ing a simplified optical system in conjunction with the electro-optical line array sensor W~ ,ill the lens can be 30 simpler and less expensive because it esselllially requires quality focus in only one dimension and in only one place. The lllas~ive amounts of digital data genE,dted by the camera are compressed, and any motion in-11lcecl or panoramic errors are easily correctable in the digital data while such errors were nearly impossible to correct in film images. The culll~l~ssed digital image data may be stored and retrieved later for utilization in conlpu~el type networks 35 or altellldlively can be tr~n~mittP~l from the aircraft to a ground station for prompt lltili7Ation.

W 09613~03 PCTAUS9~0677 Brie~ Desc~iption of the D~wi..~i FIG. 1 is a sch~om~ti-~ pictorial diagram of one embodiment of applicants' improved airbor~e direct digital panoramic camera.
FIG. 2 is a geomebric optical diagram illu~LldLillg pixel design slim~n~iQns of S applicallts' improved airborne panoramic camera system.
~ IG. 3 is a partial block diagram of the ~r~ d embodiment of applicants' ~ v~d airborne pallol~l,ic c~m~r~.
FIG. 4 is a ~c.~e~;live view of an slecbro-optical in-line im~gin~ device usable in applicants' il,~l~ved airborne panoramic camera system and method.
10FIG. 5 is a top view of an improved in-line elecbro-optical sensor in accordance with a,~oll.el aspect of applicants' invention.
FIG. 6A is a top view of an alternate embodiment of an illl~lvvt;d in-line elecbro-optical sensor utilizable in applicants' improved airborne digital l.anol~llic camera.
lFIG. 6B is a perspective view of another ~lt-~rn~te embodiment of an electro-optical 15sensor utilizable in applicants' improved digital panoramic camera system and method.
lFIG. 7 is a pictorial l~lcs~ l ion of a portion of the improved in-line pixel array of applicants' electro-optical sensor utilizable in accordance with applicants' improved digital panoramic camera and method.
FIG. 8 is a logic block diagram illu~lld~illg p,illci~al system components of applicants' 20improved adirborne digital panoramic camera system and method.
FIG. 9 is a partial pictorial illustrating the picture or frame relationship generated in the operation of applicants' improved airborne digital pallo,~llic camera system and method.
FIG. 10 is an exp~ntle~ pictorial diagram of the multiple frame interrelationship of the video data generated in accordance with applicants' improved airborne panoramic camera and 25method.
FIG. 11 is a logic block diagram illustrating the overall system operation of applicants' improved airborne direct digital l,a.lol~llic camera and method.
FIG. 12 is a logic flow diagram illu~Llalillg another embodiment of applicants' improved airborne digital pallolalllic camera and process.
30FIG. 13 is a schl~m~tic logic diagram illustrating the overlap operation of a dual port memory utilizable in accordance with principles of applicants' improved airborne digital panoramic camera and system.
FIGS. 14A and 14B are graphs illustrating pixel calibration data utilizable in accordance with another aspect of applicants' invention.
35FIG. 15 is a block diagram of a dynamic calibration system utilizable in accordance with another aspect of applicants' improved airborne digital panoramic camera and method.

W 096/38003 PCTnUS96/06771 FIG. 16 is a graph of platform attitude parameters utilizable in accoidallce with a dynamic platform stabilization apparatus and process in accordance with another aspect of applicants' invention.
FIG. 17 is a block diagram of an error correction system and method for the improved S sensor arm positioning ~ çh~ -l utilizable in accordance with another aspect of applicants' inventlon.
FIG. 18 is a graph of a portion of the ele~;L,o,-,~gnrtir spectrum illu~L~dlulg remote multi-spectral data utilizable in accor~,ce with another aspect of applicants' improved airborne digital ~allolamic camera.
FIGS. 19 and 19A are a srll~m~tir logic illustration of data storage utilizable in applicants' improved airborne direct digital panoramic camera system and method.

W 096138003 PCTnUS9~D6772 Detafiled l[)escription In today's rapidly ÇhAn~ g world of instant co.. ~ tions, governmlont~l agencies, privalte b~ es and the news media require more intelligence in ever more detail and from more remote locations thus making the efficient, flexible gathering of image illro"~la~ion 5 more critical every day. l?allol~Lic airborne cameras are a key to these new mission requLrements due to their ability to collect, in a very short time, massive data on geographically di~ ed areas at very high image quality. For example, at IQ-7, a ~)alLIOlal~LliC camera at an ~ltih~ P of 5000 feet in one hour can collect imagery data cov~
more than 500 square miles which yields over 87 gigabytes of com~,~ssed digital image data.
This is the equivalent of 87,000 3-1/2" floppy disks which, as will be recognized by those skilled ~n the digital memory arts, is a massive volume of data, and the extremely high data rates imvolved present sub~ ial problems in terms of system design, cost and imple~ .~e~ ion.
ReÇel,...g now to FIG. 1, there is srllPm~t~ y shown an improved direc~ digital 15 airborne panoramic camera system and rnethod in accordance with applicants' invention. As shown ïn FIG. 1, an aircraft 10, flying at an altitude of 5000 feet, would yield a bowtie photo frame 12 that is a~l~o~ ately 1774 feet long and 10,000 feet wide. As will be more fully described h~l~,arl~l in connection with FIGS. 16 and 17, aircraft 10 incorporates as part of its navigation i.~L ~---ents a global position system (GPS) receiver, not shown, which 20 receives n~vigational data from a series of orbiting satellites 14 which are part of the global positioning system which are well known to those skilled in the art of navigation.
The GPS navigation system permits the exact location of the aircraft in terms ofl~tit~ e and longitude when a picture is taken and as will be hereinafter described to accurately ~ , .";"-~ and correlate the position of the sensor arm 16 of camera 18 shown in 25 FIG. 3 in relationship to a fixed position on the earth.
In order to be commercially competitive, applicants' improved airborne direct digital panoramic camera system and me~hod must be equivalent in pe.r~ lallce to the ~;ull~,lllly available film systems. While co..l~alison belweell film and electro-optical cannot be exact, set forth below in Table 1 is a set of pelro...lallce parameters based on design and system 30 tradeoffs.

W 096138003 PCTnUS96/06771 Design Objc~liYes System Parameters Image Quality at 5000 AGL IIRS-7 ~ Nadir _5~
Velocity at 5000' 300 Knots or less Sun Angle Local Noon +2 hours Modes of Operation 10% and 55 % Overlap Sll~t~in~hility > 1000 Frames of Data Table 1 As illu~ led schem~tir,.lly in FIG. 1, the above-listed p~ lf t. - ~. yield a bowtie photo frame 12 of approximately 1700 feet long and 10,000 feet wide for an aircraft flying at an altitude of 5000 feet at 300 knots or less. The number of picture elemf rlt~ (pixels) in a frame 12 can be c~lclll~te~l as:
Pixels/Frame = (Frame WidWGSD) x (Frame Length/GSD) where GSD is the ground's sample fli~t~nre as illustrated in FIG. 2. Image Quality (IQ) >7 implies a GSD of no more than 1/3 of a foot. Therefore each frame of data will contain at least 119,340,000 pixels, 26. Thus the average data rate can be calculated as:

Data Rate = Pixels P/Frame x Frames P/Sec x Bytes/Pixel Bytes/pixel is measured after cc.,llpr,_ssion and the frames per second is determined by 25 the mission parameters. For a ~-.i~.i...-.... set of re~luh~ ~, 0.25 bytes/pixel will be stored and frames or pictures will be taken as often as every 2 seconds with 0.5 frames per second.
This yields a .. i.. i.. -- data rate of 15 megabytes per second.
Turning now to FIG. 3 there is shown a partial breakout of an the airborne panoramic camera 18 having a rockably mounted sensor arm 16 which houses the lens and optical train, 30 not shown. Mounted on top of the rockably mounted sensor arm 16 is an electro-optical assembly sensor 20. There are a number of c~ llelcially available airborne panoramic cameras, however in the ~l~r~lred embo~liment, applicants have utilized an optical system of an airborne panoramic camera built by Hycon and used by the U.S. military for several years. Panoramic cameras generally take pictures over a 90-degree angle and each frame 35 represents a :~45 degree by a _ 10 degree look at the ground. As the aircraft moves forward along a line of flight 22, the sensor arm 16 sweeps from starboard to port in about 0.5 seconds thereby llal~relli~lg or gene~ating the image signals to the electro-optical sensor.

W 096/3~003 PCT/u~-~J-~77 This imode of taking picture data in a c~ross flight line direction is commonly called a whisk broorn tecllni-llle.
As is known to those skilled in the photographic arts, airborne panoramic cameras, for e~cample the KA-54A, KA-56A and KA-60 m~m-f~ct~lred by Fairchild, the KA-SSAm~mlf~hlred by Hycon and the KA-80A m~mlf~ctllred by Itek, incorporate an image motion conll,e"salion aMC) feature, not shown. The IMC virtually ensures that the picture is not blurr~d by aircraft movement because the forward motion of the aircraft in flight is removed or cony~e.L~aled for by the image cul-l~el~ali~n feature. In general, the image col~ elL~ation feature moves the lens forward before i~ a picture-taking cycle and during the picture-taking period the lens mûves aft as the aircraft moves fOl~ald thus culll~ n~ x for the airc] aft motion during frame exposure.
As is known to those skilled in the airborne ~al~l~nic camera arts, the relationship of the image colll~el~alion rate, aircra~t velocity, aircraft height and aircraft velocity are importallt ~alallle~ls in clelf....i..i,-g the pixel data rates.
The image cu~ e~ lion rate aMC) may be ~A~l~ssed as:

IMC = Focal Length x (Velocity/Height of Aircraft) R~eferring again to FIG. 2, one of the most important parameters of the electro-optical 20 sensor design is the deterrnin~tion of the a~r~ iate ground sample rli~t~nre (GSD) 24. As shown in FIG. 2, the ground sample tli~t~n~e 24 is si~lnply how much ground each picture element (pixel) leplesellLs in a frame. The conversion of the GSD to pixel pitch (~ min~
for siImplicity the pixel size for square pixels) may be stated by the following forrm~l~

Pixel Pitch = Focal Length x (Min GSD/Altitude) Therefore in applicants' pl~r~lled embodim~nt, any sensor with a pitch less than 20 microns would produce an Image Quality (IQ) of 7 at an altitude of 5000 feet. In applicants' ylc~r~lled embo~lim~nt, the pixel pi~ch of the sensor is 13 microns, which at an altitude of 5000 feet yields a GSD of 2.56 in. and 3.62 in. at nadir and ~45 degree look angle, respec~ively. This is equivalent to Image Quality 8.
Witn a pixel pitch of 13 microns the line rate can be c~ tP~l as:

Line Rate = (Cross Track Width/GSD)/Scan Time The ,u~,fell~d embodiment of applicants' improved camera has a scan time as small as 0.5 seconds and therefore at an altitude of 5000 feet has a cross-track width of 10,000 feet. From the above equation it can be seen that this yields a line rate of 73,631 lines per CA 02221683 1997-ll-20 W O 96/38003 PCTnUS96/06771 second. Kl~willg the line rate ~ 7 the c~lc~ tion of the pixels per port and the clock rate for the sensor from the following formula:

Max Pixels/Port = Clock Rate/Line Rate Thus ~llming a reasonable clock rate of 12.5 MHz, this would yield a m~ximllm pixels per port of 170. This implies that the line array sensor is preferably segm~ tecl into sections of length of 170 pixels or less. In building applicants' ill~pLuv~d camera, a number of design tradeoffs involving costs/service were made between scan time, clock rate and ms.xi.. ~.
10 pixels per port. In the plere.,~d embodiment of applicants' improved c~m~-r~ the scan time is in the order of .75 seconds, the clock rate is in the order of 12.5 MHz, and the m~ximllm pixels per port were set at 2S6. These values yield a corresponding line rate of 49,100 lines per second. This slight slowing of the camera's scan time allows very useful tradeoffs and facilitates using less costly, commercially available sensors.
Referring now to FIG. 4, there is shown an electro-optical sensor assembly 20 which is de~i~n~d to be mounted on the optical sensor arm 16 and become an integral part of the optical assembly, not shown. In this ,l,&llllel the electro-optical detector is fixedly mounted at the top of the optical path thus facilit~tin~ the m~int~ining of focus during flight operation.
In prior art panoramic cameras, failing to keep the film flat and thus at a fixed dict~n~e from the lens was a major source of errors. Applicants, by fixedly mounting the sensor 20 to the arm 16 and thus with respect to the lens, elimin~t~ a major source of so-called panoramic type errors.
The electro-optical sensor assembly comprises planer mounting board 30 which maybe made of any incn1~ting material, for example a glass epoxy type. Mounted on board 30 are four time-domain integration lined arrays 32, 34, 36 and 38. There are a number of commercially available TDI line arrays which are suitable for use, including the Dalsa 2048 x 96 sensor elements from Dalsa, Inc. in Waterloo, Ontario, C~n~ The Dalsa 2048 x 96 sensor line array ele~n~nt~ are a 13 x 13 micron pitch and can be o~e,d~ed at a 15 megahertz clock rate. An al~"~live TDI line array is commercially available from Loral Fairchild which includes a 1024 x 128 element with a 15 x 15 micron pixel pitch. Both ~e Dalsa and the Loral arrays are front side ill~ d charge coupled (CCD) devices and im~ging is accomplished by integrating photons which must pass through one of more levels of polysilicone. Thus the reslllting image is heavily filtered in the blue and green response of those imagers. The plain sensor array assembly 20 further includes a plurality of power and control connections 40, a digital timing and control processor 42, and thirty-two channel video ~ ,plifiers 44. Preamplifiers 44 on the sensor array assembly perform the first step of video signal conditioning and may be any of the commercially available preamplifiers used, for example an emitter follower circuit MMBT 3904, m~nllf~r~lred by Motorola. As W 096/3~003 PCTJu'~ 771 shown in FIG. 4 the electro-optical sensor assembly 30 preferably includes the sensors 32, 34, 36 and 38, drive electronics 42 and plcd~ lifiers 44 to precondition the image signals.
The .~,;..;,..~1 inter-array spacing is desirable because it facilitates the use of relatively short interconnPcting wires, thus facilit~ting impedance ~ c~ g of ~e CCD clocking signals 5 which helps in m~int~inin~ the low signal-to-noise ratio required when dealing with low-level signals and the short integration times, as will be (1i~c~ e~1 helcil~rl~l.
Design of the sensor assembly 30 is in a large measure ~lirt~te(l by the active picture area of the airborne camera. Airborne panoramic cameras generally have an active picture area of 4 inches, and thcl~ore an ideal sensor array would use a 4-inch line array with somPtllling in the order of 2000 pixels per inch. While an in-line array on the order of 4 inches is certainly co~ ;ially feasible, the actual design of applicants' i~l~rov~d sensor array is a tradeoff lltili7ing cost, number of ports and clock speed as the primary determin~nt~. In the ~l~r~led embodiment of applicants' improved electro-optical sensor, 4 Dalsa 2048 x 96 el~m~nt TDI line arrays 32, 34, 36 and 38 are utilized as shown which 15 gives an active optical area of four inches. Obviously for other applications a single one-inch line array, or alt~ ivt;ly up to seven or eight line arrays in straight line or staggered parallel lines, could be utilized for dirre,~llt applications, as will be understood by those ~,killed in the photographic and airborne ,~co~ s~nre arts.
It should be understood that the one-inch line array sensor will result in a subst~nti~lly 20 less resolution or picture quality and the seven or eight-inch sensor would be colllpalable to a framing camera operation which could be utilized for .l.apping.
In another embodiment of applicants' improved airborne panoramic digital camera and method, the electro-optical sensor 30 or one or more of the line arrays 32, 34, 36 or 38 of FIG. 4 or line arrays illustrated in other embo-lim~nt~ may be replaced by or supplemented 25 with an additional infrared sensor for various applications. The structure and function of commercially available infrared sensors, for example HgCdTe (Mercury C~ millm Telluride) are well known to those skilled in the remote sensing arts. By lltili7ing various sensors, detectors or filters alone or in combination facilitates remote sensing of radiation from 0.3 microns wavelength up to about 14 microns wavelength. In general, multi-spectral imagery 30 data is collected in several discrete bands, as illustrated in FIG. 18, to obtain good quality da~ .
Referring now to FIG. 5, sensor line array overlap, as illustrated in FIGS. 4 and 5, will be described. Four electro-optical sensors, for example of the Dalsa type each c(Jmplising a 2048 x 96 element TDI line array are mounted on a glass member substrate 50 35 which is, for example 1.5 inches long by 0.5 inches wide. The pixel line array length of the illustrated Dalsa sensor is one inch with in the order of 2000 pixels per inch. The overlap 52 as shown in FIG. 5 is n~cess~ry in order to avoid gaps in picture line data which if uncorrected would be a major source of error in the picture data. The optical sensor 20 W 096l38003 PCTnUS96/06771 moves as the optical colllmn scans the ground scene for each frame. If a single line of pixel elç-mP~t~ were employed this movement would not present any optical ~imrultiPs in the resnlting picture data. However with the use of Timed Domain Integration (TDI) it is critical that the line arrays be oriented p~.~elldicularly to the travel of the camera scan. As is known to those skilled in the art of airborne photography this can be accomplished through the ~r~,iate positioning of the rLx lules and the use of optical correction cl-~-;x..-, to be more fully explained hereinafter. In applicants' improved airborne digital pd~ lliC camera a new picture (frame) is~t ken every 1/75 of a second. With 2000 pixels per sensor el~-mPnt and four sensor elPm~nt~ there are thus 320 million pixel signals per 10 second ~.~.al~d by the operation of the airborne camera. The analog video signals are read from the sensor array 30, as will he,cillarlel more fully be described, via four sets of 8-channel graphic data outputs 44 from each Dalsa sensor line array at an effective clock speed of 28 mPg~hPrtz. A new picture scan is initi~tPcl every 2.25 seconds. As shown in FIG. 5 there is a pixel line seam s~aldlion 52 of approximately 3/4-inch, which for the 15 Dalsa line array is equal to approximately 1500 pixel widths. There is also a sensor line overlap of approximately one millimPtPr ~ ali"g duplicate pixel data on both sides of the seam 52. The seam s~a,alion is a potential error source which requires control of the scan arm velocity VO to within 0.1%. As will be hereinafter ~ c~ e~l in co-l.-e-;lion with FIG.
11 the post proce~ing of the seam data uses the overlap data to calibrate the actual velocity 20 of the scan arm. The post processing uses a Fourier type analysis in two dimensions to calculate the actual velocity errors which can be used to col-~pe"sal~ or correct the graphic or picture data.
R~fe~ g now to FIG. 6, an alternate embodiment of applicants improved electro-optical line sensor array 60 is shown which is particularly useful in multi-spectral 25 imagery of agricultural crops. For years the U.S. Agriculture Depa,l,ne"l has undert~ken a number of research projects to explore the feasibility of developing improved remote multi-spectral sensing to identify and quantify various ground conditions which would be useful in improving land use. Remote spectral sensing in agriculture is concerned with a clel. . . ~ ion of various parameters and characteristics of crops through an analysis of data 30 taken at a tii~t~nre. Remote multi-spectral sensing in agriculture has the broad objective to increase the yield and quality of agricultural cultivation and decrease losses in production which are due to disease or weed or insect infestation thereby increasing the yua"lily and quality of agricultural production. As will be hereinafter more fully described, certain portions of the electromagnetic ~ecl,.l", are particularly useful in agricultural multi-spectral 35 im~ging. A biological mass or other properties of certain vegetation can be estim~terl as a ratio of the infrared to the green-yellow spectrum. An improved electro-optical line sensor assembly 60 illustrated in FIG. 6A and 6B would be useful in such agricultural remote multi-spectral imagery of crops. As shown in FIG. 6A there are two electro-optical line CA 0222l683 l997-ll-20 W 096/38003 PC~nUS96/D6771 arrays 62 and 64 which are overlaid by color filters 66 and 68. As hereinabove described, the color filter on sensor 62 may colllplise an infrared filter while the filter on array 64 might be a green/yellow filter thus yielding the picture data useful in d~ the biomass of certain crops overflown by a plane e~luipped with a detector similar to that in FIG. 6A.
Al~e. . ~ /ely~ as illustrated in FIG. 6B, a single sensor 36 could be utilized in conjunction with a colQr wheel 63 which when rotated would generate output data taken through the respective filter se~ e~ 65. Elecho-optical sensors have dynamic ranges which are greater than that offered by film and Lllelerulc applicants' improved electro-optical sensor has a spectr,al ~s~ Lse which offers distinct advantages in such applications as crop surveys and ol)el~d~ g under hazy or less than ideal conditions.
Prior art film-based pallolalllic carneras have x~ which are set by an ~ulc,lllaLiC
exposure control system with shutter speeds of between 1/100 to 1/3000 of a second being select~hle. If a single elechro-optical line array were lltili7PCl~ the coll~spol~ lg shutter speed for the modified prior art camera would be 1/49,100 second which would be too fast to obta~in reasonable signal-to-noise ratios in the video data. Th~,~rol~ in applicants' ~ r~ d embodiment the effective shutter speed is lowered through the use of time domain ~ eglalion (TDI~. As is known to those skilled in the electro-optical sensor arts, commercially available ~electro-optical line sca~ c-l~, such as those referred to above by Dalsa and Loral Fairchild, utilize TDI to lower the effective shutter speed. In FIG. 7 the length of the pixel line is divided into 2000 rows with each row co.~ ;.. il.g 100 pixels. In operation the charge from pixel 1 of row 1 (Pll) is clocked or lldl~r~ d to pixel 2 of row 1, i.e. the adjacent pixel (P12). The velocity of the scan arm V with 100 stages (i.e. pixels in each row) is thus V
~tl %. Thus the incoherent integraLtion accomplished by the TDI effectively increases the exposure time in this example by a factor of 100, thus increasing the dynamic range of the 25 sensor and compen~ting for variations of the individual pixel output. For a complete underst~n-ling of the TDI incorporated in the Dalsa sensor, l~felcnce may be had to the Dalsa technical lil~ldlule for the electro-optical sensors available from Dalsa, Inc., Waterloo, Ontario, ~n~
As shown in FIG. 7, a source of clock pulses is co~n~ctrd to ~e~ l 70 and at 30 s~ccessive clock times the ~ccllm~ tecl charge for example from the first pixel, Pll is clocked or transferred to the ~ rent pixel P12. Thus with the illustrative hundred stages of TDI, the incoherent hl~ ,ldlion effectively increases exposure time by a factor of 100 thus increasing the dynamic range of the sensor and c~ elLsdlillg for any variations in the individual pixel outputs. As is known tv those skilled in the electro-optical sensor art, 35 commercially available scanners commonly utilize a TDI type principle to achieve these desirable features.
Referring now to FIG. 8, there is shown a logic block diagram of the applicants'improved airborne digital panorarnic system and method. As illustrated in FIG. 8 applicants' W 096/38003 PCTrUS96/06771 improved system and method includes four major components: an electro-optical sensor 80 which as shown gencla~s a bowtie photo frame approximately 1700 feet long and 10,000 feet wide at an aircraft altitude of 5000 feet (above ground length AGL), a data h~n(lling unit 82 which calibrates the electro-optical sensor and fligiti7~5, coll~ ,ses and stores the video data in a high speed buffer memory, a controller unit 84 which includes a colll~ulcr to control the overall calibration and operation of the system and real time archive unit 86.
The data h~ntlling re~luil~.llcllL~ for applicants' improved airborne direct digital ~allol~llic camera and method are quite high. The output of the camera is more than 450 megapixels per second. Utilizing a ten-bit analog-to-digital converter, as will hereinafter be described, the data rate increases to over 550 megabytes per second. To put this number into pcl~ec~ ;, a single page of printed text is in the order of 10,000 ch&lacLcl~ or bytes of data.
Applicants' irnproved camera thus can be viewed as clcalillg the equivalent of over 40,000 pages of printed text every second. Applicants ~~h~llenge in ~l~cigning their improved ahl,ollle digital ~allolalnic camera and method was to deal with this very high data rate in a cost effective, commercially colllycliLi~e n~al.,l~l which meant that the data rate had to be subst~nti~lly reduced through the system in order to be ~;olll~aLible with current commercially available recorders having up to 32 megabytes per second inputs, as will hereinafter be more fully described in connection with FIG. 11. Applicants ~ler.,ll~,d system embodiment employs a number of data rate reduction sch~m~s to effectively reduce the camera data rate of over 550 megabytes per second peak to the data rate in the order of 32 megabytes per second which can be h~ntll.od by current state-of-the-art commercially available data recorders. The first major data reduction process involves utili7~tion of the camera's duty cycle. Although each frame of data is taken in about 0.75 seconds, pictures normally will not be taken any more often than every 2 seconds. Thus by employing applicants' leveling buffer, the graphic data after being ~ligiti7P~l is stored in the buffer at the 0.75 second rate and read out every 2 seconds thus yielding a 2.85: 1 reduction in the data rate. The next step to achieve reduced data rates is to employ data c~lllplc~ion techniques. As is known to those skilled in the data h~n~lling arts, there are a number of commercially available cc,lllp,~s~ion circuits for digital signals. For example, the Joint Photographic Experts Group (JPEG) culn~ ,sion circuits are available which yield col~ ,ssion ratios in the order of 5: 1 in the data rate. Thus data colllplession would yield in the case of applicants' l~lc~fell~d design that there would be 2 bits per pixel after culllplcs~ion as opposed to the original 10 bits per pixel. An additional reduction in the data rate is accomplished by utili7ing a process of calibrating the individual pixels. As will hereinafter be described, a look-up table is utili_ed to map the sample values from each individual pixel into a corrected value utili7.ing a log conversion. This is similar to the gamma correction sometimes utili_ed with film. As is known to those skilled in the panoramic arts, the calibration of the output of individual pixels in each line is required either during the collection of the data or in the post-collection W O9~r38003 PCTAUS96J06771 proces~ing procedure. Without the individual calibration of the pixels, lines generally would appear in the im~in~ making the pictures very streaked or bandy. These lines would also appear as data fedlul~s to the col,lplession algoliLlllll which would have to work very hard to k.eep them in the picture. Therefore by calibrating the pixels before cu~ s~ion, there S is an additional advantage that the ec,lll~l~ssion function will be more ~rr~;Live and further reduce the data stream rate. In the yl~r~llcd embodiment of applicants' hll~roved airborne panoramic camera, the JPEG algoiill..., is employed in the data co,l~,c;ssion function. A set of JPEG data colllpl~.,sion chips are commercially available from LSI Logic, Inc. (L64735 QC~35 Discrete Cosine Transfer and L64745 QC-30 JPEG Co~ J,cs~ion). In the data 10 redllction processes as described above, the composite reduction is in the order of 14:1 and thus the output of the data h~n~lling Imit may be recorded on collull~cially available m~gnPtir tape or disk. As will be he.eil~rLelr more fully described, the data h~nrllin~ unit colll~ises a data procec~in~ unit, for example of the Intel 486/66, with a~pl~liate sorlwdl~ programs to provide setup, m~i..l~n~ e and overall operation control. The two major modes of the 15 dat~ proces~in~ unit are (a) single frame, where a single frame of data is taken and displayed on a local monitor, and (b) operation where multiple seq~lenti~l frames of data are taken conti~uously and stored on m~gnPti~ tape, e.g. tape unit 88 or disk 89. The real time archive unit receives digital data from the data h~n~lling unit at rates up to 32 megabytes per second.
The unit, for example, TSC/STX Interface m~mlf~rtllred by Tri-Plex Systems, Inc. Tape 20 and disk units 88 and 89 respectively may be of any type co~ ;ially available, for example, the Sony Dir 1000 Tape unit which employs the standard D-l tape in which the tape c~el~e stores approximately 100 gigabytes of data which in applicants' hl.ploved camlera would be more than 1000 frames of data or an IBM 3350 type disk unit.
Turning now to FIG. 9, there is shown a pi~;lo~ial ,e~,cse..~ ion of a portion of an imaged ground strip 94 on which the path of flight 96 is inriic~tPcl by an arrow. The sequential bowtie shaped photo frames are shown along the flight path and numbered Fl through FR which as hereinabove described are taken, for example, every two seconds during normal reconn~i~s~nre flight opera~ions. As described in connection with FIGS. 1 and 8, a GPS receiver in conju,l~;lion with otller normal navigation instrumentation, not shown, can give the exact latitude and longitude for the start, begi,ll-illg or center of each frame in relationship to the GPS coordinate system. As is known to those skilled in the mapping and surveying arts, the U.S. National Geodetic Survey data sheets describe the location of known or f~ed survey points and such data can be used to ,efe,e~ce the aircraft data to such fixed points on the earth.
As described above in connection with FIGS. 1 and 8, the data stream out of the camera is in the order of 450 megapixels per second. The lltili7:ing a 10-bit analog-to-digital converter m~gnifiPs the tligiti~P~ data stream to more than 550 megabytes per second, which is roughly equivalent to over 40,000 pages of printed text every second. Panoramic cameras W 096/38003 PCTIU~ C771 generally have the ability to collect massive amounts of data in a relatively short time. For example, at IQ-7 the KA-55 in one hour can collect imagery data covering over 500 square miles and generate more than 87 gigabytes of imagery data. This is equivalent roughly to 87,000 three and one-half inch floppy disks. As is known to those skilled in the co~ .uLel S and data procçecin~ arts, ul~ulLing massive amounts of data into a colll~,uLt:l system is a time-con~ il.g and basically inefficient process. In addition, the massive data stream of applir~ntc' improved airborne digital pallo~ lic camera system and method is colllplcssed using for example the JPEG chip set. As is known to those skilled in the arts, in order to utilize coll~ ssed data it must be decolll~ ,sed, that is in effect run through the decoul~l~ss chip set or similar software routine to restore its original noncolll~,lcssed state. Since the hlrollll,lLion from applicants' improved airborne digital camera and system will go in the oldil~ly course to multiple users, it is desirable that collvcl~ient data address systems be developed to permit the user to access the exact portion of the data which is relevant to their particular current needs.
As will be fully explained hcleil~rL~l in connection with FIGS. 11 and 17, knowing where the sensor arm is pointing in terms of exact latitude and longitude coordinates is important in accordance with anoLllel aspect of applicants' invention, basically by using the 'dirrcl~llLial GPS data to pinpoint the exact position of the aircraft GPS antenna to il centim~ter. This can be utilized to correct errors and to simplify addressing specific portions of the imagery data.
Referring now to FIG. 10, there is shown a pictorial representation of the rows and columns which make up a picture width and length respectively. As hereinabove described, each of the eight row segments, row 1, row 2 . . . row 8, includes the data from 64,000 pixels divided into eight 8 by 1000 sub-blocks. Each picture has up to 4000 rows and therefore 32,000 sub-blocks, 100 are illustrated in FIG. 10. It would be possible to address the image digital data segments 100 by a standard row column address scheme as is well known to those skilled in the collll,uLel memory arts. However in applicants' improved airborne panoramic camera and system, it is possible to address a sub-block 100 by its geo-lerelence, i.e. the latitude and longitude coordinates determined at the beginning, edge or center of a frame. As will hel~il~rLeL be described in detail in connection with FIGS. 11 and 16, geo-lefelc~llce of sub-blocks of data by latitude-longitude cooldil~L~ data will permit the user of applicants' camera and method to access, decolllpress and load into the COlll~uL~l system only those sub-blocks as small as 8000 pixel bytes, desired and then expand from there rather than loading an entire data base and accessing it in a more conventional and less efficient manner.
Thus in accordance with another aspect of applicants' invention, the sub-blocks 100 may be individually addressed in the co~ ssed data by specifying the latitude-longitude WO 96/38003 P~JV~ 6771 coo,d~ale data at the begi~ g of the frame and cle~i~n~ting which of the eight sub-blocks are of interest.
Rt;r~llhlg now to FIG. 11, there is shown an oveMll logic block diagram of applicants' improved airborne digital pa~ ic system and mrtho~l The electro-optical sensor 80, as S hereinabove described, is a four inch electro-optical line sensor with a 13 micron pixel size and 96 stage time ~oln~in illle~,dlion. The analog signal from the electro-optical sensor 80 is p~L~ aled in parallel channels from the sensor to amplifier 110 where the analog signals are amplified and subjected to a correlated ~ouble sampling. The c~ ,laled double sampling requires two sample-and-hold circuits and a bulJLl~li~ amplifier, not shown, which is available as an integrated circuit from Kodak as product number KASP-120. During each clock period, one of the sample-and-hold circuits is loaded with the "noisy zero" value of the line array, while the other sample-and-hold circuit is loaded with the pixel value plus "noisy zero." Through analog subtraction the "noisy zero" value which l~r~,sellL~ the most ~ignific~nt noise source in the CCD array output signal is el;...i~ r~1. The resnl~ing signal is thell amplified and fed to a stan~ard analog-to-digital co,~.t,_. 112 such as Analog Devices product number AD 773. The output of the A/D CO~ LL~ which may be of anyof the well-known commercially available types capable of ~,ld~ g at clock rate of 15 -megahertz, is fed to the pixel calibration unit and log function unit 114. The pixel calibration unit 114 comprises a lookup table which maps the sample values for each pixel into a corrected or norm~li7efl value. FIGS. 14A and B illustrate a pixel calibration curve where 116 is the desired linear relationship between light i~lle,~ y e~lessed in lumens and the ~umplitude of the pixel analog signal. The actual output curve for each pixel is ~e~ign~tP~l 118 with the cro~sh~trh~l area represenfing the dir~ic.llial plus or minus deviation from the linear desired relationship. This data is developed using a standard light source and measuring the signal output amplitude of each pixel. FIG. 14B shows the output of the calibration log unit in a digital eight-bit value with the desired linear relationship being in-lir~te(l in a curve 120 and the dyn~nic response in-lir~trrl as 122. After each pixel in the electro-optical array has been st~tir~lly calibrated, the values and log function are loaded into the pixel calibration unit which in turn provides data for the calibration unit 114. As will hereinafter be more fully explail~ed, the static pixel calibration table can be mo-lifie~l in flight, preferably with the input of picture data taken during the flight so that the pixel calibration unit can be dyn~mir~lly altered to correspond to the actual mission conditions.
The output of the A/D co~lv~llel 112 is at a data rate of 3.28 gigabits per second and with the 10:8 log function conversion table 114, the output of the pixel calibration unit is at a data rate of 2.624 gigabits per second which is inputted into the dual port memory 126.
The first major data rate reduction occurs by taking advantage of the camera's duty cycle with each frame of data being taken in about 0.75 seconds with pictures being taken on the order of every 2 seconds. By using a leveling dual port memory which may comprise any W O9~/3~3 PCTrUS96/06771 semiconductor memory system having dual, se~d,dlely clocked input and output ports, data are stored in a buffer at the 0.75 second rate and read out at the 2 second rate, thereby yielding a 2.85:1 reduction in data rate. Operation of the dual port leveling memory is shown in FIG. 13 with the data read in oc~;u~ g at tr~ i.e. the be~i,."h~g of the initial 5 2-minute interval. The subsequent frame data is read in 1.25 sec. later at tl and the initial data is read.out at t2 which is 2 min. after it was read in to the leveling memory 126. The output of the dual port memory 126 is inputted into the JPEG coll,yl~sion circuit 128.
There are a number of co"--"e,~;ially available data co",~r~ion chip sets which could be employed in applicants' illl~loved airborne digital panoramic camera, however in the 10 ~r~,r~ d embodiment applicants employ the JPEG chip set of illLe~ ted circuits from LSI
Logic Inc. The LSI Logic integrated circuit chip set was selected because of its very high implem~-nt~tion speed and high image quality with respect to co,~ ,ssion ratio. The output of the JPEG compression circuit 128 is fed to memory control unit 130. With the 5:1 reduction in the data rate out of the JPEG co~ ression circuit, the input to the memory 15 control unit is at a rate of 123 megabytes per second. The memory control unit takes the cc".~,~;ssed data and fOllllaL~ it for the output devices. In the prerell~d embodiment, memory control unit 130 col"~,ises an 8-bit ECL interface to the TRI-Plex interface. The output of the memory control unit may be selectively directed to tape storage unit 132 or L,~...~.,.ill~r modem 134. The function of the memory control unit is to record all camera 20 data on removable memory without adversely affecting the mission planning, e.g. record in real time. To accomplish this, the key palal,ltL~,~ are the m~ximllm data rate and data storage capacity of the memory control unit 130 and tape storage units 132~ A number of tape storage units are compatible with the peak and average data rates encou"lered in applicants' improved airborne digital panolalllic camera system and method. For a typical 25 mission taking pictures every 4 seconds and where 10% of the mission time is taken in execllting turns, the peak data rate of 32 megabytes per second would equate to an average mission data rate of 14.4 megabytes per second. One such col,lllltl~;ially available high speed tape storage unit is available from General Electric in its family of high pelro""a,lce 19 millim~ter Rotary Digital Recorders which include both ground and airborne units.
30 Alternatively, the data from applicants' improved airborne panoramic camera may be directed from the memory control unit 130 to a Ll n~i,nill~ modem 134. Several design options are available for trsln~mitting the video data via ~nt~nn~ 136 to a ground station, not shown. For example, the data ,c~,ese"li"g a single frame or a few frames of data could be segregated in the memory control unit 130 and, for example, reduced from the order of 64 megabytes 35 to the order of 2 megabytes of data. This coll-l,rcssed data stream file could then be tr~n~mitt~d over a standard 54 kilobaud tran~mitter in the order of 6 mimltes which is consistent with worldwide low-cost satellite co~..",~ tions systems.

WO 9613$003 PCT/US96/0677I

In accordance with another aspect of applicants' invention, data from global position system satellites 14 (FIGS. 1 and 8) is utilized to co~ ellsaL~ for panoramic or pointing errors in the imagery data which result from the sensor arm 16 (FIG. 3) not being normal to the ground due to motion of the aircraft 10. As shown in FIG. 11, a GPS Receiver and - 5 Calibration unit 129 receives signals from a plurality of global position system satellites 14.
~Ant~nlt~ 13.1 is mounted on plane 10 and thus through tne GPS Receiver 129 col"~uLes the latitude-longitude coold~lldL~S of the GPS ~nt~nn~ 131, and thus the exact position coordinates of the pixel array can be c~lc~ te(1. As h~cil~rL~l will be described in co~ e~;lion with FIG. 17, the dynamic calibration appaldlus and method may comprise a plurality of airborne 10 GPS receivers in combination with an inertial g~ n~e system to ~ccllr~tely d~l~llllhle the ~ttit~ e of the sensor arm relative to motion in~lce~l by roll, pitch or yaw movements. The dynaLmic calibration setup table data is calc~ te~l using mission setup data from the sensor 80, and static pixel calibration data is utilized to update the Pixel Calibration Unit 114 during missioin setup procedures.
Rerellillg now to FIG. 12, there is shown a logic flow diagram of applicants' improved digital panoramic camera and method. Before operation is inhi~t~tl by a frame start comm~n~l, initial camera calibration and test m~ e and setup procedures, not 'shown, are accomplished. In the camera calibration process various lookup tables are loaded with ~,~rop.iate values in accordance with the mission data, e.g. anticipated altitude, velocity 20 and overldp parameters. In normal operation of the camera the pixel calibration lookup table func~ion would be loaded with static conditions data which are gel~ldled under ideal or laboratory test conditions. To provide better imagery data the pixel calibration lookup table func~ion may be calibrated for each particular flight or mission by generating dynamic pixel calibration data from the camera at the initiation of the flight. The dynamic pixel calibration 25 data would then be compared with the static pixel data and a modified lookup table would be generated which would better define the actual flight conditions.
During the setup procedure, the normal functions of various elementc of the camera system would be tested to ensure proper operation. In the picture-taking mode the next step is to e~:ecute the fixed scan, as hereinabove described for the whisk broom type across flight 30 line calmera involves a sweep from starboard to port in approximately 0.75 seconds thereby Lldl~,rellillg an image through the optical train to the electro-optical CCD sensor. The data OUtpl1t of the camera is in the order of ~50 megapixels per second and as hereinabove described in connection with FIG. 4, the sensor head provides 32 parallel channels for the four line arrays with each having eight output ports. Imagery data is read from the 35 electro-optical scanner in a burst mode with CCD line array having a plurality of parallel output sections and each section having four A/D collvelL.,l." each sampling in a 14.28 mega-sample per second data rate with ten data bits of resolution. The analog signals are converted to digital signals through app,o~liate A/D converter circuits. The next step in the W 096/38003 PCTrUS96/06771 imagery data h~n-lling process is to calibrate the output signal from the individual pixel elem~nt~. For each CCD sensor output (there are 8096 pixels per array) as shown in FIG.
4, a conversion table is created which maps the 10-bit output from the A/D collvt;ll~r to a 8 bit output which is suitable for colllplession. This also reduces the data rate by 1.25:1.
In addition as shown in FIGS. 14 A and B the output of each pixel element is calibrated by creating a lookup conversion table data which depending upon the light hll~llsily and the output of each individual pixel in terms of voltage is calibrated in the dynamic output. As hereinabove described, the ideal operatio~ involves the genel~tion of a dynamic pixel calibration table data which would reflect the actual mission conditions and that dynamic data is used to modify the static calibration data table.
As hereinabove described, the calibration of the pixels is desirable prior to the data compression process in order to elimin~te lines or streaks which may appear in the image data making them almost unusable and further complicating the colllpLession algc~ lls which would treat such lines as picture data. Therefore by calibrating pixels before con~,ession, additional advantages are gained in that the c-Jn,~,~s~.ion is less costly and more effective.
A major data rate reduction occurs by taking advantage of the camera's duty cycle.
Although each frame of data is taken in about 0.75 seconds, pictures will be taken generally no faster than every 2 seconds. Thus by storing data in a buffer memory at the 0.75 second rate and reading it out at the 2 second rate yields a substantial 2.85:1 reduction in data rate.
The next step in the imagery data h~n-lling process is to C~J111~)1C;SS the ~ iti7rd data.
There are a number of commercially available compression circuits, for example the National Imagery Tr~n~mi~sion Format Standard (NITFS) or the JPEG which are commercially available as data compression chip sets. In the pl-,r~ d embodiment, after colllpression it is anticipated that there will be two bits per pixel as opposed to the original ten bits per pixel thus yielding a 5:1 reduction in data rate which when combined with the other reductions, reduces the imagery data rate by a factor of more than 14:1.
The next step is to record the digital imagery data on approp~iaL~ m~gn-otir recorders or similar memory devices. With the substantial data rate reductions described above, there are a number of m~gn.otic recording systems capable of h,.ntlling the imagery data, which as hereinabove described is at a peak data rate of 32 mega bytes/second.
During initial setup, the frame start command triggers the calculation by the global position system receiver, not shown, in the aircraft which c~lr~ ttos the l~tit~ e/longitude coordinates at the time the frame start command is issued. The next step in the process is to record the l~fit~lde/longitude coordinate data for each frame start command pulse which can then be utilized to coordinate or address the storing of the data for each frame. As hereinabove described l~tihl(le/longitude coordinate data facilitates or permits the location of the frame and subframe data when stored. As shown, this l~tit~ e/longitude address data can W O~Ç~3~q~ P ~ AUS96~06771 be r~corded with the recorded data imagery so that it is tracked and easily accessible in that .l.al]llel. For high quality images, as hereinabove described, it is ,.~ce~. y to ensure that the electro-optical sensor's position and pointing is known with respect to the surface of the earth. In order to correct and calibrate imagery data, as hereinabove described, it is 5 desi;rable to monitor the three degrees of motion of the electro-optical sensor. As described in con,nection with FIGS. 15, 16 andL 17, locating a plurality of GPS units on the aircraft and on the ground can be utilized to dt:L~ le not only the exact position of the aircraft and therefore the sensor arm within the ai]rcraft~ but also the roll, pitch and yaw motions of the sensor arm due to turbulence, etc.
This motion correction data is then correlated with, for example as shown in cormection with FIG. 17, the inertial g~ n~e data to give the exact position and of the sensor arm 16 (FIG. 3). By lltili7ing such motion coll~lion data and the data from the same picture taken by the seam or overlap of the four line sensors, the rligiti7~d imagery data can be compared and corrected to elimin~te errors which result when the film or image plane is not kept exactly parallel to the earth's surface during picture or frame operations.
l~ollowing the correction of the imagery data to elimin~t~ ~allolalllic and other types of elTors, the corrected ~1igiti7sd data may be OU~ull~d to various users, for example being ~rcesse~l by the lat/lon cooldinates which were recorded with the digital data at the start of each frame.
Referring now to FIG. 15 there is shown a logic block diagram of a dynamic calibration system in accordance with another aspect of applicants' invention. As hereinabove described, a static calibration table 140 is created by measuring the voltage out of each pixel for a controlled level of ill~ ion as illustrated in FIGS. 14A and 14B. This calibration table data is utilized in correcting output variations between the individual pixel eleml~nt~. In addition to the static calibration, for improved results it is desirable to inco~[porate dynamic calibration during an actual mission flight. The dynamic calibration table l.42 in effect sets the exposure or shutter speed for actual mission conditions. In accordance with the dynamic calibration feature of applicants' improved camera, the initial step is to take a pic~ure as the mission flight begins. The camera data along with the static calibration table data 140 are fed as inputs to the mission setup data controller 144.
The mission data controller 144 calculates the DC offsets and gains for the new picture elemel~t data and conl~alcs it to the static calibration table data and feeds the output to a new dynamic calibration table unit 142. By recalc~ ting the lookup table values and comparing themL with the static calibration table in the COlllpal~Ol circuit 146, the a~propliaL~ offsets for the actual mission conditions are integrated into and reloaded into the static calibration table 140 which then would be utilized as hereinabove described in connection with FIG. 11.
The static calibration table 140 and the dynamic calibration table 142 may comprise semiconductor or disk memory devices, for example MTSC1005DJ m~mlf~ctllred by Micron W O 96/38003 PCT/U',.'JC771 Technologies which include a sepa~ memory location for each of the individual pixel elem~ontc. The mission data controller 144 comprises a procçscing unit, for exarnple an Intel 486, which accepts the input data stream from the static calibration table 140 and the camera data taken from the calibration picture data and pairs them and loads the dirr~lcnce into the 5 dynamic calibration table 142. The c~,l,lpaldtol circuit which, for example may comprise a colll~dlalor algc,liLhlll or an adder circuit of any known type, COlll~dl~,S the input from the StdtiC calibration table and the dynamic calibration table for each pixel, and the dirÇ~lcnce being the variation of the dynamic calibration table 142 from the static calibration table values is reloaded into the static calibration table 140 for that flight.
One of the ~linci~al problems with airborne panoramic cameras is the assullll!lion that the camera pointing angle is fixed with respect to the ground. For any variation in the camera position relative to the normal position to the ground results in pointing errors.
Referring now to FIG. 16, there is shown a graphic plot of the amplitude of variations which occur in normal flight as a function of time. The amplitude variations may be due to the roll, pitch or yaw type movements of the aircraft about the three axis as the plane is in flight due to variations in wind and general air turbulence. As illustrated in FIG. 16, the amplitude variations as a function of time may vary from -10 to +10 units. The graph illustrates several points 150 which lc;~senL the i~ ~ous variation at a point in time due to such variations in upper air turbulence. A similar graph would be made with regard to each of the major axis of the plane and would be indicative of the amplitude of the motion variations due to either roll, pitch or yaw movements of the aircraft about its normal line of flight.
Referring now to FIG. 17, there is shown a pictorial and block diagram of an improved motion cc,lllpellsaLion circuit which can be employed to further calibrate the pixel calibration table and unit 114 to correct panoramic errors in the imagery data. In ~e case of the airborne film-based reconnAicc~nre systems, fixed ground points are commonly used to recalculate the film data. In accordance with applicants' improved airborne panoramic system and method, the ~ ous attitude of the aircraft at any instant in time and in e,ellce to a known ground position enables post procescing to correct pointing errors introduced by the movement of the aircraft about its normal flight path.
As shown in FIG. 17, aircraft 10 has four antennas, 150, 152, 154, and 156 whichco"""~ ir~t~ with GPS satellites 14. In addition, one ground GPS unit 158 is opel~Lillg cimnlt~n~ously. The ground unit 158 can be in radio c~----------i~tion with the GPS 162 and 164 on plane 10 which thus yield real time dirr~.cllLial correction data. The three ADF
aircraft mounted receivers having antennas 152, 154 and 156 track the roll, pitch and yaw movements of the aircraft respectively. The GPS receivers on the aircraft and on the ground respectively through direct triangulation on the three or four GPS satellites 14 develop the latitude-longitude coordinates of the exact position on the aircraft antenna 150 within a few centimeters. By combining the inertial navigation system, shown sch~m~tic~lly in block 160 WO96138003 PC'r~u~ C77 which may be of the inertial platform or accelerometer type, yields a very high accuracy angular position of sensor arm 16 ~FIG. 3) and when interpolated with the GPS data from an~ a 150 yields position accuracy data for the GPS ~nt~ 150 with a ~t 1 cm accuracy.
Thus by using the dynamic calibration system described in FIG. 17, the position of sensor S arm 16 can be calculated to better than 200 micro radians. With the dirr~ ial GPS
processors 168 in combination with the inertial navigation system motion dirr.,.~ ial processor 170 it is possible to c~lr~ tç the position of the sensor arm 16 to a fraction of a radian. Using the dynarnic error correction system described in FIG. 17 it is possible to remove any .~ignifir~nt positional errors in the imagery data allowing the data to be ortho-rectifiçcl to the earth. Thus the ~ laly errors introduced in digital p~lUlallliC airborne cameras, that of a moving focal point, can be corrected using applicants' dynamic calibration system andl method.
As hereinabove described, comparison of remotely sensed spectral response data is useful in many appli~tion~ including remote multi-spectral sensing of agriculture crops.
Remote multi-spectral sensing for agriculture crops is primarily concerned with de~. .--i--;~-g various crop parameters by analyzing spectral data to enable farmers to detect and correct crop conditions due, for example, to disease, weed or insect infection or fertili_er or 'irrigation needs. As shown in FIGS. 6A and 6B, l~tili7ing applicants' improved elecltro-optical sensors and system facilitates the collection of imagery data by lltili7ing a plurality of color filters 63, 66 or 68. As described above, applicants' improved sensor arrays 60 and 62 have a spectral response superior to film systems.
Referring now to FIG. 18 there is shown a plot of spectral bands for blue, green, red, near infrared and infrared respectively which may be generated lltili~ing the combination of applicants' in-line optical sensors and the color filters 63-66 shown in FIGS. 6A and 6B. As hereinabove described, the biological mass or other p~ ies of agricultural crops can be detelmined for example as a functior of the ratio of green-yellow or other spectral data.
Such data is useful to a farmer in detecting crop conditions such as disease, weed or insect infestation thereby permitting the falmer to take ~pl~liate actions to correct or counteract such conditions to increase the quality and quantity of production.
Refe,lil,~, now to FIG. 19, there is shown a diskette 190 having thereon m~gn~ti7~
tracks 192, 194 and 196. With li;felence to track 192, for example, an imagery data file 198 may be recorded as digital bits in a ll.anllel well known to those skilled in the coll,plllel arts.
Along with the imagery data, related lat/ lon coordinate data may likewise be recorded on the sarne track in a combined address-data format or alternatively they may be related through an additional related addressing scheme. For example, as shown in FIG. l9A, by incorporating a tagging approach, related lat/lon coordinate data and the imagery data generate-i by that frame may be associated and recorded at pred~L~llllined addresses shown as 198 or through a scheme of look up tables well known to those skilled in the m~gnPtic and W 095'3~C03 PCTnUS96/06771 recording arts. But in accoldallce with another aspect of applicants' invention, the imagery data file associated with each frame is preferably recorded with or proximate the corresponding lat/lon data as illustrated in FIG. l9A to enable easy access to the recorded imagery data. In this l-~allllel a convenient ~crc~cnce to the lat/lon cool-lhlalc data recorded 5 at the time the frame imagery data was genc~alcd greatly simplifies ~cce~ing the associated imagery da~a. Thus by incorporating this lat/lon data tagging approach illustrated as 198 in FIG. 19, it is possible to access specific frame imagery data without the i-~ces~i~y for deco",~lcssing and ~ealcllillg an entire vo!ulllh,uus data file gen~ldted during a particular operation or mission.
While applicants' invention has been described with lcrerellce to several particular illustrative embodi,llc"ls, it is not to be lc~I,icled by those embodiments but only by the appended claims. Those skilled in the art will al~plccia~c that the illustrative embo~im.on can be changed or modified without departing from the scope of the present invention.
1~

Claims (16)

WHAT IS CLAIMED IS:

1. In a direct digital airborne whisk broom-type panoramic camera having an optical system for focusing a ground area to be imaged onto a focal plane, image motion compensation means for correcting aircraft motion during camera operation, a rockably mounted sensor arm for housing a lens and optical train, drive means for moving said sensor arm across the line of flight of the aircraft during camera operation, the improvement comprising:
electro-optical multi-port line array sensor means for converting images of said ground area to be imaged into analog electric image signals having an initial image data rate in the order of at least 100 million pixel signals per second, means for fixedly mounting said electro-optical sensor means on said sensing arm at the focal plane of the camera, image signal processing means for digitizing said analog image signals, data compression means for compressing said digitized image signals, and memory means for storing said compressed digitized image signals whereby the overall image data rate is substantially reduced before storage.

2. The improved airborne panoramic camera of Claim 1 wherein said electro-optical multi-port line array sensor means comprises a plurality of pixel elements wherein the pixel pitch is in the range of 5 to 20 microns and additionally including buffer memory means for temporarily storing said digitized image signals and for slowing said data rate as a function of a frame duty cycle rate of said camera.

3. The improved airborne panoramic camera of Claim 1 wherein said sensor means comprises at least one group of a plurality of pixel lines arranged in a staggered, overlapping arrangement with at least one of the others of said plurality of lines of pixel elements and wherein the overlapping pixel elements generate like image signals at different time intervals during each fame operation of the camera and wherein said multi-port line array includes a plurality of stages of time-domain integration for each pixel element.

4. The improved airborne panoramic camera of Claim 1 additionally including leveling buffer memory means for storing digitized image signals at a first data rate and for delivering the digitized image signals from the buffer memory at a second data rate which is substantially lower than said first data rate.

5. The improved airborne panoramic camera of Claim 1 additionally including a plurality of optical filter means functionally associated with said sensor means.

6. The improved airborne panoramic camera of Claim 1 additionally including differential global position system means for determining position data for said sensor arm in terms of latitude and longitude coordinates for each picture sub-frame of operation of the camera and data addressing means for tagging digitized image signals corresponding to a particular sub-frame as a function of the determined latitude-longitude coordinate data.

7. A method for generating and processing remote image data from a direct digital airborne panoramic camera having a predetermined frame and picture cycle time of operation comprising the steps of:
generating analog image signals by exposing a multi-output port line array of pixel elements having in the order of 8 to 16 output data ports to illumination from an area to be imaged, converting said analog image signals into digital image signals, first reducing the output data rate of said digital image signals as a function of the frame cycle time and picture cycle time of said camera operations, further reducing the rate of said digital image data by subjecting the digital data to a compression algorithm process, and recording the compressed digital data in a digital memory device whereby the image data rate is substantially reduced before storage.

8. The method of claim 7 further including the step of:
generating a static calibrating data table with unique values for each pixel element, correcting said digital image signals as a function of said static calibrating data for each pixel element, and further reducing the data rate of said digital image signals by utilizing individuals pixel calibration data.

9. The method of claim 7 further including the steps of:
generating static calibrating data with data conversion values for each pixel element, generating dynamic calibrating data with data derived from setup picture data representative of actual mission conditions, and correcting said digital image signals as a function of a differential between said static calibrating data and said dynamic calibrating data.

10. The method of claim 7 further including the steps of:
determining the exact latitude and longitude coordinates for the location of said line array of pixel elements by differential global position system techniques, and designating a memory storage location for digital image signals for each picture line sub-frame of camera operations as a function of the latitude-longitude coordinates data corresponding to the initiation of each respective picture line sub-frame operation of the camera.

11. The process of claim 8 including the additional steps of:
determining the magnitude of any motion of the pixel element array by comparing the differential outputs of a plurality of ADF receivers fixedly mounted on the airborne platform and the output of an inertial guidance system monitoring the motion of said pixel element array, correcting any panoramic errors introduced into the digital image signals by any roll, pitch or yaw movements of the airborne platform or pixel element array, and wherein the step of correcting is within the range of 50 to 200 micro radians.

12. The improved camera of claim 1 wherein at least one of said sensor means comprises an infrared sensing element.

13. The improved camera of claim 12 additionally including a plurality of color filter means for filtering portions of the spectrum of incident radiation passing though said lens means.

14. The method of claim 7 additionally including the steps of generating latitude/longitude coordinate data associated with each frame of camera operation, and storing a portion of said compressed digital data stored in said digital memory utilizing said latitude/longitude coordinate data for a specific frame of compressed data.

15. The method of claim 7 wherein the step of generating analog image signals includes the step of converting image-wise radiation into analog image signals utilizing a plurality of parallel, offset sensor arrays to generate overlap gap image data signals, and additionally including the step or correcting panoramic type errors in said digital image signals as a function of said gap image data.

16. The improved panoramic camera of claim 1 wherein said senor means comprises a plurality of parallel lines of sensor elements arrayed in a stagged, overlapped arrangement, and additionally including means for correcting panoramic type errors in said digital image signals as a function of data from said overlapped sensor elements.