Title of Invention	"A VIDEO CAMERA"
Abstract	A video camera characterized in that picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory; object recognizing and tracking means for recognizing extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory; tracked object initialization means for storing characteristic values of said tracked object; tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first average value; recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating corresponding luminance information so as to calculate a second average value; and exposure control means for controlling an iris and/or an automatic gain controller of said video camera by utilizing said first average value obtained by said tracked object extraction and light measurement means or said second average value obtained by said recognized color extraction and light measurement means and by utilizing a luminance reference value, and for adjusting the luminance of said object so as to be substantially constant.

Title of Invention

"A VIDEO CAMERA"

Abstract

A video camera characterized in that picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory; object recognizing and tracking means for recognizing extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory; tracked object initialization means for storing characteristic values of said tracked object; tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first average value; recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating corresponding luminance information so as to calculate a second average value; and exposure control means for controlling an iris and/or an automatic gain controller of said video camera by utilizing said first average value obtained by said tracked object extraction and light measurement means or said second average value obtained by said recognized color extraction and light measurement means and by utilizing a luminance reference value, and for adjusting the luminance of said object so as to be substantially constant.

Full Text	BACKGROUND OF THE INVENTION The present invention relates to a video camera technique for automatically following a change of an object in the field of view of a video camera and, more particularly, to such technique for controlling the exposure so as to properly adjust the luminance of the object. Numerous methods may be employed for controlling the exposure of a video camera. For example, such methods may include an automatic multi-split light measurement exposure control method, an automatic centralized or spot light measurement, exposure control method, a manual exposure control method and so forth. The automatic multi-split light measurement exposure control method utilizes a multi-split light measurement pattern to measure luminance information obtained from picture information. The measured luminance information may be processed to obtain an average value, a distribution, and/or a histogram of such luminance information which may be utilized to control exposure. The automatic centralized or spot light measurement exposure control method utilises a light measurement pattern which may be respectively located in or around the center portion or a screen ana in a relatively enall or spot area in or around such center portion. Luminance information obtained from picture information is measured by use of the light measurement pattern which may ba utilized to control exposure. In the manual exposure control method, an operator or cameraman miy determine and/or manually adjust the luminance of an object ts be photographed to obtain a desired or acceptable value. Each of the above-mentioned exposure control methods has disadvantages associated therewith. For example, the automatic multi-split light measurement exposure control method may have difficultly in obtaining a proper exposures of an object in a seen saving a relatively large luminance difference between the object and its background (such as in a scene having a backlight or excessive normal light). In such situation, the background information may be incorporated with information of the object to be photographed, thereby making it difficult to obtain a proper exposure of the object. Further, with regard to the automatic centralized or spot light measurement exposure control met nod, ouch method may be difficult to uste when the object to be photographed is moving. That is, in such method, the -operator or cameraman has to photograph the object such that it is always located at the center of a screen. As is to be appreciated, it may be difficult to continuously photograph the object so as to keep it centered on the screen while the object is moving. Furthermore, with regard to the manual exposure control method, an operator may not be able to properly adjust th exposure control in response to a sudden luminance change of an object. Additionally, a problem may arise in that the background nay not be set to an optimum luminance level when an object suddenly disappears from a photographing screen. An object of the present invention is to provide a system and method for properly controlling the exposure of a video camera for an object to be photographed. Another object of the present invention is to provide a system and method as aforesaid in which luminance values are selected from among a plurality or luminance values and in which the selected luminance values are utilized to control the exposure of the video camera. A further object of the present invention is to provide a system and method as aforesaid in which the plurality of luminance values include a plurality of average luminance values each calculated by a respective technique and at least one reference luminance value. in accordance with an aspect of the present invention, a video camera system is provided. Such system comprises a picture information temporary storage device havirg a temporary memory for dividing a video signal obtained from at least one video camera into luminance information and color information and for storing picture information composed of the luminance information and the color information in tne temporary memory at an arbitrary timing; an object recognizing and tracking device for recognizing, extracting and tracking a displayed object from this picture information stored in the temporary metaory; a tracked object initialization device for storing characteristic values of the tracked object; a tracked object extraction and light measurement device for extracting and integrating only the luminance information of the tracked object and for calculating an average value; a recognized color extraction and light measurement device for extracting color information having the same color ac the recognized object, integrating the corresponding luminance information, and calculating an average value; a centralized light measurement device for integrating the luminance information of the object in or around the center portion of a screen and calculating an average value; a spot light measurement device for integrating the luminance information in a relatively small area in or around the center portion of the screen and calculating an average value; a light-measuring method selection device for selecting the luminance information measured by the tracked object extraction and light measurement device, the recognized coioz extraction and light measurement device, the centralized light measurement device or epot light measurement device in accorda.nce with the object: condition; and an exposure control device lor controlling an iris and/or an automatic gain controller of this video camera and adjusting the luminance of an output object to a constant value by using a selected luminance average value and a luminance reference value. In accordance with another aspect of the present invention, a video camera exposure control method is provided. Such method comprises dividing a video signal obtained from a video camera into luminance information and color information and storing picture information including the luminance information and the color information in a temporary memory at an arbitrary timing; recognizing, extracting and tracking a displayed object from the picture information stored in the temporary memory; storing characteristic values or the tracked object; extracting and integrating only the luminance information of the tracked object and calculating a first average value; extracting color information having the same color as the recognized object, integrating the corresponding luminance information, and calculating a second average value; integrating the luminance information of the object around the center portion of a screen and calculating a third average value; integrating the luminance information in a relatively small area around the center portion of the screen and calculating a fourth average value; selecting the luminance information of the first average value, the second avaraga value, the third average value or the fourth average value in accordance with the object condition; and controlling an iris and/or an automatic gain controller of the viieo camera and adjusting the luminance or an output object to a constant value by using the selected luminance average value and a luminance reference value. The luminance information and color intonation stored in the picture information temporary storage device nay be read out and supplied to the centralized light measurement device or spot light measurement device, the recognized color extraction and light measurement device, the object recognizing and tracking device, and the tracked object extraction and lighr. measurement device in accordance with a command signal which may be supplied by the light-measuring method selection device. The recognized color extraction and light measurement device calculates a luminance average value of pixels having information which is relatively close to the recognized color. The objact recognizing and tracking device and the tracked object extraction and light measurement device extract only the recognized object and calculate its average luminance value. The centralized light measurement device or spot light measurement device calculates a average luminance value of a center portion of a screen or in a relatively small area of such center portion, respectively. The light-measuring method selection device determines or selects a luminance average value from among the calculated luminance average values in accordance with the objoct condition or present opera-ting conditions, and further selects a reference value. The selected luminance average value and reference value may be compared by the exposure control device BO as to obtain a signal which may be utilized to properly adjust the luminance value of an object. Accordingly, the present invention relates to a video camera, comprising: picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory; object recognizing and tracking means for recognizing extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory; tracked object initialization means for storing characteristic values of said tracked object; tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first average value; recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating corresponding luminance information so as to calculate a second average value; and exposure control means for controlling an iris and/or an automatic gain controller of said video camera by utilizing said first average value obtained by said tracked object extraction and light measurement means or said second average value obtained by said recognized color extraction and light measurement means and by utilizing a luminance reference value, and for adjusting the luminance of said object so as to be substantially constant. Other objects, features and advantages of the present invention will become apparent from the following detailed description of the illustrated embodiments when read in conjunction with the accompanying drawings in which corresponding components are identified by the same reference numerals or characters. BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS Fig. 1 is a diagram of a video camera ay stem fur performing object recognition and automatic exposure adjustment according to an embodiment, of the present invention; Fig. 2 illustrates a section of an image screen wherein a centralized light measurement device of the video camera system of Fig. 1 performs luminance integrating and average value calculating; Fig. 3 illustrates a section of an image screen wherein a spot light measurement device of the video camera system of Fig. 1 performs luminance integrating and average value calculating; Fig. 4A is a diagram of an image to which reference will be made in explaining light measurement methods; Figs. 4B and 4C are diagrams to which reference will be made in explaining light measurements performed by the centralized light measurement device of the video camera system of Fig. 1; Pigs. 5A to 5D are diagrams to which reference will be made in explaining light measurements performed by other devices of the video camera system of Fig. 1; Figs. 6 and 7 are flow charts to which rnference will be nade in explaining a recognized color extraction and light measurement procedure; and Fig. 8 is a flow Chart to which referencn will be made in explaining a light-maaauring method selection procedure. DESCRIPTION OF THE PREFEREED EMBODIMENTS Preferred embodiments of the present, invention will now be described with reference to the accompanying drawings. Fig. 1 illustrates a video camera system (VCS) which is adapted to perform object recognition and automatic: exposure adjustment, AS shown therein, such vcs generally i.ncludes a lens block section 50, a sample and hold (S/H) and autoiiatic gain control (AGC) circuit 4, an analog-to-digital (A/DJ circuit 5, a luminance and color difference signal generator 6, an image temporary memory 7/ a recognized color extraction and light measurement unit 9, an object recognizing and tracking unit 10, a tracked object extraction and light measurement unr.t 11, a •tracked object initialization unit 8, a cerrtralizaa light measurement/spot light measurement unit 12, a light-measuring method selector 15, an exposure controller 3.6 and a driver 17 Which are connected as shown, the lens block section So includes a lens 1, an iris 2 and a solid pickup or charge cnupled device (CCD) 3. Image pickup light LA from an object is supplied through the lens 1 and the iris 2 to the CCD 3, whereupon a pickup signal SI which represents a field-of-view picture is generated and supplied to the S/H and AGC 4. The K/H and AGC is adapted to sample and hold the received pickup signal si and to perform or adjust the gain thereof in accordance w;.th an AGC control signal S16 from the exposure controller 16 so as to produce A pickup signal S2. Such pickup signal S2 is converted to a digital pickup signal S3 by the A./D converter 5. The digital pickup signal S3 is supplied to the luminance and color difference signal generator 6, The luminance and color difference signal generator 6 is adapted to receive the digital pickup signal S3 and to generate therefrom a luroinance (Y) signal S4, a color difference (R-Y) signal S5 and a color difference (B-YJ signal SB. such luminance and color difference signals are supplied to the image temporary memory 7, whereupon they are respectively preprocessed as an abject tracking detection signal and an object luminance light-measuring signal, such preprocessing may include a thinning out operation in which the number of canplaa of the luminance data are decreased, whereupon the total number of data bits processed are reduced thereby reducing or improving the sampling frequency and/or processing speed. The creprocessed signals are stored in an image memory or memories. That is, three image memories 51, 52 and 53 may be utilized, wherein the luminance picture information is stored in the image memory 51, the color difference (R-Y) picture information is stared in the image memory 52, and the color difference (B-Y) picture information is stored in the image memory 53. such stared picture information may be randomly-accessed by the centralized light measurement/spot light Measurement unit 12, the recognized color extraction and light: measurement unit: 9, the object recognizing and tracking unit 10, and tne tracked object extraction and light measurement unit n. More specifically, when stored picture information is desired, the unit requesting such information supplies a signal, such as an address signal (adi, ad2 or ad3) which indicates the address or location of the desired stored picture information, to the image memories. As a result, the requested information is supplied to the respective unit by way of signal dal, da2 or da3. As an example, consider the situation in which the centralized light/spot light measurement unit 12 Wishes to obtain the stored luminance picture information from the image memory 51. In such situation, the unit 12 supplies a request: or addrefis signal adi to the image memory 51, whereupon signal dal having the requested luminance picture information is supplied to the unit 12. The centralized light measurement/spot light measurement unit 12 receives the requested stored luminance picture information from the image temporary memory 7, as previously described, and generates therefrom an average value of the luminance information contained within a predetermined portion. For example, the centralized light measurement/spot light measurement unit 12 may generate an average value of the luminance information in a center portion 98 of an image screen 97 which is indicated by a cross-hatched or shaded portion in Fig. 2. In this example, the center portion or areta 98 has a siae of X/3 x V/3, in which the horizontal dimension of the screen 97 is X and the vertical dimension of the ac;reen is Y As is to be appreciated, other sizes of such center portion may be utilized. The centralized light measurement unit :.2 supplies the generated average value of the center luminance information (which may be referred to as a centralized light measurement) as an output signal S7 to one terminal of a switch sw;. of the light-measuring method selector 15. An example of the procedure performed by the centralized light measurement unit 12 is illustrated in Figs. 4A. 4B and 4C. That is, Fig. 4A illustrates a image on which the unit 12 is to determine an average value of the center luminance information- The light measured or center portion of such image is indicated as a shaded area in Pig. 4B. The information contained within this shaded area LG processed by the unit 13 eo the centralized light measurement or average luminance levels (light values) as illustrated in :?ig. 4C. The centralized light measurement unit/apot light measurement unit 12 may further generate an average value of the luminance information obtained from the signal dal pertaining to a relatively small predetermined portion or spot of the image screen 97. Such predetermined portion may be at the center portion of the image screen 97 as, for example, indicated by a shaded portion 99 in Fig. 3. In this example, and in a manner similar tn that previously described with refaranou to Pig. 2, the unit 12 determines an average value of the luminance information contained within the center portion or area having a size of X/10 x Y/10 (in which the horizontal dimension of the screen 97 is X and the vertical dimension of the screen is Y). Such area 99 is smaller or narrower than the area 98 of Fig. 2. As is to be appreciated, other sizes of the spot area may be utilized. The spot light measurement unit 12 supplies the generated average value of the center spot luminance information (which may be referred to as a spot light measuremtsnt) as the output signal S7 to one terminal of the switch swi of the light-measuring method selector 15. In response to a request, the object recognizing and tracking unit 10 may receive the luminance information signal dai and the color difference information signals da2 and da3 from the imaga temporary memory 7. The object recoynizinq and tracking unit 10 may further receive an instructing or trigger signal Si7. In response to such signal S17, the object recognising and tracking unit 10 extracts and holds characteristic values or information, which may include position information, of a desired object contained within the image represented by the respective luminance and color difference signals in a hold area of the unit 10. The hue of such object, or recognized hue, is supplied to the recognized color extraction and light measurement unit 9. The tracked object extraction and light neasurement unit 11 may receive an instructing signal S17 and information pertaining to the recognized and tracked object from the object recognizing and tracking unit 10. The tracked objisct extraction and light measurement unit 11 may integrate only the luminance information of the object extracted by the object recognizing and tracking unit 10 and average the same so as to forn an average value S9. Such average value S9 is supplied from l;he tracked object extraction and light measurement unit 11 to one terminal of a switch sw3 of the light-measuring method selector 15. An example of the procedure performed by the tracked object extraction and light measurement unit 11 is illustrated in Figs. 5A and SB. In this example! the center piece of fruit or apple of the image of Pig. 4A is the recognized or tracked object and, as such, is shown as a shaded object in Fig. 5A. The information contained within this shaded area is processed by the unit 11 so as to determine the light measurement or average luminance values &s illustrated in Fig. 5B. Further, in response to the signal S17, 1:he tracked abject extraction and light measurement unit 11 may hold the luminance average value S9 of the tracked object in a hold area of the unit 11 and may cause such average value to be supplied to an object recognized color extraction/object recognizing, tracking and light-measuring reference luminance value generating circuit 13 so that a reference luminance value signal S1Q may be produced therefrom. Such signal SlO, which way represent an average of the luminance at the extracted object prior to auto-tracking, ia supplied Croia the circuit 13 to one tormina 1 of a switch sw4 of the light-measuring method selector :.5. The recognized color extraction and light: measurement unit 9 may receive the luminance Information dal, the color difference information da2 and da3, and the recognized hue information from the object recognizing and tracking unit 10 as previously described. The unit 9 is adapted to process the received information so ag to determine other locav.ions or objects of the respective image (such as that of Fi.g. 4A) which have hues substantially similar to that of the recognized hue from the unit 10. More specifically, the unit 9 extracts information from the color dtfferanca signals R-Y and B-Y so as to determine locations or points having substantially the same hue as the recognized color among the luminance information dal from the image memory 7, integrates the values of the determined points, averages the integrated values to obtain an averaged value SB, and supplies such average value S8 to one terminal of a switch sw2 of the light measuring method selector 15. To avoid or minimize count errors of the luminance of the same hue of noise component(s) in a black object or an environment color in a white object, black and environment colors may be checked. An example of the procedure performed by the recognized color extraction and light measurement unit 9 is illustrated in Pigs. SC and so. In thie axanple, the hue or color of the center piece of fruit (or apple) of Fig, 5A is the extracted or recognized color. Accordingly, tne unit 9 determines other locations or objects within the image of Fig. 4A having substantially the same hue aa that of the apple. AS a result, objects such as the three cherries which have substantially the same color as that of the apple are identified or extracted. These objects are shaded in a similar manner to that of the apple as shown in Fig. 5C. The area of the apple having an oval shape which is not shaded represents an area which may be affected by ao-called "white saturation" or "black crushing". (White saturation may be caused by over-exposure, whereas black crushing may be caused by under-exposure.) The information contained within the shaded areas of Fig. 5C ar processed by the recognized color extraction and light meaaurHinnt unit 9 so as to determine the light measurement or average luminance values as illustrated in Fig. 5D. AS way be observed from Figs. 4C, SB and 5D, the average luminance levels or light values of similar objects (such as the apple) appear slightly different from figure to figure. Such minor discrepancy may be due to somewhat different conditions which may exist when these figures were captured or picked-up (such as different background brightneae;}. The processing performed by the recognised color extraction and light measurement unit 9 will now tie described with to the flow chart illustrated in Figs. 6 and 7. Pig. 6 illustrates the processing in which luminar.ee levels except a black portion and noise are averaged and the presence of an object is determined. Pig. 7 illustrates the processing wherein, in the presence of an object, Ymin/Ymax are set again to calculate Y average. Processing is initiated in step SPO and proceeds therefrom to steps SPi and SP2 wherein a luminance level and a degree of saturation are respectively set. Thereafter, processing proceeds to step SP3 whereupon the degree of saturation of each pixel is checked- If the saturation is less than a predetermined amount, then processing returns to step SP3. If, however, the saturation is more than the predetermined amount, then processing proceeds to step SP4 wherein the hue of each pixel is checxea. if the hue is not the recognized color, then processing returns to step SP3. if the hue is the recognized color, then processing proceeds to step SP5 whereat the luminance of each pixel is checked. If the luainanee is less than a predetermined amount, then processing returns to step SP3. However, if th luminance is more than the predetermined amount, then processing proceeds to step SP6 whereat the luminance values are integrated. Thereafter, the number of pixels having the same hue are counted at step SP7. After step SP7, processing proceeds to step SP8, Whereupon a determination is made as to whethor all the pixels were observed. If such determination ia negative, processing returns to step SP3. If, however, this determination is affirmative, processing proceeds to step SP9 wherein a determination is made as to whether the object is present by determining it the count number is less than a reference value, If the determination in step SP9 is affirmative, processing proceeds to slep SP21 wherein it is decided that there was no object. Thereafter, processing is terminated at step SP20. On the other hand, if the determination in step SP9 is negative, then processing proceeds to step SPlO whurein an averaqe luminance value is calculated. Thereafter, processing proceeds to step SP11. wnereupon upper and lower Unite of the luminance are set. Prom step SP11, processing proceeds through steps SP12-SP19 which, with an exception at step SPU. are similar to steps SP3-SP9 and, as such, will not be further described. At step SP14 the luminance of each pixel is checked. If the luminance is less than a predetermined amount or greater than another predetermined amount, then processing returns to step sp12, if, however, such luminance lies within a predetermined range, then processing proceeds to ctap SP15. After step SP19, processing is terminated at step SP20. Returning to Fig. 1, the tracked object Initialization unit 8 produces the signal S17 which may be suppliod to the object recognizing and tracking unit 10 and the tracked object extraction and light measurement unit 11 as previously described. Such signal $17 nay start and/or stop automatic exposure adjustment and object tracking. The tracked object: initialization unit a may include a memory switch 3.8 arranged on the body or housing of the video camera which, upon activating, may cause a signal to be transmitted therefrom which may cause the signal S17 to be produced and/or the previously-mentioned operations to be initiated. Alternatively, a recording start switch of the video camera, a signal from the recording start switch, or a memory switch of a radio or cable remote controller or the like may be utilised instead of the memory swltcn 18 arranged on the video camera body. Mi previously described, the trigger signal S17 may be supplied to the object recognizing and tracking unit 10 so as to cause characteristic values to be extracted and held and the recognized hue to be supplied to the recognized color extraction and light measurement unit 9, and may also be supplied to the tracked object extraction and light measurement unit 11 so as to cause the luminance average value S9 of the tracked object to be held and an object recognized color extraction;objact recognizing, tracking and light-measuring reference luminance value signal to be generated. Additionally, th tracked object initialization unit a may generate a signal S19 and may supply the same '~o the light-measuring method selector 15. Such signal S19 provides an indication or one or a so-called pre-memory, memory and object tracking and exposure adjustment modes and the lik to the light-measuring method selector 15. Such modes will be hereinafter more fully described with reference to steps SPSI, SP35 and SP41-SP33 Of Fig. 8. As previously described, the light-measuring method selector 15 receives the signal S7 from the centralized light measurement/spot light measurement unit 12, the sitmal S8 from the recognized color and light measurement unit 9, the signal S9 from the tracked object extraction and light weaevurdswmt unit 11, and the reference signal S10 from the reference luminance value unit 13. Additionally, the light-measuring method unit 15 receives a reference signal Sll from a centralized light measurement reference luminance value generating circuit 14. in Jparticular, such circuit 14 supplies the signal si I to one terminal of a switch sw5 of the light-measuring method selector 15. The circuit 14 may include a memory, such as .read only memory (ROM), wherein the reference signal sn is utored. Accordingly, the light-ncaturing method selector 15 receives signals S7, S3 and S9 which contain average values of luminance information respectively obtained from units 12, 9 and 10, and the light-measuring method selector 1b further receives reference value signals S10 and Sll from reference luminance value circuits 13 and 14. The light-measuring method selector 15 further receives the signal S19 from the tracked object initialization unit 8, as previously described, in response to the received signals, the light-measuring method selector 15 selects from among the luminance average values S7, S8 and S9 so as to form a selected signal S12, and selects from among luminance reference values SIO and Sll so as to form a selected reference signal 313. Such selection of signals may be performed by the selective opening and/or closing of switches swl-aw5. The light-measuring method selector 15 may detect a tracked object unrecognizable or abnormal state in which a tracked object cannot be extracted by the tracked object extraction and light measurement unit 11. such situation may occur, for example, during a white saturation or Black crushing state which way result from a sudden luminance change or the like. in this situation, the light-measuring method selector 15 selects from among the luminance average value signals s? and SB from the centralized light measurement unit or spot light measurement unit 12 and the recognized color and light measurement unit 9, and the reference luminance value signals corresponding thereto, as luminance information to obtain or recover an operating state in which exposure may be controlled through tracked object extraction and light measurement. The light-measuring method selector is may further detect a atate in which luminance other- than that af a tracked object is abnormal, for example, an extreme saturation state or extreme black crushing state. Such situation or state may occur when the tracked object hides or moves off of the screen, in such situation, the light-measuring method selector 15 selects the luminance average value S7 from the centralized light measurement unit or spot light measurement unit 12 and the reference luminance value sn (which corresponds to the signal S7) as luminance information. Furthermore, the light-measuring method selector 15 may select the signal S7 from the centralized light measurement unit or spot light measurement unit 12 and the reference luminance value Sll (corresponding to the signal S7) when the salsutur receives the instruction signal S19 from the tracked abject initialization unit 8 prior to the initialization of an object as the luminance information. As a result, the lumirance of the object to be recognized and extracted may be adjusted to an optimum exposure. The light-measuring method selector 15 aiay further provide a picture information update signal Sis alter obtaining light measurement data. Such signal sis is supplied to the picture information temporary memory 7. The above-mentioned operations performed by the light-measuring method selector 15 will now be further described with reference to the flow chart: of Fig. 8. Processing initially proceeds from step SP30 to step SP31 whereupon a determination is made as to whether storing has begun. If the determination is affirmative, processing proceeds to step SP32 whereupon the light measurement value signal S7 from the unit 12 and the corresponding reference luminance value are selected. Thereafter, processing proceeds to step SP33 whereupon a picture information update signal S18 is generated and supplied to the memory 7. If, on the other hand, the determination in step SP31 is negative, processing proceeds to step SP35 whereupon a determination is made as to whether it is the first time through such step. If it is the first time, processing proceeds to step SP36 wherein the characteristics of tha recognized object are stored. Processing then proceeds to step SP37 wherein the hue of the recognized color ia stored. Thereafter, object recognition processing is performed in step SP38. The reference luminance value sio is formed in step SP39. Processing then proceeds to step SP4Q wherein the light measurement value signal 58 from the unit 9 and the corresponding reference luminance value are selected. Thereafter, processing proceeds to step SP33. If the determination of step spas indicates that it is not the first tine through, then processing proceeds to step SP41 wherein object recognition processing is performed. Thereafter, processing proceeds to step SP42 whereupon a determination is made as to whether it is possible to extract the desired object. If it is possible, the light: measurement value signal S9 and the corresponding reference luminance value are selected in step SP43. Processing then proceeds to step SP33. If the determination of step SP2 is negative, processing proceeds to step SP44 wherein a determination is made as to whether the object moves off of the screen. If such determination is affirmative, processing proceeds to step SP45 wherein the light measurement value signal S7 and the corresponding reference luminance value are selected. Processing then proceeds to step SP33. If, however, the determination in step SP44 is negative, processing proceeds to step SP46 wherein the light measurement value signal S9 from the unit 11 and the corresponding reference luminance value are selected. Processing then proceeds to step 2P33. Although preferred embodiments of the present invention and modifications thereof have been described in detail herein, it is to be understood that this invention is not limited to theae embodiments and modifications, and that othor modifications and variations may ba offootod by one skilled in the art, without-departing from the spirit and scope ot the invention as defined bv the anoendcd claims. Processing proceeds from step SP33 to stup SP34, whereupon the selected signals are supplied to the exposure controller 16. Returning to Fig. 1, the selected signals S12 and S13 from the light-measuring method selector 15 are respectively supplied to positive and negative inputs of a comparator or gain circuit 60 of the exposure controller is, which may function as an error detector. As a result, tha comparator 60 produces a compared or error signal S14 Which is supplied to an iris/AGC circuit: el which, in turn, produces an iris signal SIS and the AGC signal 516. The Iris signal 515 is supplied to an iris driver circuit 17 so as to drive the iris, and the AGC signal S16 is supplied to the S/H and AGC circuit 4 as previously described. As a result, the exposure controller 16 may control the luminance and color difference signal generator 6, Therefore, by extracting only the luminance of an object to be photographed, the present invention e.iables a light measurement to be obtained without being influenced by the background or other objects. Further, since the present invention enables an object, to be light-neasured while tracking, a cameraman photographing an object does net have to maintain such object in a light measurement area of a light measurement pattern. Furthermore, the present invention enables light measurement to be performed on the background around the center of an image screen when tne object being photographed moves off the screen. As a result, normal luminance may be maintained. Thus, the present invention provides a video camera which determines a plurality of average values of luminance and a plurality of values of reference luminance values, and which selects therefrom the appropriate averay luminance and reference luminance values in accordance with operating conditions. Such selected values are utilized to generate control signals for controlling or adjusting the exposure of the video camera so as to provide automatic and proper exposure control even when photographing under adverse or harsn conditions, such as, when backlight and excessive normal light (spot light) sxists which may present serious problems for a conventional video camera. Although the present video camera was described as having a specific number of measurement units and reference value generating circuits, the present invention is not BO limited. Instead, other numbers of such units and reference value generating circuits may be utilized. Although preferred embodiments of the present invention and modifications thereof have been described in detail herein, it is to be understood that this invention is not limited to these embodiments and modifications, and that other modifications and variations may be effected by one skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims. WE CLAIM: 1. A video camera characterized in that : picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory; object recognizing and tracking means for recognizing extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory; tracked object initialization means for storing characteristic values of said tracked object; tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first: average value; recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating corresponding luminance information so as to calculate a second average value; and exposure control means for controlling an. iris and/or an automatic gain controller of said video camera by utilizing said first average value obtained by said tracked object extraction and light measurement means or said second average value obtained by said recognized color extraction and light measurement means and by utilizing a luminance reference value, and for adjusting the luminance of said object so as to be substantiallv constant. 2 A video camera as claimed in claim 1, comprising picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and lor storing picture information including said luminance information and color information in said temporary memory; object recognizing and tracking means for recognizing, extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory; tracked object initialization means for storing characteristic values of said tracked object; tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first average value; recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating said corresponding luminance information so as to calculate a second average value; centralized light measurement means for integrating said luminance information of a first area in a center portion of a screen so as to calculate a third average value; sport light measurement means for integrating said luminance information in a second area around the center portion of said screen which is smaller than said first area, so as to calculate a fourth average value; light-measuring method selection means for selecting said first average value obtained by said tracked object extraction and light measurement means, said second average value obtained by said recognized colour extraction and light measurement means, said third average value obtained by said centralized light measurement means, or said fourth average value obtained by said sport light measurement means in accordance with an object condition; and exposure control means for controlling an iris and/or an automatic gain controller of said video camera by using the selected luminance average value and a luminance reference value, and for adjusting the luminance of said object so as to be substantially constant. 3 The video camera as claimed in claim 1, wherein said tracked object extraction and light measurement means includes means for generating said luminance reference value used for exposure control upon receiving a command signal from said tracked object initialization means. 4 The video camera as claimed in claim 1, wherein said tracked object initialization means includes a storage switch provided on said video camera, said storage switch is capable of initiating and/or terminating exposure operations of said video camera system. 5 The video camera as claimed in claim 1, wherein said tracked object initialization means includes a storage switch provided on one of a cable and a radio remote control for said video camera, said storage switch is capable of initiating and/or terminating exposure operations of said video camera system. 6 The video camera as claimed in claim 1, wherein said tracked object initialization means includes a recording start switch of said video camera, said recording start switch is capable of initiating and/or terminating exposure operation o! said video camera system. 7 The video camera as claimed in claim 1, wherein said tracked object initialization means includes a recording start switch provided on one of a cable and radio remote control for said video camera, said recording start switch is capable of initiating and/or terminating exposure operations of said video camera system. 8 A video camera as claimed in claim 1, comprising: picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory; object recognizing and tracking means for recognizing, extracting and tracking a displayed object in said picture information stored in said temporary memory:: tracked object extraction and light measurement means for extracting said luminance information of said tracked object, for integrating only the extracted luminance information, and for averaging the integrated luminance information so as to produce a first average value; recognized color extraction and light measurement means for extracting objects having the same color as said tracked object, for integrating corresponding luminance information of said objects, and for averaging the integrated information so as to produce a second average value; reference generating means for generating reference luminance value signals; and exposure control means for controlling an exposure operation of said video camera by utilizing selected ones of said first and second average values and said luminance reference values so that the luminance of said object is properly adjusted. 9 A video camera as claimed in claim 1, comprising: picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory; tracked object extraction and light measurement means for extracting the luminance information of a tracked object from the picture information stored in said temporary memory, for integrating the extracted luminance information, and for averaging the integrated luminance information so as to produce a first average value; recognized color extraction and light measurement means for extracting other objects having the same color as said tracked object, for integrating corresponding luminance information of said other objects, and for averaging the integrated information so as to produce a second average value; centralized light measurement means for integrating said luminance information of a first area in a center portion of a screen, and for averaging the integrated luminance information so as to produce a third average value; reference generating means for generating a plurality of reference luminance value signals; light-measuring method selection means for selecting (i) one of said first average value from said tracked object extraction and light: measurement means, said second average value from said recognized color extraction and light measurement means, and said third average value from said centralized light measurement means, and (ii) one of said reference luminance value signals in accordance with current operating conditions; and exposure control means for controlling an exposure operation of said video camera by utilizing the selected ones of said first, second and third average values and said reference luminance value signals so that the luminance of said object is properly adjusted. 10 A video camera as claimed in claim 9, optionally comprising sport light measurement means for integrating said luminance information in a second area around the center portion of said screen which is smaller than said first area, and for averaging the integrated information so as to produce a fourth average value. 11 An apparatus for controlling an exposure of a video camera, as claimed in any one of the preceding claims, said apparatus comprising: means for receiving a video signal having luminance information and color information; a plurality of light measurement means for producing a plurality of luminance value signals from the received luminance and color information; means for generating a plurality of reference luminance value signals; means for selecting one of the produced luminance value signals and one of said reference luminance value signals; and means for controlling an exposure operation of said video camera by utilizing the selected ones of said produced luminance value signals and said reference luminance value signals so as to properly maintain the exposure of a desired object. 12 A video camera exposure control method for controlling an exposure of a video camera as claimed in any one of the preceding claims. 13 A video camera system substantially as herein described with reference to and as illustrated in the accompanying drawings. 14 A method for controlling video camera exposure substantially as herein described with reference to and as illustrated in the accompanying drawings.

Full Text

BACKGROUND OF THE INVENTION
The present invention relates to a video camera technique for automatically following a change of an object in the field of view of a video camera and, more particularly, to such technique for controlling the exposure so as to properly adjust the luminance of the object.
Numerous methods may be employed for controlling the exposure of a video camera. For example, such methods may include an automatic multi-split light measurement exposure control method, an automatic centralized or spot light measurement, exposure control method, a manual exposure control method and so forth.
The automatic multi-split light measurement exposure control method utilizes a multi-split light measurement pattern to measure luminance information obtained from picture information. The measured luminance information may be processed to obtain an average value, a distribution, and/or a histogram of such luminance information which may be utilized to control
exposure.
The automatic centralized or spot light measurement
exposure control method utilises a light measurement pattern which may be respectively located in or around the center portion or a screen ana in a relatively enall or spot area in or around such center portion. Luminance information obtained from picture

information is measured by use of the light measurement pattern which may ba utilized to control exposure.
In the manual exposure control method, an operator or cameraman miy determine and/or manually adjust the luminance of an object ts be photographed to obtain a desired or acceptable value.
Each of the above-mentioned exposure control methods has disadvantages associated therewith. For example, the automatic multi-split light measurement exposure control method may have difficultly in obtaining a proper exposures of an object in a seen saving a relatively large luminance difference between
the object and its background (such as in a scene having a backlight or excessive normal light). In such situation, the background information may be incorporated with information of the object to be photographed, thereby making it difficult to obtain a proper exposure of the object. Further, with regard to the automatic centralized or spot light measurement exposure control met nod, ouch method may be difficult to uste when the object to be photographed is moving. That is, in such method, the -operator or cameraman has to photograph the object such that it is always located at the center of a screen. As is to be appreciated, it may be difficult to continuously photograph the object so as to keep it centered on the screen while the object is moving. Furthermore, with regard to the manual exposure control method, an operator may not be able to properly adjust
th exposure control in response to a sudden luminance change of an object. Additionally, a problem may arise in that the background nay not be set to an optimum luminance level when an object suddenly disappears from a photographing screen.
An object of the present invention is to provide a system and method for properly controlling the exposure of a video camera for an object to be photographed.
Another object of the present invention is to provide a system and method as aforesaid in which luminance values are selected from among a plurality or luminance values and in which the selected luminance values are utilized to control the exposure of the video camera.
A further object of the present invention is to provide a system and method as aforesaid in which the plurality of luminance values include a plurality of average luminance values each calculated by a respective technique and at least one reference luminance value.
in accordance with an aspect of the present invention, a video camera system is provided. Such system comprises a picture information temporary storage device havirg a temporary memory for dividing a video signal obtained from at least one video camera into luminance information and color information and for storing picture information composed of the luminance information and the color information in tne temporary memory at
an arbitrary timing; an object recognizing and tracking device for recognizing, extracting and tracking a displayed object from this picture information stored in the temporary metaory; a tracked object initialization device for storing characteristic values of the tracked object; a tracked object extraction and light measurement device for extracting and integrating only the luminance information of the tracked object and for calculating an average value; a recognized color extraction and light measurement device for extracting color information having the same color ac the recognized object, integrating the corresponding luminance information, and calculating an average value; a centralized light measurement device for integrating the luminance information of the object in or around the center portion of a screen and calculating an average value; a spot light measurement device for integrating the luminance information in a relatively small area in or around the center portion of the screen and calculating an average value; a light-measuring method selection device for selecting the luminance information measured by the tracked object extraction and light measurement device, the recognized coioz extraction and light measurement device, the centralized light measurement device or epot light measurement device in accorda.nce with the object: condition; and an exposure control device lor controlling an iris and/or an automatic gain controller of this video camera and adjusting the luminance of an output object to a constant
value by using a selected luminance average value and a luminance reference value.
In accordance with another aspect of the present invention, a video camera exposure control method is provided. Such method comprises dividing a video signal obtained from a video camera into luminance information and color information and storing picture information including the luminance information and the color information in a temporary memory at an arbitrary timing; recognizing, extracting and tracking a displayed object from the picture information stored in the temporary memory; storing characteristic values or the tracked object; extracting and integrating only the luminance information of the tracked object and calculating a first average value; extracting color information having the same color as the recognized object, integrating the corresponding luminance information, and calculating a second average value; integrating the luminance information of the object around the center portion of a screen and calculating a third average value; integrating the luminance information in a relatively small area around the center portion of the screen and calculating a fourth average value; selecting the luminance information of the first average value, the second avaraga value, the third average value or the fourth average value in accordance with the object condition; and controlling an iris and/or an automatic gain controller of the viieo camera and adjusting the luminance or an output object to a constant value
by using the selected luminance average value and a luminance reference value.
The luminance information and color intonation stored in the picture information temporary storage device nay be read out and supplied to the centralized light measurement device or spot light measurement device, the recognized color extraction and light measurement device, the object recognizing and tracking device, and the tracked object extraction and lighr. measurement device in accordance with a command signal which may be supplied by the light-measuring method selection device. The recognized color extraction and light measurement device calculates a luminance average value of pixels having information which is relatively close to the recognized color. The objact recognizing and tracking device and the tracked object extraction and light measurement device extract only the recognized object and calculate its average luminance value. The centralized light measurement device or spot light measurement device calculates a average luminance value of a center portion of a screen or in a relatively small area of such center portion, respectively. The light-measuring method selection device determines or selects a luminance average value from among the calculated luminance average values in accordance with the objoct condition or present opera-ting conditions, and further selects a reference value. The selected luminance average value and reference value may be compared by the exposure control device BO as to obtain a signal
which may be utilized to properly adjust the luminance value of an object.
Accordingly, the present invention relates to a video camera, comprising:
picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory;
object recognizing and tracking means for recognizing extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory;
tracked object initialization means for storing characteristic values of said tracked object;
tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first average value;
recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating corresponding luminance information so as to calculate a second average value; and
exposure control means for controlling an iris and/or an automatic gain controller of said video camera by utilizing said first average value obtained by said tracked object extraction and light measurement means or said second average value obtained by said recognized color extraction and light measurement means and by utilizing a luminance reference value, and for adjusting the luminance of said object so as to be substantially constant.
Other objects, features and advantages of the present invention will become apparent from the following detailed description of the illustrated embodiments when read in conjunction with the accompanying drawings in which corresponding components are identified by the same reference numerals or characters.
BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS
Fig. 1 is a diagram of a video camera ay stem fur performing object recognition and automatic exposure adjustment according to an embodiment, of the present invention;
Fig. 2 illustrates a section of an image screen wherein a centralized light measurement device of the video camera system of Fig. 1 performs luminance integrating and average value calculating;
Fig. 3 illustrates a section of an image screen wherein a spot light measurement device of the video camera system of Fig. 1 performs luminance integrating and average value calculating;
Fig. 4A is a diagram of an image to which reference will be made in explaining light measurement methods;
Figs. 4B and 4C are diagrams to which reference will be made in explaining light measurements performed by the
centralized light measurement device of the video camera system of Fig. 1;
Pigs. 5A to 5D are diagrams to which reference will be made in explaining light measurements performed by other devices of the video camera system of Fig. 1;
Figs. 6 and 7 are flow charts to which rnference will be nade in explaining a recognized color extraction and light measurement procedure; and
Fig. 8 is a flow Chart to which referencn will be made in explaining a light-maaauring method selection procedure.
DESCRIPTION OF THE PREFEREED EMBODIMENTS
Preferred embodiments of the present, invention will now be described with reference to the accompanying drawings.
Fig. 1 illustrates a video camera system (VCS) which is adapted to perform object recognition and automatic: exposure adjustment, AS shown therein, such vcs generally i.ncludes a lens block section 50, a sample and hold (S/H) and autoiiatic gain control (AGC) circuit 4, an analog-to-digital (A/DJ circuit 5, a luminance and color difference signal generator 6, an image temporary memory 7/ a recognized color extraction and light measurement unit 9, an object recognizing and tracking unit 10, a tracked object extraction and light measurement unr.t 11, a •tracked object initialization unit 8, a cerrtralizaa light measurement/spot light measurement unit 12, a light-measuring method selector 15, an exposure controller 3.6 and a driver 17
Which are connected as shown, the lens block section So includes a lens 1, an iris 2 and a solid pickup or charge cnupled device (CCD) 3.
Image pickup light LA from an object is supplied through the lens 1 and the iris 2 to the CCD 3, whereupon a pickup signal SI which represents a field-of-view picture is generated and supplied to the S/H and AGC 4. The K/H and AGC is adapted to sample and hold the received pickup signal si and to perform or adjust the gain thereof in accordance w;.th an AGC control signal S16 from the exposure controller 16 so as to produce A pickup signal S2. Such pickup signal S2 is converted to a digital pickup signal S3 by the A./D converter 5. The digital pickup signal S3 is supplied to the luminance and color difference signal generator 6,
The luminance and color difference signal generator 6 is adapted to receive the digital pickup signal S3 and to generate therefrom a luroinance (Y) signal S4, a color difference (R-Y) signal S5 and a color difference (B-YJ signal SB. such luminance and color difference signals are supplied to the image temporary memory 7, whereupon they are respectively preprocessed as an abject tracking detection signal and an object luminance light-measuring signal, such preprocessing may include a thinning out operation in which the number of canplaa of the
luminance data are decreased, whereupon the total number of data
bits processed are reduced thereby reducing or improving the
sampling frequency and/or processing speed. The creprocessed signals are stored in an image memory or memories. That is, three image memories 51, 52 and 53 may be utilized, wherein the luminance picture information is stored in the image memory 51, the color difference (R-Y) picture information is stared in the image memory 52, and the color difference (B-Y) picture information is stored in the image memory 53. such stared picture information may be randomly-accessed by the centralized light measurement/spot light Measurement unit 12, the recognized color extraction and light: measurement unit: 9, the object recognizing and tracking unit 10, and tne tracked object extraction and light measurement unit n. More specifically, when stored picture information is desired, the unit requesting such information supplies a signal, such as an address signal (adi, ad2 or ad3) which indicates the address or location of the desired stored picture information, to the image memories. As a result, the requested information is supplied to the respective unit by way of signal dal, da2 or da3. As an example, consider the situation in which the centralized light/spot light measurement unit 12 Wishes to obtain the stored luminance picture information from the image memory 51. In such situation, the unit 12 supplies a request: or addrefis signal adi to the image memory 51, whereupon signal dal having the requested luminance picture information is supplied to the unit 12.
The centralized light measurement/spot light measurement unit 12 receives the requested stored luminance picture information from the image temporary memory 7, as previously described, and generates therefrom an average value of the luminance information contained within a predetermined portion. For example, the centralized light measurement/spot light measurement unit 12 may generate an average value of the luminance information in a center portion 98 of an image screen 97 which is indicated by a cross-hatched or shaded portion in Fig. 2. In this example, the center portion or areta 98 has a siae of X/3 x V/3, in which the horizontal dimension of the screen 97 is X and the vertical dimension of the ac;reen is Y As is to be appreciated, other sizes of such center portion may be utilized. The centralized light measurement unit :.2 supplies the generated average value of the center luminance information (which may be referred to as a centralized light measurement) as an output signal S7 to one terminal of a switch sw;. of the light-measuring method selector 15.
An example of the procedure performed by the
centralized light measurement unit 12 is illustrated in Figs. 4A. 4B and 4C. That is, Fig. 4A illustrates a image on which the unit 12 is to determine an average value of the center luminance information- The light measured or center portion of such image is indicated as a shaded area in Pig. 4B. The information contained within this shaded area LG processed by the unit 13 eo
the centralized light measurement or average luminance levels (light values) as illustrated in :?ig. 4C.
The centralized light measurement unit/apot light measurement unit 12 may further generate an average value of the luminance information obtained from the signal dal pertaining to a relatively small predetermined portion or spot of the image screen 97. Such predetermined portion may be at the center portion of the image screen 97 as, for example, indicated by a shaded portion 99 in Fig. 3. In this example, and in a manner similar tn that previously described with refaranou to Pig. 2, the unit 12 determines an average value of the luminance information contained within the center portion or area having a size of X/10 x Y/10 (in which the horizontal dimension of the screen 97 is X and the vertical dimension of the screen is Y). Such area 99 is smaller or narrower than the area 98 of Fig. 2. As is to be appreciated, other sizes of the spot area may be utilized. The spot light measurement unit 12 supplies the generated average value of the center spot luminance information (which may be referred to as a spot light measuremtsnt) as the output signal S7 to one terminal of the switch swi of the light-measuring method selector 15.
In response to a request, the object recognizing and tracking unit 10 may receive the luminance information signal dai and the color difference information signals da2 and da3 from the imaga temporary memory 7. The object recoynizinq and tracking
unit 10 may further receive an instructing or trigger signal Si7. In response to such signal S17, the object recognising and tracking unit 10 extracts and holds characteristic values or information, which may include position information, of a desired object contained within the image represented by the respective luminance and color difference signals in a hold area of the unit 10. The hue of such object, or recognized hue, is supplied to the recognized color extraction and light measurement unit 9. The tracked object extraction and light neasurement unit 11 may receive an instructing signal S17 and information pertaining to the recognized and tracked object from the object recognizing and tracking unit 10. The tracked objisct extraction and light measurement unit 11 may integrate only the luminance information of the object extracted by the object recognizing and tracking unit 10 and average the same so as to forn an average value S9. Such average value S9 is supplied from l;he tracked object extraction and light measurement unit 11 to one terminal of a switch sw3 of the light-measuring method selector 15. An example of the procedure performed by the tracked object extraction and light measurement unit 11 is illustrated in Figs. 5A and SB. In this example! the center piece of fruit or apple of the image of Pig. 4A is the recognized or tracked object and, as such, is shown as a shaded object in Fig. 5A. The information contained within this shaded area is processed by the unit 11 so

as to determine the light measurement or average luminance values &s illustrated in Fig. 5B.
Further, in response to the signal S17, 1:he tracked abject extraction and light measurement unit 11 may hold the luminance average value S9 of the tracked object in a hold area of the unit 11 and may cause such average value to be supplied to an object recognized color extraction/object recognizing, tracking and light-measuring reference luminance value generating circuit 13 so that a reference luminance value signal S1Q may be produced therefrom. Such signal SlO, which way represent an average of the luminance at the extracted object prior to auto-tracking, ia supplied Croia the circuit 13 to one tormina 1 of a switch sw4 of the light-measuring method selector :.5.
The recognized color extraction and light: measurement unit 9 may receive the luminance Information dal, the color difference information da2 and da3, and the recognized hue information from the object recognizing and tracking unit 10 as previously described. The unit 9 is adapted to process the received information so ag to determine other locav.ions or objects of the respective image (such as that of Fi.g. 4A) which have hues substantially similar to that of the recognized hue from the unit 10. More specifically, the unit 9 extracts information from the color dtfferanca signals R-Y and B-Y so as to determine locations or points having substantially the same hue as the recognized color among the luminance information dal
from the image memory 7, integrates the values of the determined points, averages the integrated values to obtain an averaged value SB, and supplies such average value S8 to one terminal of a switch sw2 of the light measuring method selector 15. To avoid or minimize count errors of the luminance of the same hue of noise component(s) in a black object or an environment color in a white object, black and environment colors may be checked.
An example of the procedure performed by the recognized color extraction and light measurement unit 9 is illustrated in Pigs. SC and so. In thie axanple, the hue or color of the center piece of fruit (or apple) of Fig, 5A is the extracted or recognized color. Accordingly, tne unit 9 determines other locations or objects within the image of Fig. 4A having substantially the same hue aa that of the apple. AS a result, objects such as the three cherries which have substantially the same color as that of the apple are identified or extracted. These objects are shaded in a similar manner to that of the apple as shown in Fig. 5C. The area of the apple having an oval shape which is not shaded represents an area which may be affected by ao-called "white saturation" or "black crushing". (White saturation may be caused by over-exposure, whereas black crushing may be caused by under-exposure.) The information contained within the shaded areas of Fig. 5C ar processed by the recognized color extraction and light meaaurHinnt unit 9 so as to
determine the light measurement or average luminance values as illustrated in Fig. 5D.
AS way be observed from Figs. 4C, SB and 5D, the average luminance levels or light values of similar objects (such as the apple) appear slightly different from figure to figure. Such minor discrepancy may be due to somewhat different conditions which may exist when these figures were captured or picked-up (such as different background brightneae;}.
The processing performed by the recognised color extraction and light measurement unit 9 will now tie described with to the flow chart illustrated in Figs. 6 and 7. Pig. 6 illustrates the processing in which luminar.ee levels except a black portion and noise are averaged and the presence of an object is determined. Pig. 7 illustrates the processing wherein, in the presence of an object, Ymin/Ymax are set again to calculate Y average.
Processing is initiated in step SPO and proceeds therefrom to steps SPi and SP2 wherein a luminance level and a degree of saturation are respectively set. Thereafter, processing proceeds to step SP3 whereupon the degree of saturation of each pixel is checked- If the saturation is less than a predetermined amount, then processing returns to step SP3. If, however, the saturation is more than the predetermined amount, then processing proceeds to step SP4 wherein the hue of each pixel is checxea. if the hue is not the recognized color,
then processing returns to step SP3. if the hue is the recognized color, then processing proceeds to step SP5 whereat the luminance of each pixel is checked. If the luainanee is less than a predetermined amount, then processing returns to step SP3. However, if th luminance is more than the predetermined amount, then processing proceeds to step SP6 whereat the luminance values are integrated. Thereafter, the number of pixels having the same hue are counted at step SP7.
After step SP7, processing proceeds to step SP8, Whereupon a determination is made as to whethor all the pixels were observed. If such determination ia negative, processing returns to step SP3. If, however, this determination is affirmative, processing proceeds to step SP9 wherein a determination is made as to whether the object is present by determining it the count number is less than a reference value,
If the determination in step SP9 is affirmative, processing proceeds to slep SP21 wherein it is decided that there was no object. Thereafter, processing is terminated at step SP20.
On the other hand, if the determination in step SP9 is negative, then processing proceeds to step SPlO whurein an averaqe luminance value is calculated. Thereafter, processing proceeds to step SP11. wnereupon upper and lower Unite of the luminance are set.
Prom step SP11, processing proceeds through steps SP12-SP19 which, with an exception at step SPU. are similar to steps SP3-SP9 and, as such, will not be further described. At step SP14 the luminance of each pixel is checked. If the luminance is less than a predetermined amount or greater than another predetermined amount, then processing returns to step sp12, if, however, such luminance lies within a predetermined range, then processing proceeds to ctap SP15.
After step SP19, processing is terminated at step SP20.
Returning to Fig. 1, the tracked object Initialization unit 8 produces the signal S17 which may be suppliod to the object recognizing and tracking unit 10 and the tracked object extraction and light measurement unit 11 as previously described. Such signal $17 nay start and/or stop automatic exposure adjustment and object tracking. The tracked object: initialization unit a may include a memory switch 3.8 arranged on the body or housing of the video camera which, upon activating, may cause a signal to be transmitted therefrom which may cause the signal S17 to be produced and/or the previously-mentioned operations to be initiated. Alternatively, a recording start switch of the video camera, a signal from the recording start switch, or a memory switch of a radio or cable remote controller or the like may be utilised instead of the memory swltcn 18 arranged on the video camera body.
Mi previously described, the trigger signal S17 may be supplied to the object recognizing and tracking unit 10 so as to cause characteristic values to be extracted and held and the recognized hue to be supplied to the recognized color extraction and light measurement unit 9, and may also be supplied to the tracked object extraction and light measurement unit 11 so as to cause the luminance average value S9 of the tracked object to be held and an object recognized color extraction;objact recognizing, tracking and light-measuring reference luminance value signal to be generated.
Additionally, th tracked object initialization unit a may generate a signal S19 and may supply the same '~o the light-measuring method selector 15. Such signal S19 provides an indication or one or a so-called pre-memory, memory and object tracking and exposure adjustment modes and the lik to the light-measuring method selector 15. Such modes will be hereinafter more fully described with reference to steps SPSI,
SP35 and SP41-SP33 Of Fig. 8.
As previously described, the light-measuring method selector 15 receives the signal S7 from the centralized light measurement/spot light measurement unit 12, the sitmal S8 from the recognized color and light measurement unit 9, the signal S9 from the tracked object extraction and light weaevurdswmt unit 11, and the reference signal S10 from the reference luminance value unit 13. Additionally, the light-measuring method unit 15
receives a reference signal Sll from a centralized light measurement reference luminance value generating circuit 14. in Jparticular, such circuit 14 supplies the signal si I to one terminal of a switch sw5 of the light-measuring method selector 15. The circuit 14 may include a memory, such as .read only memory (ROM), wherein the reference signal sn is utored.
Accordingly, the light-ncaturing method selector 15 receives signals S7, S3 and S9 which contain average values of luminance information respectively obtained from units 12, 9 and 10, and the light-measuring method selector 1b further receives reference value signals S10 and Sll from reference luminance value circuits 13 and 14. The light-measuring method selector 15 further receives the signal S19 from the tracked object initialization unit 8, as previously described, in response to the received signals, the light-measuring method selector 15 selects from among the luminance average values S7, S8 and S9 so as to form a selected signal S12, and selects from among luminance reference values SIO and Sll so as to form a selected reference signal 313. Such selection of signals may be performed by the selective opening and/or closing of switches swl-aw5.
The light-measuring method selector 15 may detect a tracked object unrecognizable or abnormal state in which a tracked object cannot be extracted by the tracked object extraction and light measurement unit 11. such situation may occur, for example, during a white saturation or Black crushing
state which way result from a sudden luminance change or the like. in this situation, the light-measuring method selector 15 selects from among the luminance average value signals s? and SB from the centralized light measurement unit or spot light measurement unit 12 and the recognized color and light measurement unit 9, and the reference luminance value signals corresponding thereto, as luminance information to obtain or recover an operating state in which exposure may be controlled through tracked object extraction and light measurement.
The light-measuring method selector is may further detect a atate in which luminance other- than that af a tracked object is abnormal, for example, an extreme saturation state or extreme black crushing state. Such situation or state may occur when the tracked object hides or moves off of the screen, in such situation, the light-measuring method selector 15 selects the luminance average value S7 from the centralized light measurement unit or spot light measurement unit 12 and the reference luminance value sn (which corresponds to the signal S7) as luminance information.
Furthermore, the light-measuring method selector 15 may select the signal S7 from the centralized light measurement unit or spot light measurement unit 12 and the reference luminance value Sll (corresponding to the signal S7) when the salsutur receives the instruction signal S19 from the tracked abject initialization unit 8 prior to the initialization of an object as
the luminance information. As a result, the lumirance of the object to be recognized and extracted may be adjusted to an optimum exposure.
The light-measuring method selector 15 aiay further provide a picture information update signal Sis alter obtaining light measurement data. Such signal sis is supplied to the picture information temporary memory 7.
The above-mentioned operations performed by the light-measuring method selector 15 will now be further described with reference to the flow chart: of Fig. 8.
Processing initially proceeds from step SP30 to step SP31 whereupon a determination is made as to whether storing has begun. If the determination is affirmative, processing proceeds to step SP32 whereupon the light measurement value signal S7 from the unit 12 and the corresponding reference luminance value are selected. Thereafter, processing proceeds to step SP33 whereupon a picture information update signal S18 is generated and supplied to the memory 7.
If, on the other hand, the determination in step SP31 is negative, processing proceeds to step SP35 whereupon a determination is made as to whether it is the first time through such step. If it is the first time, processing proceeds to step SP36 wherein the characteristics of tha recognized object are
stored. Processing then proceeds to step SP37 wherein the hue of the recognized color ia stored. Thereafter, object recognition
processing is performed in step SP38. The reference luminance value sio is formed in step SP39. Processing then proceeds to step SP4Q wherein the light measurement value signal 58 from the unit 9 and the corresponding reference luminance value are selected. Thereafter, processing proceeds to step SP33.
If the determination of step spas indicates that it is not the first tine through, then processing proceeds to step SP41 wherein object recognition processing is performed. Thereafter, processing proceeds to step SP42 whereupon a determination is made as to whether it is possible to extract the desired object. If it is possible, the light: measurement value signal S9 and the corresponding reference luminance value are selected in step SP43. Processing then proceeds to step SP33.
If the determination of step SP2 is negative,
processing proceeds to step SP44 wherein a determination is made as to whether the object moves off of the screen. If such determination is affirmative, processing proceeds to step SP45 wherein the light measurement value signal S7 and the corresponding reference luminance value are selected. Processing then proceeds to step SP33. If, however, the determination in step SP44 is negative, processing proceeds to step SP46 wherein the light measurement value signal S9 from the unit 11 and the
corresponding reference luminance value are selected. Processing then proceeds to step 2P33.
Although preferred embodiments of the present invention and modifications thereof have been described in detail herein, it is to be understood that this invention is not limited to theae embodiments and modifications, and that othor modifications and variations may ba offootod by one skilled in the art, without-departing from the spirit and scope ot the invention as defined bv the anoendcd claims.
Processing proceeds from step SP33 to stup SP34, whereupon the selected signals are supplied to the exposure controller 16.
Returning to Fig. 1, the selected signals S12 and S13 from the light-measuring method selector 15 are respectively supplied to positive and negative inputs of a comparator or gain circuit 60 of the exposure controller is, which may function as an error detector. As a result, tha comparator 60 produces a compared or error signal S14 Which is supplied to an iris/AGC circuit: el which, in turn, produces an iris signal SIS and the AGC signal 516. The Iris signal 515 is supplied to an iris driver circuit 17 so as to drive the iris, and the AGC signal S16 is supplied to the S/H and AGC circuit 4 as previously described. As a result, the exposure controller 16 may control the luminance and color difference signal generator 6,
Therefore, by extracting only the luminance of an object to be photographed, the present invention e.iables a light measurement to be obtained without being influenced by the background or other objects. Further, since the present invention enables an object, to be light-neasured while tracking, a cameraman photographing an object does net have to maintain such object in a light measurement area of a light measurement pattern. Furthermore, the present invention enables light measurement to be performed on the background around the center
of an image screen when tne object being photographed moves off the screen. As a result, normal luminance may be maintained.
Thus, the present invention provides a video camera which determines a plurality of average values of luminance and a plurality of values of reference luminance values, and which selects therefrom the appropriate averay luminance and reference luminance values in accordance with operating conditions. Such selected values are utilized to generate control signals for controlling or adjusting the exposure of the video camera so as to provide automatic and proper exposure control even when photographing under adverse or harsn conditions, such as, when backlight and excessive normal light (spot light) sxists which may present serious problems for a conventional video camera.
Although the present video camera was described as having a specific number of measurement units and reference value generating circuits, the present invention is not BO limited. Instead, other numbers of such units and reference value generating circuits may be utilized.
Although preferred embodiments of the present invention and modifications thereof have been described in detail herein, it is to be understood that this invention is not limited to these embodiments and modifications, and that other modifications and variations may be effected by one skilled in the art without
departing from the spirit and scope of the invention as defined
by the appended claims.

WE CLAIM:
1. A video camera characterized in that : picture information temporary storage
means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory;
object recognizing and tracking means for recognizing extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory;
tracked object initialization means for storing characteristic values of said tracked object;
tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first: average value;
recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating corresponding luminance information so as to calculate a second average value; and exposure control means for controlling an. iris and/or an automatic gain controller of said video camera by utilizing said first average value obtained by said tracked object extraction and light measurement means or said second average value obtained by said recognized color extraction and light measurement means and by utilizing a luminance reference value, and for adjusting the luminance of said object so as to be substantiallv constant.
2 A video camera as claimed in claim 1, comprising picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and lor storing picture information including said luminance information and color information in said temporary memory;
object recognizing and tracking means for recognizing, extracting and tracking a displayed object in accordance with said picture information stored in said temporary memory;
tracked object initialization means for storing characteristic values of said tracked object;
tracked object extraction and light measurement means for extracting and integrating only said luminance information of said tracked object so as to calculate a first average value;
recognized color extraction and light measurement means for extracting said color information having the same color as said tracked object, and integrating said corresponding luminance information so as to calculate a second average value;
centralized light measurement means for integrating said luminance information of a first area in a center portion of a screen so as to calculate a third average value;
sport light measurement means for integrating said luminance information in a second area around the center portion of said screen which is smaller than said first area, so as to calculate a fourth average value;
light-measuring method selection means for selecting said first average value obtained by said tracked object extraction and light measurement means, said
second average value obtained by said recognized colour extraction and light measurement means, said third average value obtained by said centralized light measurement means, or said fourth average value obtained by said sport light measurement means in accordance with an object condition; and
exposure control means for controlling an iris and/or an automatic gain controller of said video camera by using the selected luminance average value and a luminance reference value, and for adjusting the luminance of said object so as to be substantially constant.
3 The video camera as claimed in claim 1, wherein said tracked object extraction
and light measurement means includes means for generating said luminance
reference value used for exposure control upon receiving a command signal from
said tracked object initialization means.
4 The video camera as claimed in claim 1, wherein said tracked object
initialization means includes a storage switch provided on said video camera, said
storage switch is capable of initiating and/or terminating exposure operations of said
video camera system.
5 The video camera as claimed in claim 1, wherein said tracked object
initialization means includes a storage switch provided on one of a cable and a radio
remote control for said video camera, said storage switch is capable of initiating
and/or terminating exposure operations of said video camera system.
6 The video camera as claimed in claim 1, wherein said tracked object
initialization means includes a recording start switch of said video camera, said
recording start switch is capable of initiating and/or terminating exposure operation
o! said video camera system.
7 The video camera as claimed in claim 1, wherein said tracked object
initialization means includes a recording start switch provided on one of a cable and
radio remote control for said video camera, said recording start switch is capable of
initiating and/or terminating exposure operations of said video camera system.
8 A video camera as claimed in claim 1, comprising:
picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory;
object recognizing and tracking means for recognizing, extracting and tracking a displayed object in said picture information stored in said temporary memory::
tracked object extraction and light measurement means for extracting said luminance information of said tracked object, for integrating only the extracted luminance information, and for averaging the integrated luminance information so as to produce a first average value;
recognized color extraction and light measurement means for extracting objects having the same color as said tracked object, for integrating corresponding luminance information of said objects, and for averaging the integrated information so as to produce a second average value;
reference generating means for generating reference luminance value signals; and
exposure control means for controlling an exposure operation of said video camera by utilizing selected ones of said first and second average values and said luminance reference values so that the luminance of said object is properly adjusted.
9 A video camera as claimed in claim 1, comprising:
picture information temporary storage means having a temporary memory for dividing a video signal obtained from a video camera into luminance information and color information, and for storing picture information including said luminance information and color information in said temporary memory;
tracked object extraction and light measurement means for extracting the luminance information of a tracked object from the picture information stored in said temporary memory, for integrating the extracted luminance information, and for averaging the integrated luminance information so as to produce a first average value;
recognized color extraction and light measurement means for extracting other objects having the same color as said tracked object, for integrating corresponding luminance information of said other objects, and for averaging the integrated information so as to produce a second average value;
centralized light measurement means for integrating said luminance information of a first area in a center portion of a screen, and for averaging the integrated luminance information so as to produce a third average value;
reference generating means for generating a plurality of reference luminance value signals;
light-measuring method selection means for selecting (i) one of said first average value from said tracked object extraction and light: measurement means, said second average value from said recognized color extraction and light measurement means, and said third average value from said centralized light

measurement means, and (ii) one of said reference luminance value signals in accordance with current operating conditions; and
exposure control means for controlling an exposure operation of said video camera by utilizing the selected ones of said first, second and third average values and said reference luminance value signals so that the luminance of said object is properly adjusted.
10 A video camera as claimed in claim 9, optionally comprising sport light
measurement means for integrating said luminance information in a second area
around the center portion of said screen which is smaller than said first area, and
for averaging the integrated information so as to produce a fourth average value.
11 An apparatus for controlling an exposure of a video camera, as claimed in any
one of the preceding claims, said apparatus comprising:
means for receiving a video signal having luminance information and color information;
a plurality of light measurement means for producing a plurality of luminance value signals from the received luminance and color information;
means for generating a plurality of reference luminance value signals;
means for selecting one of the produced luminance value signals and one of said reference luminance value signals; and
means for controlling an exposure operation of said video camera by utilizing the selected ones of said produced luminance value signals and said reference luminance value signals so as to properly maintain the exposure of a desired object.
12 A video camera exposure control method for controlling an exposure of a video
camera as claimed in any one of the preceding claims.
13 A video camera system substantially as herein described with reference to and
as illustrated in the accompanying drawings.
14 A method for controlling video camera exposure substantially as herein
described with reference to and as illustrated in the accompanying drawings.

Documents:

2483-del-1995-abstract.pdf

2483-del-1995-claims.pdf

2483-del-1995-complete specification (granted).pdf

2483-DEL-1995-Correspondence-Others.pdf

2483-del-1995-correspondence-po.pdf

2483-del-1995-description (complete).pdf

2483-del-1995-drawings.pdf

2483-del-1995-form-1.pdf

2483-del-1995-form-2.pdf

2483-del-1995-form-3.pdf

2483-del-1995-form-4.pdf

2483-del-1995-pa.pdf

2483-del-1995-petition-137.pdf

2483-del-1995-petition-138.pdf

« Previous Patent

Next Patent »

Patent Number

197221

Indian Patent Application Number

2483/DEL/1995

PG Journal Number

38/2008

Publication Date

19-Sep-2008

Grant Date

22-Dec-2006

Date of Filing

29-Dec-1995

Name of Patentee

Sony Corporation

Applicant Address

7-35 Kitashinagawa, 6-Chome, Shingawa-ku, Tokyo,Japan.

Inventors:

#	Inventor's Name	Inventor's Address
1	Masakazu Koyanagi	C/o Sony Corporation, of 7-35 Kitashinagawa, 6-Chome, Shinagaqwa-ku, Tokyo,Japan.
2	Tadafusa Tomitaka	C/o Sony Corporation, 7-35 Kitashinagawa, 6-Chome, Shinagaqwa-ku, Tokyo,

PCT International Classification Number

G03B 7/00

PCT International Application Number

N/A

PCT International Filing date

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1			NA