Indian Patents. 231272:A METHOD OF CONTROLLING AN IMAGE COMBINING APPARATUS

Title of Invention	A METHOD OF CONTROLLING AN IMAGE COMBINING APPARATUS
Abstract	In combining a handwritten image with an arbitrary image in an image processing apparatus, a sheet is prepared so as to allow confirmation a positional relationship of the arbitrary image and the handwritten image.

Full Text	IMAGE COMBINING APPARATUS, AND CONTROL METHOD AND PROGRAM THEREFOR BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to an image combining apparatus for combining photographic image data entered from image input means, and data read from a handwritten original (manuscript). Related Background Art Along with the recent pervasiveness and the technical advances in photographic image input apparatuses such as a digital camera and a color scanner, and in image output apparatus such as a color printer, various methods of use are being proposed on digital photo images. Such images are not only captured and printed on a printing medium, but are often utilized with various processing and editing in various manners. For example it is proposed to send a photograph with an inscribed message as a letter, and also to paste a photograph with an inscribed title or illustration to an album. For adding a handwritten message or illustration to a photo image entered from an image input apparatus, Japanese Patent Application Laid-open No. S63-040933 discloses a method of displaying the input photo image on a display apparatus such as a liquid crystal panel, and inputting a handwritten image through a touch-panel input apparatus provided in the display apparatus, thereby combining the photo image and the handwritten image. Also Japanese Patent Application Laid-open No. H09-139832 discloses a method in which a handwritten original, written on a paper sheet different from an input photo image, is read by a reading apparatus and is combined with the photo image. Also the printing apparatus is increasingly provided, not only a printing function, with complex functions such as an image reading function and a memory card slot for accepting a memory card, and Japanese Patent Application Laid-open No. H05-122496 discloses a method, in such printing apparatus, of combining and printing a photo image in a memory card and data read in a reading unit. Also such image combining is not limited to a photo image data and a handwritten image, but the data superposed with the photo image also include a photo frame or a message of a predetermined pattern. However, in the case of drawing a message or an illustration on a photo image, by selecting a background area other than a person recorded as a main object, the prior methods involve following limitations. 10021170 The touch-panel method mentioned above allows to draw a handwritten character or an illustration directly on a photo image while observing the photo image, but requires a display apparatus and a touch-panel input apparatus which lead to a high hardware cost, and an ability of expression is inevitably restricted as the image resolution of the handwritten input is limited by the resolution of the display apparatus or the touch panel. On the other hand, the original reading method mentioned above has the advantages that a handwritten original can be prepared in a simple and easy manner and that various expressions are possible depending on the type of the writing utensil, but involves a drawback that, as the handwritten original is drawn on an original sheet separate from the photo image, a superposing positional relation of the handwritten image with the photo image cannot be confirmed exactly at the preparation of the handwritten original Also with an increase in the data to be combined, such as photo image data, data handwritten by the user, a photo frame etc., the combining has to be executed plural times on the data of plural contents, thereby resulting in a large burden on the resources used for the combining and also in a long process time* 10021170 Also at the image combining operation, it is not possible to know how the content to be combined and the content written by the user are positioned after the image combining. SUMMARY OF THE INVENTION An object of the present invention is to provide an image combining apparatus which allows to draw a handwritten original, while confirming, at the drawing operation of the handwritten original on a sheet, a positional relationship how the handwritten original is to be combined with respect to an object photo image, and a control method and a program therefor. Another object of the present invention is to provide an image combining apparatus that does not involve a large burden on resources required for image combining nor a long process time, even with a large amount of data to be combined such as photo image data, data written by a user, a photo frame etc., and an image processing method and a program therefor. BRIEF DESCRIPTION OF THE DRAWING Fig. 1 is a block diagram of an image combining apparatus 100 constituting an embodiment 1 of the present invention; 10021170 Fig. 2 is a view showing an original sheet SHI in the embodiment 1; Fig. 3 is a flow chart showing, in the embodiment 1, a process executed by a CPU 2 of the image combining apparatus 100 for printing the original sheet SHI; Fig. 4 is a flow chart showing, in the embodiment 1, a process executed by the CPU 2 in reading the original sheet SHI bearing a handwritten image and combining and printing a reference image 41 and a handwritten image 51; Fig. 5 is a view showing an example of a combined output 7 0 in the embodiment 1; Fig. 6 is a view showing a configuration of an original sheet SH2 in an embodiment 2 of the present invention; Fig. 7 is a view showing an example of a combined output in the embodiment 2, in case a half-area layout is selected by the user; ig. 8 is a view showing a configuration of an original sheet SH3 in an embodiment 3 of the present invention; Fig. 9 is a view showing a specific example of an operation panel 16 in the image combining apparatus 100; Figs 10-1 and 10-2 are views showing a configuration of an original sheet SH4 to be 10021170 inscribed by the user in an embodiment 6 of the present invention; Figs. 11-1 and 11-2 are views showing operations showing a photo image PI selection and a font/illustration selection (combination print image 52) for printing the original sheet SH4; Fig. 12 is a flow chart showing a process, in the embodiment 6, of printing the original sheet SH4; Fig. 13 is a flow chart showing operations, in the embodiment 6, of printing the original sheet SH4; Fig. 14 is a flow chart showing a process, in the embodiment 6, of reading the printed original sheet SH4 and executing a combining process; Fig. 15 is a flow chart showing a process, in the embodiment 6, of analyzing the original sheet SH4; Figs. 16-1 and 16-2 are views showing a configuration of an original sheet SH5 in an embodiment 7 of the present invention; Fig. 17 is a flow chart showing a process, in the embodiment 7, of printing the original sheet SH5; Figs 18-1 and 18-2 are views showing, in an embodiment 8, an original sheet SH6, bearing a printed image 52 for combining, corresponding to the font/illustration FI, as a reference image 41; and Fig. 19 is a flow chart showing a process, in the embodiment 8, of printing the original sheet SH6. 10021170 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Fig. 1 is a block diagram showing an image combining apparatus 100 constituting an embodiment 1 of the present invention. The image combining apparatus 100 is provided with a CPU 2, an internal bus 3, a program memory 4, a data memory 5, a read control portion 6, a read mechanism 7, a data conversion portion 9, a print control portion 11, a print mechanism 12, an operation panel 16, a memory card control portion 17, an interface control portion 19, an interface 20 and a communication control portion 21. The data memory 5 includes a read buffer memory 8, a print buffer memory 10, and an image memory 13. The operation panel 16 includes a display portion 14 and an operation portion 15. The CPU 2 is constructed as a microprocessor, which functions according to a control program, stored in the program memory 4, constituted of a ROM and connected through the internal bus 3, and to a content of the data memory constituted of a RAM. The CPU 2 operates the read mechanism 7 through the read control portion 6, and stores original image data, read by an unillustrated image sensor provided in the read mechanism 7, in the read buffer memory 8 provided in the data memory 5. The original image data, stored in the read 10021170 buffer memory 8, are read by the data conversion portion 9 for conversion into print data, which are stored in the print buffer memory 10 provided in the data memory 5. The data conversion portion 105 executes image data converting operations, such as an analysis of page description language (PDL) and a CG (computer graphics) development of character data. The CPU 2 operates the print mechanism 12 through the print control portion 11, and also reads the print data stored in the print buffer memory 10 for supply to the print mechanism 12, and executes a printing operation on a print medium, thereby executing a copying operation. The data memory 5 is provided with the image memory 13, which temporarily stores various image data, in addition to the original image data read by the read mechanism 7, and executes an editing thereon Also the image data stored in the image memory 13 are converted by the data conversion portion 9 into print data for printing, whereby a printing operation of a photo image and the like can be realized in addition to the copying operation. The read buffer memory 8, the print buffer memory 10 and the image memory 13, provided in the data memory 5, are managed by the CPU 2, and capacities of these memories are dynamically changed 10021170 and assigned according to the operation status such as an operation mode and a setting by the user. The operation panel 16 is equipped with numeral input keys, character input keys, one-touch telephone number keys, mode setting keys, a determination key, a cancel key etc., and is constituted of an operation part to be used by the user for selecting a destination of image transmission and for registering various registration data or an energy saving mode, various keys, LED (light-emitting diodes) and an LCD (liquid crystal display), thus being utilized for various input operations by the user and for displaying operation statuses of the image combining apparatus 100. The memory card control portion 17 controls an access to a memory card 18 serving as a data memory medium, and reads and writes the image data and the like stored in the memory card 18. Thus the photo image data, stored in the memory card 18, are read, then stored in the image memory 13, further converted into print data and printed. The interface control part 19 controls a communication process by the interface 20, thereby executing data exchanges with an external apparatus 21. As the external apparatus 21, a personal computer may be connected for receiving print data, 10021170 prepared by a printer driver functioning on the computer, and printing such print data. Also a digital camera or the like may be connected for reading photo image data, for storage in the image memory 13 and conversion into print data for printing The communication control portion 22, including a MODEM (modulation-demodulation unit) and an NCU (network control unit), is connected to an analog communication line 23 and executes a communication control and a line control such as a call making or a call reception. Now reference is made to Fig. 9, showing details of the operation panel 16 shown in Fig. 1. The operation panel 16 is equipped with an LCD display 201, a power key 202, a copy mode key 203, a film mode key 204, a scan mode key 205, a photo mode key 206, a menu key 207, a print sheet key 208, a scan sheet key 209, a minus key 210, a plus key 211, a set key 212, a slide show key 213, a sheet setting key 214, a feeder selection key 215, a stop key 216, a monochromatic start key 217, and a color start key 218. The LCD display 201 is used for displaying various information such as a message and an operation prompt. The power key 202 is used for turning on/off the power supply of the image processing apparatus 10021170 100. The copy mode key 203 is used for setting the image processing apparatus 100 in a state capable of a copying operation, and the image processing apparatus 100 is shifted to a copy mode when the copy mode key 203 is depressed. The film mode key 204 is used for setting the image processing apparatus 100 in a state capable of a printing operation from a film, and the image processing apparatus 100 is shifted to a film mode when the film mode key 204 is depressed. The scan mode key 205 is used for setting the image processing apparatus 100 in a state capable of a scanning operation, and the image processing apparatus 100 is shifted to a scan mode when the scan mode key 205 is depressed. The photo mode key 206 is used for setting the image processing apparatus 100 in a state capable of a direct printing operation by a signal from a digital photo card or a digital camera, and the image processing apparatus 100 is shifted to a photo mode when the photo mode key 206 is depressed. The menu key 207 is used for displaying set items for example in executing a copying operation, a facsimile operation or a direct printing from a card. A depression of the menu key 207 causes a display of the set items for execution on the LCD 2 01, and such 10021170 set items may be selected by the plus key 211 and the minus key 210 and set by the set key 212. The print sheet key 208 is’used for printing an original sheet SHI as shown in Fig. 3. The scan sheet key 209 is used for reading the sheet SHI. In the image processing apparatus 100, information written on the sheet/ printed by the print sheet key 208, is read by the read portion 107 for job execution, and the scan sheet key 209 is used for instructing such sheet reading. The minus key 210 and the plus key 211 are used by the user for making a selection from plural candidates in the menu or in a user registration. “Plus” and “minus” respectively indicate a forward shift and a backward shift. The set key 212 is used for determining a selected item. The slide show key 213 is used for displaying images of the memory card MC in succession on the display of the operation panel 16. The sheet setting key 214 is depressed for a sheet setting in the copying or other printing operation. The feeder selection key 215 is used, in case a sheet feeding portion 114 has plural sheet feeder portions, for switching such sheet feeder. The stop key 216 is used for terminating an operation such as a facsimile transmission-reception, a copying operation or other operations - The 10021170 monochromatic start key 217 is used for starting a monochromatic facsimile transmission or a monochromatic copying. The color start key 218 is used for starting a color facsimile transmission or a color photo printing. Fig. 2 shows an original sheet SHI in the embodiment The original sheet SHI is used in case of synthesizing a handwritten image with a photo image serving as a reference image, and includes a setting area 30, a reference image area 40, a handwriting area 50, and a bar code 60. The setting area 30 is provided, for selecting a size of the print medium, a mark 31 for selecting an A4 size, a mark 32 for selecting a postcard, and a mark 33 for selecting an L-size• The user blacks out one of the mark 31 for selecting the A4 size, the mark 32 for selecting the postcard, and the mark 33 for selecting the L-size, thereby selecting a size of the print medium to be used for printing the combined result. The reference image area 4 0 prints a reference image 41. The “reference image” corresponds to a photo image selected by the user, and it is not the photo image itself but is printed from such photo image. The handwriting area 50 is an area where the 10021170 user writes a handwritten image 51 to be combined with the reference image 41. The reference image area 4 0 and the handwriting area 50 have substantially same sizes and substantially same shape, and the handwriting area 50 is provided close to the reference image area 40. The read control portion 6, the read mechanism 7, the data memory 5, the print control portion 11, and the print mechanism 12 constitute an example of original sheet print means for printing the original sheet SHI. As shown in Fig. 2, under the reference image area 4 0 bearing the reference image 41, the handwriting area 50 of a same size as the reference image 41 is provided and the user can draw the handwritten image 51 by an arbitrary writing utensil in the handwriting area 50. The bar code 60 is used for specifying an image stored in the memory card 18, and is for example a file name of the image stored in the memory card 18. In the following, there will be explained a process of the embodiment 1, in which the CPU 1 of the image combining apparatus 100 prints the original sheet SHI. Fig. 3 is a flow chart showing a process of the embodiment 1, in which the CPU 1 of the image combining apparatus 100 prints the original sheet SHI 10021170 The process shown in Fig. 3 is executed when the user selects “original sheet printing” in the menu displayed on the operation panel 16. At first, a step SI reads a photo image data file in the memory card 18 and displays it on the display portion 14. A step S2 monitors a key operation on the operation portion 15 and discriminates whether a photo image is selected. In case a photo image is selected, a step S3 reads the selected photo image (reference image 41) from the memory card 18, and combines it with print data, stored in advance in the program memory 4 for printing in the setting area 30, thereby preparing print data for the original sheet SHI. A step S4 control the print mechanism 12 based on thus prepared print data, thereby printing the original sheet SHI. In the following, there will be explained an operation of the embodiment 1 for reading the original sheet SHI bearing the handwritten image and printing the reference image 41 and the handwritten image 51 in a combined image. Fig. 4 is a flow chart showing the functions of the GPU 2 in the embodiment 1, in case of reading the original sheet SHI bearing the handwritten image and printing the reference image 41 and the handwritten image 51 in a combined image. 10021170 The process shown in Fig. A is executed when the user selects “original sheet reading and combined printing” in the menu displayed on the operation panel 16. At first a step S11 controls the read mechanism 7 thereby reading the original sheet SH1… A step S12 extracts, from the image data of the read original sheet SHI, data of the setting area 30 and analyzes a marked state of the marks 31, 32 and 33 in the setting area 30, thereby acquiring information on the size of the selected print medium. A step S13 extracts, from the image data of the read original sheet SHI, an image of the handwriting area 50 as handwritten image data. The extraction of the handwriting area 50 is executed for example by detecting a frame line indicating the handwriting area 50 of the original sheet SHI in the reading operation, and by extracting the data within such detected frame line. A step S14 analyzes the bar code 60 on the read original sheet SHI, and reads a photo image same as the photo image (reference image 41) selected by the user, from the memory card 18. The bar code 60 indicates, for example, a file name of the image stored in the memory card 18. The read image is combined with the handwritten image data, extracted in S13, to obtain combined image data. 10021170 A step SI5 executes, based on the size selection information of print medium obtained in S12, a process of enlarging or reducing the combined image data, and controls the print mechanism 12 to print a combined image on an output print medium (S16). Fig. 5 shows an example of a combined output result 70 in the embodiment 1. In the combined output result 70, as shown in Fig. 5, the reference image 41 and the handwritten image 51, inscribed in the handwriting area 50, are combined in a superposed manner, and enlarged or reduced so as to match the print medium size selected in the setting area 30, and thus enlarged or reduced combination data (combination result) are outputted on the print medium 71. In the embodiment 1, the user can observe, in drawing the handwritten image 51 in the handwriting area 50 of the original sheet SHI, the reference image 41 provided in an adjacent position in a same size, and can therefore draw the handwritten image 51 while recognizing the positional relationship of the reference image 51 and the handwritten image 51 when both are combined in superposition - In the embodiment 1, the reference image 41 and the handwriting area 50 are arrayed along a vertical direction on the original sheet SHI, but they may be arrayed in a lateral direction. In the embodiment 1, 10021170 the reference image 41 and the handwriting area 50 are provided in a same and in a mutually adjacent relation, but the reference image 41 and the handwriting area 50 need not necessarily be in a same size, as long as the positional relationship does not become very unclear. Embodiment 2 Fig 6 shows a configuration of an original sheet SH2 in an embodiment 2 of the present invention. The original sheet SH2 includes a setting area 30a, a reference image area 40, a handwriting area 50, and a bar code 60. The setting area 30a is provided, for selecting a size of the print medium, a mark 31 for selecting an A4 size, a mark 32. for selecting a postcard, and a mark 33 for selecting an L-size, and for selecting a print layout, a mark 34 for selecting an entire area and a mark 35 for selecting a half area. The user blacks out one of the mark 31 for selecting the A4 size, the mark 32 for selecting the postcard, and the mark 33 for selecting the L-size, thereby selecting a size of the print medium to be used for printing the combined result. Also, when the user blacks out the mark 34 for selecting the entire area (selecting an entire area layout), an image formed by superposing the reference image 41 and the handwritten image 51 is printed on the entire 10021170 area of the print medium 71, while, when the user blacks out the mark 35 for selecting the half area (selecting a half-area layout), the reference image 41 is printed in a half area of the print medium 71 and the handwritten image 51 is printed on a remaining half area of the print medium 71. Fig. 7 shows an example of a combined output result when the user selects the half-area layout in the embodiment 2. In such case, the reference image 41 and the handwritten image 51, drawn in the handwriting area 50, are combined side by side (that is, a mutually abutting combination of the photo image corresponding to the reference image and the handwritten image), then the combined image is enlarged or reduced according to the print medium size selected by the setting area 30a, and printed with the layout according to the designation by the user. The combined output result, in case the user selects the entire-area layout, is same as the result 70 in the embodiment 1. In the embodiment 2, the reference image 41 and the handwriting area 50 are positioned on the original sheet SH2 as shown in Fig. 6, and, when the entire-area layout is selected, the reference image 41 and the handwritten image 51 drawn in the handwriting area 50 are combined in a superposed 10021170 manner, while, when the half-area layout is selected, the reference image 41 and the handwritten image 51 drawn in the handwriting area 50 are arrayed in the same positional relationship. There the user, in drawing the handwritten image 51 in the handwriting area 50 of the original sheet SH2, can easily recognize the result of combination with the reference image 41. Also the embodiment 2 allows to achieve the entire-area layout and the half-area layout by a single original sheet SH2. The embodiment 1 shows a case where the original photo image (reference image 41) is a laterally oblong so-called landscape image, but, in a case where the original photo image (reference image 41) is a vertically oblong so-called portrait image, the reference image 41 is printed in a laterally oblong position so that the user can conveniently understand the direction of the image. Embodiment 3 Fig. 8 shows a configuration of an original sheet SH3 in an embodiment 3 of the present invention The original sheet SH3 includes a setting area 30a, a reference image area 4 0a, a handwriting area 50a, and a bar code 60. The setting area 30a is same as that shown in Fig. 6. 10021170 Then reference image area 40a prints a reference image 41, which is provided with a frame line 42, scale lines 4 3 and grid lines 4 4. The frame line 42 surrounds the reference image area 4 0a. The scale lines 4 3 are provided to divide the vertical and horizontal lines of the frame line 42, and the grid lines 44 are provided to form a grid including the scale lines 43. The handwriting area 50a is used by the user for drawing a handwritten image 51 and is provided with a frame line 52 and scale lines 53. The frame line 52 surrounds the handwriting area 50a. The scale lines 53 are provided to divide the vertical and horizontal lines of the frame line 52. The frame lines 42 and 52 have substantially same shapes, and the scale lines 43, 53 are provided in mutually approximately same positions of the frame lines. The print mechanism 12 and the print control portion 11 constitute an example of the original sheet printing means, which serves to print the reference image frame line 42 on the reference image 41, also to print the handwriting area frame line 52 on the handwriting area 50, to print also the scale lines dividing the reference image frame line 43, and also to print the scale lines 53 in the positions, 10021170 on the handwriting area frame line 52, substantially corresponding to those of the scale lines 43 on the reference image frame line 42. The original sheet printing means also prints, on the reference image 41, the grid lines including the scale lines 43 on the reference image frame line 42. In the embodiment 3, the frame lines, the scale lines and the grid lines are attached to the handwriting area 50 and the reference image 41 on the original sheet SH3, thereby further facilitating to understand the positional relationship at the image combining operation. The embodiment 3 shows a case of adding all the frame lines 42, the scale lines 43 and the grid lines 44, but it is also possible to add the frame lines 42 and the scale lines 43 only. Embodiment 4 In an embodiment 4 of the invention, in the printing of the original sheet in any one of the embodiments 1 to 3, the reference image 41 is printed with a lowered print density. As the final output of the combined image utilizes the image read out from the memory card 18, while the reference image 41 printed on the original sheet is used only temporarily as a reference for the drawing of the handwritten image 51 by the user, so 10021170 that the density of the reference image 41 is not reflected on the final output. Therefore, the reference image 41 may be printed with a lowered print density as long as it allows to confirm the mutual positional relationship of the handwritten image 51 and the reference image 41 at the image combining, and such lowered print density is effective in reducing an ink consumption in case an ink jet printing apparatus is employed as the print mechanism 12. In outputting a combined result, the reference image to be printed in the reference area may be printed, instead of lowering the print density, with a smaller number of colors, or as a monochromatic image. Stated differently, upon outputting the combined result, printing may be made with an image quality different from that in the normal printing operation. Such “image quality” means parameters controlling the quality of an image, such as an ink density, a number of colors, a monochromatic or color, and an image resolution. Embodiment 5 In an embodiment 5 of the present invention, in any of the embodiments 1 to 4, the photo image data are entered, instead of from the memory card 18, from the reading means, same as that used for reading the original. 10021170 The photo image data may also be obtained, instead of from the reading means used for reading the original, from an external equipment through the communication line 23 or through the interface 20. It is also possible, in any one of the memory medium such as the memory card 18, or to embodiments 1 to 4, to confirm the combined image result by displaying on a display portion of the operation panel 16, or to store the combined result in a transfer the combined result to another equipment through the communication line 23 or through the interface In this manner, the image after combining operation may be edited in such another equipment. Figs. 10-1 and 10-2 show an original sheet SH4 to be inscribed by the user in the embodiment 6. More specifically, Fig. 10-1 shows an original sheet SH4 used in the embodiment 6, and Fig. 10-2 shows an example of the combined output 90 in the embodiment 6. The original sheet SH4 includes a setting area 30, a reference image area 40, a handwriting area 50 and a bar code 60. The original sheet SH4 is printed utilizing the operation panel 16 of the image processing apparatus 100, and a photo image, stored in the memory card MC connected to a USB host control portion 117, and a 10021170 content inscribed in the original sheet SH4 are subjected to a combining process. More specifically, at first a setting for printing the original sheet SH4 is executed in the image processing apparatus 100 through the operation panel 16, and the recording portion 113 executes a printing operation according to such setting. In such printing, the handwriting area 50 for inscribing a content for image synthesis is printed on the original sheet SH4, then the reading portion 107 reads such handwriting area 50 of the original sheet SH4 for synthesis with a photo image PI corresponding to the reference image 41, and the result of such synthesis is printed by the recording portion 113 to achieve an image combining process. In the embodiment 6, the print sheet key 208 is depressed for printing the recording sheet SH4 . In the following, details of the printing operation of the original sheet SH4 will be explained with reference to Figs. 11-1 and 11-2, and details of the operations for reading the original sheet SH4 and executing the image combining will be explained with reference to Fig. 12. The setting area 30 is used for designating a size of a sheet for printing the image combined by using the original sheet SH4. In the embodiment 6, such size designation is executed by a marking sheet, 10021170 with printed marks corresponding to different sheet sizes. At the image combining operation, the reading portion 107 reads the setting area 30 while reading the original sheet SH4, and the output size of the combined image is determined according to the marked size in the setting area 30. The reference image area 4 0 prints, as a sample, an image same as the photo image PI, and, in the image combining operation utilizing the original sheet SH4, data obtained by reading the image drawn in the handwriting area 50 and a photo image corresponding to the reference image 41 printed in the reference image area 4 0 are combined. The reference image 41 indicates an object image for combination, to be used as a reference when the user inscribes in the handwriting area 50, but the data for actual combining operation are not extracted from the original sheet SH4 but are obtained from the photo image stored in the digital camera DC. The handwriting area 50 is used for inscribing a content, to be combined with a photo image PI, corresponding to the reference image 41 printed in the reference image area 40. An inscription is made in the handwriting area 50, and, when the reading portion 107 reads the original sheet SH4 at the image combining operation, a content inscribed in the handwriting area 50 is taken as a content for image 10021170 combination. In the embodiment 6, at the image combining operation, a white background is disregarded, and a handwritten image 51 drawn by the user and a printed combining image 52 are regarded as a content of image combination and are used for the image combining operation. The handwriting area 50 and the reference image area 4 0 have mutually same sizes, so that the user can judge an actual size at the.inscription in the handwriting area 50. The combination print image 52 is an image of a font/illustration selected by the user at the printing of the original sheet SH4. The data for such image are present in the image processing apparatus 100 for printing the original sheet SH4, and, at the printing of the original sheet SH4, the frame of the handwriting area 50 and the image 52 for combination are printed in such a manner that the image 52 for combination is printed within the frame of the handwriting area 50 as shown in Fig. 10-1. The “font” above indicates at least one of a character font, a style, a size, a color and a character decoration. The handwritten image 51 shows an example of the content which the user actually draws on the original sheet SH4. The handwritten image 51 is absent at the printing of the original sheet SH4 and is later inscribed by the user in the handwriting 10021170 area 50 after the original sheet SH4 is printed, and is thereafter combined with the photo image corresponding to the reference image 41. In the image combining operation utilizing the original sheet SH4, the print image 52 and the handwritten image 51 in the handwriting area 50 are read as a single integral image and are combined with the photo image PI corresponding to the reference image 41 printed in the reference image area 40. In the bar code 60 recorded on the original sheet SH4, embedded is information of the photo image corresponding to the reference image 41 (for example ID of the photo image). A photo image corresponding to the reference image 41 can be extracted by reading and analyzing the data of the bar code 60. The output example 90 of the combined image shows an example of printing after the image combining process. In the image combining utilizing the original sheet SH4, a first object image for combining and a second object image for combining are combined. The “first object image for combining” is a photo image corresponding to the reference image 41 printed in the reference image area 4 0. Also the “second object image for combining” is an image obtained by integrally reading the print image 52 and the handwritten image 51 described in the handwriting 10021170 area 50 as a single image. Such two object images for combining are combined and printed to provide an output example 90 of the combined image. The embodiment 6 allows, by a single combining process, to synthesize three images, namely the combination print image 52, the handwritten image 51 and the photo image PI corresponding to the reference image 41. Figs. 11-1 and 11-2 illustrate a process of selecting the photo image PI and font/illustration (combination print image 52), in order to print the original sheet SH4. In printing the recording sheet SH4, the reference image 41 and the combination print image 52 are printed thereon. For this purpose, the reference image 41 and the combination print image 52 are selected prior to the printing of the original sheet SH4. Fig. 11-1 shows a photo image selecting display screen 100 in the embodiment 6. The photo image selecting display screen 100 is an example of the image displayed on the display portion 201 for selecting a photo image. The display portion 201 displays a photo image PI, which is a candidate to be used in image combining. A desired photo image PI is selected among plural photo images 10021170 by operating minus/plus buttons 210, 211 provided on the operation panel 16, and is determined by the set key 212. Fig. 11-2 shows a print image selecting display screen 110 in the embodiment 6. The print image selecting display screen 110 is an example of the display screen displayed on the display portion 201, in order to select the combination print image (font/illustration FI) 52. The font/illustration FI is a candidate of the font/illustration to be printed in the handwriting area 50 as the combination print image 52. A desired font/illustration FI is selected among plural font/illustrations by operating minus/plus buttons 210, 211 provided on the operation panel 16, and is determined by the set key 212. Fig. 12 is a flow chart showing operation in the embodiment 6 for printing the original sheet SH4. In the embodiment 6, the operation panel 16 includes the print sheet key 208, which is used for executing the printing operation. Also the LCD 201 displays a photo image PI and a font/illustration FI. At first a step SI selects a printing of the original sheet SH4. In the embodiment 4, a mode of printing the original sheet SH4 can be selected by operating the print sheet key 208 in the operation panel 16 as shown in Fig. 9. 10021170 Then a step S2 displays, as shown in Fig. 11-1, a photo image selecting display screen 100. More specifically, there is displayed a photo image PI stored in the memory card MC, connected to the LCD 201 of the operation panel 16. Then, in a step S3, the user operates the minus/plus keys 210, 211 provided in the operation panel 16 to select a photo image PI, and then operates the set key 212 to determine the displayed photo image PI. On the other hand, in case the photo image is not yet selected, the step S2 continues to displays the photo image selecting image display screen 100. Then a step S4 displays, as shown in Fig. 11-2, a print image selecting display screen 110, namely a font/illustration FI. A step S5 confirms whether the font/illustration FI has been selected. The selection of the font/illustration FI can be made, as 20 in the step S3, by the minus/plus keys 210, 211 and the set key 212. In the case that the font/illustration FI is selected, the sequence proceeds to a step S6. In case the selection is not made, the step S4 continues to display the printed image selecting display screen 110, thus showing the font/illustration FI. A step S6 prints the original sheet SH4. In 10021170 this operation, the recording portion 113 prints the original sheet SH4, based on the photo image PI selected in S3 and the font/illustration FI selected in S5, and according to the user description data in a data storage portion 116 of the image processing apparatus 100. Fig. 13 is a flow chart showing a printing process of the original sheet SH4 in the embodiment 6 At first a step S30 prepares an image, selected in S2, as an image for the original sheet. In the embodiment 6, a photo image PI, selected by the user as a reference for the inscription by the user, in the original sheet SH4. A step S31 confirms a memory capacity, in order to confirm whether the combining of the photo image PI and the font/illustration FI is possible. Then the sequence proceeds to S35 or S32, respectively according to whether the memory capacity, necessary for the image combining, is available or not. In case the memory capacity necessary for the image combining is not available, the font/illustration FI may be written in advance into the handwriting area 50, whereby, at the combining with the photo image PI later, the handwritten image 51 and the font/illustration FI can be processed as single image data. Therefore, in the embodiment described above, 10021170 the image combining process require only two memories for developing two sets of data. A step S32 prepares an image of the font/illustration, selected in S4, for printing on the original sheet SH4 . A step S33 matches the images, prepared in S30 and S32, with the image of the original sheet SH4 to be actually printed. A step S34 executes the printing of the original sheet SH4. A step S35, utilizing the memory for image combining, matches the image prepared in S30 with the image of the original sheet SH4 to be actually printed. A step S34 executes the printing of the original sheet SH4. In this operation, in the bar code 60 of the original sheet SH4, there is embedded information of the photo image to be combined, and, in case the font/illustration is selected in S4, there is embedded information indicating such selection. In this manner, the original sheet SH4 can be printed with the photo image selected prior to the printing of the original sheet SH4 as explained in Fig. 12, and the printed image (image of font/illustration). In the embodiment 6, the handwritten image 51, 10021170 the photo image PI selected by the user and the combination print image 52 are printed in the original sheet SH4. Fig. 14 is a flow chart showing an operation of the embodiment 6 for reading the printed original sheet SH4 and executing an image combining process. Thus the operations shown in Fig. 14 are those for reading the printed original sheet SH4 by the reading portion 107 and combining thus read image and the photo image PI corresponding to the reference image 41. This process is executed, after the printing of the original sheet SH4, for actually combining the photo image, selected in S3 in Fig. 12, and a content of combination desired by the user (image drawn in the handwriting area 50) At first a step S40 selects a “mode for original sheet reading and photo printing” displayed on the operation panel 16. In the embodiment 6, such 20 “mode for original sheet reading and photo printing” is selected by depressing the scan sheet mode 209. A step S41 displays, on the display portion (LCD) 201 of the operation panel 16, a message requesting the user to place the original sheet SH4 on an original support of the reading portion 107. A step S42 confirms that the original sheet SH4 is placed on the original support. Such confirmation 10021170 may be executed by the image processing apparatus 100, or by a request to the user. When the original sheet SH4 is placed on the original support, the sequence proceeds to S43, but, in case the original sheet cannot be confirmed, S41 is executed again to display the message requesting the placement of the original sheet SH4. A step S44 reads the original sheet SH4 placed on the original support of the reading portion 107. The data thus read are stored in the image memory 104 and are analyzed. Information of the combination print image 52 is thus obtained, by such analysis, from the original sheet SH4. The detailed process will be explained later with reference to Fig. 15. A step S45 combines and prints the photo image PI selected in S3, the combination print image 52 and the handwritten image 51, read and analyzed in S44. Fig. 15 is a flow chart showing operations of analyzing the original sheet SH4 in the embodiment 6. At first, a step S50 reads the information of the bar code 60 of the original sheet SH4. A step S51 acquires image information from the information of the read bar code 60. The information of the selected photo image is incorporated, in S3, in the bar code 60. Thus the original sheet SH4 contains information indicating the photo image constituting the object of combining, and it is 10021170 possible to avoid the trouble of selecting the photo image again, by merely reading the original sheet SH4 Also in case S31 in Fig. 13 identifies that the memory required for the image combining is available, the process of printing the font/illustration image on the original sheet SH4 is not executed. When the font/illustration FI selected in S4 is not printed in the handwriting area 50 of the original sheet SH4, information indicating the font/illustration FI of the object of image combining is incorporated in the bar code 60, and the step S51 also extracts such information. A step S52 detects the position of the handwriting area 50 in the original sheet SH4. A step S53 reads the combination print image 52 printed in the handwriting area 50 and the content (handwritten image) drawn by the user and extracts the analyzed data. In such extracting operation, there is also extracted a mark position in the setting area 30, positioned above the handwriting area 50. Through such extracting operation, the printed data and the drawn data are handled as a single read image for image combining. S53 also extracts the information on print size, and the data of the combination print image 52, printed in the handwriting area 50 of the original sheet SH4, and 10021170 the data of the handwritten image 51 drawn by the user can be handled as a single set of data, whereby a pseudo image synthesis is thus achieved. In the embodiment 6, in printing the original sheet SH4, the reference image 41 corresponding to the photo image PI to be combined is printed in advance on the original sheet SH4, and then, in reading the original sheet SH4, the handwritten image 51 drawn by the user and the combination print image 52 are read and discriminated as same combining data and synthesized with the photo image PI. Therefore, at the image combining operation, the combination print image 52 and the handwritten image drawn by the user are processed as a single image, and a combining process of the combination print image 52 and the handwritten image 51 can be dispensed with. Embodiment 7 In an embodiment 7 of the present invention, the reference image 41 used as an index printing is not printed on the original sheet SH4. Figs. 16-1 and 16-2 show original sheets SH5 of the embodiment 7. The operation panel 16 shown in Fig. 9 and the photo image selecting display screen and the combining print image selecting display screen shown in Fig. 11-1 are used in the same manner also in the embodiment 7. 10021170 The original sheet SH5 includes a setting area 30, a handwriting area 50 and a bar code 60. In the embodiment 7, as the reference image 41, printed in the embodiment 6, is not printed on the original sheet SH5, it is possible to reduce the ink consumption in printing the original sheet SH5. The original sheet SH5, not containing the reference image as an index, is printed through the operation panel.16 of the image processing apparatus 100. In the image processing apparatus 100, the operation panel 16 executes a setting for printing the original sheet SH5, and the recording portion 113 executes the printing of the original sheet SH5. The thus printed original sheet SH5 contains the handwriting area 50 for drawing a handwritten image to be combined, then the handwriting area 50 of the original sheet SH5 is read by the reading portion 107 and is combined with the photo image PI corresponding to the reference image 41, and a result of such combining is printed by the recording portion 113 to achieve an image combining. In the embodiment 7, the print sheet key 208 is used for printing the original sheet SH5. The details of the printing of the original sheet SH5 are shown in Fig. 12, and those of the operation for reading the original sheet SH5 for image combining 10021170 are shown in Fig. 13. The setting area 30 is used for designating the size of a sheet for printing the combined image. In the embodiment 7, the setting area 30 is formed as a marking sheet, with printed marks corresponding to different sheet sizes. When the reading portion 107 reads the original sheet SH5 for image combining, it also reads the setting area 30 and determines the output size of the combined image according to the marked size. The handwriting area 50 is used for inscribing a content to be combined. An inscription is made in the handwriting area 50, and, when the reading portion 107 reads the original sheet SH5 at the image combining operation, a content inscribed in the handwriting area 50 is taken as a content for image combination. In the embodiment 7, at the image combining operation, a white background is disregarded, and a handwritten image 51 drawn by the user for example by a pen and a printed combining image 52 are regarded as a content of image combination and are used for the image combining operation. The combination print image 52 is an image of a font/illustration selected by the user at the printing of the original sheet SH5. The data for such image are present in the image processing 10021170 apparatus 100 for printing the original sheet SH5, and, at the printing of the original sheet SH5, are printed in the handwriting area 50 of the original sheet SH5. The handwritten image 51 is an example of the content inscribed by the user on the original sheet SH5. The handwritten image 51 is not yet recorded when the original sheet SH5 is printed. In the image combining process utilizing the original sheet SH5, the handwriting area 50 is read, and the combination print image 52 and the handwritten image 51 are read as a single object for image combining. In the bar code 60, embedded is information of the photo image PI selected for image combining and the combination print image 52. Such data are read and analyzed to extract the image information. The embodiment 7 utilizes a single combining operation, which combines three objects, namely the combination print image 52, the handwritten image 51 and the photo image PI. In the following, there will be explained the operations of printing the original sheet SH5. Fig. 17 is a flow chart showing a printing process of the original sheet SH5 in the embodiment 7. At first a step S60 prepares an image of the font/illustration, selected in S4, for printing on 10021170 the original sheet SH5. A step S61 matches the font/illustration FI, prepared in S60, with the combination print image 52 of the original sheet SH5 to be actually printed. A step S62 executes the printing of the original sheet SH5. In this manner, the combination print image 52 is printed on the original sheet SH5, then the user draws the handwritten image 51 on the original sheet SH5, whereby the images 51, 52 are coupled (inscribed in a same area) but are not subjected to an image combining. In reading the original sheet SH5, the combination print image 52 and the handwritten image 51 drawn by the user are read and combined, as a single object for combining, with the photo image PI. In the embodiment 7, the combination print image 52 and the handwritten image 51 are used as a single object of image combining, without an image synthesis, and the number of combining operations can be reduced in comparison with a case where the combination print image 52 and the handwritten image 51 are combined and the result of such combining is combined with the photo image PI. Embodiment 8 In an embodiment 8 of the present invention, a font/illustration image CI is printed also on the 10021170 reference image 41 on the original sheet SH4. The operation panel 16 shown in Fig. 9 and the photo image selecting display screen and the combination print image selecting display screen shown in Fig. 11-1 are used in the same manner also in the embodiment 8. Figs. 18-1 and 18-2 show original sheets SH6 of the embodiment 8, in which a combining image 52, corresponding to the font/illustration FI, is printed in the reference image 41. In the embodiment 8, the original sheet SH6 is utilized to combine the photo image, stored in the memory card MC connected to the USB host control portion 117, and a content inscribed in the original sheet SH6. More specifically, at first a setting for printing the original sheet SH6 is executed in the operation panel 16, and the recording portion 113 prints the original sheet SH6. The printed original sheet SH6 contains the handwriting area 50 for inscribing a content to be combined, then the reading portion 107 reads such handwriting area 50 for synthesis with a photo image PI corresponding to the reference image 41, and the result of such synthesis is printed by the recording portion 113 to obtain a combined print example 73 as shown in Fig. 18-2. In the embodiment 8, the print sheet key 208 is 10021170 used for printing the recording sheet SH6. Details of the printing operation of the original sheet SH6 are same as those shown in Fig. and those of the operations for reading the original sheet SH6 and executing the image combining are same as those shown in Fig. 13. The setting area 30 is used for designating the size of printing of the combined image, and is formed as a marking sheet, with printed marks corresponding to different sizes. When the reading portion 107 reads the original sheet SH6 for image combining, it also reads the setting area 30 and determines the output size of the combined image according to the marked size. The handwriting area 50 is used for inscribing a content to be combined (combination print image 52 and handwritten image 51). An inscription is made in the handwriting area 50, and, when the reading portion 107 reads the original sheet SH6 at the image combining operation, a content inscribed in the handwriting area 50 is taken as a content for image combination. At the image combining operation, a white background is disregarded, and a handwritten image 51 drawn by the user for example by a pen and a combination print image 52 are regarded as a content of image combination and are used for the image 10021170 combining operation. The reference image 41 is an image corresponding to the photo image PI. In printing the original sheet SH6, a combination print image 52, to be printed in the handwriting area 50, and a photo image PI, corresponding to the reference image 41 to be printed in the reference image area 40, are combined. The reference image 41 is a sample indicating the photo image PI constituting the object of combining, for the drawing of the handwritten image 51 by the user in the handwriting area 50, and, in the actual combining operation, the data of the photo image PI are not extracted from the original sheet SH6 but from the bar code 60. The handwriting area 50 and the reference image area 40 have mutually same sizes, so that the user can judge an actual size at the inscription in the handwriting area 50. The combination print image 52 is an image of a font/illustration selected by the user at the printing of the original sheet SH6. The data for such image are present in the image processing apparatus 100 for printing the original sheet SH6, and, at the printing of the original sheet SH6, are printed in such a manner that the image 52 for combination is printed within the handwriting area 50 The handwritten image 51 is an example of the 10021170 content inscribed by the user on the original sheet SH6. The handwritten image 51 is not yet recorded when the original sheet SH6 is printed. In the image combining process utilizing the original sheet SH6, the handwriting area 50 is read, and the combination print image 52 and the handwritten image 51 are read as a single object for image combining, which is then synthesized with the photo image PI, corresponding to the reference image 41 printed in the reference image area 4 0. The bar code 60 is information embedded in the original sheet SH6. In the embodiment 8, embedded is the information of the photo image PI, selected as the object of combining. Such data are read and analyzed to extract the image information. The font/illustration FI is combined with the photo image PI, and the combination print image 52 and the reference image 41 are printed. As the reference image area 4 0 prints the reference image 41 and the combination print image 52, the user can know in advance, when inscribing the handwritten image 51 in the handwriting area 50, a state after the combination print image 52 and the reference image 41 are combined. The example of combined printing 73 shows an example of printing. In a combining operation utilizing the original sheet SH6, the handwritten 10021170 image 51 and the combination print image 52 in the handwriting area 50 are used as an object of image combining, and are subjected to a combining process with original data of the image printed in the reference image area 4 0, whereby a combined image 7 3 can be printed in the reference image area 4 0 as shown in Fig. 18-2. In the following, there will be explained operations of printing the original sheet SH6 in the embodiment 8. Fig. 19 is a flow chart showing a printing process of the original sheet SH6 in the embodiment 8 At first a step S70 prepares a photo image PI, selected in S2, as a reference image 41 for the original sheet SH6. In the embodiment 8, a reference image 41 corresponding to the photo image PI, selected by the user, is printed in the original sheet SH6; A step S71 prepares an image of the font/illustration FI, selected in S4, as a combination print image 52 for the original sheet SH6 a step S72 combines the font/illustration FI prepared in S71, with the photo image PI prepared in S70. Such combined image (reference image 41 and combination print image 52) is used as a reference index of the original sheet SH6. A step S73 matches the images, prepared in S72, 10021170 with the original sheet SH6 to be actually printed. A step S7 4 executes the printing of the original sheet SH6. In this manner, the original sheet SH6 can be printed with the reference image 41 corresponding to the photo image PI selected prior to the printing of the original sheet SH6 as explained in Fig, 12, and the combination print image 52 corresponding to the font/illustration FI. In the embodiment described above, at the printing of the original sheet SH6 for inputting a handwritten information of the user for image combining operation, an image to be combined is printed in advance on the original sheet SH6, and, in reading the original sheet SH6, the handwritten image 51 drawn by the user and the combination print image 52 are read as a single object of combining, and such read image is combined with the photo image PI. Therefore, in such embodiment, a process of synthesizing the printed combining image 52 and the handwritten image 51 is not required in the image combining operation, thereby allowing to reduce the number of image combining operations. The embodiments 6, 7 and 8 are different in whether the user prints the font/illustration FI as a reference index in the reference image area 40 of the original sheet, but all these embodiments are 10021170 effective regardless whether such reference index is printed or not. The foregoing embodiments can also be understood as inventions of a program. Stated differently, the foregoing embodiments constitute examples of a program which causes an image processing apparatus to execute a photo image selecting step of selecting a desired photo image from plural photo images stored in photo image storage means, a font/illustration selecting step of selecting a desired font/illustration from plural font/illustrations, and an original sheet preparing step of printing a reference image, corresponding to the selected photo image, in a reference image area, also printing an image for combining corresponding to the selected font/illustration in a handwriting area, in such a manner that the reference image area and the handwriting area are mutually adjacent, thereby preparing an original sheet. Also the foregoing embodiments constitute examples of a program which causes an image processing apparatus to execute a photo image selecting step of selecting a desired photo image from plural photo images stored in photo image storage means, a font/illustration selecting step of selecting a desired font/illustration from plural font/illustrations, an original sheet preparing step 10021170 of printing a combining image, corresponding to the selected font/illustration in a handwriting area, a read step of reading the handwritten image inscribed in the handwriting area and the combination print image, and a combining step of combining the combination print image and the handwritten image, which are read in the reading step, and a photo image corresponding to the reference image. The present invention provides an effect allowing to draw a handwritten original, at the time of such drawing on a sheet, while confirming the positional relationship how the handwritten original• is combined with the photo image as the object of image combining. The present invention also provides an effect of not requiring a large burden on resources necessary for image combining nor a long process time, even with a large amount of data to be combined such as photo image data, data written by a user, a photo 2 0 frame and the like.

Full Text

IMAGE COMBINING APPARATUS, AND CONTROL METHOD AND
PROGRAM THEREFOR
BACKGROUND OF THE INVENTION
Field of the Invention
The present invention relates to an image combining apparatus for combining photographic image data entered from image input means, and data read from a handwritten original (manuscript). Related Background Art
Along with the recent pervasiveness and the technical advances in photographic image input apparatuses such as a digital camera and a color scanner, and in image output apparatus such as a color printer, various methods of use are being proposed on digital photo images.
Such images are not only captured and printed on a printing medium, but are often utilized with various processing and editing in various manners. For example it is proposed to send a photograph with an inscribed message as a letter, and also to paste a photograph with an inscribed title or illustration to an album.
For adding a handwritten message or
illustration to a photo image entered from an image input apparatus, Japanese Patent Application Laid-open No. S63-040933 discloses a method of displaying

the input photo image on a display apparatus such as a liquid crystal panel, and inputting a handwritten image through a touch-panel input apparatus provided in the display apparatus, thereby combining the photo image and the handwritten image.
Also Japanese Patent Application Laid-open No. H09-139832 discloses a method in which a handwritten original, written on a paper sheet different from an input photo image, is read by a reading apparatus and is combined with the photo image.
Also the printing apparatus is increasingly provided, not only a printing function, with complex functions such as an image reading function and a memory card slot for accepting a memory card, and Japanese Patent Application Laid-open No. H05-122496 discloses a method, in such printing apparatus, of combining and printing a photo image in a memory card and data read in a reading unit.
Also such image combining is not limited to a photo image data and a handwritten image, but the data superposed with the photo image also include a photo frame or a message of a predetermined pattern.
However, in the case of drawing a message or an illustration on a photo image, by selecting a background area other than a person recorded as a main object, the prior methods involve following limitations.

10021170
The touch-panel method mentioned above allows to draw a handwritten character or an illustration directly on a photo image while observing the photo image, but requires a display apparatus and a touch-panel input apparatus which lead to a high hardware cost, and an ability of expression is inevitably restricted as the image resolution of the handwritten input is limited by the resolution of the display apparatus or the touch panel.
On the other hand, the original reading method mentioned above has the advantages that a handwritten original can be prepared in a simple and easy manner and that various expressions are possible depending on the type of the writing utensil, but involves a drawback that, as the handwritten original is drawn
on an original sheet separate from the photo image, a superposing positional relation of the handwritten image with the photo image cannot be confirmed exactly at the preparation of the handwritten
original
Also with an increase in the data to be combined, such as photo image data, data handwritten by the user, a photo frame etc., the combining has to be executed plural times on the data of plural
contents, thereby resulting in a large burden on the resources used for the combining and also in a long process time*

10021170
Also at the image combining operation, it is not possible to know how the content to be combined and the content written by the user are positioned after the image combining.
SUMMARY OF THE INVENTION
An object of the present invention is to provide an image combining apparatus which allows to draw a handwritten original, while confirming, at the
drawing operation of the handwritten original on a
sheet, a positional relationship how the handwritten original is to be combined with respect to an object photo image, and a control method and a program therefor.
Another object of the present invention is to provide an image combining apparatus that does not involve a large burden on resources required for image combining nor a long process time, even with a large amount of data to be combined such as photo
image data, data written by a user, a photo frame etc., and an image processing method and a program therefor.
BRIEF DESCRIPTION OF THE DRAWING
Fig. 1 is a block diagram of an image combining apparatus 100 constituting an embodiment 1 of the present invention;

10021170
Fig. 2 is a view showing an original sheet SHI in the embodiment 1;
Fig. 3 is a flow chart showing, in the embodiment 1, a process executed by a CPU 2 of the image combining apparatus 100 for printing the original sheet SHI;
Fig. 4 is a flow chart showing, in the embodiment 1, a process executed by the CPU 2 in reading the original sheet SHI bearing a handwritten image and combining and printing a reference image 41 and a handwritten image 51;
Fig. 5 is a view showing an example of a combined output 7 0 in the embodiment 1;
Fig. 6 is a view showing a configuration of an original sheet SH2 in an embodiment 2 of the present invention;
Fig. 7 is a view showing an example of a combined output in the embodiment 2, in case a half-area layout is selected by the user;
ig. 8 is a view showing a configuration of an original sheet SH3 in an embodiment 3 of the present invention;
Fig. 9 is a view showing a specific example of an operation panel 16 in the image combining apparatus 100;
Figs 10-1 and 10-2 are views showing a configuration of an original sheet SH4 to be

10021170
inscribed by the user in an embodiment 6 of the present invention;
Figs. 11-1 and 11-2 are views showing operations showing a photo image PI selection and a font/illustration selection (combination print image 52) for printing the original sheet SH4;
Fig. 12 is a flow chart showing a process, in the embodiment 6, of printing the original sheet SH4;
Fig. 13 is a flow chart showing operations, in the embodiment 6, of printing the original sheet SH4;
Fig. 14 is a flow chart showing a process, in the embodiment 6, of reading the printed original sheet SH4 and executing a combining process;
Fig. 15 is a flow chart showing a process, in the embodiment 6, of analyzing the original sheet SH4;
Figs. 16-1 and 16-2 are views showing a configuration of an original sheet SH5 in an embodiment 7 of the present invention;
Fig. 17 is a flow chart showing a process, in
the embodiment 7, of printing the original sheet SH5;
Figs 18-1 and 18-2 are views showing, in an embodiment 8, an original sheet SH6, bearing a printed image 52 for combining, corresponding to the font/illustration FI, as a reference image 41; and
Fig. 19 is a flow chart showing a process, in the embodiment 8, of printing the original sheet SH6.

10021170
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Fig. 1 is a block diagram showing an image combining apparatus 100 constituting an embodiment 1 of the present invention.
The image combining apparatus 100 is provided with a CPU 2, an internal bus 3, a program memory 4, a data memory 5, a read control portion 6, a read mechanism 7, a data conversion portion 9, a print control portion 11, a print mechanism 12, an
operation panel 16, a memory card control portion 17, an interface control portion 19, an interface 20 and a communication control portion 21.
The data memory 5 includes a read buffer memory 8, a print buffer memory 10, and an image memory 13.
The operation panel 16 includes a display portion 14 and an operation portion 15.
The CPU 2 is constructed as a microprocessor, which functions according to a control program, stored in the program memory 4, constituted of a ROM and connected through the internal bus 3, and to a
content of the data memory constituted of a RAM. The CPU 2 operates the read mechanism 7 through the read control portion 6, and stores original image data, read by an unillustrated image sensor provided in the read mechanism 7, in the read buffer memory 8 provided in the data memory 5.
The original image data, stored in the read

10021170
buffer memory 8, are read by the data conversion portion 9 for conversion into print data, which are stored in the print buffer memory 10 provided in the data memory 5.
The data conversion portion 105 executes image data converting operations, such as an analysis of page description language (PDL) and a CG (computer graphics) development of character data.
The CPU 2 operates the print mechanism 12
through the print control portion 11, and also reads the print data stored in the print buffer memory 10 for supply to the print mechanism 12, and executes a printing operation on a print medium, thereby executing a copying operation.
The data memory 5 is provided with the image memory 13, which temporarily stores various image data, in addition to the original image data read by the read mechanism 7, and executes an editing thereon Also the image data stored in the image memory 13 are
converted by the data conversion portion 9 into print data for printing, whereby a printing operation of a photo image and the like can be realized in addition to the copying operation.
The read buffer memory 8, the print buffer memory 10 and the image memory 13, provided in the data memory 5, are managed by the CPU 2, and capacities of these memories are dynamically changed

10021170
and assigned according to the operation status such as an operation mode and a setting by the user.
The operation panel 16 is equipped with numeral input keys, character input keys, one-touch telephone number keys, mode setting keys, a determination key, a cancel key etc., and is constituted of an operation part to be used by the user for selecting a destination of image transmission and for registering various registration data or an energy saving mode,
various keys, LED (light-emitting diodes) and an LCD (liquid crystal display), thus being utilized for various input operations by the user and for displaying operation statuses of the image combining apparatus 100.
The memory card control portion 17 controls an access to a memory card 18 serving as a data memory medium, and reads and writes the image data and the like stored in the memory card 18. Thus the photo image data, stored in the memory card 18, are read,
then stored in the image memory 13, further converted into print data and printed.
The interface control part 19 controls a communication process by the interface 20, thereby executing data exchanges with an external apparatus 21.
As the external apparatus 21, a personal computer may be connected for receiving print data,

10021170
prepared by a printer driver functioning on the computer, and printing such print data. Also a digital camera or the like may be connected for reading photo image data, for storage in the image memory 13 and conversion into print data for printing The communication control portion 22, including a MODEM (modulation-demodulation unit) and an NCU (network control unit), is connected to an analog communication line 23 and executes a communication control and a line control such as a call making or a call reception.
Now reference is made to Fig. 9, showing details of the operation panel 16 shown in Fig. 1. The operation panel 16 is equipped with an LCD display 201, a power key 202, a copy mode key 203, a film mode key 204, a scan mode key 205, a photo mode key 206, a menu key 207, a print sheet key 208, a scan sheet key 209, a minus key 210, a plus key 211, a set key 212, a slide show key 213, a sheet setting key 214, a feeder selection key 215, a stop key 216, a monochromatic start key 217, and a color start key 218.
The LCD display 201 is used for displaying various information such as a message and an operation prompt.
The power key 202 is used for turning on/off the power supply of the image processing apparatus

10021170
100.
The copy mode key 203 is used for setting the image processing apparatus 100 in a state capable of a copying operation, and the image processing apparatus 100 is shifted to a copy mode when the copy mode key 203 is depressed.
The film mode key 204 is used for setting the image processing apparatus 100 in a state capable of a printing operation from a film, and the image processing apparatus 100 is shifted to a film mode when the film mode key 204 is depressed.
The scan mode key 205 is used for setting the image processing apparatus 100 in a state capable of a scanning operation, and the image processing apparatus 100 is shifted to a scan mode when the scan mode key 205 is depressed.
The photo mode key 206 is used for setting the image processing apparatus 100 in a state capable of a direct printing operation by a signal from a digital photo card or a digital camera, and the image processing apparatus 100 is shifted to a photo mode when the photo mode key 206 is depressed.
The menu key 207 is used for displaying set items for example in executing a copying operation, a facsimile operation or a direct printing from a card. A depression of the menu key 207 causes a display of the set items for execution on the LCD 2 01, and such

10021170
set items may be selected by the plus key 211 and the minus key 210 and set by the set key 212.
The print sheet key 208 is’used for printing an original sheet SHI as shown in Fig. 3.
The scan sheet key 209 is used for reading the sheet SHI. In the image processing apparatus 100, information written on the sheet/ printed by the print sheet key 208, is read by the read portion 107 for job execution, and the scan sheet key 209 is used
for instructing such sheet reading.
The minus key 210 and the plus key 211 are used by the user for making a selection from plural candidates in the menu or in a user registration. “Plus” and “minus” respectively indicate a forward
shift and a backward shift. The set key 212 is used for determining a selected item. The slide show key 213 is used for displaying images of the memory card MC in succession on the display of the operation panel 16.
The sheet setting key 214 is depressed for a sheet setting in the copying or other printing operation. The feeder selection key 215 is used, in case a sheet feeding portion 114 has plural sheet feeder portions, for switching such sheet feeder.
The stop key 216 is used for terminating an
operation such as a facsimile transmission-reception, a copying operation or other operations - The

10021170
monochromatic start key 217 is used for starting a monochromatic facsimile transmission or a monochromatic copying. The color start key 218 is used for starting a color facsimile transmission or a color photo printing.
Fig. 2 shows an original sheet SHI in the embodiment
The original sheet SHI is used in case of synthesizing a handwritten image with a photo image serving as a reference image, and includes a setting area 30, a reference image area 40, a handwriting area 50, and a bar code 60.
The setting area 30 is provided, for selecting a size of the print medium, a mark 31 for selecting an A4 size, a mark 32 for selecting a postcard, and a mark 33 for selecting an L-size• The user blacks out one of the mark 31 for selecting the A4 size, the mark 32 for selecting the postcard, and the mark 33 for selecting the L-size, thereby selecting a size of the print medium to be used for printing the combined result.
The reference image area 4 0 prints a reference image 41. The “reference image” corresponds to a photo image selected by the user, and it is not the photo image itself but is printed from such photo image.
The handwriting area 50 is an area where the

10021170
user writes a handwritten image 51 to be combined with the reference image 41.
The reference image area 4 0 and the handwriting area 50 have substantially same sizes and substantially same shape, and the handwriting area 50 is provided close to the reference image area 40.
The read control portion 6, the read mechanism 7, the data memory 5, the print control portion 11, and the print mechanism 12 constitute an example of original sheet print means for printing the original sheet SHI.
As shown in Fig. 2, under the reference image area 4 0 bearing the reference image 41, the handwriting area 50 of a same size as the reference image 41 is provided and the user can draw the
handwritten image 51 by an arbitrary writing utensil in the handwriting area 50.
The bar code 60 is used for specifying an image stored in the memory card 18, and is for example a file name of the image stored in the memory card 18.
In the following, there will be explained a process of the embodiment 1, in which the CPU 1 of the image combining apparatus 100 prints the original sheet SHI.
Fig. 3 is a flow chart showing a process of the embodiment 1, in which the CPU 1 of the image combining apparatus 100 prints the original sheet SHI

10021170
The process shown in Fig. 3 is executed when the user selects “original sheet printing” in the menu displayed on the operation panel 16.
At first, a step SI reads a photo image data file in the memory card 18 and displays it on the display portion 14. A step S2 monitors a key operation on the operation portion 15 and discriminates whether a photo image is selected. In case a photo image is selected, a step S3 reads the selected photo image (reference image 41) from the memory card 18, and combines it with print data, stored in advance in the program memory 4 for printing in the setting area 30, thereby preparing print data for the original sheet SHI.
A step S4 control the print mechanism 12 based on thus prepared print data, thereby printing the original sheet SHI. In the following, there will be explained an operation of the embodiment 1 for reading the
original sheet SHI bearing the handwritten image and printing the reference image 41 and the handwritten image 51 in a combined image.
Fig. 4 is a flow chart showing the functions of the GPU 2 in the embodiment 1, in case of reading the
original sheet SHI bearing the handwritten image and printing the reference image 41 and the handwritten image 51 in a combined image.

10021170
The process shown in Fig. A is executed when the user selects “original sheet reading and combined printing” in the menu displayed on the operation panel 16.
At first a step S11 controls the read mechanism 7 thereby reading the original sheet SH1… A step S12 extracts, from the image data of the read original sheet SHI, data of the setting area 30 and analyzes a marked state of the marks 31, 32 and 33 in the setting area 30, thereby acquiring information on the size of the selected print medium.
A step S13 extracts, from the image data of the read original sheet SHI, an image of the handwriting area 50 as handwritten image data. The extraction of the handwriting area 50 is executed for example by detecting a frame line indicating the handwriting area 50 of the original sheet SHI in the reading operation, and by extracting the data within such detected frame line.
A step S14 analyzes the bar code 60 on the read original sheet SHI, and reads a photo image same as the photo image (reference image 41) selected by the user, from the memory card 18. The bar code 60 indicates, for example, a file name of the image
stored in the memory card 18. The read image is
combined with the handwritten image data, extracted in S13, to obtain combined image data.

10021170
A step SI5 executes, based on the size selection information of print medium obtained in S12, a process of enlarging or reducing the combined image data, and controls the print mechanism 12 to print a combined image on an output print medium (S16).
Fig. 5 shows an example of a combined output result 70 in the embodiment 1.
In the combined output result 70, as shown in Fig. 5, the reference image 41 and the handwritten image 51, inscribed in the handwriting area 50, are combined in a superposed manner, and enlarged or reduced so as to match the print medium size selected in the setting area 30, and thus enlarged or reduced combination data (combination result) are outputted on the print medium 71.
In the embodiment 1, the user can observe, in drawing the handwritten image 51 in the handwriting area 50 of the original sheet SHI, the reference image 41 provided in an adjacent position in a same size, and can therefore draw the handwritten image 51 while recognizing the positional relationship of the reference image 51 and the handwritten image 51 when both are combined in superposition -
In the embodiment 1, the reference image 41 and the handwriting area 50 are arrayed along a vertical direction on the original sheet SHI, but they may be arrayed in a lateral direction. In the embodiment 1,

10021170
the reference image 41 and the handwriting area 50 are provided in a same and in a mutually adjacent relation, but the reference image 41 and the handwriting area 50 need not necessarily be in a same size, as long as the positional relationship does not become very unclear. Embodiment 2
Fig 6 shows a configuration of an original sheet SH2 in an embodiment 2 of the present invention.
The original sheet SH2 includes a setting area
30a, a reference image area 40, a handwriting area 50, and a bar code 60.
The setting area 30a is provided, for selecting a size of the print medium, a mark 31 for selecting
an A4 size, a mark 32. for selecting a postcard, and a mark 33 for selecting an L-size, and for selecting a print layout, a mark 34 for selecting an entire area and a mark 35 for selecting a half area.
The user blacks out one of the mark 31 for
selecting the A4 size, the mark 32 for selecting the postcard, and the mark 33 for selecting the L-size, thereby selecting a size of the print medium to be used for printing the combined result. Also, when the user blacks out the mark 34 for selecting the
entire area (selecting an entire area layout), an image formed by superposing the reference image 41 and the handwritten image 51 is printed on the entire

10021170
area of the print medium 71, while, when the user blacks out the mark 35 for selecting the half area (selecting a half-area layout), the reference image 41 is printed in a half area of the print medium 71 and the handwritten image 51 is printed on a remaining half area of the print medium 71.
Fig. 7 shows an example of a combined output result when the user selects the half-area layout in the embodiment 2.
In such case, the reference image 41 and the handwritten image 51, drawn in the handwriting area 50, are combined side by side (that is, a mutually abutting combination of the photo image corresponding to the reference image and the handwritten image),
then the combined image is enlarged or reduced
according to the print medium size selected by the setting area 30a, and printed with the layout according to the designation by the user.
The combined output result, in case the user
selects the entire-area layout, is same as the result 70 in the embodiment 1.
In the embodiment 2, the reference image 41 and the handwriting area 50 are positioned on the original sheet SH2 as shown in Fig. 6, and, when the
entire-area layout is selected, the reference image 41 and the handwritten image 51 drawn in the handwriting area 50 are combined in a superposed

10021170
manner, while, when the half-area layout is selected, the reference image 41 and the handwritten image 51 drawn in the handwriting area 50 are arrayed in the same positional relationship. There the user, in drawing the handwritten image 51 in the handwriting area 50 of the original sheet SH2, can easily recognize the result of combination with the reference image 41.
Also the embodiment 2 allows to achieve the entire-area layout and the half-area layout by a single original sheet SH2.
The embodiment 1 shows a case where the original photo image (reference image 41) is a laterally oblong so-called landscape image, but, in a case where the original photo image (reference image 41) is a vertically oblong so-called portrait image, the reference image 41 is printed in a laterally oblong position so that the user can conveniently understand the direction of the image. Embodiment 3
Fig. 8 shows a configuration of an original sheet SH3 in an embodiment 3 of the present invention
The original sheet SH3 includes a setting area 30a, a reference image area 4 0a, a handwriting area 50a, and a bar code 60.
The setting area 30a is same as that shown in Fig. 6.

10021170
Then reference image area 40a prints a reference image 41, which is provided with a frame line 42, scale lines 4 3 and grid lines 4 4.
The frame line 42 surrounds the reference image area 4 0a. The scale lines 4 3 are provided to divide the vertical and horizontal lines of the frame line 42, and the grid lines 44 are provided to form a grid including the scale lines 43.
The handwriting area 50a is used by the user for drawing a handwritten image 51 and is provided with a frame line 52 and scale lines 53.
The frame line 52 surrounds the handwriting area 50a. The scale lines 53 are provided to divide the vertical and horizontal lines of the frame line 52.
The frame lines 42 and 52 have substantially same shapes, and the scale lines 43, 53 are provided in mutually approximately same positions of the frame lines.
The print mechanism 12 and the print control portion 11 constitute an example of the original sheet printing means, which serves to print the reference image frame line 42 on the reference image 41, also to print the handwriting area frame line 52 on the handwriting area 50, to print also the scale
lines dividing the reference image frame line 43, and also to print the scale lines 53 in the positions,

10021170
on the handwriting area frame line 52, substantially corresponding to those of the scale lines 43 on the reference image frame line 42.
The original sheet printing means also prints, on the reference image 41, the grid lines including the scale lines 43 on the reference image frame line 42.
In the embodiment 3, the frame lines, the scale lines and the grid lines are attached to the handwriting area 50 and the reference image 41 on the original sheet SH3, thereby further facilitating to understand the positional relationship at the image combining operation.
The embodiment 3 shows a case of adding all the frame lines 42, the scale lines 43 and the grid lines 44, but it is also possible to add the frame lines 42 and the scale lines 43 only. Embodiment 4
In an embodiment 4 of the invention, in the printing of the original sheet in any one of the
embodiments 1 to 3, the reference image 41 is printed with a lowered print density.
As the final output of the combined image utilizes the image read out from the memory card 18, while the reference image 41 printed on the original sheet is used only temporarily as a reference for the drawing of the handwritten image 51 by the user, so

10021170
that the density of the reference image 41 is not reflected on the final output. Therefore, the reference image 41 may be printed with a lowered print density as long as it allows to confirm the mutual positional relationship of the handwritten image 51 and the reference image 41 at the image combining, and such lowered print density is effective in reducing an ink consumption in case an ink jet printing apparatus is employed as the print mechanism 12.
In outputting a combined result, the reference image to be printed in the reference area may be printed, instead of lowering the print density, with a smaller number of colors, or as a monochromatic
image. Stated differently, upon outputting the
combined result, printing may be made with an image quality different from that in the normal printing operation. Such “image quality” means parameters controlling the quality of an image, such as an ink
density, a number of colors, a monochromatic or color, and an image resolution. Embodiment 5
In an embodiment 5 of the present invention, in any of the embodiments 1 to 4, the photo image data
are entered, instead of from the memory card 18, from the reading means, same as that used for reading the original.

10021170

The photo image data may also be obtained, instead of from the reading means used for reading the original, from an external equipment through the communication line 23 or through the interface 20.
It is also possible, in any one of the memory medium such as the memory card 18, or to embodiments 1 to 4, to confirm the combined image result by displaying on a display portion of the operation panel 16, or to store the combined result in a transfer the combined result to another equipment through the communication line 23 or through the interface In this manner, the image after combining operation may be edited in such another equipment.
Figs. 10-1 and 10-2 show an original sheet SH4 to be inscribed by the user in the embodiment 6.
More specifically, Fig. 10-1 shows an original sheet SH4 used in the embodiment 6, and Fig. 10-2 shows an example of the combined output 90 in the embodiment 6.
The original sheet SH4 includes a setting area 30, a reference image area 40, a handwriting area 50 and a bar code 60.
The original sheet SH4 is printed utilizing the operation panel 16 of the image processing apparatus 100, and a photo image, stored in the memory card MC connected to a USB host control portion 117, and a

10021170
content inscribed in the original sheet SH4 are subjected to a combining process.
More specifically, at first a setting for printing the original sheet SH4 is executed in the image processing apparatus 100 through the operation panel 16, and the recording portion 113 executes a printing operation according to such setting. In such printing, the handwriting area 50 for inscribing a content for image synthesis is printed on the
original sheet SH4, then the reading portion 107
reads such handwriting area 50 of the original sheet SH4 for synthesis with a photo image PI corresponding to the reference image 41, and the result of such synthesis is printed by the recording portion 113 to
achieve an image combining process.
In the embodiment 6, the print sheet key 208 is depressed for printing the recording sheet SH4 . In the following, details of the printing operation of the original sheet SH4 will be explained with reference to Figs. 11-1 and 11-2, and details of the operations for reading the original sheet SH4 and executing the image combining will be explained with reference to Fig. 12.
The setting area 30 is used for designating a size of a sheet for printing the image combined by using the original sheet SH4. In the embodiment 6, such size designation is executed by a marking sheet,

10021170
with printed marks corresponding to different sheet sizes. At the image combining operation, the reading portion 107 reads the setting area 30 while reading the original sheet SH4, and the output size of the combined image is determined according to the marked size in the setting area 30.
The reference image area 4 0 prints, as a sample, an image same as the photo image PI, and, in the image combining operation utilizing the original sheet SH4, data obtained by reading the image drawn in the handwriting area 50 and a photo image corresponding to the reference image 41 printed in the reference image area 4 0 are combined. The reference image 41 indicates an object image for combination, to be used as a reference when the user inscribes in the handwriting area 50, but the data for actual combining operation are not extracted from the original sheet SH4 but are obtained from the photo image stored in the digital camera DC.
The handwriting area 50 is used for inscribing a content, to be combined with a photo image PI, corresponding to the reference image 41 printed in the reference image area 40. An inscription is made in the handwriting area 50, and, when the reading portion 107 reads the original sheet SH4 at the image combining operation, a content inscribed in the handwriting area 50 is taken as a content for image

10021170
combination. In the embodiment 6, at the image combining operation, a white background is disregarded, and a handwritten image 51 drawn by the user and a printed combining image 52 are regarded as a content of image combination and are used for the image combining operation. The handwriting area 50 and the reference image area 4 0 have mutually same sizes, so that the user can judge an actual size at the.inscription in the handwriting area 50.
The combination print image 52 is an image of a font/illustration selected by the user at the printing of the original sheet SH4. The data for such image are present in the image processing apparatus 100 for printing the original sheet SH4,
and, at the printing of the original sheet SH4, the frame of the handwriting area 50 and the image 52 for combination are printed in such a manner that the image 52 for combination is printed within the frame of the handwriting area 50 as shown in Fig. 10-1.
The “font” above indicates at least one of a character font, a style, a size, a color and a character decoration.
The handwritten image 51 shows an example of the content which the user actually draws on the original sheet SH4. The handwritten image 51 is absent at the printing of the original sheet SH4 and is later inscribed by the user in the handwriting

10021170
area 50 after the original sheet SH4 is printed, and is thereafter combined with the photo image corresponding to the reference image 41.
In the image combining operation utilizing the original sheet SH4, the print image 52 and the handwritten image 51 in the handwriting area 50 are read as a single integral image and are combined with the photo image PI corresponding to the reference image 41 printed in the reference image area 40.
In the bar code 60 recorded on the original sheet SH4, embedded is information of the photo image corresponding to the reference image 41 (for example ID of the photo image). A photo image corresponding to the reference image 41 can be extracted by reading
and analyzing the data of the bar code 60.
The output example 90 of the combined image shows an example of printing after the image combining process. In the image combining utilizing the original sheet SH4, a first object image for combining and a second object image for combining are combined.
The “first object image for combining” is a photo image corresponding to the reference image 41 printed in the reference image area 4 0. Also the “second object image for combining” is an image obtained by integrally reading the print image 52 and the handwritten image 51 described in the handwriting

10021170
area 50 as a single image.
Such two object images for combining are combined and printed to provide an output example 90 of the combined image.
The embodiment 6 allows, by a single combining process, to synthesize three images, namely the combination print image 52, the handwritten image 51 and the photo image PI corresponding to the reference image 41.
Figs. 11-1 and 11-2 illustrate a process of selecting the photo image PI and font/illustration (combination print image 52), in order to print the original sheet SH4.
In printing the recording sheet SH4, the
reference image 41 and the combination print image 52 are printed thereon. For this purpose, the reference image 41 and the combination print image 52 are selected prior to the printing of the original sheet SH4.
Fig. 11-1 shows a photo image selecting display screen 100 in the embodiment 6.
The photo image selecting display screen 100 is an example of the image displayed on the display portion 201 for selecting a photo image. The display portion 201 displays a photo image PI, which is a
candidate to be used in image combining. A desired photo image PI is selected among plural photo images

10021170
by operating minus/plus buttons 210, 211 provided on the operation panel 16, and is determined by the set key 212.
Fig. 11-2 shows a print image selecting display screen 110 in the embodiment 6.
The print image selecting display screen 110 is an example of the display screen displayed on the display portion 201, in order to select the combination print image (font/illustration FI) 52. The font/illustration FI is a candidate of the font/illustration to be printed in the handwriting area 50 as the combination print image 52. A desired font/illustration FI is selected among plural font/illustrations by operating minus/plus buttons 210, 211 provided on the operation panel 16, and is determined by the set key 212.
Fig. 12 is a flow chart showing operation in the embodiment 6 for printing the original sheet SH4.
In the embodiment 6, the operation panel 16 includes the print sheet key 208, which is used for executing the printing operation. Also the LCD 201 displays a photo image PI and a font/illustration FI.
At first a step SI selects a printing of the original sheet SH4. In the embodiment 4, a mode of printing the original sheet SH4 can be selected by operating the print sheet key 208 in the operation panel 16 as shown in Fig. 9.

10021170
Then a step S2 displays, as shown in Fig. 11-1, a photo image selecting display screen 100. More specifically, there is displayed a photo image PI stored in the memory card MC, connected to the LCD 201 of the operation panel 16.
Then, in a step S3, the user operates the minus/plus keys 210, 211 provided in the operation panel 16 to select a photo image PI, and then operates the set key 212 to determine the displayed photo image PI. On the other hand, in case the photo image is not yet selected, the step S2 continues to displays the photo image selecting image display screen 100.
Then a step S4 displays, as shown in Fig. 11-2, a print image selecting display screen 110, namely a font/illustration FI.
A step S5 confirms whether the font/illustration FI has been selected. The selection of the font/illustration FI can be made, as 20 in the step S3, by the minus/plus keys 210, 211 and the set key 212. In the case that the font/illustration FI is selected, the sequence proceeds to a step S6. In case the selection is not made, the step S4 continues to display the printed image selecting display screen 110, thus showing the font/illustration FI.
A step S6 prints the original sheet SH4. In

10021170
this operation, the recording portion 113 prints the original sheet SH4, based on the photo image PI selected in S3 and the font/illustration FI selected in S5, and according to the user description data in a data storage portion 116 of the image processing apparatus 100.
Fig. 13 is a flow chart showing a printing process of the original sheet SH4 in the embodiment 6 At first a step S30 prepares an image, selected
in S2, as an image for the original sheet. In the
embodiment 6, a photo image PI, selected by the user as a reference for the inscription by the user, in the original sheet SH4.
A step S31 confirms a memory capacity, in order
to confirm whether the combining of the photo image PI and the font/illustration FI is possible. Then the sequence proceeds to S35 or S32, respectively according to whether the memory capacity, necessary for the image combining, is available or not. In
case the memory capacity necessary for the image
combining is not available, the font/illustration FI may be written in advance into the handwriting area 50, whereby, at the combining with the photo image PI later, the handwritten image 51 and the
font/illustration FI can be processed as single image data.
Therefore, in the embodiment described above,

10021170
the image combining process require only two memories for developing two sets of data.
A step S32 prepares an image of the font/illustration, selected in S4, for printing on the original sheet SH4 .
A step S33 matches the images, prepared in S30 and S32, with the image of the original sheet SH4 to be actually printed.
A step S34 executes the printing of the original sheet SH4.
A step S35, utilizing the memory for image combining, matches the image prepared in S30 with the image of the original sheet SH4 to be actually printed. A step S34 executes the printing of the
original sheet SH4. In this operation, in the bar code 60 of the original sheet SH4, there is embedded information of the photo image to be combined, and, in case the font/illustration is selected in S4, there is embedded information indicating such selection.
In this manner, the original sheet SH4 can be printed with the photo image selected prior to the printing of the original sheet SH4 as explained in Fig. 12, and the printed image (image of font/illustration).
In the embodiment 6, the handwritten image 51,

10021170
the photo image PI selected by the user and the combination print image 52 are printed in the original sheet SH4.
Fig. 14 is a flow chart showing an operation of the embodiment 6 for reading the printed original sheet SH4 and executing an image combining process.
Thus the operations shown in Fig. 14 are those for reading the printed original sheet SH4 by the reading portion 107 and combining thus read image and the photo image PI corresponding to the reference image 41.
This process is executed, after the printing of the original sheet SH4, for actually combining the photo image, selected in S3 in Fig. 12, and a content of combination desired by the user (image drawn in the handwriting area 50)
At first a step S40 selects a “mode for original sheet reading and photo printing” displayed on the operation panel 16. In the embodiment 6, such 20 “mode for original sheet reading and photo printing” is selected by depressing the scan sheet mode 209.
A step S41 displays, on the display portion (LCD) 201 of the operation panel 16, a message requesting the user to place the original sheet SH4 on an original support of the reading portion 107.
A step S42 confirms that the original sheet SH4 is placed on the original support. Such confirmation

10021170
may be executed by the image processing apparatus 100, or by a request to the user. When the original sheet SH4 is placed on the original support, the sequence proceeds to S43, but, in case the original sheet cannot be confirmed, S41 is executed again to display the message requesting the placement of the original sheet SH4.
A step S44 reads the original sheet SH4 placed on the original support of the reading portion 107.
The data thus read are stored in the image memory 104 and are analyzed. Information of the combination print image 52 is thus obtained, by such analysis, from the original sheet SH4. The detailed process will be explained later with reference to Fig. 15.
A step S45 combines and prints the photo image PI selected in S3, the combination print image 52 and the handwritten image 51, read and analyzed in S44. Fig. 15 is a flow chart showing operations of analyzing the original sheet SH4 in the embodiment 6.
At first, a step S50 reads the information of the bar code 60 of the original sheet SH4.
A step S51 acquires image information from the information of the read bar code 60. The information of the selected photo image is incorporated, in S3,
in the bar code 60. Thus the original sheet SH4 contains information indicating the photo image constituting the object of combining, and it is

10021170
possible to avoid the trouble of selecting the photo image again, by merely reading the original sheet SH4 Also in case S31 in Fig. 13 identifies that the memory required for the image combining is available, the process of printing the font/illustration image on the original sheet SH4 is not executed.
When the font/illustration FI selected in S4 is not printed in the handwriting area 50 of the original sheet SH4, information indicating the
font/illustration FI of the object of image combining is incorporated in the bar code 60, and the step S51 also extracts such information.
A step S52 detects the position of the handwriting area 50 in the original sheet SH4.
A step S53 reads the combination print image 52 printed in the handwriting area 50 and the content (handwritten image) drawn by the user and extracts the analyzed data. In such extracting operation, there is also extracted a mark position in the
setting area 30, positioned above the handwriting area 50.
Through such extracting operation, the printed data and the drawn data are handled as a single read image for image combining. S53 also extracts the
information on print size, and the data of the combination print image 52, printed in the handwriting area 50 of the original sheet SH4, and

10021170
the data of the handwritten image 51 drawn by the user can be handled as a single set of data, whereby a pseudo image synthesis is thus achieved.
In the embodiment 6, in printing the original sheet SH4, the reference image 41 corresponding to the photo image PI to be combined is printed in advance on the original sheet SH4, and then, in reading the original sheet SH4, the handwritten image 51 drawn by the user and the combination print image 52 are read and discriminated as same combining data and synthesized with the photo image PI. Therefore, at the image combining operation, the combination print image 52 and the handwritten image drawn by the user are processed as a single image, and a combining process of the combination print image 52 and the handwritten image 51 can be dispensed with. Embodiment 7
In an embodiment 7 of the present invention, the reference image 41 used as an index printing is not printed on the original sheet SH4.
Figs. 16-1 and 16-2 show original sheets SH5 of the embodiment 7.
The operation panel 16 shown in Fig. 9 and the photo image selecting display screen and the combining print image selecting display screen shown in Fig. 11-1 are used in the same manner also in the embodiment 7.

10021170
The original sheet SH5 includes a setting area 30, a handwriting area 50 and a bar code 60.
In the embodiment 7, as the reference image 41, printed in the embodiment 6, is not printed on the original sheet SH5, it is possible to reduce the ink consumption in printing the original sheet SH5.
The original sheet SH5, not containing the reference image as an index, is printed through the operation panel.16 of the image processing apparatus 100.
In the image processing apparatus 100, the operation panel 16 executes a setting for printing the original sheet SH5, and the recording portion 113 executes the printing of the original sheet SH5. The thus printed original sheet SH5 contains the
handwriting area 50 for drawing a handwritten image to be combined, then the handwriting area 50 of the original sheet SH5 is read by the reading portion 107 and is combined with the photo image PI corresponding to the reference image 41, and a result of such
combining is printed by the recording portion 113 to achieve an image combining.
In the embodiment 7, the print sheet key 208 is used for printing the original sheet SH5. The details of the printing of the original sheet SH5 are shown in Fig. 12, and those of the operation for reading the original sheet SH5 for image combining

10021170
are shown in Fig. 13.
The setting area 30 is used for designating the size of a sheet for printing the combined image.
In the embodiment 7, the setting area 30 is formed as a marking sheet, with printed marks
corresponding to different sheet sizes. When the reading portion 107 reads the original sheet SH5 for image combining, it also reads the setting area 30 and determines the output size of the combined image
according to the marked size.
The handwriting area 50 is used for inscribing a content to be combined. An inscription is made in the handwriting area 50, and, when the reading portion 107 reads the original sheet SH5 at the image
combining operation, a content inscribed in the
handwriting area 50 is taken as a content for image combination. In the embodiment 7, at the image combining operation, a white background is disregarded, and a handwritten image 51 drawn by the
user for example by a pen and a printed combining image 52 are regarded as a content of image combination and are used for the image combining operation.
The combination print image 52 is an image of a
font/illustration selected by the user at the
printing of the original sheet SH5. The data for such image are present in the image processing

10021170
apparatus 100 for printing the original sheet SH5, and, at the printing of the original sheet SH5, are printed in the handwriting area 50 of the original sheet SH5.
The handwritten image 51 is an example of the content inscribed by the user on the original sheet SH5. The handwritten image 51 is not yet recorded when the original sheet SH5 is printed.
In the image combining process utilizing the original sheet SH5, the handwriting area 50 is read, and the combination print image 52 and the handwritten image 51 are read as a single object for image combining.
In the bar code 60, embedded is information of the photo image PI selected for image combining and the combination print image 52. Such data are read and analyzed to extract the image information.
The embodiment 7 utilizes a single combining operation, which combines three objects, namely the combination print image 52, the handwritten image 51 and the photo image PI.
In the following, there will be explained the operations of printing the original sheet SH5.
Fig. 17 is a flow chart showing a printing process of the original sheet SH5 in the embodiment 7.
At first a step S60 prepares an image of the font/illustration, selected in S4, for printing on

10021170
the original sheet SH5.
A step S61 matches the font/illustration FI, prepared in S60, with the combination print image 52 of the original sheet SH5 to be actually printed. A step S62 executes the printing of the original sheet SH5.
In this manner, the combination print image 52 is printed on the original sheet SH5, then the user draws the handwritten image 51 on the original sheet SH5, whereby the images 51, 52 are coupled (inscribed in a same area) but are not subjected to an image combining.
In reading the original sheet SH5, the combination print image 52 and the handwritten image 51 drawn by the user are read and combined, as a
single object for combining, with the photo image PI.
In the embodiment 7, the combination print image 52 and the handwritten image 51 are used as a single object of image combining, without an image synthesis, and the number of combining operations can be reduced in comparison with a case where the combination print image 52 and the handwritten image 51 are combined and the result of such combining is combined with the photo image PI. Embodiment 8
In an embodiment 8 of the present invention, a font/illustration image CI is printed also on the

10021170
reference image 41 on the original sheet SH4.
The operation panel 16 shown in Fig. 9 and the photo image selecting display screen and the combination print image selecting display screen shown in Fig. 11-1 are used in the same manner also in the embodiment 8.
Figs. 18-1 and 18-2 show original sheets SH6 of the embodiment 8, in which a combining image 52, corresponding to the font/illustration FI, is printed in the reference image 41.
In the embodiment 8, the original sheet SH6 is utilized to combine the photo image, stored in the memory card MC connected to the USB host control portion 117, and a content inscribed in the original sheet SH6.
More specifically, at first a setting for printing the original sheet SH6 is executed in the operation panel 16, and the recording portion 113 prints the original sheet SH6. The printed original sheet SH6 contains the handwriting area 50 for
inscribing a content to be combined, then the reading portion 107 reads such handwriting area 50 for synthesis with a photo image PI corresponding to the reference image 41, and the result of such synthesis is printed by the recording portion 113 to obtain a combined print example 73 as shown in Fig. 18-2.
In the embodiment 8, the print sheet key 208 is

10021170
used for printing the recording sheet SH6.
Details of the printing operation of the original sheet SH6 are same as those shown in Fig. and those of the operations for reading the original sheet SH6 and executing the image combining are same as those shown in Fig. 13.
The setting area 30 is used for designating the size of printing of the combined image, and is formed as a marking sheet, with printed marks corresponding
to different sizes. When the reading portion 107
reads the original sheet SH6 for image combining, it also reads the setting area 30 and determines the output size of the combined image according to the marked size.
The handwriting area 50 is used for inscribing a content to be combined (combination print image 52 and handwritten image 51). An inscription is made in the handwriting area 50, and, when the reading portion 107 reads the original sheet SH6 at the image
combining operation, a content inscribed in the
handwriting area 50 is taken as a content for image combination.
At the image combining operation, a white background is disregarded, and a handwritten image 51
drawn by the user for example by a pen and a
combination print image 52 are regarded as a content of image combination and are used for the image

10021170
combining operation.
The reference image 41 is an image
corresponding to the photo image PI. In printing the original sheet SH6, a combination print image 52, to be printed in the handwriting area 50, and a photo image PI, corresponding to the reference image 41 to be printed in the reference image area 40, are combined. The reference image 41 is a sample indicating the photo image PI constituting the object
of combining, for the drawing of the handwritten
image 51 by the user in the handwriting area 50, and, in the actual combining operation, the data of the photo image PI are not extracted from the original sheet SH6 but from the bar code 60.
The handwriting area 50 and the reference image area 40 have mutually same sizes, so that the user can judge an actual size at the inscription in the handwriting area 50.
The combination print image 52 is an image of a
font/illustration selected by the user at the
printing of the original sheet SH6. The data for such image are present in the image processing apparatus 100 for printing the original sheet SH6, and, at the printing of the original sheet SH6, are
printed in such a manner that the image 52 for
combination is printed within the handwriting area 50 The handwritten image 51 is an example of the

10021170
content inscribed by the user on the original sheet SH6. The handwritten image 51 is not yet recorded when the original sheet SH6 is printed.
In the image combining process utilizing the original sheet SH6, the handwriting area 50 is read, and the combination print image 52 and the handwritten image 51 are read as a single object for image combining, which is then synthesized with the photo image PI, corresponding to the reference image
41 printed in the reference image area 4 0.
The bar code 60 is information embedded in the original sheet SH6. In the embodiment 8, embedded is the information of the photo image PI, selected as the object of combining. Such data are read and
analyzed to extract the image information.
The font/illustration FI is combined with the photo image PI, and the combination print image 52 and the reference image 41 are printed. As the reference image area 4 0 prints the reference image 41
and the combination print image 52, the user can know in advance, when inscribing the handwritten image 51 in the handwriting area 50, a state after the combination print image 52 and the reference image 41 are combined.
The example of combined printing 73 shows an example of printing. In a combining operation utilizing the original sheet SH6, the handwritten

10021170
image 51 and the combination print image 52 in the handwriting area 50 are used as an object of image combining, and are subjected to a combining process with original data of the image printed in the reference image area 4 0, whereby a combined image 7 3 can be printed in the reference image area 4 0 as shown in Fig. 18-2.
In the following, there will be explained operations of printing the original sheet SH6 in the embodiment 8.
Fig. 19 is a flow chart showing a printing process of the original sheet SH6 in the embodiment 8
At first a step S70 prepares a photo image PI, selected in S2, as a reference image 41 for the original sheet SH6. In the embodiment 8, a reference image 41 corresponding to the photo image PI, selected by the user, is printed in the original sheet SH6;
A step S71 prepares an image of the font/illustration FI, selected in S4, as a
combination print image 52 for the original sheet SH6
a step S72 combines the font/illustration FI prepared in S71, with the photo image PI prepared in S70. Such combined image (reference image 41 and combination print image 52) is used as a reference index of the original sheet SH6.
A step S73 matches the images, prepared in S72,

10021170
with the original sheet SH6 to be actually printed.
A step S7 4 executes the printing of the original sheet SH6.
In this manner, the original sheet SH6 can be printed with the reference image 41 corresponding to the photo image PI selected prior to the printing of the original sheet SH6 as explained in Fig, 12, and the combination print image 52 corresponding to the font/illustration FI.
In the embodiment described above, at the
printing of the original sheet SH6 for inputting a handwritten information of the user for image combining operation, an image to be combined is printed in advance on the original sheet SH6, and, in
reading the original sheet SH6, the handwritten image
51 drawn by the user and the combination print image
52 are read as a single object of combining, and such read image is combined with the photo image PI. Therefore, in such embodiment, a process of synthesizing the printed combining image 52 and the handwritten image 51 is not required in the image combining operation, thereby allowing to reduce the number of image combining operations.
The embodiments 6, 7 and 8 are different in
whether the user prints the font/illustration FI as a reference index in the reference image area 40 of the original sheet, but all these embodiments are

10021170
effective regardless whether such reference index is printed or not.
The foregoing embodiments can also be understood as inventions of a program. Stated differently, the foregoing embodiments constitute examples of a program which causes an image processing apparatus to execute a photo image selecting step of selecting a desired photo image from plural photo images stored in photo image
storage means, a font/illustration selecting step of selecting a desired font/illustration from plural font/illustrations, and an original sheet preparing step of printing a reference image, corresponding to the selected photo image, in a reference image area,
also printing an image for combining corresponding to the selected font/illustration in a handwriting area, in such a manner that the reference image area and the handwriting area are mutually adjacent, thereby preparing an original sheet.
Also the foregoing embodiments constitute examples of a program which causes an image processing apparatus to execute a photo image selecting step of selecting a desired photo image from plural photo images stored in photo image
storage means, a font/illustration selecting step of selecting a desired font/illustration from plural font/illustrations, an original sheet preparing step

10021170
of printing a combining image, corresponding to the selected font/illustration in a handwriting area, a read step of reading the handwritten image inscribed in the handwriting area and the combination print image, and a combining step of combining the
combination print image and the handwritten image, which are read in the reading step, and a photo image corresponding to the reference image.
The present invention provides an effect
allowing to draw a handwritten original, at the time of such drawing on a sheet, while confirming the positional relationship how the handwritten original• is combined with the photo image as the object of image combining.
The present invention also provides an effect of not requiring a large burden on resources necessary for image combining nor a long process time, even with a large amount of data to be combined such as photo image data, data written by a user, a photo
2 0 frame and the like.

Documents:

1053-che-2006 abstract duplicate.pdf

1053-che-2006 claims duplicate.pdf

1053-CHE-2006 CORRESPONDENCE OTHERS.pdf

1053-CHE-2006 CORRESPONDENCE PO.pdf

1053-che-2006 description (complete) duplicate.pdf

1053-che-2006 drawings duplicate.pdf

1053-CHE-2006 FORM 1.pdf

1053-che-2006-abstract.pdf

1053-che-2006-claims.pdf

1053-che-2006-correspondence-others.pdf

1053-che-2006-description-complete.pdf

1053-che-2006-drawings.pdf

1053-che-2006-form 1.pdf

1053-che-2006-form 18.pdf

1053-che-2006-form 26.pdf

1053-che-2006-form 3.pdf

1053-che-2006-form 5.pdf

1053-che-2006-priority.pdf

« Previous Patent

Next Patent »

Patent Number

231272

Indian Patent Application Number

1053/CHE/2006

PG Journal Number

13/2009

Publication Date

27-Mar-2009

Grant Date

04-Mar-2009

Date of Filing

20-Jun-2006

Name of Patentee

CANON KABUSHIKI KAISHA

Applicant Address

3-30-2, Shimomaruko, Ohta-Ku, Tokyo 146-8501,

Inventors:

#	Inventor's Name	Inventor's Address
1	MAEDA, Masao	c/0 Canon Kabushiki Kaisha, 3-30-2, Shimomaruko, Ohta-Ku, Tokyo 146-8501,
2	FUJINAGA,Seiya	c/o Canon Kabushiki Kaisha, 3-30-2, Shimomaruko, Ohta-Ku, Tokyo ,

PCT International Classification Number

H04L

PCT International Application Number

N/A

PCT International Filing date

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1	2005-179627	2005-06-20	Japan
2	2005-185261	2005-06-24	Japan