Title of Invention	METHOD AND DEVICE FOR DISPLAYING INFORMATION WITH RESPECT TO SELECTED IMAGE ELEMENTS OF IMAGES OF A VIDEO SEQUENCE
Abstract	The invention relates to a method for displaying information with respect to selected image elements. The aim of the invention is to simplify the interactive work via, for example, a frame. To this end, an image element is selected from the images of a video sequence (1) with which an information is associated. The selection proceeds partially via an additional image element or signal element that is likewise presented on the display unit (2) that displays the frame.

Title of Invention

METHOD AND DEVICE FOR DISPLAYING INFORMATION WITH RESPECT TO SELECTED IMAGE ELEMENTS OF IMAGES OF A VIDEO SEQUENCE

Abstract

The invention relates to a method for displaying information with respect to selected image elements. The aim of the invention is to simplify the interactive work via, for example, a frame. To this end, an image element is selected from the images of a video sequence (1) with which an information is associated. The selection proceeds partially via an additional image element or signal element that is likewise presented on the display unit (2) that displays the frame.

Full Text	Description Method and device for displaying information with respect to selected image elements of images of a video sequence The present Invention relates to a method for displaying information with respect to selected image elements of images or a video sequence, a recording medium with a video sequence, with information data and criteria data for performance of the method, a device for preparation of video sequences in accordance with the generic farm of claim 29 and a recording medium and a display unit for performance of the method. it is well known that certain image elements' of static single images. In particular of Internet pages in HTML script and of window-oriented operating systems can be linked with each other, These information may for example be operating supports or links to other Internet homepages. Therefore, In the HTML-programming language, the coordinates of rectangular image fields are defined and linked with information or a link to further homepages. When the image element appears on the screen and the user navigates the cursor onto the defined rectangular image field and selects it, the stored Information is shown or the Internet site, i e, the link is effected. When transmitting video sequences on TV, such as sport events, talk shows or seriate, the spectator is often exposed to advertising messages, which pass by in the background. Therefore, for example certain objects are provided with advertising banners. Furthermore, the spectators attention is drawn to certain products by the so called product placement, with the products being introduced repeatedly and clearly visible in the telecast. Thereby, the spectator is massively confronted with said product. The problem of the traditional advertising methods is the insufficient control of the advertising success. Furthermore, advertising is quite obtrusive and is poorly accepted by the spectator. Thus, It is an object of the present Invention to provide a method for displaying information with respect to selected image elements of a video sequence or a device for preparation of video sequences, in order to simplify and improve interactive work via the image contents of a video sequence. 1 Further it is an object of the invention to develop the method for displaying information with respect to selected image elements of a video sequence or a device for preparation of video sequences, in a way that improved advertising with telecasts is possible and the spectators are interactively confronted with advertising elements. The above-mentioned objects are, with respect to the method, solved in accordance with the features of claim 1, and, with respect to the device, solved in accordance with the features of claim 29. Since, in accordance with the invention, the generally known method of static images, to link defined image fields with information and display them in a separate window on the display unit as soon as the image element has been selected with a cursor, has been transferred onto video sequences with a serial of slides, generally new possibilities, in particular with regard to advertising, could have been created. Therefore, not only the traditional video sequence of the display unit, e.g. a television receiver, is transmitted. Additionally, defined information of selectable image elements are transmitted, which can be displayed after selection of the corresponding image element in a window. Thus it will be achieved, that the traditionally available video sequences can remain unmodified up to the ultimate consumer and, furthermore, the ultimate consumer generally is free to decide to watch a video sequence, i. e. for example a television film, also without adaption, with respect to the method, of the video sequence. Depending on the embodiment of said method, the spectator thus has the possibility to traditionally satisfy his television consume- Due to said method according to the invention, it will thus be guarantied to keep traditional television habits on the one hand, and to simply switch on or switch to the interactive use. Therefore, criteria for linkage of a selected image element are transmitted with the appendant information with the video sequence. Further preferred embodiments of the present invention are subject matter of the sub claims. In accordance with claim 2 it is advantageous, if the criteria for linkage of a selected image element are the image coordinates of the image element in the respective image, with therefore an allocation schedule being created for allocation of information to image coordinates of selected image elements in the images of the video sequence 2 in temporal dependence of the video sequence. The data of the allocation schedule are transmitted as criteria to the display unit However it has to be considered that, when preparing the video sequence, it must be paid attention to the fact that the position of the selectable image elements generally persistently changes. By this, the determination of the linkage of the image elements to the image coordinates becomes more expensive. In general a video sequence can be prepared advantageously by determining a time trace or track, denoting an image field in order to define a selectable image element, the image element being searched in the subsequent Images, e. g supported by well known pattern-recognition methods and the image coordinates or the displacement vector of the Imaga fields of the image element in the images of the video sequence, the respective time information of the time trace and the information appending the image elements being stored. By determining the time trace, a clear allocation of the image coordinates to the respective images is possible. Thereby, attention must be paid to the fact that the traditionally used video sequences of the broadcasting stations are often analogue and, if the video sequences are not respectively tagged, the image coordinates, stored in an independent data base, can later not be allocated to the individual images. The transmittable data for allocation of the information to the image coordinates of selectable image elements can advantageously be reduced by the fact, that the course of the image coordinate of an image element is described as matbematic approximation function, which is transmitted as criteria. Therefore, the image coordinates can be described as function of the body with the coordinates x and y via time, e.g. supported by Spline functions. In this case, only the parameters of the approximation function need to be transmitted. Advantageously, the time information of the time trace are transmitted synchronously to the video sequence and the image coordinates and respective Information asynchronously. However, it is necessary to transmit the criteria of a video sequence regularly, in order to avoid allocation mistakes, which appear by a change of transmitter. The user often "zappes" expeditiously through the particular transmitters, so that a wrong or no allocation at all to the images elements is possible, if the criteria are not updated regulariy. 3 Alternatively to the transmission of image coordinates as criteria for allocation of image elements to information it is also possible, to transmit the selectable image elements with the corresponding information to the display unit. When an image element has been selected, said element is compared to the transmitted and stored image elements, which are selectable and have information allocated. This comparison can for example be effected with known pattern-recognition procedure, which are executed in the display unit. Advantageously, for displaying the information a window is overlain over the video sequence on the display unit it is particularly advantageous if the information consist of interactive elements, so that data can be transmitted from the display unit to a target computer, after an image element has been selected from the images of the video sequence onto the display unit. The data and/or the target computer are hereby dependent on the information, . which correspond to the selected image element. The interactive information can for example be created like an Internet link. Advantageously, input elements can be provided in the information window, in which the user, after superimposing the information window onto the display unit, can enter further data, such as his address and order information for a product corresponding to the image element. This data are transmitted to the target computer and, for example, an order is effected. Thus it is possible to use video sequences as advertising media and to perform an advertising-success control supported by interaction and to provide an order system. Advantageously, the data transfer to the target computer can be effected via a wideband network, an electric network, a telephone network or vial a radio and satellite network or similar. Advantageously, the criteria for linkage of a selected image element with the corresponding information are transmitted In the blanking interval of the signal of the video sequence. When transmitting the video sequence as digital data stream, a defined unit of the data stream is provided for the criteria and the information. For a compressed data stream in accordance with the MPEG-2-standard, data units or areas can be used, which are kept free for additional user-specific functions. 4 For execution of the method, information data of selected image elements of images of the video sequence and data as criteria for linkage of a selected image element with the corresponding information are stored on the traditionally known recording media. If a selected image element is additionally transferred as so-called additional image element of the display unit, i.e. the display unit provides twice of a selected image element, it is advantageously achieved that the user is optically shown, which image element is stored with information and that he can effect the displaying of the information via additional image elements. Thus, the user can follow the video sequence unmodified and, if necessary activate the selectable image elements in the display unit, i. e. bring the additional image element onto the screen with being signalised, which image elements are stored with information. To provide the user optically with a certain structure or orderliness it is advantageous, if additional image elements or one additional image element in form of an icon listing are provided at a predetermined place on the screen. To enable the user to follow his normal television habits, the additional image element is only superimposed when necessary. To generally inform the user that selected image elements exist in the particular video sequence which are stored with information, it is advantageous, if this possibility of superimposition is preferably shown on the margin of the display unit or the screen with a separate hint tag, which may for example be a bar or a particularly for the user configured sign. However, should the user prefer to watch the video sequence with tagged or marked selected image elements it may be advantageously if the user can individually determine, which image elements should be shown in the video sequence as selected images. The user might for example determine, that due to his technical interest, exclusively technical devices, for which an information is stored, are determined as additional image element, so that, due to the tagging in the video sequence he can see, which of the technical devices in the video sequence is stored with an information. It has proved advantageously to show the additional image elements in a certain order, for example in sequence, and that the displaying of the information is effected via a corresponding digit input, which corresponds to the sequence diagram. 5 Should a user generally disfavour to be shown too many image elements on the display unit or on a television screen it is, in accordance with claim 19, advantageously, if only one signal element is provided to each selected image element, which for example can follow the selected image element on the video sequence. With this measure the user is thus signalled that an image element is selectable and it can, due to the placement of the signal element clearly be determined, which image element is stored with information. Due to general habits it has proved advantageously, if the particular place is preferably determined on the right above the selected image element. Attention is drawn to the fact, that each allocation of the signal element to the selected image element is imaginable. If furthermore, in correspondence with claim 21, a signal element is provided with a hint tag or hint marking, e.g. a numerical digit or an integer, the user can for example select the image element by pressing the corresponding digit, in order to demand the display of the information. If, in accordance with claim 22, the hint tag is linked with the signal element via a connection line, it is clearly visible for the user that now the selected image element, which is linked with signal element is stored with information. However, this presentation can only be effected for a certain period, preferably one second, in order to generally avoid any disturbance in the video sequence. Furthermore it will be achieved with this measure to show the user the possibility, that selected image elements with information are provided in the special video sequence. With the possibility to only activate the hint tag when a selected image element is provided in the video sequence, the user has the possibility to easily recognize when he can demand information to a selected image element A further possibility to indicate the user during the video sequence when he can obtain information is for example a pulsing and/or glancing hint tag. A pulsing and/or glancing hint tag has the particular advantage, that each time when further new selected image elements are added to the presentation of a video sequence, the hint tag or hint marking for the already existing selected image element begins to glance or pulse. If, in accordance with claim 25, the method is separated into different modes, different actions are imaginable within the particular modes. A first mode can, in accordance with claim 25. be provided for displaying the signal element, with which it is possible to 6 switch from the normal presentation of the video sequence, i. e. presentation of a film, to the activating mode, if a selected image element is provided in the video sequence. in contrast to the first mode, a second mode may be provided, which will he, used for displaying first information with respect to the selected image elements. Thus, the user can twice get a general survey about the stored Information with respect to a selected image element . If, in accordance with claim 27, in the second mode all selected image elements of a video sequence, which has been available when activating the second mode, are shown, the user will directly and simply get a total overview of the selected image elements stored with information. Should the user show greater interest in a selected image element, more detailed second information of the selected image element can be demanded and shown by a third mode. If, In accordance with claim 29, the activating of the first, the second and/or the third mode or any further existing mode is effected fay limited in time and/or different pressing of action buttons, a maximum success is offered to the user, in particular when he has to press the action buttons, which he usually also uses when presenting the video sequence, i.e. when watching TV. A device for preparation of video sequences for allowing to display information of selected image elements of Images of the video sequence has advantageously one reproduction unit for reproducing and displaying the video sequence, one gathering unit for selecting and gathering of image elements of images of the video sequence, one detecting unit for detecting the selected image elements in sequent images of She video sequence and for determining the respective image coordinates of the image elements and of criteria for the later linkage of image coordinates of selected image elements with information. The display unit preferably provides of a data-processing unit for performance of the method. As data-processing unit for example a set-top box may be used, which preferably allows access to the Internet Hereinafter, the invention will be disclosed more detailed in accordance with the accompanying drawings. 7 It is shown: Fig. 1 - a flow diagram of a first embodiment for the preparation of video sequences for performance of the method; Fig. 2- a schematic block diagram of a system for performance of the method; Fig. 3 shows in a second embodiment an image on the display unit, visible for the user; Fig. 4 - shows in a third embodiment an image on the display unit, visible for the user; Fig. 5 shows in a fourth embodiment an image on the display unit, visible for the user; Fig. 6 - shows a schematic block diagram of a device for performance of the method in accordance with a fifth embodiment; Fig. 7 - schematically shows the transition between the normal operating mode and the first mode in accordance with the fifth embodiment-Fig. 8 - shows the transition between the first mode and the second mode in accordance with the fifth embodiment; Fig. 9 - schematically shows the operational mode of the second mode; Fig. 10 - exemplarily shows the possible presentation of the first information in the second mode, in accordance with the fifth embodiment; Fig. 11 - schematically shows the operational mode of the third mode; Fig. 12- exemplarily shows second, more detailed information in the third mode; Fig. 13 - schematically shows the operational mode of the third mode. 8 Fig. 1 shows a flow diagram for preparation of video sequences for performance of the method according to the invention. First, a rough video sequence of a client is digitaiized, if it is available as analogue video sequence. Then a time trace is written onto the video sequence in order to allow a later allocation and choosing of image elements in dependence of time. After this an empty database is created and information are registered in said database. Therefore, the selectable image elements, which have to be linked with particular information, are framed. The frame or the image element is in this embodiment sequenced by the scenes of the video sequence and the respective image coordinates are stored in dependence of the time trace. It is advantageous, if only the alterations of the image coordinates are stored. This can save storages space. The recording of information in the database and the determination of the corresponding image coordinates is effected consecutively for each information item to register. Thus, a digital or analogue video sequence with a time trace and a linked database with information and image coordinates, which are stored in dependence with the time trace, has been created. The video sequence is transmitted synchronously with the time trace to a display unit, e.g. a television. Asynchronously to that, the data of the database are for example transmitted in the blanking interval of the television signal. When playing such prepared video sequence on TV, the user can select the image elements with a cursor, preferably mouse controlled. The image coordinates of the selected image element are compared with the transferred image coordinates and the corresponding information are shown, when the image coordinates match with those of an image element, for which information have been stored. Therefore, one window is opened on the screen and the information of the displayed video sequence is superimposed. St is thus possible to transfer and display background information of video sequences. Furthermore, advertising messages and order facilities can be demanded interactively by the user. The method in accordance with the first embodiment is explained more detailed in the block diagram of a system for performance of the method, designed in Fig. 2. The digital or analogue video sequence 1 with the time trace is transmitted to a display unit 2, which consists of a television 3 and a data-processing unit 4, The time information is transmitted synchronously to the television signal in a hidden transmission area. Additionally, information li are transmitted to selectable image elements and the respective image coordinates Xi, Yi are transmitted asynchronously to the television 9 signal to the. data-processing unit 4 of the display unit and are stored there. Supported by a mouse, an infrared or radio pointer 5, a keyboard 6 or similar, the user can select image elements on the screen of TV 3. The corresponding image coordinates are transmitted to the data-processing unit 4 and compared with the stored-image coordinates Xi Yi of the selectable image elements. If the image coordinates are in the image field of selectable image elements, the finked information II is shown in a separate window on the screen. If this Information li is connected with a link and an interaction has been started, a linkage to a target computer 7 is built in a central and data corresponding to information li, e.g. for advertising control, for order execution, etc. are transmitted to the central computer. Therefore, advantageously the internet,is used. Hereby, the data-processing unit 4 can be a so-called set-top boxionsimilar. in Fig. 3 a total image 13a, which is visible to the user, e.g. of the display unit 3 of Fig, 2 is shown, as it may appear in accordance with a second embodiment The total image 13a shows a video sequence with a plurality of image elements, within this example appearing five selected image elements I, II, III, IV and V, At a suitable position on the display image 13a an icon listing is provided, in which the selected image elements I, III, HI, IV and V are illustrated as additional image elements XI, XII, XIII, IX and XV, Due to this allocation, the user is signalled via the additional image elements XI, XII, XIII, IX and XV, which of the image elements I, II, III, IV and V appearing in the video sequence are selectable to display the stored information, it is drawn attention to the fact, that the icon fisting can also be superimposed or overlain or faded-in and that In particular this possibility of superimposing can be superimposed for example by a special movement of a mouse or hand-control element. This presentation also shows, that for example the user, should he wish to get the information to the additional image element XIII, due to position 3 in the icon list, could demand the stored information for example by pressing no. 3, e.g. on the teleguidance, which is then displayed for example as superimposed window also on the display unit (not shown). The principle basing on this second embodiment is, that during the audio- video transmission an icon listing 13b is superimposed with the additional image elements XI, XII, XIII, IX and XV as the selectable image elements. This icon listing depends on the support of an input device, (e.g. mouse, cursor button, control pad, teleguidance elements, radio pointer, etc.) the additional image element being selected and depending on the selection of an action, e.g. order, ordering information, etc, i.e. display the respective information. For the realization of this concept only the additional image element needs to be transmitted, and the time, when said additional image element, i.e. by means of the corresponding time trace of the video sequence is visible. A transmission of for example the image 10 coordinates of selected image elements in the images of the video sequence, which correspond to the image elements of the image transmission is thus not necessary, und no so-called hot spots need to be transmitted. In Fig. 4 a third embodiment of the present invention is shown, which essentially uses the same presentation mechanics for the selected image elements as the second embodiment. A display image 23a is shown, which also shows selected image elements I, II, III, IV and V and an icon listing 23b with the additional image elements XI, XII, XIII, IX and XV. Opposite to the second embodiment, now the selected image elements I to V are provided with a tag or marking in form of a selection frame, to allow the user to discern, in addition to the icon listing 23b in the display image/which image element is activated or stored with information. Furthermore it is possible to provide the currently selectable selected image element e.g. I with one opposed to the other selected image element with another selection frame, e.g. thicker frame contour or other colours, with the corresponding additional image element, here XI, also being optically highlighted in contrary to the other additional image elements II to V. Due to this highlighting it is granted, that the user can only demand the stored information of the selected or additional image element with a generally defined user-button, e.g. the "OK" button of the teleguidance. However, for this method additionally the frames for the selected image elements and its coordinates need to be transmitted, but no so-called Hot-Spots are transmitted or any actions depend on those Hot-Spots. If the additional image elements are reproduced in correspondence with the icon listing shown in Fig. 4, for example the user can only select with a corresponding "right-left" button in the icon listing the interesting additional image element and demand it respectively with the general input button. The activating of the selected image element does thus not effect via so-called Hot-Spots but only via the additional image elements XI to XV. In Fig. 5 a fourth embodiment of the present invention is shown. In this embodiment another display image of display unit 3 is shown, in which again selected image elements I to V are visible. In this embodiment a signal element 35 is provided for each selected image element 1 to IV, with each signal element 35 being provided with a hint tag or hint marking, e.g. a digit or letters which can directly be selected, with pressing of the respective digit of the letter, the stored information of the selected image element being shown. For this method only the coordinates of the signal elements need to be transmitted, which for example may depend on the image coordinates of the selected image elements. The displaying of the information, which is allocated to the selected 11 image element, thus depends, on the activating of a separate input device and, again, no so-called Hot-Spot is used. It is pointed out that the optical preparation can be contrasted similarly as in the third embodiment by highlighting, i.e. possibly by different border margins or colorations. Furthermore it shall be pointed out that the respective method of first to fourth embodiment can also concurrently be used and combined, so that a manifold presentation possibility is given to the user. Subsequent to the third embodiment, in which the selected image elements are provided with a selection frame, it shall be pointed out that the so-called transparent GIF-image formats can bo transmitted separately to the display unit, which can be exactly as large as the display unit 13a to 33b and have the selection frame for the selected image element at an appropriate position or predetermined coordinates. It is thus possible to save the transmission of image coordinates, so that only over the time trace corresponding to the image format and respective selected image elements the display of information is allowed. Thus, said method and device allow a product placement of telecasts with active advertising-success control. Furthermore, information for an image element can be connected with a link, so that after selection of the image element an order of the product presented in the image of the video sequence is possible online via data transfer. If the user favours a product which is shown in the video sequence, such as a garment, he can select said product with the mouse and get the information of said product, such as name and address of the producer, price, quality etc. Now a further information field is selectable, which allows the user to enter order data and order the product directly from the producer. Therefore, the order information are transferred to the provider via a back channel, i.e. via broadband, electric, radio or satellite network or supported by a telephone connection, who arranges the dispatch and the invoicing for the product. Thus, very inconspicuous advertising is possible, which additionally uses the curiosity of the spectator. Furthermore an advertising-success control is also possible for the product placement. 12- Finally, the functionality of the method or device in accordance with the fifth embodiment shall be shown more detailed supported by Fig. 6 - 13. As shown in Fig. 6, the method or device use different modes, with allowing to change from one general operating mode, i.e. TV mode to another mode, which activates the hint tags e.g. in shape of a star, the so-called Starzone mode (see Fig. 7). Depending on the actuating of the action button it is possible to change from the Starzone mode into the first mode, i.e. IPP mode, in which the signal element to a selected image element is activated. From the first mode, i.e. IPP mode, it is now possible to change into a second mode, which for example contains a cart, also by actuating of an action button or other action measures. Furthermore it is also possible to directly change from the first mode into a third mode, which shows more detailed information of the selected image elements. However it is also possible to change from the second mode into the third mode. As clearly shown in Fig. 6 it is also possible to change by a special operating measure from the operating mode, i.e. TV mode and/or the Starzone mode directly in the cart of the second mode. It has proved advantageously, that the order could not have been effected, if from the cart of the second mode firstly is changed into a so-called order box, from which either a confirmation of order or an error hint has been received. Furthermore it is advantageously, that if from the first mode it is changed into a mode, the user is signalled that the information corresponding to the selected image element have already been activated, e.g. that the product corresponding to the selected image element is already registered. Fig. 8 shows the possibilities to change from the so-called Starzone mode into the first and second modes by different actuating of action measures. If the system is in the Starzone mode, it is for example possible to change into the IPP mode, i.e. the first mode, by shortly pressing a certain button, e.g. a button green. If for example the user presses the same button, i.e. button green longer, the system will change into the cart, as shown in Fig. 8. In the IPP mode, i.e. first mode, the selected image elements are stored with information, with a particular signal element pursuing the respective selected image element during the current operating mode or consecutively sequencing video sequences. When pursuing the signal elements with the selected image elements, two different variations can be used. In the first variation, the signal element is stuck to the selected image element and thus skips at a scene change of different video sequences. At a second variation, the signal element is softly drawn at a scene change. This means, that, at a scene change by animation, the digit or integer is newly 13 placed at the selected image element or at the predetermined position corresponding to the image element. In the IPP mode (see Fig. 9) the user has now the possibility to store the selected image element and the respective information in a separate data storage by respective pressing of an activating button, which corresponds to a signal element If the user desires for example detailed information of a selected image element, he can, by pressing an action button corresponding to the signal element of the selected image element, directly change for example into the third mode, which consists of a product box. If the user desires to leave the IPP mode, he can return to an action button corresponding to the signal element into the Starzone mode. If he for example presses the action button longer, the complete system is interrupted. If he presses an action button different to the mentioned action button, he can for example directly change into the second mode, which consists of the cart. Fig. 10 and 11 schematically show the different possibilities with respect to the second mode, which consists of the cart, and the presentation of first information with respect to the selected image elements. In Fig. 10 the selected image elements of the IPP mode are preferably listed tabularly or shown in the so-called cart. Each selected imago element is allocated for example a digit or integer, which can be demanded consecutively. In this connection it is pointed out, that the selected image elements can be shown in dependence of the status, with the following states being differed. Firstly those selected image elements, which have not been stored with further information, e.g. order information such as size, etc, and those selected image elements, which have been stored with further information, such as order information. The latter information elements can for example be highlighted in colour. If the user selects one of the digits allocated to the selected image elements, this selected image element is highlighted in colour. Depending on the actuating of the action button, the product box of the third mode opens or the selected image element is deleted from the cart. Referring to this it is also possible to provide further action measures, which for example open the product box. If the user presses for example a different action button in addition to the former action button, for example the order box opens. It is pointed out to the fact, that the action button remains inactively until the respective information of the selected image element has been entered by the user. If the user shortly presses for example a further action button in addition to the above-mentioned, the system changes back into the IPP mode. However, it is also possible to interrupt the system by further different action measures. As a safety measure it is for example possible to 14 provide a further action button, which needs to be pressed to delete selected image elements. In Fig. 12 the product box, which is allocated to the third mode is shown, with the possible action inputs being shown schematically in Fig. 13. As it can be taken from Fig 12, in the third mode two more detailed information of selected image elements are shown. If the user presses a certain action button, the respective input unit can be highlighted in colour, with the user then being able to enter the respective information. If the user presses a further action button, it is possible to re-skip for example in the second mode, which consists of the cart, and the order information are adopted. If the user presses a different action button additionally to the above-mentioned action button, he can for example also be lead into the second mode, but the information are. not adopted. By pressing of further action buttons, the third mode can be interrupted or the selected image element can be deleted from the cart in accordance with the second mode. Fig. 13 shows the further possibilities, how further information can be entered by different action buttons. Fig. 14 shows the operating system of the order box, with additionally pointing out, that the order box is operated with a security code, in order to avoid involuntary activating of information, i. c. orders. If the information or orders have been effected, the user is signalled by a confirmation of order, that finally the interactivity is finished. Basically it is also possible to support each individual method step with an online support. 15 WE CLAIM : 1. A method for displaying information (li) with respect to selected image elements of images of a video sequence (1), comprising the following steps of: a) receiving the video sequence (1) through a display unit (2), receiving defined information (li) on image elements, and receiving criteria for linking a selected image element to the associated information (li); b) selecting an image element from the images of the video sequence (1) displayed on the display unit (2); c) displaying information (li), provided the criteria revealed that said information (li) is assigned to said selected image element; characterized by d) providing an additional signal element with respect to the selected image element, wherein said additional signal element is assigned to the selected image element and said additional signal element is used to display said information. 2. A method as claimed in claim 1, wherein the image coordinates (Xi, Yi) of the selected image elements serve as criteria for linking a selected image element to the associated information (li), wherein an assignment table for assigning information (li) to image coordinates (Xi, Yi) of selected image elements is created in the images of the video sequence (1) and that the data of the assignment table is transferred as criteria. 3. A method as claimed in claim 2, comprising the following steps : - defining a time track for the video sequence (1); choosing an image area for defining image elements to be selected; -16- searching the image element in the subsequent images; storing the image coordinates or displacement vectors of the image areas of the image element in the images of the video sequence (1), the appropriate time information of the time track and the information (li) for the image element 4. A method as claimed in claim 3, wherein a mathematical approximation function for describing the course of the image coordinates (Xi, Yi) of an image element is determined and transferred as criteria. 5. A method as claimed in claim 3 or 4, wherein the time information of the time track is transferred synchronously with the video sequence (1) and that the image coordinates (Xi, Yi) and the associated information (li) is transferred asynchronously with the video sequence (1). 6. A method as claimed in claim 1, comprising the following steps of: - choosing an image element to be selected; storing the image element chosen and the associated information (li) as criteria; - transferring the video sequence (1), the image elements stored and the associated information (li) to a display unit checking whether a stored image element corresponds to a selected image element and displaying the associated information (li), 7. A method as claimed in any one of claims 3 to 6, wherein the steps of searching or checking the image elements are earned out with methods of pattern recognition. -17- 8. A method as claimed in any one of the preceding claims, wherein a window is superimposed on the video sequence (1) on the display unit (2) for displaying the information (li). 9. A method as claimed in any one of the preceding claims, wherein data is transferred from the display unit (2) to a destination computer (7) after an image element has been selected from the images of the video sequence (1) on the display unit (2), wherein the data and / or the destination computer (7) depend(s) on the information (li) associated with the image element selected. 10. A method as claimed in claim 9, wherein the transfer of the data to the destination computer (7) is achieved via a broad-band network, via a power supply network or via a telephone network or via a radio and / or satellite network. 11. A method as claimed in anyone of the preceding claims, wherein the criteria for linking a selected image element to the associated information (li) are transferred in the blanking interval of the signal of the video sequence (1). 12. A method as claimed in anyone of claims 1 to 10, wherein the criteria for linking a selected image element to the associated information (li) are transferred in a defined range of a video sequence (1) that is provided as a digital video data stream. 13. A method as claimed in anyone of claims 1 to 12, wherein the selected image element is, in addition, transmitted to the display unit and the information is displayed via the additional image element. 14. A method as claimed in claim 13, wherein the additional image element is faded in the form of an icon bar providing the image containing the image element selected. 15. A method as claimed in claim 13 or 14, wherein, if necessary, the additional image element is inserted in the image containing the selected image element. -18- 18. A method as claimed in anyone of claims 1 to 15, wherein the option of the fading in is shown on the display unit by means of a separate reference sign. 17. A method as claimed in anyone of claims 13 to 16, wherein the additional image element is used to define the selected image element that is to be chosen. 18. A method as claimed in anyone of claims 13 to 17, wherein the additional image elements are shown successively one after the other and that the information is displayed by entering an appropriate number of the successively shown additional image elements. 19. A method as claimed in anyone of claims 1 to 18, wherein the signal element is each provided at a specific location in relation to the image element selected on the display unit. 20. A method as claimed in anyone of claims 1 to 19, wherein the signal element is provided to the upper right of the image element selected on the display unit. 21. A method as claimed in claim 20, wherein the signal element is provided with a reference sign, preferably a number. 22. A method as claimed in claims 16 to 21, wherein the reference marking is, via a connection line, connected to the signal element for a specific time interval, preferably for 1 second. 23. A method as claimed in claims 16 to 22, wherein the reference marking is activated only if the video sequence contains a selected image element. -19- 24. A method as claimed in claim 23, wherein the activation of the reference marking is represented by a pulsating and / or flashing reference marking. 25. A method as claimed in anyone of claims 19 to 24, wherein a first mode of displaying the signal element is provided. 26. A method as claimed in anyone of claims 19 to 25, wherein a second mode of displaying a first information is provided with respect to the selected image elements. 27. A method as claimed in claim 26, wherein all the selected image elements existing at the time of activation of the second mode are displayed in the second mode, wherein the selected image elements are preferably listed in a table. 28. A method as claimed in claim 27, wherein a third mode is provided, which is used to display detailed second information on the selected image element. 29. A method as claimed in anyone of claims 22 to 28, wherein the activation of the first, second and / or third mode is triggered by a time-conditioned and / or different actuation of action buttons, preferably of a radio control. 30. A recording medium with information data (li) on selected image elements of images of a video sequence (1) and with criteria data for linking a selected image element to the associated information (li) for applying the method as claimed in claims 1 to 29. 31. A device for preparing video sequences (1) to permit the display of information (li) on selected image elements of images of said video sequences (1), a reproducing unit for reproducing and displaying the video sequence (1); -20- - an acquisition unit for selecting and acquiring image elements of images of the video sequence (1); a detection unit for detecting the selected image elements in the subsequent images of the video sequence (1) and for determining the particular image coordinates (Xi, Yi) of the image elements; - a recording unit for storing information (li) on selected image elements and of criteria for subsequently (inking image coordinates (Xi, Yi) of selected image elements to information; characterized by - means for providing an additional signal element with respect to each selected image element on the reproducing unit, wherein the additional signal element is assigned to the image element selected and the information is displayed via the signal element. 32. A device as claimed in claim 31, wherein the recording unit is designed for recording the criteria as image coordinates (Xi, Yi) in relation to time, 33. A device as claimed in claim 32, wherein the recording unit is designed for calculating and recording motion vectors of the acquired image elements in the images of the video sequence (1) as criteria. 34. A device as claimed in anyone of claims 31 to 33, wherein the selected image element is, in addition, transmitted to the reproducing unit. 35. A device as claimed in any one of claims 31 to 34, wherein an icon bar with at least one selected image element is provided. 36. A device as claimed in anyone of claims 31 to 35, wherein a fading in unit for the icon bar and / or the image element selected is provided. -21- 37. A device as claimed in any one of claims 31 to 36, wherein a reference sign is provided for each signal element 38. A device as claimed in anyone of claims 31 to 37, wherein each signal element is provided in the form of a number. 39. A device as claimed in claim 38, wherein the recording unit is designed for calculating and recording motion vectors of the acquired image elements in the images of the video sequence (1) as criteria. 40. A device as claimed in any one of claims 31 to 39, wherein the recording unit is designed for calculating and recording an approximation function for describing the image coordinates (Xi, Yi) in relation to time with the parameters of the approximation function as criteria. 41. A device as claimed in any one of claims 31 to 40, wherein the acquisition unit is designed for applying methods of pattern recognition for tracking a selected image element in the images of the video sequence (1). 42. A device as claimed in anyone of claims 31 to 39, having a destination computer (7) for receiving data from a remote display unit (2), in the event of an image element being selected from the remote display unit (2) and information (!i) being linked to the image element. 43. A display unit (2) for displaying information (li) with respect to selected image elements of images of a video sequence (1) with a data processing unit (4), wherein the data processing unit (4) is designed for applying the method as claimed in anyone of claims 1 to 29. -22- The invention relates to a method for displaying information with respect to selected image elements. The aim of the invention is to simplify the interactive work via, for example, a frame. To this end, an image element is selected from the images of a video sequence (1) with which an information is associated. The selection proceeds partially via an additional image element or signal element that is likewise presented on the display unit (2) that displays the frame.

Full Text

Description
Method and device for displaying information with respect to selected image elements
of images of a video sequence
The present Invention relates to a method for displaying information with respect to selected image elements of images or a video sequence, a recording medium with a video sequence, with information data and criteria data for performance of the method, a device for preparation of video sequences in accordance with the generic farm of claim 29 and a recording medium and a display unit for performance of the method.
it is well known that certain image elements' of static single images. In particular of Internet pages in HTML script and of window-oriented operating systems can be linked with each other, These information may for example be operating supports or links to other Internet homepages. Therefore, In the HTML-programming language, the coordinates of rectangular image fields are defined and linked with information or a link to further homepages. When the image element appears on the screen and the user navigates the cursor onto the defined rectangular image field and selects it, the stored Information is shown or the Internet site, i e, the link is effected.
When transmitting video sequences on TV, such as sport events, talk shows or seriate, the spectator is often exposed to advertising messages, which pass by in the background. Therefore, for example certain objects are provided with advertising banners. Furthermore, the spectators attention is drawn to certain products by the so called product placement, with the products being introduced repeatedly and clearly visible in the telecast. Thereby, the spectator is massively confronted with said product.
The problem of the traditional advertising methods is the insufficient control of the advertising success. Furthermore, advertising is quite obtrusive and is poorly accepted by the spectator.
Thus, It is an object of the present Invention to provide a method for displaying information with respect to selected image elements of a video sequence or a device for preparation of video sequences, in order to simplify and improve interactive work via the image contents of a video sequence.
1
Further it is an object of the invention to develop the method for displaying information with respect to selected image elements of a video sequence or a device for preparation of video sequences, in a way that improved advertising with telecasts is possible and the spectators are interactively confronted with advertising elements.
The above-mentioned objects are, with respect to the method, solved in accordance with the features of claim 1, and, with respect to the device, solved in accordance with the features of claim 29.
Since, in accordance with the invention, the generally known method of static images, to link defined image fields with information and display them in a separate window on the display unit as soon as the image element has been selected with a cursor, has been transferred onto video sequences with a serial of slides, generally new possibilities, in particular with regard to advertising, could have been created.
Therefore, not only the traditional video sequence of the display unit, e.g. a television receiver, is transmitted. Additionally, defined information of selectable image elements are transmitted, which can be displayed after selection of the corresponding image element in a window.
Thus it will be achieved, that the traditionally available video sequences can remain unmodified up to the ultimate consumer and, furthermore, the ultimate consumer generally is free to decide to watch a video sequence, i. e. for example a television film, also without adaption, with respect to the method, of the video sequence. Depending on the embodiment of said method, the spectator thus has the possibility to traditionally satisfy his television consume- Due to said method according to the invention, it will thus be guarantied to keep traditional television habits on the one hand, and to simply switch on or switch to the interactive use. Therefore, criteria for linkage of a selected image element are transmitted with the appendant information with the video sequence.
Further preferred embodiments of the present invention are subject matter of the sub claims.
In accordance with claim 2 it is advantageous, if the criteria for linkage of a selected image element are the image coordinates of the image element in the respective image, with therefore an allocation schedule being created for allocation of information to image coordinates of selected image elements in the images of the video sequence
2
in temporal dependence of the video sequence. The data of the allocation schedule are transmitted as criteria to the display unit However it has to be considered that, when preparing the video sequence, it must be paid attention to the fact that the position of the selectable image elements generally persistently changes. By this, the determination of the linkage of the image elements to the image coordinates becomes more expensive.
In general a video sequence can be prepared advantageously by determining a time trace or track, denoting an image field in order to define a selectable image element, the image element being searched in the subsequent Images, e. g supported by well known pattern-recognition methods and the image coordinates or the displacement vector of the Imaga fields of the image element in the images of the video sequence, the respective time information of the time trace and the information appending the image elements being stored. By determining the time trace, a clear allocation of the image coordinates to the respective images is possible. Thereby, attention must be paid to the fact that the traditionally used video sequences of the broadcasting stations are often analogue and, if the video sequences are not respectively tagged, the image coordinates, stored in an independent data base, can later not be allocated to the individual images.
The transmittable data for allocation of the information to the image coordinates of selectable image elements can advantageously be reduced by the fact, that the course of the image coordinate of an image element is described as matbematic approximation function, which is transmitted as criteria. Therefore, the image coordinates can be described as function of the body with the coordinates x and y via time, e.g. supported by Spline functions. In this case, only the parameters of the approximation function need to be transmitted.
Advantageously, the time information of the time trace are transmitted synchronously to the video sequence and the image coordinates and respective Information asynchronously. However, it is necessary to transmit the criteria of a video sequence regularly, in order to avoid allocation mistakes, which appear by a change of transmitter. The user often "zappes" expeditiously through the particular transmitters, so that a wrong or no allocation at all to the images elements is possible, if the criteria are not updated regulariy.
3
Alternatively to the transmission of image coordinates as criteria for allocation of image elements to information it is also possible, to transmit the selectable image elements with the corresponding information to the display unit. When an image element has been selected, said element is compared to the transmitted and stored image elements, which are selectable and have information allocated. This comparison can for example be effected with known pattern-recognition procedure, which are executed in the display unit.
Advantageously, for displaying the information a window is overlain over the video sequence on the display unit
it is particularly advantageous if the information consist of interactive elements, so that data can be transmitted from the display unit to a target computer, after an image element has been selected from the images of the video sequence onto the display unit. The data and/or the target computer are hereby dependent on the information, . which correspond to the selected image element. The interactive information can for example be created like an Internet link. Advantageously, input elements can be provided in the information window, in which the user, after superimposing the information window onto the display unit, can enter further data, such as his address and order information for a product corresponding to the image element. This data are transmitted to the target computer and, for example, an order is effected. Thus it is possible to use video sequences as advertising media and to perform an advertising-success control supported by interaction and to provide an order system.
Advantageously, the data transfer to the target computer can be effected via a wideband network, an electric network, a telephone network or vial a radio and satellite network or similar.
Advantageously, the criteria for linkage of a selected image element with the corresponding information are transmitted In the blanking interval of the signal of the video sequence.
When transmitting the video sequence as digital data stream, a defined unit of the data stream is provided for the criteria and the information. For a compressed data stream in accordance with the MPEG-2-standard, data units or areas can be used, which are kept free for additional user-specific functions.
4
For execution of the method, information data of selected image elements of images of the video sequence and data as criteria for linkage of a selected image element with the corresponding information are stored on the traditionally known recording media.
If a selected image element is additionally transferred as so-called additional image element of the display unit, i.e. the display unit provides twice of a selected image element, it is advantageously achieved that the user is optically shown, which image element is stored with information and that he can effect the displaying of the information via additional image elements. Thus, the user can follow the video sequence unmodified and, if necessary activate the selectable image elements in the display unit, i. e. bring the additional image element onto the screen with being signalised, which image elements are stored with information.
To provide the user optically with a certain structure or orderliness it is advantageous, if additional image elements or one additional image element in form of an icon listing are provided at a predetermined place on the screen. To enable the user to follow his normal television habits, the additional image element is only superimposed when necessary.
To generally inform the user that selected image elements exist in the particular video sequence which are stored with information, it is advantageous, if this possibility of superimposition is preferably shown on the margin of the display unit or the screen with a separate hint tag, which may for example be a bar or a particularly for the user configured sign.
However, should the user prefer to watch the video sequence with tagged or marked selected image elements it may be advantageously if the user can individually determine, which image elements should be shown in the video sequence as selected images. The user might for example determine, that due to his technical interest, exclusively technical devices, for which an information is stored, are determined as additional image element, so that, due to the tagging in the video sequence he can see, which of the technical devices in the video sequence is stored with an information.
It has proved advantageously to show the additional image elements in a certain order, for example in sequence, and that the displaying of the information is effected via a corresponding digit input, which corresponds to the sequence diagram.
5
Should a user generally disfavour to be shown too many image elements on the display unit or on a television screen it is, in accordance with claim 19, advantageously, if only one signal element is provided to each selected image element, which for example can follow the selected image element on the video sequence. With this measure the user is thus signalled that an image element is selectable and it can, due to the placement of the signal element clearly be determined, which image element is stored with information. Due to general habits it has proved advantageously, if the particular place is preferably determined on the right above the selected image element. Attention is drawn to the fact, that each allocation of the signal element to the selected image element is imaginable.
If furthermore, in correspondence with claim 21, a signal element is provided with a hint tag or hint marking, e.g. a numerical digit or an integer, the user can for example select the image element by pressing the corresponding digit, in order to demand the display of the information.
If, in accordance with claim 22, the hint tag is linked with the signal element via a connection line, it is clearly visible for the user that now the selected image element, which is linked with signal element is stored with information. However, this presentation can only be effected for a certain period, preferably one second, in order to generally avoid any disturbance in the video sequence. Furthermore it will be achieved with this measure to show the user the possibility, that selected image elements with information are provided in the special video sequence.
With the possibility to only activate the hint tag when a selected image element is provided in the video sequence, the user has the possibility to easily recognize when he can demand information to a selected image element
A further possibility to indicate the user during the video sequence when he can obtain information is for example a pulsing and/or glancing hint tag. A pulsing and/or glancing hint tag has the particular advantage, that each time when further new selected image elements are added to the presentation of a video sequence, the hint tag or hint marking for the already existing selected image element begins to glance or pulse.
If, in accordance with claim 25, the method is separated into different modes, different actions are imaginable within the particular modes. A first mode can, in accordance with claim 25. be provided for displaying the signal element, with which it is possible to
6
switch from the normal presentation of the video sequence, i. e. presentation of a film, to the activating mode, if a selected image element is provided in the video sequence. in contrast to the first mode, a second mode may be provided, which will he, used for displaying first information with respect to the selected image elements. Thus, the user can twice get a general survey about the stored Information with respect to a selected image element
. If, in accordance with claim 27, in the second mode all selected image elements of a video sequence, which has been available when activating the second mode, are shown, the user will directly and simply get a total overview of the selected image elements stored with information.
Should the user show greater interest in a selected image element, more detailed second information of the selected image element can be demanded and shown by a third mode.
If, In accordance with claim 29, the activating of the first, the second and/or the third mode or any further existing mode is effected fay limited in time and/or different pressing of action buttons, a maximum success is offered to the user, in particular when he has to press the action buttons, which he usually also uses when presenting the video sequence, i.e. when watching TV.
A device for preparation of video sequences for allowing to display information of selected image elements of Images of the video sequence has advantageously one reproduction unit for reproducing and displaying the video sequence, one gathering unit for selecting and gathering of image elements of images of the video sequence, one detecting unit for detecting the selected image elements in sequent images of She video sequence and for determining the respective image coordinates of the image elements and of criteria for the later linkage of image coordinates of selected image elements with information.
The display unit preferably provides of a data-processing unit for performance of the method. As data-processing unit for example a set-top box may be used, which preferably allows access to the Internet
Hereinafter, the invention will be disclosed more detailed in accordance with the accompanying drawings.
7
It is shown:
Fig. 1 - a flow diagram of a first embodiment for the preparation of video
sequences for performance of the method;
Fig. 2- a schematic block diagram of a system for performance of the method;
Fig. 3 shows in a second embodiment an image on the display unit, visible
for the user;
Fig. 4 - shows in a third embodiment an image on the display unit, visible for
the user;
Fig. 5 shows in a fourth embodiment an image on the display unit, visible for
the user;
Fig. 6 - shows a schematic block diagram of a device for performance of the
method in accordance with a fifth embodiment;
Fig. 7 - schematically shows the transition between the normal operating mode
and the first mode in accordance with the fifth embodiment-Fig. 8 - shows the transition between the first mode and the second mode in accordance with the fifth embodiment;
Fig. 9 - schematically shows the operational mode of the second mode;
Fig. 10 - exemplarily shows the possible presentation of the first information in
the second mode, in accordance with the fifth embodiment;
Fig. 11 - schematically shows the operational mode of the third mode;
Fig. 12- exemplarily shows second, more detailed information in the third
mode;
Fig. 13 - schematically shows the operational mode of the third mode.
8
Fig. 1 shows a flow diagram for preparation of video sequences for performance of the method according to the invention. First, a rough video sequence of a client is digitaiized, if it is available as analogue video sequence. Then a time trace is written onto the video sequence in order to allow a later allocation and choosing of image elements in dependence of time. After this an empty database is created and information are registered in said database. Therefore, the selectable image elements, which have to be linked with particular information, are framed. The frame or the image element is in this embodiment sequenced by the scenes of the video sequence and the respective image coordinates are stored in dependence of the time trace. It is advantageous, if only the alterations of the image coordinates are stored. This can save storages space. The recording of information in the database and the determination of the corresponding image coordinates is effected consecutively for each information item to register.
Thus, a digital or analogue video sequence with a time trace and a linked database with information and image coordinates, which are stored in dependence with the time trace, has been created. The video sequence is transmitted synchronously with the time trace to a display unit, e.g. a television. Asynchronously to that, the data of the database are for example transmitted in the blanking interval of the television signal. When playing such prepared video sequence on TV, the user can select the image elements with a cursor, preferably mouse controlled. The image coordinates of the selected image element are compared with the transferred image coordinates and the corresponding information are shown, when the image coordinates match with those of an image element, for which information have been stored. Therefore, one window is opened on the screen and the information of the displayed video sequence is superimposed. St is thus possible to transfer and display background information of video sequences. Furthermore, advertising messages and order facilities can be demanded interactively by the user.
The method in accordance with the first embodiment is explained more detailed in the block diagram of a system for performance of the method, designed in Fig. 2. The digital or analogue video sequence 1 with the time trace is transmitted to a display unit 2, which consists of a television 3 and a data-processing unit 4, The time information is transmitted synchronously to the television signal in a hidden transmission area. Additionally, information li are transmitted to selectable image elements and the respective image coordinates Xi, Yi are transmitted asynchronously to the television
9
signal to the. data-processing unit 4 of the display unit and are stored there. Supported by a mouse, an infrared or radio pointer 5, a keyboard 6 or similar, the user can select image elements on the screen of TV 3. The corresponding image coordinates are transmitted to the data-processing unit 4 and compared with the stored-image coordinates Xi Yi of the selectable image elements. If the image coordinates are in the image field of selectable image elements, the finked information II is shown in a separate window on the screen. If this Information li is connected with a link and an interaction has been started, a linkage to a target computer 7 is built in a central and data corresponding to information li, e.g. for advertising control, for order execution, etc. are transmitted to the central computer. Therefore, advantageously the internet,is used. Hereby, the data-processing unit 4 can be a so-called set-top boxionsimilar.
in Fig. 3 a total image 13a, which is visible to the user, e.g. of the display unit 3 of Fig, 2 is shown, as it may appear in accordance with a second embodiment The total image 13a shows a video sequence with a plurality of image elements, within this example appearing five selected image elements I, II, III, IV and V, At a suitable position on the display image 13a an icon listing is provided, in which the selected image elements I, III, HI, IV and V are illustrated as additional image elements XI, XII, XIII, IX and XV, Due to this allocation, the user is signalled via the additional image elements XI, XII, XIII, IX and XV, which of the image elements I, II, III, IV and V appearing in the video sequence are selectable to display the stored information, it is drawn attention to the fact, that the icon fisting can also be superimposed or overlain or faded-in and that In particular this possibility of superimposing can be superimposed for example by a special movement of a mouse or hand-control element. This presentation also shows, that for example the user, should he wish to get the information to the additional image element XIII, due to position 3 in the icon list, could demand the stored information for example by pressing no. 3, e.g. on the teleguidance, which is then displayed for example as superimposed window also on the display unit (not shown). The principle basing on this second embodiment is, that during the audio- video transmission an icon listing 13b is superimposed with the additional image elements XI, XII, XIII, IX and XV as the selectable image elements. This icon listing depends on the support of an input device, (e.g. mouse, cursor button, control pad, teleguidance elements, radio pointer, etc.) the additional image element being selected and depending on the selection of an action, e.g. order, ordering information, etc, i.e. display the respective information. For the realization of this concept only the additional image element needs to be transmitted, and the time, when said additional image element, i.e. by means of the corresponding time trace of the video sequence is visible. A transmission of for example the image
10
coordinates of selected image elements in the images of the video sequence, which correspond to the image elements of the image transmission is thus not necessary, und no so-called hot spots need to be transmitted.
In Fig. 4 a third embodiment of the present invention is shown, which essentially uses the same presentation mechanics for the selected image elements as the second embodiment. A display image 23a is shown, which also shows selected image elements I, II, III, IV and V and an icon listing 23b with the additional image elements XI, XII, XIII, IX and XV. Opposite to the second embodiment, now the selected image elements I to V are provided with a tag or marking in form of a selection frame, to allow the user to discern, in addition to the icon listing 23b in the display image/which image element is activated or stored with information. Furthermore it is possible to provide the currently selectable selected image element e.g. I with one opposed to the other selected image element with another selection frame, e.g. thicker frame contour or other colours, with the corresponding additional image element, here XI, also being optically highlighted in contrary to the other additional image elements II to V. Due to this highlighting it is granted, that the user can only demand the stored information of the selected or additional image element with a generally defined user-button, e.g. the "OK" button of the teleguidance. However, for this method additionally the frames for the selected image elements and its coordinates need to be transmitted, but no so-called Hot-Spots are transmitted or any actions depend on those Hot-Spots. If the additional image elements are reproduced in correspondence with the icon listing shown in Fig. 4, for example the user can only select with a corresponding "right-left" button in the icon listing the interesting additional image element and demand it respectively with the general input button. The activating of the selected image element does thus not effect via so-called Hot-Spots but only via the additional image elements XI to XV.
In Fig. 5 a fourth embodiment of the present invention is shown. In this embodiment another display image of display unit 3 is shown, in which again selected image elements I to V are visible. In this embodiment a signal element 35 is provided for each selected image element 1 to IV, with each signal element 35 being provided with a hint tag or hint marking, e.g. a digit or letters which can directly be selected, with pressing of the respective digit of the letter, the stored information of the selected image element being shown. For this method only the coordinates of the signal elements need to be transmitted, which for example may depend on the image coordinates of the selected image elements. The displaying of the information, which is allocated to the selected
11
image element, thus depends, on the activating of a separate input device and, again, no so-called Hot-Spot is used. It is pointed out that the optical preparation can be contrasted similarly as in the third embodiment by highlighting, i.e. possibly by different border margins or colorations.
Furthermore it shall be pointed out that the respective method of first to fourth embodiment can also concurrently be used and combined, so that a manifold presentation possibility is given to the user.
Subsequent to the third embodiment, in which the selected image elements are provided with a selection frame, it shall be pointed out that the so-called transparent GIF-image formats can bo transmitted separately to the display unit, which can be exactly as large as the display unit 13a to 33b and have the selection frame for the selected image element at an appropriate position or predetermined coordinates. It is thus possible to save the transmission of image coordinates, so that only over the time trace corresponding to the image format and respective selected image elements the display of information is allowed.
Thus, said method and device allow a product placement of telecasts with active advertising-success control. Furthermore, information for an image element can be connected with a link, so that after selection of the image element an order of the product presented in the image of the video sequence is possible online via data transfer.
If the user favours a product which is shown in the video sequence, such as a garment, he can select said product with the mouse and get the information of said product, such as name and address of the producer, price, quality etc. Now a further information field is selectable, which allows the user to enter order data and order the product directly from the producer. Therefore, the order information are transferred to the provider via a back channel, i.e. via broadband, electric, radio or satellite network or supported by a telephone connection, who arranges the dispatch and the invoicing for the product.
Thus, very inconspicuous advertising is possible, which additionally uses the curiosity of the spectator. Furthermore an advertising-success control is also possible for the product placement.
12-
Finally, the functionality of the method or device in accordance with the fifth embodiment shall be shown more detailed supported by Fig. 6 - 13. As shown in Fig. 6, the method or device use different modes, with allowing to change from one general operating mode, i.e. TV mode to another mode, which activates the hint tags e.g. in shape of a star, the so-called Starzone mode (see Fig. 7). Depending on the actuating of the action button it is possible to change from the Starzone mode into the first mode, i.e. IPP mode, in which the signal element to a selected image element is activated. From the first mode, i.e. IPP mode, it is now possible to change into a second mode, which for example contains a cart, also by actuating of an action button or other action measures. Furthermore it is also possible to directly change from the first mode into a third mode, which shows more detailed information of the selected image elements. However it is also possible to change from the second mode into the third mode. As clearly shown in Fig. 6 it is also possible to change by a special operating measure from the operating mode, i.e. TV mode and/or the Starzone mode directly in the cart of the second mode. It has proved advantageously, that the order could not have been effected, if from the cart of the second mode firstly is changed into a so-called order box, from which either a confirmation of order or an error hint has been received.
Furthermore it is advantageously, that if from the first mode it is changed into a mode, the user is signalled that the information corresponding to the selected image element have already been activated, e.g. that the product corresponding to the selected image element is already registered.
Fig. 8 shows the possibilities to change from the so-called Starzone mode into the first and second modes by different actuating of action measures. If the system is in the Starzone mode, it is for example possible to change into the IPP mode, i.e. the first mode, by shortly pressing a certain button, e.g. a button green. If for example the user presses the same button, i.e. button green longer, the system will change into the cart, as shown in Fig. 8. In the IPP mode, i.e. first mode, the selected image elements are stored with information, with a particular signal element pursuing the respective selected image element during the current operating mode or consecutively sequencing video sequences. When pursuing the signal elements with the selected image elements, two different variations can be used. In the first variation, the signal element is stuck to the selected image element and thus skips at a scene change of different video sequences. At a second variation, the signal element is softly drawn at a scene change. This means, that, at a scene change by animation, the digit or integer is newly
13
placed at the selected image element or at the predetermined position corresponding to the image element.
In the IPP mode (see Fig. 9) the user has now the possibility to store the selected image element and the respective information in a separate data storage by respective pressing of an activating button, which corresponds to a signal element If the user desires for example detailed information of a selected image element, he can, by pressing an action button corresponding to the signal element of the selected image element, directly change for example into the third mode, which consists of a product box. If the user desires to leave the IPP mode, he can return to an action button corresponding to the signal element into the Starzone mode. If he for example presses the action button longer, the complete system is interrupted. If he presses an action button different to the mentioned action button, he can for example directly change into the second mode, which consists of the cart.
Fig. 10 and 11 schematically show the different possibilities with respect to the second mode, which consists of the cart, and the presentation of first information with respect to the selected image elements. In Fig. 10 the selected image elements of the IPP mode are preferably listed tabularly or shown in the so-called cart. Each selected imago element is allocated for example a digit or integer, which can be demanded consecutively. In this connection it is pointed out, that the selected image elements can be shown in dependence of the status, with the following states being differed. Firstly those selected image elements, which have not been stored with further information, e.g. order information such as size, etc, and those selected image elements, which have been stored with further information, such as order information. The latter information elements can for example be highlighted in colour. If the user selects one of the digits allocated to the selected image elements, this selected image element is highlighted in colour. Depending on the actuating of the action button, the product box of the third mode opens or the selected image element is deleted from the cart. Referring to this it is also possible to provide further action measures, which for example open the product box. If the user presses for example a different action button in addition to the former action button, for example the order box opens. It is pointed out to the fact, that the action button remains inactively until the respective information of the selected image element has been entered by the user. If the user shortly presses for example a further action button in addition to the above-mentioned, the system changes back into the IPP mode. However, it is also possible to interrupt the system by further different action measures. As a safety measure it is for example possible to
14
provide a further action button, which needs to be pressed to delete selected image elements.
In Fig. 12 the product box, which is allocated to the third mode is shown, with the possible action inputs being shown schematically in Fig. 13. As it can be taken from Fig 12, in the third mode two more detailed information of selected image elements are shown. If the user presses a certain action button, the respective input unit can be highlighted in colour, with the user then being able to enter the respective information. If the user presses a further action button, it is possible to re-skip for example in the second mode, which consists of the cart, and the order information are adopted. If the user presses a different action button additionally to the above-mentioned action button, he can for example also be lead into the second mode, but the information are. not adopted. By pressing of further action buttons, the third mode can be interrupted or the selected image element can be deleted from the cart in accordance with the second mode.
Fig. 13 shows the further possibilities, how further information can be entered by different action buttons.
Fig. 14 shows the operating system of the order box, with additionally pointing out, that the order box is operated with a security code, in order to avoid involuntary activating of information, i. c. orders. If the information or orders have been effected, the user is signalled by a confirmation of order, that finally the interactivity is finished.
Basically it is also possible to support each individual method step with an online support.

15
WE CLAIM :
1. A method for displaying information (li) with respect to selected image elements of images of a video sequence (1), comprising the following steps of:
a) receiving the video sequence (1) through a display unit (2), receiving defined information (li) on image elements, and receiving criteria for linking a selected image element to the associated information (li);
b) selecting an image element from the images of the video sequence (1) displayed on the display unit (2);
c) displaying information (li), provided the criteria revealed that said information (li) is assigned to said selected image element;
characterized by
d) providing an additional signal element with respect to the selected image element, wherein said additional signal element is assigned to the selected image element and said additional signal element is used to display said information.
2. A method as claimed in claim 1, wherein the image coordinates (Xi, Yi) of the selected image elements serve as criteria for linking a selected image element to the associated information (li), wherein an assignment table for assigning information (li) to image coordinates (Xi, Yi) of selected image elements is created in the images of the video sequence (1) and that the data of the assignment table is transferred as criteria.
3. A method as claimed in claim 2, comprising the following steps :
- defining a time track for the video sequence (1);
choosing an image area for defining image elements to be selected;
-16-
searching the image element in the subsequent images;
storing the image coordinates or displacement vectors of the image areas of the image element in the images of the video sequence (1), the appropriate time information of the time track and the information (li) for the image element
4. A method as claimed in claim 3, wherein a mathematical approximation function for describing the course of the image coordinates (Xi, Yi) of an image element is determined and transferred as criteria.
5. A method as claimed in claim 3 or 4, wherein the time information of the time track is transferred synchronously with the video sequence (1) and that the image coordinates (Xi, Yi) and the associated information (li) is transferred asynchronously with the video sequence (1).
6. A method as claimed in claim 1, comprising the following steps of:
- choosing an image element to be selected;
storing the image element chosen and the associated information (li) as
criteria;
- transferring the video sequence (1), the image elements stored and the associated information (li) to a display unit
checking whether a stored image element corresponds to a selected image element and displaying the associated information (li),
7. A method as claimed in any one of claims 3 to 6, wherein the steps of searching or checking the image elements are earned out with methods of pattern recognition.
-17-
8. A method as claimed in any one of the preceding claims, wherein a window is superimposed on the video sequence (1) on the display unit (2) for displaying the information (li).
9. A method as claimed in any one of the preceding claims, wherein data is transferred from the display unit (2) to a destination computer (7) after an image element has been selected from the images of the video sequence (1) on the display unit (2), wherein the data and / or the destination computer (7) depend(s) on the information (li) associated with the image element selected.
10. A method as claimed in claim 9, wherein the transfer of the data to the destination computer (7) is achieved via a broad-band network, via a power supply network or via a telephone network or via a radio and / or satellite network.
11. A method as claimed in anyone of the preceding claims, wherein the criteria for linking a selected image element to the associated information (li) are transferred in the blanking interval of the signal of the video sequence (1).
12. A method as claimed in anyone of claims 1 to 10, wherein the criteria for linking a selected image element to the associated information (li) are transferred in a defined range of a video sequence (1) that is provided as a digital video data stream.
13. A method as claimed in anyone of claims 1 to 12, wherein the selected image element is, in addition, transmitted to the display unit and the information is displayed via the additional image element.
14. A method as claimed in claim 13, wherein the additional image element is faded in the form of an icon bar providing the image containing the image element selected.
15. A method as claimed in claim 13 or 14, wherein, if necessary, the additional image element is inserted in the image containing the selected image element.
-18-
18. A method as claimed in anyone of claims 1 to 15, wherein the option of the fading in is shown on the display unit by means of a separate reference sign.
17. A method as claimed in anyone of claims 13 to 16, wherein the additional image element is used to define the selected image element that is to be chosen.
18. A method as claimed in anyone of claims 13 to 17, wherein the additional image elements are shown successively one after the other and that the information is displayed by entering an appropriate number of the successively shown additional image elements.
19. A method as claimed in anyone of claims 1 to 18, wherein the signal element is each provided at a specific location in relation to the image element selected on the display unit.
20. A method as claimed in anyone of claims 1 to 19, wherein the signal element is provided to the upper right of the image element selected on the display unit.
21. A method as claimed in claim 20, wherein the signal element is provided with a reference sign, preferably a number.
22. A method as claimed in claims 16 to 21, wherein the reference marking is, via a connection line, connected to the signal element for a specific time interval, preferably for 1 second.
23. A method as claimed in claims 16 to 22, wherein the reference marking is activated only if the video sequence contains a selected image element.
-19-
24. A method as claimed in claim 23, wherein the activation of the reference marking is represented by a pulsating and / or flashing reference marking.
25. A method as claimed in anyone of claims 19 to 24, wherein a first mode of displaying the signal element is provided.
26. A method as claimed in anyone of claims 19 to 25, wherein a second mode of displaying a first information is provided with respect to the selected image elements.
27. A method as claimed in claim 26, wherein all the selected image elements existing at the time of activation of the second mode are displayed in the second mode, wherein the selected image elements are preferably listed in a table.
28. A method as claimed in claim 27, wherein a third mode is provided, which is used to display detailed second information on the selected image element.
29. A method as claimed in anyone of claims 22 to 28, wherein the activation of the first, second and / or third mode is triggered by a time-conditioned and / or different actuation of action buttons, preferably of a radio control.
30. A recording medium with information data (li) on selected image elements of images of a video sequence (1) and with criteria data for linking a selected image element to the associated information (li) for applying the method as claimed in claims 1 to 29.
31. A device for preparing video sequences (1) to permit the display of information (li) on selected image elements of images of said video sequences (1),
a reproducing unit for reproducing and displaying the video sequence (1);
-20-
- an acquisition unit for selecting and acquiring image elements of images of the video sequence (1);
a detection unit for detecting the selected image elements in the subsequent images of the video sequence (1) and for determining the particular image coordinates (Xi, Yi) of the image elements;
- a recording unit for storing information (li) on selected image elements and of criteria for subsequently (inking image coordinates (Xi, Yi) of selected image elements to information;
characterized by
- means for providing an additional signal element with respect to each selected image element on the reproducing unit, wherein the additional signal element is assigned to the image element selected and the information is displayed via the signal element.
32. A device as claimed in claim 31, wherein the recording unit is designed for recording the criteria as image coordinates (Xi, Yi) in relation to time,
33. A device as claimed in claim 32, wherein the recording unit is designed for calculating and recording motion vectors of the acquired image elements in the images of the video sequence (1) as criteria.
34. A device as claimed in anyone of claims 31 to 33, wherein the selected image element is, in addition, transmitted to the reproducing unit.
35. A device as claimed in any one of claims 31 to 34, wherein an icon bar with at least one selected image element is provided.
36. A device as claimed in anyone of claims 31 to 35, wherein a fading in unit for the icon bar and / or the image element selected is provided.
-21-
37. A device as claimed in any one of claims 31 to 36, wherein a reference sign is provided for each signal element
38. A device as claimed in anyone of claims 31 to 37, wherein each signal element is provided in the form of a number.
39. A device as claimed in claim 38, wherein the recording unit is designed for calculating and recording motion vectors of the acquired image elements in the images of the video sequence (1) as criteria.
40. A device as claimed in any one of claims 31 to 39, wherein the recording unit is designed for calculating and recording an approximation function for describing the image coordinates (Xi, Yi) in relation to time with the parameters of the approximation function as criteria.
41. A device as claimed in any one of claims 31 to 40, wherein the acquisition unit is designed for applying methods of pattern recognition for tracking a selected image element in the images of the video sequence (1).
42. A device as claimed in anyone of claims 31 to 39, having a destination computer (7) for receiving data from a remote display unit (2), in the event of an image element being selected from the remote display unit (2) and information (!i) being linked to the image element.
43. A display unit (2) for displaying information (li) with respect to selected image elements of images of a video sequence (1) with a data processing unit (4), wherein the data processing unit (4) is designed for applying the method as claimed in anyone of claims 1 to 29.
-22-
The invention relates to a method for displaying information with respect to selected image elements. The aim of the invention is to simplify the interactive work via, for example, a frame. To this end, an image element is selected from the images of a video sequence (1) with which an information is associated. The selection proceeds partially via an additional image element or signal element that is likewise presented on the display unit (2) that displays the frame.

Documents:

« Previous Patent

Next Patent »

Patent Number

207725

Indian Patent Application Number

IN/PCT/2002/00679/KOL

PG Journal Number

25/06/2007

Publication Date

22-Jun-2007

Grant Date

21-Jun-2007

Date of Filing

20-May-2002

Name of Patentee

STARZONE GMBH

Applicant Address

EXPO PLAZA 12, D-30539 HANNOVER GERMANY

Inventors:

#	Inventor's Name	Inventor's Address
1	OTTO CHRISTIAN	LANDSBERGSTRASSE 19, D-31141 HILDESHEIM, GERMANY
2	KNISCHEWSKI FRANK	AN DER MAINTE 3A, D-31039 RHEDEN, GERMANY

PCT International Classification Number

H 04 N 5/445

PCT International Application Number

PCT/EP00/10388

PCT International Filing date

2000-10-20

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1	100 33 134.3	2000-07-07	Germany
2	199 50 939.5	1999-10-21	Germany