Title of Invention	METHOD FOR EDITING LIVE AUDIO AND VIDEO
Abstract	The present invention proposes a real time Audio Video (AV) editor for an embedded device with camera facility. The invention provides new editing options for recording live video in already existing file. This helps the user to organize the videos in a systematic way depending on the user requirement or importance of the event to be recorded. The present invention helps in efficient utilization of memory space and battery life of the embedded device and supports quick access and indexing of the recorded data.

Title of Invention

METHOD FOR EDITING LIVE AUDIO AND VIDEO

Abstract

The present invention proposes a real time Audio Video (AV) editor for an embedded device with camera facility. The invention provides new editing options for recording live video in already existing file. This helps the user to organize the videos in a systematic way depending on the user requirement or importance of the event to be recorded. The present invention helps in efficient utilization of memory space and battery life of the embedded device and supports quick access and indexing of the recorded data.

Full Text	FIELD OF THE INVENTION The present invention relates to the field of embedded systems and specifically to the field of embedded devices with camera facility. More particularly, the present invention relates to a system and method of real time Audio Video (AV) editing. DESCRIPTION OF THE RELATED ART In the existing technology, digital recorders available in embedded devices record live Audio Video (AV) in various media file formats. When a session is recorded a new file gets created with a file name automatically generated by the firmware. These automatically generated file names (for example Recorded 34.3gp, Recorded 43.3 gp etc.) create an annoying situation for the user. Further, with the existing technology in digital recorders, it is not possible to record a new session in an already existing or pre-recorded file. LIMITATIONS 1. Existing recorders does not provide an option to insert a new session of live AV into an already existing or pre-recorded file at a particular point selected by the user. In case if the user wants to record live video after a specific event in an already recorded video session or if the user have missed out some important events while recording, then the user has to re-record and insert the particular event at the specific points in the recorded video. In the above cases, the user records each session in different files and then with the help of different hardware and software the files are merged offline. But this results in wastage of time. 2. Existing recorders do not provide any option for changing the order of scenes or order of events in an already recorded video session that can make the already recorded video more meaningful. If a user wants to record some event that is expected to happen and incase the user misses the same, the user records the next event instead. Existing technology does not provide an option to insert the particular expected event at a particular point in the already recorded video, when the expected event is ready to record. 3. Existing recorders do not provide any option to overwrite live AV (new session) in already existing or pre-recorded file at a particular point selected by the user. Hence if the user is not interested with the quality of recording for a specific reason in a pre-recorded file or if some portion of the event is not captured properly or if some portion of the file is useless or less important or if the recorder is having memory constraints the user cannot overwrite a new session of live audio video into an already existing or pre-recorded file at a particular point selected by the user. 4. Existing recorders stop the process of recording when there is no sufficient memory in the recording system. This may result in missing important data due to memory non-availability. 5. Existing recorders do not provide any mechanism to record an unexpected event in an already existing or-pre-recorded file. 6. Existing recorders provide the options of 'pause' and 'resume' for reinitiating an ongoing recording session. But as the recorder application remains active during the 'pause' period hence the battery consumption is more. 7. With the existing technology in recorders, if a user wants to record certain events in the same file then it requires recording of unwanted events also to continue recording in the same file. This results in wastage of useful memory space. 8. Existing recorders create a new file for each recording session. If this happens for a number of times then keeping track of proper sequence and renaming files with proper file names becomes difficult. Also if the user switches off the recorder application or switches off the device then he cannot record live video next time in the same file. 9. Existing recorders do not allow a user to close or suspend or switch off (or shut down the device) the recorder application so as to use the device for some other purposes like making/receiving a call and to resume recording again in the same file. 10. Existing technology in recorders does not support real-time editing options and hence requires merging of multiple files using different hardware and software to create a single file. This results in more time consumption. 11. With the existing methodology in recorders it is not possible to correct the action recorded by repeating the events and then re-recording in the same file. 12. Existing recorders do not support a quick access and indexing for recorded video. 13. With the help of existing recorders it is not possible to create interesting video by combining existing video and live video as per user wish. Hence there is a need to help the user in situations like recording device running short of memory, battery low condition of the recording device, merging different parts of recorded events in the same file, removing unwanted portion in the recorded video, inserting live video at a desired point etc. According to a technique disclosed in a patent application, a Digital Video Recorder (DVR) device records television content from a particular channel in a temporary buffer. Once the temporary buffer is full, the DVR device starts overwriting the temporary buffer from the beginning, replacing the earlier recorded video content. The application however, does not mention the feature of inserting or appending video content in a pre-recorded video file. According to another technique, a video recording apparatus gives an option to a user to choose the area on the memory where he wants to record the new video content. In one of the embodiments, the user can select pre-recorded video files in the memory to be overwritten by the new video content to be recorded. The new video, thus, gets recorded on these selected files. The patent however, does not explicitly mention inserting video within a pre-recorded video file or appending video content at the end of the pre-recorded video file. The present invention has been designed to address these needs. SUMMARY OF THE INVENTION It is therefore a primary object of the present invention to provide a real time Audio Video (AV) editor in an embedded device with camera facility. It is another object of the present invention to provide the user with an option for recording live data in an already existing pre-recorded file. It is another object of the present invention to provide an editing option that helps user to insert or record live data anywhere in the already existing file. It is yet another object of the present invention to provide an option to record live data multiple times in one single file with one key press in the video recording device. It Is yet another object of the present Invention to provide an editing option that helps user to overwrite live data anywhere in the already existing file. It is yet another object of the present invention to provide a feature in an embedded device with camera facility to record an unexpected event and to insert or overwrite the particular event at a selected point in an existing file in real time. The present invention relates to the field of video recording. In particular, the invention describes a video recording application with new editing options. The video recording application provides real time editing options such as append, insert, overwrite, etc. The insert option provides the facility of inserting a new video in a pre-recorded video file whereas the append option allows to add a live video at the end of an already existing video file. The overwrite option provides better memory management and permits to record a video on a pre-existing video file if the user device is running short of memory. For effective memory management, there are other features provided as well, such as re-start and window recording. While recording a file, if the recording device runs out of memory, the user can use the re-start option to start recording from the beginning of the file, such that the new recording overwrites the already recorded data. The window recording option on the other hand has a circular buffer where the latest data overwrites the data at the beginning of the buffer. For example, if the user defines the duration as 20 seconds, then the circular buffer stores the last 20 seconds of recording. The invention also helps in effective battery management. Accordingly, the user can switch off the recorder when not in use and start recording from the desired place using the available options when the recorder is switched on again. Accordingly, the invention explains a system for real time AV editing comprising, a camera pre-recording preview module for previewing live frames captured by a camera ; live AV recording module for recording audio and video encoded /compressed data in real time; media player for playback of selected recorded audio and video encoded data; and editing module for editing the encoded audio and video data. Accordingly, the invention explains a method for real time AV editing comprising the steps of, previewing live frames captured by a camera using a camera pre-recording preview module ; recording audio and video encoded /compressed data in real time using live AV recording module; playing selected recorded audio and video encoded data using media player; and editing the encoded audio and video data using editing module. These and other objects, features and advantages of the present invention will become more apparent from the ensuing detailed description of the present invention taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE ACCOMPANYING FIGURES Figure 1 depicts a block diagram illustrating the components involved for implementing a real time Audio Video (AV) editor; Figure 2 depicts the 'continue' option according to an embodiment of the present invention; Figure 3 depicts the 'insert' option according to another embodiment of the present invention; and Figure 4 depicts the 'overwrite' option according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION The preferred embodiments of the present invention will now be explained with reference to the accompanying drawings. It should be understood however that the disclosed embodiments are merely exemplary of the invention, which may be embodied in various forms. The following description is not to be construed as limiting: the invention and numerous specific details are described to provide a thorough understanding of the present invention, as the basis for the claims and as a basis for teaching one skilled in the art how to make and/or use the invention. However, in certain instances, well-known or conventional details are not described in order not to unnecessarily obscure the present invention in detail. The present invention proposes a new use case for Video Recording application that provides editing facilities in real time while recording live audio video (AV). Accordingly, the present invention provides a method for efficiently utilizing the existing AV files further providing easy and quick access to the AV content, which the user has appended, added or overwritten. The present invention basically provides new options to the user for recording new video in already existing or pre-recorded file. The invention provides an option to record new session in already existing or pre-recorded file and therefore user is relieved from creating multiple files, remembering relevance and then merging the multiple files using multiple devices and different applications or softwares. User has a better control to organize his recorded files based on subject or theme of recording and based on the memory space in the recording device, battery life and situational requirements. The invention provides an option to insert live video session in already existing or pre-recorded file. This is a great advantage to the user to alter the sequence of events in video and create interesting video. Accordingly, the new options provided by the present invention are explained below. Continue- Using this option user can record live data in already existing file. When user wants to record an event, he can choose any existing or pre¬recorded file to continue recording in the same file using 'continue' option. The application appends the file with live recorded data. The facility of appending pre-recorded file is extended further with 'quick Start' option in an attractive way to help user to save battery life and to avoid multiple creation of the files. Pre-pend- This is similar to the "'continue' feature and uses the "'continue' functionality as explained above to add live media content to an existing file at the beginning. Quick start- This option is useful to record live data multiple times in one single file with one key press. It helps user to save battery life and to avoid creation of multiple files of the same event and also relieves the complexity associated with merging and editing multiple files. The back end functionality of this option is similar to 'continue 'option and is already explained in 'continue' operation. The user interface implemented for this option is different than "continue"' option. In order to record multiple times in one single file user can select already existing file or can create a new file. Proper user interface is provided to the user to mark any file for quick start option. Once user marks any media file with 'quick start' option then the real time AV editor application remembers the file name always. The real time AV editor application also assigns a single key to launch recorder application any time. Whenever user presses the assigned single key then real time AV editor application (also referred to as application in the below context) launches video recorder module in preview mode. As the application already knows or remembers the file name in which user wants to record, whenever user starts the recording operation the application starts recording data in the user selected file. Using the same key press user can terminate recording operation. So whenever user terminates recording, real time AV editor application releases all resources and exits. Thus, when real time AV editor application is totally switched off ,the user is free to use the device for any other purpose. Thus with the help of single key press user can start and terminate recording in the selected file. As the file name is stored in non-volatile memory it is possible to remember and record the live recording in the same file even if user switches off the device for some time. User can unmark the file if he wishes to record in new file and can mark new file for 'quick start' option. User interface highlights the file name of marked file to indicate that file is marked for 'quick start' option. 'Quick start' option helps user to avoid unwanted recording in the same file and as user can switch off and switch on the application with single key press it is very convenient to use the same device or equipment for other purposes if the device has that kind of capability. Use case: if user is recording a video event using a mobile device having real time AV editor application and if the user finds that he needs to wait for some more time to record one important event, then with single key press he can terminate real time AV editor application. In this way battery life can be saved as real time AV editor application releases all the resources required for recording. Also the user can use that waiting time to use the device for different purposes like making/receiving calls. After the waiting time when the event is ready the user can again start recording with single key press in the same file. Quick pick up point- If a user wants to append already existing file with live video then with the help of this option the user can see last 4 seconds of recorded data and can identify the correct point to start or append with the live data to be recorded in the same file. This helps the user to remember what he had recorded before. Insert - This editing option helps the user to insert or record live data anywhere in the already existing file. User can mark single or multiple points in the already existing file to insert live data. User selects a file for playing. During playback user marks a point using User Interface (Ul) control or pauses playback at desired point from where he wants to insert live data. The real time AV editor application inserts live AV data from this point using 'Insert' option till user selects 'stop insertion. The real time AV editor application allows user to have multiple insertions at various points of the original file. After first insertion of the live data user may resume playing of the original file from the last inserted point and can repeat the insertion of live data anywhere. The real time AV editor Application also provides a facility of specifying duration of inserting live data. After inserting live data of specified duration from selected point, the real time AV editor application automatically stops inserting live data and alerts the user. Overwrite - This editing option helps user to overwrite live data anywhere in the already existing file over existing data. User can mark single or multiple points in the already existing file to overwrite live data. User can select a file and can play the file. During playback user can mark start and end points in the existing file to indicate the duration for which he wants to overwrite live data. The real time AV editor application overwrites live AV data from this point using 'overwrite' option. In this case the original data is replaced with live recorded data. The real time AV editor Application allows user to have multiple overwriting at various points of the original file. After first overwriting of the live data, user may resume playing of the original file from the last overwritten point and can repeat overwriting of live data anywhere. The real time AV editor application also provides a facility of specifying duration of overwriting live data. After overwriting live data of specified duration from selected point, the real time AV editor application automatically stops overwriting live data and alerts the user. If the user still wants to continue recording, then the real time AV editor application stops overwriting at that point and starts inserting live data at that point as explained in 'insert' operation. Re-start - This is an extension of the overwrite feature and uses the overwrite functionality as explained in overwrite operation. User can start recording new or existing file. While recording live data if complete memory is used or exhausted or if memory is not available, then with the help of "restart" option user can continue recording but the application starts overwriting recorded data from the starting of the file using the techniques described in 'overwrite'operation. Window recording - With 'window recording' option user can record unexpected event and can insert or overwrite at a particular point in existing file in real time. User can insert or overwrite last "n" seconds of recorded data at desired point in a file. Using this feature user can select a file and can play the file. During playback user can mark a point or can stop playback at desired point from where he wants to create a window. User also needs to specify duration 'n' in seconds. User needs to specify whether he wants to insert last 'n' seconds of live data or overwrite last "n" seconds at that point using suitable Ul option. Once this is done the real time AV editor application marks a memory region or window in the selected file at the point selected by user and the memory size of this region is sufficient to record live data equal to the duration 'n' specified by the user. The memory region selected by the real time AV editor application depends on 'insert' or 'overwrite' option selected by user. If the option is 'insert' then the real time AV editor application does not overwrite over previous data as explained in 'insert' operation but if the option is 'overwrite' then the application overwrites above the previous 'n' seconds data as explained in 'overwrite' operation. The window or memory region acts as a circular memory as a part of file and user can start recording. Initially live data is stored in the window region of the file and after that at any point of time whenever it reaches the end of the window region it discards the oldest frame and replaces it with the received frame. Because of this technique window region always has last 'n' seconds live data recorded. User can continue recording till the event of user's interest is captured. Whenever user stops recording, the window region will have last "n" seconds recorded data. This helps user to record the event for which time of occurrence is not known but has less duration by efficiently making use of memory space. Easy access: If user records an event with 'append', 'insert' and 'overwrite' options then user can watch the result of all 'append', 'insert' and 'overwrite' operations as thumbnails and can directly seek a particular sub-event for playing. Thus indexing and quick access of AV content is possible. The real time AV editor application maintains a record of each editing option used by user and whenever user selects such a file for playback , the first frame of each such operation is shown as a Joint Photographic Experts Group (JPEG) image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails , the real time AV editor application starts playing directly from that point. There are many standard media formats available like 3gp, mp4, in which the information is stored in two forms i.e. meta data and media data. Real time AV editor application takes care of updating meta data and media data in an optimal way in order to support various editing options in already existing or pre¬recorded file. The only restriction on real time AV editor is that if user wants to record in already existing or pre-recorded file then the real time AV editor application uses same codec types which are used in the existing file. The editing options described here can be applied to any standard format having only audio, only video or audio, video file. The present invention requires the following for implementation • A mobile terminal or any handheld device or Personal Computer (PC) with camera facility, • Software/hardware to encode raw audio video frames • Software/hardware to decode audio video frames • Display device • Microphone, speakers • Media file reader, parser .writer, and merger. Figure 1 depicts a block diagram illustrating the components involved for implementing a real time Audio Video (AV) editor. As shown in the diagram User Interface (Ul) is provided to the user to interact with the real time AV editor application and to select various options. Based on the user input, the user interface module interacts with other blocks to provide the functionality requested by user. User interface provides functionality like selecting media file for the real time AV editing, providing editing options like continue, insert etc, launching player module to select or mark the points for insertion, overwrite etc, providing playback functionality like play, pause, resume, fast forward, reverse rewind, stop playback, showing thumbnails for all edited events, playback from selected thumbnail position, exit application etc. It also provides recording related functionality like launching recorder, camera preview mode, recording mode, pause, resume ,stop and exit recorder functionality. The real time AV editor application needs the following support functionalities. 1. Camera preview (pre-recording mode) - This functionality is provided by camera sensor, camera processing unit and LCD display. The live frames captured by camera are displayed on LCD for preview. 2. Live AV recording that gives audio and video encoded/compressed data in real time - The frames captured by camera are encoded/compressed using video encoder. The video controller controls this operation and provides encoded data on frame by frame basis. The camera frames are also displayed on LCD during recording. 3. Media player for playback of selected media file. - The functionality is provided using media file reader, audio and video decoder, audio video controller and LCD. This provides complete media player functionality. 4. Edit algorithms- This is provided by file reader, file writer and Real Time Audio Video editor (RTAVE) control block. This provides all functionality required for all editing options like 'continue', ' insert', 'overwrite', 'restart', 'window recording' etc. The editing options provided by real time AV editor application can be easily implemented in ISO based media file formats like 3gp, 3g2,mp4 etc. ISO based media file format stores the description of its media separately from the media data. The description, or metadata, is called the movie resource, movie atom, or simply the movie, and contains information such as the number of tracks, the video compression format, and timing information. The movie resource also contains an index describing where all the media data is stored. The media data is the actual sample data, such as video frames and audio samples, used in the movie. The media data may be stored in the same file, in a separate file, in multiple files, in alternate sources such as databases or real-time streams, or in some combination of these. The media data file format supports random storage of media data (a container box which can hold the actual media data for a presentation ('mdat')) irrespective of its sequence. ISO file format restricts the sequential storage only to movie box (A container box whose sub-boxes define the metadata for a presentation ('moov')) or meta data which describes media data. Real time AV editor application makes use of these features of ISO file format to provide editing option in real time like continue, insert, overwrite, re-start etc. The same methodology or concept can also be used in other file formats that has similar kind of arrangements for meta data and media data like quicktime (.mov) files. Figure 2 depicts the 'continue' option according to an embodiment of the present invention. Using the 'continue' option user can record live data in already existing file. To do this suitable user interface is provided to the user to select an already existing media file in which user wants to continue live recording. Suitable user interface is provided to the user to select continue option. After selecting this option real time AV editor launches video recorder module in camera preview mode. From here onwards the real time AV editor application works like a video recording application. User can preview the live camera frames and then the user can start recording any time as per his wish .The user can start and stop recording along with pause, resume functionality while recording. But the live video and audio data is recorded in the file selected by the user whenever recording is in progress. With the "continue " option selected, whenever user starts recording, the real time AV editor application exploits the flexibility provided by ISO media file format and uses following method to record live audio and video data in the user selected file. • As explained earlier each media file stores the description of its media called as metadata separately from the media data. Since metadata of the existing file is stored in moov atom of the file, to keep the existing media content meaningful the moov data is copied and stored. • Now since ISO media file format supports random storage of media data, live data which is captured and encoded/compressed is written to existing file on frame by frame basis during live recording. This media data consist of both audio and video data which is written frame by frame as generated by video recorder module. In order to use file memory effectively the area occupied by old meta data which is already copied is used to write live media data and continued till completion without touching original media data. • The real time AV editor application also stores the metadata information for this newly added content in separate location (not in existing file). • Whenever user stops recording and once all the media data is written to the file then application forms a new moov atom or new metadata information by processing each field and atom using old meta data and meta data of live recording. This newly formed metadata that corresponds to old data and live recorded data is then written to the file at the end. • Therefore when user plays the output file after closing the recording user can see live or newly recorded data in continuation with the old data in the file as shown in figure 2. It is possible to record live video multiple times in the same file in continuation with previously recorded file. Also the media file used for 'continue' operation can be used next time to perform 'continue' operation again or any kind of editing option provided by real time AV editor application. The real time AV editor application has a provision to record the details of each editing operation performed by the user. This helps user to access or seek a particular edited point and start playback directly from that point. The application maintains a record of each editing option used by user and whenever user selects such file for playback, first frame of each such operation is shown as JPEG image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails then the application starts playing directly from that point. For example if user records live data 5 times in already existing file using 'continue' operation then application shows thumbnail or starting/first image of each such portion i.e. 5 images each corresponding to 'continue' operation performed by user in sequence. User can select any one as per his wish and can directly start playing from that point. Figure 3 depicts the 'insert' option according to another embodiment of the present invention. The 'insert' editing option helps user to insert or record live data anywhere in the already existing file. User can mark single or multiple points in the already existing file to insert live data. User can select a file and can play the file. During playback user can mark a point using Ul control or can pause playback at desired point from where he wants to insert live data. The application inserts live AV data from this point using 'insert' option till user selects 'stop insertion'. The application allows the user to have multiple insertions at various points of the original file. After first insertion of the live data user may resume playing of the original file from the last inserted point and can mark a new point or can pause playback at new point for new insertion. The application also provides a facility of specifying duration of inserting live data. After inserting live data of specified duration from selected point, the real time AV editor application automatically stops inserting live data and alerts the user. Suitable Ul is provided to the user to select an already existing media file in which user wants to insert live recording. Suitable Ul is provided to the user to select 'insert' option. After selecting this option real time AV editor launches player module to play the selected file. This is useful to the user to mark the point at which user wants to insert live data during playback. The real time AV editor application then starts playing the file selected by user. During playback user can pause the playback and can choose 'start insert' option. After selecting this option real time AV editor launches video recorder module in camera preview mode. From here onwards the real time AV editor application works like a video recording application. User can preview the live camera frames and can start recording any time as per his wish .He can start and stop recording along with pause, resume functionality while recording. But the live video and audio data is recorded in the file selected by user whenever recording is in progress. With the 'insert' option selected, whenever user starts recording, the real time AV editor application exploits the flexibility provided by ISO media file format and uses following method to record live audio and video data in the user selected file. • Whenever user marks the point for insertion then Real time AV editor application gets the exact timing details of the insertion point. • Then the nearest "I" frame is found by parsing the tables in the track atom corresponding to the time given by user. • This is required as without this the subsequent frames would be meaningless and hence results in blocky video. So insertion needs to be made just before "1" frame in time line. Thus the time instant of "insert" operation depends on time instant of "1" frame in time line. • As explained earlier each media file stores the description of its media called as metadata separately from the media data. Since metadata of the existing file is stored in moov atom of the file, to keep the existing media content meaningful the moov data is copied and stored. • Now since ISO media file format supports random storage of media data, live data which is captured and encoded/compressed is written to existing file on frame by frame basis during live recording. This media data consist of both audio and video data which is written frame by frame as generated by video recorder module. In order to use file memory effectively the area occupied by old meta data which is already copied is used to write live media data and continued till completion without touching or overwriting original media data. • The application also stores the metadata information for this newly added content in separate location (not in existing file). • Whenever user stops recording and once all the media data is written to the file then the real time AV editor application forms a new moov atom or new metadata information by processing each field and atom using old meta data and meta data of live recording. This newly formed metadata that corresponds to old data and live recorded data is then written to the file at the end. Therefore when user plays the edited file after closing the recording, user can see live or newly recorded data very near to marked point (due to nearest "I" frame logic) along with the old data in the file as shown in figure 3. It is possible to record or insert live video multiple times in the same file. Also, the media file used for 'insert' operation can be used next time to perform again 'insert' or any kind of editing option provided by real time AV editor application. The real time AV editor application has a provision to record the details of each editing operation performed by the user. This helps user to access or seek a particular edited point and start playback directly from that point. The application maintains a record of each editing option used by user and whenever user selects such file for playback first frame of each such operation is shown as JPEG image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails then the application starts playing directly from that point. For example if user records live data 5 times in already existing file using 'insert' operation then application shows thumbnail or starting/first image of each such portion i.e. 5 images each corresponding to 'insert' operation performed by user in sequence. User can select any one as per his wish and can directly start playback from that point. Figure 4 depicts the 'overwrite' option according to another embodiment of the present invention. The 'overwrite' editing option helps user to overwrite live data anywhere in the already existing file over existing data. User can mark single or multiple points in the already existing file to overwrite live data. User can select a file and can play the file. During playback user can mark start and end points in the existing file to indicate the duration for which he wants to overwrite live data. The application overwrites live AV data from this point using 'overwrite' option. In this case the original data is replaced with live recorded data. The real time AV editor Application allows user to have multiple overwriting at various points of the original file. After first overwriting of the live data user may resume playing of the original file from the last overwritten point and can repeat the overwriting of live data anywhere. The real time AV editor application also provides a facility of specifying duration of overwriting live data. After overwriting live data of specified duration from selected point, the real time AV editor application automatically stops overwriting live data and alerts the user. If user still wants to continue recording, then the real time AV editor application stops overwriting at that point and starts inserting live data at that point as explained in 'insert' operation. Suitable Ul is provided to the user to select an already existing media file in which user wants to insert live recording. Suitable Ul is provided to the user to select 'overwrite' option. After selecting this option real time AV editor launches player module to play the selected file. This is useful to the user to mark the point at which user wants to overwrite live data during playback. Therefore the application starts playing the file selected by user. User can mark the start and end points during playback and can choose start overwrite option. After selecting this option, the real time AV editor application launches video recorder module in camera preview mode. From here onwards the real time AV editor application works like a video recording application. User can preview the live camera frames and then can start recording any time as per his wish .He can start and stop recording along with pause, resume functionality while recording. But the live video and audio data is recorded in the file selected by user whenever recording is in progress. With the 'overwrite' option whenever user starts recording, the real time AV editor application exploits the flexibility provided by ISO media file format and uses following method to record live audio and video data in the user selected file. 1. First the user selects the 2 points (start and end point) within which data is to be replaced. 2. Then T frames corresponding to each point is found out. The method remains similar as used in 'insert' operation. 3. As explained earlier each media file stores the description of its media called as metadata separately from the media data. Since metadata of the existing file is stored in moov atom of the file, to keep the existing media content meaningful the moov data is copied and stored. 4. The exact location (file offset) of each audio and video frame that are to be replaced or to be overwritten is found out within the file using the sample tables. These parts of the file can be replaced/overwritten. 5. Using the information in step 4, big or continuous memory blocks are formed by combining adjacent frames in memory. The formation of blocks depends on how the audio, video samples or frames are stored. The purpose of this process is to find the perfect memory place in file to replace existing data with live data in optimal or efficient way. This helps to fit the live or incoming data into the memory blocks in optimal way. The combining of adjoining blocks is done to utilize available space in efficient and optimal way for replacement. This method keeps a record of all continuous or non-continuous memory blocks and uses these blocks to write live data in optimal way so that the available memory can be utilized to the full extent. If the sufficient memory is not available in the block then it writes the new data at the end of the file. 6. The real time AV editor application also stores the metadata information for this newly added content in separate location (not in existing file). 7. Whenever user stops recording and once all the media data is written to the file then application forms a new moov atom or new metadata information by processing each field and atom using old meta data and meta data of live recording. This newly formed metadata that corresponds to old data and live recorded data is then written to the file at the end. Therefore when user plays the edited file after closing the recording user can see live or newly recorded data very near to marked point (due to nearest "1" frame logic) in place of old data in the file as shown in figure 4. It is possible to record or overwrite live video multiple times in the same file. Also the media file used for 'overwrite' operation can be used next time to perform 'overwrite' operation again or any kind of editing option provided by real time AV editor application. The real time AV editor application has a provision to record the details of each editing operation performed by the user. This helps user to access or seek a particular edited point and start playback directly from that point. The application maintains a record of each editing option used by user and whenever user selects such file for playback, first frame of each such operation is shown as JPEG image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails then the application starts playing directly from that point. For example if user records live data 5 times in already existing file using 'overwrite'operation then the application shows thumbnail or starting/first image of each such portion i.e. 5 images each corresponding to 'overwrite' operation performed by user in sequence. User can select any one as per his wish and can directly start playback from that point. All editing options provided by the real time AV editor application to record live video in already existing file can also be used for non-real time situations. The method has a provision to get or extract complete or part of the portion from other media file (audio, video) instead of getting data from live sources. User can record in existing file using all editing options just like recording live data if the input source is some media file. According to an embodiment of the present invention the recording device can be any embedded device with camera facility which include but not limited to a mobile phone, smart phone, handheld devices with camera etc. Accordingly the present invention has the following advantages • The append option is very useful to record longer duration event as per user wish without waiting for the event to happen. It also helps user to organize his videos in more systematic way by appending various themes to various files. • The 'prepend' option allows the user to record live data at the beginning of the file and thus provides easy way to change or alter the sequence of events thereby saving memory and battery life of the recording device. • The 'quick start' option helps in recording live data multiple times in one single file with one key press. It helps user to avoid unwanted recording in the same file. As user can switch off and switch on the real time AV editor application with single key press it is very easy to use the same device or equipment for other purposes if required and with single key press user can continue recording in the same file when user has done with other application. • The editing option 'insert' helps user to insert or record live data anywhere in the already existing file. User can mark single or multiple points in the already existing file to insert live data in real time. 'Overwrite' option helps to select a portion of recorded event and overwrite it with live video. If user is not happy with the captured /recorded event the user can repeat the event and can overwrite to previously recorded event With the 'overwrite' option the user can make use of available memory in optimal way by overwriting live video on unwanted portion and can alter the sequence of recording by inserting new data. The 're-start' option helps to continue recording from beginning of a file when the user device is running short of memory which overwrites the live data available in the file. User can use this option to record important data by continuing recording in the same file by loosing less important data. The present invention allows a user to record an unexpected event in already existing file by providing a circular buffering scheme as a part of file wherein user can record live data. The oldest data in the circular buffer is replaced by new data till user presses stop. User can select the insertion point and duration of buffering before starting recording. This helps user to merge unexpected event recording in already recorded file with efficient use of memory The present invention allows user to continue recording in the same file. Hence user can close the recorder whenever needed and can start recording at any time in the same file with hot key. This saves the battery life of the recorder. • The present invention avoids merging of multiple files to form a single file thereby saving time. • With the present invention, it is possible to correct actions by repeating events and then re-recording in the same file. • The present invention provides quick access and indexing method for recorded video. If user records an event with 'append', 'insert' and 'overwrite' options then user can watch result of all 'append', 'insert' and 'overwrite' operations as thumbnails and can directly seek a particular sub-event for playing. Thus indexing and quick access of AV content is possible • With the editing options provided by the proposed invention for real time recording, it is possible to create interesting videos using one single application Although the present invention has been fully described in connection with the preferred embodiments thereof with reference to the accompanying drawings, it is to be noted that various changes and modifications are possible and are apparent to those skilled in the art. Such changes and modifications are to be understood as included within the scope of the present invention as defined by the appended claims unless they depart there from. We Claim: 1. A system for real time AV editing comprising, a camera pre-recording preview module for previewing live frames captured by a camera ; live AV recording module for recording audio and video encoded /compressed data in real time; media player for playback of selected recorded audio and video encoded data; and editing module for editing the encoded audio and video data. 2. The system according to claim 1, wherein the editing module is adapted to append, insert and overwrite audio and video data in an existing file. 3. The system according to claim 2, wherein the editing module is adapted to append, insert and overwrite audio and video data single or multiple times in an existing files. 4. The system according to claim 2, wherein the editing module is adapted to insert or overwrite data at locations for specified duration of time in the already existing file by marking single or multiple points in the encoded audio and video data. 5. The system according to claim 1, wherein the editing module comprises of a 'continue' option to record live data in the existing file. 6. The system according to claim 1, wherein the editing module comprises of a 'pre-pend' option to add live media content to the beginning of the existing file. 7. The system according to claim 1, wherein the editing module comprises of a 'quick start' option to record live data multiple times in one single file with one key press. 8. The system according to claim 1, wherein the editing module comprises of a 'quick pick up point' option to see last 'n' seconds of recorded data and to identify a correct point to start or append the live data to be recorded in the same file. 9. The system according to claim 1, wherein the editing module comprises a re¬start module to continue overwriting recorded data from the starting of the file. 10. The system according to claim 1 wherein, the editing module comprises 'window recording' option to insert or overwrite last 'n' seconds of recorded data at desired point in a file. 11. The system according to claim 1, wherein the editing module comprises a means to preview edited data. 12. A method for real time AV editing comprising the steps of, previewing live frames captured by a camera using a camera pre-recording preview module ; recording audio and video encoded /compressed data in real time using live AV recording module; playing selected recorded audio and video encoded data using media player; and editing the encoded audio and video data using editing module. 13. The method according to claim 12, wherein insertion comprises the step of: getting exact timing details of an insertion point by the editing module when a user marks the insertion point; finding a nearest "I" frame by parsing the tables in a track atom corresponding to a time given by the user; copying and storing moov data which contain the metadata of the existing file; writing the live data captured and encoded/compressed to existing file on frame by frame basis during live recording where area occupied by old meta data which is already copied is used to write live media data and continued till completion without touching or overwriting original media data; storing the metadata information for the newly added content in separate location; and forming a new moov atom or new metadata information by processing each field and atom using old meta data and meta data of live recording whenever user stops recording. 14. The method according to claim 13,wherein overwriting comprises the step of: selecting the start and end point by the user within which data is to be replaced; finding the T frames corresponding to each point; copying and storing moov data which contain the metadata of the existing file; finding the exact location of audio and video frame that are to be replaced or to be overwritten within the file using the sample tables; writing new data in an optimal way in memory or at the end of the file; storing the meta data information of the newly added content in separate location; writing the newly formed meta data that corresponds to old data and live recorded data to the file at the end whenever user stops recording. 15. A system for real time AV editing substantially described particularly with reference to the accompanying drawings. 16. A method for real time AV editing substantially described particularly with reference to the accompanying drawings.

Full Text

FIELD OF THE INVENTION
The present invention relates to the field of embedded systems and specifically to the field of embedded devices with camera facility. More particularly, the present invention relates to a system and method of real time Audio Video (AV) editing.
DESCRIPTION OF THE RELATED ART
In the existing technology, digital recorders available in embedded devices record live Audio Video (AV) in various media file formats. When a session is recorded a new file gets created with a file name automatically generated by the firmware. These automatically generated file names (for example Recorded 34.3gp, Recorded 43.3 gp etc.) create an annoying situation for the user. Further, with the existing technology in digital recorders, it is not possible to record a new session in an already existing or pre-recorded file.
LIMITATIONS
1. Existing recorders does not provide an option to insert a new session of live AV into an already existing or pre-recorded file at a particular point selected by the user. In case if the user wants to record live video after a specific event in an already recorded video session or if the user have missed out some important events while recording, then the user has to re-record and insert the particular event at the specific points in the
recorded video. In the above cases, the user records each session in different files and then with the help of different hardware and software the files are merged offline. But this results in wastage of time.
2. Existing recorders do not provide any option for changing the order of scenes or order of events in an already recorded video session that can make the already recorded video more meaningful. If a user wants to record some event that is expected to happen and incase the user misses the same, the user records the next event instead. Existing technology does not provide an option to insert the particular expected event at a particular point in the already recorded video, when the expected event is ready to record.
3. Existing recorders do not provide any option to overwrite live AV (new session) in already existing or pre-recorded file at a particular point selected by the user. Hence if the user is not interested with the quality of recording for a specific reason in a pre-recorded file or if some portion of the event is not captured properly or if some portion of the file is useless or less important or if the recorder is having memory constraints the user cannot overwrite a new session of live audio video into an already existing or pre-recorded file at a particular point selected by the user.
4. Existing recorders stop the process of recording when there is no sufficient memory in the recording system. This may result in missing important data due to memory non-availability.
5. Existing recorders do not provide any mechanism to record an unexpected event in an already existing or-pre-recorded file.
6. Existing recorders provide the options of 'pause' and 'resume' for reinitiating an ongoing recording session. But as the recorder application remains active during the 'pause' period hence the battery consumption is more.
7. With the existing technology in recorders, if a user wants to record certain events in the same file then it requires recording of unwanted events also to continue recording in the same file. This results in wastage of useful memory space.
8. Existing recorders create a new file for each recording session. If this happens for a number of times then keeping track of proper sequence and renaming files with proper file names becomes difficult. Also if the user switches off the recorder application or switches off the device then he cannot record live video next time in the same file.
9. Existing recorders do not allow a user to close or suspend or switch off (or shut down the device) the recorder application so as to use the device for some other purposes like making/receiving a call and to resume recording again in the same file.
10. Existing technology in recorders does not support real-time editing options and hence requires merging of multiple files using different hardware and software to create a single file. This results in more time consumption.
11. With the existing methodology in recorders it is not possible to correct the action recorded by repeating the events and then re-recording in the same file.
12. Existing recorders do not support a quick access and indexing for recorded video.
13. With the help of existing recorders it is not possible to create interesting video by combining existing video and live video as per user wish.
Hence there is a need to help the user in situations like recording device running short of memory, battery low condition of the recording device, merging different parts of recorded events in the same file, removing unwanted portion in the recorded video, inserting live video at a desired point etc.
According to a technique disclosed in a patent application, a Digital Video Recorder (DVR) device records television content from a particular channel in a temporary buffer. Once the temporary buffer is full, the DVR device starts overwriting the temporary buffer from the beginning, replacing the earlier recorded video content. The application however, does not mention the feature of inserting or appending video content in a pre-recorded video file.
According to another technique, a video recording apparatus gives an option to a user to choose the area on the memory where he wants to record the new video content. In one of the embodiments, the user can select pre-recorded video files in the memory to be overwritten by the new video content to be recorded. The new video, thus, gets recorded on these selected files. The patent however, does not explicitly mention inserting video within a pre-recorded video file or appending video content at the end of the pre-recorded video file. The present invention has been designed to address these needs.
SUMMARY OF THE INVENTION
It is therefore a primary object of the present invention to provide a real time Audio Video (AV) editor in an embedded device with camera facility.
It is another object of the present invention to provide the user with an option for recording live data in an already existing pre-recorded file.
It is another object of the present invention to provide an editing option that helps user to insert or record live data anywhere in the already existing file.
It is yet another object of the present invention to provide an option to record live data multiple times in one single file with one key press in the video recording device.
It Is yet another object of the present Invention to provide an editing option that helps user to overwrite live data anywhere in the already existing file.
It is yet another object of the present invention to provide a feature in an embedded device with camera facility to record an unexpected event and to insert or overwrite the particular event at a selected point in an existing file in real time.
The present invention relates to the field of video recording. In particular, the invention describes a video recording application with new editing options. The video recording application provides real time editing options such as append, insert, overwrite, etc. The insert option provides the facility of inserting a new video in a pre-recorded video file whereas the append option allows to add a live video at the end of an already existing video file. The overwrite option provides better memory management and permits to record a video on a pre-existing video file if the user device is running short of memory.
For effective memory management, there are other features provided as well, such as re-start and window recording. While recording a file, if the recording device runs out of memory, the user can use the re-start option to start recording from the beginning of the file, such that the new recording overwrites the already recorded data. The window recording option on the other hand has a circular buffer where the latest data overwrites the data at the beginning of the buffer. For example, if the user defines the duration as 20 seconds, then the circular buffer stores the last 20 seconds of recording. The invention also helps in effective battery management. Accordingly, the user can switch off the recorder when not in use and start recording from the desired place using the available options when the recorder is switched on again.
Accordingly, the invention explains a system for real time AV editing comprising,
a camera pre-recording preview module for previewing live frames captured by a camera ;
live AV recording module for recording audio and video encoded /compressed data in real time;
media player for playback of selected recorded audio and video encoded data; and
editing module for editing the encoded audio and video data.
Accordingly, the invention explains a method for real time AV editing comprising the steps of,
previewing live frames captured by a camera using a camera pre-recording preview module ;
recording audio and video encoded /compressed data in real time using live AV recording module;
playing selected recorded audio and video encoded data using media player; and
editing the encoded audio and video data using editing module.
These and other objects, features and advantages of the present invention will become more apparent from the ensuing detailed description of the present invention taken in conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE ACCOMPANYING FIGURES
Figure 1 depicts a block diagram illustrating the components involved for implementing a real time Audio Video (AV) editor;
Figure 2 depicts the 'continue' option according to an embodiment of the present invention;
Figure 3 depicts the 'insert' option according to another embodiment of the present invention; and
Figure 4 depicts the 'overwrite' option according to another embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
The preferred embodiments of the present invention will now be explained with reference to the accompanying drawings. It should be understood however that the disclosed embodiments are merely exemplary of the invention, which may be embodied in various forms. The following description is not to be construed as limiting: the invention and numerous specific details are described to provide a thorough understanding of the present invention, as the basis for the claims and as a basis for teaching one skilled in the art how to make and/or use the invention. However, in certain instances, well-known or conventional details are not described in order not to unnecessarily obscure the present invention in detail.
The present invention proposes a new use case for Video Recording application that provides editing facilities in real time while recording live audio video (AV). Accordingly, the present invention provides a method for efficiently utilizing the existing AV files further providing easy and quick access to the AV content, which the user has appended, added or overwritten. The present invention basically provides new options to the user for recording new video in already existing or pre-recorded file. The invention provides an option to record new session in already existing or pre-recorded file and therefore user is relieved from creating multiple files, remembering relevance and then merging the multiple files using multiple devices and different applications or softwares. User has a better control to organize his recorded files based on subject or theme of recording and based on the memory space in the recording device, battery life and situational requirements.
The invention provides an option to insert live video session in already existing or pre-recorded file. This is a great advantage to the user to alter the sequence of events in video and create interesting video.
Accordingly, the new options provided by the present invention are explained below.
Continue- Using this option user can record live data in already existing file. When user wants to record an event, he can choose any existing or pre¬recorded file to continue recording in the same file using 'continue' option. The application appends the file with live recorded data. The facility of appending pre-recorded file is extended further with 'quick Start' option in an attractive way to help user to save battery life and to avoid multiple creation of the files.
Pre-pend- This is similar to the "'continue' feature and uses the "'continue' functionality as explained above to add live media content to an existing file at the beginning.
Quick start- This option is useful to record live data multiple times in one single file with one key press. It helps user to save battery life and to avoid creation of multiple files of the same event and also relieves the complexity associated with merging and editing multiple files.
The back end functionality of this option is similar to 'continue 'option and is already explained in 'continue' operation. The user interface implemented for this option is different than "continue"' option. In order to record multiple times in one single file user can select already existing file or can create a new file. Proper user interface is provided to the user to mark any file for quick start option.
Once user marks any media file with 'quick start' option then the real time AV editor application remembers the file name always. The real time AV editor application also assigns a single key to launch recorder application any time. Whenever user presses the assigned single key then real time AV editor application (also referred to as application in the below context) launches video recorder module in preview mode. As the application already knows or remembers the file name in which user wants to record, whenever user starts the recording operation the application starts recording data in the user selected file. Using the same key press user can terminate recording operation. So whenever user terminates recording, real time AV editor application releases all resources and exits. Thus, when real time AV editor application is totally switched off ,the user is free to use the device for any other purpose. Thus with the help of single key press user can start and terminate recording in the selected file. As the file name is stored in non-volatile memory it is possible to remember and record the live recording in the same file even if user switches off the device for some time. User can unmark the file if he wishes to record in new file and can mark new file for 'quick start' option. User interface highlights the file name of marked file to indicate that file is marked for 'quick start' option.
'Quick start' option helps user to avoid unwanted recording in the same file and as user can switch off and switch on the application with single key press it is very convenient to use the same device or equipment for other purposes if the device has that kind of capability.
Use case: if user is recording a video event using a mobile device having real time AV editor application and if the user finds that he needs to wait for some more time to record one important event, then with single key press he can terminate real time AV editor application. In this way battery life can be saved as real time AV editor application releases all the resources required for recording. Also the user can use that waiting time to use the device for different purposes like making/receiving calls. After the waiting time when the event is ready the user can again start recording with single key press in the same file.
Quick pick up point- If a user wants to append already existing file with live video then with the help of this option the user can see last 4 seconds of recorded data and can identify the correct point to start or append with the live data to be recorded in the same file. This helps the user to remember what he had recorded before.
Insert - This editing option helps the user to insert or record live data anywhere in the already existing file. User can mark single or multiple points in the already existing file to insert live data.
User selects a file for playing. During playback user marks a point using User Interface (Ul) control or pauses playback at desired point from where he wants to insert live data. The real time AV editor application inserts live AV data from this point using 'Insert' option till user selects 'stop insertion.
The real time AV editor application allows user to have multiple insertions at various points of the original file. After first insertion of the live data user may resume playing of the original file from the last inserted point and can repeat the insertion of live data anywhere.
The real time AV editor Application also provides a facility of specifying duration of inserting live data. After inserting live data of specified duration from selected point, the real time AV editor application automatically stops inserting live data and alerts the user.
Overwrite - This editing option helps user to overwrite live data anywhere in the already existing file over existing data. User can mark single or multiple points in the already existing file to overwrite live data. User can select a file and can play the file. During playback user can mark start and end points in the existing file to indicate the duration for which he wants to overwrite live data. The real time AV editor application overwrites live AV data from this point using 'overwrite' option. In this case the original data is replaced with live recorded data. The real time AV editor Application allows user to have multiple overwriting at various points of the original file. After first overwriting of the live data, user may resume playing of the original file from the last overwritten point and can repeat overwriting of live data anywhere. The real time AV editor application also provides a facility of specifying duration of overwriting live data. After overwriting live data of specified duration from selected point, the real time AV editor application automatically stops overwriting live data and alerts the user.
If the user still wants to continue recording, then the real time AV editor application stops overwriting at that point and starts inserting live data at that point as explained in 'insert' operation.
Re-start - This is an extension of the overwrite feature and uses the overwrite functionality as explained in overwrite operation. User can start recording new or existing file. While recording live data if complete memory is used or exhausted or if memory is not available, then with the help of "restart" option user can continue recording but the application starts overwriting recorded data from the starting of the file using the techniques described in 'overwrite'operation.
Window recording - With 'window recording' option user can record unexpected event and can insert or overwrite at a particular point in existing file in real time. User can insert or overwrite last "n" seconds of recorded data at desired point in a file. Using this feature user can select a file and can play the file. During playback user can mark a point or can stop playback at desired point from where he wants to create a window. User also needs to specify duration 'n' in seconds. User needs to specify whether he wants to insert last 'n' seconds of live data or overwrite last "n" seconds at that point using suitable Ul option.
Once this is done the real time AV editor application marks a memory region or window in the selected file at the point selected by user and the memory size of this region is sufficient to record live data equal to the duration 'n' specified by the user. The memory region selected by the real time AV editor application depends on 'insert' or 'overwrite' option selected by user. If the option is 'insert' then the real time AV editor application does not overwrite over previous data as explained in 'insert' operation but if the option is 'overwrite' then the application overwrites above the previous 'n' seconds data as explained in 'overwrite' operation. The window or memory region acts as a circular memory as a part of file and user can start recording. Initially live data is stored in the window region of the file and after that at any point of time whenever it reaches the end of the window region it discards the oldest frame and replaces it with the received frame. Because of this technique window region always has last 'n' seconds live data recorded. User can continue recording till the event of user's interest is captured. Whenever user stops recording, the window region will have last "n" seconds recorded data. This helps user to record the event for which time of occurrence is not known but has less duration by efficiently making use of memory space.
Easy access: If user records an event with 'append', 'insert' and 'overwrite' options then user can watch the result of all 'append', 'insert' and 'overwrite' operations as thumbnails and can directly seek a particular sub-event for playing. Thus indexing and quick access of AV content is possible. The real time AV editor application maintains a record of each editing option used by user and whenever user selects such a file for playback , the first frame of each such operation is shown as a Joint Photographic Experts Group (JPEG) image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails , the real time AV editor application starts playing directly from that point.
There are many standard media formats available like 3gp, mp4, in which the information is stored in two forms i.e. meta data and media data. Real time AV editor application takes care of updating meta data and media data in an optimal way in order to support various editing options in already existing or pre¬recorded file.
The only restriction on real time AV editor is that if user wants to record in already existing or pre-recorded file then the real time AV editor application uses same codec types which are used in the existing file. The editing options described here can be applied to any standard format having only audio, only video or audio, video file.
The present invention requires the following for implementation
• A mobile terminal or any handheld device or Personal Computer (PC) with camera facility,
• Software/hardware to encode raw audio video frames
• Software/hardware to decode audio video frames
• Display device
• Microphone, speakers
• Media file reader, parser .writer, and merger.
Figure 1 depicts a block diagram illustrating the components involved for implementing a real time Audio Video (AV) editor. As shown in the diagram User Interface (Ul) is provided to the user to interact with the real time AV editor
application and to select various options. Based on the user input, the user interface module interacts with other blocks to provide the functionality requested by user. User interface provides functionality like selecting media file for the real time AV editing, providing editing options like continue, insert etc, launching player module to select or mark the points for insertion, overwrite etc, providing playback functionality like play, pause, resume, fast forward, reverse rewind, stop playback, showing thumbnails for all edited events, playback from selected thumbnail position, exit application etc. It also provides recording related functionality like launching recorder, camera preview mode, recording mode, pause, resume ,stop and exit recorder functionality.
The real time AV editor application needs the following support functionalities.
1. Camera preview (pre-recording mode) - This functionality is provided by camera sensor, camera processing unit and LCD display. The live frames captured by camera are displayed on LCD for preview.
2. Live AV recording that gives audio and video encoded/compressed data in real time - The frames captured by camera are encoded/compressed using video encoder. The video controller controls this operation and provides encoded data on frame by frame basis. The camera frames are also displayed on LCD during recording.
3. Media player for playback of selected media file. - The functionality is provided using media file reader, audio and video decoder, audio video controller and LCD. This provides complete media player functionality.
4. Edit algorithms- This is provided by file reader, file writer and Real Time
Audio Video editor (RTAVE) control block. This provides all functionality required for all editing options like 'continue', ' insert', 'overwrite', 'restart', 'window recording' etc.
The editing options provided by real time AV editor application can be easily implemented in ISO based media file formats like 3gp, 3g2,mp4 etc.
ISO based media file format stores the description of its media separately from the media data. The description, or metadata, is called the movie resource, movie atom, or simply the movie, and contains information such as the number of tracks, the video compression format, and timing information. The movie resource also contains an index describing where all the media data is stored. The media data is the actual sample data, such as video frames and audio samples, used in the movie. The media data may be stored in the same file, in a separate file, in multiple files, in alternate sources such as databases or real-time streams, or in some combination of these.
The media data file format supports random storage of media data (a container box which can hold the actual media data for a presentation ('mdat')) irrespective of its sequence. ISO file format restricts the sequential storage only to movie box (A container box whose sub-boxes define the metadata for a presentation ('moov')) or meta data which describes media data. Real time AV editor application makes use of these features of ISO file format to provide editing option in real time like continue, insert, overwrite, re-start etc. The same methodology or concept can also be used in other file formats that has similar kind of arrangements for meta data and media data like quicktime (.mov) files.
Figure 2 depicts the 'continue' option according to an embodiment of the present invention. Using the 'continue' option user can record live data in already existing file. To do this suitable user interface is provided to the user to select an already existing media file in which user wants to continue live recording. Suitable user interface is provided to the user to select continue option. After selecting this option real time AV editor launches video recorder module in camera preview mode. From here onwards the real time AV editor application works like a video recording application. User can preview the live camera frames and then the user can start recording any time as per his wish .The user can start and stop recording along with pause, resume functionality while recording. But the live video and audio data is recorded in the file selected by the user whenever recording is in progress.
With the "continue " option selected, whenever user starts recording, the real time AV editor application exploits the flexibility provided by ISO media file format and uses following method to record live audio and video data in the user selected file.
• As explained earlier each media file stores the description of its media called as metadata separately from the media data. Since metadata of the existing file is stored in moov atom of the file, to keep the existing media content meaningful the moov data is copied and stored.
• Now since ISO media file format supports random storage of media data, live data which is captured and encoded/compressed is written to existing file on frame by frame basis during live recording. This media data consist of both audio and video data which is written frame by frame as generated by video recorder module. In order to use file memory effectively the area occupied by old meta data which is already copied is used to write live media data and continued till completion without touching original media data.
• The real time AV editor application also stores the metadata information for this newly added content in separate location (not in existing file).
• Whenever user stops recording and once all the media data is written to the file then application forms a new moov atom or new metadata information by processing each field and atom using old meta data and meta data of live recording. This newly formed metadata that corresponds to old data and live recorded data is then written to the file at the end.
• Therefore when user plays the output file after closing the recording user can see live or newly recorded data in continuation with the old data in the file as shown in figure 2. It is possible to record live video multiple times in the same file in continuation with previously recorded file. Also the media file used for 'continue' operation can be used next time to perform 'continue' operation again or any kind of editing option provided by real time AV editor application. The real time AV editor application has a provision to record the details of each editing operation performed
by the user. This helps user to access or seek a particular edited point and start playback directly from that point. The application maintains a record of each editing option used by user and whenever user selects such file for playback, first frame of each such operation is shown as JPEG image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails then the application starts playing directly from that point. For example if user records live data 5 times in already existing file using 'continue' operation then application shows thumbnail or starting/first image of each such portion i.e. 5 images each corresponding to 'continue' operation performed by user in sequence. User can select any one as per his wish and can directly start playing from that point.
Figure 3 depicts the 'insert' option according to another embodiment of the present invention. The 'insert' editing option helps user to insert or record live data anywhere in the already existing file. User can mark single or multiple points in the already existing file to insert live data. User can select a file and can play the file. During playback user can mark a point using Ul control or can pause playback at desired point from where he wants to insert live data. The application inserts live AV data from this point using 'insert' option till user selects 'stop insertion'.
The application allows the user to have multiple insertions at various points of the original file. After first insertion of the live data user may resume playing of the original file from the last inserted point and can mark a new point or can pause playback at new point for new insertion.
The application also provides a facility of specifying duration of inserting live data. After inserting live data of specified duration from selected point, the real time AV editor application automatically stops inserting live data and alerts the user.
Suitable Ul is provided to the user to select an already existing media file in which user wants to insert live recording. Suitable Ul is provided to the user to select 'insert' option. After selecting this option real time AV editor launches player module to play the selected file. This is useful to the user to mark the point at which user wants to insert live data during playback. The real time AV editor application then starts playing the file selected by user. During playback user can pause the playback and can choose 'start insert' option. After selecting this option real time AV editor launches video recorder module in camera preview mode. From here onwards the real time AV editor application works like a video recording application. User can preview the live camera frames and can start recording any time as per his wish .He can start and stop recording along with pause, resume functionality while recording. But the live video and audio data is recorded in the file selected by user whenever recording is in progress. With the 'insert' option selected, whenever user starts recording, the real time AV editor application exploits the flexibility provided by ISO media file format and uses following method to record live audio and video data in the user selected file.
• Whenever user marks the point for insertion then Real time AV editor application gets the exact timing details of the insertion point.
• Then the nearest "I" frame is found by parsing the tables in the track atom corresponding to the time given by user.
• This is required as without this the subsequent frames would be meaningless and hence results in blocky video. So insertion needs to be made just before "1" frame in time line. Thus the time instant of "insert" operation depends on time instant of "1" frame in time line.
• As explained earlier each media file stores the description of its media called as metadata separately from the media data. Since metadata of the existing file is stored in moov atom of the file, to keep the existing media content meaningful the moov data is copied and stored.
• Now since ISO media file format supports random storage of media data, live data which is captured and encoded/compressed is written to existing file on frame by frame basis during live recording. This media data consist of both audio and video data which is written frame by frame as generated by video recorder module. In order to use file memory effectively the area occupied by old meta data which is already copied is used to write live media data and continued till completion without touching or overwriting original media data.
• The application also stores the metadata information for this newly added content in separate location (not in existing file).
• Whenever user stops recording and once all the media data is written to the file then the real time AV editor application forms a new moov atom
or new metadata information by processing each field and atom using old meta data and meta data of live recording. This newly formed metadata that corresponds to old data and live recorded data is then written to the file at the end.
Therefore when user plays the edited file after closing the recording, user can see live or newly recorded data very near to marked point (due to nearest "I" frame logic) along with the old data in the file as shown in figure 3. It is possible to record or insert live video multiple times in the same file. Also, the media file used for 'insert' operation can be used next time to perform again 'insert' or any kind of editing option provided by real time AV editor application. The real time AV editor application has a provision to record the details of each editing operation performed by the user. This helps user to access or seek a particular edited point and start playback directly from that point. The application maintains a record of each editing option used by user and whenever user selects such file for playback first frame of each such operation is shown as JPEG image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails then the application starts playing directly from that point. For example if user records live data 5 times in already existing file using 'insert' operation then application shows thumbnail or starting/first image of each such portion i.e. 5 images each corresponding to 'insert' operation performed by user in sequence. User can select any one as per his wish and can directly start playback from that point.
Figure 4 depicts the 'overwrite' option according to another embodiment of the present invention. The 'overwrite' editing option helps user to overwrite live data anywhere in the already existing file over existing data. User can mark single or multiple points in the already existing file to overwrite live data.
User can select a file and can play the file. During playback user can mark start and end points in the existing file to indicate the duration for which he wants to overwrite live data. The application overwrites live AV data from this point using 'overwrite' option. In this case the original data is replaced with live recorded data. The real time AV editor Application allows user to have multiple overwriting at various points of the original file. After first overwriting of the live data user may resume playing of the original file from the last overwritten point and can repeat the overwriting of live data anywhere. The real time AV editor application also provides a facility of specifying duration of overwriting live data. After overwriting live data of specified duration from selected point, the real time AV editor application automatically stops overwriting live data and alerts the user.
If user still wants to continue recording, then the real time AV editor application stops overwriting at that point and starts inserting live data at that point as explained in 'insert' operation.
Suitable Ul is provided to the user to select an already existing media file in which user wants to insert live recording. Suitable Ul is provided to the user to select 'overwrite' option. After selecting this option real time AV editor launches player module to play the selected file. This is useful to the user to mark the point at which user wants to overwrite live data during playback. Therefore the application starts playing the file selected by user. User can mark the start and end points during playback and can choose start overwrite option. After selecting this option, the real time AV editor application launches video recorder module in camera preview mode. From here onwards the real time AV editor application works like a video recording application. User can preview the live camera frames and then can start recording any time as per his wish .He can start and stop recording along with pause, resume functionality while recording. But the live video and audio data is recorded in the file selected by user whenever recording is in progress.
With the 'overwrite' option whenever user starts recording, the real time AV editor application exploits the flexibility provided by ISO media file format and uses following method to record live audio and video data in the user selected file.
1. First the user selects the 2 points (start and end point) within which data is to be replaced.
2. Then T frames corresponding to each point is found out. The method remains similar as used in 'insert' operation.
3. As explained earlier each media file stores the description of its media called as metadata separately from the media data. Since metadata of the existing file is stored in moov atom of the file, to keep the existing media content meaningful the moov data is copied and stored.
4. The exact location (file offset) of each audio and video frame that are to be replaced or to be overwritten is found out within the file using the sample tables. These parts of the file can be replaced/overwritten.
5. Using the information in step 4, big or continuous memory blocks are formed by combining adjacent frames in memory. The formation of blocks depends on how the audio, video samples or frames are stored. The purpose of this process is to find the perfect memory place in file to replace existing data with live data in optimal or efficient way. This helps to fit the live or incoming data into the memory blocks in optimal way. The combining of adjoining blocks is done to utilize available space in efficient and optimal way for replacement. This method keeps a record of all continuous or non-continuous memory blocks and uses these blocks to write live data in optimal way so that the available memory can be utilized to the full extent. If the sufficient memory is not available in the block then it writes the new data at the end of the file.
6. The real time AV editor application also stores the metadata information for this newly added content in separate location (not in existing file).
7. Whenever user stops recording and once all the media data is written to the file then application forms a new moov atom or new metadata information by processing each field and atom using old meta data and meta data of live recording. This newly formed metadata that corresponds to old data and live recorded data is then written to the file at the end.
Therefore when user plays the edited file after closing the recording user can see live or newly recorded data very near to marked point (due to nearest "1" frame logic) in place of old data in the file as shown in figure 4. It is possible to record or overwrite live video multiple times in the same file. Also the media file used for 'overwrite' operation can be used next time to perform 'overwrite' operation again or any kind of editing option provided by real time AV editor application. The real time AV editor application has a provision to record the details of each editing operation performed by the user. This helps user to access or seek a particular edited point and start playback directly from that point. The application maintains a record of each editing option used by user and whenever user selects such file for playback, first frame of each such operation is shown as JPEG image that helps user to remember what was inserted last time and if user selects anyone of these thumbnails then the application starts playing directly from that point. For example if user records live data 5 times in already existing file using 'overwrite'operation then the application shows thumbnail or starting/first image of each such portion i.e. 5 images each corresponding to 'overwrite' operation performed by user in sequence. User can select any one as per his wish and can directly start playback from that point.
All editing options provided by the real time AV editor application to record live video in already existing file can also be used for non-real time situations. The method has a provision to get or extract complete or part of the portion from other media file (audio, video) instead of getting data from live sources. User can record in existing file using all editing options just like recording live data if the input source is some media file.
According to an embodiment of the present invention the recording device can be any embedded device with camera facility which include but not limited to a mobile phone, smart phone, handheld devices with camera etc.
Accordingly the present invention has the following advantages
• The append option is very useful to record longer duration event as per user wish without waiting for the event to happen. It also helps user to organize his videos in more systematic way by appending various themes to various files.
• The 'prepend' option allows the user to record live data at the beginning of the file and thus provides easy way to change or alter the sequence of events thereby saving memory and battery life of the recording device.
• The 'quick start' option helps in recording live data multiple times in one single file with one key press. It helps user to avoid unwanted recording in the same file. As user can switch off and switch on the real time AV editor application with single key press it is very easy to use the same device or equipment for other purposes if required and with single key press user can continue recording in the same file when user has done with other application.
• The editing option 'insert' helps user to insert or record live data anywhere in the already existing file. User can mark single or multiple points in the already existing file to insert live data in real time.
'Overwrite' option helps to select a portion of recorded event and overwrite it with live video. If user is not happy with the captured /recorded event the user can repeat the event and can overwrite to previously recorded event
With the 'overwrite' option the user can make use of available memory in optimal way by overwriting live video on unwanted portion and can alter the sequence of recording by inserting new data.
The 're-start' option helps to continue recording from beginning of a file when the user device is running short of memory which overwrites the live data available in the file. User can use this option to record important data by continuing recording in the same file by loosing less important data.
The present invention allows a user to record an unexpected event in already existing file by providing a circular buffering scheme as a part of file wherein user can record live data. The oldest data in the circular buffer is replaced by new data till user presses stop. User can select the insertion point and duration of buffering before starting recording. This helps user to merge unexpected event recording in already recorded file with efficient use of memory
The present invention allows user to continue recording in the same file. Hence user can close the recorder whenever needed and can start
recording at any time in the same file with hot key. This saves the battery life of the recorder.
• The present invention avoids merging of multiple files to form a single file thereby saving time.
• With the present invention, it is possible to correct actions by repeating events and then re-recording in the same file.
• The present invention provides quick access and indexing method for recorded video. If user records an event with 'append', 'insert' and 'overwrite' options then user can watch result of all 'append', 'insert' and 'overwrite' operations as thumbnails and can directly seek a particular sub-event for playing. Thus indexing and quick access of AV content is possible
• With the editing options provided by the proposed invention for real time recording, it is possible to create interesting videos using one single application
Although the present invention has been fully described in connection with the preferred embodiments thereof with reference to the accompanying drawings, it is to be noted that various changes and modifications are possible and are apparent to those skilled in the art. Such changes and modifications are to be understood as included within the scope of the present invention as defined by the appended claims unless they depart there from.

We Claim:
1. A system for real time AV editing comprising,
a camera pre-recording preview module for previewing live frames captured by a camera ;
live AV recording module for recording audio and video encoded /compressed data in real time;
media player for playback of selected recorded audio and video encoded data; and
editing module for editing the encoded audio and video data.
2. The system according to claim 1, wherein the editing module is adapted to append, insert and overwrite audio and video data in an existing file.
3. The system according to claim 2, wherein the editing module is adapted to append, insert and overwrite audio and video data single or multiple times in an existing files.
4. The system according to claim 2, wherein the editing module is adapted to insert or overwrite data at locations for specified duration of time in the already existing file by marking single or multiple points in the encoded audio and video data.
5. The system according to claim 1, wherein the editing module comprises of a 'continue' option to record live data in the existing file.
6. The system according to claim 1, wherein the editing module comprises of a 'pre-pend' option to add live media content to the beginning of the existing file.
7. The system according to claim 1, wherein the editing module comprises of a 'quick start' option to record live data multiple times in one single file with one key press.
8. The system according to claim 1, wherein the editing module comprises of a 'quick pick up point' option to see last 'n' seconds of recorded data and to identify a correct point to start or append the live data to be recorded in the same file.
9. The system according to claim 1, wherein the editing module comprises a re¬start module to continue overwriting recorded data from the starting of the file.
10. The system according to claim 1 wherein, the editing module comprises 'window recording' option to insert or overwrite last 'n' seconds of recorded data at desired point in a file.
11. The system according to claim 1, wherein the editing module comprises a means to preview edited data.
12. A method for real time AV editing comprising the steps of,
previewing live frames captured by a camera using a camera pre-recording preview module ;
recording audio and video encoded /compressed data in real time using live AV recording module;
playing selected recorded audio and video encoded data using media player; and
editing the encoded audio and video data using editing module.
13. The method according to claim 12, wherein insertion comprises the step of:
getting exact timing details of an insertion point by the editing module when a user marks the insertion point;
finding a nearest "I" frame by parsing the tables in a track atom corresponding to a time given by the user;
copying and storing moov data which contain the metadata of the existing file;
writing the live data captured and encoded/compressed to existing file on frame by frame basis during live recording where area occupied by old meta data which is already copied is used to write live media data and continued till completion without touching or overwriting original media data; storing the metadata information for the newly added content in separate location; and forming a new moov atom or new metadata information by processing each field and atom using old meta data and meta data of live recording whenever user stops recording.
14. The method according to claim 13,wherein overwriting comprises the step of:
selecting the start and end point by the user within which data is to be replaced;
finding the T frames corresponding to each point;
copying and storing moov data which contain the metadata of the existing file;
finding the exact location of audio and video frame that are to be replaced or to be overwritten within the file using the sample tables;
writing new data in an optimal way in memory or at the end of the file; storing the meta data information of the newly added content in separate location;
writing the newly formed meta data that corresponds to old data and live recorded data to the file at the end whenever user stops recording.
15. A system for real time AV editing substantially described particularly with reference to the accompanying drawings.
16. A method for real time AV editing substantially described particularly with reference to the accompanying drawings.

Documents:

0117-che-2007 abstract.pdf

0117-che-2007 claims.pdf

0117-che-2007 descrption(complete).pdf

0117-che-2007 drawings.pdf

0117-che-2007 form-5.pdf

117-CHE-2007 AMENDED PAGES OF SPECIFICATION 18-09-2013.pdf

117-CHE-2007 AMENDED CLAIMS 18-09-2013.pdf

117-CHE-2007 EXAMINATION REPORT REPLY RECEIVED 18-09-2013.pdf

117-CHE-2007 FORM-1 18-09-2013.pdf

117-CHE-2007 FORM-13 18-09-2013.pdf

117-CHE-2007 POWER OF ATTORNEY 18-09-2013.pdf

117-CHE-2007 CORRESPONDENCE OTHERS.pdf

117-CHE-2007 FORM-18.pdf

117-che-2007-correspondnece-others.pdf

117-che-2007-description(provisional).pdf

117-che-2007-drawings.pdf

117-che-2007-form 1.pdf

117-che-2007-form 26.pdf

« Previous Patent

Next Patent »

Patent Number

257580

Indian Patent Application Number

117/CHE/2007

PG Journal Number

42/2013

Publication Date

18-Oct-2013

Grant Date

17-Oct-2013

Date of Filing

19-Jan-2007

Name of Patentee

SAMSUNG INDIA SOFTWARE OPERATIONS PRIVATE LIMITED

Applicant Address

BAGMANE LAKEVIEW, BLOCK 'B' NO. 66/1, BAGMANE TECH PARK, C V RAMAN NAGAR, BYRASANDRA, BANGALORE-560 093.

Inventors:

#	Inventor's Name	Inventor's Address
1	GIRISH KULKARNI	EMPLOYED AT SAMSUNG INDIA SOFTWARE OPERATIOS PVT. LTD., HAVING ITS OFFICE AT BAGMANE LAKEVIEW, BLOCK 'B' NO. 66/1, BAGMANE TECH PARK, C V RAMAN NAGAR, BYRASANDRA, BANGALORE-560 093.

PCT International Classification Number

G06Q10/00

PCT International Application Number

N/A

PCT International Filing date

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1			NA