Title of Invention

"A CODING METHOD AND AN APPARATUS FOR CODING A MOVING PICTURE"

Abstract ABSTRACT The present invention relates to a coding method for generating a coded signal corresponding to each picture by coding a plurality of coded signals, a switching picture capable of switching a plurality of coded signals and following pictures of the switching picture can refer to only a group of pictures of same time in the coded signals. More specifically, the case where picture numbers of an adjacent picture of an S picture are not continuous is not considered as an error.
Full Text DESCRIPTION
Technical Field
The present invention relates to a picture coding method for efficiently compressing moving picture signals using correlation between pictures, a picture decoding method for decoding the signals correctly, and a recording medium on which a program for executing these methods using software.
Background Art
Recently, the multi-media era has come in which sound, pictures and other pixel values are integrated into one media, and conventional information media as communication tools like newspapers, magazines, TV, radio and telephone are regarded as the targets of multi-media. Generally, multi-media is a form of simultaneous representation of not only characters but also graphics, sound, and especially pictures. In order to handle the above-mentioned conventional information media as multi-media, it is a requis:te to represent the information digitally.
However, it is unrealistic to directly process a huge amount of information digitally using the above-mentioned conventional information media because, when calculating the data amount of each information medium mentioned above as digital data amount, data amount per character is 1 ~-2 byte while that of sound per second is not less than 64 kbits (telephone speech quality) and that of moving pictures per second is not less than lOOMbits (present TV receiving quality). For example, a TV telephone has already become commercially practical thanks to Integrated Services Digital Network (ISDN) with a transmission speed of 64kbps ~ 1.5 Mbps, but it is impossible to transmit pictures of TV camera as they are using ISDN.

That is why information compression technique is necessary. For example, a moving picture compression technique standard of H.261 or H.263 internationally standardized by International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) is used for TV telephones. Also, it is possible to store picture information with sound information in ordinary music compact discs (CDs) using the information compression technique of the MPEG-1 standard.
Here, Moving Picture Experts Group (MPEG) is an international standard to digitally compress moving picture signals, and MPEG-1 is the standard to compress moving picture signals to 1.5Mbps, that is, compress TV signal information to about one hundredth. Also, the quality that satisfies the MPEG-1 standard is medium level that-can be realized at a transmission rate of about 1.5Mbps. MPEG-2 is thus standardized in order to satisfy the nggrj for hioher picture oualitv. and it comoresses moving picture signals to 2 ~ 15Mbps.
At present, the work group (ISO / IECJTC1 / SC29 / WG11), which standardized MPEG-1 and MPEG-2, has standardized MPEG-4 with a higher compression rate. MPEG-4 introduced not only efficient coding at a low bit rate, but also a powerful error-resist technique that lessens subjective picture deterioration in case a transmission error occurs. Also, as a picture coding system for the next generations, ISO / IEC and ITU-T are jointly working for the standardization of Joint Video Team (JVT). At oresent, the standard called Joint Model 2 (3M2) is the latest version.
The picture for intra predictive coding without any reference picture is called Intra Coded Picture (I picture). Also, the picture or inter predictive coding with a reference picture is called Predictive Coded Picture (P picture). Also, the picture for inter predictive coding in which two reference pictures are referred to

simultaneously is called 3i-predictive Coded Picture (3 picture).
"Picture" used here is a term representing one picture. In a progressive picture, a picture means a frame, but in an interlace picture, it means a frame or a fieid. An "interlace picture" mentioned here means a frame composed of two fields with a slight time lag. In the coding and decoding processes of interlace pictures, it is possible to process a frame as it is, as two fields, or by each block in a frame in a frame-by-frame structure or in a field-by-field structure.
In JVT, it is possible to choose an arbitrary picture as a forward reference picture from a plurality of pictures, unlike conventional moving picture coding. A!so, a system to switch coded bit streams at specific pictures, that is, Switching Coded Pictures (S pictures) has been introduced. (There are SI pictures and SP pictures in S pictures, and these are the pictures for intra predictive coding or inter predictive coding respectively.)
S picture system is to guarantee that streams after S Pictures can be decoded correctly in the case of switching from stream to stream just before S pictures. Also, it is possible to switch streams at a server such as a moving picture distribution server according to the communication capacity of receiving terminals or preference of receivers.
In the conventional picture coding method or picture decoding method, S pictures are introduced so that (1) it can choose an arbitrary picture as a forward reference picture from a plurality of pictures, and (2) it can also switch pictures at specific pictures. In spite of the introduction of these two techniques, regrettably, problems that occur when these two techniques are combined have not yet considered well. In reality, it is difficult to use both the techniques together because of the problems shown
below.
FIG. 1 is an illustration showing the relations between

pictures and picture numbers (PN) when coding an input picture-signal (VIN). The same picture signal is coded at different picture rates (the number of pictures per second) to make Streams 1, 2 and 3. Picture numbers (PN) are numbers to identify coded pictures. In JM2, pictures to be referred to as reference pictures in the following coding are assigned numbers incremented by 1. To simplify the explanation, the example of FIG. 1 shows only the case that all pictures in each stream are referred to as reference pictures in the following coding, and the picture numbers are always incremented by 1. Pictures that are not referred to in the following coding are unrelated to the increase or decrease in the picture numbers, and not stored in a memory. Therefore, explanation as to pictures that are not referred to in the following coding is -omitted because the pictures are unrelated to the- following explanation of operations.
As shown in FIG. 1, at the time cf tB, the diagonally shaded pictures are coded as S pictures'. FIG. 2 is a diagram showing picture numbers (PN) of the pictures to be stored in the reference picture memory when coding or decoding S pictures.
FIG. 2 shows pictures stored in the reference picture memory (Mem) and their positions. In the reference picture memory (Mem), pictures in the left position are newer in time than pictures in the right position. At the time of predictive coding, the same pictures must be referred to in coding and decoding. When it is possible to choose each reference picture from a plurality of reference pictures like in the 3M2, it is necessary to specify which pictures are referred to.
There are two following methods to show reference pictures, and JM2 uses the two methods properly according to purposes. (D Clearly expressing how many pictures there are prior to a newer
picture © Cleariy expressing a reference picture by a picture number (PN)

In order to correctly code S pictures and the following pictures and to decode these pictures correctly at the time of decoding when switching streams at S pictures, the contents of the reference picture memory (Mem) must be the same in every case of switching streams at S pictures.
However, as shown in the illustration of FIG. 2 showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem), the contents of the reference picture memory (Mem) are not the same at the start of coding or decoding an S picture in each stream. Unless such a conventionai method is improved, it is impossible to use the coding method to choose reference pictures from reference picture memory (Mem) in combination with the S picture system for switching streams.
The present invention aims at solving all the above-mentioned problems, making the 5 picture system usable in combination with the other coding method to choose reference pictures in reference picture memory (Mem), and thus providing picture coding and decoding methods that improve compression rates in the above-mentioned combined coding method using S pictures
Disclosure of Invention
In order to resolve this problem,
The first invention Is a picture coding method for specifying a reference picture stored in a memory using a picture number for reference and generating a coded stream of moving pictures, the picture coding method comprising a picture number coding step of coding picture numbers corresponding to the current pictures, a coding step of coding the current pictures, a step of making aii the pictures stored in the memory except the current pictures released for reference after the picture coding step, a picture number initializing step of initializing the picture numbers of the current

pictures in the memory, and an all picture release Information coding step of coding all picture release information for instructing a picture decoding apparatus to release all the pictures that have been already stored in the memory except the current pictures.
The second invention is a picture decoding method for specifying a reference picture stored in a memory using a picture number for reference and decoding the coded stream of moving pictures, the picture decoding method comprising an all picture release information decoding step of checking and decoding an all picture release information meaning releasing all the pictures in a coded stream stored in a memory except the current pictures to be decoded, a decoding step of decoding the current pictures in the coded stream, a step of releasing all the pictures stored in the memory except the current pictures according to the decoded all picture release information after the decoding step, and a picture number initializing step of assigning an initialized picture number to the current picture in the memory."
The third invention is a picture coding method for coding a plurality of picture signals and generating coded signals corresponding to respective pictures, wherein a switching picture capable of switching a plurality of coded signals and the following pictures of the switching picture can refer to only a group of pictures of same time in the coded signals.
The fourth invention is a picture decoding method for decoding a coded signal, wherein information on pictures released for reference before a switchable switching picture is decoded, decoded pictures in a reference picture memory are released based on the decoding result, and after the switching picture, a coded signal is decoded referring to a reference picture that is not released.
The fifth invention is a picture coding method for coding a plurality of picture signals and generating coded signals

corresponding to respective oictures,. wherein the picture coding method has 3 step of changing a picture number of a switching picture capable of switching a plurality of coded signals into same value at respective one of coded signals.
The sixth invention is a picture decoding method for decoding a coded signal comprising, a step of changing a picture number of a reference picture into same value in a switchable coded signal at the time of switching coded signals at a switchable switching picture.
As mentioned above, with the picture coding method and the picture decoding method in the present invention, it is possible to use the feature of S pictures and a coding method for choosing a reference picture in a reference picture memory in combination, which makes it possible to provide a picture coding method and a " picture decoding method to improve compression rates even when using S pictures in the ceding method, and thus these methods are nighly practical.
Brief Description of Drawings
FIG. 1 is an illustration of relations between pictures and picture numbers (PN) when coding an input picture signal (Vin).
FIG. 2 is a diagram showing picture numbers (PN) of pictures to be stored in a reference picture memory (Mem) when coding and decoding S pictures.
FIG. 3 is an illustration of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem).
FIG. 4A, 4B and 4C are flow charts of how to code and decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture coding method and the picture decoding method of the present invention.
FIG. 5A and 5B are illustrations of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem).

FIG. 6Af 6B and 5C are flow charts of how to code and decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture coding method and the picture decoding method of the present invention.
FIG. 7 is an illustration of the relation between pictures and picture numbers (PN) when coding input picture signal (Vin) of the present invention.
FIG. 8A and 8B are flow charts of how to decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture decoding method of the present invention.
FIG. 9 is a block diagram showing the structure of picture coding apparatus of the present invention.
FIG. 10A, 10B, IOC and 10D-are diagrams showing data structure examples of coded signals, Str of the present invention.
RG, 11 is a block diagram showing the structure cf 3 picture decoding apparatus of the present invention.
FIG. 12 is an illustration of the relation between pictures and picture numbers (PN) for coding an input picture signal (Vin) of the present invention.
FIG. 13 is a flow chart showing a coding method by giving a picture number to each picture of each stream in the present invention.
FIG. 14 is a flow chart showing a decoding method in a seventh embodiment.
FIG. 15 is a diagram showing a memory structure in the seventh embodiment.
FIG. 16A and 16B are flow charts showing a coding method in the seventh, embodiment.
FIG. 17A and 17B are flow charts showing another coding method in the seventh embodiment.
FIG. 18 is a flow chart showing another coding method in

the seventh embodiment,
FIG. 19 is a block diagram showing a structure of a coding apparatus in an eighth embodiment.
FIG. 20 is a block diagram showing a structure of another coding apparatus in the eighth embodiment.
FIG. 21A and 21B are flow charts showing a decoding method in a tenth embodiment.
FIG. 22 is a block diagram showing a structure of a decoding apparatus in an eleventh embodiment.
FIG. 23A and 23B are flow charts showing the processing for making coded signals and decoding the coded signals.
FIG. 24 is a block diagram showing a structure of a picture
coding apparatus realizing a coding method in a twelfth
embodiment, ------ - -
FIG. 25 is a block diagram showing a structure of a picture decoding apparatus realizing the decoding method in the twelfth - embodiment.
FIG. 26 is an illustration as to a recording medium storing a program for realizing the picture coding methods and picture decoding methods in the first to the twelfth embodiments using a computer system.
FIG. 27 is a block diagram showing the whole structure of a content supplying system realizing a content distribution service concerning the present invention.
FIG. 28 is a diagram showing an example of cellular phones relates to the present invention.
FIG. 29 is a block diagram showing the structure of the same cellular phones, and
FIG. 30 is a diagram showing the structure of a digital broadcasting system concerning the present invention.
Best Mode for Carrying Out the Invention

i he present embodiments of the present invention wi!! be
explained below with reference to FIG. 3~30.
(First Embodiment)
FIG. 3 is an illustration showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem). The difference between the same figure and FIG. 2 showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem) will be explained below.
It has already been explained that, after switching coded signals when coding and decoding S pictures, the content of the reference picture memory (Mem) is not the same. Therefore, in the coding and the decoding methods of the present invention in FIG. 1 showing the relations of pictures and their picture numbers (PN) when coding input picture signal (Vin), only pictures at the times of t0; tl and t2 "het ere exactly the same respectively in 5!! streams are stored In the reference picture memory (Mem), while the other pictures are deleted In the reference picture memory (Mem) before coding and decoding S pictures. FIG. 3 shows the result of this processing as an illustration showing the picture numbers (PN) of the pictures to be stored in the reference picture memory (Mem).
As shown in FIG. 3 explaining picture numbers of pictures to be stored in the reference picture memory (Mem), when the method of "Clearly expressing how many pictures there are prior to a newer picture" is employed to specify reference pictures in coding and decoding, it is possible to code and decode the pictures correctly because exactly the same pictures of the same time are referred to in any case of stream 1, 2 or 3.
FIG. 4 is a flow chart showing coding and decoding methods of information concerning picture control of pictures to be stored in the reference picture memory (Mem) in the picture coding

decoding methods of the present invention,
FIG. 4A as a flow cha-t of the coding method that shows how to realize the method of the operation explained in FIG. 3. and the coding and decoding methods of the necessary information for realizing the operation.
In Step 0, pictures at the same points of time in a plurality of coding information (streams) are chosen. In Step 1, it is possible to code delete information showing deletion of the other pictures which are not chosen in Step 0. In Step 2, pictures which are not chosen in Step 0 are deleted from the reference picture memory (Mem). Up to this point, as shown in FIG. 3, it is possible to realize the storage condition in the reference picture memory (Mem) to realize decodable streams even after switching coded signals.
Also, it is possible to change the order of Step 1 and 2, and if it is changed, the flow chsrt of the picture coding method shown in FIG: 4B is used.
Decoding the delete information coded according to the coding method shown as the flow chart in FIG. 4A using the decoding method shown as the flow chart in FIG. 4C makes it possible to realize the storage condition of the reference picture memory (Mem) to realize decodable streams using the picture decoding method even after switching coded signals as shown In FIG. 3.
By decoding the delete information in Step 5, it is possible to show pictures that are not the same-time pictures in a plurality of coding information (streams). These pictures are the rest of the pictures chosen in Step 0 of FIG. 4A as pictures of the same time. Next,, in Step 6, pictures chosen in Step 5 are deleted in the reference picture memory (Mem). Specifically, when pictures stored in the reference picture memory (Mem) are deleted (or erased), pictures to be deleted are assigned IDs (identification

information) such as "release" prohibiting from using the pictures concerned as reference pictures. For that purpose, picture decoding unit (PicDec) and picture encoding unit (PicEnc) always check whether IDs of "release" are assigned or not each time these units refer to pictures stored in the reference picture memory (Mem). Only pictures that are not assigned IDs of "release" are referred to by these units while pictures assigned IDs of "release" are not referred to by these units. Likewise, pictures in the reference picture memory (Mem) are deleted (or erased) in the following embodiments. Of course, as this deletion method is an example, it goes without saying that it is possible to delete the above-mentioned picture data in the reference picture memory (Mem) by actually deleting or erasing these data. Up to this point, it is possible to realize the storage condition in the reference picture memory (Mem) to realize decodable streams even after switching the coded signals as shown in FIG. 3.
(Second Embodiment)
FIG. 5A is an illustration showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem). The difference between FIG. 5A and FIG. 3 showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem) is whether picture numbers (PN) in the reference picture memory (Mem) are the same or not.
As not only the times of pictures stored in the reference picture memory (Mem), but also picture numbers (PN) in each stream in the reference picture memory (Mem) are the same in the time frame, it is possible to use the method of "Ciearly expressing reference pictures by picture numbers (PN)" when specifying reference pictures in coding and decoding, and thus it becomes possible to code and decode the pictures correctly because exactly the same pictures of the same time are referred to in any case of

stream 1, 2 or 3 =
it is possible to realize zhls when replacing the picture numbers of pictures stored in the reference picture memory (Mem) with a new same picture number, and coding and decoding the information for replacing the old picture numbers with the new one before coding and decoding S pictures.
In addition, there is a need to match picture numbers (PN) of S pictures in any of the streams because the same picture numbers (PN) must be used when storing S pictures next time.
FIG. 6 is a flow chart showing coding and decoding methods of information for controlling pictures to be stored in the reference picture memory (Mem) according to the picture coding and decoding methods of the present invention, and shows the realization methods of the operation explained In FIG. 5A and the coding and decoding methods of information necessary for the realization.
In Step 10, the maximum value of picture numbers (PN) ("8" in the example of FIG. 5A.) of pictures in the coded signals to be switched in the reference memory (Mem) are detected. In Step 12, information for reassigning picture numbers (PN) of each picture stored in the reference picture memory (Mem) is coded with reference to the maximum value of picture numbers (PN). Also, as the need arises, picture numbers (PN) to be assigned to the next S pictures are coded. As stream 3 in FIG. 5A is the same as stream 3 in FIG. 3, there is no need to reassign picture numbers of pictures in stream 3. Therefore, picture numbers are reassigned to necessary pictures only, only information on necessary reassignment needs to be coded in Step 11. Lastly, picture numbers shown by the coded information in Step 11 is reassigned in Step 12. Up to this point, as shown in FIG. 5, the storage condition in the reference memory (Mem) for realizing decodable streams even after switching the coded signals is

realized.
Also, as the picture number (PN) of S pictures are 12, in order to make picture numbers (DN) continuous after coding and decoding these S pictures., it is possible to use picture number 11, that is the picture number (PN) immediately before an S picture (immediately before an S picture of stream 1 in FIG. 1) as shown in FIG. 5B. In this case, picture numbers (PN) always increase in the process of coding and decoding as the picture number (PN) of the S picture is 12, which is more effective because the error check function to regard decrease in picture numbers (PN) as an error is also realized.
FIG. 7 is an illustration showing the relations of pictures and picture numbers (PN) when coding an input picture signal (VIN) of the present invention. FIG. 7 is an example of reassignment of picture numbers (PN) using the method explained in FIG. 5B, ail the picture numbers of S pictures are 12. Therefore, it is dear • -c-hat ail the pictures after S pictures are correctly decodable -even -after switching streams at S pictures because pictures in the reference memory (Mem) are identical irrespective of streams when coding and decoding S pictures. Also, it is possible to change the operational order of Step 11 and 12, and in this case, a flo.w chart of picture coding method shown in FIG. 6B is used.
Decoding the coded delete information in FIG. 6A shown in the flow chart of the coding method using the decoding method shown in the flow chart in FIG. 6C of the decoding method makes it possible to realize the storage condition of the reference picture memory (Mem) to realize decodable streams using the picture decoding method even after coded signals are switched as shown in FIG. 5A.
Decoding the information on reassignment of picture numbers (PN) in Step 15 makes it possible to specify the pictures necessary for reassignment of picture numbers (PN) and the

method. Next, in Step 16, picture numbers (PN) of pictures in the reference picture memory (Mem) is reassigned based on the pictures decoded in Step 15 and also requires for reassignment of picture numbers (PN) and the reassignment method. Up to this point, it is possible to realize the storage condition in the reference picture memory (Mem) to realize decodable streams even after switching the coded signals as shown in FIG, 5.
While this second embodiment explains the effectiveness in combination with the first embodiment, as second embodiment can solely realize the merit of coding and decoding correctly in the case of "Clearly expressing reference pictures by picture numbers (PN)", it is possible to use only second embodiment instead of the second embodiment in combination with the first embodiment if the effect of the-second embodiment is sufficiently effective.
(Third Ernbodiment)
FIG. 8 is another embodlments'o as to realize an illustration of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem) in FIG. 5.
Picture types are identified by picture type information (PicType). Therefore, when a picture is identified as an 5 picture that can change streams by picture type information (PicType), making a rule for reassigning picture numbers (PN) of pictures in the reference picture memory (Mem) to coincide with picture numbers (PN) of S pictures makes it possible to omit coding and decoding the information on reassignment method for each picture number (PN) of pictures in the reference picture memory (Mem).
The operation shown in FIG, 8A will be explained below. In Step 20, picture numbers (PN) of pictures are gotten by decoding coded signals. The picture type information (PicType) of the pictures is gotten in Step 21. When the picture type information (PicType) is proved to be S pictures, picture numbers (PN) of

pictures in the reference picture memory (Mem) in a way that they coincide with picture numbers ;PN) of 5 pictures are reassigned based on a specified method in Step 22. Up to this point, as shown in FIG. 5, it is possible to realize the storage condition in the reference picture memory (Mem) that realizes decotiable streams even after switching coded signals.
Also, it is possible to change the order of Step 21 and 22, and if it is changed, the flow chart of the picture coding method shown in FIG. 8B is used.
Also, it is possible to code and decode only part of
reassignment information of picture numbers (PN) (which cannot
be represented by the rule for reassigning picture numbers (PN) in
the reference picture memory (Mem) in a way that they coincide
with picture numbers (PN) of S pictures) in Step 11 and 15 in FIG.
6 by combining the illustration of FIG. 8 and 6 concerning picture
numbers (PN) of pictures to be stored in the reference picture
memory (Mem) in FIG. 5. "
(Fourth Embodiment)
FIG. 9 is 3 block diagram showing the structure of the picture coding apparatus of the present invention. FIG. 9 as a block diagram concerning the picture coding apparatus of the present invention is an example for realizing the picture coding method in the first embodiment and the second embodiment.
Picture number generating unit (PNGen) generates picture numbers (PN). Picture numbers (PN) are IDs to identify the pictures stored in the reference picture memory (Mem), each picture stored in the reference picture memory (Mem) is assigned to an exclusive picture number (PN). Usually, picture numbers (PN) are incremented by 1 each time a picture is stored in the reference picture memory (Mem). When a picture number (PN) received by a picture decoding apparatus is incremented by 2 or

more, the picture decoding apparatus can found that a picture to be stored is missing because of transmission line error, and can correct the error and make the error less conspicuous.
Maximum picture number checking unit (MaxPN) compares other coded signal picture numbers (OtherPN) and picture numbers (PN) generated in the picture number generating unit (PNGen), detects the maximum value of picture numbers (PN), notifies variable length unit (VLC) and the picture number generating unit (PNGen) of the maximum value of picture numbers (PN), and initializes picture numbers (PN) to be generated in the picture number generating unit (PNGen) using the maximum value of picture numbers (PN). Other coded signal picture numbers (OtherPN) are picture numbers of pictures in a different stream in parade! with the pictures to be coded. Consequently, after that, the picture number generating unit (PNGen) starts to output Dicture numbers (PN) larger than the maximum value of the picture numbers (PN).
Coded picture time comparing unit (TimeCmp) compares the frame time of each picture in the input picture signai (Vin) coded so far and the frame time of each picture coded as other coded signals (streams), and notifies picture deleting unit (PicDel) of picture information concerning the frame time coded in ail the streams.
When picture type information (PicType) shows that next picture is an S picture, picture deleting unit (PicDel) orders reference picture memory (Mem) to delete the pictures outside the time frame in all the streams stored in the reference picture memory (Mem) based on the information notified by the coded picture time comparing unit (TimeCmp), and notifies variable length coding unit (VLC) of the same information at the same time.
Picture encoding unit (PicEnc) refers to the pictures in the reference picture memory (Mem), codes the input picture sianal

(Vin) includingfrequency conversion and quantization as a Dicture type showin in tne picture type information (PicType), sends the result to picture decoding unit (PicDec) and variable length coding unit (VLC). Picture decoding unit (PicDec) performs inverse quantization and frequency conversion of the coding result in picture encoding unit (PicEnc) as picture types shown in the picture type information (PicType), and stores the picture types as picture numbers (PN) in the reference picture memory (Mem) so as to refer to the picture types in the following picture coding process.
Variable length coding unit (VLC) makes the coded result in picture encoding unit (PicEnc) into variable length codes so as to make a bit stream, codes the information needed for decoding, that isthe information for deleting pictures stored in the reference picture memory (Mem) notified by the picture deleting unit (PicDei), the maximum value of the picture numbers (PN) and picture numbers (PN) mentioned above so as to output the information as coded signals (Str). Variable length coding unit (VLC) also codes information notified by picture deleting unit (PicDsl) and information for reassigning picture numbers (PN) of pictures stored in the reference picture memory (Mem) based on the method shown in the second embodiment.
FIG. 10 shows a structural example of coded signals (Str) in the present invention. Each data in FIG. 10A will be explained below.
First, picture numbers (PN) are coded. Next, the maximum picture number (PN) to be reassigned, information for deleting pictures stored in the reference picture memory (Mem) and information for reassigning picture numbers stored in the reference picture memory (Mem) are coded. After that, picture type information (PicType) and picture coded data which is outputted by picture encoding unit (PicEnc) are located.

As FIG. IOA is simp!y an example of d3ta location, it is possible to change order of dats as shown in FIG. 103 so as to perform picture coding.
It is possible to prepare the picture coding apparatus consists of the above-mentioned units that realizes the picture coding method shown in the first and the second embodiments.
(Fifth Embodiment)
FIG. 11 is a block diagram showing the structure of the picture decoding apparatus of the present invention, FIG. 11 as a block diagram concerning the picture decoding apparatus of the present invention is an example of a picture decoding apparatus that realizes the first, the second and the third embodiments. Its function will be explained below-..-----.-
Variable length decoding unit (VLD) decodes coded signals (Str),. outputs various information (such as an order for deleting pictures stored in the reference picture memory "(Mem), picture type information (PicType), picture numbers (PN), information for reassigning picture numbers (PN) and picture data.)
The order for deleting pictures stored in the reference picture memory (Mem) gotten in variable length decoding unit (VLD) is firstly sent to picture deleting unit (PicDei). Picture deleting unit (PicDei) deletes the specified pictures stored in the reference picture memory (Mem).
Picture type information (PicType) gotten in the variable length decoding unit (VLD) is sent to picture decoding unit (PicDec) so as to show the decoding method.
Picture numbers (PN) gotten in variable length decoding unit (VLD) is sent to the reference picture memory (Mem) as picture numbers (PN) when storing the pictures decoded in picture decoding unit (PicDec).
Information for reassigning picture numbers (PN) of

pictures stored in the reference picture memory (Mem) gotten in the variable length decoding unit (VLD) is sent to picture number changing unit (PNchg). Picture number changing unit (PNchg) follows the directions and reassign picture numbers (PN) of pictures stored in the reference picture memory (Mem). To be more specific, picture number changing unit (PNchg) reads out picture numbers (PN) of pictures stored in the reference picture memory (Mem), reassigns the values of picture numbers (PN) read out, and then writes the new picture numbers (PN) in the reference picture memory (Mem).
In the picture decoding unit (PicDec), picture data gotten in variable length decoding unit (VLD) are decoded in a suitable decoding method for the specified picture type shown as picture type information (PicType). In other words, I pictures are decoded- ■■■ -without referring pictures in the reference picture memory (Mem) while P pictures and 3 pictures are decoded by referring to pictures in the reference picture memory (Mem). The decoded pictures gotten in this way are stored in the reference picture memory (Mem) and outputted as decoded picture signals (Vout).
Up to this point, it is possible to prepare the picture decoding apparatus consists of the above-mentioned units that realizes the picture decoding method shown in the first, the second, 3nd the third embodiments.
(Sixth Embodiment)
In the picture coding apparatus shown in the first to the fifth embodiments, when switching streams at S pictures, the picture numbers of the pictures before the switchabie pictures are switched so as to make the picture numbers continuous with the picture numbers of the switchabie pictures. In this sixth embodiment, picture numbers are switched at the switchabie pictures.

As to coding of a plurality of streams having a -different picture rate or a bit rate, or 3 different structure, here is an example of a stream switching method that enables coding after switching coded pictures from a picture under coding in a stream to a picture in another stream. For convenience of explanation, the simpler phrase of "switching streams" is used below.
In addition, in this sixth embodiment, whether pictures to be coded should be stored in the reference memory or not is judged based on the increment in picture number between pictures to be coded and pictures forwardly adjacent to the pictures to be coded (Simply, "the preceding picture" is used below.) in coding order. To be specific, when an increment in picture number between a preceding picture and a picture to be coded is 1, which means the pictures to be coded is stored in the reference memory. When the picture number of the pictures to be coded is the same as the picture number of the preceding pictures, which means the current picture is not stored in the reference memory.
The processing of switching picture numbers (PN) of switchable pictures will be explained concretely with reference to FIG. 12.
FIG. 12 is a diagram showing an example of the relations between pictures and picture numbers (PN) when sn input picture number (Vin) is coded. An identical picture signal is coded at different picture rates to make streams of 1, 2 and 3. In FIG. 12, pictures are located according to the cording order in each stream.
In stream 1, picture numbers (PN) are assigned to each picture so that the picture numbers are incremented by 1. Aiso, in stream 2, there are pictures assigned picture numbers (PN) incremented by 1 and pictures assigned the same picture numbers (PN) as the precedent pictures. Also, in stream 3, picture number (PN) are assigned to each picture so that the picture numbers are

incremented by 1 like in stream 1,
Therefore, as the picture numbers are incremented by 1 in streams 1 and 3, pictures to be coded are stored in the reference memory. In stream 2, pictures assigned picture numbers (PN) in a way that the picture numbers are incremented by 1 are stored in the reference memory, and pictures assigned the same picture numbers (PN) as the precedent pictures are not stored in the reference memory.
Also, pictures assigned the picture number of "O" in streams of 1, 2 and 3 are pictures to be displayed at the time of tO. Likewise, the groups of pictures fisted below are pictures to be displayed at the same time: Picture F14 in stream 1, picture F22 in stream 2 and picture F31 in stream 3 are pictures to be displayed at the time of tl. Picture F18 in stream 1, picture F24 in stream 2 and picture F32 in stream 3 are pictures to be displayed at the time of t2. Picture F112 in stream 1, picture F26 in stream 2 and picture F33 in "stream 3 are pictures to be displayed at the time of t3. Picture F117 in stream I, picture F215 in stream 2 and picture F34 in stream 3 are pictures to be ■ displayed at the time of t4. Note that pictures F112, F26 and F33 correspond to S pictures in the first and second embodiments.
In FIG. 12, streams are switched by way of picture BPI and BP2 that exist between the picture before switching and the picture after switching, and both of the BPI and the BP2 are switching pictures to be coded in a way that they have the same time as their precedent pictures in the respective switching streams.
For example, in the case where picture F026 in stream 2 (a picture in the stream before switching) is switched to picture F113 in stream 1 (a picture in the stream after switching), switching picture BPI that exists between F26 and F113 is used as a picture at the time of t3. In this case, the picture number of switching

picture BP1 which is a switching pictureis changed to v12" to make the number contiguous with the picture number 13 of picture Fil3 in the stream after switching.
Also, likewise, in the case where picture F32 in stream 3 (a picture in the stream before switching) is switched to picture F213 in stream 2 (a picture in the stream after switching), switching picture BP2 that exists between F32 and F213 is used as a picture at the time of t3. In this case, the picture number of switching picture BP2 which is a switching picture is changed to make the number continuous with the picture number 13 of picture F213 in the stream after switching.
In this way, by assigning picture numbers (PN) of switching pictures to make the numbers continuous with the picture numbers (PN) of pictures in the stream after switching, picture numbers (PN) of pictures in the stream after switching are changed to be identical ;n eny case of coded pictures within each stream or switching streams.
Next, the processing of assigning picture numbers (PN) in the case of switching streams will be explained below.
FIG. 13 is a flow chart showing a coding method after assigning picture numbers (PN) to respective pictures in the streams in FIG. 12.
In step 1401, whether each of the pictures to be coded is S picture or not is judged. When the current pictures are S pictures, picture numbers (PN) of the current pictures are changed to the initial values of M in step 1402. When the current pictures coded are not S pictures, picture numbers (PN) of the current pictures are not changed.
In step 1403, whether each of the pictures to be coded is the next pictures of S pictures or not is judged. When the current pictures are the next pictures of 5 pictures, whether each of S pictures is stored in the memory or not is judged in step 1404.

When the current pictures 3re not the next pictures of S pictures, whether each of the current pictures Is stored in the memory or not is judged in step 1405.
When S pictures are judged to be stored in the memory in step 1404, the picture number "M" are incremented by 1 to make M + l in step 1406, the incremented picture numbers (PN) replace the old picture numbers (PN).
When S pictures are not judged to be stored in the memory in step 1404, the picture numbers (PN) are regarded as "M"s in step 1407. The picture numbers (PN) are not changed. In step 1405, whether the pictures to be coded are stored in the memory or not is judged. When the current pictures are judged to be stored in the memory, the picture numbers (PN) are incremented to PN + 1 in step 1408, and the incremented picture-numbers (PN) replace the old picture numbers (PN).
When the pictures to be coded are judged net to be stored in the memory, the picture numbers (PN)" ere hot changed.
In step 1409, target pictures are coded. In step 1410, whether ail the current pictures have been coded or not is judged. When ail the current pictures have not been coded, step 1401 must be repeated. When all the current pictures have been coded, step 1410 finishes.
The processing shown in FIG. 13 makes it possible to produce coded data streams whose picture numbers (PN) are continuous in the streams after switching coded pictures.
Also,, the coded signals (Str) produced in this way can be decoded based on the decoding method by the picture decoding apparatus in the fifth embodiment. In this way, the picture decoding apparatus that decodes coded signals in the sixth embodiment is realized.
Aiso, the coding and the decoding methods shown in the above-mentioned first to sixth embodiments can be implemented

in mobile communication instruments such as cellular ohones and car navigation systems and cameras sucn as digital video cameras or digital steel cameras by using semiconductors such as LSI. Also, there are 3 types of apparatuses suitable for the implementation: a sending and receiving type terminal with both of a coding and a decoding apparatuses, a sending terminal with a coding apparatus only and a receiving terminal with a decoding apparatus only.
(Seventh Embodiment)
Pictures that should be referred to by the pictures to be decoded are shown in a form of picture numbers (PN). Also, errors of picture numbers (PN) can be detected based on the .increase and the decrease in picture numbers (PN).- FIG. 14 shows the processing of checking and correcting errors of picture numbers (PN) based on picture numbers (PN).
First/ picture numbers (PN) are detected in step-20. Next; ' picture types (PicType) are detected in step 21. After that, whether picture numbers (PN) detected in step A2 are continuous or not is judged. When the picture numbers (PN) are continuous in step A2, error checking and correcting processing of picture numbers (PN) is completed. When the picture numbers (PN) are not continuous in step A.2, errors must be corrected in. step A3. Note that the processing of checking the stored maximum PN and reassigning "PN"s can be performed either way of solely done after this error checking and correcting processing or concurrently done with this error checking and correcting processing.
The first conceivable method of error correcting processing in step A3 is requesting for resending the data concerning the picture numbers with errors and following the error checking processing of picture numbers (PN) again after receiving the resent data. However, the cause of the discontinuity in picture

numbers (PN) of S pictures is not a transmission e^ror. That is, as the number of pictures in each stream stored in ihe memory prior to the S pictures may vary in the case where the discontinuity in picture numbers (PN) of S pictures is found, the picture whose picture number (PN) is required for being resent may not exist, and thus it is highly unlikely that a missing picture can be sent. Therefore, the requests for the pictures that cannot be sent are made endlessly, which may be a hindrance in displaying pictures. For this reason, as to countermeasures in the case of trouble in displaying pictures, further expianation will be made in a tenth embodiment below.
Also, if the number of pictures in the stream after switching in the memory at the time of switching streams is not the same as that of pictures in the original stream, displaying the pictures may not be done properly.
First, there are memories of an FIFO memory for short-time storage and a memory for long-time storage that cerv directly specify storage places without employing "first-in, first-out" so as to store pictures longer than the above-mentioned memory for short-time storage. When the memory for short-time storage can store 7 pictures and the memory for long-time storage can store 4 pictures, reference picture are specified based on the order calculated from the memory for short-time storage. For example, used L.T2 can be specified as the eighth picture (Idx = 7) in the memory for long-time storage. In this way, reference pictures are specified based on the relative positions.
When there are 3 streams as shown in FIG. 7, the locations in the memory to specify the identical pictures (such as S pictures shown in FIG. 7) vary from stream to stream as shown in FIG. 2. And, when referring to pictures in another stream from S pictures, the locations in the memory to specify reference pictures vary according to the memory in. each stream. When there exist a

plurality of streams at the time of switching streams from a predetermined one to another stream, S pictures are those pictures whose precedent pictures in the streams before and after switching are identical.
Not only S pictures but also I pictures can be the pictures at which streams are switched when the pictures decoded in a plurality of streams in the reference memory are exactly the same, and thus I pictures can be used for the same purpose as S pictures (for switching streams).
Considering various conditions like this, it is difficult to specify reference pictures correctly if the number of pictures varies from stream to stream, and it is highly likely that there emerges any error even when the reference pictures are specified.
Therefore, this seventh embodiment will show the coding and decoding methods of additional information used for avoiding an endless error checking processing of picture numbers (?FM) triggered by problems like discontinuity of picture numbers (PN) or incongruity in the content of the memory. This additional information (all picture delete information) is the order for showing that pictures except I pictures and S pictures to be coded must be deleted from the memory for reference in coding and decoding in order to prevent any error from occurring in the process of coding pictures after coding I pictures capable of intra-picture coding and the above-mentioned S pictures.
This process makes each memory condition identical in a plurality of streams after switching streams from a predetermined stream to another stream and makes it possible to specify predetermined pictures in the memory correctly even when reference pictures are needed for Intra-predictive coding and so on. Also, preventing discontinuity in picture numbers (PN) from being detected and corrected as an error resolves the problem of hindering decoding caused by requests for resending nonexistent

pictures,
The coding method will be explained below in FIG. ISA. FIG. 16A shows the making processing of coded signals in this seventh embodiment.
First, picture numbers (PN) are detected in step 20. Next, picture types (PicType) are detected in step 21. Whether the picture types detected are I pictures or not is judged in step Al. When the picture types detected are I pictures, all the pictures except I pictures to be coded in the memory are deleted in step A10. In the following step All, ai! picture delete information meaning deleting all the pictures in the memory is coded, and this is the last procedure of coding additional information.
Also, step Al in FIG. ISA can be used as a step for judging whether the picture types are S pictures or not in the same coding processing as shown in FIG. 16B. Also, it is possible to combine step Al with step A2 and judge whether the picture types are I pictures or S pictures after checking the picture types in'step 21.
As shown in FIG. 17A, when the picture types to be coded are proved to be I pictures in step Al at the time of checking the picture types in step 21 and when the picture numbers (P!M) are judged to be discontinuous after the judgment on whether the picture numbers are continuous or not like the judgment made in step A3, it is also possible to delete all the pictures except I pictures to be coded in the memory. On the other hand, when picture numbers (PN) are continuous in step A3, the pictures in the memory are not deleted. When checking S pictures as picture types, the same explanation shown in FIG. 17A holds true. Also, it is possible to combine step Al with step A2 and judge whether the picture types are I pictures or S pictures after checking the picture types in step 21.
It is also possible to perform the processing of step 30 judging whether the numbers of pictures stored in the memory are

the same or not so as to avoid the occurrence of errors caused bv the difference in the number of pictures stored in the memory as shown in FIG, 173 after the processing of step A3 shown in FIG. I7A. It is possible to perform tne processing of step A30 before going on to the processing of step A3 shown in FIG. 17. And it is possible to delete all the pictures after performing Step A30 before performing Step A3 in FIG. 17B when the numbers of pictures vary from stream to stream, and it is possible to delete all the pictures shown in step A10 when the numbers of pictures do not vary from stream to stream and the picture numbers (PN) are discontinuous. (FIG. 18)
In this way, the processing of FIG. 17 makes it possible to keep storing the pictures that may be reference pictures in the memory as many as possible and improve the reproducibility of pictures decreasing errors. As to I pictures or S pictures, they need no error correction when the numbers of pictures stored in the memory-vary or the picture-numbers (PN) are discontinuous/ which simplifies memory control in the coding apparatus.
Note that specifying I pictures and ordering deletion of all the pictures in the memory can be shown by picture types that specifies special pictures 3S I pictures.
(Eight Embodiment)
FIG. 19 is a block diagram showing the structure of the picture coding apparatus of the present invention. The block diagram of the picture coding apparatus in the present invention shown in FIG. 19 is an example for realizing the picture coding method in FIG. 16.
Picture number generating unit (PNGen) generates picture numbers (PN). Picture numbers (PN) are IDs that identify pictures stored in the reference picture memory (Mem), and each different picture stored in the reference picture memory (Mem) is

given an exclusive picture number (PM). Usually, pictu-e numbers (P\;) are incremented by 1 each time a picture is stored in the reference picture memory. If picture numbers (PN) received in the picture decoding apparatus are incremented by 2 or more, it is possible to detect the lack of pictures to be stored in the transmission line by the picture decoding apparatus and carry out error correction processing such as picture improvement (making the error less conspicuous) or error correction (retransmitting the picture with no errors to reproduce the picture).
When the picture type information (PicType) shows that the
pictures concerned are S pictures (corresponding to the
processing of step A2 in FIG. 16), picture deleting unit 3 (PicDel)
orders the reference picture memory (Mem) to delete the.pictures
stored in the reference picture memory (Mem) except pictures to
be coded, and sends the information to variable length coding unit
(VLC) at the same time. ------ ...
On the other hand, the picture type information (PicType) shows that the pictures concerned are I pictures (corresponding to the processing of step Al in FIG. IS), picture deleting unit (PicDel) orders the reference picture memory (Mem) to delete the pictures stored in the reference picture memory (Mem) except pictures to be coded, and also sends the information to variable length coding unit (VLC) at the same time.
Picture encoding unit (PicEnc) refers to the pictures stored in the reference picture memory (Mem) so as to code the input picture signal (Vin) including frequency conversion and quantization as picture types shown by picture type information (PicType) and send the result to the picture decoding unit (PicDec) or the variable length coding unit (VLC).
Picture decoding unit (PicDec) inversely quantizes and converts the frequency of the information coded in the picture

encoding unit (Pictnc) as picture types shown as picture type information (PicType), and stores the information in the reference picture memory (Mem) as picture numbers (PN) to refer so as to the picture numbers in the following picture coding.
Variable length coding unit (VLC) performs variable length coding on the information coded in picture encoding unit (PicEnc) so as to make a bit stream, and codes necessary information for decoding such as the information for deleting pictures stored in the reference picture memory (Mem) notified by picture deleting unit 3 (PicDel), picture numbers (PN), and picture type information (PicType) to output the information as coded signals (Str).
The structure of the coded signals (Str) in the present invention will be shown in FIG. IOC and 10D. The data will be explained below.
First, Picture numbers (PN) are coded. Next., information for deleting pictures stored in the reference picture memory (Mem), then picture type information (PicType) and picture coding data outputted by the in picture encoding unit (PicEnc) are iocated.
As FIG. IOC is merely an example of data location, it is possible to switch data coding orders as shown in FIG. 10D.
The above-mentioned processing enables the picture coding apparatus to realize the picture coding method shown in FIG. 16 and provide a coding apparatus with high error-resistance.
(Ninth Embodiment)
FIG. 20 is a block diagram showing the structure of the picture coding apparatus in the present invention. The block diagram of the picture coding apparatus of the present invention shown in FIG. 20 is an example for realizing the picture coding method in FIG. 17. Explanation concerning the same units as

explained in FIG. 19 will be omitted from the following explanations.
FIG. 20 and FIG. 19 differ in the process in picture deleting unit 4 (PicDei). To be specific, when the picture type information {PicType) shows that the pictures concerned are S pictures (corresponding to the processing of step A2 in FIG. 17) and the number of pictures varies from stream to stream when comparing the numbers, (corresponding to the processing of step A30 in FIG. 17), picture deleting unit 4 (PicDe!) orders the reference picture memory (Mem) to delete the pictures stored in the reference picture memory (Mem) except pictures to be coded, and sends the information to variable length coding unit (VLC) at the same time. The same is done in the case of I pictures shown by picture type information (PicType). The structure of coded signals of the present invention is the same as the ones shown in FIG. IOC and 10D.
The above-mentioned processing enables the picture coding apparatus to realize the picture coding method shown in FIG. 17 and provide the coding apparatus with high . error-resistance.
(Tenth Embodiment)
The seventh embodiment above showed that picture display may be hindered because requests for resending pictures that cannot be resent are repeatedly made when discontinuity of picture numbers occurred at an S picture. The problem-solving method for the example of picture display with troubles caused by this reason will be explained below.
FIG. 21A shows the processing of decoding the coded pictures.
First, picture numbers (PN) are detected in step 20. Next,
picture types (PicTy :ected in step 21, Whether the

picture types detected are I pictures or not is judged in step Al. When :he picture types detected are not I pictures, whether the picture numbers (PN) are continuous or not is judged in step A3. On the other hand, when the picture types detected are I pictures, there is no need to detect or correct errors and a series of processing finishes.
When the picture numbers (PN) are not continuous in step A3, errors are corrected in step A4. On the other hand, the picture numbers (PN) are continuous in step A3, error check and correction are completed.
Error correction in step A4 can be, for example, processing of detecting the stored maximum picture numbers (PN) as explained in the above-mentioned embodiment, or deleting all the pictures in the memory after receiving all picture delete information meaning deleting ail the pictures in the memory in the processing of reassigning picture numbers (?N).
As shown in FIG. 21B, of step AI in FIG. 21A, the same coding processing can be performed as a step of judging whether the picture types are S pictures or not. Also, it is possible to combine step Al with step A2 and judge which picture types of I pictures or S pictures they have after checking the picture types in step 21.
In this way, it is possible to avoid hindering decoding as a result of repeating requests for resending the missing pictures so as to correct errors when picture numbers of I pictures or S pictures are not continuous. This processing at an I picture is especially useful in the case of a special I picture capable of switching streams.
(Eleventh Embodiment)
FIG. 22 is a block diagram showing the structure of the picture decoding apparatus of the present invention. The block

diagram of picture decoding apparatus of the present invention shown in FIG, 22 shows an example for realizing the picture decoding method shown in FIG. 21. Explanation concerning the same units as explained in FIG. 11 will be omitted from the following explanations.
The different point of FIG. 22 from FIG, 11 is the processing in the error checking unit (ErrChk) using picture types (PicType) by the picture number checking unit (PNchk). To be specific, when picture numbers (PN) to be inputted in picture number checking unit (PNchk) are not continuous and picture types are neither I pictures nor S pictures, an error correction order (Err) is outputted by the error checking unit (ErrChk). With an error correction order, processing such as the stored maximum PN check, PN reassignment or deleting all the pictures in the memory according to all picture delete information meaning deleting all the pictures in the memory.
The above-mentioned processing enables the picture coding " apparatus to realize the picture coding method shown in FIG. 21 and provide a decoding apparatus with high error-resistance.
(Twelfth Embodiment)
This embodiment explains another countermeasure against an endless error check of picture numbers (PN) caused by the troubles such as discontinuity of picture numbers (PN) or incongruity in memory contents. This twelfth embodiment differs from the seventh embodiment in that the picture numbers (PN) are reassigned from "0" when all the pictures are deleted after the same step of deleting ail the pictures in coding shown in the seventh embodiment.
This process makes the respective memory conditions in a plurality of streams identical after switching streams from a predetermined stream to another stream, and initializes the

picture numbers (PN). and thus it makes It possible to correctly specify the predetermined pictures in the memory even when reference pictures are required in intra-predictive coding and so on. It is also possible to resolve the problem of hindering decoding when switching the coded streams to be decoded from the predetermined stream to another stream by avoiding correcting any picture number discontinuity as errors.
As explained, each picture in the stream is given an exclusive picture number (PN) continuous in display time order in the coded streams gotten by coding moving pictures. The reason why the picture numbers (PN) are given exclusive picture numbers (PN) continuous in display time order is that it makes it possible to check the lack of pictures in the coded streams caused by an error in transmission line in the case where the picture decoding apparatus receives the coded streams by way of the transmission line. When a picture number (PN) of a picture to be Inputted in -oisplay time order is incremented by 2 or more while the received coded stream is being decoded, this picture decoding apparatus can check the transmission error occurred just before the receiving time of the picture concerned and request the sender to resend the missing pictures. Therefore, as long as the picture decoding apparatus is decoding one coded stream continuously, it can check transmission errors effectively and receive the resent missing pictures to decode the coded streams perfectly.
However, this error check causes a glitch of endless error checking processing, in the case of the picture decoding apparatus used for continuing decoding after switching to another coded stream with a different picture rate while decoding a coded stream after inputting a plurality of coded streams gotten by coding the same.moving picture at a different picture rate. The cause of this glitch is that picture numbers (PN) of pictures except the first picture in each stream vary among coded pictures with

different picture rates even in the case of the picture's to be displayed at the same time, In other words, picture numbers (PN) are continuous within each coded stream in the display time order. Therefore, when the decoding target is switched to another stream in the middle of decoding one coded stream in a picture decoding apparatus, picture numbers (PN) are discontinuous even in the case of the pictures to be displayed at the same time. The coding method using additionaf information (all picture delete information) has been explained in the seventh embodiment so as to avoid the endless error check of picture numbers (PN) caused by the problems such as discontinuity of picture numbers (PN) or incongruity of memory contents in this way. This additional information is the order for deleting all the pictures except pictures to be coded from the memory for reference in coding or decoding so that any error should not occur at the time of switching streams in the process of picture coding after coding I pictures for intra-coding or the above-mentioned S pictures. ~~
Coding method will be explained below using FIG. 23. FIG. 23A shows the making processing of coded signals in this twelfth embodiment.
First, picture numbers (PN) are detected in step 1. Next, picture numbers (PN) detected in step 1 are coded in step 2. And, picture types (PicType) are detected in step 3. In the step 3, whether the detected picture types are S pictures or not is judged.
When the detected picture types are S pictures, al! picture delete information meaning deleting all the pictures in the memory is coded in step 5. Next, S pictures are coded in step 6A. And, picture numbers are initialized in step 7, and in the following step 8, all the pictures except S pictures to be coded in the memory are deleted in step 8. Up to this point, the processing of coding additiona! information and initializing picture numbers (PN) finishes.

As picture numbers (PN) are continuous unless the detected picture types are 3 pictures, these pictures are coded in step SB, Put the processing is completed without coding additional information, initializing picture numbers (PN), and deleting all the pictures.
Initialization of picture numbers (PN) in step 7 is such as giving picture number of "0" to the coded S pictures. In other words, initializing picture numbers (PN) of S pictures means giving the pictures after 5 pictures in display time order new picture numbers (for example, PN 1) starting from the picture number (PN 0) of S pictures. Consequently, picture numbers (PN) are initialized after coding S pictures (that is, after coding the picture numbers (PN) of S pictures.).
The judgment on whether the pictures are S pictures or not is made in step 4, the judgment on whether the pictures are I picturesare I pictures or not can be made. Also, when there is a step of deleting all the pictures in step 23A, picture nurnbers (PN) should be initialized concurrently because whether the pictures are I pictures or S pictures or not is not a sole judgmental standard as to whether picture numbers (PN) should be initialized. Also, picture number (PN) coding processing in Step 2 can be performed at any time between picture number checking processing in step 1 and the picture number initializing processing in step 7. Also, it Is possible to initialize picture numbers (PN) in step 7 after deleting all the pictures except S pictures to be coded in the memory in step 8. Also, the processing of coding ali picture delete information meaning deleting all the pictures in the memory in step 5 is the processing after the judgment whether the pictures are S pictures or not and can be performed at any time before the processing shown in FIG. 23A is finished. Also, it is possible not to code additional information by using special picture types (PicType) that include additional information meaning deleting ail the pictures

except the pictures to be coded from the memory for reference in coding or decoding. Reassigning picture numbers (PN) so as to switch streams at S pictures or I pictures is effective, but the effectiveness is not limited to the case where streams are switched at S pictures of I pictures, in other words, picture numbers (PN) can be reassigned in the same way on condition that other pictures such as P pictures are capable of switching streams and that there is a step of deleting all the pictures unnecessary for reference.
FIG. 24 is a block diagram showing the structure of the picture coding apparatus capable of realizing the coding method in the twelfth embodiment,
Picture number generating unit (PNGen) generates Picture numbers (PN). Picture numbers (PN) are IDs that identify the pictures stored in the reference, picture memory (Mem), each picture stored in the reference picture memory (Mem) is given an exclusive picture number (PN). 3asicaMy, picture numbers (PN) are incremented by 1 each time a picture is stored in the reference picture memory (Mem). Also, picture numbers (PN) of- the current S pictures are initialized to "0" after S pictures are coded according to the notification from picture encoding unit (PicEnc).
When picture type information (PicType) shows that the pictures are S pictures (corresponding to the processing of step 3 in FIG-. 23), picture deleting unit 5 (PicDel) notifies the reference picture memory (Mem) of an order of deleting the pictures except the pictures to be coded (all picture delete information) stored in the reference picture (Mem) memory and notifies the variable length coding unit (VLC) of the information at the same time.
The picture encoding unit (PicEnc) refers to the pictures stored In the reference picture memory (Mem) so as to code the input picture signal (Vin) into picture types shown by picture type information (PicType) concurrently performing frequency conversion and quantization and send the result to the picture

decoding unit (PicDec) and the variable length coding unit (VLC). Also, the picture encoding unit (PicEnc) notifies the picture number generating unit 2 (PNGen) of the order of initializing picture numbers (PN) after coding S pictures.
The picture decoding unit (PicDec) inversely quantizes and inversely converts the information coded in the picture encoding unit (PicEnc) into picture types shown as picture type information (PicType), and stores the information in the reference picture memory (Mem) associating with picture numbers (PN) so as to refer to the picture types in the following picture coding.
The variable length coding unit (VLC) performs variable length coding on the information coded in picture encoding unit (PicEnc) so as to make a bit stream, and codes necessary information in decoding such as the information for deleting pictures stored in the reference picture memory (Mem) (that is, all picture delete inform at ion) notified by picture deleting ur.lt 5 (PicDer)--ptctore numbers (PN), and picture type information"""' (PicType) to output the information as coded signals (Str).
Next, decoding method will be explained below using FIG. 233. FIG. 233 shows the decoding processing of coded signals.
First, picture numbers (PN) are decoded in step 9. Next, whether all picture delete information is coded or not is judged in step 10.
When ali picture delete information is judged to be coded in step 10, all picture delete information is decoded in step 11. And, pictures are decoded in step 12A. After that, ali the pictures except the pictures to be decoded in the memory are deleted in step 13, and picture numbers (PN) are initialized in step 14. Up to this point, the processing of decoding additional information and initialization of picture numbers (PN) finishes.
When all picture delete information is judged not to be coded in step 10, pictures are decoded in step 12B, and the

processing of decoding additional information and initializing Picture numbers (PNJ is completed in step 123.
Initialization of picture numbers (PN) in step 14 is such as giving picture number of "0" to the decoded pictures, In other words, when decoding the coded signals coded according to the coding procedure shown in FIG. 23A, initializing picture numbers (PN) of S pictures means giving the pictures after S pictures in display time order new continuous picture numbers starting from the picture number of S pictures.
When there is a step of deleting all the pictures in FIG. 23B, the processing of initializing picture numbers (PN) is necessary, in other words, the judgment on whether the picture numbers should be initialized is not influenced by picture types to be decoded. Also-, the processing of initializing picture numbers (PN) in step 14 can be performed before the processing of deleting all the pictures except the pictures to be codec ;r, the memory in step 13. It is "possible not to code additional Information by usiTig'specia! picture types (PicType) that include additional information meaning deleting all the pictures except the pictures to be decoded from the memory for reference in decoding.
FIG. 25 is a block diagram shewing the structure of the picture decoding apparatus that realizes the decoding method of this twelfth embodiment.
Variable length decoding unit (VLD) decodes coded signals (Str), and outputs various information (such as an order for deleting the pictures stored in the reference picture memory (Mem), picture type information (PicType), picture numbers (PN), information for reassigning picture numbers (PN), and picture data.)
First, the order for deleting the pictures stored in the reference picture memory (Mem) gotten in variable length decoding unit (VLD) (all picture delete information) is sent to

Dicture deleting unit 6 (PicDei). Next, picture deleting unit 5 (PicDet) deletes the specified pictures stored in rhe reference picture memory (Mem).
Picture type information (PicType) gotten in variable length decoding unit (VLD) is sent to picture decoding unit (PicDec) to specify the decoding method.
Picture numbers (PN) gotten in variable length decoding unit (VLD) is sent to the reference picture memory (Mem) to be used as picture numbers (PN) at the time of storing the pictures decoded in the picture decoding unit (PicDec).
All picture delete information gotten in the variable length decoding unit (VLD) is sent to the picture number changing unit 2 (PNchg). Picture number changing unit 2'(PNchg) reassigns (initializes) picture numbers (PN) of the pictures stored in the reference picture memory (Mem) according to the order. To be specific, after all the pictures except the pictures to be decoded (S
pictures) in the reference picture memory (Mem), picture number "
changing unit 2 (PNchg) reads out the picture numbers (PN) of the pictures stored in the reference picture memory (Mem), changes the values of the read-out picture numbers (PN) to "0", and writes the picture numbers (PN) in the reference picture memory (Mem).
Picture data gotten in variable length decoding unit (VLD) are decoded using the decoding method based on the picture type shown by picture type information (PicType) in the picture decoding unit (PicDec). In other words, P pictures and B pictures are decoded referring to the pictures stored in the reference picture memory (Mem) while I pictures are decoded without referring to pictures in the reference picture memory. The decoded picture gotten in this way are stored in the reference picture memory (Mem) and outputted as decoded picture signals (Vout).

The above-mentioned structure makes it possible to "ealize :he picture decoding apparatus to realize the picture decoding method shown in FIG. 23 and provide a decoding apparatus with high error-resistance.
The processing of the coding and the decoding methods ■ shown in this twelfth embodiment makes the memory conditions in a plurality of streams identical after switching streams from a predetermined stream to another stream, and thus makes it possible to correctly specify the predetermined pictures in the memory even when reference pictures are required in intra-predictive coding and so on.
It is possible to change I pictures into special pictures at which streams can be reproduced by deleting all the pictures in the reference memory when .using .1 pictures while the above-mentioned embodiment explains that additional information (ail picture delete information) and picture types -(-PicType) can be coded ali at once. This special I "pictures are called Instantaneous Decoder Refresh (IOR). An IDR picture is effective as a leading I picture of Group of Pictures (GOP) because IDR pictures become a starting position of random access. 5y determining that all the pictures except the current pictures in the memory are deleted and picture numbers (PN) are initialized after coding the current pictures each time these IDR pictures are coded, there is no need to code additional information even when all the pictures except the current pictures in the memory are deleted in the picture decoding apparatus. In this case, the picture decoding apparatus detects IDR pictures in the coded streams based on picture types, deletes ai! the pictures except the current IDR pictures in the memory, and initializes picture numbers (PN) after coding and decoding the current IDR pictures even when any additional information is not coded each time IDR pictures are decoded.

(Thirteenth embodiment)
Moreover, storing programs to realize the structures of the picture coding and the decoding methods shown in the above-mentioned embodiments on a storage medium iike a flexible disc makes it possible to easily perform the processing shown in the above-mentioned embodiments in the independent computer system.
FIG. 26 is an illustration concerning "the storage medium to store the program to realize the coding and the decoding methods shown in the first to the twelfth embodiments mentioned above in the computer systems.
FIG. 26B shows a flexible disc and the front view and the cross-sectional view of the appearance of the flexible disc, and FIG. 26A shows an example of a physical format of a flexible disc as a . recording medium body. A flexible disc (FD) is contained in a case F, a plurality of tracks (Tr) are formed concentrically on the surface of the di-sc from t-he periphery into the inner radius of-the - " ' disc, and each track'is divided into 16 sectors (Se) in the angular direction. Therefore, in the case of the flexible disc storing the above-mentioned program, the picture coding method and the picture decoding method as the program is recorded in an area allocated for it on the flexible disc (FD).
Also, FIG. 26C shows the structure for recording and reading out the program on the flexible disc (FD). When the program is recorded on the flexible disc (FD), the computer system (Cs) writes in the picture coding method or the picture decoding method as a program via a flexible disc drive. When the picture coding method and the decoding method mentioned above are constructed in the computer system by the program on the flexible disc, the program is read out from the flexible disc drive and transferred to the computer system.
The above explanation is made using a flexible disc as a

recording medium., but the same processing can also be performed jsing an optical disc. In addition, the recording medium Is not limited to flexibie discs and optica! discs, in other words, any other medium capable of recording a program such as DC-ROMs, memory cards, and ROM cassettes can be used.
Here, the applications of the picture coding method and the picture decoding method shown in the above-mentioned embodiment and the system using them will be further explained.
FIG. 27 is a block diagram showing the overall configuration of a content supply system exlOO for realizing content distribution service. The area for providing communication service is divided into cells of desired sizes, and cell sites exl07~exll0 of fixed wireless stations are placed in the respective cells.
This content supply system_. _exl00 is connected =to apparatuss such as a computer exlll, a Personal Digital Assistant (PDA) exl 12. 3 camera exl13, a ceil phone exl 14 and a cell phone with a camera exll5 via, for example, a combination of the Internet exlQl, an Internet service provider ex!02, a telephone network exl04 and cell sites exlQ7-~exllO.
However, the content supply system exlOO is not limited to the configuration as shown in FIG. 27, and may be connected to a combination of any of them. Also, each apparatus can be connected directly to the telephone network exl04, not through the ceil sites as fixed radio stations exl07~exll0.
The camera exil3 is a apparatus capable of shooting video (moving pictures) such as a digital video camera. The cell phone can be a cell phone of a Personal Digital Communication (PDC) system, a Code Division Multiple Access (CDMA) system, a Wideband-Code Division Multiple Access (W-CDMA) system or a Global System for Mobile Communications (GSM) system, a Personal Handy-phone system (PHS) or the like.
A streaming server exl03 is connected to the camera exll3

via the telephone network exlC^ and the cell site ex!09, which enables live distribution or "he like using the camera ex 113 based on the copied data transmitted from the user. Either the camera ex!13 or the server for transmitting the data can code the shot data. Also, the moving picture data shot by a camera exll6 can be transmitted to the streaming server exl03 via the computer exlll. The camera exll6 is a apparatus capable of shooting still and moving pictures such as a digital camera. Either the camera exllo or the computer exlll can code the moving picture data. An LSI ex 117 included in the computer exlll or the camera exll6 performs coding processing. Software for coding and decoding pictures can be integrated into any type of storage media such as CD-ROMs, fiexible discs and hard discs) that is a recording medium which is. readable by the computer exlll or the like. Furthermore, a cell phone with a camera exll5 can transmit the moving picture data. This moving picture data is the data coded by the LSI inciuded-in the cell phone ex!15.
The contents supply system exlOO codes contents (such as a music live video) shot by users using the camera exl!3, the camera exll6 or the like in the same manner as the above-mentioned embodiments and transmits them to the streaming server exl03, while the streaming server ex!03 makes stream distribution of the contents data to the clients upon their request. The clients include the computer exlll, the PDA exll2r the camera exll3, the cell phone exll4 and so on that are capable of decoding the above-mentioned coded data. In the contents supply system exlOO, the clients can thus receive and reproduce the coded data, and further can receive, decode and reproduce the data in real time so as to realize persona! broadcasting in this way.
When each apparatus in this system performs coding or decoding, the picture coding apparatus or the picture decoding apparatus can be used, as shown in the above-mentioned

embodiments,
A cell phone will be explained as an example of ;he apparatus.
FIG. 28 is a diagram showing the cell phone ex!15 using the picture coding method and the picture decoding method explained in the above-mentioned embodiments. The celi phone exil5 has an antenna ex201 for communicating with the ceil site exllQ via radio waves, a camera unit ex203 capable of shooting moving and stitl pictures such as a CCD camera, a display unit ex202 such as a liquid crystal display for displaying the data obtained by decoding pictures and the like shot by the camera unit ex203 and received by the antenna ex201, a body unit including a set of operation keys ex204, a voice output unit ex208 such as a speaker for outputting voices, a voice input unit 205 such as a microphone for inputting voices, a storage medium ex207 for storing coded or decoded data such as data of moving or still pictures shot by the c-3frner3,-data of received e-mail and data of moving or stilS-pietu-resy - ■ and a slot unit ex206 for attaching the storage medium ex207 to the celi phone exllS. The storage medium ex207 is Equipped with a flash memory element, a kind of Electrically Erasable and Programmable Read Only Memory (EEPROM) that is an electrically erasable and rewritable nonvolatile memory, in a plastic case such as SD cards.
Next, the cell phone exll5 will be explained with reference to FIG. 29. In the cell phone exllS, a main control unit 6x311 for overall controlling each unit of the body unit including the display unit ex202 and operation keys ex204 is connected to a power supply circuit unit ex310, an operation input control unit ex304, a picture coding unit ex312, a camera interface unit ex303, an Liquid Crystal Display (LCD) control unit ex302, a picture decoding unit ex309, a demultiplexing unit ex308, a recording and reproducing unit ex307, a modem circuit unit ex306 and a voice

processing unit ex305 to each other via a synchronous bus ex313.
When 3 call-end key or a sower key is turned ON by 3 user's operation, the power supply circuit unit ex310 supplies respective components with power from a battery pack so as to activate the digital cell phone with a camera exllS for making it into a ready state.
In the cell phone exll5, the voice processing unit ex3Q5 converts the voice signals received by the voice input unit ex205 in conversation mode into digital voice data under the control of the main control unit ex311 including a CPU, a ROM and a RAM, the modem circuit unit ex306 performs spread spectrum processing of the digital voice data, and the communication circuit unit ex301 performs digital-to-analog conversion and frequency transform of the data so as to transmit it via the antenna ex201. Also, in the cell phone exllS, the communication circuit unit ex301 amplifies the data received by the antenna ex201 in conversation mode and performs frequency transform and anaLo.g-10-ciigitai conversion for the data, the modern circuit unit ex306 performs inverse spread spectrum processing of the data, and the voice processing unit ex305 converts it into analog voice data so as to output it via the voice output unit 208.
Furthermore, when transmitting e-mai) in data communication mode, the text data of the e-mail inputted by operating the operation keys ex204 on the body unit is sent out to the main control unit ex311 via the operation input control unit ex304. In the main control unit ex311, after the modem circuit unit ex306 performs spread spectrum processing of the text data and the communication circuit unit ex301 performs digital-to-analog conversion and frequency transform for it, the data is transmitted to the cell site exllO via the antenna ex201. When picture data is transmitted in data communication mode, the moving picture data shot by the camera unit ex203 is

supplied to the picture coding unit ex312 via the camera interface unit ex303. When the picture data is not transmitted, it is also possible to display the picture data shot by the camera unit ex203 directly on the display unit 202 via the camera interface unit ex303 and the LCD control unit ex302.
The picture coding unit ex312, which includes the picture coding apparatus as explained in the present invention, compresses and codes the picture data supplied from the camera unit ex203 using the coding method used for the picture coding apparatus as shown in the above-mentioned embodiments so as to transform it into coded picture data, and sends it out to the demultiplexing unit ex308. At this time, the cell phone exll5 sends out the voices received by the voice input unit ex205 during shooting by the camera unit ex203 to the demultiplexing unit ex308 as digital voice data via the voice processing unit ex305.
The demultiplexing un';t ex303 multiplexes the coded picture data supplied from the picture coding unit ex312and the - - -voice data supplied from the voice processing unit ex305 using a predetermined method, the modem circuit unit ex30S performs spread spectrum processing of the multiplexed data obtained as a result of the multiplexing, and the communication circuit unit ex301 performs digital-to-analog conversion and frequency transform of the data for transmitting via the antenna ex201.
As for receiving data of a moving picture file which is linked to a Web page or the like in data communication mode, the modem circuit unit ex306 performs spread spectrum processing of the data received from the cell site exilO via the antenna ex201, and sends out the multiplexed data obtained as a result of the processing to the demultiplexing unit ex308.
In order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into a bit stream of picture data and a bit stream

of voice data, and supplies the current coded picture data to the picture decoding unit ex309 and the current voice data to the voice processing unit ex3C5 respectively via the synchronous bus ex313.
Next, the picture decoding unit ex309, which includes the
picture decoding apparatus as explained in the present invention,
decodes the bit stream of picture data using the decoding method
corresponding to the coding method as shown in the
above-mentioned embodiments to generate reproduced moving
picture data, and supplies this data to the display unit ex202 via
the LCD control unit ex302, and thus the moving picture data
included in a moving picture file linked to a Web page, for instance,
is displayed. At the same time, the voice processing unit ex305
converts the voice data into analog voice data, and supplies this'
data to the voice output unit ex208, and thus voice data included
in a moving picture file linked to a Web page, for instance, is
reproduced. ...
The present invention is not limited to the above-mentioned system, and at least either the picture coding apparatus or the picture decoding apparatus in the above-mentioned embodiments can be incorporated into a digital broadcasting system as shown in FIG. 30. Such ground-based or satellite digital broadcasting has been in the news lately. More specifically, a bit stream of video information is transmitted from a broadcast station ex409 to a communication or a broadcast satellite ex410 via radio waves. Upon receipt of it, the broadcast satellite ex410 transmits radio waves for broadcasting, a home-use antenna ex406 with a satellite broadcast reception function receives the radio waves, and a television (receiver) ex401 or a set top box (STB) ex4G7 decodes and reproduce the bit stream. The picture decoding apparatus as shown in the above-mentioned embodiments can be implemented in the reproduction apparatus ex403 for reading off

and decoding the bit stream recorded on a storage medium ex402 ?h3t Is a recording medium such £s a CD and a DVD. In inls case. ttie reproduced moving picture signals are displayed on a monitor ex404. It is also conceived to implement the picture decoding apparatus in the set top box ex407 connected to a cable ex405 for a cable television or the antenna ex406 for satellite and/or ground-based broadcasting so as to reproduce them on a monitor ex408 of the television. The picture decoding apparatus may be incorporated into the television, in stead of in the set top box. Or, a car ex412 having an antenna ex411 can receive signals from the satellite ex4I0 or the cell site exl07 for reproducing moving pictures on a display apparatus such as a car navigation system ex413.
Furthermore, the picture coding apparatus shown in the above-mentioned embodiments can code picture signals for recording on a recording medium.. As a concrete example, there is 3 recorder ex420 such 3S a DVD recorder for recording picture signals on a DVD disc ex421 and a disc recorder for recording them on a hard disc. They can be recorded on an SD card ex422. If the recorder ex420 includes the picture decoding apparatus shown in the above-mentioned embodiment, the picture signals recorded on the DVD disc ex42l or the SD card ex422 can be reproduced for display on the monitor ex408.
Note that a conceivable structure of the car navigation system ex413,is the structure without the camera unit ex203, the camera interface unit ex303 and the picture coding unit ex312 that are existing components in FIG. 29. The same goes for the computer exlll, the television (receiver) ex401 and the like.
In addition, three types of implementations can be conceived for a terminal such as the above-mentioned cell phone exll4; a sending/receiving terminal implemented with both an encoder and a decoder, a sending termina! implemented with an

encoder only., and a receiving terminal implernented with 3
decoder only. ;.
As described above, it is possible to use the picture coding method or the picture decoding method in the above-mentioned embodiments in any of the above-mentioned apparatuss and systems, and by using this method, the effects explained in the above embodiments can be obtained.
From the invention thus described, it will be obvious that the embodiments of the invention may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended for inclusion within the scope of the following claims.
Industrial Applicability
The picture coding apparatus concerning "he present invention is useful as the picture coding apparatus- instaHed'-in personal computers with communication" functions, PDAs, digital broadcasting stations and cellular phones.
Also, the picture decoding apparatus concerning the present invention is useful as the picture decoding apparatus installed in personal computers with communication functions, PDAs, STBs receiving digital broadcasting and celiular phones.
DESCRIPTION
Technical Field
The present invention relates to a picture coding method for efficiently compressing moving picture signals using correlation between pictures, a picture decoding method for decoding the signals correctly, and a recording medium on which a program for executing these methods using software.
Background Art
Recently, the multi-media era has come in which sound, pictures and other pixel values are integrated into one media, and conventional information media as communication tools like newspapers, magazines, TV, radio and telephone are regarded as the targets of multi-media. Generally, multi-media is a form of simultaneous representation of not only characters but also graphics, sound, and especially pictures. In order to handle the above-mentioned conventional information media as multi-media, it is a requiste ~o represent the information digitally.
However, it is unrealistic to directly process a huge amount of information digitally using the above-mentioned conventional information media because, when calculating the data amount of each information medium mentioned above as digital data amount, data amount per character is 1 ~ 2 byte while that of sound per second is not less than 64 kbits (telephone speech quality) and that of moving pictures per second is not less than lOOMbits {present TV receiving quality). For example, a TV telephone has already become commerda))y practical thanks to Integrated Services Digital Network (ISDN) with a transmission speed of 64kbps ~ 1.5 Mbps, but it is impossible to transmit pictures of TV camera as they are using ISDN;

That is why information compression technique is necessary. For example, a moving picture compression technique standard of H.261 or H.263 internationally standardized by International Telecommunication Union-Telecommunication Standardization Sector (ITU-T) is used for TV telephones. Also, it is possible to store picture information with sound information in ordinary music compact discs (CDs) using the information compression technique of the MPEG-1 standard.
Here, Moving Picture Experts Group (MPEG) is an international standard to digitally compress moving picture signals, and MPEG-1 is the standard to compress moving picture signals to 1.5Mbps, that is, compress TV signal information to about one hundredth. Also, the quality that satisfies the MPEG-1 standard is medium level that can be realized at a transmission rate of about 1.5Mbps. MPEG-2 is thus standardized in order to satisfy the need for hioner picture ousiitv, and it compresses moving picture signals to 2 - 15Mbps.
At present, the work group (ISO / IEQJTC1 / SC29 / WG11), which standardized MPEG-1 and MPEG-2, has standardized MPEG-4 with a higher compression rate. MPEG-4 introduced not only efficient coding at a low bit rate, but also a powerful error-resist technique that lessens subjective picture deterioration in case a transmission error occurs. Also, as a picture coding system for the next generations, ISO / IEC and ITU-T are jointly working for the standardization of Joint Video Team (JVT). At present, the standard called Joint Model 2 (JM2) is the latest version.
The picture for intra predictive coding without any reference picture is called Intra Coded Picture (I picture). Also, the picture for inter predictive coding with a reference picture is called Predictive Coded Picture (P picture). Also, the picture for inter predictive coding in which two reference pictures are referred to

simultaneously is called 3i-predictive Coded Picture (B picture).
"Picture" used here is a term representing one picture. In a progressive picture, a picture means a frame, but in an interlace picture, it means a frame or a field. An "interlace picture" mentioned here means a frame composed of two fields with a slight time lag. In the coding and decoding processes of interlace pictures, it is possible to process a frame as it is, as two fields, or by each block in a frame in a frame-by-frame structure or in a field-by-field structure.
In JVT, it is possible to choose an arbitrary picture as a forward reference picture from a plurality of pictures, unlike conventional moving picture coding. Also, a system to switch coded bit streams at specific pictures, that is, Switching Coded Pictures (S pictures) has been introduced. (There are SI pictures and SP pictures in S pictures, and these are the pictures for intra predictive coding or inter predictive ceding respectively.)
S picture system is to guarantee that streams after S Pictures can be decoded correctly in the case of switching from stream to stream just before S pictures. Also, it is possible to switch streams at a server such as a moving picture distribution server according to the communication capacity of receiving terminals or preference of receivers.
In the conventional picture coding method or picture decoding method, S pictures are introduced so that (1) it can choose an arbitrary picture as a forward reference picture from a plurality of pictures, and (2) it can also switch pictures at specific pictures. In spite of the introduction of these two techniques, regrettably, problems that occur when these two techniques are combined have not yet considered well. In reality, it is difficult to use both the techniques together because of the problems shown below.
FIG. 1 is an illustration showing the relations between

pictures and picture numbers (PN) when coding an input picture signal (VIN). The same picture signal is coded at different picture rates (the number of pictures per second) to make Streams 1, 2 and 3. Picture numbers (PN) are numbers to Identify coded pictures. In 3M2, pictures to be referred to as reference pictures in the foliowing coding are assigned numbers incremented by i. To simplify the explanation, the example of FIG. 1 shows only the case that all pictures in each stream are referred to as reference pictures in the following coding, and the picture numbers are always incremented by 1, Pictures that are not referred to in the following coding are unrelated to the increase or decrease in the picture numbers, and not stored in a memory. Therefore, explanation as to pictures that are not referred to in the following coding is omitted because the pictures are unrelated to the following explanation of operations.
As shown in FIG. 1, at the time cf:3, the diagonally shaded pictures are coded as 5 pictures, FIG. 2 is a diagram showing picture numbers (PN) of the pictures to be stored in the reference picture memory when coding or decoding S pictures.
FIG. 2 shows pictures stored in the reference picture memory (Mem) and their positions. In the reference picture memory (Mem), pictures in the left position are newer in time than pictures in the right position. At the time of predictive coding, the same pictures must be referred to in coding and decoding. When it is possible to choose each reference picture from a plurality of reference pictures like in the JM2, it is necessary to specify which pictures are referred to.
There are two following methods to show reference pictures, and 3M2 uses the two methods properly according to purposes, (p Clearly expressing how many pictures there are prior to a newer
picture (2) Clearly expressing a reference picture by a picture number (PN)

In order to correctly code S pictures and the following pictures and to decode tnese pictures correctly at ihe time of decoding when switching streams at S pictures, the contents of the reference picture memory (Mem) must be the same in every case of switching streams at S pictures,
However, as shown in the illustration of FIG. 2 showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem), the contents of the reference picture memory (Mem) are not the same at the start of coding or decoding an S picture in each stream. Unless such a conventional method is improved, it is impossible to use the coding method to choose reference pictures from reference picture memory (Mem) in combination with the S picture system for switching streams.
The present invention aims at solving all the above-mentioned problems, making the S picture system usable in combination with the other coding method to choose reference pictures in reference picture memory (Mem), and thus providing picture coding and decoding methods that improve compression rates in the above-mentioned combined coding method using S pictures
Disclosure of Invention
In order to resolve this problem,
The first invention is a picture coding method for specifying a reference picture stored in a memory using a picture number for reference and generating a coded stream of moving pictures, the picture coding method comprising a picture number coding step of coding picture numbers corresponding to the current pictures, a coding step of coding the current pictures, a step of making all the pictures stored in the memory except the current pictures released for reference after the picture coding step, a picture number initializing step of initializing the picture numbers of the current

pictures in the memory, and an a!! picture "eiease Information coding step of coding aii picture release information for instructing a picture decoding apparatus to release ai! the pictures that have been already stored in the memory except the current pictures.
The second invention is a picture decoding method for specifying a reference picture stored in a memory using a picture number for reference and decoding the coded stream of moving pictures, the picture decoding method comprising an all picture release information decoding step of checking and decoding an all picture release information meaning releasing all the pictures in a coded stream stored in a memory except the current pictures to be decoded, a decoding step of decoding the current pictures in the coded stream, a step of releasing ali the pictures stored in the memory except the cur-rent pictures according to the decoded all picture release information after the decoding step, and a picture number initializing step of assigning an initialized picture number to the current picture in the memory.
The third invention is a picture coding method for coding a plurality of picture signals and generating coded signals corresponding to respective pictures, wherein a switching picture capable of switching a plurality of coded signals and the following pictures of the switching picture can refer to only a group of pictures of same time in the coded signals.
The fourth invention is a picture decoding method for decoding a coded signal, wherein information on pictures released for reference before a switchable switching picture is decoded, decoded pictures in a reference picture memory are released based on the decoding result, and after the switching picture, a coded signal is decoded referring to a reference picture that is not released.
The fifth invention is a picture coding method for coding a plurality of picture signals and generating coded signals

corresponding to respective pictures, wherein the picture coding method has a step of changing a picture number of a switching picture capable of switching a piurality of coded signals into same value at respective one of coded signals.
The sixth invention is a picture decoding method for decoding a coded signal comprising, a step of changing a picture number of a reference picture into same value in a switchable coded signal at the time of switching coded signals at a switchable switching picture.
As mentioned above, with the picture coding method and the picture decoding method in the present invention, it is possible to use the feature of S pictures and a coding method for choosing a reference picture in a reference picture memory in combination, - which makes it possible to provide a picture coding method and a picture decoding method to improve compression rates even when using S pictures in the coding method, and thus these methods are "highly practice!.
Brief Description of Drawings
FIG. 1 is an illustration of relations between pictures and picture numbers (PN) when coding an input picture signal (Vin).
FIG. 2 is a diagram showing picture numbers (PN) of pictures to be stored in a reference picture memory (Mem) when coding and decoding S pictures.
FIG. 3 is an illustration of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem).
FIG. 4A, 4B and 4C are flow charts of how to code and decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture coding method and the picture decoding method of the present invention.
FIG. 5A and 5B are illustrations of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem).

FIG. 6A, 6B and 5C are flow charts of how to code and decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture coding method and the picture decoding method of the present invention.
FIG. 7 is an illustration of the relation between pictures and picture numbers (PN) when coding input picture signal (Vin) of the present invention.
FIG. 8A and SB are flow charts of how to decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture decoding method of the present invention.
FIG. 9 is a block diagram showing the structure of picture coding apparatus of the present invention.
FIG. 10 A, 10B, IOC and-lOD are diagrams showing data structure examples of coded signals, Str of the present invention.
FIG, 11 is a block diagram showing the structure cf a picture decoding apparatus of the present invention.
FIG. 12 is an Illustration of the relation between pictures and picture numbers (PN) for coding an input picture signal (Vin) of the present invention.
FIG. 13 is a flow chart showing a coding method by giving a picture number to each picture of each stream in the present invention.
FIG. 14 is a flow chart showing a decoding method in a seventh embodiment.
FIG. 15 is a diagram showing a memory structure in the seventh embodiment.
FIG. ISA and 16B are flow charts showing a coding method in the seventh embodiment.
FIG. 17A and 17B are flow charts showing another coding method in the seventh embodiment.
FIG. 18 is a flow chart showing another coding method in

the seventh embodiment.
FIG. 19 is a block diagram showing 3 structure of a coding apparatus in an eighth embodiment.
FIG. 20 is a block diagram showing a structure of another coding apparatus in the eighth embodiment.
FIG. 21A and 21B are flow charts showing a decoding method in a tenth embodiment.
FIG. 22 is a biock diagram showing a structure of a decoding apparatus in an eleventh embodiment.
FIG. 23A and 23B are flow charts showing the processing for making coded signals and decoding the coded signals.
FIG. 24 is a block diagram showing a structure of a picture coding apparatus realizing a coding method in a twelfth embodiment..
FIG. 25 is a block diagram showing a structure of a picture decoding apparatus resizing the decoding method in the twelfth embodiment.
FIG. 26 is an illustration as to a recording medium storing a program for realizing the picture coding methods and picture decoding methods in the first to the twelfth embodiments using a computer system.
FIG. 27 is a block diagram showing the whole structure of a content supplying system realizing a content distribution service concerning the present invention.
FIG. 28 is a diagram showing an example of cellular phones relates to the present invention.
FIG. 29 is a block diagram showing the structure of the same celiular phones, and
FIG. 30 is a diagram showing the structure of a digital broadcasting system concerning the present invention.
Best Mode for Carrying Out the Invention

[he present embodiments of the present Invention will be explained below with reference to FIG. 3~3Q.
(First Embodiment)
FIG. 3 is an illustration showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem). The difference between the same figure and FIG. 2 showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem) will be explained below.
It has already been explained that, after switching coded signals when coding and decoding S pictures, the content of the reference picture memory (Mem) is not the same. Therefore, in the coding and the decoding methods of the present invention in FIG. 1 showing the relations of pictures and their picture numbers ■ -(PN) when coding input picture signal (Vin), only pictures at the times of tO. tl and t2 ~nat are exactly the same respectively in an streams are stored in the reference picture memory (Mem), while the other pictures are deleted in the reference picture memory (Mem) before coding and decoding S pictures. FIG. 3 shows the result of this processing as an illustration showing the picture numbers (PN) of the pictures to be stored in the reference picture memory (Mem).
As shown in FIG. 3 explaining picture numbers of pictures to be stored in the reference picture memory (Mem), when the method of "Clearly expressing how many pictures there are prior to a newer picture" is employed to specify reference pictures in coding and decoding, it is possible to code and decode the pictures correctly because exactly the same pictures of the same time are referred to in any case of stream i, 2 or 3.
FIG. 4 is a flow chart showing coding and decoding methods of information concerning picture control of pictures to be stored in the reference picture memory (Mem) in the picture coding

decoding methods of che present invention,
FIG. 4A as a flow chart of the coding method tnat shows how to realize the method of the operation explained in FIG. 3, and the coding and decoding methods of the necessary information for realizing the operation.
In Step 0, pictures at the same points of time in a plurality of coding information (streams) are chosen. In Step 1, it is possible to code delete information showing deletion of the other pictures which are not chosen in Step 0. In Step 2, pictures which are not chosen in Step 0 are deleted from the reference picture memory (Mem). Up to this point, as shown in FIG. 3, it is possible to realize the storage condition in the reference picture memory (Mem) to realize decodable streams even after switching coded signals.
Also, it is possible to change the order of Step 1 and 2, and
if it is changed, the flow chart of the picture coding method shown
-in F-IG. 43 is used. ......
Decoding the delete information coded according to the coding method shown as the flow chart in FIG. 4A using the decoding method shown as the flow chart in FIG. 4C makes it possible to realize the storage condition of the reference picture memory (Mem) to realize decodable streams using the "picture decoding method even after switching coded signals as shown in FIG. 3.
By decoding the delete information in Step 5, it is possible to show pictures that are not the same-time pictures in a plurality of coding information (streams). These pictures are the rest of the pictures chosen in Step 0 of FIG. 4A as pictures.of the same time. Next, in Step 6, pictures chosen in Step 5 are deleted in the reference picture memory (Mem). Specifically, when pictures stored in the reference picture memory (Mem) are deleted (or erased), pictures to be deleted are assigned IDs (identification

information) such as "release" prohibiting from using the pictures concerned as reference pictures. For that purpose, picture decoding unit (PicDec) and picture encoding unit (PicEnc) always check whether IDs of "release" are assigned or not each time these units refer to pictures stored in the reference picture memory (Mem). Only pictures that are not assigned IDs of "release" are referred to by these units whiie pictures assigned IDs of "release" are not referred to by these units. Likewise, pictures in the reference picture memory (Mem) are deleted (or erased) in the following embodiments. Of course, as this deletion method is an example, it goes without saying that it is possible to delete the above-mentioned picture data in the reference picture memory (Mem) by actually deleting or erasing these data. Up to this point, it is possible to realize the storage condition in the reference picture memory (Mem) to realize decodable streams even after switching the coded signals as shown in FIG. 3.
(Second embodiment)
FIG. 5A is an illustration showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem). The difference between FIG. 5A and FIG. 3 showing picture numbers (PN) of pictures to be stored in the reference picture memory (Mem) is whether picture numbers (PN) in the reference picture memory (Mem) are the same or not.
As not only the times of pictures stored in the reference picture memory (Mem), but also picture numbers (PN) in each stream in the reference picture memory (Mem) are the same in the time frame, it is possible to use the method of "Clearly expressing reference pictures by picture numbers (PN)" when specifying reference pictures in coding and decoding, and thus it becomes possible to code and decode the pictures correctly because exactly the same pictures of the same time are referred to in any case of

stream lr 2 or 3,
Ii is possible to realize this when replacing the picture numbers of pictures stored in the reference picture memory (Mem) with a new same picture number, and coding and decoding the information for replacing the old picture numbers with the new one before coding and decoding S pictures.
In addition, there is a need to match picture numbers (PN) of S pictures in any of the streams because the same picture numbers (PN) must be used when storing S pictures next time.
FIG. 6 is a flow chart showing coding and decoding methods of information for controlling pictures to be stored in the reference picture memory (Mem) according to the picture coding and decoding methods of the present invention, and shows the realization methods.of .the operation explained in FIG. 5A and the coding and decoding methods of information necessary for the
re3MZ-3t!OP .
In Step-10, the maximum value of picture numbers (PN) ("8" in the example of FIG. 5A.) of pictures in the coded signals to be switched in the reference memory (Mem) are detected. In Step 12, information for reassigning picture numbers (PN) of each picture stored in the reference picture memory (Mem) is coded with reference to the maximum value of picture numbers (PN). Also, as the need arises, picture numbers (PN) to be assigned to the next S pictures are coded. As stream 3 in FIG. 5A is the same as stream 3 in FIG. 3, there is no need to reassign picture numbers of pictures in stream 3. Therefore, picture numbers are reassigned to necessary pictures only, onty information on necessary reassignment needs to be coded in Step 11. Lastly, picture numbers shown by the coded information in Step 11 is reassigned in Step 12. Up to this point, as shown in FIG. 5, the storage condition in the reference memory (Mem) for realizing decodable streams even after switching the coded signals is
* •

realized.
Also, as the picture number (PN) of 5 pictures are 12, in order to make picture numbers (PN) continuous after coding and decoding these S pictures, it is possible to use picture number 11, that is the picture number (PN) immediately before an S picture (immediately before an S picture of stream 1 in FIG. 1) as shown in FIG. 5B. In this case, picture numbers (PN) always increase in the process of coding and decoding as the picture number (PN) of the S picture is 12, which is more effective because the error check function to regard decrease in picture numbers (PN) as an error is also realized.
FIG. 7 is an illustration showing the relations of pictures and picture numbers (PN) when coding an input picture signal (VIN) of the present invention. FIG. 7 is an. example of reassignment of picture numbers (PN) using the method explained in FIG. 5B, all the picture numbers of S pictures are 12. Therefore, it is clear ■that ail the pictures after S pictures 3re correctly decodable even after switching streams at S pictures because pictures in the reference memory (Mem) are identical irrespective of streams when coding and decoding S pictures. Also, it is possible to change the operational order of Step 11 and 12, and in this case, a flow chart of picture coding method shown in FIG. 6B is used.
Decoding the coded delete information in FIG. 5A shown in the flow chart of the coding method using the decoding method shown in the flow chart in FIG. 6C of the decoding method makes it possible to realize the storage condition of the reference picture memory (Mem) to realize decodable streams using the picture decoding method even after coded signals are switched as shown In FIG. 5A.
Decoding the information on reassignment of picture numbers (PN) in Step 15 makes it possible to specify the pictures necessary for reassignment of nicfurp numbers fPi\n and thp

method. Next, in Step 16.. picture numbers (PN) of pictures in ihe reference picture memory (Mem) is reassigned based on the pictures decoded in Step 15 and also requires for reassignment of picture numbers (PN) and the reassignment method. Up to this point, it is possible to realize the storage condition in the reference picture memory (Mem) to realize decodable streams even after switching the coded signals as shown in FIG. 5.
While this second embodiment explains the effectiveness in combination with the first embodiment, as second embodiment can solely realize the merit of coding and decoding correctly in the case of "Clearly expressing reference pictures by picture numbers (PN)", it is possible to use only second embodiment instead of the second embodiment in combination with the first embodiment if the effect of the second embodiment is sufficiently effective.
(Third Embodiment)
FIG. 8 is another embodiment so as to realize an illustration of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem) in FIG. 5.
Picture types are identified by picture type information (PicType). Therefore, when a picture is identified as an S picture that can change streams by picture type information (PicType), making a rule for reassigning picture numbers (PN) of pictures in the reference picture memory (Mem) to coincide with picture numbers (PN) of S pictures makes it possible to omit coding and decoding the information on reassignment method for each picture number (PN) of pictures in the reference picture memory (Mem).
The operation shown in FIG. 8A will be explained below. In Step 20, picture numbers (PN) of pictures are gotten by decoding coded signals. The picture type information (PicType) of the pictures is gotten in Step 21. When the picture type information (PicType) is proved to be S pictures, picture numbers (PN) of

pictures in the reference picture memory (Mem) in a way that they coincide with picture numbers (PN) of 5 pictures are reassigned based on a specified method in Step 22. Up to this point, as shown in FIG. 5, it is possible to realize the storage condition in the reference picture memory (Mem) that realizes decodable streams even after switching coded signals.
Also, it is possible to change the order of Step 21 and 22, and if it is changed, the flow chart of the picture coding method shown in FIG. 8B is used.
Also, It is possible to code and decode only part of reassignment information of picture numbers (PN) (which cannot be represented by the rule for reassigning picture numbers (PN) in the reference picture memory (Mem) in a way that they coincide with picture numbers (PN) of S pictures) in Step 11 and 15 in FIG. 6 by combining the illustration of FIG. 8 and 6 concerning picture numbers (PN) of pictures to be stored in the reference picture ■ memory (Mem) in FIG. 5.
(Fourth Embodiment)
FIG. 9 is a block diagram showing the structure of the picture coding apparatus of the present invention. FIG. 9 as a block diagram concerning the picture coding apparatus of the present invention is an example for realizing the picture coding method in the first embodiment and the second embodiment.
Picture number generating unit (PNGen) generates picture numbers (PN). Picture numbers (PN) are IDs to identify the pictures stored in the reference picture memory (Mem), each picture stored in the reference picture memory (Mem) is assigned to an exclusive picture number (PN). Usually, picture numbers (PN) are incremented by 1. each time a picture is stored in the reference picture memory (Mem). When a picture number (PN) received by a picture decoding apparatus is incremented by 2 or

more, the picture decoding apparatus can found that 3 picture to be stored is missing because of transmission line error, and can correct the error and make the error less conspicuous.
Maximum picture number checking unit (MaxPN) compares other coded signal picture numbers (OtherPN) and picture numbers (PN) generated.in the picture number generating unit (PNGen), detects the maximum value of picture numbers (PN), notifies variable length unit (VLC) and the picture number generating unit (PNGen) of the maximum value of picture numbers (PN), and initializes picture numbers (PN) to be generated in the picture number generating unit (PNGen) using the maximum value of picture numbers (PN). Other coded signal picture numbers (OtherPN) are picture numbers of pictures in a different stream in parallel with the pictures to.be coded. Consequently, after that,- - --- - • -
the picture number generating unit (PNGen) starts to output Dicture numbers (PN) larger than the maximum value of the picture numbers (PN).
Coded picture time comparing unit (TimeCmp) compares the frame time of each picture in the input picture signal (Vin) coded so far and the frame time of each picture coded as other coded signals (streams), and notifies picture deleting unit (PicDel) of picture information concerning the frame time coded in all the streams.
When picture type information (PicType) shows that next picture is an S picture, picture deleting unit (PicDel) orders reference picture memory (Mem) to delete the pictures outside the time frame in all the streams stored in the reference picture memory (Mem) based on the information notified by the coded picture time comparing unit (TimeCmp), and notifies variable length coding unit (VLC) of the same information at the same time.
Picture encoding unit (PicEhc) refers to the pictures in the reference picture memory (Mem), codes the input picture signa!

(Vin) including frequency conversion and quantization as a picture type snowin in the picture type information (PicType), sends the result to picture decoding unit (PicDec) and variable length coding unit (VLC). Picture decoding unit (PicDec) performs inverse quantization and frequency conversion of the coding result in picture encoding unit (PicEnc) as picture types shown in the picture type information (PicType), and stores the picture types as picture numbers (PN) in the reference picture memory (Mem) so as to refer to the picture types in the following picture coding process.
Variable length coding unit (VLC) makes the coded result in picture encoding unit (PicEnc) into variable length codes so as to make a bit stream, codes the information needed for decoding, -. that is the information for deleting pictures stored in the reference -picture memory (Mem) notified by the picture deleting unit (PicDei). the maximum value of the picture numbers (PN) and picture numbers (PN) mentioned above" so as to output" the" information as coded signals (Str). Variable length coding unit (VLC) also codes information notified by picture deleting unit (PicDei) and information for reassigning picture numbers (PN) of pictures stored in the reference picture memory (Mem) based on the method shown in the second embodiment.
FIG. 10 shows a structural example of coded signals (Str) in the present invention. Each data in FIG. 10A will be explained below.
First, picture numbers (PN) are coded. Next, the maximum picture number (PN) to be reassigned, information for deleting pictures stored in the reference picture memory (Mem) and information for reassigning picture numbers stored in the reference picture memory (Mem) are coded. After that, picture type information (PicType) and picture coded data which is outputted by picture encoding unit (PicEnc) are located.

As FIG, 10A is simply an example of data location, it is possible to change order of data as shown in FIG. 103 so as to perform picture coding.
It is possible to prepare the picture coding apparatus consists of the above-mentioned units that realizes the picture coding method shown in the first and the second embodiments.
(Fifth Embodiment)
FIG. 11 is a block diagram showing the structure of the picture decoding apparatus of the present invention. FIG. 11 as a block diagram concerning the picture decoding apparatus of the present invention is an example of a picture decoding apparatus that realizes the first, the second and the third embodiments. Its function will be.explained below.
Variable length decoding unit (VLD) decodes coded signals (Str), outputs various information (such as an order for deleting pictures stored in the reference picture memory (Mem), picture type information (PicType), picture numbers (PN), information for reassigning picture numbers (PN) and picture data.)
The order for deleting pictures stored in the reference picture memory (Mem) gotten in variable length decoding unit (VLD) is firstly sent to picture deleting unit (PicDei). Picture deleting unit (PicDei) deletes the specified pictures stored in the reference picture memory (Mem).
Picture type information (PicType) gotten in the variable length decoding unit (VLD) is sent to picture decoding unit (PicDec) so as to show the decoding method.
Picture numbers (PN) gotten in variable length decoding unit (VLD) is sent to the reference picture memory (Mem) as picture numbers (PN) when storing the pictures decoded in picture decoding unit (PicDec).
Information for reassigning picture numbers (PN) of

pictures stored in the reference picture memory (Mem) gotten in the variable length decoding unit (VLD) is sent to picture number changing unit (PNchg). Picture number changing unit (PNchg) follows the directions and reassign picture numbers (PN) of pictures stored in the reference picture memory (Mem). To be more specific, picture number changing unit (PNchg) reads out picture numbers (PN) of pictures stored in the reference picture memory (Mem), reassigns the values of picture numbers (PN) read out, and then writes the new picture numbers (PN) in the reference picture memory (Mem).
In the picture decoding unit (PicDec), picture data gotten in variable length decoding unit (VLD) are decoded in a suitable decoding method for the specified picture type shown as picture type information (PicType). In other words, I pictures are decoded without referring pictures in the reference picture memory (Mem) while ? pictures end 3 pictures ere decoded by referring to pictures "in the reference picture memory (Mem). The decoded pictures gotten in this way are stored in the reference picture memory (Mem) and outputted as decoded picture signals (Vout).
Up to this point, it is possible to prepare the picture decoding apparatus consists of the above-mentioned units that realizes the picture decoding method shown in the first, the second, and the third embodiments.
(Sixth Embodiment)
In the picture coding apparatus shown in the first to the fifth embodiments, when switching streams at S pictures, the picture numbers of the pictures before the switch able pictures are switched so as to make the picture numbers continuous with the picture numbers of the switchable pictures. In this sixth embodiment, picture numbers are switched at the switchable pictures.

As to coding of a plurality of streams having 5 different picture rate or a bit rate, or 3 different structure, here is an example of a stream switching method that enables coding after switching coded pictures from a picture under coding in a stream to a picture in another stream. For convenience of explanation, the simpler phrase of "switching streams" is used below.
In addition, in this sixth embodiment, whether pictures to be coded should be stored in the reference memory or not is judged based on the increment in picture number between pictures to be coded and pictures forwardly adjacent to the pictures to be coded (Simply, "the preceding picture" is used below.) in coding order. To be specific, when an increment in picture number between a preceding picture and a picture to be coded is 1, which means the pictures to be coded is stored in the reference memory. When the picture number of the pictures to be coded is the same as the picture number of the preceding pictures, which means the current picture is not stored in the reference memory.
The processing of switching picture numbers (PN) of switchable pictures will be explained concretely with reference to FIG. 12.
FIG. 12 is 3 diagram showing an example of the relations between pictures and picture numbers (PN) when an input picture number (Vin) is coded. An identical picture signal is coded at different picture rates to make streams of 1, 2 and 3. In FIG. 12, pictures are located according to the cording order in each stream.
In stream 1, picture numbers (PN) are assigned to each picture so that the picture numbers are incremented by 1. Also, in stream 2, there are pictures assigned picture numbers (PN) incremented by 1 and pictures assigned the same picture numbers (PN) as the precedent pictures. Also, in stream 3, picture number (PN) are assigned to each picture so that the picture numbers are

incremented by 1 like in stream 1.
Therefore, as the picture numbers are incremented by 1 in streams 1 and 3, pictures to be coded are stored in the reference memory. In stream 2, pictures assigned picture numbers (PN) in a way that the picture numbers are incremented by 1 are stored in the reference memory, and pictures assigned the same picture numbers (PN) as the precedent pictures are not stored in the reference memory.
Also, pictures assigned the picture number of "0" in streams of 1, 2 and 3 are pictures to be displayed at the time of tO. Likewise, the groups of pictures listed below are pictures to be displayed at the same time: Picture F14 in stream 1, picture F22 in stream 2 and picture F31 in stream 3 are pictures to be . displayed at the time of tl. Picture F1S in stream 1, picture F2-4 — in stream 2 and picture F32 in stream 3 are pictures to be displayed at the time of t2. Picture F112 in stream 1, picture F25 in stream 2" and "'picture F33 in stream 3 are pictures to" "be displayed at the time of t3. Picture F117 in stream 1, picture F215 in stream 2 and picture F34 in stream 3 are pictures to be displayed at the time of t4. Note that pictures F112, F26 and F33 correspond to S pictures in the first and second embodiments.
In FIG. 12, streams are switched by way of picture BP1 and 3P2 that exist between the picture before switching and the picture after switching, and both of the BP1 and the BP2 are switching pictures to be coded in a way that they have the same time as their precedent pictures in the respective switching streams.
For example, in the case where picture F026 in stream 2 (a picture in the stream before switching) is switched to picture F113 in stream 1 (a picture in the stream after switching), switching picture BP1 that exists between F26 and F113 is used as a picture at the time of t3. In this case, the picture number of switching

picture 3P1 which is a switching pictureis changed to "12" to make The number continuous with the picture number 13 of picture F113 in the stream after switching.
Also, likewise, in the case where picture F32 in stream 3 (a picture in the stream before switching) is switched to picture F213 in stream 2 (a picture in the stream after switching), switching picture BP2 that exists between F32 and F213 is used as a picture at the time of t3. In this case, the picture number of switching picture BP2 which is a switching picture is changed to make the number continuous with the picture number 13 of picture F213 in the stream after switching.
In this way, by assigning picture numbers (PN) of switching pictures to make the numbers continuous with the picture numbers (PN) of pictures in the.-str.e-am after switching, picture numbers (PN) of pictures in the stream after switching are changed to be identical in sny case of coded pictures within each stream or switching streams.
Next, the processing of assigning picture numbers (PN) in the case of switching streams wili be explained below.
FIG. 13 is a flow chart shewing a coding method after assigning picture numbers (?U) to respective pictures in the streams In FIG. 12.
In step 1401, whether each of the pictures to be coded is S picture or not is judged. When the current pictures are S pictures, picture numbers (PN) of the current pictures are changed to the initial values of M in step 1402. When the current pictures coded are not S pictures, picture numbers (PN) of the current pictures are not changed.
In step 1403, whether each of the pictures to be coded is the next pictures of S pictures.or not is judged. When the current pictures are the next pictures of S pictures, whether each of S pictures is stored in the memory or not is judged in step 1404,

When the current pictures are not the next pictures of 5 pictures, whether each of the current pictures is stored in the memory or not is judged in step 1405.
When S pictures are judged to be stored in the memory in step 1404, the picture number "M" are incremented by 1 to make M4-1 in step 1406r the incremented picture numbers (PN) replace the old picture numbers (PN).
When S pictures are not judged to be stored in the memory in step 1404, the picture numbers (PN) are regarded as "M"s in step 1407. The picture numbers (PN) are not changed. In step 1405, whether the pictures to be coded are stored in the memory or not is judged. When the current pictures are judged to be stored in the memory, the picture numbers (PN) are incremented to PP4 + 1 In.step 1408, and the incremented picture numbers- (PN) replace the old picture numbers (PN).
When the pictures to be coded ere judged not to be stored in the memory,-the picture numbers (PN) are not changed."
In step 1409, target pictures are coded. In step 1410, whether ail the current pictures have been coded or not is judged. When ail the current pictures have not been coded, step 1401 must be repeated. When all the current pictures have been coded, step 1410 finishes.
The processing shown in FIG. 13 makes it possible to produce coded data streams whose picture numbers (PN) are continuous in the streams after switching coded pictures.
Also, the coded signals (Str) produced in this way can be decoded based on the decoding method by the picture decoding apparatus in the fifth embodiment. In this way, the picture decoding apparatus that decodes coded signals in the sixth embodiment is realized.
Also, the coding and the decoding methods shown in the above-mentioned first to sixth embodiments can be implemented

in mobile communication instruments such as cellular ohones and car navigation systems and cameras such as digital video cameras or digital steel cameras by using semiconductors such as LSI. Also, there are 3 types of apparatuses suitable for the implementation: a sending and receiving type terminal with both of a coding and a decoding apparatuses, a sending terminal with a coding apparatus only and a receiving terminal with a decoding apparatus only.
(Seventh Embodiment)
Pictures that should be referred to by the pictures to be decoded are shown in a form of picture numbers (PN). Also, errors of picture numbers (PN) can be detected based on the increase and the decrease in picture numbers (PN). FIG. 14 shows the processing of checking and correcting errors of picture numbers (PN) based on picture numbers (PN).
First, picture numbers (PN) are detected in step 20. Next, picture types (PicType) are detected in step 21. After that, whether picture numbers (PN) detected in step A2 are continuous or not is judged. When the picture numbers (PN) are continuous in step A2, error checking and correcting processing of picture numbers (PN) is completed. When the picture numbers (PN) are not continuous In step A2, errors must be corrected in step A3. Note that the processing of checking the stored maximum PN and reassigning "PN"s can be performed either way of soleiy done after this error checking and correcting processing or concurrently done with this error checking and correcting processing.
The first conceivable method of error correcting processing in step A3 is requesting for resending the data concerning the picture numbers with errors, and following the error checking processing of picture numbers (PN) again after receiving the resent data. However, the cause of the discontinuity in picture

numbers (PN) of 5 pictures is not a transmission error. That is, as the number of pictures in each stream stored in the memory prior to the S pictures may vary in the case where the discontinuity in picture numbers (PN) of S pictures is found, the picture whose picture number (PN) is required for being resent may not exist, and thus it is highly unlikely that a missing picture can be sent. Therefore, the requests for the pictures that cannot be sent are made endlessly, which may be a hindrance in displaying pictures. For this reason, as to countermeasures in the case of trouble in displaying pictures, further explanation will be made in a tenth embodiment below.
Also, if the number of pictures in the stream after switching in the memory at the time of switching streams is not the same as that of pictures in the original stream, displaying the'pictures may not be done properly.
First, there are memories of an FIFO memory for short-time storage and a memory for long-time storage that can directly specify storage places without employing iVfirst-in, first-out" so as to store pictures longer than the above-mentioned memory for short-time storage. When the memory for short-time storage can store 7 pictures and the memory for long-time storage can store 4 pictures, reference picture are specified based on the order calculated from the memory for short-time storage. For example, used LT2 can be specified as the eighth picture (Idx = 7) in the memory for long-time storage. In this way, reference pictures are specified based on the relative positions.
When there are 3 streams as shown in FIG. 7, the locations in the memory to specify the identical pictures (such as S pictures shown in FIG. 7) vary from stream to stream as shown in FIG. 2. And, when referring to pictures.in another stream from S pictures, the locations in the memory to specify reference pictures vary according to the memory in each stream. When there exist a

plurality of streams at the time of switching streams from a predetermined one to another stream, S oictures are those pictures whose precedent pictures in the streams before and after switching are identical.
Not only S pictures but also I pictures can be the pictures at which streams are switched when the pictures decoded in a plurality of streams in the reference memory are exactly the same, and thus I pictures can be used for the same purpose as S pictures (for switching streams).
Considering various conditions like this, it is difficult to specify reference pictures correctly if the number of pictures varies from stream to stream, and it is highly likely that there emerges any error even when the reference pictures are specified.
Therefore, this seventh embodiment will show-the coding and decoding methods of additional information used for avoiding an endless error checking processing of picture numbers (PN) triggered by problems like discontinuity of picture numbers (PN) or incongruity in the content of the memory. This additional information (all picture delete information) is the order for showing th3t pictures except I pictures and S pictures to be coded must be deleted from the memory for reference in coding and decoding in order to prevent any error from occurring in the process of coding pictures after coding I pictures capable of intra-picture coding and the above-mentioned S pictures.
This process makes each memory condition identical in a plurality of streams after switching streams from a predetermined stream to another stream and makes it possible to specify predetermined pictures in the memory correctly even when reference pictures are needed for intra-predictive coding and so on. Also, preventing discontinuity in picture numbers (PN) from being detected and corrected as an error resolves the problem of hindering decoding caused by requests for resending nonexistent

pictures.
The coding method will be explained below in FIG. 16A. FIG. 16A shows the making processing of coded signals in this seventh embodiment.
First, picture numbers (PN) are detected in step 20, Next, picture types (PicType) are detected in step 21. Whether the picture types detected are I pictures or not is judged in step Al. When the picture types detected are I pictures, all the pictures except I pictures to be coded in the memory are deleted in step A10. In the following step All, all picture delete information meaning deleting all the pictures in the memory is coded, and this is the last procedure of coding additional information.
Also, step Al in FIG. 16A can be used as a step for judging whether the picture types are S.pictures or not-in the same coding processing as shown in FIG. 16B. Also, it is possible to combine step Al with step A2 snd judge whether the picture types are I pictures or S pictures after checking the picture types in step 21.
As shown in FIG. 17A, when the picture types to be coded are proved to be I pictures in step Al at the time of checking the picture types in step 21 and when the picture numbers (PN) are judged to be discontinuous after the judgment on whether the picture numbers are continuous or not like the judgment made in step A3, it is also possible to delete ail the pictures except I pictures to be coded in the memory. On the other hand, when picture numbers (PN) are continuous in step A3, the pictures in the memory are not deleted. When checking S pictures as picture types, the same explanation shown in FIG. 17A holds true. Afso, it is possible to combine step Al with step A2 and judge whether the picture types are I pictures or S pictures after checking the picture types in step 21.
It is also possible to perform the processing of step 30 judging whether the numbers of pictures stored in the memory are

the same or not so as to avoid the occurrence of errors caused by the difference in the number of pictures stored in the memory as shown in FIG. 173 after the processing of step A3 shown in FIG. 17A. It is possible to perform the processing of step A30 before going on to the processing of step A3 shown in FIG. 17. And it is possible to delete ai! the pictures after performing Step A30 before performing Step A3 in FIG. 17B when the numbers of pictures vary from stream to stream, and it is possible to delete all the pictures shown in step A10 when the numbers of pictures do not vary from stream to stream and the picture numbers (PN) are discontinuous. (FIG. 18)
In this way, the processing of FIG. 17 makes it possible to keep storing the pictures that may be reference pictures in the memory as. many as. possible and improve the reproducibiiity-of pictures decreasing errors. As to I pictures or S pictures, they need no error correction when the numbers of pictures stored in the rnemor-y vary or the picture numbers (PNJ-are discontinuous, which simplifies memory control in the coding apparatus.
Note that specifying I pictures and ordering deletion of all the pictures in the memory can be shown by picture types that specifies special pictures 3S I pictures.
(Eight Embodiment)
FIG. 19 is a block diagram showing the structure of the picture coding apparatus of the present invention. The block diagram of the picture coding apparatus in the present invention shown in FIG. 19 is an example for realizing the picture coding method in FIG. 16.
Picture number generating unit (PNGen) generates picture numbers (PN). Picture numbers (PN) are IDs that identify pictures stored in the reference picture memory (Mem), and each different picture stored in the reference picture memory (Mem) is

given an exclusive picture number (PN). Usually, picture numbers 'PN) are incremented by 1 each time a picture Is stored in the reference picture memory. If picture numbers (PN) received in the picture decoding apparatus are incremented by 2 or more, it is possible to detect the lack of pictures to be stored in the transmission line by the picture decoding apparatus and carry out error correction processing such as picture improvement (making the error less conspicuous) or error correction (retransmitting the picture with no errors to reproduce the picture).
When the picture type information (PicType) shows that the pictures concerned are S pictures (corresponding to the processing of step A2 in FIG. 16), picture deleting unit 3 (PicDe!) orders the reference picture memory (Mem) to delete the pictures stored in the reference picture memory (Mem) except pictures to be coded, end sends the information to variable length coding unit (V'LC) at the same time-On the other hand, the picture type information (PicType) shows that the pictures concerned are I pictures (corresponding to the processing of step Al in FIG, 16), picture deleting unit (PicDe!) orders the reference picture memory (Mem) to delete the pictures stored in the reference picture memory (Mem) except pictures to be coded, and also sends the information to variable length coding unit (VLC) at the same time.
Picture encoding unit (PicEnc) refers to the pictures stored in the reference picture memory (Mem) so as to code the input picture signal (Vin) including frequency conversion and quantization as picture types shown by picture type information (PicType) and send the result to the picture decoding unit (PicDec) or the variable length coding unit (VLC).
Picture decoding unit (PicDec) inversely quantizes and converts the frequency of the information coded in the picture

encoding unit (PicEnc) as picture types shown as picture type information (PicType), and stores the information in the reference picture memory (Mem) as picture numbers (PN) to refer so as to the picture numbers in the following picture coding.
Variable length coding unit (VLC) performs variable length coding on the information coded in picture encoding unit (PicEnc) so as to make a bit stream, and codes necessary information for decoding such as the information for deleting pictures stored in the reference picture memory (Mem) notified by picture deleting unit 3 (PicDel), picture numbers (PN), and picture type information (PicType) to output the information as coded signals (Str).
The structure of the coded signals (Str) in the present invention will be shown in FIG. IOC and 10D. The data will be explained below.
First, Picture numbers (PN) are coded. f\!extf information for .deleting pictures stored in the reference picture memory (Mem), then picture type information (PicType) and picture coding data outputted by the in picture encoding unit (PicEnc) are located.
As FIG. IOC is merely an example of data location, it is possible to switch data coding orders as shown in FIG. 10D.
The above-mentioned processing enables the picture coding apparatus to realize the picture coding method shown in FIG. 16 and provide a coding apparatus with high error-resistance.
(Ninth Embodiment)
FIG. 20 is a block diagram showing the structure of the picture coding apparatus in the present invention. The block diagram of the picture coding apparatus of the present invention shown in FIG. 20 is an example for realizing the picture coding method in FIG. 17. Explanation concerning the same units as

explained in FIG. 19 will be omitted from the following explanations.
FIG. 20 and FIG. 19 differ in the process in picture deleting unit 4 (PicDel). To be specific, when the picture type information (PicType) shows that the pictures concerned are S pictures (corresponding to the processing of step A2 in FIG. 17) and the number of pictures varies from stream to stream when comparing the numbers, (corresponding to the processing of step A30 in FIG. 17), picture deleting unit 4 (PicDel) orders the reference picture memory (Mem) to delete the pictures stored in the reference picture memory (Mem) except pictures to be coded, and sends the information to variable length coding unit (VLC) at the same time. The same is done in the case of I pictures shown by picture type information (PicType). The structure of coded signals of the present invention is the same as the ones shown in FIG, IOC and 10D.
The above-mentioned processing enables- the picture coding apparatus to realize the picture coding method shown in FIG. 17 and provide the coding apparatus with high ■ error-resistance.
(Tenth Embodiment)
The seventh embodiment above showed that picture display may be hindered because requests for resending pictures that cannot be resent are repeatedly made when discontinuity of picture numbers occurred at an S picture. The problem-solving method for the example of picture display with troubles caused by this reason will be explained below.
FIG. 21A shows the processing of decoding the coded pictures.
First, picture numbers (PN) are detected in step 20. Next, picture types (PicType) are detected in step 21. Whether the

picture types detected are I pictures or not is judged in step Al. When the picture types detected are not I pictures, whether the picture numbers (PN) are continuous or not is judged in step A3. On the other hand, when the picture types detected are I pictures, there is no need to detect or correct errors and a series of processing finishes.
When the picture numbers (PN) are not continuous in step A3, errors are corrected in step A4. On the other hand, the picture numbers (PN) are continuous in step A3, error check and correction are completed.
Error correction in step A4 can be, for example, processing of detecting the stored maximum picture numbers (PN) as explained in the above-mentioned embodiment, or deleting a!l the pictures in the memory after receiving ali picture delete information meaning deleting ail the pictures in the memory in the processing of reassigning picture numbers (PN).
.. As .shown in FIG. 2-1B, of step Al In FIG. 21A, the same- -coding processing can be performed as a step of judging whether the picture types are S pictures or not. Also, it is possible to combine step Al with step A2 and judge which picture types of I pictures or S pictures they have after checking the picture types in step 21.
In this way, it is possible to avoid hindering decoding as a result of repeating requests for resending the missing pictures so as to correct errors when picture numbers of I pictures or S pictures are not continuous. This processing at an I picture is especially useful in the case of a special I picture capable of switching streams.
(Eleventh Embodiment)
FIG. 22 is a block diagram showing the structure of the picture decoding apparatus of the present invention. The block

diagram of picture decoding apparatus of the present invention shown in FIG, 22 shows an example for realizing the picture decoding method shown in FIG. 21. Expianation concerning the same units as explained in FIG. II will be omitted from the following explanations.
The different point of FIG. 22 from FIG. 11 is the processing in the error checking unit (ErrChk) using picture types (PicType) by the picture number checking unit (PNchk). To be specific, when picture numbers (PN) to be inputted in picture number checking unit (PNchk) are not continuous and picture types are neither I pictures nor S pictures, an error correction order (Err) is outputted by the error checking unit (ErrChk). With an error correction order, processing such as the stored maximum PN check, PN reassignment or deleting all the pictures in the memory according to all picture delete information meaning deleting all the pictures in the memory.
The-above-mentioned processing enables ihe picture coding ' apparatus to rea!i2e the picture coding method shown in FIG. 21 and provide a decoding apparatus with high error-resistance.
(Twelfth Embodiment)
This embodiment explains another countermeasure against an endless error check of picture numbers (PN) caused by the troubles such as discontinuity of picture numbers (PN) or incongruity in memory contents. This twelfth embodiment differs from the seventh embodiment in that the picture numbers (PN) are reassigned from *0" when all the pictures are deleted after the same step of deleting all the pictures in coding shown in the seventh embodiment.
This process makes the respective memory conditions in a plurality of streams identical after switching streams from a predetermined stream to another stream, and initializes the

picture numbers (PN), and thus it makes it possible to correctly specify the predetermined pictures in the memory even when reference pictures are required in intra-predictive coding and so on. It is also possible to resolve the problem of hindering decoding when switching the coded streams to be decoded from the predetermined stream to another stream by avoiding correcting any picture number discontinuity as errors.
As explained, each picture in the stream is given an exclusive picture number (PN) continuous in display time order in the coded streams gotten by coding moving pictures. The reason why the picture numbers (PN) are given exclusive picture numbers (PN) continuous in display time order is that it makes it possible to check the lack of pictures in the coded streams caused by an error in transmission line in the case where the picture decoding apparatus receives the coded streams by way of the transmission line. When a picture number (PN) cf a picture to be inputted in oispiay time order is incremented by 2 or more while the received coded stream is being decoded, this picture decoding apparatus can check the transmission error occurred just before the receiving time of the picture concerned and request the sender to resend the missing pictures. Therefore, as long as the picture decoding apparatus is decoding one coded stream continuously, it C3n check transmission errors effectively and receive the resent missing pictures to decode the coded streams perfectly.
However, this error check causes a glitch of endless error checking processing, in the case of the picture decoding apparatus used for continuing decoding after switching to another coded stream with a different picture rate while decoding a coded stream after inputting a plurality of coded streams gotten by coding the same moving picture at a different picture rate. The cause of this glitch is that picture numbers (PN) of pictures except the first picture in each stream vary among coded pictures with

different picture rates even in the case of the pictures to be displayed at the same time, in o:her words., picture numbers (PN) are continuous within each coded stream in the display time order. Therefore, when the decoding target is switched to another stream in the middle of decoding one coded stream in a picture decoding apparatus, picture numbers (PN) are discontinuous even in the case of the pictures to be displayed at the same time. The coding method using additional information (all picture delete information) has been explained in the seventh embodiment so as to avoid the endless error check of picture numbers (PN) caused by the problems such as discontinuity of picture numbers {?N) or incongruity of memory contents in this way. This additional information is the order for deleting all the pictures except pictures to be coded from the memory for reference in coding or decoding so that any error should not occur at the time of switching streams in the process of picture coding after coding I pictures for intra-coding or the above-mentioned S pictures.
Coding method will be explained beiow using FIG. 23. FIG. 23A shows the making processing of coded signals in this twelfth embodiment.
First, picture numbers (PN) are detected in step 1. Next, picture numbers (PN) detected in step 1 are coded in step 2. And, picture types (PicType) are detected in step 3. In the step 3, whether the detected picture types are S pictures or not is judged.
When the detected picture types are S pictures, all picture delete information meaning deleting all the pictures in the memory is coded in step 5. Next, S pictures are coded in step 6A. And, picture numbers are initialized in step 7, and in the following step 8, all the pictures except S pictures to be coded in the memory are deleted in step 8. Up to this point, the processing of coding additional information and initializing picture numbers (PN) finishes.

As picture numbers (PN) g-e continuous unless the detected picture types are 5 pictures, these pictures are coded in step SB, but the processing is completed without coding additional information, initializing picture numbers (PN), and deleting all the pictures.
Initialization of picture numbers (PN) in step 7 is such as giving picture number of "0" to the coded S pictures. In other words, initializing picture numbers (PN) of S pictures means giving the pictures after S pictures in display time order new picture numbers (for example, PN 1) starting from the picture number (PN 0) of S pictures. Consequently, picture numbers (PN) are initialized after coding S pictures (that is, after coding the picture numbers (PN) of S pictures.).
The judgment on whether the pictures are S pictures or not is made in step 4, the judgment on whether the pictures are I picturesare I pictures or not can be made. Also, when there is a step of deleting all the pictures in step Z3A, picture numbers (PN) should be initialized concurrently because whether the pictures are I pictures or S pictures or not is not a sole judgmental standard as to whether picture numbers (PN) should be initialized. Also, picture number (PN) coding processing in Step 2 can be performed at any time between picture number checking processing in step I and the picture number initializing processing in step 7. Also, it is possible to initialize picture numbers (PN) in step 7 after deleting all the pictures except S pictures to be coded in the memory in step 8. Also, the processing of coding ail picture delete information meaning deleting all the pictures in the memory in step 5 is the processing after the judgment whether the pictures are S pictures or not and can be performed at any time before the processing shown in FIG. 23A is finished. Also, it is possible not to code additional information by using special picture types (PicType) that include additional information meaning deleting all the pictures

except the pictures to be coded from the memory fo.r reference in coding or decoding. Reassigning picture numbers (PN) so as to switch streams at S pictures or I pictures is effective, but the effectiveness is not limited to the case where streams are switched at 5 pictures of I pictures, in other words, picture numbers (PN) can be reassigned in the same way on condition that other pictures such as P pictures are capable of switching streams and that there is a step of deleting ail the pictures unnecessary for reference.
FIG. 24 is a block diagram snowing the structure of the picture coding apparatus capable of realizing the coding method in the twelfth embodiment.
Picture number generating unit (PNGen) generates Picture numbers (PN). Picture numbers (PN) are IDs that identify the pictures stored in the reference picture memory (Mem)., each picture stored in the reference picture memory (Mem) is given an exclusive picture number (PN). Basically, picture numbers (PN) -are-m-cremented by 1 each time a picture is stored in the reference picture memory (Mem). Also, picture numbers (PN) of- the current S pictures 3re initialized to "0" after S pictures are coded according to the notification from picture encoding unit (PicEnc).
When picture type information (PicType) shows that the pictures are S pictures (corresponding to the processing of step 3 in FIG. 23), picture deleting unit 5 (PicDel) notifies the reference picture memory (Mem) of an order of deleting the pictures except the pictures to be coded (all picture delete information) stored in the reference picture (Mem) memory and notifies the variable length coding unit (VLC) of the information at the same time.
The picture encoding unit (PicEnc) refers to the pictures stored'in the reference picture memory (Mem) so as to code the input picture signal (Vin) into picture types shown by picture type information (PicType) concurrently performing frequency conversion and quantization and send the result to the picture

decoding unit (PicDec) and the variable length coding unit (VLCV Also, the picture encoding unit (PicEnc) notifies -he picture number generating unit 2 (PNGen) of the order of initializing picture numbers (PN) after coding S pictures.
The picture decoding unit (PicDec) inversely quantizes and inversely converts the information coded in the picture encoding unit (PicEnc) into picture types shown as picture type information (PicType), and stores the information in the reference picture memory (Mem) associating with picture numbers (PN) so as to refer to the picture types in the following picture coding.
The variable length coding unit (VLC) performs variable length coding on the information coded in picture encoding unit (PicEnc) so as to make a bit stream, and codes necessary information in decoding such as th-e information for deleting pictures stored in the reference picture memory (Mem) (that is, all picture delete information) notified by picture deleting jnit 5 (PicDei), picture numbers ("PN); and' picture type information (PicType) to output the information as coded signals (Str).
Next, decoding method will be explained below using FIG. 233. FIG. 233 shows the decoding processing of coded signals.
First, picture numbers (PN) are decoded in step 9, Next, whether all picture delete information is coded or not is judged in step 10.
When aii picture delete information is judged to be coded in step 10, all picture delete information is decoded in step 11. And, pictures are decoded in step 12A. After that, all the pictures except the pictures to be decoded in the memory are deleted in step 13, and picture numbers (PN) are initialized in step 14. Up to this point, the processing of decoding additional information and initialization of picture numbers (PN) finishes.
When all picture delete information is judged not to be coded in step 10, pictures are decoded in step 12B, and the

processing of decoding additional information and Jni.ial;::in~ picture numbers (PN) :s completed in step 123.
Initialization of picture numbers (PN) in step 14 is such as giving picture number of "0" to the decoded pictures. In other words, when decoding the coded signals coded according to the coding procedure shown in FIG. 23A, initializing picture numbers (PN) of S pictures means giving the pictures after S pictures in display time order new continuous picture numbers starting from the picture number of S pictures.
When there is a step of deleting all the pictures in FIG. 23B, the processing of initializing picture numbers (PN) is necessary, in other words, the judgment on whether the picture numbers should be initialized is not influenced by picture types to be decoded. Also-, the processing of initializing picture numbers (PN) in step 14-can be performed before the processing of deleting all the pictures except the pictures to be coded :.r, ihe memory in step 13. It is possible not to code additional information by using special'picture' types (PicType) that include additional ■ information meaning deleting all the pictures except the pictures to be decoded from the memory for reference in decoding.
FIG. 25 is a block diagram showing the structure of the picture decoding apparatus that realizes the decoding method of this twelfth embodiment.
Variable iength decoding unit (VLD) decodes coded signals (Str), and outputs various information (such as an order for deleting the pictures stored in the reference picture memory (Mem), picture type information (PicType), picture numbers (PN), information for reassigning picture numbers (PN), and picture data,)
First, the order for deleting the pictures stored in the reference picture memory (Mem) gotten in variable length decoding unit (VLD) (all picture delete information) is sent to

Picture deleting unit 5 (PIcDel). Next, picture deleting unit 5 (PicDei) deletes the specified pictures stored in the reference picture memory (Mem).
Picture type information (PicType) gotten in variable length decoding unit (VLD) is sent to picture decoding unit (PicDec) to specify the decoding method.
Picture numbers (PN) gotten in variable length decoding unit (VLD) is sent to the reference picture memory (Mem) to be used as picture numbers (PN) at the time of storing the pictures decoded in the picture decoding unit (PicDec).
All picture delete information gotten in the variable length decoding unit (VLD) is sent to the picture number changing unit 2 (PNchg). Picture number changing unit 2 (PNchg) reassigns (initializes) picture numbers (PN) of the pictures stored in the reference picture memory (Mem) according to the order. To be specific, after all the pictures except the pictures to be decoded (S pictures) in the reference picture memory'(Mem), picture number" changing unit 2 (PNchg) reads out the picture numbers (PN) of the pictures stored in the reference picture memory (Mem), changes the values of the read-out picture numbers (PN) to "0", and writes the picture numbers (PN ) in the reference picture memory (Mem).
Picture data gotten in variable length decoding unit (VLD) are decoded using the decoding method based on the picture type shown by picture type information (PicType) in the picture decoding unit (PicDec). In other words, P pictures and B pictures are decoded referring to the pictures stored in the reference picture memory (Mem) while I pictures are decoded without referring to pictures in the reference picture memory. The decoded picture gotten in this way.are stored in the reference picture memory (Mem) and outputted as decoded picture signals (Vout).

The above-mentioned structure makes it possible to -eaiize the picture decoding apparatus to realize the picture decoding method shown in FIG. 23 and provide a decoding apparatus with high error-resistance.
The processing of the coding and the decoding methods shown in this twelfth embodiment makes the memory conditions in a plurality of streams identical after switching streams from a predetermined stream to another stream, and thus makes it possible to correctly specify the predetermined pictures in the memory even when reference pictures are required in intra-predictive coding and so on.
It is possible to change I pictures into special pictures at which streams can be reproduced by deleting ali the pictures in the reference memory when using I pictures while the above-mentioned embodiment explains that additional information (ali picture delete information) and picture types (PicType) can be coded all at once. This special I pictures are called Instantaneous Decoder Refresh (IDR). An IDR picture is. effective as a leading I picture of Group of Pictures (GOP) because IDR pictures become a starting position of random access. 3y determining that all the pictures except the current pictures in the memory are deleted and picture numbers (PN) are initialized after coding the current pictures each time these IDR pictures are coded, there is no need to code additional information even when all the pictures except the current pictures in the memory are deleted in the picture decoding apparatus. In this case, the picture decoding apparatus detects IDR pictures in the coded streams based on picture types, deletes all the pictures except the current IDR pictures in the memory, and initializes picture numbers (PN) after coding and decoding the current IDR pictures even when any additional information is not coded each time IDR pictures are decoded.

(Thirteenth Embodiment)
Moreover, storing programs to realize the structures of the picture coding and the decoding methods Ihown in the above-mentioned embodiments on a storage medium like a :iexible disc makes it possible to easily perform the processing shown in the above-mentioned embodiments in the independent romputer system.
FIG. 26 is an illustration concerning "the storage medium to ;tore the program to realize the coding and the decoding methods mown in the first to the twelfth embodiments mentioned above in he computer systems.
FIG. 26B shows a flexible disc and the front view and the
ross-sectional view of the appearance of the flexible disc, and FIG.
>6A shows an example of a physical format of a flexible disc as a
ecording medium body. A flexible disc (FD) is contained in a
:ase F, a plurality of tracks (Tr) are formed concentrically on the
urfsce of the d-is-c from the periphery into the inner radius of the
lisc, and each track'is divided into 16 sectors (Se) in the angular
irection. Therefore, in the case of the flexible disc storing the
bove-nnentioned program, the picture coding method and the
icture decoding method as the program is recorded in an area
ilocated for it on the flexible disc (FD).
Also, FIG. 26C shows the structure for recording and eading out the program on the flexible disc (FD). When the rogram is recorded on the flexible disc (FD), the computer ystem (Cs) writes in the picture coding method or the picture ecoding method as a program via a flexible disc drive. When the icture coding method and the decoding method mentioned above re constructed in the computer system by the program on the exible disc, the program is read out from the flexible disc drive nd transferred to the computer system.

Brief Description of Drawings
FIG. 1 is an illustration of relations between pictures and picture numbers (PN) when coding an input picture signal (Vin).
FIG. 2 is a diagram showing picture numbers (PN) of pictures to be stored in a reference picture memory (Mem) when coding and decoding S pictures.
FIG. 3 is an illustration of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem).
FIG. 4A, 4B and 4C are flow charts of how to code and decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture coding method and the picture decoding method of the present invention.
FIG. 5A and 5B are illustrations of picture numbers (PN) of pictures to be stored in the reference picture memory (Mem).

FIG. 6A, 6B and 5C are flow charts of how to code and decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture coding method and the picture decoding method of the present invention.
FIG. 7 is an illustration of the relation between pictures and picture numbers (PN) when coding input picture signal (Vin) of the present invention.
FIG. 8A and SB are flow charts of how to decode information on controlling pictures to be stored in the reference picture memory (Mem) in the picture decoding method of the present invention.
FIG. 9 is a block diagram showing the structure of picture coding apparatus of the present invention.
FIG. 10 A, 10B, IOC and-lOD are diagrams showing data structure examples of coded signals, Str of the present invention.
FIG, 11 is a block diagram showing the structure cf a picture decoding apparatus of the present invention.
FIG. 12 is an Illustration of the relation between pictures and picture numbers (PN) for coding an input picture signal (Vin) of the present invention.
FIG. 13 is a flow chart showing a coding method by giving a picture number to each picture of each stream in the present invention.
FIG. 14 is a flow chart showing a decoding method in a seventh embodiment.
FIG. 15 is a diagram showing a memory structure in the seventh embodiment.
FIG. ISA and 16B are flow charts showing a coding method in the seventh embodiment.
FIG. 17A and 17B are flow charts showing another coding method in the seventh embodiment.
FIG. 18 is a flow chart showing another coding method in

the seventh embodiment.
FIG. 19 is a block diagram showing 3 structure of a coding apparatus in an eighth embodiment.
FIG. 20 is a block diagram showing a structure of another coding apparatus in the eighth embodiment.
FIG. 21A and 21B are flow charts showing a decoding method in a tenth embodiment.
FIG. 22 is a biock diagram showing a structure of a decoding apparatus in an eleventh embodiment.
FIG. 23A and 23B are flow charts showing the processing for making coded signals and decoding the coded signals.
FIG. 24 is a block diagram showing a structure of a picture coding apparatus realizing a coding method in a twelfth embodiment..
FIG. 25 is a block diagram showing a structure of a picture decoding apparatus resizing the decoding method in the twelfth embodiment.
FIG. 26 is an illustration as to a recording medium storing a program for realizing the picture coding methods and picture decoding methods in the first to the twelfth embodiments using a computer system.
FIG. 27 is a block diagram showing the whole structure of a content supplying system realizing a content distribution service concerning the present invention.
FIG. 28 is a diagram showing an example of cellular phones relates to the present invention.
FIG. 29 is a block diagram showing the structure of the same celiular phones, and
FIG. 30 is a diagram showing the structure of a digital broadcasting system concerning the present invention.
Best Mode for Carrying Out the Invention

[he present embodiments of the present Invention will be explained below with reference to FIG. 3~3Q.
(First Embodiment)
WE CLAIM:
1. A coding method for coding a moving picture, comprising :
(a) assigning a picture number to a current picture;
(b) coding the picture number of the current picture ;
(c) determining (Step03, Step04) whether the current picture is a picture having all reference picture release information instructing a decoder to release all reference pictures stored in a memory;
in the case where it is determined that the current picture is judged to have the all reference picture release information, performing the following steps of:
(d) coding (Step05) the all reference picture release information;
(e) coding (Step06A) the current picture to obtain a coded picture ;
(f) decoding the coded picture to obtain a decoded reference picture;
(g) releasing (Step08) all reference pictures which are stored in the memory after the decoding of the coded picture;
(h) storing the decoded reference picture into the memory; characterized by the further steps of:
(i) initializing (Step07) the picture number of the current picture after the decoding of the current picture by assigning a new picture number of "0" to the current picture, and assigning a picture number starting with the new picture number of the current picture to a following picture to be coded;
(j) coding the picture number of the following picture; and
(k) coding the following picture.
2. The coding method as claimed in claim 1, wherein the decoded picture stored in the memory is a reference picture, and the all pictures that are released in said releasing (Step08) of all pictures are reference pictures.

3. The coding method as claimed in claim 1, wherein said releasing of all pictures stored in the memory is executed by marking information, which indicates releasing, to all decoded pictures stored in the memory.
4. The coding method as claimed in claim 1, wherein the all reference picture release information is coded as information indicating that the current picture is an instantaneous decoding refresh picture.
5. The coding method as claimed in claim 4, wherein said instantaneous decoding refresh picture is an intra-coded picture.
6. The coding method as claimed in claim 1, wherein the all reference picture release information is coded as picture type information indicating that said current picture is an intra-coded picture.
7. A coding apparatus that codes a moving picture, comprising :
a picture number assigning unit (PNGen2) operable to assign a picture number
to a current picture;
a picture number coding unit operable to code the picture number of the current
picture;
a release information coding unit (PicDel5) operable to determine whether the
current picture is a picture having all reference picture release information
instructing a decoder to release all reference pictures stored in a memory, and
in the case where it is determined that the current picture is judged to have the
all reference picture release information, code the all reference picture release
information;
a picture coding unit (PicEnc) operable to code the current picture to obtain a
coded picture;

a picture decoding unit (PicDel) operable to decode the coded picture to obtain
a decoded reference picture.
a memory management unit operable to release all reference pictures which are
stored in the memory (Mem) after the decoding of the coded picture, in the
case where the current picture has the all reference picture release information ;
a storage unit operable to store the decoded reference picture into the memory;
characterized by;
an initializing unit operable to initialize the picture number of the current
picture after the decoding of the current picture by assigning a new picture
number of "0" to the current picture and operable to assign a picture number
starting with the new picture number of the current picture to a following
picture to be coded,
wherein said picture number coding unit operable to code the picture number of
the following picture, and
said picture coding unit operable to code the following picture.
8. The coding apparatus as claimed in claim 7, wherein the decoded picture stored in the memory is a reference picture, and the all pictures that are released by the releasing unit are also reference pictures.
9. The coding apparatus as claimed in claim 7, wherein said releasing of all pictures stored in the memory is executed by marking information, which indicates said releasing, to all decoded reference pictures stored in the memory.
10. The coding apparatus as claimed in claim 7, wherein the all reference picture release information is information indicating that the current picture is coded as an instantaneous decoding refresh picture.

11. The coding apparatus as claimed in claim 10, wherein said instantaneous
decoding refresh picture is an intra-coded picture.
12. The coding apparatus as claimed in claim 7, wherein the all reference picture
release information is picture type information indicating that said current picture is an
intra coded picture.


Documents:

http://ipindiaonline.gov.in/patentsearch/GrantedSearch/viewdoc.aspx?id=gP0+vwTKzWJwD3mA/SWE1Q==&loc=egcICQiyoj82NGgGrC5ChA==


Patent Number 277252
Indian Patent Application Number 5616/CHENP/2008
PG Journal Number 48/2016
Publication Date 18-Nov-2016
Grant Date 16-Nov-2016
Date of Filing 17-Oct-2008
Name of Patentee Godo Kaisha IP Bridge 1
Applicant Address c/o Sakura Sogo Jimusho, 1-11 Kanda Jimbocho, Chiyoda-ku, Tokyo 101-0051
Inventors:
# Inventor's Name Inventor's Address
1 KADONO, SHINYA 7-25-204, KOSHIENGUCHI 1-CHOME, NISHINOMIYA-SHI, HYOGO 663-8113
2 KONDO, SATOSHI 7-17, OTOKOYAMASHIGETSU, YAWATA-SHI, KYOTO 614-8361
3 HAGAI, MAKOTO; ABE, Kiyofumi 22-402, OEDAMINAMI-MACHI, MORIGUCHI-SHI, OSAKA 570-0051; 16-1-213, Miyamae-cho, Kadoma-shi, Osaka 571-0074
4 ABE, KIYOFUMI 16-1-213, MIYAMAE-CHO, KADOMA-SHI, OSAKA 571-0074
PCT International Classification Number H04N7/26
PCT International Application Number PCT/JP03/04539
PCT International Filing date 2003-04-10
PCT Conventions:
# PCT Application Number Date of Convention Priority Country
1 2002-192533 2002-07-01 U.S.A.
2 2002-204718 2002-07-12 U.S.A.
3 60/377,638 2002-05-06 U.S.A.
4 2002-112787 2002-04-16 U.S.A.
5 2003-092490 2003-03-28 U.S.A.