Title of Invention

AN APPARATUS AND METHOD FOR REPRODUCING AND RECORDING A DIGITAL STREAM

Abstract The invention relates to a reproduction apparatus for reproducing a digital stream generated by multiplexing a video stream and a graphics stream, comprising a video decoder(5) operable to decode the video stream to generate a moving picture; a graphics decoder(12) ; and a plane memory (8), wherein the graphics stream is a sequence of packets, the packets constituting the graphics stream include two types, one of which is packets containing graphics data (ODS) and the other is packets containing control data (PCS), the sequence of packets includes a pair of a packet containing graphics data (ODS) and a packet containing control data (PCS), the pair is followed by a packet containing control data (PCS) that performs a forward reference,the control data (PCS) that performs the forward reference indicates that graphics, which corresponds to the graphics data (ODS) contained in the pair of packets which precedes the control data (PCS) that performs the forward reference, is to be displayed, the graphics decoder (12) comprises: an object buffer (15) ;a processor (14) operable to generate graphics on the object buffer (15) by decoding the graphics data (ODS) contained in the pair of packets; and a controller (17) operable to execute a graphics display by writing the graphics generated on the object buffer (15) into a partial area in the plane memory (8), based on the control data (PCS) contained in the pair of packets, wherein the plane memory (8) is a memory for overlay with moving pictures and is provided in the reproduction apparatus, and when control data (PCS) that performs a forward reference has been read into the graphics decoder (12) from a recording medium, the controller (17) updates the graphics display by writing the graphics generated on the object buffer into the partial area again based on the read control data (PCS) that performs the forward reference.
Full Text

Description
Technical Field
The present invention relates to a recording medium on
which a digital stream generated by multiplexing a video stream
and a graphics stream is recorded, and a reproduction apparatus
for reproducing the digital stream. In particular, the
present invention relates to techniques whereby the
reproduction apparatus separately decodes the video stream
and the graphics stream and overlays resulting video and
graphics to produce a reproduction image.
Background Art
Video/graphics overlay techniques mentioned above
enable to select graphics and display the selected graphics
in accordance with a language setting or display setting of
a reproduction apparatus. Also, the video/graphics overlay
techniques enable to select whether or not to display graphics
according to need. Because of their capability of offering
such selections, the video/graphics overlay techniques are
employed in player models of DVD-Video and ETSI EN 300 743.
In these standards, the graphics stream is an arrangement
of PES packets . The PES packets can be classified into two
types : a PES packet carrying control information and a PES
packet carrying graphics data which represents graphics . The

control information is arranged before the graphics data,
and one pair of control information and graphics data produces
one graphics display. In more detail, the reproduction
apparatus reads the control information and the graphics data
in sequence, decodes the control information, and also decodes
the graphics data to generate uncompressed graphics. The
reproduction apparatus displays the uncompressed graphics
with a desired presentation timing, according to a decoding
result of the control information.
According to this technique, decoding of graphics data
needs to be performed for each graphics display. This being
so, a time interval from one graphics display to the next
graphics display depends on a time period required for a
decoding process of graphics data. If graphics has a higher
resolution and a decoding process of graphics data takes a
longer time, a time interval with which graphics is updated
lengthens.
In the case of subtitling in a movie film or the like
where graphics is updated with a time interval of 2 to 3 seconds,
proper graphics displays can still be produced even if graphics
has a higher resolution and decoding of graphics data takes
a longer time. In the case where graphics is used specially
as a means of amusement, however, the need to shorten the
graphics update interval arises.
For example, the use of graphics for amusement purposes
includes such applications where graphics representing a
dialog of a character is moved to attract the viewer' sattention
or varied in color to surprise the viewer. Producers of video

content such as variety shows are especially eager to adopt
such graphics displays.
To realize smooth movements of graphics, however, an
operation of decoding graphics data and presenting graphics
needs to be performed within a short time of one frame period
(1/29.97 seconds in NTSC). Repeating this operation of
decoding graphics data and presenting graphics within each
frame period requires a tremendous load, and is hard to be
realized with hardware specifications of reproduction
apparatuses that are expected to be provided at low cost.
The aforementioned graphics movements and variations
in color can be achieved by overlaying graphics on each frame
of video beforehand so that graphics forms a part of a picture
pattern of each frame. This method of overlaying graphics
on each frame beforehand, however, lacks flexibility of being
able to selectively display graphics in accordance with a
language setting or display setting of a reproduction
apparatus, and therefore does not have much future potential.
Disclosure of Invention
The present invention aims to provide a recording medium
which enables graphics to be moved smoothly like moving
pictures.
The stated aim can be achieved by a recording medium
used for storing data, including a digital stream generated
by multiplexing a video stream and a graphics stream, wherein:
the graphics stream is a sequence of a plurality of packets
which include a packet containing control Information; and

the control information indicates that graphics data
contained in a preceding packet in the sequence is to be
displayed at a predetermined time in a state of being overlaid
on the video stream.
Control information indicates that a display is to be
produced using graphics data that precedes the control
information. Accordingly, a display position of graphics can
be changed by feeding only control information showing new
coordinates of the graphics into a reproduction apparatus.
Equally, a color of graphics can be changed by feeding only
appropriate control information into the reproduction
apparatus.
Since a graphics display can be updated using only
control information, graphics is easily brought into
synchronization with video.
Usually, a picture pattern of graphics itself need not
be varied in order to change a display position of the graphics .
This is because the human eye cannot perceive a moving object
so distinctly. This being so, the technique provided by the
present invention is suitable for displaying the same graphics
while changing its position at high speed.
Brief Description of Acccompanying Drawings
FIG. 1 shows an example application of a recording medium
to which embodiments of the present invention relate.
FIG. 2 shows a structure of a BD-ROM shown in FIG. 1.
FIG. 3 shows a structure of an AV Clip.
FIG. 4A shows a structure of a Presentation graphics

stream.
FIG. 4B shows PES packets which contain functional
Segments.
FIG . 5 shows a logical structure made up of various types
of functional Segments.
FIG. 6 shows a relationship between subtitle display
positions and Epochs.
FIG. 7A shows a data structure of an ODS.
FIG. 7B shows a data structure of a PDS.
FIG. 8A shows a data structure of a WDS.
FIG. 8B shows a data structure of a PCS.
FIG . 9 shows an example description of DSs for displaying
subtitles.
FIG. 10 shows an example description of a PCS and a WDS
in DS1.
FIG. 11 shows an example description of a PCS in DS2.
FIG. 12 shows an example description of a PCS in DS3 .
FIG. 13 shows a memory space in an Object Buffer when
performing graphics updates such as those shown in FIGS. 10
to 12.
FIG. 14 shows an example algorithm of calculating a
DECODEDURATION.
FIG. 15 is a flowchart of the algorithm shown in FIG.
14.
FIGS. 16A and 16B are flowcharts of the algorithm shown
in FIG. 14.
FIG. 17A shows a situation where one graphics Object
exists in one Window.

FIGS. 17B and 17C are timing charts showing parameters
used in the algorithm shown in FIG. 14.
FIG. 18A shows a situation where two graphics Objects
exist in one Window.
FIGS. 18B and 18C are timing charts showing parameters
used in the algorithm shown in FIG. 14.
FIG. 19A shows a situation where two graphics Objects
exist respectively in two Windows.
FIG. 19B is a timing chart when decode period (2) is
longer than a sum of clear period (1) and write period (31) .
FIG. 19C is a timing chart when the sum of clear period
(1) and write period (31) is longer than decode period (2) .
FIG. 20 shows forward reference PCSs.
FIG. 21A shows an Epoch that produces a display effect
of moving graphics on a screen.
FIG. 21B shows contents and PTSs of PCSs in DS1 to DS8 .
FIG. 22A shows an ODS in DSO.
FIG. 22B shows positions of coordinates (x1,y1), (x2,y2),
(x3 , y3 ) , ... , (x8, y8) in a coordinate system defined by a Window.
FIG. 23 shows a settings of a time stamp of a functional
Segments in each DS.
FIG. 24 shows a specific example of successive graphics
updates.
FIG . 25 shows a series of DSs for realizing a Palette-only
Display Update.
FIG. 26A shows contents of PDSs in DSO.
FIG. 26B shows contents of PCSs in DSO to DS3.
FIG. 27 shows a display effect achieved by feeding the

four DSs.
FIG . 28 shows an internal construction of a reproduction
apparatus to which the embodiments of the present invention
relate.
FIG. 29 shows transfer rates Rx, Re, and Rd and sizes
of a Graphics Plane, a Coded Data Buffer, and an Object Buffer
shown in FIG. 28.
FIG. 30 is a timing chart of pipeline processing in the
reproduction apparatus.
FIG. 31 is a timing chart of pipeline processing when
decoding of ODSs ends before clearing of the Graphics Plane.
FIG. 32 is a timing chart showing changes in occupancy
of the Composition Buffer, the Object Buffer, the Coded Data
Buffer, and the Graphics Plane.
FIG. 33 is a flowchart of an operation of loading
functional Segments.
FIG. 34 shows a case when a skip operation is performed.
FIG. 35 shows a situation where DS10 is loaded into the
Coded Data Buffer when the skip operation is performed as
shown in FIG. 34.
FIG. 3 6 shows a case when normal reproduction is
performed.
FIG. 37 shows a situation where DS1 and DS20 are loaded
into the Coded Data Buffer when the normal reproduction is
performed as shown in FIG. 36.
FIG. 38 is a flowchart of an operation of a Graphics
Controller shown in FIG. 28.
FIG. 39 is a flowchart of the operation of the Graphics



As illustrated, the BD-ROM 100 has a BDMV directory below
a ROOT directory. The BDMV directory contains a file
(XXX.M2TS) storing an AV Clip, a file (XXX.CLPI) storing
management information of the AV Clip, and a file (YYY.MPLS)
defining a logical playback path (playlist) for the AV Clip.
The BD-ROM 100 can be realized by generating such an application
format. If there are more than one file for each of the above
file types, three directories named STREAM, CLIPINF, and
PLAYLIST may be provided below the BDMV directory, to store
files of the same type as XXX.M2TS, files of the same type
as XXX.CLPI, and files of the same type as YYY.MPLS
respectively.
The AV Clip (XXX.M2TS) in this application format is
explained below.
The AV Clip (XXX.M2TS) is a digital streamof the MPEG-TS
(Transport Stream) format, and is obtained by multiplexing
a video stream, at least one audio stream, and a Presentation
graphics stream. The video streamrepresents amoving picture
of the film, the audio stream represents audio of the film,
and the Presentation graphics stream represents subtitles
of the film. FIG . 3 shows a structure of the AV Clip (XXX. M2TS) .
In the drawing, the middle level shows the AV Clip. This
AV Clip can be created as follows. The video stream made up
of a plurality of video frames (pictures pj1, pj2, pj3, ...)
and the audio stream made up of a plurality of audio frames
on the upper first level are each converted to PES packets
on the upper second level, and further converted to TS packets
on the upper third level. Likewise, the Presentation graphics

stream on the lower first level is converted to PES packets
on the lower second level, and further converted to TS packets
on the lower third level. These TS packets of the video, audio,
and Presentation graphics streams are multiplexed to form
the AV Clip.
FIG. 3 shows an example where only one Presentation
graphics stream is multiplexed in the AV Clip. If the BD-ROM
100 supports multiple languages, however, a Presentation
graphics stream for each of the languages is multiplexed in
the AV Clip. The AV Clip generated in the above manner is
divided into a plurality of extents in the same way as computer
files, and stored on the BD-ROM 100.
The following explains the Presentation graphics stream .
FIG. 4A shows a structure of the Presentation graphics stream.
In the drawing, the first level shows the TS packet string
which constitutes the AV Clip. The second level shows the
PES packet string which constitutes the Presentation graphics
stream. This PES packet string is formed by connecting
payloads of TS packets having a predetermined PID from the
TS packet string on the first level.
The third level shows the structure of the Presentation
graphics stream. The Presentation graphics stream is made
up of functional Segments that include a PCS (Presentation
Composition Segment), a WDS (Window Definition Segment), a
PDS (Palette Definition Segment), an ODS (Object Definition
Segment) , and an END (End of Display Set Segment) . Of these
functional Segments, the PCS is a screen Composition Segment,
whereas the WDS, the PDS, and the ODS are Definition Segments .

One functional Segment corresponds to either one PES packet
or a plurality of PES packets . Which is to say, one functional
Segment is converted to one PES packet and recorded on the
BD-ROM 100, or split into fragments which are converted to
PES packets and recorded on the BD-ROM 100.
FIG . 4B shows PES packets containing functional Segments .
As illustrated, each PES packet is made up of a packet header
and a payload. The payload carries a functional Segment, and
the packet header carries a DTS and a PTS associated with
the functional Segment. Hereafter, a DTS and a PTS in a packet
header of a PES packet which contains a functional Segment
are regarded as a DTS and a PTS of that functional Segment.
These various types of functional Segments form a logical
structure such as the one shown in FIG. 5. In the drawing,
the third level shows the functional Segments, the second
level shows DSs (Display Sets), and the first level shows
Epochs.
ADS on the second level is a group of functional Segments,
in the Presentation graphics stream, which are used for
creating one screen of graphics. Dashed lines hk2 show which
DS the functional Segments on the third level belong to. As
can be seen from the drawing, the series of functional Segments
PCS-WDS-PDS-ODS-END composes one DS. The reproduction
apparatus 200 reads these functional Segments which compose
the DS from the BD-ROM 100, to produce one screen of graphics .
An Epoch on the first level refers to one time unit of
continuous memory management on a reproduction time axis of
the AV Clip, and to a group of data allocated to that time

unit. Memory mentioned here includes a Graphics Plane for
storing one screen of graphics and an Object Buffer for storing
uncompressed graphics data. Continuous memory management
means that throughout the Epoch neither the Graphics Plane
nor the Object Buffer is flushed and deletion and rendering
of graphics are performed only within a predetermined
rectangular area of the Graphics Plane (to flush means to
clear the entire Graphics Plane or the entire Object Buffer) .
A size and a position of this rectangular area are fixed during
the Epoch. So long as deletion and rendering of graphics are
performed within this fixed rectangular area of the Graphics
Plane, synchronization of video and graphics is guaranteed.
In other words, the Epoch is a time unit, on the reproduction
time axis of the AV Clip, during which synchronization of
video and graphics can be guaranteed. To change the graphics
deletion/rendering area in the Graphics Plane, it is necessary
to define a point of change on the reproduction time axis
and set a new Epoch from the point onward. Synchronization
of video and graphics is not guaranteed in a boundary between
the two Epochs.
In regard to subtitling, the Epoch is a time period,
on the reproduction time axis, during which subtitles appear
within the fixed rectangular area on a screen. FIG. 6 shows
a relationship between subtitle display positions and Epochs .
In the drawing, subtitle display positions are changed
depending on patterns of pictures. In more detail, three
subtitles "Actually", "I lied to you.", and "Sorry." are
positioned at the bottom of the screen, whereas two subtitles

"Three years have passed" and "since then." are positioned
at the top of the screen . Thus, the subtitle display positions
are changed from one margin to another on the screen, to enhance
visibility. In such a case, on the reproduction time axis
of the AV Clip, a time period during which the subtitles are
displayed at the bottom of the screen is Epochl, and a time
period during which the subtitles are displayed at the top
of the screen is Epoch2. These two Epochs each have an
individual subtitle rendering area. In Epochl, the subtitle
rendering area is Windowl that corresponds to the bottommargin
of the screen. In Epoch2, the subtitle rendering area is
Window2 that corresponds to the top margin of the screen.
In each of Epochl and Epoch2, memory management of the Object
Buffer and the Graphics Plane is continuous, so that the
subtitles are displayed seamlessly in the corresponding
margin of the screen. This completes the explanation on an
Epoch.
The following explains a DS.
In FIG. 5, dashed lines hk1 indicate which Epoch DSs
on the second level belong to. As illustrated, a series of
DSs that are an Epoch Start DS, an Acquisition Point DS, and
a Normal Case DS constitutes one Epoch on the first level.
Here, Epoch Start, Acquisition Point, and Normal Case are
types of DSs. Though the Acquisition Point DS precedes the
Normal Case DS in FIG. 5, they may be arranged in reverse
order.
The Epoch Start DS provides a display effect "new
display", and indicates a start of a new Epoch. The Epoch

Start DS contains all functional Segments necessary for the
next screen composition. The Epoch Start DS is provided in
a position which is to be a destination of a skip operation,
such as a start of a chapter in a film.
The Acquisition Point DS provides a display effect
"display refresh", and is identical to the preceding Epoch
Start DS. The Acquisition Point DS is not the start of the
Epoch, but contains all functional Segments necessary for
the next screen composition. Therefore, graphics can be
displayed reliably when reproduction is started from the
Acquisition Point DS . Which is to say, the Acquisition Point
DS enables a screen composition to be made from a midpoint
in the Epoch.
The Acquisition Point DS is provided in a position which
can be a destination of a skip operation, such as a position
that may be designated by a time search. The time search is
an operation of locating a reproduction point corresponding
to a time input by a user in minutes/seconds. The time input
is made in a relatively large unit such as ten minutes or
ten seconds. Accordingly, the Acquisition Point DS is
provided in a position that can be designated by a time search
made in units of ten minutes or ten seconds. By providing
the Acquisition Point DS in a position that can be designated
by a time search, the graphics stream can be smoothly reproduced
when a time search is conducted.
The Normal Case DS provides a display effect "display
update", and contains only a difference from the previous
screen composition. For example, if DSv has the same subtitle

as immediately preceding DSu but a different screen
composition from DSu, DSv is a Normal Case DS which contains
only a PCS and an END. This makes it unnecessary to provide
overlapping ODSs in DSs, with it being possible to reduce
the amount of data stored on the BD-ROM 100 . Since the Normal
Case DS contains only the difference, graphics cannot be
displayed with the Normal Case DS alone.
The following explains the ODS, the WDS, and the PDS
(Definition Segments).
The ODS is a functional Segment for defining a graphics
Object. AV Clips recorded on BD-ROMs feature an image quality
as high as high-definition television. This being so,
graphics Objects are set at a high resolution of 1920x1080
pixels. This high resolution allows theater screen-style
subtitles, i.e. elegant handwriting-style subtitles, to be
reproduced vividly on BD-ROMs. A graphics Object is made up
of a plurality of pieces of run-length data. Run-length data
expresses a pixel string using a Pixel Code which shows a
pixel value and a continuous length of the pixel value. The
Pixel Code has 8 bits, and shows one of the values from 1
to 255. Through the use of this Pixel Code, the run-length
data sets arbitrary 25 6 pixel colors out of full color
(16,777,216 colors). Note that it is necessary to place a
character string on a background of a transparent color in
order to display a graphics Object as a subtitle.
The ODS defines a graphics Object according to a data
structure shown In FIG. 7A. As shown in the drawing, the ODS
includes a segment_type field showing a Segment type "ODS",

a segment_length field showing a data length of the ODS, an
object_id field identifying the graphics Object in the Epoch,
an obj ect_version_number field showing a version of the ODS
in the Epoch, a last_in_sequence_flag field, and an
object_data_fragment field carrying a consecutive sequence
of bytes corresponding to part or all of the graphics Object.
In more detail, the object_id field shows an identifier
which identifies the graphics Object and a storage area in
the Object Buffer that is occupied by the graphics Object,
when the ODS is decoded and the graphics Object is buffered
in the Object Buffer . This being so, when one or more graphics
Objects are present in the Object Buffer, each individual
storage area in the Object Buffer is identified by an object_id
field value . Suppose one ob j ect_id is assigned to two or more
ODSs . In such a case, after a graphics Object corresponding
to one ODS is stored in the Object Buffer, that graphics Object
is overwritten by a graphics Object corresponding to a
succeeding ODS with the same object_id . Such an update intends
to prevent occurrence of many small free spaces in the Object
Buffer and scattering of graphics Objects in the Object Buffer .
When displaying graphics, graphics Objects in the Object
Buffer are constantly transferred to the Graphics Plane . This
being so, if many small free spaces exist in the Object Buffer
or one graphics Object is scattered in the Object Buffer,
overhead for reading graphics Objects causes a reduction in
efficiency of transfer from the Object Buffer to the Graphics
Plane. S.u.ch...a...reduction in transfer efficiency may affect
synchronous display of graphics and video . To prevent this,

an existing graphics Object in the Object Buffer is overwritten
by a new graphics Object having the same object_id.
Here, the new graphics Object overwriting the existing
graphics Object needs to be equal in size to the existing
graphics Object, that is, the new graphics Object can be neither
smaller nor larger than the existing graphics Object. At the
time of authoring, therefore, an author needs to make these
graphics Objects equal in size. This size constraint that
graphics Objects having the same object__id need be equal in
width and height applies only within an Epoch. Graphics
Objects having the same object_id need not be equal in size
if they belong to different Epochs.
The last_in_sequence_flag field and the
object_data_flagment field are explained next. Due to a
constraint of payloads of PES packets, uncompressed graphics
constituting one subtitle may not be able to be contained
in one ODS. In such a case, the graphics is split into a
plurality of fragments and one of such fragments is carried
in the ob j ect_data_f ragment field. When storing one graphics
Object across a plurality of ODSs, every fragment except the
last fragment is of the same size. That is, the last fragment
is less than or equal to the size of the preceding fragments.
The ODSs carrying these fragments of the graphics Ob j ect appear
in the DS in sequence. The last_in_sequence_flag field
indicates an end of the graphics Object. Though the above
ODS data structure is based on a method of storing fragments
in consecutive PES packets without a gap, the fragments may
instead be stored in PES packets so as to leave some gaps

in the PES packets.
The PDS is a functional Segment for defining a Palette
used for color conversion. The Palette is data showing
combinations of Pixel Codes of 1 to 255 and pixel values.
A pixel value referred to here is made up of a red color
difference component (Cr value), a blue color difference
component (Cb value), a luminance component (Y value), and
a transparency (T value). Substituting a Pixel Code of each
piece of run-length data into a pixel value on the Palette
produces a color. FIG. 7B shows a data structure of the PDS.
As shown in the drawing, the PDS includes a segment_type field
showing a Segment type "PDS", a segment_length field showing
a data length of the PDS, a palette_id field uniquely
identifying the Palette, a palette_version_number field
showing a version of the PDS within the Epoch, and a
palette_entry field carrying information for each entry. The
palette_entry field shows a red color difference component
(Cr_value), a blue color difference component (Cb_value),
a luminance component (Y_value), and a transparency (T_value)
for each entry.
The WDS is a functional Segment for defining a
rectangular area on the Graphics Plane. As mentioned earlier,
memory management is continuous within an Epoch during which
clearing and rendering are performed in a fixed rectangular
area on the Graphics Plane. This rectangular area on the
Graphics Plane is called a Window, which is defined by the
WDS. FIG. 8A shows a data structure of the WDS. As shown
in the drawing, the WDS includes a window_id field uniquely

identifying the Window on the Graphics Plane, a
window_horizontal_position field specifying a horizontal
position of a top left pixel of the Window on the Graphics
Plane, a window_vertical_position field specifying a vertical
position of the top left pixel of the Window on the Graphics
Plane, a window_width field specifying a width of the Window
on the Graphics Plane, and a window_height field specifying
a height of the Window on the Graphics Plane.
The window_horizontal_position field, the
window_vertical_position field, the window_width field, and
the window_height field can take the following values. The
Graphics Plane serves as a coordinate system for these field
values. This Graphics Plane has a two-dimensional size
defined by video_height and video_width parameters.
The window_horizontal_position field specifies the
horizontal position of the top left pixel of the Window on
the Graphics Plane, and accordingly takes a value in a range
of 0 to (videp_width)~l. The window_vertical_position field
specifies the vertical position of the top left pixel of the
Window on the Graphics Plane, and accordingly takes a value
in a range of 0 to (video_height)-1.
The window_width field specifies the width of the Window
on the Graphics Plane, and accordingly takes a value in a
range of 1 to (video_width)-(window_horizontal_position).
The window_height field specifies the height of the Window
on the Graphics Plane, and accordingly takes a value in a
range of 1 to (video_height) - (window_vertical_position) .
A position and size of a Window can be defined for each

Epoch, using these window_horizontal_position,
window_vertical_position, window_width, and window_height
fields in the WDS. This makes it possible for the author to
adjust, at the time of authoring, a Window to appear in a
desired margin of each picture in an Epoch so as not to interfere
with a pattern of the picture. Graphics for subtitles
displayed in this way can be viewed clearly. The WDS can be
defined for each Epoch. Accordingly, when the pattern of the
picture changes with time, graphics can be moved based on
such a change so as not to decrease visibility. This enhances
the quality of the film to the same level as in the case where
subtitles are integrated in a moving picture.
The following explains the END. The END is a functional
Segment indicating that the transmission of the DS is complete .
The END is positioned immediately after the last ODS in the
DS. The END includes a segment_type field showing a Segment
type "END" and a segment_length field showing a data length
of the END. These fields are not main features of the present
invention and therefore their explanation has been omitted.
The following explains the PCS (Composition Segment).
The PCS is a functional Segment for composing a screen
that can be synchronized with a moving picture. FIG. 8B shows
a data structure of the PCS. As shown in the drawing, the
PCS includes a segment_type field, a segment_length field,
a composition_number field, a composition_state field, a
palette_update_flag field, a palette_id_ref field, and
composition_object(l) to composition_object(m) fields.
The composition_number field uniquely identifies a

graphics update in the DS, using a number from 0 to 15 . In
more detail, the composition__number field is incremented by
1 for each graphics update from the beginning of the Epoch
to the PCS.
The composition_state field indicates whether the DS
is a Normal Case DS, an Acquisition Point DS, or an Epoch
Start DS.
The palette_update_flag field shows whether the PCS
describes a Palette-only Display Update. The Palette-only
Display Update refers to such an update that only replaces
a previous Palette with a new Palette. To indicate a
Palette-only Display Update, the palette_update_flag field
is set to 1.
The palette_id_ref field specifies the Palette to be
used in the DS.
The composition_object(1) to composition_object(m)
fields each contain control information for realizing a screen
composition in the DS. In FIG. 8B, dashed lines wdl indicate
an internal structure of composition_object (i) as one example.
As illustrated, composition_object(i) includes an
object_id_ref field, a window_id_ref field, an
object_cropped_flag field, an object_horizontal_position
field, an object_vertical_position field, and
cropping_rectangle information(1) to cropping_rectangle
information(n).
The object_id_ref field shows a reference value of an
identifier of a graphics Object (object_id) . This reference
value specifies the graphics Object that is to be used in

order to produce a screen composition corresponding to
composition_object(i).
The window_id_ref field shows a reference value of an
identifier of a Window (window_id) . This reference value
specifies the Window in which the graphics Object is to be
displayed in order to produce the screen composition
corresponding to composition_object(i).
The object_cropped_flag field shows whether the
graphics Object cropped in the Object Buffer is to be displayed
or not. When the object_cropped_flag field is set to 1, the
graphics Object cropped in the Object Buffer is displayed.
When the object_cropped_flag field is set to 0, the graphics
Object cropped in the Object Buffer is not displayed.
The object_horizontal_position field specifies a
horizontal position of a top left pixel of the graphics Object
on the Graphics Plane.
The object_vertical_position field specifies a
vertical position of the top left pixel of the graphics Object
on the Graphics Plane.
The cropping_rectangle information(1) to
cropping_rectangle information(n) fields are valid when the
object_cropped_flag field value is 1. Dashed lines wd2
indicate an internal structure of cropping_rectangle
information(i) as one example. As illustrated,
cropping_rectangle information(i) includes an
object_cropping_horizontal_position field, an
object_cropping_vertical_position field, an
object_cropping_width field, and an object_cropping_height

field.
The object_cropping_horizontal_position field
specifies a horizontal position of a top left corner of a
cropping rectangle in the graphics Object. The cropping

rectangle is used for taking out one part of the graphics
Object, and corresponds to a "region" in ETSI EN 300 743.
The object_cropping_vertical_position field specifies
a vertical position of the top left corner of the cropping
rectangle in the graphics Object.
The object_cropping_width field specifies a horizontal
length of the cropping rectangle in the graphics Object.
The object_cropping_height field specifies a vertical
length of the cropping rectangle in the graphics Object.
The following explains a specific description of the
PCS,'using an example where the three subtitles "Actually",
"I lied to you.", and "Sorry." shown in FIG. 6 are displayed
sequentially by three operations of writing to the Graphics
Plane as the reproduction of the moving picture progresses.
FIG. 9 shows an example description for realizing such
subtitling. In the drawing, an Epoch has DS1 (Epoch Start
DS), DS2 (Normal Case DS), and DS3 (Normal Case DS). DS1
includes a WDS defining a Window in which the subtitles are
to be displayed, an ODS showing the line "Actually I lied
to you. Sorry.", and a PCS . DS2 includes a PCS . DS3 includes
a PCS.
Each of these PCSs has the following description . FIGS .
10 to 12 show example descriptions of the WDS and the PCSs
belonging to DS1 to DS3.

FIG. 10 shows descriptions of the PCS and the WDS in
DS1. In the drawing, a window_horizontal_position field
value and a window_vertical_position field value in the WDS
specify top left coordinates LP1 of the Window on the Graphics
Plane, and a window_width field value and a window_height
field value in the WDS specify a width and height of the Window.
An ob j ect_cropping_horizontal_position field value and
an object_cropping_vertical_position field value of
cropping_rectangle information in the PCS specify top left
coordinates ST1 of a cropping rectangle in a coordinate system
whose origin is top left coordinates of the graphics Object
in the Object Buffer. The cropping rectangle is an area
(enclosed by a thick-line box) defined by an
object_cropping_width field value and an
object_cropping_height field value from top left coordinates
STl. A cropped graphics Object is positioned in area cp1
(enclosed by a dashed-line box) so that a top left corner
of the cropped graphics Object lies at a pixel specified by
an object_horizontal_position field value and an
ob j ect_vertical_position field value in the coordinate system
of the Graphics Plane. In this way, the subtitle "Actually"
out of "Actually I lied to you. Sorry." is written into the
Window on the Graphics Plane. The subtitle "Actually" is
overlaid on a picture and a resultant image is displayed.
FIG. 11 shows a description of the PCS in DS2. Since
the description of the WDS in the drawing is the same as that
in FIG. 10, its explanation has been omitted. Meanwhile, the
description of cropping_rectangle information differs from

that in FIG. 10. In FIG. 11, an
object_cropping_horizontal_position field value and an
object_cropping_vertical_position field value in
cropping_rectangle information specify top left coordinates
of a cropping rectangle corresponding to the subtitle "I lied
to you. " in the Object Buffer, and an object_cropping_height
field value and an object_cropping_width field value specify
a height and width of the cropping rectangle. As a result,
the subtitle "I lied to you." is written into the Window on
the Graphics Plane . The subtitle "I lied to you . " is overlaid
on a picture and a resultant image is displayed.
FIG. 12 shows a description of the PCS in DS3 . Since
the description of the WDS in the drawing is the same as that
in FIG. 10, its explanation has been omitted. Meanwhile, the
description of cropping_rectangle information differs from
that in FIG. 10. In FIG. 12, an
object_cropping_horizontal_position field value and an
object_cropping_vertical_position field value specify top
left coordinates of a cropping rectangle corresponding to
the subtitle "Sorry." in the Object Buffer, and an
object_cropping_height field value and an
object_cropping_width field value specify a height and width
of the cropping rectangle. As a result, the subtitle "Sorry. "
is written into the Window on the Graphics Plane. The subtitle
"Sorry." is overlaid on a picture and a resultant image is
displayed. FIG. 13 shows a memory space of the Object Buffer
when performing graphics updates such as those shown in FIG.
10 to 12. As illustrated, the Object Buffer has four storage

areas A to D which each have a fixed height and width and
a fixed position . Of storage areas A to D, the subtitle shown
in FIG. 10 is stored in storage area A. Each of storage areas
A to D is identified by an object_id corresponding to a graphics
Object to be stored in that storage area. In detail, storage
area A is identified by object_id=l, storage area B is
identified by object_id=2, storage area C is identified by
object_id=3, and storage area D is identified by object_id=4 .
To maintain an efficiency of transfer from the Object Buffer

to the Graphics Plane, the height and width of each of storage
areas A to D are fixed. This being so, when a graphics Object
having some object_id is obtained as a result of decoding,
that graphics Object is written in a storage area identified
by the ob j ect_id over an existing graphics Ob j ect. For example,
to display a subtitle in the same position and size as the
subtitles displayed in FIGS. 10 to 12, an ODS having the same
object_id as the ODS in DS1 needs to be provided in a succeeding
DS. By adding the same object_id in such a way, a graphics
Object in the Object Buffer is overwritten by a new graphics
Object, which is displayed in the same position and size as
the overwritten graphics Object.
The following explains constraints for achieving
display effects. To display subtitles smoothly, it is
necessary to perform clearing and rendering on a Window. When
performing Window clearing and Window rendering at a frame
rate of video frames, the following rate of transfer from
the Object Buffer to the Graphics Plane is required.
First, a constraint on the size of the Window is examined.

Let Rc be the transfer rate from the Object Buffer to the
Graphics Plane. In a worst-case scenario, the Window clearing
and the Window rendering need to be performed at Rc. In other
words, each of the Window clearing and the Window rendering
needs to be performed at half of Rc (Rc/2).
To synchronize the Window clearing and the Window
rendering with a video frame,
(Window size) x (frame rate)≈Rc/2
needs to be satisfied. If the frame rate is 29.97,
Rc=(Window size)x2x29.97
To display a subtitle, the size of the Window needs to
be at least about 25% to 33% of the entire Graphics Plane.
If a total number of pixels of the Graphics Plane is 1920x1080
and a bit length of an index per pixel is 8 bits, a total
capacity of the Graphics Plane is 2 Mbytes (≈1920x1080x8).
Suppose the size of the Window is 1/4 of the Graphics
Plane, i.e., 500 Kbytes (=2Mbytes/4). Substituting this to
the above formula yields Rc=256Mbps (500Kbytesx2x29.97) .
Thus, if the size of the Window is about 25% to 33% of
the Graphics Plane, display effects of subtitles can be
achieved without losing synchronization with a moving picture,
so long as the subtitles are displayed with Rc=256Mbps.
If the Window clearing and the Window rendering may be
performed at 1/2 or 1/4 of the video frame rate, the size

of the Window can be doubled or quadrupled with the same Rc.
The following explains a position and range of a Window. As
mentioned earlier, a position and range of a Window are fixed
within an Epoch, for the following reason.
If the position or range of the Window varies in the
Epoch, a write address to the Graphics Plane needs to be changed.
This incurs overhead, which causes a drop in transfer rate
Rc from the Object Buffer to the Graphics Plane.
A number of graphics Objects that can be displayed
simultaneously in one Window is limited, in order to reduce
overhead when transferring decoded graphics Objects to the
Graphics Plane. The overhead mentioned here occurs when
setting addresses of edge parts of the graphics Objects . This
overhead increases if the number of edge parts is greater.
If there is no limitation on the number of graphics
Objects that can be displayed in one Window, the overhead
occurs unlimitedly when transferring graphics objects to the
Graphics Plane, which increases a variation in transfer load.
On the other hand, if the number of graphics Objects in one
Window is limited to 2, transfer rate Rc can be set on an
assumption that the number of instances of overhead is 4 at
the worst. Hence a minimum standard for transfer rate Rc can
be determined easily. This completes the explanation on a
Window.
The following explains how DSs carrying functional
Segments such as PCSs and ODSs described above are allocated
on the reproduction time axis of the AV Clip. An Epoch is
a time period on the reproduction time axis during which memory

management is continuous, and is made up of one or more DSs .
Hence it is important to effectively allocate DSs on the
reproduction time axis of the AV Clip. The reproduction time
axis of the AV Clip mentioned here is a time axis for defining
decoding times and presentation times of individual pictures
which constitute the video stream multiplexed in the AV Clip.
Decoding times and presentation times on the reproduction
time axis are expressed with a time accuracy of 90KHz. DTSs
and PTSs of PCSs and ODSs in DSs specify timings for synchronous
control on this reproduction time axis. In other words, the
DSs are allocated on the reproduction time axis by exercising
synchronous control using the DTSs and PTSs of the PCSs and
ODSs.
Synchronous control exercised using a DTS and a PTS of
an ODS is explained first.
The DTS shows a time at which a decoding process of the
ODS is to be started, with an accuracy of 90KHz . The PTS shows
a time at which the decoding process of the ODS is to be completed,
with an accuracy of 90KHz.
The decoding process is made up of decoding the ODS and
transferring an uncompressed graphics Object generated by
the decoding to the Object Buffer. This decoding process does
not complete instantaneously, but requires a certain length
of time. The DTS and the PTS of the ODS respectively show
the decoding start time and the decoding end time of the ODS,
to specify the beginning and end of the decoding process.
Since the time shown by the PTS is a deadline, it is
necessary to decode the ODS and store an uncompressed graphics

Object in the Object Buffer by the time shown by the PTS.
A decoding start time of arbitrary ODSj in DSn is
specified by DTS (DSn [ODSj ] ) with an accuracy of 90KHz. This
being so, a decoding end time of ODSj in DSn (i.e.
PTS (DSn [ODSj] ) is a sum of DTS (DSn [ODSj ] ) and a maximum time
required for a decoding process.
Let SIZE (DSn [ODSj] ) denote a size of ODSj , and Rd denote
an ODS decoding rate. Then the maximum time required for the
decoding process (in seconds) is SIZE(DSn[ODSj])//Rd. The
symbol "//" represents an operator for a division with a
fractional part being rounded up.
By converting this maximum time to the accuracy of 9 0KHz
and adding the result to the DTS of ODSj , the decoding end
time of ODSj specified by the PTS is calculated with the
accuracy of 90KHz.
This PTS of ODSj in DSn can be expressed by the following
formula:
PTS(DSn[ODSj])=DTS(DSn[ODSj])+90, 000
x(SIZE(DSn[ODSj])//Rd)
Also, two adjacent ODSs (ODSj and ODSj+1) in DSn need
to satisfy the following relationship:
PTS(DSn[ODSj]) An END in DSn indicates an end of DSn. Therefore, the
END shows a decoding end time of a last ODS (ODSlast) in DSn.

The decoding end time of ODSlast is shown by a PTS of ODSlast
(PTS(DSn[ODSlast] ) ) , so that a PTS of the END is set as follows:
PTS(DSn[END])=PTS(DSn[ODSlast])
Meanwhile, a DTS and a PTS of a PCS in DSn are set in
the following manner.
The DTS of the PCS shows either a decoding start time
of a top ODS (0DS1) in DSn or a time earlier than that. This
is because the PCS needs to be loaded in a buffer of the
reproduction apparatus 200 at the same time as or earlier
than the decoding start time of ODS1 (DTS(DSn[ODS1] ) ) and
a time at which a top PDS (PDS1) in DSn becomes valid
(PTS(DSn[PDSl] ) ) . Which is to say, the DTS of the PCS needs
to satisfy the following formulas:
DTS(DSn[PCS]) DTS(DSn[PCS]) On the other hand, the PTS of the PCS is calculated as
follows:
PTS(DSn[PCS])>DTS(DSn[PCS])
+DECODEDURATION(DSn)
Here, DECODEDURATION(DSn) indicates a time required for
decoding and presenting all graphics Objects used for updates
described in the PCS in DSn. Though DECODEDURATION(DSn) is

not a fixed value, it will not be affected by factors such
as differences in state or implementation of reproduction
apparatuses. When a graphics Object used for a screen
composition described by the PCS in DSn is denoted by
DSn.PCS.OBJ[j], DECODEDURATION(DSn) is varied by (i) a time
required for Window clearing, (ii) a time required for decoding
DSn.PCS.OBJ[j], and (iii) a time required for writing
DSn.PCS.OBJ[j] on the Graphics Plane. Accordingly,
DECODEDURATION (DSn) is the same regardless of implementations
of reproduction apparatuses, so long as Rd and Rc are
predetermined. Therefore, the length of each of the above
time periods is calculated to specify the PTS of the PCS,
at the time of authoring.
The calculation of DECODEDURATION(DSn) is carried out
based on a program shown in FIG. 14. FIG. 15 and FIGS. 16A
and 16B are flowcharts showing an algorithm of this program.
A procedure of calculating DECODEDURATION (DSn) is explained
below, with reference to these drawings. In FIG. 15, a
PLANEINITIALIZATIONTIME function is called, and a return
value is added to decode_duration (S1) . The
PLANEINITIALIZATIONTIME function (FIG. 16A) is a function
for calculating a time required for initializing the Graphics
Plane in order to produce display for DSn. In step S1, this
PLANEINITIALIZATIONTIME function is called using DSn,
DSn.PCS.OBJ[0], and decode_duration as arguments.
FIG. 16A shows a procedure of the
PLANEINITIALIZATIONTIME function. In the drawing,
initialize_duration is a variable indicating a return value

of the PLANEINITIALIZATIONTIME function.
Step S2 judges whether a composition_state field of the
PCS in DSn shows Epoch Start. If the composition_state field
shows Epoch Start (S2:YES,
DSn.PCS.composition_state==EPOCH_START in FIG. 14 ) , a time
required for clearing the Graphics Plane is set as
initialize_duration (S3).
Suppose transfer rate Rc from the Object Buffer to the
Graphics Plane is 256, 000, 000 and a total size of the Graphics
Plane is (video_width)*(video_height), as mentioned above.
Then the time required for clearing the Graphics Plane (in
seconds) is (video_width)*(video_height)//256,000,000.
This is multiplied by 90, 000Hz , to express in the PTS accuracy .
Hence the time required for clearing the Graphics Plane is
90,000x(video_width)*(video_height)//256,000,000. This
is added to initialize_duration, which is returned as a return
value.
If the composition_state field does not show Epoch Start
(S2:NO), an operation of adding a time required for clearing
Window[i] to initialize_duration is carried out for all
Windowsfi] (S4). Suppose transfer rate Rc from the Object
Buffer to the Graphics Plane is 256,000,000 as mentioned
earlier, and a total size of Windows [i] is ∑SIZE(WDS.WIN[i]).
Then a time required for clearing all Windows [i] (in seconds)
is ∑SIZE(WDS.WIN[i])//256,000,000. This is multiplied by
90,000Hz to express in the PTS accuracy. Hence the time
required for clearing all Windows[i] is
90, 000x∑SIZE(WDS.WIN[i])//256,000,000. This is added to

initialize_duration, which is returned as a return value.
This completes the PLANEINITIALIZATIONTIME function.
Referring back to FIG . 15, step S5 judges whether a number
of graphics Objects in DSn is 1 or 2
(if(DSn.PCS.num_of_objects==2,
if(DSn.PCS.num_of_objects==1 in FIG. 14). If the number of
graphics Objects in DSn is 1 (S5:=1), a wait time for decoding
that graphics Object to complete is added to decode_duration
(S6) . The wait time is calculated by calling a WAIT function
(decode_duration+=WAIT(DSn, DSn.PCS.OBJ[0],
decode_duration) in FIG. 14). The WAIT function is called
using DSn, DSn . PCS . OBJ [ 0 ] , and decode_duration as arguments,
and wait_duration showing the wait time is returned as a return
value.
FIG. 16B shows a procedure of the WAIT function.
In the WAIT function, current_duration is a variable
to which decode_duration is set, and
object_definition_ready_time is a variable indicating a PTS
of graphics Object OBJ[i] in DSn.
Also, current_time is a variable indicating a sum of
current_duration and a DTS of the PCS in DSn. If
object_definition_ready_time is greater than current_time
(S7.-YES, if (current_time FIG. 14) , a difference between object_definition_ready_time
and current_time is set to wait_duration, which is returned
as a return value (S8, wait_duration +=
object_definition_ready_time - current_time in FIG. 14).
This completes the WAIT function.

Referring back to FIG. 15, a sum of the return value
of the WAIT function and a time required for rendering on
a Window to which OBJ[0] belongs
(90,000*(SIZE(DSn.WDS.WIN[0]))//256,000,000) is set to
decode_duration (S9).
The above procedure relates to the case when the number
of graphics Objects in DSn is 1. If the number of graphics
Objects is 2 (S5:=2, if(DSn.PCS.num_of_objects==2) in FIG.
14), the WAIT function is called using DSn, DSn.PCS.OBJ[0] ,
and decode_duration as arguments, and a return value of the
WAIT function is added to decode_duration (S10).
Step S11 judges whether the Window to which OBJ[0]
belongs is the same as a Window to which OBJ[l] belongs
(if(DSn.PCS.OBJ[0].window_id == DSn.PCS.OBJ[1].window_id)
in FIG. 14). If the judgement is in the affirmative (S11: YES) ,
the WAIT function is called using DSn, DSn.PCS.OBJ[1], and
decode_duration as arguments, and a return value of the WAIT
function is added to decode_duration (S12). Furthermore, a
time required for rendering on the Window to which OBJ[0]
and OBJ[l] belong
(90,000*(SIZE(DSn.WDS.OBJ[0] . window_id)//256,000 , 000) is
added to decode_duration (S13).
If the judgement is in the negative (S11:NO), on the
other hand, the time required for rendering on the Window
to which OBJ[0] belongs
(90,000*(SIZE(DSn.WDS.OBJ[0] .window_id)//256,000,000) is
added to decode_duration (S15) . After this, the WAIT function
is called using DSn, DSn.PCS.OBJ[1], and decode_duration as

arguments, and a return value of the WAIT function is added
to decode_duration (S16). Furthermore, a time required for
rendering on the Window to which 0BJ[1] belongs
(90,000*(SIZE(DSn.WDS.OBJ[l] .window_id)//256,000 , 000) is
added to decode_duration (S17). In this way,
DECODEDURATION(DSn) is calculated.
The following explains how a PTS of a PCS in one DS is
set, using specific examples.
FIG. 17A shows a situation where one OBJ (OBJ1)
corresponding to one ODS (ODS1) belongs to one Window. FIGS.
17B and 17C are timing charts showing a relationship between
parameters used in FIG. 14. Each of these timing charts has
three levels. Of the three levels, the "Graphics Plane access"
level and the "ODS decode" level indicate two processes which
are performed in parallel when reproducing the ODS . The above
algorithm is based on an assumption that these two processes
are performed in parallel.
Graphics Plane access is made up of clear period (1)
and write period (3). Clear period (1) indicates either a
time required for clearing the entire Graphics Plane
(90,000*((size of the Graphics Plane)//256,000,000)) or a
time required for clearing all Windows on the Graphics Plane
(2(90,000x((size of Window[i])//256,000,000)) ) .
Write period ( 3 ) indicates a time required for rendering
on the entire Window (90,000x((size of the
Window)//256,000,000)).
ODS decode is made up of decode period (2) . Decode period
(2) indicates a time period from a DTS to a PTS of ODS1.

Clear period (1), decode period (2), and write period
(3) can vary depending on the range to be cleared, the size
of an ODS to be decoded, and the size of a graphics Object
to be written to the Graphics Plane . In FIG. 17, the beginning
of decode period ( 2) is assumed to be the same as the beginning
of clear period (1), for simplicity's sake.
FIG. 17B shows a case where decode period (2) is longer
than clear period (1). In this case, decode_duration is a
sum of decode period (2) and write period (3).
FIG. 17C shows a case where clear period (1) is longer
than decode period (2). In this case, decpde_duration is a
sum of clear period (1) and write period (3).
FIGS. 18A to 18C show a situation where two OBJs (0BJ1
and 0BJ2) corresponding to two ODSs (0DS1 and 0DS2) belong
to one Window. In FIGS. 18B and 18C, decode period (2)
indicates a total time required for decoding 0DS1 and 0DS2 .
Likewise, write period (3) indicates a total time required
for writing 0BJ1 and 0BJ2 to the Graphics Plane. Though the
number of ODSs is two, decode_duration can be calculated in
the same way as in FIG. 17. In detail, if decode period (2)
of 0DS1 and 0DS2 is longer than clear period (1),
decode_duration is a sum of decode period (2) and write period
(3) as shown in FIG. 18B.
If clear period (1) is longer than decode period (2),
decode_duration is a sum of clear period (1) and write period
(3) as shown in FIG. 18C.
FIGS. 19A to 19C show a situation where OBJ1 belongs
to Windowl and 0BJ2 belongs to Window2. In this case too,

if clear period (1) is longer than decode period (2) of 0DS1
and 0DS2, decode_duration is a sum of clear period (1) and
write period (3). If clear period (1) is shorter than decode
period (2), on the other hand, 0BJ1 can be written to Windowl
without waiting for the end of decode period (2). In such
a case, decode_duration is not simply a sum of decode period
(2) and write period (3) . Let write period (31) denote a time
required for writing OBJ1 to Windowl and write period (32)
denote a time required for writing 0BJ2 to Window2 . FIG . 19B
shows a case where decode period (2) is longer than a sum
of clear period (1) and write period (31). In this case,
decode_duration is a sum of decode period (2) and write period
(32).
FIG. 19C shows a case where a sum of clear period (1)
and write period (31) is longer than decode period (2). In
this case, decode_duration is a sum of clear period (1) , write
period (31), and write period (32).
The size of the Graphics Plane is fixed according to
a player model. Also, sizes and numbers of Windows and ODSs
are set in advance at the time of authoring. Hence
decode_duration can be calculated as one of the sum of clear
period (1) and write period (3), the sum of decode period
(2) and write period (3), the sum of decode period (2) and
write period (32), and the sum of clear period (1), write
period (31), and write period (32). By setting the PTS of
the PCS based on such calculated decode_duration, graphics
can be synchronized with picture data with high accuracy.
Such accurate synchronous control is achieved by defining

Windows and restricting clearing and rendering operations
within the Windows. Thus, the introduction of the concept
"Window" in authoring is of great significance.
The following explains how a DTS and a PTS of the WDS
in DSn are set. The DTS of the WDS is set so as to satisfy
the following formula:
DTS(DSn[WDS])≥DTS(DSn[PCS])
The PTS of the WDS specifies a deadline for starting
writing to the Graphics Plane. Since writing to the Graphics
Plane is restricted to a Window, the time to start writing
to the Graphics Plane can be determined by subtracting a time
required for rendering on all Windows from the time shown
by the PTS of the PCS . Let ∑SIZE(WDS . WIN [ i ] ) be a total size
ofWindows[i] . Then a time required for clearing and rendering
on all Windows[i] is ∑SIZE(WDS.WIN[i])//256,000,000.
Expressing this time with the accuracy of 90,000KHz yields
90,000x∑siZE(WDS.WIN[i])//25 6,000,000.
Accordingly, the PTS of the WDS can be calculated as
follows:
PTS(DSn[WDS])=PTS(DSn[PCS])-
90,000x∑SIZE(WDS.WIN[i])//256,000,000
Since the PTS of the WDS shows the deadline, the writing
to the Graphics Plane can be launched earlier than the time
shown by this PTS. Which is to say, once decoding of one ODS

belonging to one of two Windows has completed, a graphics
Object obtained by the decoding can be immediately written
to the Window as shown in FIG. 19.
Thus, a Window can be allocated to a desired point on
the reproduction time axis of the AV Clip, using the DTS and
the PTS of the WDS. This completes the explanation on the
DTS and PTS of each of the PCS and the WDS in DSn.
The following explains a forward reference by a PCS.
A forward reference is a type of reproduction control that
references a graphics Object which has already been stored
in the reproduction apparatus 200. FIG. 20 shows forward
reference PCSs . The drawing shows three DSs, i.e. DSn, DSn+1,
and DSn+2 . PCSs of DSn+1 and DSn+2 are forward reference PCSs .
Arrows yy1 and yy2 indicate forward references made by these
PCSs. Graphics Objects defined in ODS#1 to ODS#v in DSn are
forward-referenced by these PCSs. Here, DSn contains a
plurality of graphics Objects, i.e. ODS#1 to ODS#u and ODS#u+l
to ODS#v. Of these graphics Objects, ODS#1 to ODS#u are
referenced by a PCS of DSn, but ODS#u+l to ODS#v are not
referenced by the PCS of DSn. These graphics Objects not
referenced by the PCS of DSn are forward-referenced by the
PCSs of DSn+1 and DSn+2.
Such a graphics Object forward-referenced by a PCS in
one DS has already been stored in the reproduction apparatus
200, so that there is no need to wait for decoding of an ODS.
This being so, by feeding only a PCS into the reproduction
apparatus 200 , display control based on that PCS can be carried
out. Since the display control is carried out promptly when

the PCS is fed into the reproduction apparatus 200, a graphics
update of replacing existing graphics with new graphics can
be performed with a short time interval. Repeating such a
short-interval graphics update makes it possible to realize
display control of moving graphics smoothly as the
reproduction of the moving picture progresses. Two
description examples of forward reference PCSs are given
below.
A description example of forward reference PCSs for

realizing a display effect of moving graphics on a screen
is explained first. FIG. 21A shows an Epoch having such a
display effect. This Epoch is made up of nine DSs (DSO to
DS8) . DSO, the first DS of the Epoch, includes a PCS, a PDS,
and an ODS. The ODS in DSO defines a subtitle "My heart is
fluttering" shown in FIG. 22A, and has object_id=1. Meanwhile,
DS1 to DS8 which follow DSO each include only a PCS.
FIG. 21B shows the composition_object fields and PTSs
of the PCSs in DS1 to DS8. As illustrated, the PCSs in DS1
to DS8 designate the graphics Object specified by
object_id_ref=1 to be displayed at respective coordinates
(x1,y1), (x2,y2), (x3,y3), ... , (x8,y8) specified by the
object_horizontal_position and object_vertical_position
fields. FIG. 22B shows positions of coordinates (xl,yl),
(x2,y2), (x3,y3), ... , (x8,y8) in a coordinate system defined
by a Window. As can be understood from the drawing,
coordinates (x1,y1), (x2,y2), (x3,y3), .„ , (x8,y8) lie on
a wavy line. Coordinates (x1,y1), (x2,y2), (x3,y3), ... ,
(x8,y8) are set so that the graphics Object stays within the

Window defined by the WDS . This is because the synchronization
between the graphics display of the graphics stream and the
video display of the video stream cannot be established if
the graphics Object is not contained within the Window.
In FIG. 21B, the PTSs of the PCSs in DS1 to DS8 designate
times t1, t2, t3, ... , t8, on the reproduction time axis of
the video stream, at which the graphics Object is to be
displayed at respective coordinates (x1, y1) , (x2,y2), (x3,y3),
... , (x8,y8).
FIG. 23 shows a display effect when DSO to DS8 having
the above PCSs are sequentially fed into the reproduction
apparatus 200. At t1 on the reproduction time axis of the
video stream, DS1 is fed into the reproduction apparatus 200,
and as a result the graphics is displayed at coordinates (x1, y1)
within the Window. At t4 , DS4 is fed into the reproduction
apparatus 200, and as a result the graphics is displayed at
coordinates (x4,y4) . At t6, DS6 is fed into the reproduction
apparatus 200, and as a result the graphics is displayed at
coordinates (x6,y6) . At t8, DS8 is fed into the reproduction
apparatus 200, and as a result the graphics is displayed at
coordinates (x8,y8). Since these coordinates (x1,y1),
(x4,y4), (x6,y6), and (x8,y8) lie on a wavy line, the graphics
will end up tracing the wavy line as shown in FIG. 24.
Thus, when such DSs including PCSs which each have
specified object_horizontal_position and
object_vertical_position field values are sequentially fed
into the reproduction apparatus 200, the reproduction
apparatus 200 displays the graphics Object already stored

in the Object Buffer in accordance with these PCSs. Since
the position of the graphics can be changed just by feeding
the control information into the reproduction apparatus 200,
quick movements of graphics on the screen can be achieved.
When feeding such DSs into the reproduction apparatus
200 , a new PCS may be written over an existing PCS . This allows
the reproduction apparatus 200 to hold only a newest PCS,
even if there are tens or hundreds of PCSs showing display
coordinates of graphics for smooth movement. In this way,
the memory occupancy in the reproduction apparatus 200 can
be minimized.
A Palette-only Display Update is explained next, as
another description example of forward reference PCSs. FIG.
25 shows a series of DSs for realizing a Palette-only Display
Update. In the drawing, the first level shows the series of
DSs (DS0, DS1, DS2, and DS3) for the Palette-only Display
Update . DS0 has a PCS , four PDSs ( PDS0 , PDS1, PDS2 , and PDS3 ) ,
and an ODS. DS2, DS3, and DS4 that follow DS0 each have only
a PCS.
The ODS in DS0 defines a subtitle "I will never forget" ,
as indicatedby guidelines hv1. The words" I", "will", "never",
and "forget" are each constituted by a plurality of pieces
of run-length data (indicated by rectangles shown in FIG.
25). Run-length data constituting "I" has Pixel Code 0.
Run-length data constituting "will" has Pixel Code 1.
Run-length data constituting "never" has Pixel Code 2.
Run-length data constituting "forget" has Pixel Code 3.
FIG. 26A shows the four PDSs in DS0. .

PDSO which is the first PDS in DSO (on the first level)
has palette_id=0, and assigns a red color to Pixel Code 0
and a white color to Pixel" Codes 1 to 3.
PDS1 which is the second PDS in DSO (on the second level)
has palette_id=1, and assigns the red color to Pixel Codes
0 and 1 and the white color to Pixel Codes 2 and 3.
PDS2 which is the third PDS in DSO (on the third level)
has palette_id=2, and assigns the red color to Pixel Codes
0, 1, and 2 and the white color to Pixel Code 3.
PDS3 which is the fourth PDS in DSO (on the fourth level)
has palette_id=3, and assigns the red color to Pixel Codes
0 to 3 .
FIG. 26B show descriptions of the PCSs in DSO to DS3 .
The PCS in DSO (on the first level) designates display-
control to be performed using a graphics Object specified
by object_id_ref=1. This PCS has palette_update_flag=0 and
palette_id_ref=0. Accordingly, when DSO is fed into the
reproduction apparatus 200, the graphics Object shown by the
ODS in DSO is displayed using the PDS specif ied by palette_id=0 .
The PCS in DS1 (on the second level) designates display
control to be performed using the graphics Object specified
by object_id_ref=1. This PCS has palette_update_f lag=1 and
palette_id_ref=1. Accordingly, when DS1 is fed into the
reproduction apparatus 200, the display of the graphics Ob j ect
is updated using the PDS specified by palette_id=1.
The PCS in DS2 (on the third level) designates display
control to be performed using the graphics Object specified
by object_id_ref=1. This PCS has palette_update_f lag=1 and

palette_id_ref = 2. Accordingly, when DS2 is fed into the
reproduction apparatus 200 , the display of the graphics Object
is updated using the PDS specified by palette_id=2.
The PCS in DS3 (on the fourth level) designates display
control to be performed using the graphics Object specified
by object_id_ref=1. This PCS has palette_update_flag=1 and
palette_id_ref = 3. Accordingly, when DS3 is fed into the
reproduction apparatus 200 , the display of the graphics Object
is updated using the PDS specif ied by palette_id=3 . By setting
the PCSs in DSO to DS3 in this way, it is possible to repeatedly
update the graphics Object belonging to DSO using different
palette data.
FIG. 27 shows a display effect achieved when these four
DSs are sequentially fed into the reproduction apparatus 200 .
When DSO is fed into the reproduction apparatus 200, the
subtitle "I will never forget" is displayed with the word
"I" being colored in red. WhenDS1 is fed into the reproduction
apparatus 200, the word "will" turns red. When DS2 is fed
into the reproduction apparatus 200, the word "never" turns
red. When DS3 is fed into the reproduction apparatus 200,
the word "forget" turns red, as a result of which the entire
subtitle "I will never forget" is displayed in red. Thus,
the words constituting the subtitle "I will never forget"
change from white to red one by one beginning with the leftmost
word, as the DSs are sequentially fed from the BD-ROM 100
to the reproduction apparatus 200. This produces a display
effect of changing a subtitle in color as the reproduction
of the audio stream progresses.

Note that the data structures of DSs (PCS, WDS, PDS,
and ODS) explained above are instances of class structures
written in a programming language . The author writes the class
structures according to the syntax defined in the Blu-ray
Disc Read-Only Format, to create these data structures on
the BD-ROM 100. This completes the explanation on the
recording medium according to the first embodiment of the
present invention. The following explains a reproduction
apparatus according to the first embodiment of the present
invention. FIG. 28 shows an internal construction of the
reproduction apparatus 200. The reproduction apparatus 200
is manufactured based on this internal construction. The
reproduction apparatus 200 is roughly made up of three parts
that are a system LSI, a drive device, and a microcomputer
system. The reproduction apparatus 200 can be manufactured
by mounting these parts on a cabinet and substrate of the
apparatus . The system LSI is an integrated circuit including
various processing units for achieving the functions of the
reproduction apparatus 200. The reproduction apparatus 200
includes a BD drive 1, a Read Buffer 2, a PID filter 3, Transport
Buffers 4a, 4b, and 4c, a peripheral circuit 4d, a Video Decoder
5, a Video Plane 6, an Audio Decoder 7, a Graphics Plane 8,
a CLUT unit 9, an adder 10, and a Graphics Decoder 12. The
Graphics Decoder 12 includes a Coded Data Buffer 13, a
peripheral circuit 13a, a Stream Graphics Processor 14, an
Object Buffer 15, a Composition Buffer 16, and a Graphics
Controller 17.
The BD drive 1 performs loading, reading, and ejecting

of the BD-ROM 100 . The BD drive 1 accesses to the BD-ROM 100 .
The Read Buffer 2 is a FIFO (first-in first-out) memory.
Accordingly, TS packets read from the BD-ROM 100 are removed
from the Read Buffer 2 in the same order as they arrive.
The PID filter 3 performs filtering on TS packets output
from the Read Buffer 2 . In more detail, the PID filter 3 passes
only TS packets having predetermined PIDs to the Transport
Buffers 4a, 4b, and 4c. There is no buffering inside the PID
filter 3. Accordingly, TS packets entering the PID filter
3 are instantaneously written to the Transport Buffers 4a,
4b, and 4c.
The Transport Buffers 4a, 4b, and 4c are FIFO memories
for storing TS packets output from the PID filter 3 . A speed
at which a TS packet is read from the Transport Buffer 4a
is denoted by transfer rate Rx.
The peripheral circuit 4d has a wired logic for
converting TS packets read from the Transport Buffer 4a to
functional Segments . The functional Segments are then stored
in the Coded Data Buffer 13.
The Video Decoder 5 decodes TS packets output from the
PID filter 3 to obtain uncompressed pictures, and writes them
to the Video Plane 6.
The Video Plane 6 is a plane memory for a moving picture.
The Audio Decoder 7 decodes TS packets output from the
PID filter 3, and outputs uncompressed audio data.
The Graphics Plane 8 is a plane memory having a memory
area of one screen, and is capable of storing uncompressed
graphics of one screen.

The CLUT unit 9 converts index colors of the uncompressed
graphics on the Graphics Plane 8 , based on Y, Cr, and Cb values
shown in a PDS.
The adder 10 multiplies the uncompressed graphics
converted by the CLUT unit 9, by a T value (transparency)
shown in the PDS. The adder 10 then performs addition for
corresponding pixels in the resulting uncompressed graphics
and the uncompressed picture data on the Video Plane 6 , and
outputs a resultant image.
The Graphics Decoder 12 decodes a graphics stream to
obtain uncompressed graphics, and writes the uncompressed
graphics to the Graphics Plane 8 as graphics Objects. As a
result of decoding the graphics stream, subtitles and menus
appear on the screen.
This Graphics Decoder 12 executes pipeline processing,
by reading a graphics Object belonging to DSn from the Object
Buffer 15 whilst simultaneously writing a graphics Object
belonging to DSn+1 to the Object Buffer 15.
The Graphics Decoder 12 includes the Coded Data Buffer
13, the peripheral circuit 13a, the Stream Graphics Processor
14, the Object Buffer 15, the Composition Buffer 16, and the
Graphics Controller 17 .
The Coded Data Buffer 13 is used for storing functional
Segments together with DTSs and PTSs. Such functional
Segments are obtained by removing a TS packet header and a
PES packet header from each TS packet stored in the Transport
Buffer 4a and arranging remaining payloads in sequence. DTSs
and PTSs contained in the removed TS packet headers and PES

packet headers are stored in the Coded Data Buffer 13 in
correspondence with the functional Segments.
The peripheral circuit 13a has a wired logic for
transferring data from the Coded Data Buffer 13 to the Stream
Graphics Processor 14 and transferring data from the Coded
Data Buffer 13 to the Composition Buffer 16 . In more detail,
when the current time reaches a DTS of an ODS, the peripheral
circuit 13a transfers the ODS from the Coded Data Buffer 13
to the Stream Graphics Processor 14. Also, when the current
time reaches a DTS of a PCS or a PDS, the peripheral circuit
13a transfers the PCS or the PDS from the Coded Data Buffer
13 to the Composition Buffer 16.
The Stream Graphics Processor 14 decodes the ODS to
obtain uncompressed graphics having index colors, and
transfers the uncompressed graphics to the Object Buffer 15
as a graphics Object. The decoding by the Stream Graphics
Processor 14 is instantaneous, and the graphics Object
obtained by the decoding is temporarily stored in the Stream
Graphics Processor 14. Though the decoding by the Stream
Graphics Processor 14 is instantaneous, the transfer of the
graphics Object from the Stream Graphics Processor 14 to the
Object Buffer 15 is not instantaneous. This is because
transfer to the Object Buffer 15 is performed at a transfer
rate of 128Mbps in the player model of the Blu-ray Disc
Read-Only Format. An end of transfer of all graphics Objects
belonging to a DS to the Object Buffer 15 is shown by a PTS
of an END in the DS. Therefore, processing of the next DS
will not be started until the time shown by the PTS of the

END. Transfer of a graphics Object obtained by decoding each
ODS to the Object Buffer 15 starts at a time shown by a DTS
of the ODS and ends at a time shown by a PTS of the ODS.
If a graphics Object of DSn and a graphics Object of
DSn+1 have different object_ids, the Stream Graphics
Processor 14 writes the two graphics Objects in different
storage areas of the Object Buffer 15. This allows pipeline
presentation of the graphics Objects, without the graphics
Object of DSn being overwritten by the graphics Object of
DSn+1. If the graphics Object of DSn and the graphics Object
of DSn+1 have the same object_id, on the other hand, the Stream
Graphics Processor 14 writes the graphics Object of DSn+1
to a storage area in the Object Buffer 15 in which the graphics
Object of DSn is stored, so as to overwrite the graphics Object
of DSn. In this case, pipeline processing is not performed.
Also, a DS may include ODSs which are referenced by a PCS
of the DS and ODSs which are not referenced by the PCS. The
Stream Graphics Processor 14 sequentially decodes not only
the ODSs referenced by the PCS but also the ODSs not referenced
by the PCS, and stores graphics obtained by the decoding to
the Object Buffer 15.
The Object Buffer 15 corresponds to a pixel buffer in
ETSI EN 300 743. Graphics Objects decoded by the Stream
Graphics Processor 14 are stored in the Object Buffer 15.
A size of the Object Buffer 15 needs to be twice or four times
as large as that of the Graphics Plane &. This is because
the Object Buffer 15 needs to be capable of storing twice
or four times as much graphics as the Graphics Plane 8 in

order to achieve scrolling.
The Composition Buffer 16 is used for storing a PCS and
a PDS. When active periods of PCSs in DSn and DSn+1 overlap,
the Composition Buffer 16 stores the PCSs of both DSn and
DSn+1.
The Graphics Controller 17 decodes the PCSs in the
Composition Buffer 16. Based on a decoding result, the
Graphics Controller 17 writes a graphics Object of DSn+1 to
the Object Buffer 15, while reading a graphics Object of DSn
from the Object Buffer 15 and presenting it for display. The
presentation by the Graphics Controller 17 is performed at
a time shown by a PTS of the PCS in DSn.
To produce a screen composition using a PCS, the Graphics
Controller 17 performs the following processing. The
Graphics Controller 17 reads an object_id_ref which is a
reference value of an identifier of a graphics Object, from
the PCS. The Graphics Controller 17 takes out one part of
the graphics Object that is defined by an
object_cropping_horizontal_position, an
object_cropping_vertical_position, an
object_cropping_width, and an object_cropping_height. The
Graphics Controller 17 writes the taken part on the Graphics
Plane 8 at a position shown by an object_horizontal_position
and an object_vertical_position. If the graphics Object
specified by the reference value belongs to the same DS as
the PCS, the above writing to the Graphics Plane 8 cannot
be performed until after a corresponding ODS in that DS is
decoded.

On the other hand, when a composition_state of the PCS
shows Normal Case, the graphics Ob j ect has already been decoded
and stored in the Object Buffer 15. Accordingly, the above
writing to the Graphics Plane 8 can be performed without waiting
for decoding of an ODS.
To effect a Palette-only Display Update, the Graphics
Controller 17 performs the following processing. Once a DS
has been read from the BD-ROM 100, the Graphics Controller
17 judges whether a palette_update_flag of a PCS in the DS
is "1". If the palette_update_flag is "1", the Graphics
Controller 17 instructs the CLUT unit 9 to perform pixel value
conversion using palette data specified by a palette_id of
the PCS. The Graphics Controller 17 then reads a graphics
Object from the Object Buffer 15 and writes it to the Graphics
Plane 8. Thus, a graphics update can be carried out just by
setting the palette data in the CLUT unit 9.
Recommended transfer rates and buffer sizes for
realizing the PID filter 3, the Transport Buffers 4a, 4b,
and 4c, the Graphics Plane 8, the CLUT unit 9, the Coded Data
Buffer 13 , the Stream Graphics Processor 14 , the Object Buffer
15, the Composition Buffer 16, and the Graphics Controller
17 are given below. FIG. 29 shows transfer rates Rx, Rc, and
Rd and sizes of the Graphics Plane 8, the Transport Buffer
4a, the Coded Data Buffer 13, and the Object Buffer 15.
Transfer rateRc ( Pixel Composition Rate) from the Object
Buffer 15 to the Graphics Plane 8 is a highest transfer rate
in the reproduction apparatus 200, and is calculated as 256Mbps
(=500Kbytesx29 .97x2) from a Window size and a frame rate.

Transfer rate Rd (Pixel Decoding Rate) from the Stream
Graphics Processor 14 to the Object Buffer 15 does not need
to coincide with the frame rate unlike Rc, and may be 1/2
or 1/4 of Rc . Therefore, transfer rate Rd is 128Mbps or 64Mbps .
Transfer rate Rx (Transport Buffer Leak Rate) from the
Transport Buffer 4a to the Coded Data Buffer 13 is a transfer
rate of ODSs in a compressed state. Accordingly, transfer
rate Rx can be calculated by multiplexing Rd by a compression
rate of ODSs. For example, when the compression rate is 25%,
transfer rate Rx is 16Mbps (=64Mbpsx25%) .
These transfer rates and buffer sizes are merely shown
as minimum standards, and transfer rates and buffer sizes
greater than those shown in FIG. 29 are equally applicable.
In the above constructed reproduction apparatus 200,
the construction elements perform processing in a pipeline.
FIG. 3 0 is a timing chart showing pipeline processing
performed in the reproduction apparatus 200 . In the drawing,
the fifth level shows a DS on the BD-ROM 100 . The fourth level
shows write periods of a PCS, a WDS, a PDS, ODSs, and an END
to the Coded Data Buffer 13. The third level shows decode
periods of the ODSs by the Stream Graphics Processor 14 . The
second level shows storage contents of the Composition Buffer
16. The first level shows operations of the Graphics
Controller 17.
DTSs of ODS1 and 0DS2 show t31 and t32 respectively.
Therefore, ODSl and ODS2 need to be buffered in the Coded
Data Buffer 13 by t31 and t32 respectively. This being so,
writing of ODSl to the Coded Data Buffer 13 is completed by

t31 at which decode period dpi begins, and writing of 0DS2
to the Coded Data Buffer 13 is completed by t32 at which decode
period dp2 begins.
Meanwhile, PTSs of 0DS1 and 0DS2 show t32 and t33
respectively. Accordingly, decoding of 0DS1 by the Stream
Graphics Processor 14 is completed by t32, and decoding of
0DS2 by the Stream Graphics Processor 14 is completed by t3 3 .
Thus, an ODS is buffered in the Coded Data Buffer 13 by a
time shown by a DTS of the ODS, and the buffered ODS is decoded
and transferred to the Object Buffer 15 by a time shown by
a PTS of the ODS.
On the first level, cd1 denotes a time period needed
for the Graphics Controller 17 to clear the Graphics Plane
8, and td1 denotes a time period needed for the Graphics
Controller 17 to write graphics obtained in the Object Buffer
15 to the Graphics Plane 8 . A PTS of the WDS shows a deadline
for starting writing the graphics . A PTS of the PCS shows
a time at which the writing of the graphics to the Graphics
Plane 8 ends and the written graphics is presented for display .
Therefore, uncompressed graphics of one screen is obtained
on the Graphics Plane 8 at the time shown by the PTS of the
PCS. The CLUT unit 9 performs color conversion on the
uncompressed graphics, and the adder 10 overlays the graphics
on an uncompressed picture stored on the Video Plane 6 . This
produces a resultant image.
In the Graphics Decoder 12, the Stream Graphics Processor
14 continues decoding while the Graphics Controller 17 is
clearing the Graphics Plane 8. As a result of such pipeline

processing, graphics can be displayed speedily.
FIG . 30 shows an example when the clearing of the Graphics
Plane 8 ends before the decoding of the ODSs. FIG. 31, on
the other hand, is a timing chart showing pipeline processing
when the decoding of the ODSs ends before the clearing of
the Graphics Plane 8. In this case, upon completion of the
decoding of the ODSs, the graphics obtained by the decoding
cannot yet be written to the Graphics Plane 8 . Only after
the clearing of the Graphics Plane 8 is completed, the graphics
can be written to the Graphics Plane 8.
FIG. 32 is a timing chart showing changes in buffer
occupancy in the reproduction apparatus 200 . In the drawing,
the first to fourth levels show changes in occupancy of the
Graphics Plane 8, the Object Buffer 15, the Coded Data Buffer
13, and the Composition Buffer 16, respectively. These
changes are shown in the form of a line graph in which the
horizontal axis represents time and the vertical axis
represents occupancy.
The fourth level shows changes in occupancy of the
Composition Buffer 16. As illustrated, the changes in
occupancy of the Composition Buffer 16 include monotone
increase Vf0 with which the PCS output from the Coded Data
Buffer 13 is stored.
The third level shows changes in occupancy of the Coded
Data Buffer 13. As illustrated, the changes in occupancy of
the Coded Data Buffer 13 include monotone increases Vf 1 and
Vf 2 with which ODS1 andODS2 are stored, and monotone decreases
Vg1 and Vg2 with which 0DS1 and ODS2 are sequentially read

by the Stream Graphics Processor 14. Slopes of monotone
increases Vf1 and Vf2 are based on transfer rate Rx from the
Transport Buffer 4a to the Coded Data Buffer 13, whereas
monotone decreases Vg1 and Vg2 are instantaneous since
decoding by the Stream Graphics Processor 14 is performed
instantaneously. Which is to say, the Stream Graphics
Processor 14 decodes each ODS instantaneously and holds
uncompressed graphics obtained by the decoding. Since
transfer rate Rd from the Stream Graphics Processor 14 to
the Object Buffer 15 is 128Mbps, the occupancy of the Object
Buffer 15 increases at 128Mbps.
The second level shows changes in occupancy of the Object
Buffer 15. As illustrated, the changes in occupancy of the
Object Buffer 15 include monotone increases Vh1 and Vh2 with
which the graphics Objects of 0DS1 and 0DS2 output from the
Stream Graphics Processor 14 are stored. Slopes of monotone
increases Vhl and Vh2 are based on transfer rate Rd from the
Stream Graphics Processor 14 to the Object Buffer 15 . A decode
period of each of 0DS1 and 0DS2 corresponds to a time period
in which a monotone decrease occurs on the third level and
a monotone increase occurs on the second level. The beginning
of the decode period is shown by a DTS of the ODS, whereas
the end of the decode period is shown by a PTS of the ODS.
Once the uncompressed graphics Object has been transferred
to the Object Buffer 15 by the time shown by the PTS of the
ODS, the decoding of the ODS is complete. It is essential
for the uncompressed graphics Object to be stored in the Object
Buffer 15 by the time shown by the PTS of the ODS. As long

as this is satisfied, the monotone decrease and the monotone
increase in the decode period are not limited to those shown
in FIG. 32.
The first level shows changes in occupancy of the
Graphics Plane 8. As illustrated, the changes in occupancy
of the Graphics Plane 8 include monotone increase Vf3 with
which the graphics Objects output from the Object Buffer 15
are stored. A slope of monotone increase Vf3 is based on
transfer rate Rc from the Object Buffer 15 to the Graphics
Plane 8. The end of monotone increase Vf3 is shown by the
PTS of the PCS.
The graph such as the one shown in FIG. 32 can be created
through the use of the DTSs and PTSs of the ODSs, the DTS
and the PTS of the PCS, and the buffer sizes and transfer
rates shown in FIG. 29. Such a graph allows the author to
grasp how the buffer states change when the AV Clip on the
BD-ROM 100 is reproduced.
These changes of the buffer states can be adjusted by
rewriting DTSs and PTSs. Therefore, it is possible for the
author to prevent the occurrence of such a decoding load that
exceeds specifications of a decoder of the reproduction
apparatus 200, or to prevent a buffer overflow during
reproduction . This makes it easier to implement hardware and
software when developing the reproduction apparatus 200.
This completes the explanation on the internal construction
of the reproduction apparatus 200.
The following explains how to implement the Graphics
Decoder 12 . The Graphics Decoder 12 can be realized by having

a general-purpose CPU execute a program for performing an
operation shown in FIG. 33. An operation of the Graphics
Decoder 12 is explained below, with reference to FIG. 33.
FIG. 33 is a flowchart showing an operation of loading
a functional Segment. In the drawing, SegmentK is a variable
indicating a Segment (PCS, WDS, PDS, or ODS) which belongs
to a DS and is read during reproduction of the AV Clip, and
an ignore flag indicates whether SegmentK is to be ignored
or loaded. In this flowchart, after the ignore flag is reset
to 0 (S20), a loop of steps S21 to S24 and S27 to S31 is performed
for each SegmentK (S25 and S26).
Step S21 judges whether SegmentK is a PCS. If SegmentK
is a PCS, the operation proceeds to step S27.
Step S22 judges whether the ignore flag is 0 or 1. If
the ignore flag is 0, the operation proceeds to step S23 .
If the ignore flag is 1, the operation proceeds to step S24 .
In step S23, SegmentK is loaded to the Coded Data Buffer 13.
If the ignore flag is 1 (S22:NO), SegmentK is ignored
(S24 ) . This leads to the negative judgement on all functional
Segments belonging to the DS in step S22, as a result of which
the functional Segments of the DS are all ignored.
Thus, the ignore flag indicates whether SegmentK is to
be ignored or loaded. Steps S27 to S31 and S34 to S35 are
performed to set this ignore flag.
Step S27 judges whether a composition_state field of
the PCS shows Acquisition Point. If the composition_state
field shows Acquisition Point, the operation proceeds to step
S28. If the composition_state field shows Epoch Start or

Normal Case, the operation proceeds to step S31.
Step S28 judges whether an immediately preceding DS
exists in any of the buffers (the Coded Data Buffer 13, the
Stream Graphics Processor 14, the Object Buffer 15, and the
Composition Buffer 16) in the Graphics Decoder 12. The
immediately preceding DS does not exist in the Graphics Decoder
12 if a skip operation is performed. In this case, the display
needs to be started from the Acquisition Point DS, so that
the operation proceeds to step S30 (S28:NO).
In step S30, the ignore flag is set to 0 , and the operation
proceeds to step S22 .
On the other hand, the immediately preceding DS exists
in the Graphics Decoder 12 if normal reproduction is performed .
In this case, the operation proceeds to step S29 (S28:YES).
In step S29, the ignore flag is set to 1, and the operation
proceeds to step S22.
Step S31 judges whether the composition_state field
shows Normal Case. If the composition_state field shows
Normal Case, the operation proceeds to step S34. If the
composition_state field shows Epoch Start, the operation
proceeds to step S30 where the ignore flag is set to 0. Step
S34 is the same as step S28 , and judges whether the immediately
preceding DS exists in the Graphics Decoder 12. If the
immediately preceding DS exists, the ignore flag is set to
0 (S30) . Otherwise, the ignore flag is set to 1, because
enough functional Segments for composing one screen of
graphics cannot be obtained (S35). In this way, when the
immediately preceding DS does not exist in the Graphics Decoder

12, the functional Segments of the Normal Case DS are ignored.
The following gives a specific example of loading DSs,
with reference to FIG. 34. In FIG. 34, three DSs (DS1, DS10,
and DS20) are multiplexed with video. A composition_state
field of DS1 shows Epoch Start, a composition_state field
of DS10 shows Acquisition Point, and a composition_state field
of DS20 shows Normal Case.
Suppose a skip operation is performed on picture data
pt10 in an AV Clip in which these three DSs are multiplexed
with video, as indicated by arrow am1. In such a case, DS10
which is closest to ptlO is subjected to the operation shown
in FIG. 33. The composition_state field of DS10 shows
Acquisition Point (S27:YES), but the immediately preceding
DS (DS1) does not exist in the Coded Data Buffer 13 (S28:NO).
Accordingly, the ignore flag is set to 0 (S30) . As a result,
DS10 is loaded to the Coded Data Buffer 13 as indicated by
arrow md1 in FIG. 35. Suppose, on the other hand, a skip
operation is performed on picture data which is located after
DS10, as indicated by arrow am2 in FIG. 34. In this case,
DS20 is a Normal Case DS, and the immediately preceding DS
(DS10 ) does not exist in the Coded Data Buffer 13 . Accordingly,
DS20 is ignored, as indicated by arrow md2 in FIG. 35.
FIG. 37 shows how DS1, DS10, and DS20 are loaded when
normal reproduction is performed as shown in FIG. 36 . Of the
three DSs, DS1 which is an Epoch Start DS is loaded to the
Coded Data Buffer 13, as indicated by arrow rd1 (S23) . However,
the ignore flag is set to 1 for DS10 which is an Acquisition
Point DS (S29), so that the functional Segments of DS10 are

not loaded to the Coded Data Buffer 13 but ignored, as indicated
by arrow rd2 (S24). Meanwhile, DS20 which is a Normal Case
DS is loaded to the Coded Data Buffer 13, as indicated by
arrow rd3 (S23).
The following explains an operation of the Graphics
Controller 17. FIGS. 38 to 40 are flowcharts showing the
operation of the Graphics Controller 17.
Steps S41 to S44 constitute a main routine, where an
event specified by any of steps S41 to S44 is waited.
In FIG. 38, step S41 judges whether the current
reproduction time is a DTS of a PCS. If so, steps S45 to S53
are performed.
Step S45 judges whether a composition_state field of
the PCS shows Epoch Start. If so, the entire Graphics Plane
8 is cleared in step S46. Otherwise, a Window specified by
a window_horizontal_position field, a
window_vertical_position field, a window_width field, and
a window_height field of a WDS is cleared in step S47.
Step S48 is performed after step S46 or S47, and judges
whether a PTS of arbitrary ODSx has passed. Clearing the
entire Graphics Plane 8 takes a long time, so that decoding
of ODSx may already be completed by the time the entire Graphics
Plane 8 is cleared. Step S48 examines this possibility. If
the PTS of ODSx has not passed, the operation returns to the
main routine. If the PTS of ODSx has passed, steps S49 to
S51 are performed. Step S49 judges whether an
object_cropped_flag field shows 0. If so, a graphics Object
corresponding to ODSx is set to non-display (S50).

If the object_cropped_f lag shows 1, the graphics Object
cropped based on an object_cropping_horizontal_position
field, an object_cropping_vertical_position field, a
cropping_width field, and a cropping_height field is written
to the Window on the Graphics Plane 8 at a position specified
by an object_horizontal_position field and an
object_vertical_position field (S51). In this way, the
graphics Object is v/ritten to the Window.
Step S52 judges whether a PTS of another ODS (ODSy) has
passed. If decoding of ODSy is completed during when the
graphics Object of ODSx is being written to the Graphics Plane
8, ODSy is set as ODSx (S53), and the operation returns to
stepS49. As a result, steps S49 to S51 are performed on ODSy.
In FIG. 39, step S4 2 judges whether the current
reproduction time is a PTS of the WDS. If so, the operation
proceeds to step S54, to judge whether the number of Windows
is 1. If the number of Windows is 2, the operation returns
to the main routine. If the number of Windows is 1, a loop
of steps S55 to S59 is performed. In this loop, steps S57
to S59 are performed for each of at most two graphics Objects
to be displayed in the Window. Step S57 judges whether the
object_cropped_f lag field shows 0 . If so, the graphics Object
is set to non-display (S58).
If the object_cropped_flag field shows 1, the graphics
Object cropped based on an
object_cropping_horizontal_position field, an
object_cropping_vertical_position field, a cropping_width
field, and a cropping_height field is written to the Window

on the Graphics Plane 8 at a position specified by an
object_horizontal_position field and an
object_vertical_position field (S59). As a result of this
loop, one or more graphics Objects are written to the Window.
Step S4 4 judges whether the current reproduction time
is a PTS of the PCS. If so, the operation proceeds to step
S60 to judge whether a palette_update__f lag field shows 1.
If so, a Palette identified by a palette_id field is set to
the CLUT unit 9 (S61). If the palette_update_f lag field shows
0, step S61 is skipped.
After this, the CLUT unit 9 performs color conversion
of graphics on the Graphics Plane 8. The graphics is then
overlaid on video (S62).
In FIG. 40, step S43 judges whether the current
reproduction time is a PTS of an ODS. If so, the operation
proceeds to step S63 to judge whether the number of Windows
is 2 . If the number of Windows is 1, the operation returns
to the main routine.
Here, the judgements made in steps S4 3 and S6 3 have the
following meaning . I f the number of Windows is 2, two graphics
Objects are displayed respectively in the two Windows. In
such a case, each time decoding of one ODS is completed, a
graphics Object obtained by the decoding needs to be written
to the Graphics Plane 8 (see FIG. 19B) . Therefore, if the
current reproduction time is the PTS of the ODS and the number
of Windows is 2, steps S64 to S66 are performed to write each
individual graphics Object to the Graphics Plane 8 . Step S64
judges whether the object_cropped_f lag field shows 0 . If so,

the graphics Object is set to non-display (S65).
If the object_cropped_flag field shows 1, the graphics
Object cropped based on an
object_cropping_horizontal_position field, an
object_cropping_vertical_position field, a cropping_width
field, and a cropping_height field is written to a Window
on the Graphics Plane 8 at a position specified by an
object_horizontal_position field and an
object_vertical_position field (S66). By repeating this
process, two graphics Objects are written respectively to
the two Windows.
(Second Embodiment)
The second embodiment of the present invention relates
to a manufacturing process of the BD-ROM 100 explained in
the first embodiment. FIG. 41 is a flowchart showing the
manufacturing process of the BD-ROM 100.
The manufacturing process includes a material
production step of recording video, sound, and the like ( S201) ,
an authoring step of creating an application format using
an authoring device (S202), and a pressing step of creating
an original master of the BD-ROM 100 and performing stamping
and bonding to complete the BD-ROM 100 (S203).
In this manufacturing process, the authoring step
includes steps S204 to S213.
In step S204, control information, Window definition
information, Palette definition information, and graphics
are generated. In step S205, the control information, the

Window definition information, the Palette definition
information, and the graphics are converted to functional
Segments. In step S206, a PTS of each PCS is set based on
a time of a picture to be synchronized with. In step S207,
a DTS[ODS] and a PTS[ODS] are set based on the PTS [PCS] . In
step S208, aDTS[PCS], a PTS[PDS], a DTS[WDS], and a PTS[WDS]
are set based on the DTS[ODS]. In step S209, changes in
occupancy of each buffer in the player model are graphed.
In step S210, a judgement is made as to whether the graphed
changes satisfy constraints of the player model. If the
judgement is in the negative, the DTS and PTS of each functional
Segment are rewritten in step S211. If the judgement is in
the affirmative, a graphics stream is generated in step S212,
and the graphics stream is multiplexed with a video stream
and an audio stream to form an AV Clip in step S213 . After
this, the AV Clip is adapted to the Blue-ray Disc Read-Only
Format, to complete the application format.
(Modifications)
Though the present invention has been described by way
of the above embodiments, the present invention is not limited
to such. The present invention can be realized with any of
modifications (A) to (P) below. The invention of each of the
claims of this application includes extension and
generalization of the above embodiments and their
modifications below. The degree of extension and
generalization depends upon the state of the art in the
technical field of the present invention at the time when

the present invention was made.
(A) The above embodiments describe the case where the
BD-ROM is used as the recording medium. Main features of the
present invention, however, lie in a graphics stream recorded
on the recording medium, which does not rely on physical
characteristics of BD-ROMs. Therefore, the present
invention is applicable to any recording medium that is capable
of recording a graphics stream. Examples of such a recording
medium include: an optical disc such as a DVD-ROM, a DVD-RAM,
a DVD-RW, a DVD-R, a DVD+RW, a DVD+R, a CD-R, or a CD-RW;
a magneto-optical disk such as a PD or an MO; a semiconductor
memory card such as a CompactFlash card, a SmartMedia card,
a Memory Stick card, a MultiMediaCard, or a PCMCIA card; a
magnetic disk such as a flexible disk, SuperDisk, Zip, or
Clik!; a removable hard disk drive such as ORB, Jaz, SparQ,
SyJet, EZFley, or Microdrive, and a nonremovable hard disk
drive.
(B) The above embodiments describe the case where the
reproduction apparatus decodes an AV Clip on the BD-ROM and
outputs the decoded AV Clip to the television. As an
alternative, the reproduction apparatus may be equipped with
only a BD drive, with the remaining construction elements
being provided in the television. In this case, the
reproduction apparatus and the television can be incorporated
in a home network connected with an IEEE 1394 connector. The
above embodiments describe the case where the reproduction
apparatus is connected to the television, but the reproduction
apparatus may instead be integrated with a display device.

Also, the reproduction apparatus may include only the system
LSI (integrated circuit) which constitutes an essential part
of processing . The reproduction apparatus and the integrated
circuit are both an invention described in this specification .
Accordingly, regardless of whether the reproduction apparatus
or the integrated circuit is concerned, an act of manufacturing
a reproduction apparatus based on the internal construction
of the reproduction apparatus described in the first
embodiment is an act of working of the present invention.
Also, any act of assigning with charge (i.e. for sale) or
without charge (i.e. as a gift), leasing, and importing the
reproduction apparatus is an act of working of the present
invention. Likewise, an act of offering for assignment or
lease of the reproduction apparatus using storefront displays,
catalogs, or brochures is an act of working of the present
invention.
(C) Information processing using the programs shown in
the flowcharts is actually realized using hardware resources .
Accordingly, the programs which describe the operational
procedures shown in the flowcharts are themselves an invention.
The above embodiments describe the case where the programs
are incorporated in the reproduction apparatus, but the
programs can be used independently of the reproduction
apparatus. Acts of working of the programs include (1) an
act of manufacturing, (2) an act of assigning with or without
charge, (3) an act of leasing, (4) an act of importing, (5)
an act of providing to the public via a bi-directional
electronic communications network, and ( 6 ) an act of offering

for assignment or lease using storefront displays, catalogs,
or brochures.
(D) The time elements of the steps which are executed
in a time series in each of the flowcharts can be regarded
as the necessary elements of the present invention . This being
so, a reproduction method shown by these flowcharts is an
invention. If the processing shown in each flowchart is
carried out by performing the steps in a time series so as
to achieve the intended aim and the intended effect, this
is an act of working of the recording method of the present
invention.
( E ) When recording an AV Clip on the BD-ROM, an extension
header may be added to each TS packet in the AV Clip. The
extension header is called a TP_extra_header, includes an
arrival_time_stamp and a copy_permission_indicator, and has
a data length of 4 bytes. TS packets with TP_extra_headers
(hereafter "EX TS packets" ) are grouped in units of 32 packets,
and each group is written to three sectors. One group made
up of 32 EX TS packets has 6,144- bytes (=32x192), which is
equivalent to a size of three sectors that is 6144 bytes
( =2048x3 ) . The 32 EXTS packets contained in the three sectors
are called an Aligned Unit.
In a home network connected with an IEEE 1394 connector,
the reproduction apparatus transmits an Aligned Unit in the
following manner. The reproduction apparatus removes a
TP_extra_header from each of the 32 EX TS packets in the Aligned
Unit, encrypts the body of each TS packet according to the
DTCP Specification, and outputs the encrypted TS packets.

When outputting the TS packets, the reproduction apparatus
inserts an isochronous packet between adjacent TS packets.
A position where the isochronous packet is inserted is based
on a time shown by an arrival_time_stamp of the TP_extra_header .
The reproduction apparatus outputs a DTCP_descriptor, as well
as the TS packets. The DTCP_descriptor corresponds to a
copy_permission_indicator in the TP_extra_header. With the
provision of theDTCP_descriptor indicating "copy prohibited" ,
it is possible to prevent, when-using the TS packets in the
home network connected with the IEEE 13 94 connector, the TS
packets from being recorded to other devices.
(F) The above embodiments describe the case where an
AV Clip of the Blu-ray Disc Read-Only Format is used as a
digital stream, but the present invention can also be realized
with a VOB (Video Object) of the DVD-Video Format or the
DVD-Video Recording Format. The VOB is a program stream that
complies with the ISO/IEC 13818-1 Standard and is obtained
by multiplexing a video stream and an audio stream. Also,
the video stream in the AV Clip may be an MPEG4 video stream
or a WMV video stream. Further, the audio stream in the AV
Clip may be a Linear PCM audio stream, a Dolby AC-3 audio
stream, an MP3 audio stream, an MPEG-AAC audio stream, or
a dts audio stream.
(G) The film described in the above embodiments may be
obtained by encoding an analog image signal broadcast by analog
broadcasting. Also, the film may be stream data made up of
a transport stream broadcast by digital broadcasting.
Alternatively, an analog/digital image signal recorded

on a videotape may be encoded to obtain content. Also, an
analog/digital image signal directly captured by a video
camera may be encoded to obtain content. A digital work
distributed by a distribution server is applicable too.
(H) Graphics Objects described in the above embodiments
is run-length encoded raster data. Run-length encoding is
used for compression/encoding of graphics Objects, because
the run-length encoding is suitable for compression and
decompression of subtitles . Subtitles have a property in that
a continuous length of the same pixel value in a horizontal
direction is relatively long. Therefore, by performing
compression using run-length encoding, a high compression
rate can be attained. In addition, run-length encoding
reduces a load for decompression, and is therefore suitable
for realizing decoding by software. Nevertheless, the use
of run-length encoding for graphics Objects is not a limitation
of the present invention. For example, graphics Objects may
be PNG data. Also, graphics Objects may be vector data instead
of raster data . Further, graphics Objects may be transparent
patterns.
(I) Graphics of subtitles selected according to a
language setting in the reproduction apparatus may be
subjected to display effects of PCSs. As a result, display
effects achieved by using characters which are contained
within the body of video in a conventional DVD can be realized
with subtitle graphics displayed according- to the language
setting of the reproduction apparatus. This contributesto
high practicality.

Also, subtitle graphics selected according to a display-
setting of the reproduction apparatus may be subjected to
display effects of PCSs. For example, graphics of various
display modes such as wide screen, pan and scan, and letterbox
are recorded on the BD-ROM, and the reproduction apparatus
selects one of these display modes according to a display
setting of the television connected with the reproduction
apparatus and displays corresponding graphics. Since the
display effects of PCSs are applied to such graphics,
viewability increases . As a result, display effects achieved
by using characters which are contained within the body of
video in a conventional DVD can be realized with subtitle
graphics displayed according to the display setting. This
contributes to high practicality.
(J) The first embodiment describes the case where
transfer rate Rc from the Object Buffer to the Graphics Plane
is set so as to clear the Graphics Plane and render graphics
on a Window, which is 25% in size of the Graphics Plane, within
one video frame. However, transfer rate Rc may be set so that
the clearing and the rendering complete within a vertical
blanking time. Suppose the vertical blanking time is 25% of
1/29.97 seconds. Then Rc is 1Gbps. By setting Rc in this
way, graphics can be displayed smoothly.
Also, writing in sync with line scan can be used together
with writing within a vertical blanking time. This enables
subtitles to be displayed smoothly with Rc=256Mbps.
(K) The above embodiments describe the case where the
reproduction apparatus includes the Graphics Plane.

Alternatively, the reproduction apparatus may include a line
buffer for storing uncompressed pixels of one line. Since
conversion to an image signal is performed for each horizontal
row (line), conversion to an image signal can equally be
performed with the line buffer.
(L) The above embodiments describe the case where
graphics is character strings representing dialogs in a film,
but the present invention is equally applicable to any graphics
that is displayed in precise synchronization with video.
Examples of such graphics include an illustration, a pattern,
a cartoon character, and a symbol mark. Examples of such
graphics also include a combination of figures, letters, and
colors constituting a trademark, a national crest, a national
flag, a national emblem, a public symbol or seal employed
by a national government for supervision/certification, a
crest, flag, or emblem of an international organization, or
a mark of origin of a particular item.
(M) The first embodiment describes the case where a
Window is provided on the top or bottom of the Graphics Plane,
on an assumption that subtitles are displayed horizontally
on the top or bottom of the screen. Instead, a Window may
be provided on the left or right of the Graphics Plane, to
display subtitles vertically on the left or right of the screen .
This enables Japanese subtitles to be displayed vertically.
(0) The Graphics Decoder performs pipeline processing
on DSn and DSn+1, when DSn and DSn + 1 belong to the same Epoch
inthe-graphics stream . When DSn and DSn + 1 belong to different
Epochs, on "the other hand, the Graphics Decoder starts

processing DSn+1 after display of graphics of DSn begins.
Also, there, are two types of graphics streams/ i.e.,
a Presentation graphics stream which is mainly intended to
synchronize with video, and an Interactive graphics stream
which is mainly intended to realize an interactive display.
The Graphics Decoder performs pipeline processing of DSn and
DSn+1 when the graphics stream is a Presentation graphics
stream, and does not perform pipeline processing when the
graphics stream is an Interactive graphics stream.
(P) In the example shown in FIG. 20, graphics data
referenced by a PCS in a DS is set before graphics data not
referenced by the PCS in the DS . However, a plurality of sets
of referenced graphics data and a plurality of sets of
non-referenced graphics data may instead be arranged in a
DS in an order specified by their object_ids . Alternatively,
the plurality of sets of referenced graphics data may be
arranged in a scan line order. The scan line order referred
to here is an order in which scanning is performed . In general,
scanning is performed from the top left to bottom right of
a screen. This being so, graphics data whose display
coordinates lie at the top left of the screen is set before
graphics data whose display coordinates lie at the bottom
right of the screen. This accelerates presentation of
graphics data.
The present invention can be modified as explained above .
Nevertheless, the invention of each of the claims of this
application reflects means for solving, the-technical problem
encountered by the conventional techniques, so that the

technical scope of the invention according to the claims will
not extend beyond the technical scope in which one skilled
in the art acknowledges the technical problem. Hence the
invention according to the claims substantially corresponds
to the description of the specification.
Industrial Applicability
The above embodiments disclose the internal
constructions of the recording medium and the reproduction
apparatus to which the present invention relates, and the
recording medium and the reproduction apparatus can be
manufactured in volume based on the disclosed internal
constructions . In other words, the recording medium and the
reproduction apparatus are capable of being industrially
manufactured. Hence the recording medium and the
reproduction apparatus have industrial applicability.

We Claim:
1. A reproduction apparatus for reproducing a digital stream generated by
multiplexing a video stream and a graphics stream, characterized by
comprising:
a video decoder (5) operable to decode the video stream to generate a
moving picture;
a graphics decoder (12); and
a plane memory (8), wherein:
the graphics stream is a sequence of packets, the packets constituting the
graphics stream include two types, one of which is packets containing
graphics data (ODS) and the other is packets containing control data
(PCS), the sequence of packets includes a pair of a packet containing
graphics data (ODS) and a packet containing control data (PCS), the pair
is followed by a packet containing control data (PCS) that performs a
forward reference,
the control data (PCS) that performs the forward reference indicates that
graphics, which corresponds to the graphics data (ODS) contained in the
pair of packets which precedes the control data (PCS) that performs the
forward reference, is to be displayed,

the graphics decoder (12) comprises:
an object buffer (15);
a processor (14) operable to generate graphics on the object buffer (15)
by decoding the graphics data (ODS) contained in the pair of packets; and
a controller (17) operable to execute a graphics display by writing the
graphics generated on the object buffer (15) into a partial area in the
plane memory (8), based on the control data (PCS) contained in the pair
of packets, wherein
the plane memory (8) is a memory for overlay with moving pictures and is
provided in the reproduction apparatus, and
when control data (PCS) that performs a forward reference has been read
into the graphics decoder (12) from a recording medium, the controller
(17) updates the graphics display by writing the graphics generated on
the object buffer into the partial area again based on the read control
data (PCS) that performs the forward reference.
2. A method of recording onto a recording medium, characterized by
comprising the steps of:
generating application data (S202); and
recording the application data to the recording medium (S203), wherein:

the application data includes a digital stream generated by multiplexing a
video stream and a graphics stream;
the graphics stream is a sequence of packets, the packets constituting the
graphics stream include two types, one of which is packets containing
graphics data (ODS) and the other is packets containing control data (PCS),
the sequence of packets includes a pair of a packet containing graphics data
(ODS) and a packet containing control data (PCS), the pair is followed by a
packet containing control data (PCS) that performs a forward reference; and
the control data (PCS) that performs the forward reference indicates that
graphics, which corresponds to the graphics data (ODS) contained in the pair
of packets which precedes the control data (PCS) that performs the forward
reference, is to be displayed.
3. A method of reproducing a digital stream generated by multiplexing a video
stream and a graphics stream, characterized by comprising the steps of:
decoding the video stream to generate a moving picture; and
decoding the graphics stream to generate graphics; wherein

the graphics stream is a sequence of packets, the packets
constituting the graphics stream include two types, one of which is
packets containing graphics data (ODS) and the other is packets
containing control data (PCS), the sequence of packets includes a pair of a
packet containing graphics data (ODS) and a packet containing control
data (PCS), the pair is followed by a packet containing control data (PCS)
that performs a forward reference, the control data that performs the
forward reference indicates that graphics, which corresponds to the
graphics data (ODS) contained in the pair of packets which precedes the
control data (PCS) that performs the forward reference, is to be displayed,
the graphics decoding step includes the sub-steps of:
generating graphics on an object buffer (15) provided in the
computer by decoding the graphics data (ODS) contained in the pair of
packets; and
executing (S59) a graphics display by writing the graphics
generated on the object buffer (15) into a partial area in a plane memory
(8) provided in the computer, based on the control data (PCS) contained
in the pair of packets, wherein

the plane memory (8) is a memory for overlay with moving
pictures, and
when control data (PCS) that performs a forward reference has
been read into the graphics decoder (12) from a recording medium, the
graphics display executing sub-step updates the graphics display by
writing the graphics generated on the object buffer (15) into the partial
area again based on the read control data (PCS) that performs the
forward reference.
4. A reproduction apparatus for reproducing a digital stream generated by
multiplexing a video stream and a graphics stream, characterized by
comprising:
a video decoder (5) operable to decode the video stream to generate a
moving picture, and store a picture constituting the moving picture into a
video plane;
a graphics decoder (12) operable to decode the graphics stream to
generate graphics, and store the generated graphics into a graphics plane;
and
an adder (10) operable to overlay the graphics and the moving picture by
performing addition for corresponding pixels in the picture stored in the video
plane and the graphics stored in the graphics plane, wherein

the graphics stream is a sequence of packets, the packets constituting the
graphics stream include two types, one of which is packets containing
graphics data (ODS) and the other is packets containing control data (PCS),
the sequence of packets includes a pair of a packet containing graphics data
(ODS) and a packet containing control data (PCS), the pair is followed by a
packet containing control data (PCS) that performs a forward reference,
the control data (PCS) that performs the forward reference indicates that
graphics, which corresponds to the graphics data (ODS) contained in the pair
of packets which precedes the control data (PCS) that performs the forward
reference, is to be displayed,
the graphics decoder (12) includes;
an object buffer (12);
a processor (14) operable to generate graphics on the object buffer (12)
by decoding the graphics data (ODS) contained in the pair of packets; and
a controller (17) operable to execute a graphics display by writing the
graphics generated on the object buffer (12) into a partial area in the plane
memory (8), based on the control data (PCS) contained in the pair of
packets, wherein

the plane memory (8) is a memory for overlay with moving pictures and is
provided in the reproduction apparatus, and
when control data (PCS) that performs a forward reference has been read
into the graphics decoder (12) from a recording medium, the controller (17)
updates the graphics display by writing the graphics generated on the object
buffer (12) into the partial area again based on the read control data (PCS)
that performs the forward reference.



ABSTRACT


TITLE: AN APPARATUS AND METHOD FOR REPRODUCING AND RECORDING
A DIGITAL STREAM
The invention relates to a reproduction apparatus for reproducing a digital
stream generated by multiplexing a video stream and a graphics stream,
comprising a video decoder(5) operable to decode the video stream to
generate a moving picture; a graphics decoder(12) ; and a plane memory (8),
wherein the graphics stream is a sequence of packets, the packets constituting
the graphics stream include two types, one of which is packets containing
graphics data (ODS) and the other is packets containing control data (PCS), the
sequence of packets includes a pair of a packet containing graphics data (ODS)
and a packet containing control data (PCS), the pair is followed by a packet
containing control data (PCS) that performs a forward reference,the control
data (PCS) that performs the forward reference indicates that graphics, which
corresponds to the graphics data (ODS) contained in the pair of packets which
precedes the control data (PCS) that performs the forward reference, is to be
displayed, the graphics decoder (12) comprises: an object buffer (15) ;a
processor (14) operable to generate graphics on the object buffer (15) by
decoding the graphics data (ODS) contained in the pair of packets; and a
controller (17) operable to execute a graphics display by writing the graphics
generated on the object buffer (15) into a partial area in the plane memory (8),
based on the control data (PCS) contained in the pair of packets, wherein the
plane memory (8) is a memory for overlay with moving pictures and is
provided in the reproduction apparatus, and when control data (PCS) that
performs a forward reference has been read into the graphics decoder (12)
from a recording medium, the controller (17) updates the graphics display by
writing the graphics generated on the object buffer into the partial area again
based on the read control data (PCS) that performs the forward reference.

Documents:

02460-kolnp-2005-claims.pdf

02460-kolnp-2005-description complete.pdf

02460-kolnp-2005-drawings.pdf

02460-kolnp-2005-form 1.pdf

02460-kolnp-2005-form 2.pdf

02460-kolnp-2005-form 3.pdf

02460-kolnp-2005-form 5.pdf

02460-kolnp-2005-international publication.pdf

2460-KOLNP-2005-(07-02-2013)-AMANDED PAGES OF SPECIFICATION.pdf

2460-KOLNP-2005-(07-02-2013)-CORRESPONDENCE.pdf

2460-KOLNP-2005-ABSTRACT 1.1.pdf

2460-KOLNP-2005-ABSTRACT.pdf

2460-KOLNP-2005-CANCELLED PAGES.pdf

2460-KOLNP-2005-CANCELLED PAGES1.1.pdf

2460-KOLNP-2005-CLAIMS 1.1.pdf

2460-KOLNP-2005-CLAIMS.pdf

2460-KOLNP-2005-CORRESPONDENCE.pdf

2460-KOLNP-2005-DESCRIPTION (COMPLETE) 1.1.pdf

2460-KOLNP-2005-DRAWINGS 1.1.pdf

2460-KOLNP-2005-EXAMINATION REPORT.pdf

2460-KOLNP-2005-FORM 1.1.1.pdf

2460-KOLNP-2005-FORM 13.pdf

2460-KOLNP-2005-FORM 18.pdf

2460-KOLNP-2005-FORM 2.1.1.pdf

2460-KOLNP-2005-FORM 26.pdf

2460-KOLNP-2005-FORM 3 1.2.pdf

2460-KOLNP-2005-GPA.pdf

2460-KOLNP-2005-GRANTED-ABSTRACT.pdf

2460-KOLNP-2005-GRANTED-CLAIMS.pdf

2460-KOLNP-2005-GRANTED-DESCRIPTION (COMPLETE).pdf

2460-KOLNP-2005-GRANTED-DRAWINGS.pdf

2460-KOLNP-2005-GRANTED-FORM 1.pdf

2460-KOLNP-2005-GRANTED-FORM 2.pdf

2460-KOLNP-2005-GRANTED-FORM 3.pdf

2460-KOLNP-2005-GRANTED-FORM 5.pdf

2460-KOLNP-2005-GRANTED-SPECIFICATION-COMPLETE.pdf

2460-KOLNP-2005-INTERNATIONAL PUBLICATION.pdf

2460-KOLNP-2005-OTHER PATENT DOCUMENT.pdf

2460-KOLNP-2005-OTHERS.pdf

2460-KOLNP-2005-PETITION UNDER RULE 137.pdf

2460-KOLNP-2005-PRIORITY DOCUMENT.pdf

2460-KOLNP-2005-REPLY TO EXAMINATION REPORT.pdf

2460-KOLNP-2005-TRANSLATED COPY OF PRIORITY DOCUMENT.pdf


Patent Number 255756
Indian Patent Application Number 2460/KOLNP/2005
PG Journal Number 12/2013
Publication Date 22-Mar-2013
Grant Date 20-Mar-2013
Date of Filing 02-Dec-2005
Name of Patentee PANASONIC CORPORATION
Applicant Address 1006, OAZA KADOMA KADOMA-SHI, OSAKA 571 8501 JAPAN
Inventors:
# Inventor's Name Inventor's Address
1 JOSEPH MCCROSSAN 1724, WEXFPRD CIRCLE, SIMI VALLEY, CA, 93065 USA
2 HIROSHI YAHATA 24-10-303, SHYOUJI-CHO, KADOMA-SHI, OSAKA 571-0058 JAPAN
3 TOMOYUKI OKADA 1-8-19-303, TOMIO-MOTOMACHI, NARA-SHI, NARA 631-0078 JAPAN
4 WATARU IKEDA 1-15-18-1202, KYOMACHIBORI, NISHI-KU, OSAKA-SHI, OSAKA 550-0003 JAPAN
PCT International Classification Number H04N 5/91
PCT International Application Number PCT/JP2004/009517
PCT International Filing date 2004-06-29
PCT Conventions:
# PCT Application Number Date of Convention Priority Country
1 60/483,228 2003-06-30 U.S.A.