Analysis of MPEG-2 Video Streams

Analysis of MPEG-2Video Streams

Damir Isovi′c and Gerhard Fohler

Department of Computer Engineering

M¨a lardalen University,Sweden

damir.isovic,gerhard.fohler@mdh.se

Abstract

MPEG-2is widely used as video coding standard for contents such as DVD or dig-ital video broadcasting,DVB.It de?nes a layered structure,composing three different

types of frames into groups for temporal and spatial compression of video information.

In this paper we present an exhaustive analysis av various MPEG streams,taken from original DVDs.The purpose is to get a more clear picture about what are valid

assumptions about MPEG.The analysis showed that many common assumptions,in

particular about relation of frame sizes,and equal importance of frames,do not hold in

the general case.

1Introduction

MPEG,the Moving Picture Experts Group standard for coded representation of digital audio and video,is used in a wide range of applications.In particular MPEG-2has become the coding standard for digital video streams in consumer content and devices,such as DVD movies and digital television set top boxes for DVB,terrestrial TV broadcasts or via satellite.It should be noted that MPEG is a standard for the format,a syntax,not for the actual encoding:the same content,e.g.,a movie,can be encoded in many ways while adhering to the same standard.In fact,MPEG encoding has to meet diverse demands, depending,e.g.,on the medium of distribution,such as overall size in the case of DVD, maximum bitrate for DVB,or speed of encoding for live broadcasts.

In the case of DVD and DVB,sophisticated provisions to apply spatial and temporal compression are applied,while a very simple,but quickly coded stream will be used for the live broadcast.Consequently,video streams,and in particular their decoding demands will vary greatly between media,but also different types of contents or even different scenes within the same movie.

MPEG-2video streams have a layered structure.The layer we are considering here is the picture layer,where the video data is organized in Group of Pictures(GOP),i.e.,a sequence of pictures that consist of a number of frames.The three types of frames are frames(intra-coded pictures),frames(predicted pictures),and frames bi-directionally predicted pictures.Simply speaking,frames contain full pictures and are independent, frames build a full picture using a previous or frame as reference,and frames contain incremental changes to a full picture,based on both previous and later frames.

In this paper we present results of an analysis of realistic MPEG streams of DVD movies and match the analysis results against common assumptions.For example,an in-tuitive conclusion is that will be the largest frames,followed by and frames,and frames have similar sizes within their respective frame type.While true on average,such assumptions do not hold for a considerable number of cases.The analysis of realistic streams presented in this paper shows,e.g.,a case with9%GOPs in which have the largest size,and1%of frames,which corresponds to roughly to8and1minutes,resp., in a90minute feature?lm.Clearly,such deviations from average cannot be ignored.

1

The analysis showed that many common assumptions,in particular about relation of frame sizes,and equal importance of frames,do not hold in the general case.

2MPEG Video Streams Properties

A complete description of the MPEG compression scheme is beyond the scope of this paper.For details on MPEG see e.g.[1,4,3].Here we will focus on the MPEG video stream structure,and see how it can be analyzed and scheduled.In this work,we describe the most important characteristics of a MPEG-2video stream.The text presented in this subsection is sumarized in?gure1.

2.1Frame types

The MPEG-2standard de?nes three types of frames,,and.

frames or intra frames are simply frames coded as still images.They contain absolute picture data and are self-contained,meaning that they require no additional information for decoding.frames have only spatial redundancy providing the least compression among all frame types.Therefore they are not transmitted more frequently than necessary.

frames The second kind of frames are or predicted frames.They are forward pre-dicted from the most recently reconstructed or frame,i.e.,they contain a set of instruc-tions to convert the previous picture into the current one.frames are not self-contained, meaning that if the previous reference frame is lost,decoding is impossible.On average, frames require roughly half the data of an frame,but our analysis also showed that this is not the case for the signi?cant number of cases.

frames The third type is or bi-directionally predicted frames.They use both for-ward and backward prediction,i.e.,a frame can be decoded from a previous or frame,and/or from a later or frame.They contain vectors describing where in an earlier or later pictures data should be taken from.They also contain transformation coef?-cients that provide the correction.frames are never predicted from each other,only from or frames.As a consequence,no other frames depend on frames.frames require resource-intensive compression techniques such as Motion Compensation and Motion Es-timation but they also exhibit the highest compression ratio,on average typically requiring one quarter the data of an picture.Again,our analysis showed that this does not hold for a signi?cant number of cases.

2.2Group of Pictures

Predictive coding,i.e.,the current frame is predicted from the previous one,cannot be used inde?nitely,as it is prone to error propagation.A further problem is that it becomes im-possible to decode the transmission if reception begins part-way through.In real video signals,cuts or edits can be present across which there is little redundancy.In the ab-sence of redundancy over a cut,there is nothing to be done but to send from time to time a new picture information in absolute form,i.e.,an frame.As decoding needs no pre-vious frame,decoding can begin at coded information,for example,allowing the viewer to switch channels.An frame together with all of the frames before the next frame form a group of pictures(GOP).The GOP length is?exible,but12or15frames is a common value.Furthermore,it is common industrial practice to have a?xed pattern(e.g.

).However,more advanced encoders will attempt to optimize the placement of the three frame types according to local sequence characteristics in the

2

context of more global characteristics.Note that the last frame in a GOP requires the frame in the next GOP for decoding and so the GOPs are not truly independent.Inde-pendence can be obtained by creating a closed GOP which may contain frames but ends with a frame.

a)Frame types and Group of Pictures

b)Forward()and bidirectional()prediction

Encoding and

display order

Transmission and

decoding order

c)Changes in frame sequence

Figure1:MPEG-2video stream characteristics

2.3Transmission vs display order

As we mentioned above,frames are predicted from two or frames,one in the past and one in the future.Clearly information in the future has yet to be transmitted and so is not normally available to the decoder.MPEG gets around the problem by sending frames in the“wrong”order.The frames are sent out of sequence and temporarily stored.Figure 1-c shows that although the original frame sequence is,this is transmitted as ,so that the future frame is already in the decoder before bi-directional decoding begins.Picture reordering requires additional memory at the encoder and decoder and delay in both of them to put the order right again.The number of bi-directionally coded frames between and frames must be restricted to reduce cost and minimize delay,if delay is an issue.

3Analysis of Various MPEG streams

We have analyzed a number of realistic MPEG streams to get a more clear picture about which assumption about MPEG are valid.Some types of videos are more sensitive for frames dropping.For example,dropping4frames in an action video reduces half of the

3

original video quality,50%,while only10%in a cartoon video[2].Therefore we have analysed different types of movies such as action movie,drama,cartoons,etc.

3.1Simulation environment

We have analysed the contents of original DVD movies.The movies were not encrypted or copy protected in any sense,which means that we managed to rip their context on a hard drive by using only legal ripping software,i.e.,the one that will not try to break the CSS protection code on a DVD.

Ripped MPEG streams were analysed by an own-written piece of software(C-program). It takes approximately10minutes to analyse a100minutes long MPEG stream on a PC computer with the processor speed of1,5GHz.

3.2Analysed DVD movies

An overview of the movies we analyzed is summarized in table1.and refer to the GOP length and distance between reference frames respective,e.g.GOP(12,3)means -to-distance is12,while-to-and-to-distance is3.

3.3Analysis results:Mission Impossible2

Here is the data for the movie Mission Impossible2.Table2sumarizes GOP and frame size properties for the movie.Minumum,maximum and average size is given in bits.

The size ration between average values for respective frame type is::=4:2:1,which means that on average frames are twice as big as the frames,and times bigger than the frames.However,this does not hold for a signi?cant number of cases,which is depicted in table3.

For example,in“Mission Impossible2”we have a case with10%GOPs in which have the largest size,and1%of frames,which corresponds roughly to13and1.5 minutes,resp,in a90minute feature?lm.Clearly,such deviations from average cannot be ignored.Furthermore we can see from table3that frames in a GOP are not sorted according to their bitsize,e.g.,in81%of the cases,the frame that is closest to the frame was not the largest among all frames in the GOP.

We have also analysed the distribution of the frame sizes.We have divided the range be-tween minimum and maximum frame size for respective frame type into ten size intervals, and identi?ed the number of frames in respective interval.In that way we can e.g say that the majority of frames have bitsize between some X and Y.This is depicted in?gure2.

For example,from?gure2we can see that88%of the frames has bitsize between 197737and790684bits(200-800kB),which is a quite large interval.The assumptions about MPEG based on average frame size will not hold in this case,since the signi?cant number of frames will have twice as large repective twice as small bitsize,compared to the average frame size(which is500kB).

4

Movie title Length Resolution GOP Mission Impossible2118min720x576(12,3)

Drama258700 Chicken Run104min720x576(12,3)

Thriller309800

The Matrix122min720x480(12,3)

Music257000

The Sea55min720x480(12,3)

Table1:Analyzed MPEG streams

Item Count Minimum Maximum Average Std deviation

I16873881976584506109187598

P49679161216000234821109889

B1128603276904814820457615

GOP168738875414962222249746767 Table2:Mission Impossible2-Bitsizes for frames and GOPs

GOP property Percent

Open GOPs76%

Closed GOPs24%

GOPs with normal length(12)77%

Largest frame89%

Largest frame10%

Largest frame1%

GOPs where31%

GOPs where26%

GOPs where39%

some previous in the GOP81%

some previous in the GOP97%

Table3:Mission Impossible2-GOP properties

5

Interval From

To Nr of I Percent 1881977378765,2%2197737395386219013,0%3395386593035941055,8%4593035790684313718,6%

57906849883334262,5%698833311859821290,8%711859821383631790,5%813836311581280510,3%915812801778929230,1%101778929197658460,0%

Interval From To Nr of P Percent

1161216146377

12,8%21216142432122235545,0%32432123648101446029,1%4364810486408549611,1%54864086080068571,7%66080067296041020,2%7729604851202170,0%8851202972800110,0%9972800

109439830,0%101094398121600010,0%

Interval From

To Nr of B Percent 1327693373656,5%2769331538345993853,1%31538342307353582731,7%423073530763683707,4%530763638453711951,1%63845374614381130,1%7461438538339320,0%8538339615240130,0%961524069214130,0%1069214176904820,0%

Figure 2:Mission Impossible 2-Size distribution for I,P and B frames

6

3.4Analysis results:Leaving Las Vegas

The GOP and frame sizes for the movie Leaving Las Vegas are presented in table4.

The GOP properties are described in table5and the size distribution is shown in?gure3.

3.5Analysis results:Chicken Run

The size data and GOP properties for the cartoon Chicken Run is presented in tables6and 7.The size distribution is shown in?gure4.

3.6Analysis results:The Usual Suspect

The size data and GOP properties for the movie The Usual Suspect can be found in in tables 8and9.The size distribution is depicted in?gure5.

3.7Analysis results:The Matrix

The GOP and frame sizes for the movie The Matrix are presented in table10.The GOP properties are described in table11and the size distribution is shown in?gure6.

3.8Analysis results:New Year’s Concert

The size data and GOP properties for the cartoon New Year’s Concert is presented in tables 12and13.The size distribution is shown in?gure7.

3.9Analysis results:The Sea

The GOP and frame sizes for the movie The Sea are sumarized in table14.The GOP properties are described in table15and the size distribution is shown in?gure8.

7

Item Count Minimum Maximum Average Std deviation I137161361469848471886140329 P5286032100983223114576835 B1064783263641615243545746 GOP1371613671857682543520627856 Table4:Leaving Las Vegas-Bitsizes for frames and GOPs GOP property Percent Open GOPs98% Closed GOPs2% GOPs with normal length(12)92%

Largest frame94% Largest frame6% Largest frame0,4% GOPs where6% GOPs where2% GOPs where37%

some previous in the GOP84%

some previous in the GOP100% Table5:Leaving Las Vegas-GOP properties

Frame type Nr of frames Min Max Avg Std dev I10139574241121216674549216068 P3040612721097336255551133372 B80861126489124011518553982 GOP101396971246802002360795622665

Table6:Chicken Run-Bitsizes for frames and GOPs

GOP property Percent Open GOPs100% Closed GOPs0,2% GOPs with normal length(12)99% Largest frame92% Largest frame8% Largest frame0,1% GOPs where8% GOPs where1% GOPs where12% some previous in the GOP81%

some previous in the GOP100%

Table7:Chicken Run-GOP properties

8

Interval

From To Nr of I Percent 11361469851881,4%214698529397010897,9%3293970440955450332,8%44409555879405281

38,5%5587940734925223116,3%67349258819103442,5%78819101028895630,5%810288951175880140,1%91175880132286510,0%10

1322865146985020,0%

Interval

From To Nr of P Percent

132100984

22064,2%21009842019681599930,3%32019683029522663050,4%4302952403936709813,4%54039365049206181,2%65049206059042520,5%7605904706888350,1%870688880787230,0%9

80787290885600,0%10908856100984020,0%

Interval From

To Nr of B Percent 132636429140,9%2636421272841463013,7%31272841909262527523,7%419092625456883857,9%52545683182106930,7%6318210381852540,1%7381852445494120,0%844549450913620,0%950913657277800,0%1057277863642000,0%

Figure 3:Leaving Las Vegas -Size distribution for I,P and B frames

Item

Count Minimum

Maximum Average Std deviation

I 1340428561282720514744174307P 4008832120480828186792779B 988683276204812953748890GOP 134041331258967282324673583043

Table 8:The Usual Suspect -Bitsizes for frames and GOPs

9

Interval From To Nr of I Percent 1574241638032072,0%21638032701821651,6%32701823765616436,3%43765614829409489,4%5482940

589319114011,2%6589319695698205620,3%7695698802077209820,7%8802077908456149414,7%990845610148358788,7%10101483511212165105,0%

Interval From To Nr of P Percent 1127211087826168,6%21108782204841124537,0%3220484330090

976832,1%4330090439696447314,7%543969654930212984,3%65493026589084781,6%76589087685142790,9%87685148781201410,5%9878120987726630,2%10987726109733645

0,1%

Interval

From To Nr of B Percent

11264902612976736,8%2902611792584144951,3%3179258268255860510,6%42682553572529411,2%5357252446249830,1%644624953524690,0%753524662424350,0%862424371324000,0%971324080223710,0%1080223789124010,0%

Figure 4:Chicken Run -Size distribution for I,P and B frames GOP property

Number of GOPs

Percent

Interval From To Nr of I Percent 128561308421981,5%21308422588283562,7%3258828386814223816,7%4386814514800440832,9%5514800642786

342225,5%6642786770772157911,8%77707728987586755,0%889875810267442892,2%910267441154730880,7%1011547301282720120,1%

Interval From To Nr of P Percent 1321205097521,9%21205092409861280932,0%

32409863614632090052,1%4361463481940442811,0%54819406024177932,0%66024177228942530,6%77228948433711030,3%8843371963848330,1%99638481084325130,0%10108432512048084

0,0%

Interval From

To Nr of B Percent 1327623382558,3%2762331524346674467,5%31524342286352101521,3%422863530483620162,0%53048363810374390,4%63810374572381610,2%74572385334392020,2%8533439609640250,0%960964068584170,0%1068584176204830,0%

Figure 5:The Usual Suspect -Size distribution for I,P and B frames

Item

Count Minimum

Maximum Average Std deviation

I 146634110476000043008870920P 43920127280901624957665226B 117090318466496813672541336GOP 146677608843226482269353408220

Table 10:The Matrix -Bitsizes for frames and GOPs

11

GOP property Number of GOPs Percent

Item Count Minimum Maximum Average Std deviation I14541343218950881019897363358 P5524832145995239657998782 B11039624156596018491851664 GOP145418912106358404000410806103 Table12:New Year’s Concert-Bitsizes for frames and GOPs GOP property Number of GOPs Percent

Open GOPs8020100% Closed GOPs160% GOPs with normal length(12)767495% Largest frame767295% Largest frame3574% Largest frame70%

GOPs where131716% GOPs where153219% GOPs where3334% some previous in the GOP587273%

some previous in the GOP7997100%

Table15:The Sea-GOP properties

13

Interval From To Nr of I Percent 134321925974743,3%21925973817623582,5%33817625709275503,8%457092776009210737,4%5760092949257

303620,9%69492571138422397827,4%711384221327587194513,4%81327587151675212008,3%9151675217059178025,5%10170591718950885513,8%

Interval From To Nr of P Percent

13214602418663,4%2

14602429201620593,7%32920164380083619465,5%44380085840001392125,2%55840007299927621,4%67299928759844150,8%78759841021976270,0%81021976

116796820,0%91167968131396000,0%101313960145995220,0%

Interval From To Nr of B Percent 1241566172425522,0%21566173132108548577,4%33132104698035610,5%4469803626396730,1%562639678298950,0%678298993958240,0%7939582109617530,0%81096175125276860,0%91252768140936100,0%101409361156596030,0%

Figure 7:New Year’s Concert -Size distribution for I,P and B frames

14

Interval From To Nr of I Percent 114392949261341,7% 294926175460360,4% 3175460255994510,6% 4255994336528710,9% 5336528

4170621021,3% 6417062497596116514,5% 7497596578130248030,9% 8578130658664240329,9% 9658664739198126715,8% 107391988197362893,6%

Interval From To Nr of P Percent 132764981040,4% 276498152964

1270,5%

31529642294301800,8%

42294303058964461,9%

53058963823627593,2%

63823624588282152690,0%

74588285352947813,3%

853529461176010,0%

961176068822630,0%

1068822676469620,0%

Interval From To Nr of B Percent

132424163450,5%

242416848002620,4%

3848001271847071,1%

412718416956819243,0%

51695682119524214866,1%

62119522543361809628,4%

72543362967202000,3%

8296720339104570,1%

933910438148840,0%

1038148842388030,0%

Figure8:The Sea-Size distribution for I,P and B frames

15

Avg size ratio frames

average average average

Mission Impossible2506109234821148204

6:3:21403297683545746 Chicken Run674549255551115185

4:2:11743079277948890 The Matrix430088249576136725

6:2:13633589878251664 The Sea568826414148199928 Table16:Comparrison of bitsize properties for all analysed movies

Number of GOPs where

largest largest

Mission Impossible210%31%39%

Leaving Las Vegas5%6%37%

Chicken Run8%8%12%

The Usual Suspect11%32%8%

The Matrix7%17%10%

New Year’s Concert7%27%15%

The Sea4%16%4%

4Comments on analysis results

An overview of the movies we analyzed is summarized in table16and17.Here we mach the most common assumptions about MPE video streams with our analysis results.

Assumption1:-frames are the largest and frames are the smallest.This assumption holds on average.In all the movies that we analysed,the average sizes of the frames were larger than the average sizes of the frames,and frames were larger than frames on average,with frame size ratio::=4:2:1.Of course,the ratio depends also on the movie content,i.e.,the ratio for the New Year’s Concer movie that we analyzed was::, re?ecting the fact that the we have a quite static background which is not cahnged often, so the difference between current frame and the next one gets smaller.In other words,we need less bits for predicted frames.

However,our analysis showed that this assumption is not valid for a signi?cant number of cases.For example,in“The Usual Suspect”we have a case with11%GOPs in which have the largest size,and1%of frames,which corresponds roughly to14and2minutes of the movie.Clearly,such deviations from average cannot be ignored.

Assumption2:-frame is always the largest one in a GOP.This is not true.For example in the movie“Mission Impossible2”the frame was larger than the frame in 31%of the GOPs.The frame might be the most important one in the GOP from the reconstruction point of view,but it does not necessarily has to be the largest one.

Assumption3:-frames are always the smallest ones in a GOP.Neither this assump-tion is true.For example,in“Mission Impossible2”a frame was largest in1%of the cases.And in39%of the cases,a frame was larger than all frames in the same GOP. This implies that even the assumption that frames are always larger than frames is also not valid.Another example is GOP nr393in“Mission Impossible2”where the frame is almost100times larger than the frame().

Assumption4:-The sequence structure in a GOP is?xed to a speci?c I,P,B frame pat-tern.Not true.In23%of the GOPs in“Mission Impossible2”the GOP length was not12 frames.Not all GOPs consist of the same?xed number of and frames following the frame in a?xed pattern.That is because more advanced encoders will attempt to optimize the placement of the three picture types according to local sequence characteristics in the context of more global characteristics.For instance scene changes or large changes in video content do not occur regularly,and hence the need for frames in most video sequences is not at regular intervals.

Assumption5:-Frame properties for all movies are the same.Neither this is true.Our analysis showed big variations between frame sizes,GOP pattern and the impact on the overall output video quality depending on the number of dropped frames.Different kinds of video will also effect the perceived quality of the video.For instance,the viewer will perceive jerky motion much easier if we drop frames in an action movie than in a cartoon. Assumption6:-and frames are sorted in a GOP according to their sizes in de-scending order.This is not true.There is no such an ordering within a GOP.As a matter of fact,our analysis showed that the largest frames are placed towards the end of the GOP. So,the“best-effort”algorithms will perform badly when skipping the last frames in the GOP.

17

Assumption7:-All frames are equally important.Not true.sizes vary a lot.In our analysis we could see that e.g.in“Leaving Las Vegas”almost90%of the frames is in a pretty large interval between6000and300000bits.So,if we drop a large frame,the entire GOP could be ruined.On the other hand,more bits does not necessarily mean better quality.That is because motion vectors give the highest compression ratio,but are smallest. So,a frame with a lot of motion vectors would have less data than some annother frame with more row picture information,but still give better output quality when decoded All this implies that selection of frames to be dropped should be performed carefully. Assumption8:-Frame sizes vary with minor deviations from the average value.Not true.For example,from?gure2we can see that88%of the frames has bitsize between 197737and790684bits(200-800kB),which is a quite large interval.The assumptions about MPEG based on average frame size will not hold in this case,since the signi?cant number of frames will have twice as large repective twice as small bitsize,compared to the average frame size(which is500kB).

18

References

[1]ISO/IEC13818-2:Information technology-Generic coding of moving pictures and associated

audio information,Part2:Video.1996.

[2]J.K.Ng,K.R.Leung,W.Wong,V.C.Lee,and C.K.Hui.Quality of Service for MPEG

Video in Human Perspective.In Proceedings of the8th International Conference on Real-Time Computing Systems and Applications(RTCSA2002),Tokyo,Japan,March2002.

[3]L.Teixera and M.Martins.Video compression:The MPEG standards.In Proceedings of the1st

European Conference on Multimedia Applications Services and Techniques(ECMAST1996), Louvian-la-Neuve,Belgium,May1996.

[4]J.Watkinson.The MPEG handbook.ISBN024*******,Focal Press,2001.

19

本文来源：https://www.bwwdw.com/article/lf9e.html