Ask questions to Compression Engineer insider "drmpeg"

[newt07]

Originally Posted by drmpeg

big post

Ron

Wow, thanks for this great information. You've certainly given a lot.

Those graphs, how'd you generate them? Whatever you use to make them, it looks really useful.

[drmpeg]

Originally Posted by Mobe1969

I've a question (or two). DrMpeg, were/are you involved in SD DVD encoding? Just wondering as a long term bug-bear of mine was edge enhancement. I hate it with a passion. So many times I've put in a DVD only to look in disgust at the amount of edge enhancement. So many good DVDs have been ruined, including a lot of the Bond Lowrey remasters, all by someone god awful edge enhancing on encoding.

I've had a question I've always wanted to ask an encoding engineer. Why? Is there some reason they do it to a transfer? Do they think it looks good? Are they forced to do it by some executive? Is it easier? From my perspective I honestly can NOT think of even one reason for edge enhancement, so have been wanting to ask an insider to see if there actually is a reason.

Thankfully with HD they don't seem to be able to get away with it though, which is such a blessing. People just seemed to put up with it on DVDs though.

But my question has always been "WHY!!!!???!!!"

My take on EE is that it's a function of what the studios considered to be the average consumer display during the early and mid years of DVD. In other words, a 20 to 32 inch CRT. So to get some "pop" out of those displays, they cranked up the EE.

Fast forward to 2008, and the average consumer display is an LCD, probably starting at 27 inches, with many approaching 40 inches. Now you have displays with some "pop" pretty much built in, and EE tends to diminish the experience on these displays. Whether the studios have gotten the message, you'll have to be the judge by looking at newer titles.

As you mention, for blue laser HD content, there doesn't seem to be too many reports of EE, although I guess there are a few.

Ron

[drmpeg]

Originally Posted by 4K2K

So when do you think we will see Blu-ray players in England that can play 1080p60 and 1080p50 content and 10 or 12 bit colour? Will they have motion interpolation technology for converting 24p to 60p?

Also, do you think it would be possible to increase the bandwidth in players/media?

I'm afraid it may be too late for Blu-ray to include native 1080p@50/60 and/or extended (greater than 8) bit depth in the specification. Given that the vast majority of titles are 24p content, I'm not sure 1080p@50/60 is even that useful. Do folks really watch sports content on DVD or blue laser HD? I'm sure a few do, but I'd have to think the majority don't. At least for me, sports events are a one time view, and you can get the highlights on ESPN.

I'm a big fan of 10-bit content, but I don't think that will happen either. Again, the Blu-ray specification is locked-in, and is not likely to change for the duration (look at the static nature of the DVD specification). Also, as I've mentioned before, 10-bit codecs (both encoders and decoders) are an architectural pain, and unless there's a big motivation ($) for companies to develop 10-bit codecs, it's going to be a chicken and egg game.

Ron

[drmpeg]

Originally Posted by newt07

Wow, thanks for this great information. You've certainly given a lot.

Those graphs, how'd you generate them? Whatever you use to make them, it looks really useful.

It's an old program called xvgr that only runs on Sun Solaris workstations. I've never been able to find anything for Windows as flexible and as useful as that program. If I ever end up at a company that doesn't use Sun boxes, I'll have to buy one for myself just to run xvgr.

Ron

[drmpeg]

Originally Posted by Icemage

Ron,

Thanks so much for this primer on bitrates. I've read bits and pieces of the theory behind how "average bitrate" is achieved, as well as what I/P/B frames do, but I think this is the first time I've ever seen those theories applied with graphs, numbers, and a visual representation of what can happen when things get out of control.

During the encoding process, is the level of quantization known by the encoder, or is this a value that is derived upon hindsight analysis of the resulting stream? The reason I ask is because there is just horrid macroblocking all over that quant 19 P frame. I can just imagine what the B-frame(s) based on that P-frame will look like...

I'll have a nice graphical answer for this question. Stand by.

Ron

[Mobe1969]

Originally Posted by drmpeg

My take on EE is that it's a function of what the studios considered to be the average consumer display during the early and mid years of DVD. In other words, a 20 to 32 inch CRT. So to get some "pop" out of those displays, they cranked up the EE.

Fast forward to 2008, and the average consumer display is an LCD, probably starting at 27 inches, with many approaching 40 inches. Now you have displays with some "pop" pretty much built in, and EE tends to diminish the experience on these displays. Whether the studios have gotten the message, you'll have to be the judge by looking at newer titles.

As you mention, for blue laser HD content, there doesn't seem to be too many reports of EE, although I guess there are a few.

Ron

Thanks for that insight. So it was at the behest of the studios. And to the detriment of people with good equipment. Sigh

[damonous]

Originally Posted by drmpeg

To be honest, I don't have much exposure to compressionists, since my background is almost entirely in real-time encoders for broadcast applications. ...

drmpeg (Ron), thanks for taking the time out for some great info on this Forum. I have a couple of long-burning questions you might be able to answer...


1) Since your background is broadcast application, what do you know of Cable; specifically TWC? I've read unsubstantiated info in the past that Cable providers sometimes re-encode MPEG streams to fit their bandwidth needs. I never used to think this, because compared to my friends' Dish/satellite HD, I've always felt Cable was superior.

But, of late, certain things on TWC in my area (Cleveland) has been looking VERY inconsistent, with some very unexpected 'smelly sights'. In particular, the Super Bowl looked atrocious with gobs of messy macroblocking. Ditto to the Grammys. Other times, I see some absolutely spectacular HD, like Prison Break on Fox (in 1080i even, according to my TV's indicator).

In total, though, all channels seem to be worse (more macroblocking) than in the past. Is it possible that TWC is re-encoding and selectively cranking-down the bits?


2) You posted a page or two back about banding, and I'm curious about what it the primary cause. Is it the bit depth for color, or bit depth for encode, or the codec itself?

Case in point: I recently rented Sunshine on BD. I've always been VERY fond of the quality of Fox/MGM BD's, owning about 15.

For the most part (like 99.99999% ), this was another stunning transfer with spiderweb-fine lines clearly depicting the spaceship's communication arrays, and excellent compression and greyscale control (especially for a film advertised on the sleeve as AVC at only 14mbps!).

Yet, there is one absolutely atrocious moment at the 3/4 mark of the movie, where the ship is moving in space against an eclipse of Mercury against the Sun. Suddenly, a frightening blooming mush of muddy 256-color-looking orange halo around the ship (where a proper haze/mist cloud should have been).

It's a BD50, --so what gives? ...Was the compressionist asleep at the wheel? Is it a limitation of AVC? Is it a limitation of available colorspace?

[drmpeg]

Originally Posted by Icemage

During the encoding process, is the level of quantization known by the encoder, or is this a value that is derived upon hindsight analysis of the resulting stream? The reason I ask is because there is just horrid macroblocking all over that quant 19 P frame. I can just imagine what the B-frame(s) based on that P-frame will look like...

The level of quantization is set for each macroblock by the encoder based primarily on bitrate. Since the quantization level controls how many bits are expended for each macroblock, it's the encoders bitrate "throttle".

The typical rate control algorithm decides how many bits a picture should expend before it's encoded. This is called the picture "bit budget". It's primarily a function of frame type. Big I-frames, smaller P-frames and even smaller B-frames. A fairly common trick is to increase I-frame size (and consequently decrease P and B-frame size) during low motion or still sequences and decrease I-frame size (increase P and B-frame size) on heavy motion.

Once the encoder starts coding the picture, it keeps track of how it's doing versus the bit budget. If it's over-shooting the bit budget, the macroblock quantization will increase. If it's under-shooting, the quantization will decrease.

Here's an I-frame example from the same Fox bitstream:



The bits expended in each macroblock (hotter colors are more bits):



The quantization level for each macroblock:



Note that the block images are cutoff on the right hand side. It's just a function of my Sun workstation's 1152 pixel wide display.

The bits and quantization map shows a few things. First, the text in the lower middle of the image takes a lot of bits, so the quantization goes up for those macroblocks. Second, it's segmented vertically into six segments. This is the mark of the multi-chip encoder. In this case, a Harmonic MV-400, MV-450 or MV-500 which uses six C-Cube/LSI Logic chips for 720p. Finally, a lot of bits are expended coding the noise on the first line of the image. I'm not sure what that noise is, but you can see it's a real waste of bits (and artificially raises the quantization level at the top of the picture).

Ron

[drmpeg]

Here's the P-frame before the scene change that caused the badly coded frame shown previously.



Quantization levels:



Motion vectors:



Bits per macroblock:



And the residual:



The residual is a representation of the differences sent in a motion compensated frame. Ideally, the residual picture would be all gray, indicating no residual (and perfect motion estimation). The residual is useful because it shows how well the motion estimation did and also how many bits will be required to send motion compensation coefficients.

This frame has very little residual, so the motion estimation was pretty good (as indicated by the low quantization levels). Intra blocks are shown as the block itself, so they will tend to stick out in the residual (the fully black or white blocks).

Ron

[drmpeg]

The actual scene change occurs on a B-frame.



The encoder tries to do the right thing. Here's a map of the macroblock mode decisions. Orange is backward predicted, blue is forward predicted, purple is bi-directionally predicted, dark blue is intra and yellow is skipped. The frame is mostly backward predicted, which is the correct choice at a scene change.



The motion vectors. The long incoherent vectors are the forward vectors. The backward vectors are the dots in the middle of each block:



The residual:



And the quantization levels:



Since this B-frame occurs after the bad P-frame (in encoding order), there just no bits left over. The frame wasn't badly motion estimated, but it ends up poorly coded because it has to pay the bitrate price from the previous P-frame (and because the P-frame predictor wasn't very good).

Ron

[drmpeg]

Finally, here's the reason for the bad P-frame.

The motion vectors. Very chaotic:



The residual:



The huge residual takes a ton of bits to encode, so the quantization level soars:



Ron

[irfoton]

Isn't xvgr a Unix based plotting package? If so, I think it runs on a Mac as well. The new Mac Pros are real nice too.

[PaulGo]

Ron - thank you for those pictures and explanation! I am not trying to politicize this but I appears for a properly written encoder the higher the allowable bandwidth along with more disc space you should get a better product. The point I am trying to make is when Microsoft states VC-1 can obtain optimal results within the HD-DVD bandwidth and disc space it really is BS (or their encoder can't utilize the additional bandwidth properly).

[drmpeg]

Originally Posted by damonous

drmpeg (Ron), thanks for taking the time out for some great info on this Forum. I have a couple of long-burning questions you might be able to answer...


1) Since your background is broadcast application, what do you know of Cable; specifically TWC? I've read unsubstantiated info in the past that Cable providers sometimes re-encode MPEG streams to fit their bandwidth needs. I never used to think this, because compared to my friends' Dish/satellite HD, I've always felt Cable was superior.

But, of late, certain things on TWC in my area (Cleveland) has been looking VERY inconsistent, with some very unexpected 'smelly sights'. In particular, the Super Bowl looked atrocious with gobs of messy macroblocking. Ditto to the Grammys. Other times, I see some absolutely spectacular HD, like Prison Break on Fox (in 1080i even, according to my TV's indicator).

In total, though, all channels seem to be worse (more macroblocking) than in the past. Is it possible that TWC is re-encoding and selectively cranking-down the bits?

HD encoders are too expensive for head-ends to re-encode all of their HD channels. Instead, they use rate-shaping boxes called "cherry pickers" that reduce the bitrate of the incoming streams by re-quantizing the macroblocks in the MPEG-2 video elementary stream.

The only way to really tell what your cable company is up to is to capture some cable QAM bitstreams and compare them to an OTA capture made at the same time. If you can capture both versions, let me know, and I can analyze them for you.

Originally Posted by damonous

2) You posted a page or two back about banding, and I'm curious about what it the primary cause. Is it the bit depth for color, or bit depth for encode, or the codec itself?

Case in point: I recently rented Sunshine on BD. I've always been VERY fond of the quality of Fox/MGM BD's, owning about 15.

For the most part (like 99.99999% ), this was another stunning transfer with spiderweb-fine lines clearly depicting the spaceship's communication arrays, and excellent compression and greyscale control (especially for a film advertised on the sleeve as AVC at only 14mbps!).

Yet, there is one absolutely atrocious moment at the 3/4 mark of the movie, where the ship is moving in space against an eclipse of Mercury against the Sun. Suddenly, a frightening blooming mush of muddy 256-color-looking orange halo around the ship (where a proper haze/mist cloud should have been).

It's a BD50, --so what gives? ...Was the compressionist asleep at the wheel? Is it a limitation of AVC? Is it a limitation of available colorspace?

It's mostly a problem with 8-bit video, but the encoder (of any codec, not just AVC) can make it worse. A look at the quantization levels for those frames would show if the encoder was making things worse.

Without seeing the 10-bit source, it's difficult to know what the compressionist was thinking. The compressionist may have taken a look at an 8-bit version of the source and decided the encode was faithful (enough) to that.

Ron

[kapitalisten]

Ron,

may one inquire about what kind of tools you use to visualize quantization levels, residuals and so forth - or is that a "secret of the trade"?

[damonous]

Originally Posted by drmpeg

It's mostly a problem with 8-bit video, but the encoder (of any codec, not just AVC) can make it worse. ...

Without seeing the 10-bit source, it's difficult to know what the compressionist was thinking. The compressionist may have taken a look at an 8-bit version of the source and decided the encode was faithful (enough) to that.

Ron

Ron, thanks loads for that response! I have a Fusion HDTV card in my PC that I'd be able to capture Cable vs OTA on, but unfortunately not simultaneously, so I don't know how effective that would be.

On the question of the the banding .... wow: you kind of opened up a whole other can of worms. --Are you implying that even 10bit color could leave stair-stepping in color gradations? ...I mean, is this really then the achilles heel of digital video? Is it the one thing where film will always be superior (ie: depiction of gradual color gradations in fog-like transitions)?

[Penton-Man]

Ron,
This may be worthy of a road trip sometime down the line…………..
http://press.ucsc.edu/text.asp?pid=2142

further info……………..
http://library.ucsc.edu/speccoll/GD_archive.html

[oppopioneer]

I'm getting hammered by the "compression" fans of Dolby who don't like that I do favor LPCM for audio even if it takes up lots of space. I do believe that uncompressed is ever so faithfully closer bit-for-bit to the original masters than lossy and I like the idea of not bitstreaming so I'm taking some stress off my reciever. Again, I'll get hammered for saying this. The LPCM fans seem to be a dying breed for the bluray world.

[PeterTHX]

You're not getting "hammered".

You just get things wrong. Neither Dolby TrueHD or DTS-HD MA are "lossy". They simply pack the PCM and expand it in the decoder, saving space and losing NOTHING of the original signal. If you're worried about "stressing" your receiver (which it doesn't) the OPPO BD players can internally decode them and send the audio as PCM.

Nor is the spec going to be updated to DXD.

Nor will we see studio releases using higher capacity discs.

Nor is this an audio compression discussion, but a video one.

[oppopioneer]

I've read on another thread that for 3D and other future bd technology regarding PQ that MPEG is prefered or is compatible with it and not VC-1. Is VC-1 going to continue to be used or be phased out?

Also, are they working on an MPEG-6 ?

Which is a superior system to use for best PQ?