Spears & Munsil 4K UHD HDR Benchmark (Test Discs) May 15, 2023

[Stacey Spears]

We also developed a new pattern for Chroma Upsamping Evaluation. Mathias, from MadVR, helped inspire it.

It won't be useful for most people. The pattern actually shows off how good the AI upsamping is in MadVR or some new, unreleased, algorithms in MPC-HC are. These include bilateral and SpearsBilateral that were written by Shaindow. (if you track those projects or read Doom9)

It is really targeted at people who write 4:2:0 to 4:4:4 conversion algorithms.

Here are some sample images from both. I hope this link works. LOL

Take a look at the images in the MPC folder. Here are sections of the pattern converted back to RGB using Bicubic, Bilateral and SpearsBilateral, which uses the Y channel to reconstruct Cb and Cr. (guided bilateral) You want to view them at full resolution, not scaled down. These are from an unreleased version of MPC-HC for Windows. Because MPC renders at your monitor resolution, Shaindow (Author of Bilateral and SpearsBilateral) had to be creative to capture the images for me, hence they are cropped.

Look at the red to green and magenta to cyan transitions. In bicubic, you get the normal thicker line between them due to non-constant luminance. In the two bilateral examples, the transition is nicer, less noticable. I imagine these would look even better if ICtCp was used instead of YCbCr, but have not tested it as we have not done the work to create native ICtCp patterns yet.

In the zip file, under MadVR, take a look at row 2 and 4 on these two images:
fullFrameColoredBilateral.png
fullFrameColoredNGU.png

And the Cr only version (Converted to B&W):
fullFrameCrBilateral.png
fullFrameCrNGU.png

NGU is MadVRs AI based upscaling and I think it does a great job keeping row 2/4 sharp and artifact free vs. the bilateral. The MadVR bilateral is different from the MPC-HC bilateral. MadVR does not do as well with the color transitions as MPC-HC.

Some additional notes:

There are five rows.
Row 1: Background is 0% of code value. (64)
Row 2: Background value is the same as the Y value of red. Idea is there is no luma for a guided bilateral to use to help reconstruct chroma. There was some schmutz due to anti aliasing of the text, so we flatten the Y channel after conversion to 4:2:0 to ensure its clean of schmutz.
Row 3: Background is 50% of code value. (502)
Row 4: Same as row 2, but we don't flatten Y and we added a pattern to confuse a guided algorithm. The bilateral example shows some ugly artifacts.
Row 5: Peak luminance in HDR or 100% of code value in SDR. So when we say different nit versions, that applies specifically to row 5. The 10,000 version would be 940, etc...

1. The left half of the image uses a bilinear down sample and the right half bicubic. This impacts everything except the horizontal and vertical chroma alignment diamonds because they are generated in 4:2:0 after everything else is converted to 4:2:0.

2. The color blocks were chosen because they show either a horizontal or vertical transition that is thicker, which some algorithms eliminate. (Bilateral under MPC in the zip file)

3. The big red diamonds that are animated are to look for the old chroma bug artifact. It changes size throughout to make it more obvious if interlaced upsampling was used on progressive 4:2:0.

4. The spirals are to look for aliasing.

5. The thin red diamonds (It's what we call the chroma alignment bars) that surround the animated chroma diamond are to look at co-siting alignment. We have also found that delay is introduced in some scalers. e.g. There was a Sony TV that would introduce a half pixel delay when scaling HD to UHD. More info on those diamonds in our choosing a color space article.

6. The red on yellow is also for alignment.

7. The value of red does change based on nit level. SDR would be 100% (940, 64, 64) of code value as would 10,000 nit HDR. We also do that for the color bar patches and animated red diamonds.

The above is a copy / paste from emails I have sent around to various people interested in chroma upsampling. I keep fixing obvious errors in the text above as I find them.

That is the nuts and bolts of this new pattern. Now we just need people comparing players and writing about it to motivate companies to improve their algorithms.

[Geoff D]

Have you seen the state of AV journalism lately, Stacey? Not a pretty sight.

[Stacey Spears]

Originally Posted by Geoff D

They do indeed, and I'll be perfickly honest and say that the HLG version of the Montage was the best mapped version by FAR to my eyes on the ZD9, superior to the TV's clipping of extreme brightness/colour in HDR10 and its borked player-led Dobly implementation.

The HLG version is the 1000 nit HDR10 version of the pattern (converted from PQ to HLG in Colorfront Transkoder) since that is the peak luminance limit of HLG. Though, BBC wants to use headroom (940-1019) to go brighter while NHK would rather not.

[Stacey Spears]

Originally Posted by Geoff D

Have you seen the state of AV journalism lately, Stacey? Not a pretty sight.

Every time I am near a magazine rack, I look for AV magazines and can't find any. I miss the days of receiving Widescreen Review in the mail.

One of our goals, with the videos, is to explain how to use the patterns to review products. We won't be doing reviews ourselves though, mostly do to lack of time.

[NKB2017]

Originally Posted by Stacey Spears

Every time I am near a magazine rack, I look for AV magazines and can't find any. I miss the days of receiving Widescreen Review in the mail.

One of our goals, with the videos, is to explain how to use the patterns to review products. We won't be doing reviews ourselves though, mostly do to lack of time.

I wondered whatever happened to Gary and co over at Widescreen Review? Does anyone know? I enjoyed that magazine! Good times.

[wxman2003]

Originally Posted by Stacey Spears

On the configuration menu, move the cursor to P3D65|BT.2020. Then press the right arrow 5 times. It will now show you the hidden settings menu. If the player does not support a feature, such as Dolby Vision, it will also report that the display does not support it even if it does. The 3D support also does not ever seem to report correctly, so we will remove that one on the new disc. Everything else seems to report correctly as far as we can tell.

This menu, or rather the registers we expose in this menu, is what we use to determine if we should display the Dolby Vision and HDR10+ warnings on the montage page.

And to be clear, when you see the warning, we are not blocking playback of those formats. We get a lot of emails about the Dolby Vision warnings from Xbox One customers who insist it supports Dolby Vision. We keep explaining that it only supports it for Netflix. Most of the time they refuse to believe us, even after we send them an email from the dev manager at Microsoft, who is responsible for multimedia playback on Xbox One, saying the same thing.

Thanks for the hidden tip trick. I have a question. When I get into the hidden menu on my OPPO 203, Smooth Transition for Dolby Vision is checked, while for my Panasonic 9000 it is not. What exactly is smooth transition?

Secondly, I noticed with Dolby Vision, that the Panasonic 9000 pushes red more than the 203, and it is obvious, especially in skin tones. With HDR10, they both look alike. I also checked Dolby Vision with my Apple TV 4k, and skin tones look exactly the same as the Oppo 203 without the red push that the Panasonic 9000 has. This is on my LG E6 OLED.

[Fendergopher]

Though I likely won't make use of this disc myself, I have to say the effort and expertise going into it is truly exemplary, especially considering how HDR calibration is still evolving.

[Stacey Spears]

Originally Posted by wxman2003

Thanks for the hidden tip trick. I have a question. When I get into the hidden menu on my OPPO 203, Smooth Transition for Dolby Vision is checked, while for my Panasonic 9000 it is not. What exactly is smooth transition?

Secondly, I noticed with Dolby Vision, that the Panasonic 9000 pushes red more than the 203, and it is obvious, especially in skin tones. With HDR10, they both look alike. I also checked Dolby Vision with my Apple TV 4k, and skin tones look exactly the same as the Oppo 203 without the red push that the Panasonic 9000 has. This is on my LG E6 OLED.

Can you check your Dolby Vision settings in the OPPO under HDR? Is it set to Auto, Player-led or TV-led? Be sure it is set to TV-led as that is what the Panasonic is using. Auto, on OPPO, uses player-led.

No idea what the smooth transition actually means. It may be related to menus. This is one reason we hid them and use it for debugging devices.

If anyone has a combo where SL-HDR2 is checked, would love to know about it. Or at least the player.

[Stacey Spears]

Originally Posted by NKB2017

I wondered whatever happened to Gary and co over at Widescreen Review? Does anyone know? I enjoyed that magazine! Good times.

I checked their website after reading your posts and it says they still print magazines.

[Stacey Spears]

Originally Posted by Fendergopher

Though I likely won't make use of this disc myself, I have to say the effort and expertise going into it is truly exemplary, especially considering how HDR calibration is still evolving.

Your comment about still evolving is spot on. We, as an industry, have decades of experience with SDR, but only 5+ years with HDR.

I joked with the last disc that had we released it a year earlier, it would not be as good and a year later would be even better. This is all based on how little time we have had with HDR calibration as an industry. Five years from now, we could make an even better product than what we are working on now.

Our primary goal is to deliver tools for enthusiasts and reviewers to point out deficiencies and for display manufacturers to address them. We built the tool we wish we had when reviewing products.

Long before Don and I made our first disc, we had the goal of getting Guy Kuo (AVIA) and Joe Kane together to build a disc. That never happened, so we did it ourselves.

[Stacey Spears]

I should mention, our new HDR vs. SDR is different as well. Instead of a static butterfly, it is a continuous rotating wipe between the two.

In addition, instead of putting the SDR peak at 100 nits, we made it 203 nits. To ensure this, we used the title card that had text at 940. We made sure that 100% white, in SDR, is at 203 nits in HDR. You can measure and confirm on the title card because we put the HDR text at the same nit level.

The HDR vs. SDR is really only valid on a display that can do 1000 nits full screen w/o tone mapping. Then you will see both at their proper levels. Tone mapping will make both the HDR and SDR darker than it really is.

[Stacey Spears]

Here is a screen shot from AE rendering out one of the videos for the audio tones.

[Wendell R. Breland]

Originally Posted by Stacey Spears

Here is a screen shot from AE rendering out one of the videos for the audio tones.

Do you have a tentative target release date? Will the test disc be available through the same vendors as the last disc?

Since these disc have some many items and menus how do you folks keep track of everything? Gives me a headache just thing about it!

Will you use x265 again? Ever used Sirius Pixels? Will you author with Scenarist?

[David M]

Originally Posted by wxman2003

Thanks for the hidden tip trick. I have a question. When I get into the hidden menu on my OPPO 203, Smooth Transition for Dolby Vision is checked, while for my Panasonic 9000 it is not. What exactly is smooth transition?

I'm pretty sure it's for menus. After UHD BD launched, they added the option to disable tone mapping while the popup menu was showing over the film, to avoid sudden luminance changes of the menu graphics. Smooth transition means that in such cases where the popup menu being active disables the tone mapping, opening or closing the menu will fade between the two states.

[Stacey Spears]

Some good news this morning. DTS was able to create DTS:X for A/V Sync.

[Stacey Spears]

Originally Posted by Wendell R. Breland

Do you have a tentative target release date? Will the test disc be available through the same vendors as the last disc?

Since these disc have some many items and menus how do you folks keep track of everything? Gives me a headache just thing about it!

Will you use x265 again? Ever used Sirius Pixels? Will you author with Scenarist?

No release date yet. I am hoping this summer, but won't really know until its authored and tested. Once we are close on disc 1 being done, I will post an update. That is the most complicated disc.

The PowerPoint mock-up for the menu helps with the tracking. Then lots and lots of directories. Here is the basic layout. The numbers under baseline are an example of how the different versions of patterns are stored.


Yes, x265, Scenarist and same dealers. I don't know what Sirius Pixels is.

Edit: I just looked up Sirius Pixels. The last encoder build was 2018 according to the feature page. Not sure if that is true. Has a lot of GUI information that look useful.

[wxman2003]

Originally Posted by David M

I'm pretty sure it's for menus. After UHD BD launched, they added the option to disable tone mapping while the popup menu was showing over the film, to avoid sudden luminance changes of the menu graphics. Smooth transition means that in such cases where the popup menu being active disables the tone mapping, opening or closing the menu will fade between the two states.

Thanks. That probably the reason why when the screen remains black between main menu and start of movie in DV on the OPPO, while it fluctuates between grey and black until the movie starts on the Panny.

[David M]

Originally Posted by wxman2003

Thanks. That probably the reason why when the screen remains black between main menu and start of movie in DV on the OPPO, while it fluctuates between grey and black until the movie starts on the Panny.

No, that's a different thing there. The smooth transition relates to the interactive graphics layer being called up over the film, it doesn't have anything to do with transitioning between titles or between the film and the menu.

[wxman2003]

Originally Posted by David M

No, that's a different thing there. The smooth transition relates to the interactive graphics layer being called up over the film, it doesn't have anything to do with transitioning between titles or between the film and the menu.

Thanks. So for the average person, smooth transition doesn't have much of an impact.

[wxman2003]

One thing I have noticed with my LG E6, and my OPPO 203, if I force it to play HDR10 as DV, I need to raise my brightness 1 click, from 50 to 51. I verified this using the 0% HDR10 window on this disc when forcing this disc to display everything as DV. At 51 the window and screen remains black. 52, the screen glows. I also tested it out on several HDR10 discs forced to DV. Fade to black, and the screen is black, and also letterbox bars are black. Oddly enough, when I stop the OPPO from forcing HDR10 to DV, and let it play as HDR10, I have to lower brightness back down to 50, otherwise blacks will glow at a setting of 51. So the OPPO is doing something with the conversion and crushing blacks just a bit. With DV encoded discs, a brightness of 51 causes the blacks to glow and I need to lower to the normal setting of 50.