Spears & Munsil 4K UHD HDR Benchmark (Test Discs) May 15, 2023

[Stacey Spears]

Originally Posted by wxman2003

One thing I have noticed with my LG E6, and my OPPO 203, if I force it to play HDR10 as DV, I need to raise my brightness 1 click, from 50 to 51. I verified this using the 0% HDR10 window on this disc when forcing this disc to display everything as DV. At 51 the window and screen remains black. 52, the screen glows. I also tested it out on several HDR10 discs forced to DV. Fade to black, and the screen is black, and also letterbox bars are black. Oddly enough, when I stop the OPPO from forcing HDR10 to DV, and let it play as HDR10, I have to lower brightness back down to 50, otherwise blacks will glow at a setting of 51. So the OPPO is doing something with the conversion and crushing blacks just a bit. With DV encoded discs, a brightness of 51 causes the blacks to glow and I need to lower to the normal setting of 50.

I don't know how well the forced DV output works on the OPPO, it is not something I personally use.

Soon I will be able to compare HDR10 vs. HDR10 as Dolby Vision vs. Dolby Vision on the OPPO. Hopefully in a few weeks.

The 6 series has a really small LUT and is really limited in what it can do or the quality you can get from it. On the C9 and CX you can now calibrate the bottom end using Calman. This needs to be done after the autocal as both are crushed by default. They will be providing a download link for our Dynamic Range Low pattern in SDR, HDR10 and Dolby Vision soon'ish.

[wxman2003]

Originally Posted by Stacey Spears

I don't know how well the forced DV output works on the OPPO, it is not something I personally use.

Soon I will be able to compare HDR10 vs. HDR10 as Dolby Vision vs. Dolby Vision on the OPPO. Hopefully in a few weeks.

The 6 series has a really small LUT and is really limited in what it can do or the quality you can get from it. On the C9 and CX you can now calibrate the bottom end using Calman. This needs to be done after the autocal as both are crushed by default. They will be providing a download link for our Dynamic Range Low pattern in SDR, HDR10 and Dolby Vision soon'ish.

The reason why I do that is because HDR10 does not look good on the 2016 E6. However, forcing HDR10 to play as DV via the OPPO 203 looks much better on my E6 due to the tone mapping. So I always force the OPPO 203 to play HDR10 as DV. Oddly enough, there is no black crush when converting HDR10 to DV on my E6, but there is some black crush with DV encoded discs.

[Stacey Spears]

Originally Posted by wxman2003

The reason why I do that is because HDR10 does not look good on the 2016 E6. However, forcing HDR10 to play as DV via the OPPO 203 looks much better on my E6 due to the tone mapping. So I always force the OPPO 203 to play HDR10 as DV. Oddly enough, there is no black crush when converting HDR10 to DV on my E6, but there is some black crush with DV encoded discs.

Thank you for the notes. I will do some testing once I have a build of the disc with the right patterns on it.

I was curious about the Sony since it forces everything to Dolby Vision when enabled, specifically to use Dolby's tone mapping. I had forgotten OPPO has that ability.

Dolby also has VS10, which is their tone mapping designed for HDR10 content. VISIO had it at one point, not sure they still do. This was back in 2016/2017.

[New2019]

Hello,
i just bought an epson 3800 4K player and planning to buy Pana 420 player.
This calibration disc looks interesting. But is this something a lay man like me can do myself? or need some expertise?
If its a simple self explanatory, i can just buy this calibration disc.
Please let me know

[Stacey Spears]

Originally Posted by New2019

Hello,
i just bought an epson 3800 4K player and planning to buy Pana 420 player.
This calibration disc looks interesting. But is this something a lay man like me can do myself? or need some expertise?
If its a simple self explanatory, i can just buy this calibration disc.
Please let me know

The only documentation is on our website. It only covers the very basic video setup patterns. I would read through all of the the articles under "Ultra HD Benchmark" and the beginners guide under downloads first.

[Sledgehamma]

Originally Posted by Stacey Spears

Dolby also has VS10, which is their tone mapping designed for HDR10 content. VISIO had it at one point, not sure they still do. This was back in 2016/2017.

The VS10 engine is being used in the Realtek RTD1619DR. It can also use the VS10 engine on SDR so that everything gets outputted as HDR10 or DV (depending on your display).
It also improves the problems on HDR chroma upsampling with the Realtek decoder. So it might be more than just tone mapping, but I'm no expert.

If you are interested please have a look at my review where I used your excellent patterns:
https://www.avsforum.com/threads/rev...o-z9x.3170473/

If you want to know something specific, feel free to ask.

Originally Posted by Stacey Spears

We also developed a new pattern for Chroma Upsamping Evaluation. Mathias, from MadVR, helped inspire it.


The above is a copy / paste from emails I have sent around to various people interested in chroma upsampling. I keep fixing obvious errors in the text above as I find them.

Quite awesome test patterns! Would it be possible to send them to me as well?

Originally Posted by Stacey Spears

Every time I am near a magazine rack, I look for AV magazines and can't find any. I miss the days of receiving Widescreen Review in the mail.

One of our goals, with the videos, is to explain how to use the patterns to review products. We won't be doing reviews ourselves though, mostly do to lack of time.

Do you know the reason for the lack of those patterns on reviews? I really think its a shame and I have zero trust in reviews which just describe that something is sharper, more vivid etc. without having actual proof.

[Stacey Spears]

Originally Posted by Sledgehamma

Quite awesome test patterns! Would it be possible to send them to me as well?

HDR or SDR? I have the various HDR versions encoded. I rendered SDR over the weekend, but won't have it encoded for another month at least as I am primarily focused on rendering all of the content first and then I will encode after.

Feel free to PM me.

[Stacey Spears]

Some progress has been made on the authoring side. Here are some sample images of the pop-up help. Some have images and some don't. (More don't) You can ignore that the correct and incorrect both look the same, that was the iPhone capture that made them look the same.

Also, you can only change nit levels for HDR10. Dolby Vision and HDR10+ are always using the 10,000 cd/m² version of the patterns since they use dynamic metadata.

Up arrow shows the nit menu and down arrow shows the help screen.

[Pagey123]

Just a quick shout out to Stacey for continuing to not only develop awesome calibration tools for both professional and "prosumer" types, but for also staying actively engaged in the community. It's rare, in my experience, to find that wonderful combination of technical expertise, renown, and humbleness. Cheers, mang!

[Sledgehamma]

Originally Posted by Stacey Spears

Also, you can only change nit levels for HDR10. Dolby Vision and HDR10+ are always using the 10,000 cd/m² version of the patterns since they use dynamic metadata.

Looks great to me. This might be a dumb question as its early in the morning:
While DV and HDR10+ youse dynamic meta data couldn't you alter the peak brightness in accordance the HDR10 pattern?
When always using the 10.000 nits version it would imply that DV and HDR10+ tone mapping would be identical if your test pattern has a peak of 1.000, 4.000 or 10.000 nits, wouldn't it? I find that hard to believe but maybe I'm missing something obvious here.

Originally Posted by Pagey123

Just a quick shout out to Stacey for continuing to not only develop awesome calibration tools for both professional and "prosumer" types, but for also staying actively engaged in the community. It's rare, in my experience, to find that wonderful combination of technical expertise, renown, and humbleness. Cheers, mang!

Well put. I totally agree with you!

[Stacey Spears]

Originally Posted by Sledgehamma

While DV and HDR10+ youse dynamic meta data couldn't you alter the peak brightness in accordance the HDR10 pattern? When always using the 10.000 nits version it would imply that DV and HDR10+ tone mapping would be identical if your test pattern has a peak of 1.000, 4.000 or 10.000 nits, wouldn't it? I find that hard to believe but maybe I'm missing something obvious here.

Not sure what you mean by identical, can you clarify? All Dolby Vision displays are identical and all HDR10+ Profile B displays are identical. But Dolby vs. HDR10+ are not identical. Hope that makes sense.

This is difficult to discuss in a forum vs. in person, but I will do my best. Might even make for a good topic in one of our planned videos.

The reason we have different nit levels for HDR10 is because there is no standard for HDR10 tone mapping and the content can come in with different static values for Mastering Display Max Luminance and for MaxCLL, which is the primary thing we change. Some patterns do have a limit on peak luminance as well (Contrast / DR High, resolution) but others like ramps, contrast ratio and window patterns are identical at all nit levels.

Dolby and HDR10+ don't use the static metadata. While it is in the bitstream for HDR10 compatibility, they each use dynamic metadata that is calculated on a per shot basis, so there really is no concept of HDR10 style static metadata.

So we could have a 4000 and 10,000 Dolby contrast pattern, but there is not static metadata to change with them.

I should mention that for Dolby, we actually use static metadata. We apply the same metadata if the window pattern is 0% or 100%. This was what we agreed to with Dolby back in 2014 while working with them on how to correctly measure, profile and calibrate a Dolby Vision display. So you can say there is a standard on how metadata should be with test patterns in Dolby Vision and we follow that standard. We use the same metadata as Calman. We even offer the window patterns in perceptual, absolute and relative modes like Calman does when used with the VideoForge Pro or Murideo Six-G.

There is only one encode of the window patterns. For HDR10, we made a copy of the encoded file and changed the metadata. For Dolby, there is no metadata to change.

The primary reason we had different peak luminance was for contrast. We did not want people to freak-out asking how they can get all the boxes up to 940 to be visible, when they can't, so we limit the peak box to the code value that corresponds to the peak luminance level. e.g. 1000 cd/m² is code value 723 in 10-bit limited range, so the highest box is 722 and the background is 723. If you see it, you know you are good. Dolby does not have this issue, will always show up to 940.

Since you specifically mention tone mapping. Our tone mapping pattern is just a single pattern for all HDR10 nit levels. It always goes to 10,000 on the pattern. This way you can see what the display does when the metadata changes since it has to be a single value.

The image on some HDR displays (e.g. LG B7) gets darker the higher the HDR10 static values go. If you look at the sharpness pattern, on the current disc, on the B7 and switch between 1000 and 10,000, the image gets very dark at 10,000. The pattern itself is identical, only the static metadata changes. In the case of Dolby and 10+, there is no static metadata to change.

I would have loved to just have one set of HDR10 too, if it was only a standard.

In the specialty section, the DR high pattern is the same at all HDR10 nit levels vs. the normal pattern that changes in HDR10. And since these are specialty, they also go up to 1019 (head room) and the DR low goes down to 4 (toe room). We also added a 0 nit option for these to see what a display does when set to 0 like Fox titles have used.

The DR low on the current disc goes down to 4 as well, but we changed the normal DR low to stop at 64 on the new disc and the specialty goes down to 4.

The specialty are not tied to the normal nit menu. It has its own nit menu because of the 0 nit option. It also has Dolby absolute mode to turn off tone mapping all together and see where things really clip.

To summarize, Dolby defined the metadata specifically for test patterns to calibrate Dolby Vision and it is what we use. HDR10 has no standard of any kind, for anything, so we have to offer different options for evaluation. We usually recommend you just use 1000 for calibration though.

I am sure my rambling is as clear as mud!

One pattern I am excited about is the hue shift. That was developed with a lot of input from a forum member. Dolby crushes this and thanks to said forum member, so does the Lumagen. HDR10 on most displays have some room for improvement. And you can see this on our updated montage as well in the peacock feather, quantum dot vials and crane next to the Space Needle.

[PeterTHX]

Originally Posted by Stacey Spears

One pattern I am excited about is the hue shift. That was developed with a lot of input from a forum member. Dolby crushes this and thanks to said forum member, so does the Lumagen.

This a good thing or bad thing?
Like bad - black crush.
Good - "I crushed it!"

[Stacey Spears]

Originally Posted by PeterTHX

This a good thing or bad thing?
Like bad - black crush.
Good - "I crushed it!"

Sorry for the ambiguity. Its a good thing for Dolby, bad for HDR10.

The Dolby crush has been worked around in Calman. And then when you also perform the shadow detail adjustment for SDR, HDR10 and Dolby, using Calman, they should perform similar down to 65. At least on LG.

The above assumes we are talking about the same thing in terms of black crush.

[Sledgehamma]

Originally Posted by Stacey Spears

Not sure what you mean by identical, can you clarify? All Dolby Vision displays are identical and all HDR10+ Profile B displays are identical. But Dolby vs. HDR10+ are not identical. Hope that makes sense.

This is difficult to discuss in a forum vs. in person, but I will do my best. Might even make for a good topic in one of our planned videos.

Agreed! Thanks for trying to explain it. Let me try the same:

Originally Posted by Stacey Spears

There is only one encode of the window patterns. For HDR10, we made a copy of the encoded file and changed the metadata. For Dolby, there is no metadata to change.

If I understand you correctly the video is identical but just the meta data changes. As DV has no meta data it wouldn't make sense to make a similar DV pattern for it. I agree.
What about changing the actual brightness of the video in different nit terms. This way the Dolby engine has to apply tone mapping and due to differences in brightness in each in each video, the resulting image will look different. Maybe darker, maybe highlights get clipped (I don't know the philosophy of DV's tone mapping).
This would show how the DV tone mapping is behaving.
I think this would be interesting because, although DV being a standard and it should look the same, there are still differences from player to player (Oppo 203 behaves differently than Panasonic UB9000 for example).
But then again, it might just be a useless idea.

[Stacey Spears]

Originally Posted by Sledgehamma

I think this would be interesting because, although DV being a standard and it should look the same, there are still differences from player to player (Oppo 203 behaves differently than Panasonic UB9000 for example).

If there are differences between the OPPO and Panasonic with Dolby Vision, it is most likely due to a bug in the player.

We found a possible bug with the OPPO and Dolby Vision with the first test image David delivered last week. When you look at the color space eval pattern while in Dolby Vision mode, the top two PQ tracking boxes blend correctly on the LG. When you switch to HDR10 and back to Dolby Vision on our disc, they no longer blend. You have to power cycle the OPPO to fix it. I tried stopping and restarting the disc, no luck. I tried power cycling the display. No luck. Tried changing inputs, no luck. This might be causing the difference you are seeing between OPPO and Panasonic.

I was really confused by the bug because I had thought it blended, but then it was not blending in Dolby but was in HDR10. The next time I played the disc it was correct in Dolby and that is when I started digging into it and figured out when it would stop displaying correctly. I think it also impacted the brightness pattern as well, but will spend more time testing once we have the menu in place with background videos to make it easier.

You may have to power cycle the OPPO before playing a Dolby Vision disc if you had played HDR10 before hand with the player on. If a disc has an HDR10 menu and not Dolby Vision, then you may be out of luck.

I will need to test player vs. display-led in this context as well.

I will know more once I burn a disc to test on the Panasonic to confirm it is OPPO specific and not a disc issue. Since re-loading does not solve the issue, it makes me think it is not the disc, but that is what testing is for.

[PeterTHX]

Originally Posted by Stacey Spears

You may have to power cycle the OPPO before playing a Dolby Vision disc if you had played HDR10 before hand with the player on. If a disc has an HDR10 menu and not Dolby Vision, then you may be out of luck.

Are there discs like that? Most DV discs I've seen are end-to-end DV from the initial warning screen on disc load to the multiple multi-lingual copyright warning screens at the end of a movie.

[Stacey Spears]

Originally Posted by PeterTHX

Are there discs like that? Most DV discs I've seen are end-to-end DV from the initial warning screen on disc load to the multiple multi-lingual copyright warning screens at the end of a movie.

No idea, but nothing stopping someone from making one.

[Stacey Spears]

Confirmed the issue is an OPPO bug and does not occur on Panasonic.

However, at this point in time, I think Dolby on BD is a real disappointment.
1. They appear to be using 709, instead of 2020, upsampling so there is a half pixel vertical delay between Y and C. Verified this on both OPPO and Panasonic.
2. The 4:2:2 to 4:4:4 conversion in the Dolby engine, on the display side, is very low quality and lacks chroma resolution compared to HDR10 on the same display.
3. Dolby has chroma ringing in their upsampling, similar to Panasonic. You can see it come and go on the OPPO when you switch between Dolby Vision and HDR10.

Maybe I will invest in a Lumagen or MadVR Envy and use HDR10 on all Dolby titles.

I also tested player vs. tv-led. Player-led has the same issues where black is crushed more and the top two boxes on the CSE pattern don't blend, all the time! At first I thought the OPPO bug might simply be playing in player-led, but that was not the case as player-led has some other differences as well.

I did find that if I let a pattern play until the player loops it, it actually resolves the bug on OPPO. Not sure why it does but nothing else but a power cycle does.

[LordoftheRings]

Awesome work you do man!

[David M]

Originally Posted by PeterTHX

Are there discs like that? Most DV discs I've seen are end-to-end DV from the initial warning screen on disc load to the multiple multi-lingual copyright warning screens at the end of a movie.

Are you sure about that? Keep in mind that some Panasonic players detect if the main feature on the disc is in DoVi (possibly by searching for a playlist with associated DoVi enhancement that's longer than all the others and comparing running times), and if detects this, outputs the entire disc in DoVi - even the stuff encoded in SDR.

It probably is done based on length, because we had the DoVi montage on the first version of Spears & Munsil UHD and this didn't cause this behavior on the Panasonic.