Spears & Munsil 4K UHD HDR Benchmark (Test Discs) May 15, 2023

[BillyDuffyLove77]

Sorry guys... do we know when this disc is going to be available and where best to get it (I'm in the UK)??? Thank you!

[Sledgehamma]

Already available. You can order it here:
https://www.biaslighting.co.uk/colle...benchmark-2023

[BillyDuffyLove77]

Originally Posted by Sledgehamma

Already available. You can order it here:
https://www.biaslighting.co.uk/colle...benchmark-2023

Thank you kind sir

[Stacey Spears]

Originally Posted by batman2000

Is there a digital version?

Currently there are no plans for a USB or digital download. There are over 5000 files on the disc and re-encoding them for a digital version is an enormous amount of work.

The static patterns, on disc, are encoded as 9-frames to get the best quality. The BD player pauses on the last frame. For a digital version, we would need to make them around 1440 frames so they would play as running video, so we can't simply re-use the existing encodes.

Our Dolby Vision files are Profile 7. We would need to make them 8.1, so those would all need to be redone.

Some devices will maintain the folder structure we put them in, to organize. Others remove that and put everything into a single menu. When you have this many files, you want them broken into groups similar to the menu system on disc.

We did run into issues with at least one display that could not play some of the files we encoded. We could re-encode and it would work the next time.

There are others that offer files for download, such as Diversified Video Solutions, so that would be your best bet.

We can revisit in the future, but right now both of us are busy on other projects.

We may do a very small subset for another project.

[HDTV1080P]

Streaming and digital downloads are lower quality unless they are bit for bit 4K Blu-ray images. Only the Kaleidescape system is capable of downloading 100GB 4K Blu-ray images that are bit for bit the same as the optical disc. But the download would need to be three separate files of 2 BD-100’s and 1 BD-66. Maybe one day Spears and Munsil will be offered on the Kaleidescape system. But personally I prefer the 4K Blu-ray disc version since there exists over 100 million Blu-ray devices if one counts 4K videogame consoles versus maybe only under 100,000 Kaleidescape systems (perhaps under 10,000 since the company has not released the numbers). But the problem is the Kaleidescape system does not support Dolby Vision HDR or HDR10+ on their current models (maybe that will change with a firmware update or new hardware).

https://forum.blu-ray.com/showthread.php?t=345757

[David M]

Originally Posted by batman2000

Is there a digital version?

A lot of people ask this, and it's a good opportunity to talk about the actual authoring side. Blu-ray (and by extension UHD BD) give us a good framework for controlling things necessary to make a disc like this (HDMV - we didn't use any Java on the disc, but that would have been an option too). For example, which patterns pause after one frame, which ones loop, keeping track of the position on the disc to allow jumping between, custom menus, popup help, etc etc.

That's custom code we can write just for a project like this and easily put it on a Blu-ray Disc, and be confident it'll work in basically the same way across all licensed BD players. As far as I know, streaming services don't have as much of this, they're designed for a much less interactive "start playing the movie and allow the user to pause and rewind" scenario. (If anyone from said streaming services thinks this is incorrect, feel free to correct me).

And that's to say nothing of the video side, the most critical part. On the physical formats, the video we (Stacey and Don) encode is the video that goes onto the disc, untouched. Streaming services encode many different versions of the same content for various different devices from a mezzanine file. While I trust most of them will be getting the color science right, it's still being re-encoded. Stacey and Don would then have to test a ton of different versions - and hope none of them changed the next time the service did a refresh. Whereas, what's on the disc is one perfect version that's on the disc forever. There are a lot more moving parts with internet delivery.

Plus... network latency might not be a big deal for choosing one film and watching it. But it could make navigating an entire suite of patterns like this tedious.

Edit: Stacey beat me to the reply before I refreshed the page!

[batman2000]

The reason I asked is I don’t have a 4k blu ray player and I can’t get one currently to use the disc. Is there online software for calibrating hdr and Dolby vision?

[Stacey Spears]

Originally Posted by batman2000

The reason I asked is I don’t have a 4k blu ray player and I can’t get one currently to use the disc. Is there online software for calibrating hdr and Dolby vision?

Which display are you using? It might be best to hire someone that can come out and perform the full calibration for you.

We may work with a 3rd party for an ATV solution, but that is a long way out.

[INdetectableMAN]

Good morning Stacey.

I wonder if you would be so kind as to provide the luminance data (Y) of the light skin and dark skin frames from the colorchecker table of the model pass, would it be possible, could you provide it?

Thank you so much.

[Stacey Spears]

Here is a .7z.
https://www.dropbox.com/scl/fi/kssjm...8fl4fqygf&dl=1

The format of the files included in the .7z are HDR BT.2020.

1. Original 7680x4320 16-bit R'G'B' full range BT.2020 TIFF.
2. 3840x2160 10-bit Y'CbCr 4:2:0 limited range BT.2020 YUV. (This is an example frame that was fed into the encoder.)
3. Video Conversion spreadsheet.

• 10-bit Color conversion between Y'CbCr and R'G'B'. (Tab - ColorConversion 10-bit)
• HDR levels conversion (Tab - LevelConversion (HDR ) Limited) - Simple conversion between nit level and code value. Code value is assumed to be white. It does not support converting a triplet into a nit level, which would be useful and might be what you are asking for. You have the data to calculate the real Luminance value.
• Backlight Resolution Calculator (Tab - Backlight Resolution (PCA)) - Clearly no one has used these tests on the disc as no one has asked for the spreadsheet to calculate the end result. I believe this would have provided some useful data during the Value Electronics TV Shootout along with Peak Luminance. PCA is a single number that provides an objective quality metric. Peak Luminance, on the other hand, is a lot more data that has to be interpreted and I can understand why it was not included in this years TV Shootout. Neither test is something a judge would look at and subjectively evaluate. Classy Tech Calibrations has done a wonderful job with the Peak Luminance pattern. I have not seen anyone else utilize Peak Luminance yet. The work Classy has done reminds me of the work we did 20+ years ago with the Progressive Scan shootout over at Secrets.

4. Image of the 10-bit Y'CbCr 4:2:0 limited range .yuv values for the first chip.


5. Image of the 10-bit Y'CbCr 4:2:0 limited range .yuv values for the second chip.


The white cursor on the chip is the 16x16 (Macrobock) area being shown in the dialog box to the left with the Y'CbCr values. Sorry the crops are not identical. I used the Windows snip tool to crop the frames.

For those that might wonder why the images I posted are desaturated and flat, compared to the images posted by INdetectableMAN, that is because you are viewing an HDR image in SDR mode. These are not photos taken of an HDR display but rather a screen capture of our image analysis tool that is running in SDR mode. It is what HDR looks like when viewed in gamma.

[DisplayCalNoob]

@Stacey Spears I have an acknowledgement and a realization. First is acknowledging how wrong I was about HDMI chroma subsampling when paired ×ith Dolby Vision. Second, realizing that the disc can, help identify HDMI cables that aren't manufactured to spec. Chroma errors, posterization, color inaccuracies, Luminance issues, and black crush.

[Pagey123]

Just a quick Thanksgiving shout out to both Stacey Spears and David M., who always goes above and beyond to interact with members and answer questions completely! You guys are credits to your professions!

[INdetectableMAN]

I currently have a Samsung QD OLED, since I can't stand the anomalies produced by the white subpixel in WRGB, but I have discovered that my instruments, an i1Pro2 and an i1Display Plus, do not get along very well with QD OLED and in general with screens which have relatively narrow bandwidth as they report incorrect data so I end up with inaccurate images so I guess I trust my eyes more.

Therefore, I am using and testing the Spears & Munsil color temperature test patter, which to me looks like ground gold, beautiful.

And there comes the question. When I perform a visual inspection in a dark environment using this test pattern, my eye tells me that using the default white balance setting I am at or very close to D65.

Is there any additional element, such as a backlit optical comparator, since if it is not, I have seen that it is quite useless, that can help me when verifying with the test patter? Or can I trust that just with the test patter I could get close enough to D65?

[DMDreview]

Originally Posted by Stacey Spears

Someone wanted to see an artifact I had mentioned on OPPO, so I took some cell phone photos. Not the best quality, but they do show the issue.


OPPO was set to UHD output, so the player scaled the image. These are from the HDR10 Scaling HD pattern to make the artifact easier to see. This issue is also visible on the Sony player with MediaTek. I don't recall if it is present in Dolby Vision either on MediaTek.

I don't recall if it comes and goes or is only with 4:2:2 output. Ignore the color moire in the middle, that was from the capture of the image with my cell phone.

I confirm, I made the capture in 444 mode, the defect is there, while media players different (fire tv max, Shield tv and others )behave normally, qualitatively producing upscale chroma at the same time with upscale 1080 to 4k. Moreover, both 422 and 444 on the capture card when capturing in 444, produces the same defect.

In HDR when upscaling Oppo has an unpleasant defect of chroma, because the nearest neighbor is used for interpolation, while in DV there is no such defect, because linear interpolation for chroma is used inside DV processing.
So oppo is not the best option if you need to upscale an image from 1080 to 4k. With normal upscaling of only chroma from 420 to 422 or 444 in 1080p, there is no defect.

[aboulfad69]

Would anyone care to comment on the two attached photos using the color conversion test pattern through two devices? What aspect of that test pattern would most severely impact colours: color space conversion & clipping in the bottom middle? & how would it impact color rendition/reproduction?

There’s this excellent write-up from S&M, sadly given my limited knowledge, it’s still hard to understand fully.

Device1

Device2

[Kirk Out]

Hello I was checking the d65 pattern on my lg cx that was recently calibrated on FMM. Without DTM the last four boxes of the pattern clip when but when turned in they reappear, I check cinema with out calibration and the boxes don’t clip , I can see all the boxes of d65 with DTM and without out . Is my calibration wrong or am I reading this pattern wrong? The top is slight orange and bottom pattern is slight blue like in the info box describes. Any help thanks guys

[Geoff D]

Originally Posted by aboulfad69

Would anyone care to comment on the two attached photos using the color conversion test pattern through two devices? What aspect of that test pattern would most severely impact colours: color space conversion & clipping in the bottom middle? & how would it impact color rendition/reproduction?

There’s this excellent write-up from S&M, sadly given my limited knowledge, it’s still hard to understand fully.

Device1

Device2

Device 2 is more or less what it should look like. Device 1 is messing it up big time.

[aboulfad69]

Originally Posted by Geoff D

Device 2 is more or less what it should look like. Device 1 is messing it up big time.

Thank you, I gathered as much from reading the S&M article. Device 1 is an AppleTV (ATV) 4K gen3, device 2 is Oppo UDP-203, both devices were using the same DV test pattern.

The difficulty I have to my untrained eye, is discerning difference(s) in image color reproduction between both devices when viewing a DV movie. For movie comparison, Oppo is playing a physical DV movie (P7 MEL), ATV through Infuse is streaming the ripped & converted P7 movie to P8 (a lossless conversion).

I know or feel that Oppo’s image quality is better, I just can’t identify the differences. The latter being evident with the test pattern (one being Y’CbCr clipping).

[mrtickleuk]

Originally Posted by aboulfad69

Thank you, I gathered as much from reading the S&M article. Device 1 is an AppleTV (ATV) 4K gen3, device 2 is Oppo UDP-203, both devices were using the same DV test pattern.

The difficulty I have to my untrained eye, is discerning difference(s) in image color reproduction between both devices when viewing a DV movie. For movie comparison, Oppo is playing a physical DV movie (P7 MEL), ATV through Infuse is streaming the ripped & converted P7 movie to P8 (a lossless conversion).

I know or feel that Oppo’s image quality is better, I just can’t identify the differences. The latter being evident with the test pattern (one being Y’CbCr clipping).

(my emphasis above)
In general, this is why we use test patterns; they are designed to tease out all the issues. Then once you've used that, you can make the setup changes required and forget about it, relax and enjoy your movies where the differences can be infinitesimally small and may only happen in a few seconds of the movie.
You aren't supposed to use the test patterns as "training" to give you the ability to always be able to see these tiny things in real content. That would drive you mad, and you'd never enjoy anything!

[aboulfad69]

Originally Posted by mrtickleuk

(my emphasis above)
… you can make the setup changes required and forget about it, …

Thanks, given that the same identical DV settings are used on the TV for both devices, it seems to indicate there’s something funky being done in the Apple TV that I don’t think we (end users) may have control over.

I still watch physical movies using my UDP-203 at least once, and then for convenience occasionally use ATV/Infuse with my ripped BD.