Spears & Munsil 4K UHD HDR Benchmark (Test Discs) May 15, 2023

[p5browne]

Any new news re new disc ? Has the pandemic killed the current upgrade?

Stay healthy, stay safe.

[gkolb]

Originally Posted by p5browne

Any new news re new disc ? Has the pandemic killed the current upgrade?

Stay healthy, stay safe.

Yup, I’m curious about when new disc could be coming.

[Stacey Spears]

Originally Posted by p5browne

Any new news re new disc ? Has the pandemic killed the current upgrade?

Here is where we are as of today.

I started encoding all of the HDR content the first week of September. I have had to restart twice. Once for an encoder improvement and once for a bad setting.

One of the parts that is slowing encoding down is that I am encoding every test pattern 96 times before the final encode. This is a brute force encode to find the best settings. On the previous discs I used the highest / slowest encoding mode, assuming it would be the best. Turns out it is not, at least for test patterns. I ran into this over the summer when the next mode down (faster) was mathematically closer to the source. Visually, all 96 look the same. It is at the pixel level is where I am trying to make them closer.

The resolution menu took 13 days for the brute force and then three hours for the final encode. That menu has three pattern types: multiburst, wedge and zone plate. There are 18 patterns for each type. Multiply that by six nit levels. And multiply that by 96 encodes. The result was 31,104 encodes! LOL

Yesterday I started encoding the Windows section of the disc. I figure this will take about a week and a half to complete.

The final section are the motion patterns. Those may take a while as they are all one minute+ long. To give you an idea:
1. Dynamic Range High takes one and a half hours to encode.
2. Clipping takes three hours to encode.
3. Pixel Aging takes 24 hours to encode.

I am hoping the motion does not take more than 3-4 weeks to encode. I can provide an update around the start of December.

I am still missing:
1. New montage audio in Atmos. Early December is the best guess.
2. DTS:X tones. COVID has prevented this thus far.
3. Videos for the audio panning. Brian is trying to figure out the best way to handle this.

Once all of the HDR content is finished, I will know how much disc space it will consume and will have an idea if SDR patterns will fit on the HDR disc or if they need to be on the 2nd disc. Then I will render and encode the SDR patterns. There are 1/6th the number of SDR patterns, so it should be done before Christmas.

The new hand graded SDR montage is now encoded. Waiting on something for the HDR montage and then I will process and encode them. These run in parallel with the patterns.

To give you an idea on disc space. The current disc spends 14 GB on the patterns and the rest goes to the montage videos. The patterns will probably take 40-60 GB, if not more, on this disc. There are 75 videos just for levels and bass management in the audio section. This is almost 10 GB and there is just Dolby Vision / HDR10. I won't be including them at different nit levels.

We should have the pop-up help text completed by next week and then we will work on the printed manual. It is the beginners guide, the getting started guide and a summary of each section all merged into a super doc.

Not every pattern will have help text. e.g. None of the window patterns will have any help text. These require a meter and software, so you are expected to know how to use them.

With all that said, late Q1 is best guess right now, but I can provide an update after the New Year. We hope to start authoring in December, but the holidays will slow it down a bit.

[p5browne]

Starting to sound like a 2 disc set.

[panasonicst60]

Sounds great Stacey Spears! I'm ready to support your hard work and dedication when it's done. Thank you!

[morris_schaffer]

To clear up the confusion for myself, can I use this disc to the fullest even without measuring equipment? And with measuring equipment I do not mean pen and paper.

and would this UHD HDR benchmark disc also help in calibrating a full HD image or would I need to buy the previous calibration disc as well?

Lastly, would it also help in calibrating a 3d image (for movies) or are such measurements not included?

Thanks!

[Stacey Spears]

Originally Posted by morris_schaffer

To clear up the confusion for myself, can I use this disc to the fullest even without measuring equipment? And with measuring equipment I do not mean pen and paper.

and would this UHD HDR benchmark disc also help in calibrating a full HD image or would I need to buy the previous calibration disc as well?

Lastly, would it also help in calibrating a 3d image (for movies) or are such measurements not included?

Thanks!

Ultra HD Blu-ray does not support 3D, which is why 3D content is not included on the HDR disc. 3D is on our 2nd edition disc, which is also HD.

The current UHD disc is focused on HDR. The upcoming disc will have most of the patterns in 709 SDR as well. You really don't want to convert HDR patterns to SDR for any type of calibration or evaluation.

The majority of content on our discs is about evaluation, which is why we call it a benchmark disc and not a calibration disc. Basically we are making the disc we wish we had when we were reviewing products. We want display manufacturers and reviewers to have the best possible tools to evaluate a displays performance. If customers have the same tools, they can also be a loud voice to push for improvements in products.

To set grayscale, CMS or any type of LUT, you need a colorimeter and software along with patterns. For basics like brightness, contrast, color, tint and sharpness, you can do it by eye with a test pattern. Backlight requires a light meter. For setting audio levels, you need an SPL meter.

[Stacey Spears]

Originally Posted by p5browne

Starting to sound like a 2 disc set.

We have been assuming two discs for a while. We just don't know the final layout until we know how much disc space everything takes up. If we have SDR on one disc, it will probably also include some of the HDR demo material.

I will provide a progress update the first week of December.

[morris_schaffer]

Thanks Stacey!

[Stacey Spears]

Ultra HD Blu-ray does not support 3D, which is why 3D content is not included on the HDR disc.

I do want to clarify my comment. There are three different specs for HD 2D, HD 3D and UHD Blu-ray. Each one actually requires a different license. We currently pay for 3D and UHD. Of course a 3D disc can have 2D content on it.

A UHD player can support all of them, if the player manufacturer so desires, but a disc is limited to the format you author for.

Another difference is that 3D content can only be 23.976 and not 24. 2D HD and UHD both support 23.976 and 24.

Also, UHD cannot have 480i, 480p, 720p or 1080i. It is limited to 1080p and 2160p. I mention this as we really wanted to include 480p and 720p versions of the scaling pattern on disc, but no such luck.

[Stacey Spears]

The Windows section finished over the weekend. Currently re-rendering the Motion Interpolation, Sarah on a Hammock and Ticker clips. Ticker is waiting on final montage.

Here is an image of the HDDs I have purchased to create the new disc. The latest is a 2nd 16 TB drive over the weekend to put the SDR assets on, once rendered.

[sperezmore]

Thanks, can't wait for the release date.

Regards,

[Stacey Spears]

Sorry I have not provided an update. I decided to shut off all computers, Internet and news for a week in December. Just focused on movies and TV shows.

This week we delivered all of the encoded assets for disc 1. It actually goes over the 100 GB a bit once muxed into m2ts, so we will have to trim a bit when we hear from authoring. Hopefully most of it muxes first time around. Sometimes we have to re-encode in order for it to mux.

The current disc on the market uses ~89 GB and has 1400+ files. Disc 1 is at 95.7 GB and 4515 files. The audio tones (pink noise) and montages did not make it on disc 1. It is all test patterns, including skin tone content.

The audio tones will go onto disc 2, which is the SDR disc. The montages will either go on disc 2 or a disc 3.

I started rendering the 75 videos for the audio tones. Each one is taking 7 hours to render from AE, so ~22 days to render them. Then I will render the SDR test patterns. This is being done while disc 1 is being authored.

Perhaps in a few weeks we will have an early build of the disc to start testing internally.

Authoring started on the current disc in February and was released in June of 2019. That disc, as complex as it was, was less complicated than the new disc. We are going to figure out how to make the pop-up help work with the addition of an HDR10 350 nit based on feedback from Kris Deering. He wanted 350 for projectors. Then Dolby Vision for everything as well.

We had to cut DTS:X. They are unable to help us due to COVID. The audio for the montage has been composed and is in two parts. The first half is more majestic and the 2nd half is more electronic. The transition between the two occurs between the tulips and pocket watch. The Atmos mix should happen in the next few weeks.

We will provide a spreadsheet for some of the measurements on our website once it launches.

Once the disc is authored, we will start working on videos. We may pick a subject and create a video around it. e.g. We want to make a video for luminance loading. At least show how we had envisioned it being used. We will take requests on videos you would like to see. I will be turning my spare room into a little studio for the next year.

The AV Sync will be available in Atmos, TrueHD and DTS-HD MA. Was really hoping for at least DTS:X on this one, but no luck. The AV Sync is back in the center channel and will be in HD, UHD, 23.976 and 59.94. This should allow you to account for different frame rates and latency introduced by scaling. We made some other tweaks to the Sync-One2 pattern based on feedback from them to deal with frame rate conversion in a TV. Also added a version for calibrating the device to your room.

The disc has a few patterns at 59.94. Some players drop to 4:2:0 when its 59.94, so we needed patterns for that. Some also drop to 8-bit, so same thing. All but one 59.94 exists as 23.976. The one that does not is Pixel Aging and that is because it works better at the higher frame rate. To avoid using 60 vs. 59.94, we just call it HFR.

Still trying to decide on the final name. Thoughts on:
Ultra HD Benchmark Pro
Ultra HD Benchmark Ultimate

Some tag lines we decided not to use:
More WOW than Disney
Even more essential than Digital Video Essentials
More BRAVIA than AVIA
More X than THX


My favorite new patterns include Pixel Aging and Peak Luminance. There are little details that make the Pixel Aging work and encode cleanly. Peak Luminance, I hope, will lead to more accurate nit numbers being reported on TVs.

Like the current disc, there will be a hidden menu. This time, it will also include one specific test pattern. We hid it because it is going to ruffle some feathers. It is related to a recent video that Vincent, from HDTVTest, posted on his channel.

Does everyone know where the current hidden menu is? Its just a page that reports player and display capability based on what the player detects.

[Geoff D]

Benchmark Pro sounds betterer. It puts the ass in classier. Ultimate would indicate there's no improvements to be made, but that's rarely the case for something labelled as 'Ultimate'. Not to knock what you've done, this disc is shaping up to be a stunner, but with the lack of DTS:X alone some would say it's less than Ultimate.

Can't wait to get a look see!

[Stacey Spears]

Originally Posted by Geoff D

Benchmark Pro sounds betterer. It puts the ass in classier. Ultimate would indicate there's no improvements to be made, but that's rarely the case for something labelled as 'Ultimate'. Not to knock what you've done, this disc is shaping up to be a stunner, but with the lack of DTS:X alone some would say it's less than Ultimate.

Can't wait to get a look see!

Thank you for the quick feedback Geoff!

DTS has offered some pre-made content. Just waiting for a link to review it. It might be similar to the channel ID on their demo disc. I am undecided.

We did receive a trailer from Dolby as well, but the source does not match the encoded sample they provided. Some blacks are elevated and the HDR looks like SDR in the source.

In the end, I took a three week break from thinking about the disc.

We did improve many of the existing patterns. Some are subtle, like anti-aliasing. e.g. The Color and Tint pattern had the wrong anti-aliasing on the text. Not important for the pattern, but is a detail we care about.

Some new patterns, like the Hue Shift were done with testing from a customer. In fact, their feedback changed/improved the ADL patterns and added the Aspect Ratio patterns.

We added another element to the Scaling pattern based on feedback in this thread. Might have been you Geoff. Its all in my notes when I start working on the credits video. We added a section from the zone plate to show moire when you enable the anti-ringing filter in the Panasonic. Maybe its called Edge, I forget.

We redesigned the Color Temp pattern. People will think the logic is backwards, but we can assure you its not. The help text points this out as well. We created some patterns designed to be used with an Optical Comparator to set grayscale. I would never want to use them as it would drive me crazy! You can get Calman Home for <your display> and an i1 for a good price. But for those that can't, you can try these.

There is a Specialty menu with a small subset of patterns. The purpose is to have a version of HDR10 metadata set to 0. It is also used for a set of patterns in Dolby Vision that disable tone mapping to see where the display actually clips. It has versions of the Dynamic Range High and Low with head and toe room in HDR.

We did a re-design on layout in terms of pattern naming and which menu they belong in. Hopefully this is more intuitive. Originally I had placed any pattern with motion under Motion or Motion HFR. That changed a bit. e.g. We now have one called Panel. Some of those patterns are in motion. Here is an example. (menu mockup from PowerPoint)

Panel


We added RGB and CMY versions of the Quantization Rotate. Sarah on a Hammock, from the 2nd edition, is back. We did tweak it a bit. For the 24p version, on the 2nd edition, we took the 60p version and dropped every other frame. Then played back as 24, so its a bit slower. This time we used the 60p version and played it as if it were 24p. This actually works much better. For the 60p version, we repeated every frame twice, to make it fake 120 and then encoded as 60p. Otherwise the motion was way too fast to be useful. This does show how the motion interpolation on TVs can clean up an image and it still breaks them due to the complexity of the pattern. The ropes moving over the gravel is not motion-interpolating friendly as of today. Gives everyone a bar to reach.

We added an HD version of the Chroma Align (Num) and Color Space Evaluation patterns. The Chroma Align works well if HD is scaled to UHD as in it makes it easier to see. However, some displays actually introduce alignment errors during scaling, which is why we added those chroma diamonds to the Scaling pattern on the current disc. Nothing beats a loupe to see it in UHD or UHD2.

We will include a printed version of the Beginners Guide that is in our download section which should help new customers get started. We may include the blue filter again, though it still does not work. We ended up getting every sample of filter material we could find. We even tried mixing different filters. In the end, nothing would work on the Sony Z9D. We even tried some very expensive glass filters and they were actually worse than the Tokyo Blue we use in our existing filter. We may include it just to make people happy. (Amazon reviewers) They are brutal w/o a filter. LOL The filter is also only for one pattern, color and tint, yet so many think you can't use the disc at all without one. Sounds like a good idea for a video.

We broke the starfield out into two patterns. We also added a new element. The idea is that some displays will show a perfect starfield until you introduce something else on screen, and then the stars can dim. Think of the Star Wars logo. The starfield is great and then dims a bit when the logo appears. I would like to also include chapter stops on the starfield so you can jump to the different versions quicker within the video.

We are unhappy that we have no montages on disc 1. We could have cut more and put one version on, but we really want all the versions together so you can jump between them to compare. And we are happy we are also able to include an SL-HDR2 version of the montage. (Advanced HDR by Technicolor). The files play on a C8 and C9, but I don't think there are any BD players that support the format, so they will probably just play as HDR10. But we now have a video in all HDR formats that Blu-ray supports.

We may include an HD version of the montage. If so, it will probably be based on the 1000 nit version.

[BijouMan]

Originally Posted by Stacey Spears

Spears & Munsil UHD HDR Benchmark Blu-ray

Spears & Munsil UHD HDR Benchmark Blu-ray

Spears & Munsil UHD HDR Benchmark Blu-ray



As soon as we have discs, it will go on sale. We gave Sony the thumbs up last week, so we should have discs in 2-3 weeks. Until we have inventory, I don't want to provide a date. At CES, we thought we would have it out in March. Discs are actually being pressed now, so it is real.

I don't recall how the last thread was created. It might have been created by an admin. I don't want to get in trouble by creating my own thread. As it is, I am nervous about posting in this thread, which is why I am trying to limit my posts to just answering questions. I don't want to violate forum rules with advertising, etc...

Does the HLG portion of the disc output as HLG on Panasonic players? If so, this disc is the only one that has that.

[BijouMan]

Originally Posted by Stacey Spears

Does everyone know where the current hidden menu is? Its just a page that reports player and display capability based on what the player detects.

I'd love to know where it is! I recently got this disc myself.

[Stacey Spears]

Originally Posted by BijouMan

Does the HLG portion of the disc output as HLG on Panasonic players? If so, this disc is the only one that has that.

No. BD does not allow the HLG transfer function to be used, which is how it is normally signaled. However, we found an SEI message that does work and it will output in HLG on the Panasonic. This will be on the new disc. It does not work on the OPPO though. Sony displays let you force HLG through a setting in the UI. LG added it to a hidden menu on their 2020, which is why we included it.

Also, there was a bug in the tool that was used to insert the HDR10+ metadata into the montage and it will not playback as HDR10+ on Panasonic, unless they added support. Panasonic pointed this out to us. It does on OPPO, which is how we tested it at the time. This is also fixed on the new disc.

[Stacey Spears]

Originally Posted by BijouMan

I'd love to know where it is! I recently got this disc myself.

On the configuration menu, move the cursor to P3D65|BT.2020. Then press the right arrow 5 times. It will now show you the hidden settings menu. If the player does not support a feature, such as Dolby Vision, it will also report that the display does not support it even if it does. The 3D support also does not ever seem to report correctly, so we will remove that one on the new disc. Everything else seems to report correctly as far as we can tell.

This menu, or rather the registers we expose in this menu, is what we use to determine if we should display the Dolby Vision and HDR10+ warnings on the montage page.

And to be clear, when you see the warning, we are not blocking playback of those formats. We get a lot of emails about the Dolby Vision warnings from Xbox One customers who insist it supports Dolby Vision. We keep explaining that it only supports it for Netflix. Most of the time they refuse to believe us, even after we send them an email from the dev manager at Microsoft, who is responsible for multimedia playback on Xbox One, saying the same thing.

[Geoff D]

Originally Posted by Stacey Spears

No. BD does not allow the HLG transfer function to be used, which is how it is normally signaled. However, for the new disc, we found an SEI message that does work and it will output in HLG on the Panasonic. It does not work on the OPPO though. Sony displays let you force HLG through a setting in the UI. LG added it to a hidden menu on their 2020.

Also, there was a bug in the tool that was used to insert the HDR10+ metadata into the montage and it will not play back as HDR10+ on Panasonic, unless they added support. It does on OPPO, which is how we tested it at the time. This is also fixed on the new disc.

They do indeed, and I'll be perfickly honest and say that the HLG version of the Montage was the best mapped version by FAR to my eyes on the ZD9, superior to the TV's clipping of extreme brightness/colour in HDR10 and its borked player-led Dobly implementation.