List of UHDs that utilize 4K or Higher Masters

[alexanderg823]

I don't know if this is the right thread for this... But for all the talk of display manufacturers using AI/neural networks to upscale to 8K, do we have any detailed information on the techniques that studios use when they upscale a blu ray disc?

Some of the AI upscaling results I've seen online not related to TVs or movies have been extremely impressive, almost imperceptible from true resolution imagery.

[Geoff D]

Well, they don't upscale the "blu ray disc" for a start

[alexanderg823]

Originally Posted by Geoff D

Well, they don't upscale the "blu ray disc" for a start

I'm juggling a bunch of stuff here at once, I meant upscale for the UHD blu ray disc. Look, it's morning here okay?!

[Geoff D]

Hence the , I'm just yanking your crank

[gkolb]

Originally Posted by alexanderg823

I don't know if this is the right thread for this... But for all the talk of display manufacturers using AI/neural networks to upscale to 8K, do we have any detailed information on the techniques that studios use when they upscale a blu ray disc?

Some of the AI upscaling results I've seen online not related to TVs or movies have been extremely impressive, almost imperceptible from true resolution imagery.

Somewhere in the 2019 CES thread is a video from one of the tv makers about the image database they use ‘on the fly’ to match the video image to best upscale to the quality of a stored image. Think of a tiger in a video and they find a tiger image in the database to apply the upscale patterns.

I’m probably not explaining this in the most coherent fashion, but I’m pretty sure the Sony 4K Z9D that I have already does something similar, they just didn’t call it AI at the time.

[edit]

If anyone is interested, I found the video from CES 2019 thread in post 188, it's this:

Sony - BRAVIA - Z9G/ZG9 Series - MASTER Series 8K HDR TV - YouTube

It's a short video, but zero in on the part starting around 0:26 .

[alexanderg823]

Originally Posted by gkolb

Somewhere in the 2019 CES thread is a video from one of the tv makers about the image database they use ‘on the fly’ to match the video image to best upscale to the quality of a stored image. Think of a tiger in a video and they find a tiger image in the database to apply the upscale patterns.

I’m probably not explaining this in the most coherent fashion, but I’m pretty sure the Sony 4K Z9D that I have already does something similar, they just didn’t call it AI at the time.

Right. But I'm talking about the upscales on the discs themselves. How exactly are they processed? Are they ran through what essentially is a photoshop filter or are they using advanced AI based tools to provide results that are negligibly different from real 4k?

Because the results of the upscaling on a 4K TV, even the Z9D or the a9 LG are good, but they're MUCH closer to just basic upscaling than the results of REAL AI and neural network implementations...

Check these links:

NVIDIA's Image Restoration AI: Almost Perfect - YouTube

https://deepsense.ai/using-deep-lear...er-resolution/

[gkolb]

Originally Posted by alexanderg823

...
But I'm talking about the upscales on the discs themselves. How exactly are they processed?
...

So, to re-phrase your question, How do studios upscale a 2K DI for example into a 4K UHD presentation?

I'm not good enough to answer this, but I know it's in the main HDR thread.
I'll throw on the beetlejuice-signal and hope you get an answer:

Geoff D
Geoff D
Geoff D

[Geoff D]

Of all the shit that I pretend to know about, this exact procedure is not one of them (though they don't take the literal blu-ray disc and start with that, in case anyone was wondering, despite the joke above ).

[alexanderg823]

You're quite resourceful Geoff, perhaps you could dig something up

My entire point is that recent advances in image processing due to legitimate machine learning/AI can do such a great job I almost doubt anyone would be able to see the difference between 2K and 4K native images.

Who knows, in a few years we may see all films upscaled to 8k or 16k using machine learning then downscaled back to 4K for the UHD blu and it would be a remarkably better image than just utilizing the 4K master to begin with.

[Doctorossi]

Originally Posted by alexanderg823

Who knows, in a few years we may see all films upscaled to 8k or 16k using machine learning then downscaled back to 4K for the UHD blu and it would be a remarkably better image than just utilizing the 4K master to begin with.

You could call it "better", but it wouldn't meet my definition. I'd rather have a lower value of actual resolution than a higher value of estimated resolution, no matter how "good" the estimated resolution's quality appears. I'm looking for representations of movies, not interpretations of them.

[alexanderg823]

Originally Posted by Doctorossi

You could call it "better", but it wouldn't meet my definition. I'd rather have a lower value of actual resolution than a higher value of estimated resolution, no matter how "good" the estimated resolution's quality appears. I'm looking for representations of movies, not interpretations of them.

But what's it matter if the estimated quality is essentially indistinguishable from real quality?

I mean keep in mind this isn't like replacing guns with radios in a film, it's merely cleaning up an image to be sharper - without all of the bad side effects we've had in the past like aliasing, ringing, blurriness, etc.

Another thing to remember is that in a world of fixed pixel displays, you're going to have to have the image upscaled regardless. Do you watch 1080p movies with black borders along all sides of the image on a 4K to maintain the true pixel count representation of the original image? If not, then you're getting the TV's interpretation of the image anyways - Why not be in favor of a smarter one then?

[puddy77]

Originally Posted by alexanderg823

Right. But I'm talking about the upscales on the discs themselves. How exactly are they processed? Are they ran through what essentially is a photoshop filter or are they using advanced AI based tools to provide results that are negligibly different from real 4k?

Because the results of the upscaling on a 4K TV, even the Z9D or the a9 LG are good, but they're MUCH closer to just basic upscaling than the results of REAL AI and neural network implementations...

[Show spoiler]Check these links:

NVIDIA's Image Restoration AI: Almost Perfect - YouTube

https://deepsense.ai/using-deep-lear...er-resolution/

There are lots of different upscaling algorithms that are akin to a Photoshop filter. They are high quality and "negligibly different from real 4k" because they are not doing it in real-time. AFAIK these new AI chips and programs that 8K TVs are implementing are needed because they need to do real-time upscaling.

For reference, here's an article from Steve Yedlin.

Another item: the article talks a lot about resizing (“blowing up”), but fails to mention that there are multiple algorithms for upscaling and down-scaling, which to a large degree determine the so-called “sharpness” more than the actual resolution does. For example, I’ve done tests with highly controlled variables that show that a 2k image upscaled to 4k with one of the sharper algorithms might look more perceptually “sharp” than an actual 4k image. (I’m not just talking about debayering, but actual re-sampling: like preparing 2k acquisition for 4k distribution). This issue is not totally separate from the “artificial sharpening” mentioned above.

I also recommend watching the first 15 minutes of part two of his resolution demo.

[alexanderg823]

Originally Posted by puddy77

AFAIK these new AI chips and programs that 8K TVs are implementing are needed because they need to do real-time upscaling.

No,

"real" AI, not samsung marketing AI, is about using machine learning to see the patterns of what real high resolution imagery would look like and using that information to intelligently fill in the blanks. This type of computation just can't be done real time on a small chip inside the TV no matter what Samsung or Sony says about their new processing chips.

[puddy77]

Originally Posted by alexanderg823

No,

"real" AI, not samsung marketing AI, is about using machine learning to see the patterns of what real high resolution imagery would look like and using that information to intelligently fill in the blanks. This type of computation just can't be done real time on a small chip inside the TV no matter what Samsung or Sony says about their new processing chips.

Gotcha. My point remains. Algorithms, not AI.

[Geoff D]

Originally Posted by alexanderg823

You're quite resourceful Geoff, perhaps you could dig something up

My entire point is that recent advances in image processing due to legitimate machine learning/AI can do such a great job I almost doubt anyone would be able to see the difference between 2K and 4K native images.

Who knows, in a few years we may see all films upscaled to 8k or 16k using machine learning then downscaled back to 4K for the UHD blu and it would be a remarkably better image than just utilizing the 4K master to begin with.

You saying this brings to mind the uprezzing Lowry Process, now this is not something that is "AI" as it's based around various proprietary algorithms that still require human intervention and need a lot of processing power, but the underlying principal isn't too far afield from what you're getting at.

They scan/ingest the relevant film/digital data and then use their patented process to analyse each frame and discard all parts of the image that aren't temporally consistent from frame to frame e.g. dirt & grain, making sure that the processing is throttled back for scenes with rain or snow lest it confuse those picture elements with what needs removing. It then takes the detail from the most well endowed frame(s) and uses it to essentially rebuild the image in adjacent frames that lack for this detail. It's not physically uprezzing the spatial resolution from 2K to 4K or whatever, but nonetheless it can make the image look much sharper from a temporal standpoint. It still has to look like 24fps of course, and the option is there for the filmmakers to regrain/renoise the image to their specifications, but the impact of this detail enhancement can be remarkable.

It's fallen out of favour in recent years as it's still very expensive, and what was once deemed necessary to "supercharge" SD and HD images becomes a moo point when you've got studios doing their own 4K/6K/8K scans all over the place and doing their own downsampling to 2K and 4K, thus retaining dat sweet sweet 4K detail.

But as it's commonly held that 35mm tops out at roughly 4K's worth of information (slow camera neg shot on the best spherical glass using test patterns) then if it's going to gain any ground in the 8K world it's going to need something extra, so some folks at HBO did some tests with 10K (!) scans of 35mm compared to 4K scans. They couldn't see any major differences but when they handed the 10K to Lowry Digital to process then it brought out more detail, albeit still subtly, compared to the 4K scan. http://www.etcentric.org/the-reel-th...scans-of-film/ (I'd love to see what a Lowry processed 2K upscale vs the same scan in native 4K would look like!)

Phew, I managed to waffle out some paragraphs after all. I was sweating there for a second, I felt like Chnandler Bong once his third nipple was removed: "It's gone...the source of all my powers!".

[gkolb]

^ Yeah, I thought you knew more than you let on earlier...

[Geoff D]

Yes, but that post has got nothing to do with what Alex was actually asking at the top! It's a lot of noise, no real answers, fairly typical for me. I still don't know what methods/systems they use to upscale a 2K DI into 4K HDR, at what stage the upscale is done (do they regrade then upscale, or upscale then regrade to make sure that the upscaled grain/noise doesn't do anything funky in HDR?) etc etc. Most processes relating to finishing and grading are out there in various books and articles, I even found that precis about how Sony rebuild their stuff into genuine 4K, but this upscaling lark is virtually anonymous. You'll need an actual expert to chime in on this one.

[alexanderg823]

Well it doesn't answer the question but is certainly relevant. I can say one of the main benefits of AI/neural networking/machine learning is that the pattern recognition continuously gets better as more data is fed into it, which can save costs into development. Of course, another huge benefit is that one can leverage cloud based computing platforms which would be absolutely massive for costs savings...

I don't know what processing bandwidth studio houses have reserved for the upscaling process to get a 2K DI onto a UHD Blu Ray Disc... buuuuut if I had to guess they could probably leverage much more powerful rendering capabilities at a lower cost by renting processing from cloud based companies like nVidia with much more specialized hardware. I mean some of the stuff nVidia has shown with regards to image processing is unreal.

[Pans]

Originally Posted by Doctorossi

You could call it "better", but it wouldn't meet my definition. I'd rather have a lower value of actual resolution than a higher value of estimated resolution, no matter how "good" the estimated resolution's quality appears. I'm looking for representations of movies, not interpretations of them.

As much I'm a fan of the advances in neural network upscalers I'm of the same opinion. Also I haven't seen consistent enough results for video upscaling so far.

The thing is, a neural network has to be trained on a dataset (training sources) which can be applicable to sources other than what it was trained with. So for example let's say you have a lower resolution image that lacks a higher resolution version, obviously if you had a higher resolution to begin with there would be no need to upscale it to begin with, so instead to teach the neural network it would be trained with a dataset of similar enough images that are higher resolution and use the resulting trained model to upscale the lower resolution image.

The results for images at least can be astounding, though it can also can vary in consistency. That said it is still like Doctorossi mentioned an interpretation, filling in the gaps in information with additional details, even if in ideal (let's say, cherry-picked) examples can seem near perfect.

For video though I haven't seen anywhere near as much consistency as where still images are currently for upscaling using neural networks, for one since frame-to-frame changes are needed to be taken into account, though I'm sure with time it will be improve greatly.

[Geoff D]

I'm not closed off to the idea of such AI-led upsampling, but there's a much simpler way to achieve such things: actually mastering in 4K/8K to begin with!

And yes, as with Lowry's own 'dumb' process mistaking the rain in Citizen Kane for dirt and removing it, the AI would absolutely need to be taught how to properly handle the entire gamut of images and textures that can be capture on moving picatures. But as you say, teaching it to upscale by using a higher-rez source vs the lower-rez downsample sounds like the way to go.