Industry leading light, Bill Baggelaar generously shares revealing and comprehensive insights into HDR - and what it means for the future of Broadcast Media. Read further to find out exactly what's in store - in his in-depth, Cine Alta Magazine interview, with Peter Crithary. Part 1, of 2.
Bill Baggelaar on HDR.
Bill Baggelaar is the SVP, Production & Post Production Technology, Sony Pictures Entertainment, where he manages the technology teams for picture and sound finishing. Bill has led the team at Sony Pictures to help Sony Electronics develop the world’s first 4K video service, Sony’s 4K Video Unlimited. Sony Pictures now has over 200 titles finished as UHD IMFs. Previously, Bill worked at Warner Bros. Studios for 13 years in feature animation, vfx, DI and video mastering. He holds a BS in Computer Science, is a SMPTE member and represents the Studio in various industry technical organisations.
Q: Let’s start with the background of high dynamic range (HDR).
BILL BAGGELAAR: High dynamic range is not new. It is the way that most of us see the world every day. Our eyes are high dynamic range and wide colour gamut-sensitive equipment.
On a bright sunny day, they can see cloud highlight details while still being able to see into the shadows. In dark environments, we can start to see detail in extremely low to practically no light. Some cameras are at least getting closer to being able to detect some of the same things on those sunny days. But they have also gotten better in the dark - not as good as our eyes, but certainly better as the sensor technology improves. On the other hand, the television sets or other types of displays that we typically have used to watch content has a much lower dynamic range than our eyes, or even the cameras that are being used to capture the content. So TV viewing, up to this point, has been what we now call, Standard Dynamic Range (SDR) where we have maybe in the order of seven to nine stops of exposure depending on the display. With these new displays, we are now talking about Ultra-HD (4K/UHD) and High Dynamic Range (HDR) that could potentially go up to 20 stops. But more practically for the consumers, we’re talking maybe 12 to 14 stops. We are inherently contrast-sensitive beings and so what this increased dynamic range does, is it gives us an increased sense of sharpness, detail, clarity, color and saturation; all these things that we see in the real world, but we’re not able to realize on today’s consumer displays or even in theaters, for that matter. And since we are incredibly adaptable beings, when we’re watching a movie or a TV show in a particularly either dark or bright location, our eyes and brain adapt. We may see something right away where the contrast doesn’t necessarily feel right, but we adapt pretty quickly to it and can watch it and not be distracted by it.
1] “Annie” in HDR: increased contrast allows overall shadow detail to be held - while reducing the glow effect from the flames.
2]“Annie” in SDR: Original image
But now with these higher dynamic range displays, we’re able to really start to give people more picture details in order to provide a more immersive experience. As I mentioned, getting better color saturation goes along with this whole wider color gamut piece that is part of HDR. So HDR is not just about contrast, or about resolution; or even about color; it’s about being able to combine all three to represent images more accurately on consumer displays. This gives content creators an expanded canvas to represent things that are true to life or they can even go hyper real. For finishing movies for theaters, we work in P3 color space, which is a wider color gamut than the TV standard, Rec. 709. Often times there are very specific colors, particular purples, reds or translucent colors; that cannot be displayed in SDR/Rec.709, so we have to do additional color correction to nicely squash it down into Rec. 709 for consumer displays. There’s all sorts of saturated colors whether it goes from blue to orange to purple to green that we can now represent on consumer displays that we’ve never been able to represent before and that provides something closer to the original artistic vision, representing what the DP and Director originally intended for the viewers to see.
“Amazing Spiderman 2” in HDR: Highlight detail is retained while also keeping shadow detail. This will retain distant objects like the cables in the bridge. The overall tonal detail that is retained adds to the viewer’s sense of immersion into the images. While at the same time keeping the image noise level down so that it does not become distracting.
“Amazing Spiderman 2” in SDR: Original SDR image
Q: So this is basically getting us as close to reality as what’s technically possible.
BILL BAGGELAAR: Yes, within certain constraints. There is the reality of staring at the sun on a bright sunny day. It hurts your eyes and we obviously don’t want to get that real. We don’t want people to be hurt by the content. So, within certain limits, yes. An additional sense of reality is possible, but I don’t know that we necessarily need to focus just on the reality piece. I think the content creators can present the vision that they want the consumers to experience more accurately although that sometimes doesn’t necessarily mean real. It just means that it’s maybe more immersive or maybe going to the level of creating colors that you don’t see in the real world that can exist, but you don’t necessarily see normally, but we can represent them in a way that we’re not able to with today’s display technology.
Q: Is it the camera delivering more in order to get that result? Or is it more of a postproduction process, or perhaps both?
BILL BAGGELAAR: A bit of both. Certainly it is better to start with captured images that have more inherent information in them. So the cameras have to be able to capture a wider dynamic range and a wider color gamut in order to truly take advantage of that in post. It doesn’t mean that you absolutely have to start with wider sources, but there are diminishing returns when starting from “narrower” sources. You’re always going to have the potential to get better results when you capture with a camera that has higher resolution, wider dynamic range and wider color gamut than the intended display, which is typically what we do. We’ve got the Sony cameras that capture in S-Log, one, two or three and we’ve got S-Gamut, which is a much wider gamut than P3. And as Rec. 2020 comes along, S-Gamut is actually even wider than Rec. 2020. Film has always been very wide as well and many of the other digital cameras have much wider color gamut than the displays that we actually have today or even are being planned for the near future, so starting from a wider color gamut helps to make sure that you can represent those colors on the displays.
“The Blacklist” in HDR: Shadow detail is able to be retained while also keeping the highlights from the overhead lights in check. On an HDR display, you can see into the shadows, whereas on an SDR display this will appear clipped.
“The Blacklist” in SDR: All of the shadow detail is lifted in order to make sure that conventional TV sets can display the scene, but this has an overall effect of making the scene seem brighter than it really was
Q: If the information isn’t captured to begin with, you can’t extract it and work with that in post-production so…
BILL BAGGELAAR: Right. It’s definitely important to start from a higher resolution, higher dynamic range and wider color gamut. The Sony cameras are much higher dynamic range and much wider color gamut than we’ve been able to take advantage of to date, so everything that we’ve been capturing in RAW we’re potentially able to go back and truly apply to these HDR displays in whatever way the creatives decide they want to apply them.
Q: Does that mean that Cinematographers now need to be aware (when using the right camera that can capture that very large range) of lighting and working in a day to day environment that is optimized for high dynamic range acquisition?
BILL BAGGELAAR: Yes. I think this is going to become one of the learning points on HDR right? What does that mean and how does it affect lighting? What SDR techniques apply and which need to change or adapt?
Q: Yes, that’s what I was getting at.
BILL BAGGELAAR: Targeting HDR? And I think that it’s going to be different approaches for different situations. If you’ve got really strong shadows and you’re trying to bury details in the shadows that you don’t think or want people to see, that may not necessarily be the way it ends up in the capture or in dailies. In post-production, you can always try the normal techniques to diminish that but it is definitely going to be a learning process for people to understand how to light for HDR. Director’s and DP’s still need to direct the viewers attention where they want them to be looking with lighting cues. So I think it’s just going to be a bit of a learning process for all involved.
“Amazing Spiderman 2” in HDR: The impact of this scene on an HDR display is stunning. You can get increased saturation in Electro’s blue translucent skin while still keeping the harsh shadows from his “glow”. This makes his eyes stand out a bit more.
“Amazing Spiderman 2” in SDR: Original SDR image — Electro’s skin details start to clip to white.
Q: Let’s talk about the consumer displays. How do we identify the televisions that can do it versus the ones that almost can do it versus the ones that can’t do it at all?
BILL BAGGELAAR: On the consumer side, that’s going to be tough. I see this definitely as a challenge, and the industry is struggling with this right now. We’re having conversations over what constitutes high dynamic range. And we have some general agreement, but by no means full agreement from all studios and CE manufacturers.
Q: Right, so what constitutes HDR?
BILL BAGGELAAR: What’s the minimum level? There is no consistency on that in the industry. And I think that is one of the things that the industry has to settle on for this to be successful. This way, we know what level we are going to master our content for and we will then know what the consumer experience will be across the range of displays that will be called HDR. We’ve made suggestions on what we think high dynamic range should be, which is 1000 nits at minimum, along with contrast ratio and minimum peak brightness suggestions. In the end it will have to be easy for the consumers to understand what they should expect from an HDR experience. There’s got to be some sort of minimums to give people the confidence that the industry is helping them to make the right decisions on display technologies.
Q: Well, is Rec. 2020 the benchmark then? If they were to meet that, would that be it?
BILL BAGGELAAR:Rec. 2020 does not specify high dynamic range. So Rec. 2020 is purely a color gamut and resolution and we’ve been working to get an expansion of Rec. 2020 to include higher dynamic range language as well as those sort of dynamic range minimum brightness to help further what Rec. 2020 (4K/Ultra HD ) will actually mean. But I think that we still have a ways to go to either get consensus that it needs to go into Rec. 2020 or whether it’ll be an additional spec that goes alongside it or it becomes part of a larger industry standard.
Click to read Part 2 of this article
Read the related BVM-X300 Technology Briefing
Learn more about Sony’s HDR and OLED technologies
*It maybe hard to see the effects described, without an actual HDR display