Unable to determine effective Video LUT entry bit depth

[Emil Ingelman Sahlen]

When I run the “Report on uncalibrated display”, it fails to determine the VideoLUT depth for my externally connected BenQ SW320 with the following message:

“Unable to determine effective Video LUT entry bit depth”

OS: macOS 12.1 Monterey
DisplayCAL version: 3.8.9.3
ArgyllCMS version: 2.3.0
Computer: MacBook Pro 14″ M1 Pro 10 core
Display: BenQ SW320

[Vincent]

That means that “visually” your GPU can provide more that 8bit corrections loaded into GPU LUT. Good.

[Emil Ingelman Sahlen]

Ah, because on my Windows machine it said that the effective bit depth was 10 bits (sometimes, unless the Nvidia driver starts glitching). I interpreted the inability to determine as something being wrong but you’re saying that it’s actually good news?

[Vincent]

I test if it can measure a difference. It is not an actual test of VCGT bit depth.
“8bit” means “I can measure truncation”.
“>8bit” (or even cannot determine) means “I can measure that after varying VCGT beyond 8th bit I can measure a change in output”

8bit bad, otherwise ok (although even with >8bit actual dithering is prefered to avoid banding caused by VCGT calibration loaded to GPU)

• This reply was modified 2 years, 3 months ago by Vincent.

[Author]

Posts