Profile loader 8 bit option

[Llesy]

Hi,

Since DisplayCal/dispcal takes calibration measurements in the videoLUT (e.g measures 1.9 red in videoLUT entry and set to nearest e.g 2 red in entry index/framebuffer), (on nvidia this would truncate to 1 red as profile loader quantization setting is not respected here, tested this myself with spotread and DC curve viewer) and since calibration tables are produced and interpolated from these colorimetric values read and since later profiling characterization measurements do not respect the profile loader quantization bitdepth option either and just use the maximum resolution available for the calibration 1Dlut, is it best to leave the profile loader quantization bitdepth at 16 bit? On an nVidia card on Windows 11 with truncation on an 8 bit display. I think so because that was what it was measuring right? Rounding to 8 bit via profile loader quantization setting afterward might make grayscale errors in calibration and the profile invalid since display is now in a different state to when measured right? Or is the profile loader setting still more accurate?

–

[Vincent]

Regarding Actual TRC (bandless-dithered vs truncation banding) vs profile TRC: usually a verification report will not show a significative increase in grey range a*b*… it is a visual actifact when you compare two greys at different L* whose error to the reference is bellow X dE (all green/almost all green validation), but when shown next to the other its relative a*b* diference is high enough to notice.

If you are concerned about nvidia induced banding for working with gradients, try DWMLUT. Simulate an idealized natige gamut version of your display and let DWMLUT dithered shaders run the grey (& color volume) calibration.

Anyway, AFAIK if display accepts 10bit input and you use DP/HDMI newer nvidias should not truncate till bitdepth. Another issue is that if any Windows app uses Windows LUT loader: standby, xrite app, etc, LUT truncation will be active till reboot even on AMDs.

[Author]

Posts