Toolchain Pipeline

Toolchain Pipeline and Colorimetry

Toolchain Pipeline

Processing Order

Everything that happens to an image, from the moment you open the file to the moment it is displayed on screen or saved, takes place in a fixed order. The data flows from one module to another - this is the toolchain pipeline. RawTherapee contains four pipelines (one for the main preview, one for the saved image, one for the thumbnail, and one other that currently escaped me). The following list shows a simplified order of operations:

1. Preprocess
   1. Dark frame
   2. Flat field
   3. Bad pixels
   4. Hot pixels
   5. Scale colors (internal, no tool in UI)
   6. Raw black point
   7. Lens distortion correction
   8. Green equilibration
   9. Line noise filter
   10. Chromatic aberration Correction
   11. Raw white point
   12. Raw histogram
   13. Prepare Auto Exposure
2. Demosaic
3. Retinex
4. Highlight recovery
5. White balance
6. Crop
7. Convert colorspace
8. Noise reduction
9. Spot Removal
10. Dehaze
11. Dynamic range compression
12. (Local Adjustments branch) avoid color shift, Log encoding, blur-noise denoise, tone-mapping, dehaze & retinex, contrast by detail levels, vibrance, soflight, local contrast, wavelet, sharp, exposure, color and light, Color appearance (Cam16 & JzCzhz), avoid color shift
13. Auto-match tone curve
14. Tone response curve
15. Process RGB
    1. Channel mixer
    2. Tone curve
    3. Highlights
    4. Shadows
    5. RGB curves
    6. HSV curves
    7. Color toning
    8. Film simulation
    9. Black-and-white
    10. L*a*b* color correction grid (Lab)
16. Process Lab
    1. Shadows/Highlight (Lab)
    2. Local contrast (Lab)
    3. Lab adjustements
    4. Vibrance
    5. L*a*b* color correction grid (Lab)
    6. Vignette filter
    7. Graduated filter
    8. Tone mapping
    9. Impulse noise reduction
    10. Defringe
    11. Edges
    12. Microcontrast
    13. Sharpening
    14. Contrast by Detail Levels
    15. Wavelets
    16. Soft light
    17. Abstract Profile
    18. CIECAM02
    19. Resize
    20. Post-resize sharpening
17. Final Lab -> RGB conversion

List of All Tools in RawTherapee

• Generic/Main preview
  • Input profile
  • Monitor Color Profile
  • Working profile
  • Output profile
  • Clipping indication
  • Red/Green/Blue/Luminosity/Focus mask previews
  • Colorimetric intent
• Exposure Tab
  • Exposure
  • Shadows/Highlights
  • Tone Mapping
  • Dynamic Range Compression
  • Vignette Filter
  • Graduated Filter
  • Lab Adjustments
• Detail Tab
  • Sharpening
  • Local Contrast
  • Edges
  • Microcontrast
  • Impulse Noise Reduction
  • Noise Reduction
  • Defringe
  • Contrast by Detail Levels
  • Haze Removal
• Color Tab
  • White Balance
  • Vibrance
  • Channel Mixer
  • Black-and-White
  • HSV Equalizer
  • Film Simulation
  • Soft Light
  • RGB Curves
  • Color Toning
  • Color Management
• Advanced Tab
  • Retinex
  • CIE Color Appearance Model 2002
  • Wavelet Levels
• Transform Tab
  • Crop
  • Resize
  • Lens/Geometry
    • Rotate
    • Perspective
    • Profiled Lens Correction
    • Distortion Correction
    • Chromatic Aberration Correction
    • Vignetting Correction
• Raw Tab
  • Sensor with Bayer matrix
    • Demosaicing
    • Raw Black Points
    • Preprocessing
    • Chromatic Aberration Correction
  • Sensor with X-Trans matrix
    • Demosaicing
    • Raw Black Points
  • Raw White Points
  • Preprocessing
  • Dark Frame
  • Flat-Field
  • Film Negative
  • Capture Sharpening

Colorimetry

The Importance of CIECAM and L*a*b*

Colorimetry gives rise to a lot of debate but we have to remember that it is not an exact science. No amount of equations, however complex, can ensure that the human eye will necessarily be satisfied with an image.

Currently RawTherapee uses the L*a*b* color space and CIECAM02/16 for chromatic adaptation and work has begun on exploring other color spaces (Jzazbz) and CAM models, for HDR applications (ZCAM does not work)..

The use of the L*a*b* (or CIELAB) color space does have its limitations but many of its shortcomings can be successfully mitigated, at least for SDR applications.

For example:

• One of the most frequent criticisms is that L*a*b* is non-linear and that it "distorts" the colors, in particular for blue-violet and red-orange. This is certainly true if you simply adjust the image using curves or chromaticity sliders. However in RawTherapee, if you click on "Avoid color shift" (Munsell correction), nearly 200 LUTs will correct these shifts and make the image perfectly linear.
• It is also said that L*a*b* addresses imaginary colors if the working profile allows it. This is also true but again, this can be compensated in RawTherapee by enabling "Avoid color shift". In this case, a relative colorimetric correction is applied to the working gamut as follows:
  • It analyzes the image data.
  • If it is within gamut no action is taken.
  • If it is outside gamut,the chroma is reduced and if this is insufficient, or if it is close to L=0 or L=100, then L is adjusted.
  • However this should rarely occur if Prophoto is used in the Working Profile and is probably not important.
  • If the saturation has been adjusted (chroma, vibrance,…), a Munsell correction using nearly 200 LUTs is applied. This will correct any color shifts with a high degree of accuracy e.g a red that has turned orange because of L*a*b*, will become red again. There will still be some errors but they are very small.
  • You can use just the Munsell correction by checking “Munsell correction only”.

L*a*b*

• L*a*b* is a reversible transformation of XYZ (in simplified terms, Y is transformed into L* using a gamma of 3.0 and a slope of 9.03). L*a*b* has more or less the same characteristics in terms of its limits (those of the primaries) as XYZ, which serves as a reference for the Working Profile and determines the basis of the gamut. Therefore L*a*b* and XYZ have essentially the same characteristics (exposure range, gamut, etc.). One point however, in many processes the values of L* can be bounded (clipped), to limit artifacts (high contrasts, highlights...), but in most cases L* is unbounded. If we ever get to HDR processing, we'll probably have to switch to "HDR-Lab". The data is not lost, even for high-dynamic range images (>= 25Ev), but the progression in the highlights is not progressive enough when used with monitors capable of displaying luminance values in the range of 120 cd/m² and beyond.
• I don’t think that the RGB->Lab transformation itself prevents complete HDR processing. This is because the calculations are generally carried out using ‘float’ or ‘double’ data values (32 or 64 bits) or using SSE (128 bit - 4x32 or 2x64 bits). The linear part of the Lab transform allows shadows with values of 0.005 cd/m2 or less to be processed. The parabolic part (gamma = 3.0) limits the distribution of data in the highlights allowing them to be reproduced more accurately (on suitable monitors) with luminance values above 120 cd/m2. The XYZ<=>Lab conversion leads to hardly any loss of data (insignificant due to double conversions) and can be considered as a kind of lossless compression. Of course if we want to achieve complete HDR processing, it is necessary to ensure that the way the data is processed, prior to being sent to the monitor, allows for more progression in the highlights. The preferred approach for Rawtherapee would be to implement HDR-Lab instead of Lab. But in the meantime I have implemented the possibility of changing the gamma of Lab (3.0) for several tools (wavelets, tone-mapping, etc.) notably to make it linear. It should be noted that Rawtherapee is designed to overcome one of the problems with Lab, which is the non-preservation of the hue when the saturation changes (especially in oranges and purples), by using "Perceptual Uniform Lab". This involves using a series of Munsell LUTs, as well as gamut control to prevent virtual colors.

L*a*b* does not impact the gamut or the dynamic range of high-dynamic-range images as shown in the following example with a Dynamic Range of 25Ev

Note that the majority of digital cameras in 2024 have a maximum dynamic range of about 15Ev. This image is therefore exceptional but demonstrates the behavior of Lab* for images with a high dynamic range.

TIF file (Creative Common Attribution-share Alike 4.0): [1]

Original image - 25Ev - unprocessed

Note:

• The shadows lack detail.
• Approximately 40% of the image is 100% white.
• The restored black-to-white dynamic range is around 12 to 13 Ev.

Unprocessed

Image with Local Adjustments - Log Encoding

Note:

• The shadows and highlights occupy the entire visible range from L=1 to L=99.8 (scale 0 – 100).
• The colors appear evenly distributed as a function of luminance. The use of L*a*b* does not impact the dynamic range.
• The dynamic range of the blacks, whites and color is restored to 25Ev.
• Note the use of "White distribution = 90".

With Log Encoding

CIECAM02/16

• One criticism of Ciecam02 is that is not able to process high dynamic range and wide color gamut images, which is partially true. A number of improvements were made by the development team a few years ago to mitigate this problem (bearing in mind that a large number of user images fall within the sRGB gamut and do not pose a problem). However, by using Log Encoding in conjunction with Cam16, or Color appearance (Cam16 & JzCzHz), the vast majority of problems can be solved. Of course some images will still present problems, in particular with highlight reconstruction, but this is not specific to Ciecam. The addition of Ciecam16 (Cam16) solves some of these problems.
• Ciecam02/16 is one of the only ways to achieve true colorimetric correction because it takes into account human perception and the surrounding environment. With Ciecam for example, any adjustments to the luminosity and/or the saturation, will take into account the image and its environment.

White Balance

• White balance is also subject to debate. The Temperature Correlation module recently introduced in Rawtherapee is almost mathematically (cognitively) perfect. It makes the xyY colors of the image coincide with known spectral data. However, on images where the temperature deviates a long way from D50, the colorimetry will not be correct because there will not be the necessary chromatic adaptation expected by our eyes and brain. Ciecam however, can take this into account.

Importance of the Linear-RGB Model and Colorimetry

The merits of the RGB model, and in particular, the linear RGB model are frequently cited. It is certainly the best way to carry out “upstream” processing (demosaicing, white balance, defringing, chromatic aberration correction, etc.) and anything that can be done in this mode should be.

However, CIELAB and Ciecam02/16 still have their place despite their shortcomings. As we have seen above, they are both derived from the CIE XYZ tristimulus values with Ciecam being one of the only ways to achieve true colorimetric correction.

So what about tone curves?

• Not only are they non-linear but they only provide limited colorimetric compensation, if any (with the exception of the Perceptual mode, which uses Ciecam02). This is in contrast to the Tone Response Curves -TRC- used for output (monitor, TIF etc.).
• The Auto-Matched Tone Curve, which is a copy of the in-camera TRC, is applied mid-process and introduces non-linearities in the processing pipeline.

What about saturation?

• Maintaining RGB linearity when you change the saturation is not impossible but it is difficult and is not implemented in Rawtherapee. On the other hand, if you adjust the saturation using Ciecam, it will take into account variations in luminance (or brightness) and adapt the color accordingly.

In conclusion, RGB, L *a *b *, and Ciecam all have their advantages and disadvantages. They simply need to be understood so that they can be used appropriately.