RawTherapee cross-platform raw image processing program: a color-managed Linux workflow (Chapter I).

Partly out of curiosity, and partly to help widen the horizon of digital photography, I have worked through a color management workflow entirely in Linux, from raw photo to monitor to print.  It is actually a continuation of my page on zone system calibration in linux.  I must admit that the results were more a “proof of concept” than an ideal workflow, partly because of available alternatives, and partly because of limitations on my own ability to print in Linux.

Operating system:  Korora 20, 64-bit.  Korora is a Fedora remix that includes third-party and proprietary software that makes it easier to use out-of-the-box than Fedora, while retaining the “bleeding edge,” “state-of-the-art” qualities of the latter.  The applications below were all installed from the repositories, except for TurboPrint which was an .rpm download.

Color management application:  Argyll Color Management System 1.6.3, by Graeme Gill.  For Linux, there is no real alternative.  I mainly followed the tutorials on the Typical usage Scenarios and Examples page, together with the documentation for each individual program.

Raw converter/image editor:  RawTherapee 4.2.74.

Print utilities:  PhotoPrint 0.4.2 with Gutenprint 5.2.9, and the GNU Image Manipulation Program 2.8.14 with TurboPrint 2.35.

Camera profiling target:  ColorChecker Passport.

Color measuring instrument:  i1Pro spectrophotometer.

The following are some personal observations about the profiling process.  The names of the Argyll programs in the command lines are links to each program’s excellent documentation.  Please see these and the Typical usage Scenarios and Examples page for detailed information.

Camera profiling.

The creation of a camera .icc profile for raw images requires photographing a suitable target, rasterizing the image, measuring it with scanin, and creating the profile with colprof.

Using as I do a ColorChecker Passport, I use a spotmeter to place the light gray patch at the bottom left on zone VII, or two stops above the meter reading.  This keeps it from blowing to white while allowing plenty of latitude for the rest of the target.  My favorite profile lighting is overcast skies near midday.  .icc profiles, which are created from rasterized images, are more sensitive to color temperature and other lighting characteristics than .dcp profiles created directly from raw images.

Two files provided by Argyll, ColorChecker.cht and ColorChecker.cie (found in /usr/share/color/argyll/ref), contain the L*a*b* values and other information about the target patches that the program needs to create the profile.  The color values are also provided by X-Rite.  Note that, based on the sum of the squares of the a* and b* values, the most neutral patch is the fourth gray patch from the left, number 22 in the X-Rite illustration, or D4 in the diagnostic image below.

RawTherapee makes it relatively easy to process the raw image for measurement.  Open the raw image and choose the (Neutral) processing profile.  On the Color tab, use the Spot WB tool on patch 22 to set the white balance.  Then, under Color Management, press the “Save Reference Image for Profiling” button.  In theory, that is all we have to do to create the measurement image; RawTherapee will save a linear .tif without any color management.  Input and output profiles are both ignored; neither choice affects the image, and there is no embedded profile.  In reality, it may help to straighten and crop the image.  Very tight cropping is not required, and Argyll is able to compensate for perspective distortion.  The main consideration is being able to correct the image in 16-bit without any color management.  RawTherapee can be used to do this; open the processed .tif, set the (Neutral) profile if necesary, and use the straighten and crop tools on the main panel.  Then under Color Management, make sure no Input Profile is specified (not “No profile”; “Use embedded, if possible” should work), set Output Profile to “No ICM: sRGB Output,” make sure that Output gamma is set to “default,” and save the image as something like target.tif.  Another editor that can do this is Fotoxx.

Now we get to use Argyll.  Copy ColorChecker.cht and ColorChecker.cie into the working folder with target.tif to make things easier.  First we run scanin:

cd /home/russell/Desktop/ArgyllCMS  [your working folder here]
scanin -v -p -a -G 1.0 -dipn target.tif ColorChecker.cht ColorChecker.cie diag.tif

I won’t bore you with the meanings of all these, they are explained in the documentation.  Of note, -G 1.0 indicates that the gamma encoding of the image is 1.0, thanks to RawTherapee.  diag.tif is the name of the diagnostic output image that reality-checks the patch recognition.  target.ti3 which contains the measured patch values is also created in the working directory.

The linear gamma measurement target, and the diagnostic output.

If the target was measured successfully, we now run colprof to create the .icc profile:

colprof -v -A "Nikon" -M "D800E" -D "Nikon D800E overcast" -qh -am -nc -u target

Life is short, and art is long; go for high quality, -qh.  (Per the documenation, -qu, ultra, is a test; don’t use it.)  The algorithm switch needs some explanation; I used -am (matrix) as it is recommended for linear, raw images, and it seems to be the best.  The ColorChecker Passport software creates .dcp profiles with color lookup tables (cLUT’s); more about that later.  For now, note that 24 patches is evidently not enough for Argyll’s tables; images processed with cLUT profiles tend to have blown, pink highlights, like the first one in the second row below (note the chrome on top and the right side).  Using gamma and shaper curves appeared to make no difference.  I did not attempt to scale or otherwise manipulate the target or profile to adjust the white level; the -u swich is said to be used for targets whose white point is not expected to be media white, and allows for further adjustment of the processed image.

As you can see below, images processed with .icc and .dcp profiles are reasonably similar, especially compared to the image without any color management; but they are not as similar as the images taken with different cameras and processed with different raw processors and operating systems using .dcp profiles on the zone system calibration pages.  In fairness, a target with more patches may have helped; but note that the .dcp profiles were made from the same 24-patch target.  (All four of the images below were white balanced on the spot below the middle of the word California; there was no other manipulation.)  My assessment:  if you need .icc camera profiles for your application, or can’t or won’t use Wine for whatever reason, Argyll in all likelihood will meet your needs.  This is especially true if you are using controlled lighting.  But for real-world photography, I would recommend using the ColorChecker Passport software in Wine as described on the calibration page above, as I believe that .dcp technology is superior when the light is a moving target.

Image processed with the Argyll .icc matrix profile,
and one with a ColorChecker Passport .dcp profile for comparison.

Image processed with an XYZ cLUT .icc profile,
and one processed with no color management.

Monitor profiling.

Profiling output devices is a two-step process:  calibration, then characterization.  Calibration is like setting the hour and minute hands of a watch to NIST atomic clock time; characterization is like then observing, “it’s running about 8 seconds fast.”  For a monitor, one first uses the controls to get the brightness and color balance into the ballpark.  Then the profiling harware and software measure the output and further calibrate the system, then create an .icc color profile which characterizes the results.

Profiling the monitor with Argyll appears to be successful.  I profiled the Dell E2014H that I use for everyday browsing, and the result seemed to be a little better than the one I got using i1Profiler last year, which had undoubtedly suffered from drift.  If I had been more successful at printing, I would have profiled the U2410; but as it is, I let it go for now.

Creating a profile in one step

One first runs dispcal to see if the measurement device is detected:

dispcal -v -R

I tried using my old Monaco Optix DTP94 instrument; it was initially recognized, but then all I got was, “Instrument Access Failed - Not expected instrument model.”  [Graeme Gill says that this may be due to known bugs in Argyll version 1.6.3 that should be fixed in the next release.]  I subsequently used the i1 Pro without any major problems; one time, in the middle of a measurement, I got, “Sample read failed due to communication problem.  Hit Esc or Q to give up, any other key to retry:”, and it continued on.  The -J switch is unnecessary as the software automatically requires calibration of the instrument before each use.  The -y display type switch was ignored.

Begin by turning off any existing monitor profile:

dispwin -c

Then set your monitor’s controls to the factory defaults.  For me, that means brightness and contrast at 75, and red, green, and blue at 100.  This seemed too bright so I turned the colors down to 75.  On the U2410, the defaults are all 50.  With an LCD monitor, you will probably not adjust the contrast any further.

From here, one may take the quick single-step route, or the more complex multi-step one.  If all you want is a simple profile for web browsing and such, just run the following:

dispcal -v -ql -t 6500 -b 100 -gs -o /home/russell/Desktop/ArgyllCMS

Note, Argyll’s monitor measurement process is rather on the slow side; only use -qh if you have a real need and time to wait.  (And adjust your computer’s power managment policy to keep it from shutting down.)  The “standard” for display profiling is 6500°K and 120 cd/m2; I believe these are too blue and bright as noted below.  -gs sets the gamma to that of the sRGB profile, so web graphics will display as expected.  And -o tells Argyll to go ahead and create the .icc profile without further ado.

For a printing workflow, I think 5500°K and 80 cd/m2 are more useful; and one probably would not want to use sRGB gamma as it gives a little boost to the shadows, which your printer will probably not do.  Few people are likely to view your prints under 6500°K lighting; manufactures use this because it looks attractive in bright showrooms, or maybe because it makes PowerPoint presentations look good.  5500°K is just about the most neutral “white,” and I for one have adopted it as a standard for monitor, working space, and print.  A rather dim 80 cd/m2 brightness is often disparaged as being appropriate for working in a cave; but if your prints always seem too dark, I will propose that it is your monitor that is too bright; turn it down.  After you get used to it, the images that you make for printing may look terrible online, especially if you are using a wide-gamut monitor; but that is o.k. because those are separate workflows.  So for a better print workflow profile, use:

dispcal -v -qm -t 5500 -b 80 -g 2.2 -o /home/russell/Desktop/ArgyllCMS

After calibrating and positioning the measurement device, one gets the following, along with some measurements:

Display adjustment menu:
Press 1 .. 7
1) Black level (CRT: Offset/Brightness)
2) White point (Color temperature, R,G,B, Gain/Contrast)
3) White level (CRT: Gain/Contrast, LCD: Brightness/Backlight)
4) Black point (R,G,B, Offset/Brightness)
5) Check all
6) Measure and set ambient for viewing condition adjustment
7) Continue on to calibration
8) Exit

Steps 2 and 3 are what worked for me.  I think it would make more sense to set the brightness first, then the white point; but I did the steps in order, then repeated them, and it all seemed to work.  First, we set the white point (press 2):

Adjust R,G & B gain to get target x,y. Press space when done.
Target Br 80.00, x 0.3325 , y 0.3475
/ Current Br 80.23, x 0.3311+, y 0.3464+ DE 0.8 R+ G+ B--

Adjust the RGB controls to get x and y in the ballpark, and minimize DE (delta E).  The plus and minus signs after R, G, and B in the output above tell you what to do; in this case you would start by lowering the blue control.  They will never be perfect, just do your best.  Remember, there’s a characterization step after this.

Next, set the white level (brightness; press 3):

Adjust CRT Contrast or LCD Brightness to get target level. Press space when done.
Target 80.00
\ Current 80.65 -

Once again, follow the plus or minus sign to raise or lower brightness to as close to the the target value as it gets.

When you are finished making the adjustments, press 7 to continue.  If you have used the -o switch in the command, you will get an .icc monitor profile, along with a .cal calibration file for use in subsequent steps, if any.

Now what?  First, remember the brightness, contrast, red, green, and blue settings that you found, as proper working of the profile depends on them.  Then, copy the profile to a folder with public access, such as /home/russell/.local/share/color/icc.  Activate the profile:

dispwin /home/russell/.local/share/color/icc/monitor.icc

As long as the profile is in a location with public access, you can use this command to create a startup application entry that runs dispwin and sets the profile automatically.  The -I switch, which is intended to install the profile as the system default, did not work for me.

To inactivate the profile for comparison:

dispwin -c

To measure the final result:

dispcal -v -r

Current calibration response:
Black level = 0.2316 cd/m^2
50% level = 19.13 cd/m^2
White level = 84.20 cd/m^2
Aprox. gamma = 2.14
Contrast ratio = 364:1
White chromaticity coordinates 0.3300, 0.3449
White Correlated Color Temperature = 5608K, DE 2K to locus = 4.5
White Correlated Daylight Temperature = 5609K, DE 2K to locus = 0.3
White Visual Color Temperature = 5496K, DE 2K to locus = 4.3
White Visual Daylight Temperature = 5616K, DE 2K to locus = 0.3

More advanced profiling using multiple steps

To exercise more control over the process, one may use targen to create a set of custom test values, based on an existing profile if possible (such as a simple one created in the previous step):

targen -v -d3 -G -g33 -f300 -c monitor.icc monitor

There is a touchy-feely aspect to patch generation.  You may want to optimize your display for black-and-white images, for example.  -g33 includes 33 evenly-spaced gray steps, and -f300 brings the total to 300 by adding full spread patches.  (Note that the brief description of -f in the documentation says, “Add iterative & adaptive full spread patches to total,” which is a little ambiguous; the complete documentation correctly indicates that, “full spread patches will be added to bring the total number of patches including those generated using the -e, -s, -g -m and -b flags up to the specified number.”)  Various options may be chosen for the distribution of the patches, including spacing, and emphasis on the neutral axis and shadows.  The result is a .ti1 color patch file which is read, together with the .cal calibration file generated in the previous step, with dispread:

dispread -v -k monitor.cal monitor

This creates a .ti3 target data file.  Now we use colprof as we did when creating the camera profile:

colprof -v -D "Monitor" -qh -S ProPhoto.icm -cpp -dmt monitor

This will create a large, cLUT-based profile, which I have come to appreciate from using i1Profiler.  The -S switch specifies a color profile for gamut mapping, and the -c and -d switches specify the associated input and output viewing conditions.  I confess that I am unclear on the concept of why pre-computed gamut mapping is required when using an active color management module (CMM).  This is discussed more fully on the Gamut Mapping page.  I guess the ICC Specification ICC.1:2001-12, File Format for Color Profiles (Version 4.0.0), is trying to cover all the bases:  “The intent of requiring tags with profiles is to provide a common base level of functionality.  If a customCMM is not present, then the default CMM will have enough information to perform the requested color transformations.”  I used ProPhoto.icm as the gamut mapping profile because it is big, and it or a variation of it is my usual working space.  The resulting contrast ratio was 428:1, compared to 382:1 when using a printer profile.

Printer profiling

Printer profiling similarly involves both calibration and characterization.  Some printers incorporate a linearization utility that standardizes the output.  Argyll can do this with printcal, which I did not try.  The remainder of the calibration process involves the selection of paper and printer settings.

I hate to say it, but printing in Linux is still rather in its infancy, and resources are limited.  I began by using the Gutenprint drivers that came with Korora; unfortunately, I only own one printer that can print with it:  a small four-color Canon iP3000.  And I discovered during this process that the only supported print mode is Plain Paper.  I was unable to get the Pro9500 Mark II to work at all.  (I didn’t try the iPF6350.)  So I made the best of it, with a modest degree of success.  (I then had better luck, along with some more problems, with TurboPrint as described below.)

First, we need a patch set.  The targen page includes a list of measurement devices and the number of patches that they use with different sizes of paper.  For the i1Pro, that’s 462 patches for an 8.5×11.

targen -v -d2 -G -e2 -B2 -s16 -g16 -f462 iP3000_SemiGloss

Use your feelings to decide how many patches of what kind to choose; the above include 2 white and 2 black, ramps of 16 steps of all three channels plus gray, along with full spread patches to total 462 for one sheet.  (If this were intended to be a high-quality profile, we would undoubtedly use several sheets.)  Once again, this generates a .ti1 color patch file.  Now we need to format these patches into an image for printing.  Argyll is something of a marvel and has all this built in:

printtarg -v -ii1 -T 300 -C -p Letter iP3000_SemiGloss

The -T tag produces .tif output rather than PostScript; I used this as I virtually always print .tifs.  A .ti2 file is also generated that contains the color and layout information for the patch set; it is important to keep track of it for reading the patches.

Gutenprint

Now comes the first tricky part:  we must print the patch set using parameters selected to optimize the print quality, but without applying a profile or any other color management.  I used PhotoPrint for this as it is 16-bit and fully color-managed.  It feels like a work in progress, and will shut down without warning if you do anything wrong, but it seems to get the job done.  Below are the settings that I used.  The most important ones are outlined in red; be sure that no profile is assigned to the image or specified for printing, and tell the printer not to do any other color correction.  Also, in the menu, set Layout to Manual Size.  The rest of the settings represent calibration, and must be used exactly as-is whenever this profile is used.  Having to use the Plain Paper setting is very unfortunate; it results in a fast, dithered output that is not much to look at.  Any other setting results in a greenish-and-white, distorted image that implies that the other modes are just not supported.  Using the 600×600 DPI PHOTO HIGH resolution with the other settings maximizes the quality, to the extent that it is able.  And even though this printer’s ink type is not CcMmYK, that is the only choice with this resolution setting.  (We are winging it here . . . .)

PhotoPrint settings for printing the profiling target:

Let the print dry, preferably overnight, due to the possibility of dry-down.  Then comes the second tricky part:  reading the patches.  The process is fairly straightforward, except that in my case, the unit does not beep as it does using i1Profiler.  So the procedure is:  press the button, wait a second for an imaginary beep, slide the unit, release the button, then look at the terminal output to make sure that the row was read; if it was not, press the spacebar and read the row again.

chartread -v iP3000_SemiGloss

Ready to read strip pass C
Press 'f' to move forward, 'b' to move back, 'n' for next unread,
'd' when done, Esc or 'q' to quit without saving.
Trigger instrument switch or any other key to start:
Strip read failed due to misread (Not enough patches)
Hit Esc to give up, any other key to retry:

The measurements are stored in a .ti3 file.  Now we again use colprof to create the profile.  As before, I used ProPhoto.icm as the gamut mapping profile; see the note about it at the end of the monitor section above.  [Graeme Gill disagrees here, and recommends creating a source gamut that actually reflects the gamut of the working images.  “The best way to do this is to create a source gamut from each image or a set of images using tiffgamut. Using a device link workflow would probably be faster for this approach, since it avoids creating a lot of unnecessary tables each time. A simpler alternative is to choose more realistic source gamut such as AdobeRGB or sRGB, even if the storage format uses ProPhoto.”]

colprof -v -D "iP3000 Canon Semi-Gloss" -qh -i "D55_1.0.sp" -S ProPhoto.icm -cmt -dpp iP3000_SemiGloss

If you liked this article, subscribe to the feed by clicking the image below to keep informed about new contents of the blog: