For those just learning about perceptual colour spaces, I’d recommend exploring OKLab which is simpler to implement and overcomes some of the problems of CIELab.
Oklab is awesome and it’s such a great example of a person putting in time somewhere, where many just glanced over but still complained over the years. And it was so good that it was adopted everywhere.
This is great! The project that prompted me to submit TFA becomes ever so slightly better in OKLab (from early experiments using CSS color-mix) but I thought it would be more complicated than CIE Lab. Seems like it's not? Will try to implement it using OKLab instead!
I right away admit that I am an absolute novice in this space, but I have a few questions. The question I always had is why do we not model it closer to the actual tangible Physics and Biology going on ?
For example, the Physical reality is the different frequencies of light. The Biological reality is that different types of cells on our retina respond with differing intensity to each of those frequencies.
So to my naive mind, a way of modeling color is to have (i) a forward model that map light frequencies to response intensities of the different types of cellular light receptors and (ii) an inverse model that estimates the frequency mix of light from the cellular responses.
That is, have two spaces (i) the light frequencity space (a list of tuple of frequency and intensity/power at that frequency) and the (ii) cellular response space.
Once we have these, we can go from a pigment or excited phosphor to a biological response in a two step process.
From (a) pigment/phosphor (+ frequency mix of illuminating light) to output light frequencities (b) from those frequencities to cellular response.
For all processing, make frequencities the base space to work in (allowing us to change/personalize the forward model).
Yes, the inverse model leads to an ill posed inverse problem, but we are now very knowledgeable about how to solve these.
The frequencies may need to discretized for convenience.
I am obviously a novice and don't know much about modeling color, but this way of modeling seems more grounded in the tangibles. This also gives a way to model how a color-blind person might perceive a picture.
Is the reason that we do not do it this way it's complexity ?
You'll see it is based on the physics and human biology, and is the basis for everything else.
The thing is, this color space isn't very useful for color calculations in the perceptual/subjective sense (e.g. if I just want to change one characteristic, like luminosity, without affecting the chromaticity), so we have transformations to more useful spaces like XYZ, Lab, etc.
There's also the fact human vision is a subjective/psychological phenomenon, so only frequency response is not enough: our vision can map different frequency responses to the same perceptual color (metamerism), our vision adapts color perception based on light source, etc.
Many of these, like RGB or YUV, are intermediate "simplifications", not what you're necessarily supposed to be working in to generate them. It's like a physics calculation where everyone just decides to randomly stop half way through and use that as a transport medium. RGB exists because that's how the display physically operates, it's how bright those literal subpixels should be. Thus the RGB intermediate transport. YUV (particularly 4:2:2 and 4:2:0) is then essentially a lossy-compression format that's very easy to compress/decompress, and that's why it exists.
If you're doing rendering work, like in a game, those do operate in a more physical domain. That's the so-called "Physically Based Rendering" or PBR that you might see if you hang out in game dev circles.
Good comment. For people with physics and mathematics skills and intuition, learning or discussing about color or music theory can be a little bit frustrating experience. Ie a lot of it is about traditions and some famous people's ideas and less about how it actually works under the hood. I guess similar things exist in most fields when you dig a little bit deeper. "We always apply this fudge factor to get the correct results". Even something considered hard science.
Of course artists can be very effective with whatever the toolsets they have learned, they sort of can transcend all the obfuscations and actually can express themselves. It's a bit hard to change everything then as the "user's transformations" are then baked in.
It's also true that mathematical models are often simplifications and one has to consider the whole end to end pipeline, where a more "accurate" transformation can yield a worse end result if applied blindly. I'm always reminded of the anecdote of when in the past, at some tv channel, they switched to more accurate weather forecast models. But the audience got worse forecasts, since the resident meteorologist was used to certain errors the old model produced and could compensate those, while the new model had maybe less errors in total but they were different. Happens all the time and why some engineer thinking something is objectively better might not actually be better for the customer...
Most display technologies output RGB light, which means you’re already forced into an output methodology that’s relying on deception purpose-built for human biology and psychology.
Because of that, the layer of your model concerned with the physical reality of light doesn’t seem super useful for basic image input and output.
But of course this layer is incredibly useful for stuff like computer graphics, where it is indeed the case that physically based rendering is widely used for offline CGI rendering (and increasingly for realtime video game rendering).
We do model it according to perception, just only when it is useful. Other colour models allow different outputs, printing with coloured ink, displaying on screen by mixing coloured lights, categorising colour, finding how bright it is etc. all need different representations in numbers.
The problem is that this will only work in one direction. You can calculate the stimulation of the photoreceptors for a certain spectrum, but not the other way around. For example the eye cannot distinguish between purple light consisting of one specific wavelength and purble light mixed by red and blue wavelengths, because both give the same stimulation of the receptors. So there is an infinite number of possible spectra for any given stimulation of the photoreceptors. All we can do is take the stimulation values (X, Y and Z) and convert from there to all kinds of color models and back.
Your approach would make a lot of sense for sensors that are full spectrum analyzers, but the eye isn't one.
Yes because it's not a one to one map we cannot invert the map uniquely. But that's ok, we can maintain a distribution over the possible frequencities consistent with the response. That's how it's done in other areas of mathematics where similar non-bijections arise.
Much thanks for answering though, because I suspect I am asking a very basic question.
You're correct, for what it's worth. I too have always wished that light was modeled based on physics, not on how humans happen to see.
Unfortunately the problem is data acquisition (cameras), and data creation (artists). You need lots of data to figure out e.g. what a certain metal's spectrum is, and it's not nearly as clear-cut as just painting RGB values onto a box in a game engine.
For better or worse, all our tools are set up to work in RGB, regardless of the color space you happen to be using. So your physics-based approach would have the monumental task of redefining how to create a texture in Photoshop, and how to specify a purple light in a game engine.
I think the path toward actual photorealism is to use ML models. They should be able to take ~any game engine's rendered frame as input, and output something closer to what you'd see in real life. And I'm pretty sure it can be done in realtime, especially if you're using a GAN based approach instead of diffusion models.
No need for ML. This already exists, the keyword to look for is "spectral rendering".
To add to the general thread: the diverse color spaces are there to answer questions that inherently involve how a typical human sees colors, so they _have_ to include biology, that's their whole point. For example:
- I want objects of a specific color (because of branding), how to communicate that to contractors, and how to check it?
- What's a correct processing chain from capturing an image to display/print, that guarantees that the image will look the same on all devices?
I think this is closer to that than you think. Lab has three intensity vectors. One for each of the photoreceptors in your eye. Anything more than Lab is unnecessarily conplex IMO.
The three Lab values don’t map onto the three different wavelengths captured in the retina.
It’s more like the L is an intensity/brightness factor, and then the a and b values corresponding to the two dimensions of opponent color that neurons capture in the thalamus one step after the eye
In fact, to support your point, it is perhaps questionable from first principles if 3 dimensions not 4 is right. Leaving out tetra chromats and the (partially) color blind, normal human light perception is 1 kind of rod and 3 kinds of cones (i.e. 4 photo receptors, plus some light sensitive ganglia that don't seem to participate in vision, but diurnal regulation).
So, sure, this "4th dimension" (for normals) might be as simple as "candelas" - truly orthogonal, but one does hear an awful lot about "ambient" or "candela contrastive" (a term I just made up) kinds of effects. (EDIT: e.g. in color calibration of projectors in dark rooms vs. living rooms, for example, but I'm sure there are many.) I am just one person, but it feels like candela brightness matters for color perception. So, maybe luminous intensity is not actually exactly orthogonal. Maybe this is all covered in the 1931 CIE documents, though.
> For example, the Physical reality is the different frequencies of light. The Biological reality is that different types of cells on our retina respond with differing intensity to each of those frequencies.
Because this isn't true. And there are multiple ways with completely different combinations of intesities of light at different frequencies to have a color that most people will see as the same color. This is because the color receptors of our eye overlap.
> And there are multiple ways with completely different combinations of intesities of light at different frequencies to have a color that most people will see as the same color. This is because the color receptors of our eye overlap.
Of course, if you read back you will notice I am quite aware that the mapping is not a one to one bijection. Hence the need to solve the inverse problem.
One can maintains the inverse image (a set), or maintain a distribution over that, or certain statistics of that distribution. I have a link in my post about what inverse problems are.
We see different colors not because our eyes can tell what frequency a particular spectral color is, but because we have three types of color receptors, each of which is excitied by different broad ranges of overlapping frequencies.
No, it isn't. RGB is just a color model. You cannot convert between a color model and a color space any more than you can covert between float and Euros.
In order to convert between RGB and L*a*b you need a color space. That could be an ICC profile, sure, but it could also just be any well defined color space like sRGB, Display P3, or bt2020
It seems like you two agree on everything except what "device dependence" means...? GP acknowledged that there is no direct conversion between RGB and Lab, distinguished RGB from sRGB (implying the former is not a colour space), and indicated that a colour space like sRGB does solve the problem (since TFA is about using it to solve the problem).
I'm a fan of HSLuv (https://www.hsluv.org/comparison/) for design work (when sRGB is fine and you don't need P3 colors). It's perceptual uniformity like Lab, but has familiar looking hue, saturation and lightness sliders instead like you'd see in a regular HSL color picker.
I've used HSLuv for an accessible palette editor (https://www.inclusivecolors.com/), so you get the familiarity of an HSL color picker, but unlike with HSL, when you change hue or saturation, the lightness and WCAG contrast stay the same, which makes HSLuv a great for exploring accessible color combinations without breaking the contrast you've set up already between your existing color pairs.
OKLCH is becoming a popular choice if you need P3 colors and perceptual uniformity because it's built into CSS now, but I find the colors pickers for OKLCH tend to look really complex and unfamiliar to use so I'm skeptical it's going to get popular with designers (it's mostly developers recommending OKLCH that are interested in the technical/coding part rather than the design part?). What are good choices if you want P3, perceptual uniformity, and an intuitive color picker UI?
If you ever need to generate a gradient between colors in any of your code, interpolating colors in the Lab color space is an awesome option. A simple linear interpolation of the components gives impressively beautiful results.
(Although, like several other commenters, I do recommend OKLab.)
The colors are all roughly the same lightness? If so, for people with color deficiency, you want the lightnesses to vary because people that can't easily tell certain hues apart should still be able to tell different lightnesses apart (although, if you need a color for lots of different pie slices, the lightnesses you can pick will start getting too close together and you might have to resort to patterns).
Interesting, I wanted to have random distribution of colors for small flower pixel art generator, also chose to use other color spaces, including Lab, so my colors would be more similar to human world perception, and not machine color representation
It probably sounds silly, but this minecraft youtuber did a nice job of explaining color spaces and Oklab in particular in a way that clarified things for me: https://youtube.com/watch?v=nJlZT5AE9zY
https://bottosson.github.io/posts/oklab/
For example, the Physical reality is the different frequencies of light. The Biological reality is that different types of cells on our retina respond with differing intensity to each of those frequencies.
So to my naive mind, a way of modeling color is to have (i) a forward model that map light frequencies to response intensities of the different types of cellular light receptors and (ii) an inverse model that estimates the frequency mix of light from the cellular responses.
That is, have two spaces (i) the light frequencity space (a list of tuple of frequency and intensity/power at that frequency) and the (ii) cellular response space.
Once we have these, we can go from a pigment or excited phosphor to a biological response in a two step process.
From (a) pigment/phosphor (+ frequency mix of illuminating light) to output light frequencities (b) from those frequencities to cellular response.
For all processing, make frequencities the base space to work in (allowing us to change/personalize the forward model).
Yes, the inverse model leads to an ill posed inverse problem, but we are now very knowledgeable about how to solve these.
The frequencies may need to discretized for convenience.
I am obviously a novice and don't know much about modeling color, but this way of modeling seems more grounded in the tangibles. This also gives a way to model how a color-blind person might perceive a picture.
Is the reason that we do not do it this way it's complexity ?
Eager to be illuminated (pun intended).
https://en.wikipedia.org/wiki/CIE_1931_color_space#Color_vis...
You'll see it is based on the physics and human biology, and is the basis for everything else.
The thing is, this color space isn't very useful for color calculations in the perceptual/subjective sense (e.g. if I just want to change one characteristic, like luminosity, without affecting the chromaticity), so we have transformations to more useful spaces like XYZ, Lab, etc.
There's also the fact human vision is a subjective/psychological phenomenon, so only frequency response is not enough: our vision can map different frequency responses to the same perceptual color (metamerism), our vision adapts color perception based on light source, etc.
EDIT Thanks again. This exactly like what I had in mind, in spirit at least.
If you're doing rendering work, like in a game, those do operate in a more physical domain. That's the so-called "Physically Based Rendering" or PBR that you might see if you hang out in game dev circles.
Of course artists can be very effective with whatever the toolsets they have learned, they sort of can transcend all the obfuscations and actually can express themselves. It's a bit hard to change everything then as the "user's transformations" are then baked in.
It's also true that mathematical models are often simplifications and one has to consider the whole end to end pipeline, where a more "accurate" transformation can yield a worse end result if applied blindly. I'm always reminded of the anecdote of when in the past, at some tv channel, they switched to more accurate weather forecast models. But the audience got worse forecasts, since the resident meteorologist was used to certain errors the old model produced and could compensate those, while the new model had maybe less errors in total but they were different. Happens all the time and why some engineer thinking something is objectively better might not actually be better for the customer...
Because of that, the layer of your model concerned with the physical reality of light doesn’t seem super useful for basic image input and output.
But of course this layer is incredibly useful for stuff like computer graphics, where it is indeed the case that physically based rendering is widely used for offline CGI rendering (and increasingly for realtime video game rendering).
We do model it according to perception, just only when it is useful. Other colour models allow different outputs, printing with coloured ink, displaying on screen by mixing coloured lights, categorising colour, finding how bright it is etc. all need different representations in numbers.
Your approach would make a lot of sense for sensors that are full spectrum analyzers, but the eye isn't one.
Yes because it's not a one to one map we cannot invert the map uniquely. But that's ok, we can maintain a distribution over the possible frequencities consistent with the response. That's how it's done in other areas of mathematics where similar non-bijections arise.
Much thanks for answering though, because I suspect I am asking a very basic question.
Unfortunately the problem is data acquisition (cameras), and data creation (artists). You need lots of data to figure out e.g. what a certain metal's spectrum is, and it's not nearly as clear-cut as just painting RGB values onto a box in a game engine.
For better or worse, all our tools are set up to work in RGB, regardless of the color space you happen to be using. So your physics-based approach would have the monumental task of redefining how to create a texture in Photoshop, and how to specify a purple light in a game engine.
I think the path toward actual photorealism is to use ML models. They should be able to take ~any game engine's rendered frame as input, and output something closer to what you'd see in real life. And I'm pretty sure it can be done in realtime, especially if you're using a GAN based approach instead of diffusion models.
To add to the general thread: the diverse color spaces are there to answer questions that inherently involve how a typical human sees colors, so they _have_ to include biology, that's their whole point. For example:
- I want objects of a specific color (because of branding), how to communicate that to contractors, and how to check it?
- What's a correct processing chain from capturing an image to display/print, that guarantees that the image will look the same on all devices?
It’s more like the L is an intensity/brightness factor, and then the a and b values corresponding to the two dimensions of opponent color that neurons capture in the thalamus one step after the eye
So, sure, this "4th dimension" (for normals) might be as simple as "candelas" - truly orthogonal, but one does hear an awful lot about "ambient" or "candela contrastive" (a term I just made up) kinds of effects. (EDIT: e.g. in color calibration of projectors in dark rooms vs. living rooms, for example, but I'm sure there are many.) I am just one person, but it feels like candela brightness matters for color perception. So, maybe luminous intensity is not actually exactly orthogonal. Maybe this is all covered in the 1931 CIE documents, though.
Because this isn't true. And there are multiple ways with completely different combinations of intesities of light at different frequencies to have a color that most people will see as the same color. This is because the color receptors of our eye overlap.
Of course, if you read back you will notice I am quite aware that the mapping is not a one to one bijection. Hence the need to solve the inverse problem.
One can maintains the inverse image (a set), or maintain a distribution over that, or certain statistics of that distribution. I have a link in my post about what inverse problems are.
sRGB like Lab* is device independent so a transformation between the two is possible.
RGB on the other hand is device dependent, and would therefore require a device ICC Profile to convert to Lab*.
No, it isn't. RGB is just a color model. You cannot convert between a color model and a color space any more than you can covert between float and Euros.
In order to convert between RGB and L*a*b you need a color space. That could be an ICC profile, sure, but it could also just be any well defined color space like sRGB, Display P3, or bt2020
I've used HSLuv for an accessible palette editor (https://www.inclusivecolors.com/), so you get the familiarity of an HSL color picker, but unlike with HSL, when you change hue or saturation, the lightness and WCAG contrast stay the same, which makes HSLuv a great for exploring accessible color combinations without breaking the contrast you've set up already between your existing color pairs.
OKLCH is becoming a popular choice if you need P3 colors and perceptual uniformity because it's built into CSS now, but I find the colors pickers for OKLCH tend to look really complex and unfamiliar to use so I'm skeptical it's going to get popular with designers (it's mostly developers recommending OKLCH that are interested in the technical/coding part rather than the design part?). What are good choices if you want P3, perceptual uniformity, and an intuitive color picker UI?
(Although, like several other commenters, I do recommend OKLab.)
* https://repo.autonoma.ca/repo/delibero/blob/HEAD/source/xsl/...
* https://repo.autonoma.ca/repo/delibero/blob/HEAD/source/xsl/...
An example pie chart is on page 33, section 9.2.3:
* https://repo.autonoma.ca/repo/delibero/raw/HEAD/docs/manual/...
* https://i.ibb.co/ymDLcPNj/pie-chart.png (screenshot)
The colours are harmonious, visually distinct, and not hard-coded to a fixed number of slices.