Paint like color mixing (Kubelka-Munk)

[rvanwijnen]

Hello all, my name is Ronald van Wijnen and I’m the author of spectral.js GitHub.

Spectral.js is a javascript library for realistic paint like color mixing using the Kubelka-Munk equations.
It is already being used by the generative art scene and is well received.

With the help from a Krita developer (thank you Dmitry!) I have integrated this library in Krita.

Some technical details: a sRGB triplet is converted to spectral space by combining three (red, green and blue) premade reflectance curves (380 to 730 wavelengths in 10 nm steps) where the input sRGB triplet is the weight for the generated destination spectral curve.
These generated curves are than mixed using the Kubelka-Munk equations, which give a nice saturated color mix with the characteristics of real life paint.

To get a perceptually even distribution between the two colors spectral.js uses a concentration function which calculates the concentration for a color using the lightness value (a lighter color needs more ‘parts’ for the mix then a darker color).

The round-trip (sRGB → spectral → XYZ → sRGB) error is 0% for any given input.

A mixed color can be out of the sRGB gamut, this was the case for 0.025% in my tests, when out of gamut the deviation is maximum 0.05%, therefor clipping is used, gamut mapping is too computational heavy for this little deviance.

The reason for this little out of gamut results is because of the smooth and optimized red, green and blue base spectral reflectance curves.

Pictures say more than a thousand words:

I have been in discussions regarding this topic on this forum before and those discussion were, although unfruitful, very informative and were the basis for my implementation (thank you @urzeye for laying the groundwork).

I’m not very familiar with the Krita source code and could use some help on optimizing this and further development.

The merge request can be found here: invent.kde.org
A windows executable for testing can be found here: disk.yandex.ru
(Note by Krita developer: colour smudge brush in this build does not work in non-spectral mode, this is not a replacemtent for the current version and should be used for testing only.)

To use spectral blending create a new layer and choose Spectral as blending mode, the opacity will control the mixing factor.

Other interesting topics on this forum:
I realized the spectral mixing of mypaint in krita - Develop - Krita Artists (krita-artists.org)
Rebelle 5 Pigments - Physical Color Mixing - General Forums / off-topic talks - Krita Artists (krita-artists.org)

One more note regarding MyPaint: MyPaint has spectral mixing already but they are using the Weighted Geometric Mean (WGM) for their mixing function, in my opinion the WGM (although easy to implement and not computational heavy) give desaturated colors as seen in this example:

To my best knowledge Krita will be the first open source painting software which utilizes the Kubelka-Munk algorithm and this is big in my opinion!

Sorry for being this lengthy and technical but I’m a programmer and have been researching this for over a year now.

Looking forward to the discussion,

Ronald

[urzeye]

Hi, @rvanwijnen, thanks for creating spectral.js, I’ve just downloaded the test package and simply done some testing, I’ve found a problem, when painting with the smudge brush, the final colour seems to lean towards green no matter what colour is applied, I’m not sure if it’s just me that’s having this problem.

[rvanwijnen]

Seems to be working fine when I use it:

Do you have every color on it’s own layer? With the layer blend mode set to Spectral?
When you’re mixing it’s best to create a new layer for the color you want to mix with.

[Dynline]

Works for me on different layers as well, like @rvanwijnen is saying.

Blend mode is also available on brushes. I just tested it and it works on the same layer this way (won’t blend with the colors below unless the layer is also spectral).

If you prefer to mix colors on the same layer try having the new spectral mode on the blend brush @urzeye

[urzeye]

No, I didn’t, because this is how the smudge brush works. If someone else can replicate the above problem, I think it may be a bug.

[urzeye]

The mixing mode of the smudge brush is spectral copy, which is the same as the spectral mixing mode in terms of color mixing, except that it uses the sampled color as the painting color. Ideally, smudge brushes applying the same color will keep the color unchanged, otherwise smudge brushes will not be able to use it.

[urzeye]

In fact, this problem has always existed in spectral.js, which is also one of the reasons why I did not use spectral.js data. I can understand your desire to use spectral.js data for krita, but this is indeed a problem that needs to be solved, otherwise artists will not be able to use this function properly.

[Dynline]

I can’t replicate on the canvas the “tending toward green” you say yet, even after using random colors. I do however see in the brush preset that if the checkbox is active, then the brush preview is very slightly green.

With:

Without:

I’m not technical enough to really know how it works though

[Dynline]

normal layer but new checkbox active

Spectral layer, new checkbox active

It is very evident if you use grey on grey

[rvanwijnen]

Which checkbox are you refering to?

[Dynline]

“Use new smudge algorithm” the one from urzeye’s screenshot

[rvanwijnen]

Ah I see it an can replicate it.

This has nothing to do with spectral.js though and the problem lies more with that smudge setting. There is no tending towards green in spectral.js.
If the checkbox is turned off it works perfectly.

I will make a post on the merge request and ask if the Krita developer has an idea on where this comes from.

[rvanwijnen]

Weird part is that the brush preview on the top left corner is tending towards green in the preview.
That part is not rendered using spectral at all, spectral is only used on the layer.

I think it’s a bug with the new smudge engine and we think it’s related while it’s not.

[dkazakov]

In this branch spectral is always enabled for a part of the colour smudge brush, whatever your settings are

[rvanwijnen]

This can be fixed right?

[wolthera]

Just so people know, this is the greeness issue:

If you enable ‘new smudge algo’ and turn down the spacing, you get this:

while it should look like this:

To fix the problem we first need to understand where in the spectral algorithm the greeness is coming from.

[urzeye]

This issue is caused by the data from spectral.js. If you could build the merge branch I submitted, this issue would not occur.

[rvanwijnen]

It has nothing to do with spectral, spectral will never tend towards green if there is no dominant green in the colors.

The preview uses alternating bars with colors [140, 140, 140] and [80, 80, 80] if I mix those in spectral.js in my test suite there is no dominant green.

It could be a bug in the Spectral_Copy composite function in Krita but it has nothing to do with the spectral mixing itself.
This wil have to be looked at but don’t blame the spectral data.

Spectral.js is used a lot in the generative art scene and I have never ever heard something about tending towards green.

I am positive we will get this sorted out with the Krita devs.

[urzeye]

The smudge brush will use the colour of the spectral mixing result of the previous nib position as the next nib position mixing colour, if there is an offset in the previous mixing colour, this offset will be accumulated to the next mixing colour, and the offset will be shown after accumulating to a certain level. Spectral.js doesn’t show the offset because the mixing of spectral.js is a single mixing, and the mixing of the smudge brush is each brush tip is blended once, even though multiple blends may occur over a small distance. Spectral copy does not change the way the colours are blended in the spectral mode.

[rvanwijnen]

So basicly you’re saying it’s not the spectral data or spectral.js but the way the smudge brush engine works.

Sounds like there is an easy way to fix this and never use spectral for smudge but use the normal blend.

Spectral mixing should only happen on the layer in my opinion as the alpha/opacity determines the mix factor.

All other functionality should be handled by the already available functions in Krita.

Smudge in normal COPY mode and then apply the mixing on the layer.