I realized the spectral mixing of mypaint in krita

Develop
204

I understand what you are saying!
In the end, I’m just an artist and I have no idea of the math and knowledge that goes behind making such a tool! I’ll be more than happy to wait for proper implementation of the code, I was simply sharing my enthusiasm to see dedication and thinking going behind a request that has been going on for some time.

I simply wish to see a pigment type of implementation color mode to Krita software and it felt only natural to root for the people trying their best to come up with solutions to this task.
I wnat what’s best for everybody!
So please have fun while making this and I’ll be looking forward to it!

205

Don’t worry, this merge request hasn’t been merged yet. It’s just in the testing discussion, so even if there are some issues, it’s okay. Actually, I’m not good at coding and I look forward to your correct implementation

206

Your presentation is also interesting, but it should be noted that it should be used in a linear RGB configuration file, otherwise mixing blue and yellow will generate black. Of course, this is only the correct way I think. Everyone wants different effects, and being able to achieve their desired results is the best

2 Likes
207

No, I didn’t actually do any difficult work. @rvanwijnen 's open source spectro.js is the most important one, and in addition, this thread has also helped me solve many problems

208

I love your enthusiasm as an artist!
And I would love to have it implemented in Krita!

But if it gets implemented I want to have it done right, you can only have one first impression so let’s do this right

8 Likes
209

If you need any help regarding krita development feel free to join the IRC channel or make a post here most of the core developers are on the forum :+1:t4: You can also ping me anytime, I am not a coder but I will help in any way I can.

2 Likes
210

If you can get me in contact with a developer willing to implement this that would be awesome

211

@urzeye is handling the merge request, so you can post your thoughts in this thread itself. Other developers can also add or interact here.

212

I’ve managed to port my code to Python.
I’m not a Python developer so no guarantees but what I tested is accurate.

Input are two color arrays in srgb and mixing factor t.

##  MIT License
##
##  Copyright (c) 2023 Ronald van Wijnen
##
##  Permission is hereby granted, free of charge, to any person obtaining a
##  copy of this software and associated documentation files (the "Software"),
##  to deal in the Software without restriction, including without limitation
##  the rights to use, copy, modify, merge, publish, distribute, sublicense,
##  and/or sell copies of the Software, and to permit persons to whom the
##  Software is furnished to do so, subject to the following conditions:
##
##  The above copyright notice and this permission notice shall be included in
##  all copies or substantial portions of the Software.
##
##  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
##  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
##  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
##  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
##  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
##  FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
##  DEALINGS IN THE SOFTWARE.

SIZE = 38
GAMMA = 2.4
EPSILON = 0.00000001

SPD_C = [0.96853629, 0.96855103, 0.96859338, 0.96877345, 0.96942204, 0.97143709, 0.97541862, 0.98074186, 0.98580992, 0.98971194, 0.99238027, 0.99409844, 0.995172, 0.99576545, 0.99593552, 0.99564041, 0.99464769, 0.99229579, 0.98638762, 0.96829712, 0.89228016, 0.53740239, 0.15360445, 0.05705719, 0.03126539, 0.02205445, 0.01802271, 0.0161346, 0.01520947, 0.01475977, 0.01454263, 0.01444459, 0.01439897, 0.0143762, 0.01436343, 0.01435687, 0.0143537, 0.01435408]
SPD_M = [0.51567122, 0.5401552, 0.62645502, 0.75595012, 0.92826996, 0.97223624, 0.98616174, 0.98955255, 0.98676237, 0.97312575, 0.91944277, 0.32564851, 0.13820628, 0.05015143, 0.02912336, 0.02421691, 0.02660696, 0.03407586, 0.04835936, 0.0001172, 0.00008554, 0.85267882, 0.93188793, 0.94810268, 0.94200977, 0.91478045, 0.87065445, 0.78827548, 0.65738359, 0.59909403, 0.56817268, 0.54031997, 0.52110241, 0.51041094, 0.50526577, 0.5025508, 0.50126452, 0.50083021]
SPD_Y = [0.02055257, 0.02059936, 0.02062723, 0.02073387, 0.02114202, 0.02233154, 0.02556857, 0.03330189, 0.05185294, 0.10087639, 0.24000413, 0.53589066, 0.79874659, 0.91186529, 0.95399623, 0.97137099, 0.97939505, 0.98345207, 0.98553736, 0.98648905, 0.98674535, 0.98657555, 0.98611877, 0.98559942, 0.98507063, 0.98460039, 0.98425301, 0.98403909, 0.98388535, 0.98376116, 0.98368246, 0.98365023, 0.98361309, 0.98357259, 0.98353856, 0.98351247, 0.98350101, 0.98350852]
SPD_R = [0.03147571, 0.03146636, 0.03140624, 0.03119611, 0.03053888, 0.02856855, 0.02459485, 0.0192952, 0.01423112, 0.01033111, 0.00765876, 0.00593693, 0.00485616, 0.00426186, 0.00409039, 0.00438375, 0.00537525, 0.00772962, 0.0136612, 0.03181352, 0.10791525, 0.46249516, 0.84604333, 0.94275572, 0.96860996, 0.97783966, 0.98187757, 0.98377315, 0.98470202, 0.98515481, 0.98537114, 0.98546685, 0.98550011, 0.98551031, 0.98550741, 0.98551323, 0.98551563, 0.98551547]
SPD_G = [0.49108579, 0.46944057, 0.4016578, 0.2449042, 0.0682688, 0.02732883, 0.013606, 0.01000187, 0.01284127, 0.02636635, 0.07058713, 0.70421692, 0.85473994, 0.95081565, 0.9717037, 0.97651888, 0.97429245, 0.97012917, 0.9425863, 0.99989207, 0.99989891, 0.13823139, 0.06968113, 0.05628787, 0.06111561, 0.08987709, 0.13656016, 0.22169624, 0.32176956, 0.36157329, 0.4836192, 0.46488579, 0.47440306, 0.4857699, 0.49267971, 0.49625685, 0.49807754, 0.49889859]
SPD_B = [0.97901834, 0.97901649, 0.97901118, 0.97892146, 0.97858555, 0.97743705, 0.97428075, 0.96663223, 0.94822893, 0.89937713, 0.76070164, 0.4642044, 0.20123039, 0.08808402, 0.04592894, 0.02860373, 0.02060067, 0.01656701, 0.01451549, 0.01357964, 0.01331243, 0.01347661, 0.01387181, 0.01435472, 0.01479836, 0.0151525, 0.01540513, 0.01557233, 0.0156571, 0.01571025, 0.01571916, 0.01572133, 0.01572502, 0.01571717, 0.01571905, 0.01571059, 0.01569728, 0.0157002]
CIE_CMF_X = [0.00006469, 0.00021941, 0.00112057, 0.00376661, 0.01188055, 0.02328644, 0.03455942, 0.03722379, 0.03241838, 0.02123321, 0.01049099, 0.00329584, 0.00050704, 0.00094867, 0.00627372, 0.01686462, 0.02868965, 0.04267481, 0.05625475, 0.0694704, 0.08305315, 0.0861261, 0.09046614, 0.08500387, 0.07090667, 0.05062889, 0.03547396, 0.02146821, 0.01251646, 0.00680458, 0.00346457, 0.00149761, 0.0007697, 0.00040737, 0.00016901, 0.00009522, 0.00004903, 0.00002]
CIE_CMF_Y = [0.00000184, 0.00000621, 0.00003101, 0.00010475, 0.00035364, 0.00095147, 0.00228226, 0.00420733, 0.0066888, 0.0098884, 0.01524945, 0.02141831, 0.03342293, 0.05131001, 0.07040208, 0.08783871, 0.09424905, 0.09795667, 0.09415219, 0.08678102, 0.07885653, 0.0635267, 0.05374142, 0.04264606, 0.03161735, 0.02088521, 0.01386011, 0.00810264, 0.0046301, 0.00249138, 0.0012593, 0.00054165, 0.00027795, 0.00014711, 0.00006103, 0.00003439, 0.00001771, 0.00000722]
CIE_CMF_Z = [0.00030502, 0.00103681, 0.00531314, 0.01795439, 0.05707758, 0.11365162, 0.17335873, 0.19620658, 0.18608237, 0.13995048, 0.08917453, 0.04789621, 0.02814563, 0.01613766, 0.0077591, 0.00429615, 0.00200551, 0.00086147, 0.00036904, 0.00019143, 0.00014956, 0.00009231, 0.00006813, 0.00002883, 0.00001577, 0.00000394, 0.00000158, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
XYZ_RGB = [[3.24306333, -1.53837619, -0.49893282], [-0.96896309, 1.87542451, 0.04154303], [0.05568392, -0.20417438, 1.05799454]]

def linear_to_concentration(l1, l2, t):
    t1 = l1 * (1 - t) ** 2
    t2 = l2 * t ** 2

    return t2 / (t1 + t2)
    
def spectral_mix(color1, color2, t):
    lrgb1 = srgb_to_linear(color1)
    lrgb2 = srgb_to_linear(color2)

    R1 = linear_to_reflectance(lrgb1)
    R2 = linear_to_reflectance(lrgb2)

    l1 = dotproduct(R1, CIE_CMF_Y)
    l2 = dotproduct(R2, CIE_CMF_Y)

    t = linear_to_concentration(l1, l2, t)

    R = [0] * SIZE

    for i in range(SIZE):
        KS = (1 - t) * ((1 - R1[i]) ** 2 / (2 * R1[i])) + t * ((1 - R2[i]) ** 2 / (2 * R2[i]))
        KM = 1 + KS - (KS ** 2 + 2 * KS) ** 0.5

        R[i] = KM
    
    xyz = reflectance_to_xyz(R)
    rgb = xyz_to_srgb(xyz)

    return rgb

def uncompand(x):
    return x / 12.92 if x < 0.04045 else ((x + 0.055) / 1.055) ** GAMMA

def compand(x):
    return x * 12.92 if x < 0.0031308 else 1.055 * x ** (1.0 / GAMMA) - 0.055

def srgb_to_linear(srgb):
    r = uncompand((srgb[0] + EPSILON) / 255)
    g = uncompand((srgb[1] + EPSILON) / 255)
    b = uncompand((srgb[2] + EPSILON) / 255)

    return [r, g, b]

def linear_to_srgb(lrgb):
    r = compand(lrgb[0] - EPSILON)
    g = compand(lrgb[1] - EPSILON)
    b = compand(lrgb[2] - EPSILON)

    return [round(clamp(r, 0, 1) * 255), round(clamp(g, 0, 1) * 255), round(clamp(b, 0, 1) * 255)]
    
def reflectance_to_xyz(R):
    x = dotproduct(R, CIE_CMF_X)
    y = dotproduct(R, CIE_CMF_Y)
    z = dotproduct(R, CIE_CMF_Z)

    return [x, y, z]
    
def xyz_to_srgb(xyz):
    r = dotproduct(XYZ_RGB[0], xyz)
    g = dotproduct(XYZ_RGB[1], xyz)
    b = dotproduct(XYZ_RGB[2], xyz)

    return linear_to_srgb([r, g, b])

def spectral_weights(lrgb):
    w = c = m = y = r = g = b = 0

    if lrgb[0] <= lrgb[1] and lrgb[0] <= lrgb[2]:
        w = lrgb[0]

        if lrgb[1] <= lrgb[2]:
            c = lrgb[1] - lrgb[0]
            b = lrgb[2] - lrgb[1]
        else:
            c = lrgb[2] - lrgb[0]
            g = lrgb[1] - lrgb[2]
    elif lrgb[1] <= lrgb[0] and lrgb[1] <= lrgb[2]:
        w = lrgb[1]

        if lrgb[0] <= lrgb[2]:
            m = lrgb[0] - lrgb[1]
            b = lrgb[2] - lrgb[0]
        else:
            m = lrgb[2] - lrgb[1]
            r = lrgb[0] - lrgb[2]
    elif lrgb[2] <= lrgb[0] and lrgb[2] <= lrgb[1]:
        w = lrgb[2]

        if lrgb[0] <= lrgb[1]:
            y = lrgb[0] - lrgb[2]
            g = lrgb[1] - lrgb[0]
        else:
            y = lrgb[1] - lrgb[2]
            r = lrgb[0] - lrgb[1]

    return [w, c, m, y, r, g, b]

def linear_to_reflectance(lrgb):
    weights = spectral_weights(lrgb)

    R = [0] * SIZE

    for i in range(SIZE):
        R[i] = (
            weights[0]
            + weights[1] * SPD_C[i]
            + weights[2] * SPD_M[i]
            + weights[3] * SPD_Y[i]
            + weights[4] * SPD_R[i]
            + weights[5] * SPD_G[i]
            + weights[6] * SPD_B[i]
        )

    return R

def dotproduct(a, b):
    return sum(x * y for x, y in zip(a, b))

def clamp(value, min_value, max_value):
    return min(max(value, min_value), max_value)

Usage:

spectral_mix([0, 33, 133], [252, 210, 0], 0.5)

Full source available on github: GitHub - rvanwijnen/spectral.js

6 Likes
213

That’s awesome @rvanwijnen! You’re right it works even better. Yellow on blue no longer turns black before turning green:

I’ve updated my plugin :angel:

8 Likes
214

I would not hold my breath for that. This is something that has been tried before, but developers end up with the conclusion that it’s too computationally intensive to be viable. I’m not sure if this is the case, but I would not be surprised if it does not get in at all.

Progress does look good though.

1 Like
215

Then is there a way that I can add this locally on my machine with the code provided by @rvanwijnen ?
The windows beta version @urzeye released earlier is all good, but wont allow me to load Pigment O plugin…
Any help would be more than welcome!

216

I can’t say anything about that. I have no clue of the inner workings of Krita.

Can someone from the dev team confirm or deny this before I put any time in trying to implement this.

All te code is available so a dev could say something meaningful about this.

I don’t think this has been tried before, not to this extent.

I believe Spectral.js is the first openly available library implementing Kubelka-Munk in a simple and easy to use manner.

1 Like
217

Well let us ping them: @tiar / @halla / @dkazakov / @wolthera / @scottyp / @sh-zam, all @knowing-users and developers I’ve forgotten, can you support @rvanwijnen with your knowledge, please?

Michelist

8 Likes
218

I understand the concern of not wanting to merge in everything out of risk of making the program too bloated, but I really don’t see the harm of implementing a working system into the program. There are people clearly interested in using this (the activity of this thread is proof of that). If it’s too computationally intensive, just specify in the name that it’s experimental or intensive so the user can make an informed choice on if they want to use it or not. It’s not like this is going to be the default blending behavior or anything, people with lower end machines can just choose to not use it.

7 Likes
219

If only Krita could compile with Visual Studio, I’d add so many features… :cry:
/OT

220

The problem is not that, but rather that algorithms that are not fast enough will ultimately result in many people (especially computers with poor performance) not being able to use them properly. Especially for applying brushes.

The implementation of krita itself was many years ago and was later removed. Subsequently, we focused on mypaint, but it was still slow.

But it’s not that a fast algorithm is impossible, for example, the research on Mixbox mainly focuses on how to save performance. In addition, Artstudio and CSP have also added pigment blending, but I have not tested their performance yet.

221

I would once again invite @Deevad, as well as @RamonM, to give their point of view on whether “blending” needs special settings/controllers to show the difference between transparent watercolor and oil paint(filled with opaque white pigment) or whether this difference can be embodied by the already existing overlay modes.
I apologize if this question has already been discussed

1 Like
222

Not exactly, the script I posted is just a test, it’s what I use to check the data, so it doesn’t incorporate the linear rgb vs non linear rgb conversion function, if you use @rvanwijnen’s script at 16 bits now it will not get the correct result, if you add the conversion function to the script I provided it will work fine at 8 bits and not produce black, the reason I didn’t include the relevant conversions, as I explained in my reply above, is that it’s still slow and the linear rgb vs non linear rgb conversion may slow it down further because it uses the pow function

2 Likes
223

Actually @rvanwijnen said I was wrong, I’m not actually sure, if I said I used Z to calculate the luminosity, that’s totally subject to change at any time, I’ve replied above, it depends on whether people prefer the effect of spectral.js, if not I’ll go ahead and change it, in fact it’s been changed in the script above, apart from that I think with spectarl.js The biggest difference is that it uses different spectral data so the blending effect is a little different, they differ as the comparison image I have provided above shows, then there is the accuracy, spectral.js 2.0 is slightly less accurate, the output RGB is only accurate to the fourth decimal place, there is still some discolouration when applying white but it is very subtle, spectral.js version 1.0 is a bit less accurate than 2.0, applying white with the smear tool will turn yellow and then black straight away, as mentioned in my tests above, the code I’ve submitted now has an accuracy of 8 decimal places, which basically meets the high accuracy requirement, but of course the difference between them doesn’t look that great to the naked eye, so I’d prefer people to be able to choose between the blending results, if possible I I hope to compile two versions for you to vote on, one with spectral.js and one with the code I’ve submitted, as everyone’s preferences may be different.

This is a comparative graph of the data output:

Snipaste_2023-04-15_08-02-31
Snipaste_2023-04-15_08-02-311562×955 26 KB

On the left is spectral.js 2.0, on the right is the version I’m submitting now

This is a comparison of the actual blending effect:

test
test852×525 56 KB

On the top is spectral.js 2.0, on the bottom is the version I’m currently submitting

For the blending I would prefer that everyone could test it themselves using krita afterwards and vote on the final version to be merged to respect the different possibilities.

2 Likes