[Needs Feedback and Testing] Colorsmudge with smeared heightmap (slow but hi-quality version)

[Voronwe13]

I profusely disagree with this! Adding the lightness from the brush is explicitly different from smudging the existing height layer, and both should be possible.

For example, it should be possible to make a brush that pushes the paint around with its height information moving with the color. Right now, your proposal would make it so you can move the color, but the height information stays behind where it was, or can only be overwritten by the height information on the brush.

My proposal would be to allow smudging the height layer, so it can be moved with the color it belonged to. In order to prevent buildup, the amount that gets smudged can be inversely limited by the Lightness Strength value, since that’s how much lightness gets added by the brush. If there’s no lightness added by the brush, moving the existing height values around shouldn’t cause any problems.

[dkazakov]

Hi, @Voronwe13!

Could you please make some examples of the painted strokes? The new algorithm is in DK8 and the old one is in DK7, so you can test and compare them easily.

Adding the lightness from the brush is explicitly different from smudging the existing height layer, and both should be possible.

The only case when the new algorithm triggers is when lightness strength drops below value 0.2. The actual code is like that:

    const quint8 thresholdHeightmapOpacity = qRound(0.2 * 255.0);

    quint8 heightmapOpacity = qRound(opacity * lightnessStrengthValue * 255.0);

    if (heightmapOpacity < thresholdHeightmapOpacity) {
        heightmapOpacity =
                std::min(thresholdHeightmapOpacity,
                         std::max(smearRateOpacity(opacity, smudgeRateValue),
                                  colorRateOpacity(opacity, smudgeRateValue,
                                                   colorRateValue, maxPossibleSmudgeRateValue)));
    }

For example, it should be possible to make a brush that pushes the paint around with its height information moving with the color. Right now, your proposal would make it so you can move the color, but the height information stays behind where it was, or can only be overwritten by the height information on the brush.

Well, none of the packages in this thread smeared the heightmap layer. I didn’t change that in DK8. Smudging happens only in the color channel and heightmap channel is simply painted with COMPOSITE_OVER blending mode.

There might be some better approaches to that, e.g. use indirect painting for the heightmap (which allows COMPOSITE_ALPHA_DARKEN), but I don’t think that smearing the heightmap channel is going to work.

I tried an approach like that today. I tried erasing the height from the source dab position and adding height at the destination dab position. I guess that is something like you propose. I cannot say I liked the result, and, what is more, it didn’t solve the original problem (of zeroed lightness strength).

You can experiment with that by modifying code in KisColorSmudgeStrategyLightness::paintDab, it should be quite easy now.

[Voronwe13]

@dkazakov

I tried an approach like that today. I tried erasing the height from the source dab position and adding height at the destination dab position. I guess that is something like you propose. I cannot say I liked the result, and, what is more, it didn’t solve the original problem (of zeroed lightness strength).

The approach you mention here isn’t how smudging works for the color layer, which is why it doesn’t look right. The method for smudging the height layer needs to be exactly the same as for the color layer. The only difference is that to prevent the lightness buildup issue, you have to limit the smudging of the height layer by the inverse of the Lightness Strength value. If you do it right, it will solve the original problem, assuming you understand what the original problem actually was.

When Lightness Strength is 0, the lightness values of the brush should not be applied to the stroke at all. Your change breaks this expectation, and puts limits on what brush creators can do with the brushes they create.

I hate to take time away from my current work, but if I have to re-setup my dev environment to fix this issue, I guess I will.

[dkazakov]

The method for smudging the height layer needs to be exactly the same as for the color layer. The only difference is that to prevent the lightness buildup issue

You sound so confident as if you have already tried that or know about some papers about it… can you send me any links (perhaps on PM)?

you have to limit the smudging of the height layer by the inverse of the Lightness Strength value

But if we use the same algorithm to the heightmap as for the color layer, where will we apply lightness strength to? We will have “heightmap smudge rate” and “heightmap color rate”. How should the user configure them?

When Lightness Strength is 0, the lightness values of the brush should not be applied to the stroke at all.

You mean that when lightness strength is 0, then the brush should behave as a normal color one, right?

If, yes, then I’m not sure that is possible. It was possible in your previous implementation, when there was no separate heightmap channel, so one could apply this value to the specific dab. But now we write height values to a separate channel, so the values from different dabs (with different lightness strength values) are combined together and we cannot distinguish them anymore. Theoretically, we could somehow reduce the contrast of the heightmap dabs depending on lightness strength… but it seems like a different parameter actually…

I will try to experiment with your suggestion tomorrow, but I’m not sure I will have any success since I don’t fully understand how it is going to work

[Voronwe13]

@dkazakov

You sound so confident as if you have already tried that

I’m confident because that’s exactly how my version worked before the height information was separated into a different layer.

@dkazakov

But if we use the same algorithm to the heightmap as for the color layer, where will we apply lightness strength to? We will have “heightmap smudge rate” and “heightmap color rate”. How should the user configure them?

“Heightmap smudge rate” will be exactly the same as the Smudge Length value. While it’s possible that people could come up with creative uses for having that as a separate control, I think most brush creators would expect them to match, and it could be really confusing if they don’t, so I personally wouldn’t recommend separating it.

“Heightmap Color Rate” would basically be the Lightness Strength value. In fact, I suspect the heightmap can be drawn exactly the same way the color portion is done, except with LightnessStrength replacing ColorRate.

@dkazakov

You mean that when lightness strength is 0, then the brush should behave as a normal color one, right?

Exactly!

@dkazakov

Theoretically, we could somehow reduce the contrast of the heightmap dabs depending on lightness strength

That’s exactly what Lightness Strength is, so yes, you could do it that way… But I think just adjusting the opacity of m_heightmapPainter like you currently do will work just as well. Like I said above, I believe you can just paint the heightmap layer the same way you do the color layer, just directly substituting the LightnessStrength value in for the ColorRate value.

[dkazakov]

I’m confident because that’s exactly how my version worked before the height information was separated into a different layer

Well, your implementation worked very differently. It was far from how the real brush works (because of the missing height channel). If we get down to the physics of painting with a brush, then the simplest model will be like that:

1. The brush consists of a set of distinct bristles
2. Every bristle, depending on its position inside the brush and the current tilt/pressure value performs five separate jobs:
   1. Takes height from the canvas (by pressing into it)
   2. Takes color from the canvas (by adhering paint)
   3. Mixes the paint taken from the canvas into the own paint
   4. Puts the mixed color onto the canvas
   5. Increases the height of the canvas at the place of touching
3. The bristles can swap their position inside the brush

In this model, no “smudging” of the heightmap channel happens. Instead the height is taken from the source position with “erase” mode and then put back with “add” mode.

This model has a few technical complications though.

• the height channel in computer is limited, that is, the amount of paint you can put into a position is limited
• in real painting this limitation is overcome by the fact that the brush cannot hold infinite amount of paint (while our color rate option allows that)

So, basically, in real life the brush paints thanks to the equilibrium between “erase” and “normal” parts of the stroke. That makes the height of the strike to be somewhat uniform. I don’t know how to implement that properly in code. It seems like this balance should be achieved not by user settings, but some internal equation that we should solve.

My current implementation simplifies this model a lot. E.g. I don’t have erasing step. Instead I just paint a heightmap channel with “over” blending mode, which is basically a combination of “erase” and “add” (thanks to the presence of alpha channel).

“Heightmap Color Rate” would basically be the Lightness Strength value. In fact, I suspect the heightmap can be drawn exactly the same way the color portion is done, except with LightnessStrength replacing ColorRate.

Okay, I did a test try and I cannot say I love the result. Here is a screenshot with two strokes. One is painted with “smeared heightmap” channel and the other with “painted heightmap”. Can you guess which one is where?

The summary of my testing is the following:

1. “Smeared heightmap” implementation has less contrast and more smooth heightmap. Which is somewhat good.
2. “Smeared heightmap” solves the original problem of low values of lightness strength. That is also good.
3. “Smeared heightmap” makes the brush twice slower, which makes is hard to use. That is very bad.

Here is the branch with this test implementation: Commits · kazakov/colorsmudge-smeared-heightmap · Dmitry Kazakov / Krita · GitLab

Here is the testing package (Win64): krita-5.0.0-smeared-heightmap-dk1.zip — Яндекс Диск

[Voronwe13]

@dkazakov

Well, your implementation worked very differently.

Not really, the only difference was that the height channel was effectively baked into the color channel, so there was no choice but to smear it exactly as the color was. The only real difference is that LightnessStrength and ColorRate were necessarily tied to each other in that version. Separating the height channel out allows those to be independent, which allows lightness to still be added to brush strokes where ColorRate is 0, meaning smudging can have impasto brush marks (and this is key:) if the brush creator wants it to. If the brush creator has LightnessStrength going to 0, that means they want the brush to NOT create impasto marks at that pressure level. If it goes to 0, and the brush is still showing lightness/height/impasto values, that is a bug, and will be reported as such.

@dkazakov

If we get down to the physics of painting with a brush

This makes the big assumption that the only purpose for this feature is to mimic natural brushes. This is a digital painting medium, and artists shouldn’t be limited to how a developer thinks a natural brush works. Besides this, there are physical art tools that aren’t brushes, that brush creators may wish to mimic, but if the heightmap can’t be smudged, maybe they can’t (I’m not enough of a physical artist to say what these might be).

Either way, the original problem highlighted by @Mythmaker was not that there was no impasto added from the brush when he had Lightness Strength going to 0, because if he had it going to 0, he specifically didn’t intend for it to add impasto marks. The reason it didn’t look right was specifically that when it went to 0 with the color rate, the brush was smearing color, but not the heightmap, which looks weird. Your “solution” to just force impasto marks even when LightnessStrength is 0 just introduces buggy behavior to mask a shortcoming in the algorithm. It might make his specific scenario look more normal, but it breaks a lot of other scenarios… It’s lazy software design, and it will cause a lot of headaches if you actually release it like this.

@dkazakov

Can you guess which one is where?

When I get a chance, I’ll make an image that shows the difference quite clearly. Just making some random strokes with a single brush only proves the point that your solution can mask the issue in certain circumstances, not that it’s an actual solution.

@dkazakov

• “Smeared heightmap” implementation has less contrast and more smooth heightmap. Which is somewhat good.
• “Smeared heightmap” solves the original problem of low values of lightness strength. That is also good.
• “Smeared heightmap” makes the brush twice slower, which makes is hard to use. That is very bad.

So you tested it and determined that it actually is better, just slower? Wow, that’s exactly what I said earlier! Yes, smearing the height channel is slower, and I don’t think there’s a way around that. If you’re worried about making all the brushes slower, even for brushes that don’t care about smearing the height channel, then make “Smear Height” (or “Smear Lightness” if we want to keep consistent terminology) an option, right below the “Smear Alpha” option. Maybe even add a “(slower)” to the label, so users know there’s a performance penalty to the option. I’ll let you all decide if it should be checked or unchecked by default (that’s a tough choice between correct behavior and fast brushes). I’ll even do the write-up for the option in the manual if you want me to, with images highlighting the differences. But brush creators should absolutely have the option of smearing the height channel, even if it incurs a performance penalty.

[dkazakov]

If the brush creator has LightnessStrength going to 0, that means they want the brush to NOT create impasto marks at that pressure level

In the current implementation LightnessStrength == 0 means that impasto marks will still appear on the canvas, because of smudge rate, which is reused from the color part of the stroke.

If it goes to 0, and the brush is still showing lightness/height/impasto values, that is a bug, and will be reported as such.

Reported bugs are extra work for maintainers. If we accept this feature, we should be totally sure that the its benefit for the end user overweights extra maintenance work we should do to support that.

When I get a chance, I’ll make an image that shows the difference quite clearly

Please make an image for this usecase. The Windows package is attached to the previous message. If you need an AppImage package, please tell me, I’ll build one.

Just to make it clear, I’m not against this feature in general. I just cannot see clear benefits that would justify extra maintenance work we’ll have to do for it.

[TheFlow]

I got to test the latest build a bit. High quality height blending + smudge is crazy good for artists, like, for real. It is not so much about making it feel like a real brush, but the feeling of all real materials I can use to paint, if that makes sense, just feels more natural.

Without reading any one of the last few comments (I read them now, after I have tested it), this was the first thing that I noticed: if I set the lightness contrast to 0, it still creates an impasto effect. Performance is important, yes, but I think Peter is right here. Artistically speaking, I support what he is saying. In this case, creativity is more important than raw performance. My 4 year old PC can handle this reasonably well with a brush size of 250@A4. Photoshop’s Mixer brush tends to be pretty laggy and people still use it every day, in fact there is (not yet) anything like it. But you guys are right at it, right now.

And we’re very thankful for that

[dkazakov]

Hi, @all!

Could you please test the packages that implement the idea of @Voronwe13? The new algorithm seems to generate a smoother smudging result, although it is twice slower. It smears the height channel of the canvas as well (hence the slowdown).

Here are test packages. Please take into account that this option is always on in this version.

1. smeared-heightmap-dk1

• Win64: krita-5.0.0-smeared-heightmap-dk1.zip — Яндекс.Диск
• Linux: krita-5.0.0-prealpha-548bad2-x86_64.appimage — Яндекс.Диск

[Voronwe13]

Here are some images showing current issues:

DK9: Lightness Strength at 0% still presents as if it was 100%:

If you look closely at the beginning and ending of the 0% stroke, you can see that it looks like lightness is at 0%, but from the code I looked at earlier, I know that it was forced to be at 20% minimum, and it just built up from there. Both strokes were at ColorRate = 100%, so smudging doesn’t even factor into it. This is why forcing a minimum Lightness Strength value doesn’t work.

Smeared Heightmap DK1: Heightmap does not appear to actually be smeared:

This is using the package above that supposedly smears the Heightmap just like it does color. I haven’t looked at the code yet to see how it’s implemented, but you can see from this that the heightmap is not getting moved at all, when it is supposed to be. The second horizontal line (SR 50, CR 0 , LS 0) should show the dark and light parts of the vertical line shifted to the right, as if it were smeared over. As far as I can tell, only the color moved.

The third line shows what happens when Lightness Strength is at 100%, meaning that the lightness values of the brush should override the smeared values, and that appears to work correctly.

The fourth line is something that I wanted to test, because it’s the one case I’m not sure how it should work, as I see both options being something an artist might want. In this case, because smearing the heightmap doesn’t appear to be working correctly, it doesn’t show either option. What it should have shown (as I described the algorithm) is the lightness values of the underlying stroke shifted over, but still appearing. The other option is to have the Color Rate override the LS value, and paint the solid stroke across both lines. I think that’s the more correct option as ColorRate is understood. I believe to achieve this, the Smudge Length value would have to be multiplied by (1.0 - ColorRate) when applying the smudge copy to the heightmap. If we insist on not creating too many options to confuse artists, this is probably the one I would go with, but I’d love to see both as an option.

Maybe instead of having a checkbox for “Smear Height/Lightness”, have a radio box or dropdown with the choices “Don’t smear height” , “Smear Height, Color Rate preferred” , and “Smear Height, Lightness Strength preferred”. Or something like that. I’m not sure what the best wording for it would be, but I think it would be good to have those choices, as I could see both effects being desired. Even better would be to instead have a separate Curve Widget for how much the Color Rate affects the smudging of the heightmap versus how much the Lightness Strength affects it. Or I guess there could be a separate “Height Smudge Length” curve widget that would allow the same kind of control.

Maybe a separate topic, but since the Lightness values for the smudge engine seem to be only for impasto effect (aka paint height), I could see changing the “Lightness Strength” label to “Brush Height” or something like that instead. It still makes sense to keep it “Lightness Strength” for the pixel engine, since a primary use for it there is stamp brushes, where the concept of the light/dark values is used as shading for the stamps, rather than as any concept of height. But I don’t think people are making stamp brushes with the smudge engine, so it seems here it’s almost exclusively used for impasto effects.

Anyway, I’ll do more testing of the packages if the height smearing gets fixed. Thanks!

[Voronwe13]

In case you want to see what it should look like when the height is smeared, here is an image where I made the vertical lines with a Pixel engine brush instead of the smudge brush, so the lightness is baked in instead of in the height layer. This is what the second line in the image above should look like:

[Mythmaker]

I am following all this by the way - or at least trying to!

I did test the DK8 and felt it was improvement - using the same brush, it does smear the impasto already laid down if I sweep back over it:

I’ve just briefly tested that last build and the behaviour was very odd:

Anyway - you’re both doing great work so I hope you’re not feeling frustrated; Even as it is I’m getting usable results that are fun to paint with!

[Daishishi]

Hello all,
First let me thank the developers for this amazing Engine Update. Specially @dkazakov, I can’t even imagine the work you’re putting responding comments, coding and making test images.
My PC isn’t fast enough to fully test the new engine beyond some loose strokes. However I wanted to say my thanks, and give my two cents on this Height smudge issue.

Correct me if I’m wrong but this is what I understood from the new Engine
On DK8:

1. Color Rate: Draw color pixels on a color channel.
2. Smudge Length: ‘Smear’ the pixel from the color channel, and only the color channel.
3. Lightness Strength (LS): Draw the height pixels on a height channel. Here the strength is as the opacity of these pixels, and it is hard-coded to have at least 20% opacity.
4. The height channel is never smeared, at the lowest LS new height pixels are drawn over the previous ones.

On Smeared Height:

1. Same as DK8
2. Same as DK8
3. Lightness Strength (LS): Draw the height pixel into the height channel, no hard-coded minimum ‘opacity’.
4. The height channel is smeared, but at the rate of the LS. At 100% the smear is maximum, at 0% there is no smear of the height pixels.

If my understanding of how the Smeared Height works is correct, then it may explain the weird artifacts @Mythmaker experienced. Because not only the ‘impasto’ becomes blurry when smeared, but as the LS decreases the smear starts behaving weirdly. This also introduces the same issue present in DK7, where the height channel becomes ‘frozen’ with it’s last drawn pixels when LS is zero.
I experienced this ‘frozen’ impasto making a stroke with a regular RGBA brush, then after this I put the Color Rate at 0 (zero), left the Smudge Length as were. After this I activated Lightness Strength, turned Pen Settings OFF, and set the strength to 0 (zero). The stroke’s color was smeared but the impasto stayed intact.

Again, if I’m correct about the way the height smearing is being made, then this height smudge length should be at 100% at all time (at least on this test stage) or tied to the parameter Smudged Length.

Of course it could be a ‘bug’ or a less then optimal implementation, but I don’t have the understanding or knowledge to support this claim.

Hope this helps.

[Voronwe13]

@Daishishi

The height channel is smeared, but at the rate of the LS. At 100% the smear is maximum, at 0% there is no smear of the height pixels.

This isn’t quite right. If it’s implemented correctly, it’s inverted from that. As in, if Lightness Strength is at 100, it’s not getting smeared (or rather, you can’t see the smear because the height/lightness from the brush is completely overwriting it). When Lightness Strength is at 0%, it should be smearing the existing height channel at whatever Smudge Length is set to.

The problem you’re seeing where the color moves but the impasto stays intact, is because the height channel isn’t actually being smeared, as I discovered in my testing. Hopefully tomorrow I’ll have time to look through the code to see why, assuming @dkazakov doesn’t fix it before then.

[dkazakov]

Hi, @Voronwe13, @Daishishi and @Mythmaker!

Indeed in smeared-heightmap-dk1 there was a bug where heightmap was not correctly smeared. I have built a fixed package that should now properly smear the heightmap:

2. smeared-heightmap-dk2
   • Win64: krita-5.0.0-smeared-heightmap-dk2.zip — Яндекс.Диск
   • Linux: krita-5.0.0-prealpha-bfaec3c-x86_64-smeared-heightmap-dk2.appimage — Яндекс.Диск

[Voronwe13]

Cool, so far it looks like heightmap is smearing in this one. I am running into something that still doesn’t look right, but I’ll have to do some more testing in the morning to pinpoint what the issue is. I’ll post images here again when I get the chance. Thanks for the update!

[Voronwe13]

Here are two images of tests I did that show some issues in smeared_heightmap_dk2.

1. Smearing mode: Lightness Strength has almost no effect when Color Rate is 100, heightmap channel appears to be initialized to 0 instead of 0.5 gray.
   
   

The Lightness Strength issue shown is probably exaggerated by the extreme brush I’m using, and I haven’t tested yet with more normal brushes to see if it’s an issue with them. I’d be curious to try changing the blending mode to COMPOSITE_COPY instead, and see if that fixes the issue. I’ll do more testing with other brushes later to see if it’s a real issue or not.

The bigger issue here I see is where the (SR 50, CR 0, LS 0) brush crosses the LS 0 vertical line, and the (SR 50, CR 100, LS 0) brush crosses the blank spaces between the lines. I’m pretty sure the fact that these are black, and not the appropriate color (blue in the first case, red in the second), is because the heightmap channel is initialized as blank, or 0. For this to work properly, it needs to be initialized at Neutral Gray (0.5).

2. Dulling Mode: Lightness Strength has very little effect again (opposite to smearing, though), heightmap smearing seems to have issues.
   
   

Here, the Lightness Strength issue seems to be opposite of Smearing mode. It seems to work from 100% down to 50%, but after that it acts like Lightness Strength is basically 0. Again, I haven’t tested this with more normal brushes yet, so this might not be as much of an issue as it appears here. As with the other mode, I would be curious if changing the composite mode could fix the issue (assuming it is a real issue).

I’m not entirely sure what is happening with the heightmap smearing in this case. I’m assuming it’s using the dulling algorithm with the heightmap smear, and that might not work with this. Once the issue with initializing the heightmap to black is fixed, I’d like to try this again, and see if it still has issues. I have a feeling that even if that fixes it so it does something more predictable than what we see here, brush makers may still prefer to have the heightmap use smearing mode instead of dulling, even when dulling is the color mode chosen.

[EyeOdin]

I know I am off topic and talking about I probably don’t know enough off.

But by reading the comments it seems your using a height map to affect a baked canvas? Would it not be easier to have a zdepth on the canvas to interact with the brush depth? Impasto then would just be a add and smear could push height to the side by subtracting here and adding what was taken there. Also it would probably interact nicely with the tangent brush and filter. Erasing would take from the height until it was no more.

Interacting with baked information sounds more tricky than it should be probably. I know it would probably not work for imported stuff but for the brush build up sounds better to me.

[dkazakov]

Do you need Windows or Linux package?