[Needs Feedback and Testing] Colorsmudge with smeared heightmap (slow but hi-quality version)

[Voronwe13]

From the brief time I’ve spent looking at the code, the way the heightmap works here is that when drawing with the new smudge brush in Lightness mode, it draws the color to a color channel, and the lightness to the heightmap channel, and then the two are combined for the final presentation of the layer. The color and height information is kept separate for the purpose of the smudge brush in that mode. All other modes and brushes, etc, will only act on the combined version. Also, I did find that if you quit Krita and reopen it (or I assume if you just close the image and reopen it), the channels are no longer separated, so you could consider the height information to be baked at that point.

To do a proper impasto engine, you would actually have a truly separate height layer, and a movable light source, and the lightness/darkness values would be calculated based on the normals created by the height layer. We currently just mimic this by directly drawing the lightness/darkness values baked into the brush. This means that the highlights and shadows won’t be consistent if the brush rotates, but even so, it’s a surprisingly convincing effect.

It would be cool to have a true impasto engine, but I’m pretty happy with our pseudo-impasto version for now. Eventually we could do that, but I think it would require a new layer type and new brush engine and a light source tool. A lot of work, for a marginal improvement.

[Voronwe13]

Windows for me. Thank you!!

[EyeOdin]

I understand then. But don’t forget you have Matcaps that can bypass light setups if that would be possible.

[dkazakov]

Here is the package with heightmap device initialized to QColor(127,127,127,255). Though I don’t think that will change anything, because this device has alpha channel and before the patch the default pixel was totally transparent and the lightness application formula discards fully transparent pixels (by premultiplication).

krita-5.0.0-smeared-heightmap-dk3.zip

Btw, @Voronwe13, do you know that we have a new way of building Krita on Windows that lets you skip building all the deps by reusing prebuilt ones from the binary factory? Here is the link.

[Voronwe13]

Thank you for the new package! It actually fixed a lot of the problems I showed. I’m curious, do you think the heightmap really needs 32bit pixels? I feel like it should work perfectly in ALPHA8 mode, and that might improve the performance. I guess I’ll try to set up my dev environment, especially if building on Windows is faster now, so I can test some of these things myself before making suggestions here…
Anyway, here are the same two tests I did above, with the DK3 package:

1. Smearing mode: The heightmap channel issue is fixed. Surprisingly, this seems to improve the Lightness Strength issue slightly too, even though you didn’t say you changed anything other than initializing the heightmap to neutral gray.
   

   

2. Dulling mode: Still just erases when ColorRate is 0 to 5%, but now when Color Rate is over 5%, it draws as if there’s no height information beneath it. I think I need to test dulling mode in the other brush modes, because there might be issues that aren’t related to Lightness mode… Also, the Lightness Strength behavior doesn’t seem to have changed at all here, unlike in smearing mode.
   

   

[Mythmaker]

@Voronwe13

Eventually we could do that, but I think it would require a new layer type and new brush engine and a light source tool. A lot of work, for a marginal improvement.

Is there any way to incorporate existing image filters into the brush engine, such as the ‘phong bump map’, or ‘height to normal map’? - Those are two I used in the impasto image templates I posted last year.

I imagine the current approach is probably more efficient though. I think once the kinks are ironed out it will be my preferred method in most instances.

Thinking about it - I’ve known how to do the layered approach for years, but never use it. I think I’m much more likely to use brushes with impasto effect built in though.

[Voronwe13]

One more experiment image that may shed some light on what’s happening with dulling mode. I redid the above image, but using Smudge Length of 10% for everything, instead of 100% for the vertical lines, and 50% for the horizontal, and this produces almost perfect results.

Both the lightness strength for the vertical lines and the smearing created by the horizontal lines look like what I would expect from dulling mode. We might need to play around with how we use the Smudge Length value in the height smearing algorithm to get expected results at all values of Smudge Length…

[dkazakov]

This thread is a fork of the original thread about the new colorsmudge engine for Krita 5.0

[dkazakov]

Hi, @all!

Since @Voronwe13 confirms the package works as expected, I would like to ask for the feedback from the painters about that package. Specifically, I would like to know answers to the following questions:

1. Does smeared-heightmap-dk3 package gives better experience to the painter (in comparison to dulling-fix-dk9)?
2. What can you do in smeared-heightmap-dk3 that you cannot do in dulling-fix-dk9?
3. Is the speed in smeared-heightmap-dk3 acceptable?

UPD:
ping, @RamonM

[Voronwe13]

I did have one thought on how this might be done without the performance penalty, though I think it loses a little functionality this way…

Basically, this could be done similarly to how my original version of this feature worked, where the smudging is applied directly to the presentation layer, instead of using the height layer. This would cut out at least one composite call. The main limitation is that to prevent lightness buildup, the Lightness Strength would have to be limited by the Color Rate and Smudge Length values.

My original version effectively had the Lightness Strength limited by Color Rate, so if Color Rate was low, the lightness/impasto of the brush was also low. This prevented the lightness buildup when smudging paint with lightness, but didn’t allowing impasto marks on colorless smudging. However, I think lightness can be allowed even at 0% Color Rate as long as it isn’t higher than (100% - Smudge Length), and not result in lightness buildup, so we can still get some of that effect.

The way I see this implemented is:

1. Apply color and smudging as if done by a (new algorithm) mask smudge brush, so it smudges the existing impasto.

2. Apply brush lightness at final_opacity = Opacity * Lightness Strength * max(Color Rate, 1.0 - Smudge Length)

The main limitation with this approach is that you wouldn’t be able to apply brush impasto when Color Rate is low/0 and Smudge Length is at or near 100%. This would have the practical effect that the more you want to smudge existing height/impasto, the less you’ll see of brush height/impasto. This may have to be indicated with a message or tooltip on the option to smudge height, and will definitely have to be documented in the manual.

I haven’t actually tested this yet, to see if it would actually look good and not have the lightness buildup issue, it just seems like it would work in theory, based on my experience when I worked on the feature earlier. If I get some time, I’ll try writing a version that works this way and see if it produces decent results.

Also, I would only want this to be done when “Smudge Existing Impasto” (or whatever the option is eventually called) is added as an option. I still want to see a version that has an option for both impasto smoothing and and option for paint overlaying impasto, like I described in the other thread. So the height layer is still necessary for those modes, it just wouldn’t be used for this mode.

[Voronwe13]

I’ve successfully implemented this version. Surprisingly, it’s still slow, even though it appears to have fewer composite calls than even the non-heightmap-smearing version. I haven’t tried any benchmarks yet to see how it truly compares, though.

I’ve mostly completed a couple of other algorithms for how to handle this smudge brush, and I plan to put together a version that has every competing algorithm available through a dropdown in the Paint Thickness (formerly Lightness Strength) option. I don’t expect all of them to end up in the final version, since I’m finding that some of them have some fundamental problems. But this way, people testing can easily switch algorithms without having to keep track of several versions of Krita, and we can get feedback on all of them to see how best to go forward.

[dkazakov]

Hi, @Voronwe13!

I have tested your branch. I cannot say I love the result. Here is the screenshot:

This test shows how light stroke interacts with a heavy stroke when painted over it:

• right side: current version of “dulling-fix-dk9”.
• top-left side: “smear existing heightmap” mode
• bottom-left: “overwrite existing mode”

In both new modes, the heightmap is just blurred.You cannot emulate light brush strokes with it.

I think now I know how to formulate what I want from this algorithm. Here is a small draft that illustrates it:

As far as I can tell, the version in “dulling-fix-dk9” under some conditions can resemble behavior of the lower image, that is why I like that so much But it might be that it will be improved if we implement proper clipping of the brush…

[dkazakov]

Okay, I did a test implementation of the method I painted on paper, but I cannot say I like it either

[tiar]

Based on the screenshot, I think I like the right one too…