Looking for a volunteer to check out the FSC plugin

[tiar]

The Fast Sketch Cleanup plugin project is nearly done, but since it was done with the partnership of Intel, we need the final step, which is giving it to a final “power user” (as in, someone who uses Krita), and gather the feedback. It would have to be in a form of a blog post with screenshots or screen recording and pictures, or maybe a video.

It could be something so simple as just taking a sketch or two, testing some options in the plugin, gather the results, and making some conclusion. Something just like I did in the “Best workflow” section of the blog post, but with some summary and your thoughts.

From what I can tell, it could be useful to extract sketches from photographs of pencil sketches, or to use the Colorize Mask or Fill Tool on a messy sketch without having to clean it up if you don’t make a lineart piece, or as a good base for lineart (but will still require quite some work), or could be useful for people without a pressure-sensitive tablets to at least fake it a little bit.

The plugin is based on AI, but it’s not generative AI, but an older machine learning technique, and it’s a very small model (a few megabytes) that doesn’t invent anything.
It functions more like an advanced filter.

The main KA thread for this feature is here: Fast Sketch Cleanup plugin: first public testing
This is the older blog post introducing it with some background information: Digging into the Fast Sketch Cleanup Plugin for Krita | Krita
This is the latest blog post that describes how you can use it and shows some results: New version of the Fast Sketch Cleanup Plugin | Krita

Download:

• Linux: (beta version of Krita): krita-5.3.0-prealpha-b4601f54a8-x86_64.AppImag
• Windows: (beta version of Krita): [link soon]

Links for screens bigger than fullHD

• Linux: (beta version of Krita): krita-5.3.0-prealpha-68346790dc-x86_64.AppImage
• Windows: (beta version of Krita): krita-x64-5.3.0-prealpha-68346790.zip
• Windows: special zip version to be able to use it on top of stable Krita: FastSketchPlugin1.1.0.zip
  To use this version, you need to first download a portable version of stable Krita if you don’t have it yet: https://download.kde.org/stable/krita/5.2.9/krita-x64-5.2.9.zip and unpack it. Then download the other zip file and also unpack it in a different place. Inside the folder you’ll see two folders: lib and share, just like in the folder with Krita. Move those folders into the folder with Krita. Ad that should be it.
  If you still can’t see the plugin in Tools → Scripts → Fast Sketch Cleanup, try to go to Settings → Configure Krita → Python Plugin Manager and see whether you can find it there and enable it, and restart Krita.

If you have any issues or questions, please do tell me, since because it hasn’t been truly released yet, it probably hasn’t been battle-tested properly yet so there might be some issues.

(The final version is still not merged into Krita since it’s supposed to go into 5.3, and we can’t release 5.3 until all the text work is done, and it’s not done yet and it’s not sure when it will be).

I will really appreciate anyone taking their time to help us out with this.

[josepabloespinoza]

I have a problem, the minimum height of the dialog window is larger than the standard 1080p resolution, so I guess the Apply button is somewhere, but it’s not accessible. I tried reaching it with Tab, but no luck.

[tiar]

Oh, that’s not good… I made a quick change, and will update the post with new links and make a new comment when they’re built (linux-build (#2807753) · Jobs · Graphics / Krita · GitLab - it should take 50 min for Linux, and 100 min for Windows, not sure if they’re parallel, but Linux build already started).

[BeARToys]

So I tried it.
The first problem I had is, that the “run”-button is not visible in my linux machine. I had to upscale the resolution of my laptop to make the button visible. You may want to add a scrollbar.

I find it relatively difficult to find the correct parameters for my sketches. I have the preview, but I can not choose, which part I see. It would be great to select the section which will be displayed.

Or it could be helpfull, only display the values, which are converted.

The models are not selected upon using the FSC. You have to use ‘reset to default’ first. I like the sketch model the most, but the lines are not realy how I wanted them.

The Ink-model is more agressive and soft-Ink model is not usable in my opinion.

here is a little sketch of mine…

I applied the inkmodel without tweaks

the soft-inkmodel without tweaks

and the sketch-model without tweaks

I tried tweaking the sketchmodel a bit, but as I said it is difficult to find the right preferences.

Oh yeah and krita crashed multiple times already, while I did not use any of its tools…

[YRH]

Hi, I checked the Windows “larger than full HD” Krita build. Here are a few observations:

1. There’s an error dialog box on startup that closes automatically:
   
   
2. Would be nice if the plugin had an associated action that can be bound to a hotkey.
3. When the plugin is first opened, it can’t find the model. However, it can be fixed by clicking “reset to default”.
   
   
4. It’s not possible to pan the preview window. If the size is small, user can end up on a blank part of the sketch and the only way to see the effect better is to increase the size.
5. CPU processing is not using SMT, leading to only 50% utilization, while not being memory limited. Why?
   
   
6. Lastly, when running any model, it always gets stuck (usually at 99%) and Krita becomes unresponsive.
   
   
7. GPU acceleration is not supported.

[tiar]

Could you check this kind of workflow:

1. SketchyModel with scale adjusted to the level of detail you want (it would look kinda blurry or blurrier),
2. A white layer on top of it with opacity let’s say 50% (so that it looks greyish)
3. Then InkModel with high scale?

I got this kind of result:

Other than that, it just comes out grainy:

But it seems like it really has a problem with that grain. The contrast within the line is too much and it creates holes. I think it usually worked better for me when I was using a more blurry brush, like one of the basic ones. David Revoy’s sketches usually consistently give good results with the strategy I described.

@BeARToys

Oh yeah and krita crashed multiple times already, while I did not use any of its tools…

Which version of Krita are you using, the stable one or the unstable one?

[tiar]

@YRH

There’s an error dialog box on startup that closes automatically:

I haven’t seen that before, and I’m not sure what it can be about. It’s a startup of Krita, right?

@YRH

2. Would be nice if the plugin had an associated action that can be bound to a hotkey.
3. When the plugin is first opened, it can’t find the model. However, it can be fixed by clicking “reset to default”.
4. It’s not possible to pan the preview window. If the size is small, user can end up on a blank part of the sketch and the only way to see the effect better is to increase the size.

Noted.

@YRH

CPU processing is not using SMT, leading to only 50% utilization, while not being memory limited. Why?

I am not sure. The number of workers is determined by openvino.

@YRH

Lastly, when running any model, it always gets stuck (usually at 99%) and Krita becomes unresponsive.

I think it always becomes unresponsive immediately, it just takes time until the system notices it. I tried to make it responsive, but I think Python and Qt don’t mix together well - Python has only one thread, and while the actual processing happens I think in c++, as usual with those types of libraries, I really couldn’t find a way give the control back to GUI for at least a tiny moment. It can process multiple things at once, sure, but then waits for them all. I tried all kinds of timers and whatnot, but nothing worked. And I do use AsyncInferQueue with a callback, but it still spends all the time waiting for this queue to finish. And while it waits, Qt’s GUI doesn’t process events, because Qt’s GUI only works in one thread… and since Python scripts commonly need GUI, all Python scripts run in that one specific thread used for GUI and events in Qt. I fought with it for quite a while since I tried to make an option to cancel the processing. The best I could do was the progress bar, which works only because it increments the step within the callback function (which is the tiny bit of control I have, happens once after every processed bit) and immediately calls QCoreApplication.processEvents(). It updates the GUI and makes it feel a bit less unresponsive, but events like trying to close the window only arrive after the whole image is processed.

@YRH

GPU acceleration is not supported.

It kinda is, but only for Intel GPU. Which is… not great. But the library we’re using doesn’t support Nvidia or AMD. It’s still useful for people with Intel-based laptops, at least. I’ll make sure to change the text in the GUI to something like “Intel GPU” instead so it’s clear. At least CPU should work on any CPU.

[YRH]

@tiar

I haven’t seen that before, and I’m not sure what it can be about. It’s a startup of Krita, right?

Yes, it opens briefly during Krita startup. It shows a moment before the main window opens, and then closes automatically.

@tiar

I am not sure. The number of workers is determined by openvino.

The application is able to request that. Perhaps it could be exposed as an option, in case it performs better without HT on some systems.

config = {
    hints.performance_mode: hints.PerformanceMode.LATENCY,
    hints.enable_hyper_threading: True,
}
compiled_model = ie.compile_model(model=model, device_name=self.device, config=config)

@tiar

I think it always becomes unresponsive immediately

To clarify, it does not just become unresponsive. For me it just gets stuck at 99%, forever. I have to kill Krita and the filter doesn’t work on my system.

Regarding the blocking in general, yeah, I think you would need to somehow exit from your Python code and then have Krita call you back. I don’t know if this is possible currently (and my knowledge of Pykrita is not too deep).

In general the best way to achieve true concurrency of Python code is to use multiprocessing module and spawn a new process to handle your function, but I have a feeling your callback already does something like that.

[BeARToys]

I will try it, when I find the time.

When did the crash occur? So I tried fsc with the version you provided, I don’t think it is stable…

I used the fsc and got my results, but after some time krita closed itself. I can try to reproduce the crash next time.

[BeARToys]

So I tried it again as you told me.
I created a litle sketch

Then I used the Sketchmodel (Scale = 1.0)

Put a layer with 40% opacity on top

And finaly used the ink-model

These lines definetly look better

The window is still to big, so that I can not press run with a screen resolution of 1920 x 1080.
With a resulotion of 2560 x 1440 the run-button is visible.

And it crashed again.

I don’t know if that helps, but here is my current setup:
OS: Ubuntu 24.04.2 LTS
KERNEL: 6.11.0-24-generic
CPU: 11th Gen Intel Core i9-11900H @ 2.50GHz
GPU: NVIDIA GeForce RTX 3080 Laptop GPU
GPU DRIVER: NVIDIA 535.183.01
RAM: 32 GB

[tiar]

@BeARToys

The window is still to big, so that I can not press run with a screen resolution of 1920 x 1080.
With a resulotion of 2560 x 1440 the run-button is visible.

i thought I fixed it… I’ll add the scroll bar.

And I can’t reproduce the crash, do you have any info about it? Possibly even running the appimage through the gdb might help. It happens after the picture is processed or at the end of it, right? And pretty randomly?

[BeARToys]

How do I use gdb ? Just opening in the terminal?

[tiar]

Run in terminal:

gdb -ex "set print thread-events off" [path to krita]

Then if it asks you for more input, write “r”, make a crash, and then when it crashes, do:

bt (and copy and save the output),

and then:

thread apply all bt and also copy and save the output (you’ll probably need to press Enter a few times). (then you can close it using q command).

You can write only the first output here for now since it’s most likely the one that causes the problem.

[BeARToys]

So I could crash krita again. I made multiple layers with the FSC and afterwards I drew until crash
Is that what you were looking for ?

Thread 1 "krita" received signal SIGSEGV, Segmentation fault.
0x00007ffff54d3c8c in QMetaObject::cast(QObject const*) const ()
   from /tmp/.mount_krita.z17YRq/usr/bin/../lib/libQt5Core.so.5

[YRH]

Yes, but it would be better if you included the whole backtrace (bt) as this only shows where it eventually crashed, but not which code called it.

[tiar]

The crash looks exactly what I am looking for, yes - but could you please, after it crashes and returns the control to you (with the “>” prompt, I believe), write bt? Because that information tells me that the crash happened somewhere in QObject code, so somewhere in Qt, so it’s not very useful, I’d like to see what led to that piece of code. Especially since most things in Krita are QObjects.

[BeARToys]

Ah sry i’m not accustomed to this tool I’ll try to crash it again.
Done!

Thread 1 "krita" received signal SIGSEGV, Segmentation fault.
0x00007ffff54d3c8c in QMetaObject::cast(QObject const*) const () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
(gdb) bt -full
#0  0x00007ffff54d3c8c in QMetaObject::cast(QObject const*) const () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
No symbol table info available.
#1  0x00007ffff63c4867 in ?? () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Widgets.so.5
No symbol table info available.
#2  0x00007ffff6377575 in QApplication::notify(QObject*, QEvent*) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Widgets.so.5
No symbol table info available.
#3  0x00007ffff7a983ee in KisApplication::notify(QObject*, QEvent*) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libkritaui.so.20
No symbol table info available.
#4  0x00007ffff54caf4a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
No symbol table info available.
#5  0x00007ffff5523d8b in QTimerInfoList::activateTimers() () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
No symbol table info available.
#6  0x00007ffff552468c in ?? () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
No symbol table info available.
#7  0x00007ffff321617d in g_main_context_dispatch () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libglib-2.0.so.0
No symbol table info available.
#8  0x00007ffff3216400 in ?? () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libglib-2.0.so.0
No symbol table info available.
#9  0x00007ffff32164a3 in g_main_context_iteration () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libglib-2.0.so.0
No symbol table info available.
#10 0x00007ffff5524a58 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
No symbol table info available.
#11 0x00007ffff54c985b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
No symbol table info available.
#12 0x00007ffff54d1e14 in QCoreApplication::exec() () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
No symbol table info available.
#13 0x000055555555e04f in main ()

(gdb) bt
#0  0x00007ffff54d3c8c in QMetaObject::cast(QObject const*) const () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
#1  0x00007ffff63c4867 in ?? () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Widgets.so.5
#2  0x00007ffff6377575 in QApplication::notify(QObject*, QEvent*) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Widgets.so.5
#3  0x00007ffff7a983ee in KisApplication::notify(QObject*, QEvent*) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libkritaui.so.20
#4  0x00007ffff54caf4a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
#5  0x00007ffff5523d8b in QTimerInfoList::activateTimers() () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
#6  0x00007ffff552468c in ?? () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
#7  0x00007ffff321617d in g_main_context_dispatch () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libglib-2.0.so.0
#8  0x00007ffff3216400 in ?? () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libglib-2.0.so.0
#9  0x00007ffff32164a3 in g_main_context_iteration () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libglib-2.0.so.0
#10 0x00007ffff5524a58 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
#11 0x00007ffff54c985b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
#12 0x00007ffff54d1e14 in QCoreApplication::exec() () from /tmp/.mount_krita.Uyod3M/usr/bin/../lib/libQt5Core.so.5
#13 0x000055555555e04f in main ()

This happend when I used the color-selector

[BeARToys]

I also could generate this crash:

Thread 1 "krita" received signal SIGSEGV, Segmentation fault.
0x00007ffff54caebc in QCoreApplication::notifyInternal2(QObject*, QEvent*) ()
   from /tmp/.mount_krita.ojFb5D/usr/bin/../lib/libQt5Core.so.5
(gdb) bt
#0  0x00007ffff54caebc in QCoreApplication::notifyInternal2(QObject*, QEvent*)
    () from /tmp/.mount_krita.ojFb5D/usr/bin/../lib/libQt5Core.so.5

But that trace is not realy complete, because I did not have the correct permissions…

This happend while using the shapes fill brush

[josepabloespinoza]

Ok I tested, I used 3 examples:
uneven lighting: sketch gesture - imgpost
kraft paper: sketch study - imgpost
ink with small gaps: sketch ink - imgpost

The same setup was used for all of them (CPU, sketch model, 0–80 pre process threshold, 0–70 post process threshold). I didn’t like the results from the ink models — they bled and distorted the lines.
The larger images took about 5 to 10 minutes to process; the smaller one was much faster, about a minute.

Results:

sketch gesture fsp

sketch study fsp

sketch ink fsp

To have a baseline for comparison, I often use OpenCV to process scanned drawings — reducing paper texture, enhancing lines, and generating multiple stylized outputs.

Rough Process:
- Converts input image to grayscale.
- Applies Gaussian blur to reduce paper texture without blurring lines.
- Applies both Triangle and Otsu thresholding, picks the less lossy result.
- Normalizes the thresholded image.
- Calculates an alpha mask from the normalized image.
- Creates light and dark color variants (blue/red) using saturation and merges with alpha mask.
- Removes the background

Results:

sketch gesture opencv

sketch study opencv

sketch ink opencv

• sketch gesture opencv blue - imgpost
• sketch study opencv blue - imgpost
• sketch ink opencv blue - imgpost

Observations:
The sketch model gives a more contrast-rich result, preserves hatching, and retains original line quality. It doesn’t handle uneven lighting well, but still did better than OpenCV. I expected it to remove the background and have an option to colorize lines to non-photo blue. The ink model performed poorly, even on ink drawings, it melted lines and distorted the image.
The biggest downside is processing time. OpenCV processed all three images and generated 15 variants in under a minute. The results were good, but the long processing time makes me unsure if I’ll use it often in its current form.

Things to improve:

• Processing time
• Allow user to choose the preview section
• Improve ink model (though sketch model works well for ink)
• Load models by default
• Add support for uneven lighting

Overall, I like it. Great work.

[tiar]

I believe I fixed all the problems mentioned above:

1. CANCELING IS NOW POSSIBLE!!! I had to use a clever dirty trick, it basically processes everything in tiny 10s batches so it can check the events from Qt every 10 seconds. So the effect is not immediate, but it’s quick enough.

2. The dialog now consists of foldable widgets, most are folded on startup, and they even got some handy short info messages when they’re folded so you don’t need to unfold it to remember the settings. This is how it looks all folded up:
   

   
   
   (mentioned by @josepabloespinoza @RamonM @BeARToys )

3. You can click on the preview image now to move it around. It only updates on release, but from my testing, it shouldn’t be a big problem, and you can finally choose the part of the image you want to see previewed.
   (mentioned by @josepabloespinoza @BeARToys @YRH)

4. The folder and model should now be selected by default on startup on the first run.
   (mentioned by @BeARToys @YRH @josepabloespinoza)

5. The action now can be bound to a shortcut (which brings up the dialog).

6. It’s more clear it’s Intel GPU and Intel NPU only…

7. Hyperthreading: I implemented it and it’s now possible to turn it on, but from my understanding of this webpage: Performance Hints and Thread Scheduling — OpenVINO™ documentation it doesn’t actually improve any performance, because the logical threads are still competing for the exact same physical cores. In my experience it was even a tiny bit slower, but you can test it and tell me if it’s better or not. (mentioned by @YRH).

8. I also added time estimation to the dialog both before you run it, and during running it.

Other notes:

1. The problem with an error message encountered by @YRH was caused by incomplete instructions on the blog post about the plugin; it wasn’t explained how to actually install it over the stable version. I haven’t changed it yet but I’m going to.

Download new versions (unstable/based on master branch):
Linux: _packaging · Artifacts · linux-build (#3005878) · Jobs · Graphics / Krita · GitLab
Windows: https://invent.kde.org/graphics/krita/-/jobs/3005881/artifacts/browse/ (should be done in like 10-20 min from now).