Animation & Frame rendering clones limit: render in failure

Don’t worry, there’s no urgency to fix everything.
The most important is to be able to fix the most weird bugs :slight_smile:

And thank to you and @eoinoneill for working on that.
That’s might be hard work, my point of view is it’s harder to modify existing code on Krita than working from scratch, especially with multithreading and all things that are already implemented in way that initially might not be thought for animation :woozy_face:

Grum999

1 Like

Hi, @Grum999!

Could you please tell, what exactly the “error” is? Does it show any message or just renders the file incorrectly? Or it crashes in some way?

PS:
Btw, the default value of ‘number-of-clones = 0.5 * number-of-threads’ is somewhat optimal. Whatever the number of clones you have, Krita will still run the maximum number of threads possible. Just having a set of clones makes threads contention a bit more efficient.

Okay, I think I can reproduce the problem. It is not much related to the number of clones, but basically to the fact autosave (and probably some other background actions) can accidentally cancel the frame generation process.

The problem is easily reproducible by setting autosave interval to 1 minute and starting regeneration process that takes 3-4 minutes.

Hi, @all!

Could you please check if these packages fix the problem for you? They should fix both, cancelled frames rendering and superfluous updates of the transform masks in the hidden groups.

Hi @dkazakov

Unfortunatelly, I can’t test the appimage :frowning:
When I try to open files, krita crash with following message:

ASSERT: "!md5.isEmpty()" in file /home/appimage/persistent/krita/libs/resources/KoResource.cpp, line 154

How can I do to test it? Is there an option or something else I can activate when running appimage to avoid this kind of error? (last time I got this problem on an appimage, I had to download source code, apply modification from commit, remove assert, compile… :dizzy_face:)

Here @halla told me appimage do not enable assert/safe assert

Is it possible to generate a testing appimage with disabled assert?


I’m really curious to test this version, because as you an see in my comment here, I took care to deactivate the autosave for all tests relative to clone limits:

So if this appimage solve the problem for autosave AND the problem of clones limits AND problem of superfluous updates of transform mask, it would be really great :star_struck:

Grum999

Or is the commit https://invent.kde.org/graphics/krita/commit/00b8f331260f14792c165298857668446737d918 is available in last official appimage?

I’m a bit lost :slight_smile:
Which one I have to test to check which bug? :sweat_smile:

Grum999

Hello @Grum999, Hello @dkazakov, Hello @All,

since @dkazakov made a version today especially for @Grum999’s animation bugs, I did the tests first with today’s krita-nightly-x64-5.0.0-prealpha-17fABBA9ba and then with @dkazakov’s krita-5.0.0-fixed-frame-export-dk1.
I rendered this animation today, so very late, and with “a little fear”, because during attempts with the animation @Grum999 had provided for the autosave bug (test_bugs_animation_04.kra) the PC had crashed with blue screen and I moron interrupted a TV video recording by it then, but this animation (test_bugs_animation_06.kra) has not caused a system crash so far.

Whether there was activity in the disabled areas during rendering I couldn’t tell for sure, probably I see too bad and my monitor (LG E2442TC) is small (24’’ FHD), old and has brightness fluctuations.
But I believe that the video icon did NOT blink.

Settings quite similar to @Grum999’s, I had the swap file a bit smaller with 60 GB, but with 65487 MiB a bit more RAM, a few logging settings different and currently the “Cache Storage Backend” setting is set to “On-Disk”, if desired I can still repeat the tests with “In-memory”.

Screenshot of the performance settings

############################

Test series 1 rendered on: CPU 2x XEON E5-2643 v2 = 24 cores; Windows 10 Pro 64-bit, Krita 5 17fABBA :wink:

1st pass OK - Blur + Pixelise & Halftone test turned off / deselected (original state of downloaded file).

2nd pass OK - all layers switched on / selected

3rd pass OK - Blur + Pixelise & No filter turned off / deselected

############################

Test series 2 rendered on: CPU 2x XEON E5-2643 v2 = 24 cores; Windows 10 Pro 64-bit, krita-5.0.0-fixed-frame-export-dk1

1st pass OK - Blur + Pixelise & Halftone test turned off / deselected (original state of downloaded file).

2nd pass OK - all layers switched on / selected

3rd pass OK - Blur + Pixelise & No filter turned off / deselected

############################

Test series 3 rendered on: CPU 2x XEON E5-2643 v2 = 24 cores; Windows 10 Pro 64-bit, krita-5.0.0-fixed-frame-export-dk1, autosave 1 minute on.

1st pass NOT TO EVALUATE ENDED TO FAST (was done before autosave) - Blur + Pixelise & Halftone test turned off / deselected (original state of downloaded file).

2nd pass KO - Blur + Pixelise & Halftone Test switched off / deselected and the Transform Mask in Group 2 & Group 3 copied 3x each and then the corresponding Paint Layer copied 3x (= 8 Paint Layers with 4 Transform Masks each) - aborted at frame 24 during autosave attempt

3rd pass OK - Blur + Pixelise & Halftone test turned off / deselected and two Paint Layers added - autosave during rendering worked

I can post the other passes tomorrow, at the moment I can’t risk a system crash as I’m currently recording a video again. >>>

4th pass MISSING - all layers turned on / selected

5th pass MISSING - Blur + Pixelise & No filter turned off / deselected

If there are any other tests, settings you want me to try, I will be happy to try. Please explain in an understandable way for an animation inexperienced user, what I should do then.

Michelist

Hi, @Grum999!

The latest nightly build contains there changes now:
https://binary-factory.kde.org/job/Krita_Nightly_Appimage_Build/

Hi, @Michelist!

I feel a bit confused. Can you still reproduce the problem with my packages?

Hi!

Ok thanks, just made some quick tests, I’ll try to do more test tonight.

  1. Autosave (438760 – Animation: render in failure when when autosave is triggered)
    Seems to be OK, rendering is not interrupted anymore by autosave
  2. superfluous updates of the transform masks in the hidden groups (438538 – Krita is doing some calculations on hidden layers)
    Seems OK, no more activity on hidden layers
  3. Frames rendering (https://bugs.kde.org/show_bug.cgi?id=438754)
    Still KO with 24CPU (with autosave deactivated)

So for me, it really sound bug 438760 – Animation: render in failure when when autosave is triggered is not a duplicate of 438754 – Animation & Frame rendering clones limit: render in failure

Grum999

What does “KO” mean? Is it a misprint or it means something?

It’s KO like already described in this topic and bug 438754 – Animation & Frame rendering clones limit: render in failure

Krita is just unable to render all frames :man_shrugging:
It stop after rendering a certain number of frames, with a message “render in failure”; there’s no more information to provide because on my side Kirta doesn’t provide more information than this message :confused:

Grum999

I still don’t understand what it means :slight_smile: The bugreport doesn’t have any literal mentions of “KO” as well :slight_smile:

:sweat_smile:
KO = “Doesn’t better than before bug fix”

Grum999

I will try this afternoon, unfortunately I have a lot to do right now.

Michelist

Edit: KO probably comes from martial arts and stands for “knock out” or “knocked down and unable to fight” and I used it “figuratively” to describe the termination of the render process.

When testing tonight, please make the screenshots of the failure. Can it be that the failure is from ffmpeg’s video generation stage, but not from the frame rendering stage.

No, I always render PNG sequence files…

Grum999

Hello @dkazakov,

Continuation from yesterday, configuration unchanged, see screenshot from yesterday.

SOMETHING I DIDN’T MENTION YESTERDAY: I EXIT KRITA AFTER EACH RUN AND CLEAR ALL CACHES AND TEMP FOLDERS FROM KRITA.

##############

rendered on: CPU 2x XEON E5-2643 v2 = 24 cores; Windows 10 Pro 64-bit, krita-5.0.0-fixed-frame-export-dk1, autosave 1 minute on.

4th pass OK - all layers turned on / selected // repeated this three times, always runs through with autosave while rendering.

5th pass OK - Blur + Pixelise & No filter turned off / deselected // have repeated this three times, always runs through with autosave while rendering

##############

Repeat of 2nd pass from yesterday - Blur + Pixelise & Halftone Test switched off / deselected and the Transform Mask in Group 2 & Group 3 copied 3x each and then the corresponding Paint Layer copied 3x (= 8 Paint Layers with 4 Transform Mask’s each)
(keep in mind that this is an insane multiplication of Transform Mask, my point was to generate high computational load during autosave (that this is insane, even I as an animation noob am aware of that)):

insane settings mentioned yesterday to force workload

6th pass KO - aborted at frame 35 during autosave attempt.

7th pass KO - abort at frame 15 during autosave attempt

8th pass KO - abort at frame 54 during autosave attempt

##############

Now the insane setting without autosave for comparison!

9th pass OK

10th pass KO - abort at frame 76

11th pass KO - abort at frame 139

12th pass OK
13th pass KO - abort at frame 272 !

My interpretation as a layman is that it is probably due to the way of generating this computational load, and the autosave attempt is then too much for the PC - even if the PC apparently “gets further” here, it is the “drop that makes the barrel overflow” (a saying).

##############
now with slightly changed performance settings,

81852 MiB RAM,
64 GB swap,
“Cache Storage Backend” now changed to “In-memory” (see also screenshot)

Performance - General + Animation-Cache Modification

but still the same test
Blur + Pixelise & Halftone Test turned off / deselected and the Transform Mask in Group 2 & Group 3 copied 3x each and then the associated Paint Layer copied 3x (= 8 Paint Layers with 4 Transform Mask’s each).

##############

first without Autosave:
14th pass KO - abort after frame 422 !!!

15th pass OK

##############

now with autosave:

16th pass KO - abort after frame 82

17th pass KO - abort after frame 35

##############

And now it’s over for today, my head is smoking, I can’t do it anymore. Actually I wanted to render with less CPU’s - 16, 8, 1 CPU - but there is not enough time for that today. If you have special wishes, please formulate them in a way that is easy for me to understand.

BTW Krita “ate” my CPU’s :wink:

Michelist

I started test yesterday, but didn’t finished… rendering takes time :sweat_smile:

Grum999

1 Like

Hi, @Grum999 and @Michelist!

I have a bit of a problem with reproducing the failure. When I enable all the hidden layers it still manages to render all the frames correctly. Even Autosave succeeds during the exporting without any trouble…

The only explanation I can imagine is the timeout for rendering a single frame. The current value is set to 30 seconds. Theoretically, if you set too many “clone threads”, then one clone might be delayed for quite a long time (because it will render the layers with one thread only). That would cause a frame generation to be rejected by a timeout.

Could you test this package? If increases the frame rendering timeout to 3 minutes:

krita-5.0.0-aborted-frame-rendering-dk2.zip — Яндекс.Диск

PS:

If the package runs fine, I think we’d better do the following steps:

  1. Add a (hidden?) option to Krita that increases the frame rendering timeout
  2. Limit the user-configurable number of clones to 50% of the number of CPU cores. The best theoretical number of clones is actually 25-50%, so it should be fine. I kept the 100% value available only for the testing purposes, but it seems like it makes people shoot into their own feet.