Animation & Frame rendering clones limit: render in failure

Hi

I open a new topic about animation, as problem here are not (totally?) related to transform mask (so I continue here some discussion started in this another topic:

Problem is difficult to explain.
Not even sure there’s a process to be able to reproduce it systematically as I have big difficulties to determinate exactly what happen…

I have created a simple animation file, you can download it here (~650KB file):
http://www.grum.fr/tmp/test_bugs_animation_06.kra
It’s a full HD animation with 600frames.

The layer stack looks like this:

What I can see with this animation:

  1. Even if group layer are hidden, there’s an activity applied to transform mask (you can see ‘video’ icon blinking on transform mask from hidden layers)

    Video example of what happen (3min length… to generate 24frames)

    For me this case is not normal:

    If something is hidden (directly or by a parent) it shouldn’t be taken in account for projection calculations

    If someone can confirm there’s really a calculation activity on hidden layers, I’ll open a bug

  2. When using 19 to 24 CPU to render animation, it’s systematically in failure after few frames has been rendered.
    When using 18 or less CPU to render animation, it’s sometimes in failure (it occurs, but it’s not systematically - so harder to reproduce)

  3. Delete layers “Blur + Pixelize” and “Halftone test”
    Rendering with 24CPU is most of time complete.
    But sometime it stop before the end (598 of 600 frames rendered for example)
    This case is really hard to reproduce systematically.

    Using less CPU (4 for example) render is complete.
    Can’t say it will always work: I just can suppose that frequency of failure might be so low in this configuration that it will practically never occurs

  4. Delete layers “Blur + Pixelize” and “No filter”
    When using 19 to 24 CPU to render animation, it’s in failure after few frames has been rendered
    When using 18 or less CPU to render animation, it’s sometimes in failure (it occurs, but it’s not systematically - so harder to reproduce)

  5. I’ve tested different filter types
    It seems that failure occurs more quickly with halftone layer than combination of blur+pixelize (ie: in second case, more frames are rendered before failure than first case)

Note: with more complex animation files (more layers, more frames and more keyframes), the current limit of 18CPU is lower… (currently I have to use 1 CPU to be sure to render)

I would like to create a bug to officially formalize this, but I’m currently not satisfied of my investigations and description: if I was a developer, current provided description is “too vague” to be able to determinate exactly situation in which problem occurs…

If someone have some time to made some additional tests, and help me on it, it would be great.
@AhabGreybeard if you have ideas? I’m maybe too focused on things and maybe miss something…
@Komorebi I know you already got animation rendering errors (especially with layers styles); if you have time to made some tests on your side :slight_smile:
@Takiro if I remember you have a computer with high number of CPU (maybe more than me ^^); if you have time to made some tests and confirm the limit of 18 CPU? (limit is higher, lower for you maybe)
@emmetpdx @eoinoneill as you’re working on animation part, maybe you can have ideas of additional tests and/or additional information that could be useful? also, if you’re able to provide an appimage with additional logs (I don’t know where it could be pertinent to place them) I can use it for tests

Everyone interested by animation part is welcome to help on this, to confirm or not problem… Important thing (to help) is to provide number of CPU + Memory allocated, result (render OK/KO) and anything else that can be pertinent…

Note: due to bug with autosave and animation, I recommend to deactivate autosave functionality during tests :slight_smile:

My current configuration:



Tests made with krita-5.0.0-prealpha-c25d2e1-x86_64.appimage

Grum999

1 Like

Sure, I can give it a try no problem. I have a AMD Ryzen 9 3900X 12-Core Processor but it’s 24 logical cores and Krita recognizes it as such. So I can’t tell you if it makes a difference for more than 24 but at least maybe can confirm that it’s more reliable with fewer cores enabled.

I just tried more than 10 times and worked perfectly every time on 24 cores. No errors, no missing frames when rendering to image sequence. Even tried rendering to video a few times, no issues.

Maybe it’s not the number of cores but an issue with the chip itself?

I used krita-5.0.0-prealpha-c25d2e1-x86_64.appimage too.

Hi @Takiro

Thanks for feedback :slight_smile:

I think we have the same processor :sweat_smile:

  • Processor: AMD Ryzen 9 3900X 12-Core Processor
  • Motherboard: ROG STRIX X570-F GAMING

So problem I encounter might be related to something else :confused:

Grum999

I only have eight cores on my old desktop but I’ll report anything that seems significant when I can find time to look at it. (I tend to be busy at the weekend.)

Using the Jun 11 5.0.0-prealpha (git c25d2e1) appimage, I’ve had a very quick look and it seems that visually hidden layers are processed:

Opening the test file as provided then Playing it, it took 9 minutes to regenerate the cache.

I then manually turned off the visibility of Transform Masks and Filter Layers in the hidden groups, then Saved it, Quit, restarted and reopened it.
Playing that modified version needed one minute to regenerate the cache.

Opening the original file again and Playing it, it took nine minutes to regenerate the cache.

I’ll have a deeper look at things tomnorrow or later and report anything I notice.

1 Like

I couldn’t find the same build as you but testing on krita-nightly-x64-5.0.0-prealpha-fc927e67fe
–—
CPU – Ryzen 1700 8c 16t
MEMORY ALLOCATED (100%) – 32719MiB
WINDOWS 10

With the default setup in the file, you provided when rendering the no filters group. The animation stops after finishing frame 15 using tif, jpeg, png. This happens when I am using all 16 threads on my cpu for rendering by setting the Clone limit to 16.

For additional testing I rendered a png sequence and changed the Frame Rendering Clones Limit, here are my results.

When rendering the No Filters Group

16 threads: Frames Completed - 15 (Consistent 15 frames after few repetitions)
12 threads: Frames Completed - 24, 12, 20 (Inconsistent frame completion)
10 threads: Frames Completed - 113, 43 (Inconsistent frame completion)
9 threads: Frames Completed - 28, 600 (Inconsistent frame completion)
8 threads: Frames Completed - 600 (COMPLETED)
1 thread: Frames Completed - 600 (COMPLETED)

Note: I noticed that when rendering the animations that crashed cpu utilisation was all over the place and very low. Also, note I did have chrome open.

When Rendering Halftone Test

16 threads: Frames Completed - 15 (Consistent 15 frames after few repetitions, Same as previous?)
8 threads: Frames Completed - 600 (COMPLETED)

When Rendering Blur+Pixalize test

16 threads: Frames Completed - 15 (Consistent 15 frames after few repetitions, Same as previous?)
10 threads: Frames Completed - 27, 21 (Inconsistent frame completion)
8 threads: Frames Completed - 600 (COMPLETED)

In conclusion, it seems to be a similar story with all the tests for me.
Hope this helps somewhat :slight_smile:

1 Like

Hi @AhabGreybeard

Thanks for testing it!

As you confirmed my though, I opened a bug:
https://bugs.kde.org/show_bug.cgi?id=438538

Grum999

Hi @Komorebi

Thanks for testing! :slight_smile:

Yes I think this is because Linux & Windows are not built at the same time.
And if a commit is made between build, then the git hash is not the same.

But here it doesn’t have any importance, there’s no real difference between our tested versions.

Yes this comforts me in the fact that I’m not the only one having the problem :crazy_face:

The interesting thing is, on your side if you use more than half available CPU it doesn’t work…
Using half (or less) seems to render animation.

One thing: there’s currently a bug that make Krita doing calculations on hidden layers.
If you delete layers “Blur+Pixelize test” and “Halftone test”, and render the “No filters” with all CPU; it’s still in failure?


On my side, I’ve made additional tests on my laptop (4CPU/16GB).
Using the 4 CPU and 95% allocated memory, render is complete for “No filters” (but it’s slooooooooow :sweat_smile:)

Rendering the “Halftone test” with the 4 CPU has failed after 412 files…

Edit: the 4 CPU laptop randomly failed/success to generate all frames (but most of time it successfully finish render)

Grum999

1 Like

Yea, I can render all the tests completely out with all threads after deleting unused groups.
Still only getting 50 percent or less utilisation on my CPU though.

But as soon as I add one group that is unseen I can only complete 15 frames with 16 threads, just like in my previous tests.

Rendering the original file to an image sequence, I had no problems but my PC has only eight cores and they were all used at 100% capacity.
My PC is old and slow with DDR2 RAM so it may not be useful for these tests.

For the activity of the hidden group filter layers and animated transform masks, I turned off the visibility of the animated transform masks and played the animation and it took three minutes to regenerate the cache and showed activity on the visible filter layers of the hidden groups.
(Previous times were 1 minute for all hidden group layer masks and filter layers having visibility turned off and nine minutes for the original file.)

So, non-rendered masks and filter layers are processed with bad consequences for the total time needed.
Has anybody tried turning off the visibility of the non-rendered masks and filter layers to see if that fixes the rendering problems, to see if their processing is the only cause of the problem?

2 Likes

Interesting… seems to be the case for me @AhabGreybeard
When rendering no filters group and disabling both filter layers and masks for the hidden groups it renders out as intended. As if it was the only one there. But as soon as I enable the others it terminates after frame 15.

Here are the layers visibilities when it worked as it should:

Also, thought I would add that when only rendering no filters group my cpu is fully utilised, but like @AhabGreybeard said filter layers and transform masks seem unoptimised, as the other groups use about half of the cpu’s utilisation.

1 Like

Not sure.
If on your side it’s working (even on an old and slow computer) it can confirm that problem occurs on certain configuration only (which one, I don’t know)

With parent group layer hidden

Filter mask visible Transform mask visible activity render cache time (using 12CPU)
YES YES YES (both) ~4m00s
YES NO YES (filter mask) ~2m50s
NO YES YES (transform mask) ~21s
NO NO NO ~15s

Grum999

3 Likes

I added a comment to bug 438538 – Krita is doing some calculations on hidden layers concerning activity of filter layer/filter masks.

Grum999

1 Like

I created the following bug concerning clones limit:
https://bugs.kde.org/show_bug.cgi?id=438754

Grum999

1 Like

Today I’ve created some other bugs:

https://bugs.kde.org/show_bug.cgi?id=438771

https://bugs.kde.org/show_bug.cgi?id=438770

https://bugs.kde.org/show_bug.cgi?id=438769

https://bugs.kde.org/show_bug.cgi?id=438768

https://bugs.kde.org/show_bug.cgi?id=438767

https://bugs.kde.org/show_bug.cgi?id=438766

https://bugs.kde.org/show_bug.cgi?id=438765

https://bugs.kde.org/show_bug.cgi?id=438760

And reopen one due to 2 additional bugs

https://bugs.kde.org/show_bug.cgi?id=438341

I think I’ll do a break, this exhausted me :crazy_face:

I have some improvements to offer, but I think it’s better to fix all bugs before asking to improves things :sweat_smile:

Grum999

2 Likes

@Grum999 I’m slowly working my way through your various reports! I’m not that active on these KA threads, but I’m listening and sorting things out on bugzilla…

I really want to thank you for all of your testing and effort in creating these detailed reports, threads, test files, and videos. It’s a ton of work for you (and I bit disappointing for me to find out there are so many bugs…) but please know that we really appreciate it and that it’ll mean Krita 5 is that much better. :slight_smile:

Same to both of you @AhabGreybeard and @Komorebi – Thanks so much for all the help with testing and triage and all of that. I see you! It makes a real difference. :smile:

THANKS!

6 Likes

Great work @Grum999 thanks for putting the time in to send and explore these bugs.

@Grum999: Spend the entire weekend at Hellfest then relax for a week.
You’ve done great work and deserve a rest :slight_smile:

1 Like

Ahah thanks
It’s online so I have video on a monitor, and testing Krita on another monitor :wink:

Grum999