.ORA format extension: animation, to introduce better integration between Krita and Kdenlive (or other open source video editors)

Problem:

Krita cannot do long animations.
Krita can handle maybe a few seconds of animation, not much (yes, it depends how much RAM you have and how big resolution you want. But with reasonable 16GB and reasonable fullHD resolution and reasoable three layers, you get very small amount of frames!). (Moreover audio in Krita isn’t working very well). Users are expected to use another programs to stitch the animation together. But the only way is to either use rendered video or image sequence. Using rendered video is lossy, using image sequence is better, but a bit inconvenient (lots of files, I don’t think Kdenlive handles hold frames (missing files) well, I couldn’t confirm for sure but the parts with hold frames were behaving weirdly while the part with no gaps rendered correctly…).

Idea:

More integration!
Kdenlive (or any other open source video editor: I guess the same could be said about Blender, I’m not sure what else could we try - but I’m starting from Kdenlive because it’s KDE like Krita) could read a file saved by Krita to get the sequence (maybe even in layers!). It would work like File Layer, which means the changes made in Krita would be reflected in Kdenlive. It would be one big project in Kdenlive and separate scenes in Krita. The user would be able to open the sequence in Krita from inside Kdenlive, they could maybe even do stuff like lipsyncing inside Kdenlive (that would be more difficult since I don’t see any drawing capabilities there yet…). They should be able to export the audio at least already cut perfectly for the sequence, which would help in Krita.

Realization:

It wouldn’t be fair to expect Kdenlive to read .kra files, they are way too complicated and can change any time, since it’s just an internal format. There is a solution for that already. It’s called .ora. .Ora files are .zip files with .png files inside that represent layers. I don’t see a reason why the png files couldn’t be representing not one axis (layer stack) but two (layer stack + time), like in Krita.

Two options:

  • using APNG instead of PNG as layers. Advantages:
    • Advantage: since APNG files are easily readable as PNG, that would pose no issue for any other program that tries to use .ora
    • Disadvantage: APNG might be limiting: no hold frames (???), the file might be very big, I don’t know if APNG doesn’t have any limitations on the number of frames; a list of PNG files is easier readable than one APNG file
  • using multiple PNG file to represent animation; first image is the image that would represent the layer if the file is read in a program that doesn’t support animation.
    • Advantage: no limitations except for what .ora already have
    • need to extend the .xml definition somehow

What would it require:

  • .ORA format is maintained by Krita, but it doesn’t mean we can just change it willy-nilly, so we need to contact other developers from software that is interested to see what they think and if they think the extension would be a good idea. For sure we’d need to contact/work with MyPaint, Gimp, of course Kdenlive for other reasons as well, I believe Scribus and Drawpile can read .ora too. Although I see that the format specification says that the application can extend the format: but since we plan on making it into something we want to integrate with other programs, it would be best to discuss with other programs so we can get the best representation of animation possible.
  • Kdenlive needs to be able to read .ora files as “video” if they contain animation, which will require some coding. To get this working, we’d need to work together with Kdenlive team, or maybe some of us would need to work on Kdenlive for a bit? That depends on whether they would be interested. I believe it’s mostly good for Krita to have it, I don’t know Kdenlive team so I don’t know what they could think of something like that.
    • any additional feature mentioned above is extra:
      • cutting audio to fit Krita’s sequence
      • opening the sequence inside Krita by one click in Kdenlive
      • … other possibilities
  • Krita needs to read and write .ora files as animation

Disclaimer:
Maybe I don’t know Kdenlive enough to find a nice workflow? I only know bits of it. If so, please do tell me in the comments!

What do you guys think?

5 Likes

The problem I see with APNG is that it has no real standard. Last time I checked there were like three different implementation. All similar but not the same.

1 Like

Hi

Natron too :slight_smile:

Having a file format that Krita can export and that can be loaded immediately like a file layer in third party software would be great.
Currently, need to generate PNG sequence file and then import sequence…

But as I can understand choice of ORA file format (created by Krita as an open interchangeable file format?) I’m wondering about real benefit… :thinking:

ORA file can manage layers, but can’t manage all Krita properties: in this case if I’m working on an animation, what happen?

  • If I do “save as ORA file” I may loose some data information in my file; there’s a risk for user to loose part of their work (look at now how many topics have been opened concerning the “save/save as” in non native kra format and users who don’t understand why they loose all layers in their JPEG files… :dizzy_face:) => Consider that “export” or “render animation” as ORA is the solution…?
  • If I have 4 animated layers and 3 non animated layers, what happen in export?
    – 7 layers interpreted as 7 tracks sources by KDEnlive?
    But in this case what if some specificity can’t be exported (masks, …)
    – Export only one merged layer?
    In this case what is the benefit of ORA file over a video file format?

What about an animated ORA file imported as file layer in Krita? :slight_smile:
(Does Krita will support it?? might be performance killing…)

Choice of APNG.
APNG seems to be a sequence of PNG files inside extra data block of a PNG file…
So if one frame is modified, it seems difficult for connected software to just read the modified frame (need to re-read everything…?)
And according to wikipedia, this is not an official file format…

The PNG group officially rejected APNG as an official extension on April 20, 2007.[6] There have been several subsequent proposals for a simple animated graphics format based on PNG using several different approaches.[7]

Then, that sound better (simpler) to store all PNG sequence files directly in ORA file, with timestamp in an XML file (let the software opening ORA file being able to analyze which frames have been changed with time stamp)
=> But this mean that Krita must be able to re-export only modified frames… it sounds complex to maintain

From a personal point of view, I prefer to see in Krita a better support of WebM file format in animation rendering

Currently in my workflow:

  • I export PNG sequence (because lossless and because sometime -don’t know why- rendering animation into WebM directly badly fails…)
  • After, I import sequences and export WebM files (can be lossless or lossy) to work between Natron, Enve and KDEnlive
  • Finally a redo export to PNG sequence to import in Krita, rework on it
  • Then re-export sequences to generate WebM…

On my side I think that improving current WebM rendering could be a better thing than trying to create a new video file format.
I’m not the best concerning license compatibility, but WebM seems to be an open format, and source code is under BSD like license…

Hope my remark will not be interpreted as critics, but more as constructive arguments about this idea :slight_smile:

Grum999

3 Likes

Pinging @InkLab as @InkLab is interested in extending .ora.

While I like the idea, where would be differences between you guys’ suggestions and the EXR file format? 8 bit instead of 16/32? In professional video editing, 3D rendering etc. EXR is the gold standard and it is made by ILM and also open source. The only way I can see this make sense is because hand painted animation would be fine in less bit and therefore in smaller file sizes (and less taxing on the computer) since you would never really need the full dynamic range. Usually at work when I need to render frames in 8 bit, we always use png, of course at the cost of not having multiple layers in one file.

Edit: I just read up on the apng format, at least it seems somewhat known to the world of open source and Mozilla had a hand in it. It just comes down in the usabilty I guess. When it really works in multiple programs that should be a great format to have available.

Yeah I thought of openEXR too but I don’t know It would be a bit hacky but would totally work. If Krita maintains ORA I don’t see why not use that as the weapon of choice.

Since it is the Nuke plus Hiero approach there is a XML for animation around there somewhere on the plan? I knew it would was a big deal for Hiero until they got merged into Nuke Studio. The XML keepes track of all the content and timming. Then you could use it for animation sheets as a by product not that I am keen in it I just see people asking.

I think kdenlive to be a very good choice indeed. Would solve tons of complaints and it is very Qt too so it would talk very similar.

That’s exactly what I wanted to get: constructive feedback :slight_smile:

Losslessness. Is webm lossless? If so, then it does seem to not be needed, and the user should use .webm instead of .ora here… Especially if we don’t know what to do with layers in Kdenlive.

Well, Krita already does support .kra as File Layer but I don’t think it’s animated.

I was thinking about either saving exclusively to .ora (which limits the capabilities, but .ora is supposed to be the “.psd equivalent”, a format to interchange between programs in the middle of working), or saving to .kra and exporting to .ora. Then Kdenlive could read the .ora. I didn’t think the “and when you click on it, Krita opens” feature yet… Because if there are two files, .kra and .ora, Kdenlive wouldn’t know to open .kra if it uses .ora for its own purposes.


Regarding EXR, I think it wouldn’t be that much of a good idea.

  • .ORA is an easier file format to support - .zip is very basic format, .png is also supported everywhere (note: APNG is not that widely supported; even neither image viewer nor video player on my Mint could play it properly… Firefox could, though)
  • .ORA is better to add stuff/easier expendable: it’s just a .zip with .png files inside, there is no looking at bytes, you could put in the zip lots of stuff and programs reading .ora won’t have trouble reading it, while if you put additional stuff in .exr, then the structure will be broken
  • PNG actually can support 16bit and HDR, so it’s not that bad, although sure, the range of color spaces is smaller than in EXR
  • .ORA format is maintained by Krita, which makes things easier, too

Also note that it’s not “our suggestion”, I mean, behind this thread there is only me, Tiar; so the feature request is Tiar’s, not Krita team’s. And behind .ORA format there is libre graphics community - MyPaint uses it as an internal file format and it’s ecommended to use .ORA if you want to change programs between Gimp, MyPaint and Krita in the middle of the project (do one thing in one program, next in another). So it’s not an unknown format, the suggestion is just to add new capabilities to it.
Also note that PNG does support 16bit, afaik.

I’m aware of the 16 bit that PNG supports, I only meant that OpenEXR doesn’t support less than 16 bit, just from it’s nature of keeping as much information from renders and compositing in the files, which it was designed for.
I misunderstood what you wrote in the first post then, I understood the focus would be more on APNG and I looked it up, as I didn’t really hear much about it before.
On the other hand, I’m all in on the idea of ORA being used more widely. Kind of an open, real alternative to PSD that also suits animation workflows. That would be very nice and could be used in a lot of scenarios.

1 Like

Hi

Yes, WebM support both lossy and lossless format.

And Krita already manage this option in rendering animation :wink:

That just not stable and if sometime (to often for me) crash on my side after rendering few frames… (don’t ask me why)

I didn’t try it :sweat_smile:
But just tested it and it doesn’t manage animation…

My initial idea was more about this idea:

Then if KDEnlive implement something like File layer on ORA file to reflect automatically change made in Krita, is Krita will be able to read these ORA file and do the same? :slight_smile:

But animation (ORA or native KRA file) as file layer might kill performances… :thinking:

My idea was more, on a 500frames animation, if I modify 10 frames, exporting ORA file implies to rendering it again entirely and it could take time.
So adding/updating/removing modified frames inside ORA file was my initial though, with a timestamp in XML file to allow third party video software to be informed about “these one were modified but not these one”

But I think this idea is useless as a video software might not ‘think’ like this as it usually works with video files on which each frame have to be decoded.


Also Krita can’t open it as animation :frowning:

WebM has advantage over APNG:

  • Even in lossless mode, final file is smaller than a sequence of PNG files.
  • Already supported by most video editing softwares I think (at least KDEnlive, Natron, Enve for the one I use) and also web browsers
  • Can manage audio track
  • Decoding is fast
    WebM also support alpha channel
    Don’t know about 16bit file format…

The advantage of an extended ORA file over a WebM is eventually layers; in this case it means that video software is able to import all layers (animated or not) and synchronize tracks time automatically according to what has been defined in Krita

But maybe the best thing to do is to talk directly with KDEnlive team: they might have better experience about video file formats, what could be really usefull or not with an extended ORA file, what can be implemented or not for a better integration between both softwares.


But currently I really prefer to have a WebM file instead :slight_smile:

To be more precise, if Krita was able:

  • To import WebP/WebM files as animated layers that would be really great
  • To export WebP/WebM natively (I mean, use of library instead of external ffmpeg)

Grum999

1 Like

Kdenlive does not have a forum like Krita-artists do they? So you could ask there and see feedback from their side. they are in the KDE forum only?

What does the log say? At what point does it crash? Making the frames or encoding the video?

Krita is opening both APNG and WEBM files as animation just fine on my pc… :wink: (though not natively)

For what I remember, nothing special in log :man_shrugging:
Just it failed.
And I don’t remember at which point it failed, didn’t retried from a while now as I’m always generate PNG sequences…
I can try again when I’ll have to render animation…

On my side, for APNG and WEBP, only the first frame is loaded in a non animated frame
WEBM file format is not available in open file dialog…

I’m curious to know how you proceed :slight_smile:

Grum999

K, next time you try it, see where it fails.

APNG

WEBM

Sorry, no webp :confused:

I don’t have these dialog box…

Grum999

It should be there soon. You can also use the python plugin technically in the meantime.

Ah ok, that’s currently in development then
great :slight_smile:

Grum999