Call for help: assist development of ORA format providing KRA, PSD or other source files

Good evening.

Over the past few months, I have been working with some of the developers of free art tools such as Krita and Gimp/Glimpse to develop improvements for the ORA graphics interchange format (https://www.openraster.org/). It is like PSD format, but not proprietary. Over the course of my research so far, I have found several examples where PSD provides features which are not available in ORA. In order to propose solutions to these issues and continue to make great tools available for free, I will be hosting a panel in this years Libre Graphics Meeting (https://libregraphicsmeeting.org) to push for changes. (and I will be doing some coding too ^.^)

My plea to any digital artist who reads this: If you have any old KRA, PSD (or procreate, CSP, xcf, whatever) digital art source files that you would be able to donate prior to the meeting (May 2020), especially those which use a lot of advanced features like multiple clip or blending layers, compositing modes, etc, please let me know! I would really like to make sure that all of my proposals at LGM can address any shortcomings of the current free standards so that we can make the maximum possible progress this year!

Two of the major issues I want to address currently:

-Lack of vector / text capability

-No support for clipping layer combined with another mode, like add / multiply (which many artists use for things like highlights and shadows)

Please let me know if you are able to help and have an awesome day!

4 Likes

Welcome to the forum! I hope people provide some sample files. I’ll see if I have some test files too.

2 Likes

Important work. I might provide something, will look into my files to see if anything might be of use.

1 Like

I have provided a sample Krita file with advanced layer formatting here - https://discuss.pixls.us/t/pat-david-luminosity-mask-for-krita-4-3-tutorial/16452

Scroll down, and download that .kra files provided.

2 Likes

I’ll just note that MyPaint uses ORA as it’s native format too and they currently added spectral colour mixing.

FYI, it seems we are having a more advanced conversation about this on this thread: MyPaint 2 release & ora format for anyone who is interested in contributing to the conversation. :slight_smile:

Hello, I have some interest in advancement of ORA format, so I conjured a small example file just now, if it’s still needed:

I mainly draw in SAI, which can export to PSD, and use clipping a lot.

The main caveat I think is “passthrough” blending mode paired with other things, like partial opacity, layer mask, clipping or all at once enabled on one folder. This requires some complex setup to stay within ORA standard, and will possibly not work in some cases. This maximum combo is not included in example file above (see the planet rings for the most complex thing there is), but I can make something later if still needed.

In an piece of software I’m developing lately, I decided to add following custom XML attributes for testing:

clipping=“put anything that is not none to enable” (I write “group-to-below” here)
mask-src=“data/mask_image.png”
mask-x=“number”
mask-y=“number”

I don’t remember why I decided to go with separate attribute for clipping, it was more than a year ago, but maybe just adding a second op with “src-atop” was not enough.
I also enable passthrough mode with isolation=“auto”.

When the standard becomes capable enough, I will adjust my creation to follow it, and probably ask SAI developer about ORA support again.

1 Like

Hi f2d. Thanks! I am still open to suggestions for improvements. It is good to know that the SAI developers may be responsive. I took a look at the LibSAI project for a while to check for conversions to ORA before realizing that maintaining something like that would likely be too daunting. I am not sure the exact overlap of their features to PSD. Do the PSD that you get from SAI usually represent exactly what you intend from editing in SAI, or are there features missing?

Is the passthrough feature that you have issues with when opening in Krita, Photoshop, or SAI? I notice immediately that the PSD you provided does not open in Kritalooking the same as the rendered PNG, but I notice that might be because the transparency masks are not correctly imported from the PSD format. I can investigate if there would be any combination of blend mode that would work, but what I am guessing is that you were also interested in adding a clip mode and blend mode together as I have described earlier?

Thanks

I open files with my custom JavaScript renderer, a work in progress yet far from production quality. It uses PSD.js to parse PSD files, and as a consequence it supports even less than that (but at least the subset of features that I actually use in my files created in SAI). It does not even really support passthrough. If a folder with this mode does not have any other property that affects rendering (partial opacity, mask, etc.), it just dumps its content into the render queue (allowing to better organize layers with blending modes or just create less temporary rendering subcontexts), and otherwise ignores this mode. I have some thoughs about how to implement it properly, but it raises complexity with possibly more nested passthrough folders, and my mind breaks there so I just put it off for now. I don’t use it in my files for more than the most simple way - just to group some layers.

Today I installed Krita’s latest beta version from the site - 4.2.9-beta1 x64. It allows to set passthrough mode separately from blending mode, but thankfully one disables another (I cannot imagine how they’d work at once). Also it disables masks but not partial opacity, so if I disable it and restore inherit-alpha where the clipping was, example file above looks not as much broken, but some blending is still different.

I tested standard src-atop versus my non-standard clipping group, and it seems that the difference amounts to arithmetic precision of rounding in 8-bit calculations. I am not used to drawing in Krita, so excuse me for a not so fancy example this time:

Also for some reason “Addition” blending mode (same operation as “Linear Dodge”, “svg:plus”, etc.) in Krita looks different from my JS renderer (using “lighter” blending mode or custom-written asm.js addition, which both show the same result). “Screen” mode looks the same everywhere, so I added this variant to comparison.

Yes. After all, clip mode + blend mode looks like a good solution. I’d suggest to add “alpha-composite-op” and “color-composite-op” attributes, let applications make sense of possible combinations, and keep old combined “composite-op” for backward compatibility, but there may be better ideas.

By the way, JS canvas API suffers from the same issue, only one operation at a time - with alpha or with colors, so I do it in two or more steps, stripping alpha mask for one step and returning it for another. Maybe this is intended, but surely not convenient.

As written on its official site, SAI developer is a one-man-company and does user support via email. There is no public mailing list or forum, any roadmap or request-FAQ (like table of “planned”, “won’t do”, etc.). He only ever responded to me on some bug reports (including one not-a-bug), never on suggestions and feature requests, so I don’t know what he thinks about supporting OpenRaster. At least he read it, because I wrote something about inconsistent blending modes along the way, and in some next version blending modes were reconsidered and fixed to align closer to Photoshop.

Yes, more or less. SAI’s native format can contain editable objects like text, vectors, rulers, etc. If I export it to PSD and load back the result, everything is rasterized, but looks as it should. I can continue to draw in PSD without problems if I don’t need to keep any SAI-specific things as objects. It also looks as it should in my custom renderer, to the extent covered by my rendering implementation (PSD.js library does not have it):

  1. For some blending modes, which I never used, I have wrong or even missing formulas, as web search turned up nothing good at the time.
  2. Passthrough ignored in non-trivial cases, as mentioned before.
  3. Lower precision of calculations yields more banding in smooth gradients. I am planning to fully rewrite my renderer to 16-bit calculation later, to have closer results to SAI and not suffer from Firefox JS canvas implementation that stores colors with premultiplied-alpha, degrading precision even more.

My files also look as intended in Photopea online editor, and load much faster there than on my page, but it’s closed-source. Probably WebAssembly, maybe possible to disassemble and find some clues, but I did not try yet. Maybe I won’t even need to when the time comes.

Surprisingly, some PSD files from SAI looked broken in CSP and Photoshop itself last time I asked someone else to test, because they support clipping-group property only on layers, but not on folders for some reason. Photoshop even shows a misplaced checkbox when it’s enabled on a folder, but ignores it for rendering purposes. Maybe it’s fixed in some later version, but I would not count on it.

IIRC Photoshop had some option to save a project in compatibility mode. Maybe SAI sticks to that as a common ground.

1 Like

Hi f2d,

I really appreciate your comprehensive replies. I will likely need to respond to this multiple times to get all of my thoughts out as I think of more aspects.

In relation to your javascript renderer (For the ORA files specifically, not other raster formats), it seems that we have some duplicate efforts: as I started to get familiar with the format, I also tried to create a similar ora saver / renderer. This one uses GPU.js to vectorize each blend mode / composite. But there are definitely still areas for performance improvement. Maybe we could trade ideas on how to best speed up the rendering? I don’t think anyone else used my lib yet so I would love some feedback. In my case usually the unpacking of the PNG files from the zip was many times slower than rendering, actually. (Usually used Pepper and Carrot comics as a good large test file).

“alpha-composite-op” and “color-composite-op” sounds fine to me, but they will likely need to be argued / finalized at LibreGraphicsMeeting. I was planning on attending but with conditions this year I am not sure if it will be cancelled or not. (Hopefully if so, virtual meetings will be available)

I tried to make it easier for potential ORA users like SAI to use the format with a python library, too, but the mindset may be to do everything in c++ instead. I can perhaps gather myself to make a c or c++ version sometime in the future if that would be of greater use.

I noticed throughout this exploration that, like you, I have seen many small inconsistencies across the exact definitions / implementations of different named blend modes across programs. This is why I mentioned it might be useful to embed some kind of vectorizable spec-language into the ORA file itself describing how each pixel should be blended, but choosing how to safely serialize this type of language seems to be nontrivial and that’s why I thought it would be a good topic for LGM. Would love to know if you think this would be a nice solution in general. (Even more difficult but more useful would be to describe the non-destructive masks / filters generally in the ORA files, but that feels like an even greater security issue :pensive:)

I think the only things I could reuse are the most low level ones - to parse files and to blend two layers with a given method (not a whole tree). The rest became too specific to its purpose. To summarize:

  1. Parse a source file and get a library-specific JS object. Here I just use formats and methods that someone else made.
  2. Convert that object via library-specific wrapper into my custom generic tree form.
  3. Throw this tree into the renderer, which on the fly applies modifications and effects depending on user-selected options. This includes skipping, copying, reordering or recoloring layers (which could probably be done on a temporary copy of tree prior to supplying it to a generic whole-tree renderer), mirroring some rendered parts of a tree or generating outlines (which is too custom, but in theory could be made with some serialized formulas, or by rendering some branches separately).

If GPU.js allows calculations in 16-bit or more per channel, that would be good to use in future. For now I want to finish my specific functionality, maybe write a manual, and use it with my artwork. I wanted such a tool for years.

What I noticed is, Krita saves full-size uncompressed PNGs and relies on ZIP compression instead, which may be optimal for native code, but bad in JS (at least with libraries I tried).

But when I unpacked test ORA files from Krita, ran optimizer over PNGs and packed back, loading time, RAM and CPU usage became several times less. 10x less time to load a specific ORA file listed in examples there.

I should note that PNG decoder listed on my project page is not used anywhere in the code, still hanging in TODO. All work for now is: ZIP → PNG file → base64 or blob → store as src of image element to draw on a canvas later. And this is still resource-hungry and slow to load.

Today I tried to emulate this in Krita:

| layer 1 (clipped + blending mode 1)
| layer 2 (clipped + blending mode 2)
base layer

Like this:

folder 1 (inherit alpha + passthrough)
|- layer 1 (blending mode 1)

folder 2 (inherit alpha + passthrough)
|- layer 2 (blending mode 2)

base layer

Which works as intended, and exports into ORA without dropping anything, like this (irrelevant stuff omitted):

<stack name="folder 1" composite-op="svg:src-atop" isolation="auto">
	<layer name="layer 1" composite-op="blending mode 1"/>
</stack>
<stack name="folder 2" composite-op="svg:src-atop" isolation="auto">
	<layer name="layer 2" composite-op="blending mode 2"/>
</stack>
<layer name="base layer"/>

Resulting ORA looks in Krita exactly as its source KRA file. I added them to the folder from my previous post.

This does not work in my renderer now because it thinks of passthrough as one of blending modes, which overwrites “src-atop”. I can probably fix this later.

Also there can be more complex cases where emulation using folders would break in Krita too. Alpha masks surely break. By the way, mask-src/x/y attribute suggestion still stands.

And after all, I don’t like a solution that modifies user-made tree structure. Reading application could probably recognize patterns to reassemble it back, but this could also trigger in unwanted cases. Writing application could put some marker telling about the intention of such structures, but at this point why not add some actually useful attribute.

Sounds like a nice addition (not replacement) to have, and more headaches to support.
This would not only need built-in formula-parsing and blending library routines, but also applications rewritten to use them where specified, because their internal implementation may be different.

If done though, I think the code should not be stored on each layer, but a descriptive node separate from root layer stack, containing an element per named blending mode / filter / effect used in the document. I vaguely remember seeing something like that when searching about open layered formats before.

I have not implemented the following yet, but my idea of rendering a folder with passthrough and other properties enabled (= P) is this:

  1. Get result rendered before P (= B) and store a copy (= C).
  2. Draw an isolated queue consisting of C under the layers, contained in P, get result (= A).
  3. Interpolate B to A, using mask multiplied by opacity of P as the gradient map (degree of transition from B to A for each pixel).

Krita already does this without the mask part, only opacity.

Interpolation gradient (is this the right term?) is not the same thing as “dst-in” (the usual alpha mask operation) or any other mode, I think, which means that to support a mask in such scenario, application must either:

  • Treat topmost “dst-in” layer in a stack differently.
  • Disallow or ignore non-topmost “dst-in” layers.
  • Add a different attribute for this case (e.g. “mask-src/x/y”), which won’t be supported by some programs, but at least won’t break behavior of what is already in standard.

I wrote before that I would not like modifying user-made tree structure, but if it was a fully working solution with the least deviation from existing standard, I could live with it.

Please correct me if I got something wrong.

1 Like

Hi f2d,

I want to apologize for leaving you hanging. Things have been a little hectic.

This is fine, it seems that we have some different goals in general. However, I think discussion and documentation of the methods used for each project would help everyone involved understand exactly how the rendering is done. You might also be able to use some of the GPU.js methods for rendering two canvases together when they are fully bug tested.

This is true, I did achieve much better results using a smaller data size png file for each layer. The JS zip libraries seem to do much better with small data size as opposed to more aggressive compression, which is interesting. Unfortunately this does not help much with arbitrary files that a user might upload.

Yes, it is possible to re arrange the user layer structure to work around this limitation, but as you mentioned later, we would rather not redefine the user’s layers tree if possible.

Though I agree is is definitely a headache to get started, I think if the performance could be made to be acceptable, it could eventually be a replacement. In this way, not all of us need to re-learn every blending formula every time a new user of ORA is written. But I agree for practical purposes, this will likely not be realistic for quite a while. :frowning:

I am very interested in what proper method has been found to accomplish the stack rendering in general. In my previous implementation, I was focusing on rendering in order of absolute z-index only, but I realized that this is not a good solution for certain specific combinations of group isolation. For my next attempt I was thinking of iterating over layer groups in order of (depth first, lowest z-index second), but again the group passthrough property would make this less elegant: I would need to basically pseudo replace each passthrough group with its children, with the group visibility / opacity applied individually. Did you decide on your rendering method originally or did you peruse the Krita source algorithm to decide on it?

For passthrough + blendmode, I am a little lost because I don’t think there are examples of this in action. Does Photoshop or another program provide it? It sounds like you effectively are giving multiple blend modes between one layer and another? (Layer in a passthrough stack with ‘multiply’, passthrough stack has ‘darken_only’, the layer would need to be blended with the canvas below the stack with both ‘multiply’ and ‘darken_only’?) Have you found that this is usaful in the work that you do? I think we would need some examples of the use case before deciding on a final implementation.

I hope I don’t come across as too harsh. I really appreciate some of the examples you give and the mistakes I realized I made from it. I have a lot of catching up to do. :slight_smile:

I don’t mind, that’s fine with me, and I hope you are fine there.

What I meant by “not replacement” is, I think documents in future should still use named modes, but may contain standardized explanations for them.

Not that I know of.
Krita disables blendmode if passthrough is on, as mentioned before.
PS and SAI treat it as one of blendmodes, available only for folders.
Clipping is a separate property in all of these.

IIRC what I’ve seen of Krita sources to this moment is only some commit about “svg:add” being the same operation as “svg:plus”.

“Group opacity applied individually to its children” is a certainly no go, it will give dirty edges and such. I already tried that to emulate clipping when I was writing a JS drawpad with layers over 5 years ago (that project is abandoned in unfinished state).

I also had some crazy ideas about pipelining, maybe similar to yours, but it was not working even in my head. In the end I watched SAI results while dragging opacity sliders, and interpolation between before-after came up like a natural answer.

Today I added the transition operation I described above and it seems to work, but brings to front other problems that my renderer still has from long ago, and mirror-flipping logic is now like a broken bowl of spaghetti spilled all over the place. I’m still trying to fix it, so the new code is not yet uploaded.

The operation takes 3 buffers instead of 2 - image before, image after and the transition mask (I store it as RGBA too for simplicity, but use only its alpha channel). It may be standardized as such with attribute name like “transition-mask-src” for clarity, and hypothetically may be used for something else in future besides a transparency mask paired with passthrough.

1 Like

Okay, now that I made self-written blending work with partial alpha of both layers, next is the problem of “Fill” opacity property.

Photoshop supposedly uses it to make the layer content itself partially transparent (or even fully invisible) but keep its effects applied as normal (outlines, bezels, etc.). And then apply general “Opacity” property to the whole combined result.

Web search also showed that It affects some blending modes, e.g. “Vivid Light”, but I did not found how, and I don’t have Photoshop to drag sliders and see.

SAI2, after some update I mentioned before, now has two versions (normal and something to do with “Transparency Shapes Layer”, latter hidden by default) of the following 8 modes:

  • Linear Burn
  • Linear Dodge
  • Linear Light
  • Color Burn
  • Color Dodge
  • Vivid Light
  • Hard Mix
  • Difference

SAI developer never answered what they do and how it’s different.

In PSD files exported from SAI2, opacity is stored normally for layers with “TS” mode versions, but for non-TS mode versions partial value is stored as “Fill” opacity property, with general opacity at 100%.

When I load such files in my own renderer, I just multiply both values, so setting either means the same, but this is not a proper solution. Both versions of these modes look the same in my results (because they are called with the same name by PSD.js). At least the modes and properties I actually use in my work match my results more or less, so I don’t have to worry yet.

Krita simply ignores “Fill” opacity and gets the 100% general opacity, which is one of the reasons why my example file looked so broken. I have made some tweaks (opacity values and isolated subfolder for the planet surface) and put a Krita version into the same folder:

From this example we can see that blending operations in Krita match TS versions of SAI2, while my custom JS code matches non-TS versions, at least for the modes used in this file.

And that alpha mask + passthrough combo still does not work in Krita as of 4.2.9 release version.

I’m not sure if this is relevant to OpenRaster specs and not just how applications interpret their supported blendmodes.

1 Like

What does ‘fill’ opacity do in general compared to normal opacity?

I may have worded it confusingly, anyway there is official page, and here is example I got from someone else before, it was in Photoshop version 19.0:

P.S. SAI itself does not let user modify “Fill” opacity, only uses the property in PSD export, and probably simply multiplies both values when importing too (just a guess).

That’s for layers with layer styles. Not sure if ORA supports layer styles at all?