Call for help: assist development of ORA format providing KRA, PSD or other source files

Develop
1

Good evening.

Over the past few months, I have been working with some of the developers of free art tools such as Krita and Gimp/Glimpse to develop improvements for the ORA graphics interchange format (https://www.openraster.org/). It is like PSD format, but not proprietary. Over the course of my research so far, I have found several examples where PSD provides features which are not available in ORA. In order to propose solutions to these issues and continue to make great tools available for free, I will be hosting a panel in this years Libre Graphics Meeting (https://libregraphicsmeeting.org) to push for changes. (and I will be doing some coding too ^.^)

My plea to any digital artist who reads this: If you have any old KRA, PSD (or procreate, CSP, xcf, whatever) digital art source files that you would be able to donate prior to the meeting (May 2020), especially those which use a lot of advanced features like multiple clip or blending layers, compositing modes, etc, please let me know! I would really like to make sure that all of my proposals at LGM can address any shortcomings of the current free standards so that we can make the maximum possible progress this year!

Two of the major issues I want to address currently:

-Lack of vector / text capability

-No support for clipping layer combined with another mode, like add / multiply (which many artists use for things like highlights and shadows)

Please let me know if you are able to help and have an awesome day!

5 Likes
2

Welcome to the forum! I hope people provide some sample files. I’ll see if I have some test files too.

2 Likes
3

Important work. I might provide something, will look into my files to see if anything might be of use.

1 Like
4

I have provided a sample Krita file with advanced layer formatting here - https://discuss.pixls.us/t/pat-david-luminosity-mask-for-krita-4-3-tutorial/16452

Scroll down, and download that .kra files provided.

2 Likes
5

I’ll just note that MyPaint uses ORA as it’s native format too and they currently added spectral colour mixing.

6

FYI, it seems we are having a more advanced conversation about this on this thread: MyPaint 2 release & ora format - #12 by Reptorian for anyone who is interested in contributing to the conversation. :slight_smile:

7

Hello, I have some interest in advancement of ORA format, so I conjured a small example file just now, if it’s still needed:

I mainly draw in SAI, which can export to PSD, and use clipping a lot.

The main caveat I think is “passthrough” blending mode paired with other things, like partial opacity, layer mask, clipping or all at once enabled on one folder. This requires some complex setup to stay within ORA standard, and will possibly not work in some cases. This maximum combo is not included in example file above (see the planet rings for the most complex thing there is), but I can make something later if still needed.

In an piece of software I’m developing lately, I decided to add following custom XML attributes for testing:

clipping=“put anything that is not none to enable” (I write “group-to-below” here)
mask-src=“data/mask_image.png”
mask-x=“number”
mask-y=“number”

I don’t remember why I decided to go with separate attribute for clipping, it was more than a year ago, but maybe just adding a second op with “src-atop” was not enough.
I also enable passthrough mode with isolation=“auto”.

When the standard becomes capable enough, I will adjust my creation to follow it, and probably ask SAI developer about ORA support again.

1 Like
8

Hi f2d. Thanks! I am still open to suggestions for improvements. It is good to know that the SAI developers may be responsive. I took a look at the LibSAI project for a while to check for conversions to ORA before realizing that maintaining something like that would likely be too daunting. I am not sure the exact overlap of their features to PSD. Do the PSD that you get from SAI usually represent exactly what you intend from editing in SAI, or are there features missing?

Is the passthrough feature that you have issues with when opening in Krita, Photoshop, or SAI? I notice immediately that the PSD you provided does not open in Kritalooking the same as the rendered PNG, but I notice that might be because the transparency masks are not correctly imported from the PSD format. I can investigate if there would be any combination of blend mode that would work, but what I am guessing is that you were also interested in adding a clip mode and blend mode together as I have described earlier?

Thanks

9

I open files with my custom JavaScript renderer, a work in progress yet far from production quality. It uses PSD.js to parse PSD files, and as a consequence it supports even less than that (but at least the subset of features that I actually use in my files created in SAI). It does not even really support passthrough. If a folder with this mode does not have any other property that affects rendering (partial opacity, mask, etc.), it just dumps its content into the render queue (allowing to better organize layers with blending modes or just create less temporary rendering subcontexts), and otherwise ignores this mode. I have some thoughs about how to implement it properly, but it raises complexity with possibly more nested passthrough folders, and my mind breaks there so I just put it off for now. I don’t use it in my files for more than the most simple way - just to group some layers.

Today I installed Krita’s latest beta version from the site - 4.2.9-beta1 x64. It allows to set passthrough mode separately from blending mode, but thankfully one disables another (I cannot imagine how they’d work at once). Also it disables masks but not partial opacity, so if I disable it and restore inherit-alpha where the clipping was, example file above looks not as much broken, but some blending is still different.

I tested standard src-atop versus my non-standard clipping group, and it seems that the difference amounts to arithmetic precision of rounding in 8-bit calculations. I am not used to drawing in Krita, so excuse me for a not so fancy example this time:

Also for some reason “Addition” blending mode (same operation as “Linear Dodge”, “svg:plus”, etc.) in Krita looks different from my JS renderer (using “lighter” blending mode or custom-written asm.js addition, which both show the same result). “Screen” mode looks the same everywhere, so I added this variant to comparison.

Yes. After all, clip mode + blend mode looks like a good solution. I’d suggest to add “alpha-composite-op” and “color-composite-op” attributes, let applications make sense of possible combinations, and keep old combined “composite-op” for backward compatibility, but there may be better ideas.

By the way, JS canvas API suffers from the same issue, only one operation at a time - with alpha or with colors, so I do it in two or more steps, stripping alpha mask for one step and returning it for another. Maybe this is intended, but surely not convenient.

As written on its official site, SAI developer is a one-man-company and does user support via email. There is no public mailing list or forum, any roadmap or request-FAQ (like table of “planned”, “won’t do”, etc.). He only ever responded to me on some bug reports (including one not-a-bug), never on suggestions and feature requests, so I don’t know what he thinks about supporting OpenRaster. At least he read it, because I wrote something about inconsistent blending modes along the way, and in some next version blending modes were reconsidered and fixed to align closer to Photoshop.

Yes, more or less. SAI’s native format can contain editable objects like text, vectors, rulers, etc. If I export it to PSD and load back the result, everything is rasterized, but looks as it should. I can continue to draw in PSD without problems if I don’t need to keep any SAI-specific things as objects. It also looks as it should in my custom renderer, to the extent covered by my rendering implementation (PSD.js library does not have it):

  1. For some blending modes, which I never used, I have wrong or even missing formulas, as web search turned up nothing good at the time.
  2. Passthrough ignored in non-trivial cases, as mentioned before.
  3. Lower precision of calculations yields more banding in smooth gradients. I am planning to fully rewrite my renderer to 16-bit calculation later, to have closer results to SAI and not suffer from Firefox JS canvas implementation that stores colors with premultiplied-alpha, degrading precision even more.

My files also look as intended in Photopea online editor, and load much faster there than on my page, but it’s closed-source. Probably WebAssembly, maybe possible to disassemble and find some clues, but I did not try yet. Maybe I won’t even need to when the time comes.

Surprisingly, some PSD files from SAI looked broken in CSP and Photoshop itself last time I asked someone else to test, because they support clipping-group property only on layers, but not on folders for some reason. Photoshop even shows a misplaced checkbox when it’s enabled on a folder, but ignores it for rendering purposes. Maybe it’s fixed in some later version, but I would not count on it.

IIRC Photoshop had some option to save a project in compatibility mode. Maybe SAI sticks to that as a common ground.

1 Like
10

Hi f2d,

I really appreciate your comprehensive replies. I will likely need to respond to this multiple times to get all of my thoughts out as I think of more aspects.

In relation to your javascript renderer (For the ORA files specifically, not other raster formats), it seems that we have some duplicate efforts: as I started to get familiar with the format, I also tried to create a similar ora saver / renderer. This one uses GPU.js to vectorize each blend mode / composite. But there are definitely still areas for performance improvement. Maybe we could trade ideas on how to best speed up the rendering? I don’t think anyone else used my lib yet so I would love some feedback. In my case usually the unpacking of the PNG files from the zip was many times slower than rendering, actually. (Usually used Pepper and Carrot comics as a good large test file).

“alpha-composite-op” and “color-composite-op” sounds fine to me, but they will likely need to be argued / finalized at LibreGraphicsMeeting. I was planning on attending but with conditions this year I am not sure if it will be cancelled or not. (Hopefully if so, virtual meetings will be available)

I tried to make it easier for potential ORA users like SAI to use the format with a python library, too, but the mindset may be to do everything in c++ instead. I can perhaps gather myself to make a c or c++ version sometime in the future if that would be of greater use.

I noticed throughout this exploration that, like you, I have seen many small inconsistencies across the exact definitions / implementations of different named blend modes across programs. This is why I mentioned it might be useful to embed some kind of vectorizable spec-language into the ORA file itself describing how each pixel should be blended, but choosing how to safely serialize this type of language seems to be nontrivial and that’s why I thought it would be a good topic for LGM. Would love to know if you think this would be a nice solution in general. (Even more difficult but more useful would be to describe the non-destructive masks / filters generally in the ORA files, but that feels like an even greater security issue :pensive:)

11

I think the only things I could reuse are the most low level ones - to parse files and to blend two layers with a given method (not a whole tree). The rest became too specific to its purpose. To summarize:

  1. Parse a source file and get a library-specific JS object. Here I just use formats and methods that someone else made.
  2. Convert that object via library-specific wrapper into my custom generic tree form.
  3. Throw this tree into the renderer, which on the fly applies modifications and effects depending on user-selected options. This includes skipping, copying, reordering or recoloring layers (which could probably be done on a temporary copy of tree prior to supplying it to a generic whole-tree renderer), mirroring some rendered parts of a tree or generating outlines (which is too custom, but in theory could be made with some serialized formulas, or by rendering some branches separately).

If GPU.js allows calculations in 16-bit or more per channel, that would be good to use in future. For now I want to finish my specific functionality, maybe write a manual, and use it with my artwork. I wanted such a tool for years.

What I noticed is, Krita saves full-size uncompressed PNGs and relies on ZIP compression instead, which may be optimal for native code, but bad in JS (at least with libraries I tried).

But when I unpacked test ORA files from Krita, ran optimizer over PNGs and packed back, loading time, RAM and CPU usage became several times less. 10x less time to load a specific ORA file listed in examples there.

I should note that PNG decoder listed on my project page is not used anywhere in the code, still hanging in TODO. All work for now is: ZIP → PNG file → base64 or blob → store as src of image element to draw on a canvas later. And this is still resource-hungry and slow to load.

Today I tried to emulate this in Krita:

| layer 1 (clipped + blending mode 1)
| layer 2 (clipped + blending mode 2)
base layer

Like this:

folder 1 (inherit alpha + passthrough)
|- layer 1 (blending mode 1)

folder 2 (inherit alpha + passthrough)
|- layer 2 (blending mode 2)

base layer

Which works as intended, and exports into ORA without dropping anything, like this (irrelevant stuff omitted):

<stack name="folder 1" composite-op="svg:src-atop" isolation="auto">
	<layer name="layer 1" composite-op="blending mode 1"/>
</stack>
<stack name="folder 2" composite-op="svg:src-atop" isolation="auto">
	<layer name="layer 2" composite-op="blending mode 2"/>
</stack>
<layer name="base layer"/>

Resulting ORA looks in Krita exactly as its source KRA file. I added them to the folder from my previous post.

This does not work in my renderer now because it thinks of passthrough as one of blending modes, which overwrites “src-atop”. I can probably fix this later.

Also there can be more complex cases where emulation using folders would break in Krita too. Alpha masks surely break. By the way, mask-src/x/y attribute suggestion still stands.

And after all, I don’t like a solution that modifies user-made tree structure. Reading application could probably recognize patterns to reassemble it back, but this could also trigger in unwanted cases. Writing application could put some marker telling about the intention of such structures, but at this point why not add some actually useful attribute.

Sounds like a nice addition (not replacement) to have, and more headaches to support.
This would not only need built-in formula-parsing and blending library routines, but also applications rewritten to use them where specified, because their internal implementation may be different.

If done though, I think the code should not be stored on each layer, but a descriptive node separate from root layer stack, containing an element per named blending mode / filter / effect used in the document. I vaguely remember seeing something like that when searching about open layered formats before.

12

I have not implemented the following yet, but my idea of rendering a folder with passthrough and other properties enabled (= P) is this:

  1. Get result rendered before P (= B) and store a copy (= C).
  2. Draw an isolated queue consisting of C under the layers, contained in P, get result (= A).
  3. Interpolate B to A, using mask multiplied by opacity of P as the gradient map (degree of transition from B to A for each pixel).

Krita already does this without the mask part, only opacity.

Interpolation gradient (is this the right term?) is not the same thing as “dst-in” (the usual alpha mask operation) or any other mode, I think, which means that to support a mask in such scenario, application must either:

  • Treat topmost “dst-in” layer in a stack differently.
  • Disallow or ignore non-topmost “dst-in” layers.
  • Add a different attribute for this case (e.g. “mask-src/x/y”), which won’t be supported by some programs, but at least won’t break behavior of what is already in standard.

I wrote before that I would not like modifying user-made tree structure, but if it was a fully working solution with the least deviation from existing standard, I could live with it.

Please correct me if I got something wrong.

1 Like
13

Hi f2d,

I want to apologize for leaving you hanging. Things have been a little hectic.

This is fine, it seems that we have some different goals in general. However, I think discussion and documentation of the methods used for each project would help everyone involved understand exactly how the rendering is done. You might also be able to use some of the GPU.js methods for rendering two canvases together when they are fully bug tested.

This is true, I did achieve much better results using a smaller data size png file for each layer. The JS zip libraries seem to do much better with small data size as opposed to more aggressive compression, which is interesting. Unfortunately this does not help much with arbitrary files that a user might upload.

Yes, it is possible to re arrange the user layer structure to work around this limitation, but as you mentioned later, we would rather not redefine the user’s layers tree if possible.

Though I agree is is definitely a headache to get started, I think if the performance could be made to be acceptable, it could eventually be a replacement. In this way, not all of us need to re-learn every blending formula every time a new user of ORA is written. But I agree for practical purposes, this will likely not be realistic for quite a while. :frowning:

I am very interested in what proper method has been found to accomplish the stack rendering in general. In my previous implementation, I was focusing on rendering in order of absolute z-index only, but I realized that this is not a good solution for certain specific combinations of group isolation. For my next attempt I was thinking of iterating over layer groups in order of (depth first, lowest z-index second), but again the group passthrough property would make this less elegant: I would need to basically pseudo replace each passthrough group with its children, with the group visibility / opacity applied individually. Did you decide on your rendering method originally or did you peruse the Krita source algorithm to decide on it?

For passthrough + blendmode, I am a little lost because I don’t think there are examples of this in action. Does Photoshop or another program provide it? It sounds like you effectively are giving multiple blend modes between one layer and another? (Layer in a passthrough stack with ‘multiply’, passthrough stack has ‘darken_only’, the layer would need to be blended with the canvas below the stack with both ‘multiply’ and ‘darken_only’?) Have you found that this is usaful in the work that you do? I think we would need some examples of the use case before deciding on a final implementation.

I hope I don’t come across as too harsh. I really appreciate some of the examples you give and the mistakes I realized I made from it. I have a lot of catching up to do. :slight_smile:

14

I don’t mind, that’s fine with me, and I hope you are fine there.

What I meant by “not replacement” is, I think documents in future should still use named modes, but may contain standardized explanations for them.

Not that I know of.
Krita disables blendmode if passthrough is on, as mentioned before.
PS and SAI treat it as one of blendmodes, available only for folders.
Clipping is a separate property in all of these.

IIRC what I’ve seen of Krita sources to this moment is only some commit about “svg:add” being the same operation as “svg:plus”.

“Group opacity applied individually to its children” is a certainly no go, it will give dirty edges and such. I already tried that to emulate clipping when I was writing a JS drawpad with layers over 5 years ago (that project is abandoned in unfinished state).

I also had some crazy ideas about pipelining, maybe similar to yours, but it was not working even in my head. In the end I watched SAI results while dragging opacity sliders, and interpolation between before-after came up like a natural answer.

Today I added the transition operation I described above and it seems to work, but brings to front other problems that my renderer still has from long ago, and mirror-flipping logic is now like a broken bowl of spaghetti spilled all over the place. I’m still trying to fix it, so the new code is not yet uploaded.

The operation takes 3 buffers instead of 2 - image before, image after and the transition mask (I store it as RGBA too for simplicity, but use only its alpha channel). It may be standardized as such with attribute name like “transition-mask-src” for clarity, and hypothetically may be used for something else in future besides a transparency mask paired with passthrough.

1 Like
15

Okay, now that I made self-written blending work with partial alpha of both layers, next is the problem of “Fill” opacity property.

Photoshop supposedly uses it to make the layer content itself partially transparent (or even fully invisible) but keep its effects applied as normal (outlines, bezels, etc.). And then apply general “Opacity” property to the whole combined result.

Web search also showed that It affects some blending modes, e.g. “Vivid Light”, but I did not found how, and I don’t have Photoshop to drag sliders and see.

SAI2, after some update I mentioned before, now has two versions (normal and something to do with “Transparency Shapes Layer”, latter hidden by default) of the following 8 modes:

  • Linear Burn
  • Linear Dodge
  • Linear Light
  • Color Burn
  • Color Dodge
  • Vivid Light
  • Hard Mix
  • Difference

SAI developer never answered what they do and how it’s different.

In PSD files exported from SAI2, opacity is stored normally for layers with “TS” mode versions, but for non-TS mode versions partial value is stored as “Fill” opacity property, with general opacity at 100%.

When I load such files in my own renderer, I just multiply both values, so setting either means the same, but this is not a proper solution. Both versions of these modes look the same in my results (because they are called with the same name by PSD.js). At least the modes and properties I actually use in my work match my results more or less, so I don’t have to worry yet.

Krita simply ignores “Fill” opacity and gets the 100% general opacity, which is one of the reasons why my example file looked so broken. I have made some tweaks (opacity values and isolated subfolder for the planet surface) and put a Krita version into the same folder:

From this example we can see that blending operations in Krita match TS versions of SAI2, while my custom JS code matches non-TS versions, at least for the modes used in this file.

And that alpha mask + passthrough combo still does not work in Krita as of 4.2.9 release version.

I’m not sure if this is relevant to OpenRaster specs and not just how applications interpret their supported blendmodes.

1 Like
16

What does ‘fill’ opacity do in general compared to normal opacity?

17

I may have worded it confusingly, anyway there is official page, and here is example I got from someone else before, it was in Photoshop version 19.0:

fill_opacity
fill_opacity600×880 62 KB

P.S. SAI itself does not let user modify “Fill” opacity, only uses the property in PSD export, and probably simply multiplies both values when importing too (just a guess).

18

That’s for layers with layer styles. Not sure if ORA supports layer styles at all?

19

Not by the current standard. Although MyPaint does a number of things outside of it, some of which I intend to support later (e.g. tiled bg layers).

Speaking of which, I recently implemented ORA export from JS on my page, which adds some custom properties mentioned above. When the exported files are loaded back, they give me exactly the same result as my source PSD files from SAI2.

My code omits everything default and prefixes custom additions with a “test:” namespace. The stack.xml ends up looking like this:

<layer name="shade" x="566" y="202" composite-op="svg:src-atop" test:alpha-composite-op="svg:src-atop" test:color-composite-op="test:linear-light" src="data/layer114.png"/>
<layer name="color" x="569" y="205" composite-op="svg:src-atop" test:alpha-composite-op="svg:src-atop" test:color-composite-op="svg:multiply" src="data/layer115.png"/>
<layer name="ground shadow" x="514" y="1375" test:mask-x="448" test:mask-y="1299" src="data/layer125.png" test:mask-src="data/mask9.png"/>

I prioritized putting alpha-composition modes into the old composite-op attribute when there are two modes present, because when it’s the only thing supported, working clipping at least keeps the form instead of spilling color blobs all around the drawing.

Resulting files load fast and “fine” in Krita 4.4.1, although my custom additions are obviously ignored.

They also load “fine” in GIMP 2.10.22, but very slow - it saves a temp PNG file in a temp folder on disk and calls an external exe parser for each layer. At least it finishes. GIMP had exactly the same speed problems years ago with loading a number of plain PNGs from disk, when I tried making GIFs. It’s probably done for code portability, or versatility, or something.

MyPaint, well, is not so fine. It loads some of my exported ORA files, but shows some “unexpected NoneType” error on some others. The reason of error seems to be layers without an src or with an empty string in it. Default omissions and custom additions are finely ignored. Converting empty layers in export into empty stacks will probably do the trick, although I’m hesitant. Adding a completely transparent 1x1 PNG is probably a better solution, maybe even only one file stored and shared between layers. Reusing layer srcs in general is not a violation of the standard, I suppose.

Is src="data:base64uri" (without storing a proper image file in ZIP) a possibility, or a viable suggestion to the future standard?
I’d use it for color-fill or empty layers. Hell, I’d use it for every tiny part layer under 100 KB and even GIMP would probably load hundreds of them fast.

1 Like
20

After more testing in MyPaint, I now know the reason why just composite-op="svg:src-atop" will not be enough, and another specific clipping attribute would be needed. It is because "svg:src-atop" does not implicitly create an isolated rendering subcontext. It clips atop everything rendered before it combined. Same with masks, i.e. it is not possible to limit effects of a passthrough folder to a mask area without leaking the mask operation, until we add another isolation mode.

I propose isolation="isolate-alpha". Mask operation ("svg:dst-in" or "svg:dst-out") inside an "isolate-alpha" folder would change into a transition from merged content before the folder (B) and content inside the folder up to the mask merged with B, using the mask as transition values for each pixel. Basically the same operation I proposed for rendering top-most mask in the folder, except that with a new folder mode the mask can be placed anywhere inside between layers.

Since the last post I rewrote my webpage ORA export code to add wrapper folders and reassign attributes in a way that would not do anything incorrect (clipping groups are explicit now), and would exactly match rendering of the PSD file it was exported from.

Although out of my files, that I reexported from PSD to ORA using my new code, isolation="isolate-alpha" was added only to the “planet.ora” (created for tests in this thread), all my other ORA files that use only standard isolation="auto" and isolation="isolate" still look broken in Krita v4.4.3, and this is not just because of blending modes mismatch.

Krita does something different with its “Passthrough” toggle assigned from isolation="auto", that I don’t really understand. Can someone explain what it exactly does, or at least should or tries to do? A page from manual says it does the same as MyPaint, if “visibility” does not include opacity and masks and just disable them. Yet MyPaint and Krita show very different results on my public test files.

Also, since isolation becomes non-binary with this proposal, I would further propose to leave isolation="auto" to mean the old standard trivial thing that MyPaint also does - dump layers (including clipped-atop layers and clipping/erasing masks) from the folder into the rendering queue without creating subcontext, if the folder has 100% opacity and normal blend mode, otherwise treat as default folder and merge its content in isolated subcontext.

And then I’d add another value for a full explicit passthrough (not any kind of “auto”). So, one new value that isolates alpha masks and maybe another that does not, i.e. leaks all operations but applies folder opacity via transition, e.g. isolation="pass-through".

And by the way, with these new isolation values, other attributes proposed above become unneeded, if not for preserving exact layer tree structure. Namely, separate color/alpha-composition, mask-src and my old clipping property with implicit group subcontext, borrowed from PSD format. I still support reading them for tests and writing with a config switch, but I’m OK to never make them standard after all. My app-specific parameters relative to layer position in the tree ended up working as is in exported files even with added wrapper folders, as my export code is careful enough about what properties belong to what context, including layer names with user-written parameters in them.