Node-based brush designer (including mockups)

Krita has been wonderful for me in the year and a half or so I’ve been using it semi-professionally, but one of the issues I have with the software is the Brush Editor.

Even though I’ve made a few brushes myself, and modified other people’s for my needs, it’s hard to get a good idea about how a brush works at a glance and it’s hard to know where to start when I have an idea for a new brush in mind.

I think that offering a node-based editor for brush designing and editing would not only be useful, but also be an cool development in digital art.

Overview

  1. Background
  • what’s wrong with the existing brush editor?
  • what’s good about the existing brush editor?
  • what is a node-based editor?
  • why might a node-based editor be a suitable brush editor?
  1. Initial concept
  • concept mockups
  • exposing a node editor library for plugins (with existing examples)
  1. Accessibility

1. Background

What’s wrong with the existing brush editor?

My main issue with the current brush editor is that it’s a verbose interface that’s more an options menu than something I feel confident building tools with.

The current brush editor is a verbose interface with a lot of space dedicated to checkboxes, curves, dials, etc. that is noisy to parse if you’re not already very experienced with it. It’s useful in that it exposes all values that can be given to a brush and its verbosity lends itself to poking and prodding, but reading the brush’s behaviour at a glance and being confident in where to start on creating a new brush doesn’t come naturally to this form of brush editor.

What’s good about the existing brush editor?

The scratchpad provides a good testing ground, and I am not proposing an overhaul of the way brushes work (mainly because that would sink this proposal and be clearly too far out of scope). While I do think that a node-based brush editor might expose possibilities of more user-definable inputs to brushes in future, I feel that including that within an initial pre-proposal is premature.

What is a node-based editor?

A node-based editor (or “graph editor”, there’s not really a standardised name?) is a method of building behaviour in directed graphs. Each “node” is a card with annotations about inputs, outputs, and what it does with those inputs and outputs that can be dragged around a workspace and connected to other nodes. Nodes are connected output-to-input to “sockets” that correspond to the compatible data types (i.e. numbers, textures, position etc.) in a way that lets the user design and compose fairly complex behaviour.

Node-based editors have become popular in game engines and 3D software such as Blender (which even has an internal initiative to make as many components of the software able to be done via nodes) and Unity. Blender and Unity have node-based shader & scripting editors, and Blender has procedural geometry via node-based geometry editors in the works in an upcoming version.

What makes a node-based editor a suitable brush editor?

  • Easier to “read” brushes

As an artist I like to have an idea of what my tools do, this means I need to dig into the editor interface anyway. Opening a node-developed brush in an editor can show me how the whole thing works fairly easily, with me just needing to pan about the scene to see what’s going on if it can’t all fit on screen, and an output node always being the place to start looking for what’s going on.

  • Easier to make brushes

When I’m making or modifying brushes it can feel like I don’t know where to start or I’m unsure if the behaviour I want is even doable in the current brush engine and it’s difficult to experiment with different settings and still have a good idea of what’s going on. I’m sure you can learn the ins and outs of the editor in its current form but I don’t feel like it’s built for experimentation and planning.

  • Paths of feature extensibility

A node-based editor also opens up potential for more complex brushes in the future, if such things were able to be supported. This is more a footnote, I am aware of the amount of work that would have to i.e. go into letting us have scripted drivers for values like Blender’s shaders do, but there’s brush features with less of an overhead that I feel a node-based editor would help open up possibility for.

2. Initial concept

Note that I’m not proposing anything here 1. uses the mockup style I’ve used 2. must exist, this is just me putting forward some ideas and using a unified style to make them all look coherent. An editor would have to account for i.e. themes and such.

Concept mockups

This mockup shows:

1) An output node suited to a specific brush engine
2) Input data nodes (i.e. tilt, pressure, time)
3) Texture nodes
4) Socket compatibility (red to red, grey to grey)
5) Transforms on data (i.e. the curve transforms in the existing editor)
6) Basics of an interface for adding additional / changing nodes via a menu bar etc.

I’ve also sketched some other nodes, knowing these are only variable degrees of possible.

a) A shortform of plugging i.e. pressure directly into a curve transform.
b) Combining data, in a similar way to the way data is combined to effect brushes currently, but considering each one with a different weight.
c) Data constants, for i.e. a fixed rotation.
d) Feature switch, for determining how a brush works based on features supported by an input device (i.e. a graphics tablet, I know my old bamboo doesn’t support tilt but by cintiq does). I don’t know if Krita has the ability to poll for input device features so I expect this to not be possible without having access to a database of input device features but I think the concept is worth bringing up.

I don’t think these are necessities but instead just an exploration of how brush design might work and what different means of data input might look like.

Exposing a node editor API

One of the things that software such as Blender or Godot does is expose the tools for people to develop their own node-based editors for plugins and the like.

This involves extra work as opposed to directly writing a the node editor system for the brush editor alone, but it would also separate maintenance between the API itself and the brush editor. The Blender Foundation has a working group among its contributors that want to make as many things drivable by node editors as possible, I don’t think the same needs to apply to krita but giving this tool to plugin devs could lead to a bunch more behaviour being user-automated or easier to express.

Existing examples of general node editor APIs are:

3. Accessibility

There’s some accessibility considerations for node-based editors, namely you need to be able to visually parse them and if they work with in the QGraphicsItem class then that’s gonna be invisible to say, a screen reader. I want to know if there’s people who rely on the current layout for accessibility reasons, and if there’s a way to account for that in a possible way of implementing this.

Another thing to consider is the problems with using colours as socket type signifiers, this could be replaced with i.e. sockets that have a monochromatic icon signifying their type so that colourblind users would be able to use this system (I know of a few colourblind artists).


Anyway, that’s the work I’ve put into thinking about this concept. I’m interested in what other people think about this kind of thing? By no means do I want to dictate this kind of thing, nor do I want to get mad if this is something that’s just not possible given workloads etc. I just think it’ll be something cool, but I would be surprised if this gained traction, it’s a really hefty project in terms of the work that would be to be done and I don’t know the ins and outs of the krita brush engine infrastructure.

12 Likes

I’ve mentioned this possibility in a previous thread, and I started looking into creating it myself, since I’m already pretty familiar with the brush engine code. I have ideas how to implement it, but currently don’t have the time. It’s a great idea, though, and I would love to see it happen. Once implemented fully, it could replace all the other brush engines, especially if performance is on par with what it replaces.

6 Likes

while this is an interesting idea I don’t know if having it for brush settings would be a good thing. I can see some main problems with this approach:

  • this strays a lot from industry standards and would confuse new users
  • not everyone is familiar with a node approach (related to the first one)
  • seems a bit over-complicated for editing brushes, would increase the time to edit a brush

Now I know these things might be just my opinion but I will try to bring points that might be relevant, as I dont think I would be the only artist that has almost no experience with node based systems.

In your section of what is wrong with the current brush editor, you state and i quote

and I completely agree with you, it is verbose, which i don’t think its a bad thing in itself, though it can scare new users. I just don’t know why a node base system would help in this case, a node base approach just doesn’t seem any more intuitive than the current system.

You quote how its a common thing for 3d software like blender but I want to bring your attention that while blender uses that its never for editing brushes, both sculpting and grease pencil brushes use a more traditional settings approach.
You list some things that you consider that are pros for this new system, and I will give my view on them.

  • Easier to “read” brushes

That’s debatable, if you are used to a node system it might be intuitive, but for someone just starting this will be confusing. While the current system is by no means intuitive to beginners it follows the same format as most other applications, its somewhat familiar.

  • Easier to make brushes

Again that is pretty much debatable, I can actually see people having more trouble with this than the current approach. As in the current approach you can see all the available options, and its a simple yes or no to use them. As with a node system people also need to connect them.

  • Paths of feature extensibility

While i agree this would be interesting i cant see much of a use case for this in a brush editor, would be awesome in the context of a composition workflow though.

I understand that you put a lot of thought into this and i agree the brush editor could have a rework but, I don’t think this is the right path. To explain the points I first listed:

  • this strays a lot from industry standards and would confuse new users

Many people think the industry standards are not that much important but they help the transition of new users into the program. Imagine the amount of questions that would flood the krita help channels if the brush editor was changed so drastically. And while there will be people saying “we shouldnt stall improvements just because it confuses beginners” every time a feature confuses beginners too much krita dev team either removes it or changes it, which shows that this is a concern to be taken into account.

  • not everyone is familiar with a node approach (related to the first one)

Again this node base system is not something that everyone is familiar with, people with 3d background might be a bit more familiar but we need to consider that a huge portion of the user base is not. changing something as essential as a brush editor will cause problems

  • seems a bit over-complicated for editing brushes, would increase the time to edit a brush

You say it would make easier to create a a brush, but is it really? for creating a brush in the current workflow we check the parameters we want to control, and then check how we are going to control them tweaking the curves and strength. just some clicks and tweaking a curve.

In a node base system, add the nodes, connect the nodes, tweak the values and maybe organizer how they look. doesn’t it seems like a bit more work?

So I have been here talking about how I don’t think this is a good approach for brush editing, but I will point out one thing that I think this approach might be helpful. Creating masked brush tips, it wouldn’t be a completely drastic change as changing the whole brush settings, and i feel like it might make that part of the workflow a bit more intuitive.

Well these are my thoughts on this, for such a drastic change I believe counter opinions are just important to list.

2 Likes

I like the idea in my view node based systems are far superior to layer based ones.

But I must say I am not quite sure on this. And for reasons you might not expect. 2d people just don’t like change. I really think is a cool idea but this will just hit the wall sadly.

I do think the modular setup is more telling and easier to not have the same menu repeated several times in a row. But the time to set it up I think is the only REAL downfall for this idea. If it had a little plus sign or something to add the correct node from a list to the correct soket it would make it faster and more dumb proof. Adding manually all the time might be too much and make people bored.

Because yeah 2D people complain for the most weird things and it puzzles me to this day, and this looks like a prime target for it. This will be UNREAL issues but will be the biggest one for this idea to overcome. People here don’t know that layers suck royally and don’t expect them too really.

Despite all you have my support.

Wow… :scream:

I’m a “2D guy”, and I don’t really complain about changes, I like to have new functionalities and/or to see existing functionalities being improved, even if it means that interface is entirely redesigned.

I’m a “2D guy”, and I usually try to use software possibilities as much as possible if it make sense (because I won’t use a functionality if I don’t need it just because the functionality is here :sweat_smile:)

I’m a “2D guy”, and I build my own tools when I can and I like to test how things are implemented in other software.

I’m a “2D guy”, and I like to have many options in settings and/or for tools I use/I build.

Concerning nodes, 20years ago (approximately) I was already using it in Adobe After Effect and it was something really cool (don’t know if it’s still here, except Office at the office, I didn’t really used a commercial software from a while now)
And currently I can use it with Natron.

So, that’s not cool to talk about 2D people like this :frowning:

Some people like change, some other don’t.

But I’m not sure it’s 2D artists habits again 3D artists.
Maybe 3D artists are more used to work with complex UI because drawing in 3D is technically more complex than drawing in 2D.

I think people who don’t like change are mostly (but even here my point of view can’t be verified) people for which use of a computer is not an easy thing (and these people don’t made 3D art I think)
If it took time to them to understand how to use it, changing the way of how you have to use it will be a pain for them.

Concerning Krita brush engine interface, if a user already took time to play with brush engine to create their own brushes, having a new interface probably won’t be a problem because current interface is not really a friendly interface and can be scary for beginners.

I use it to modify some existing brushes, but currently I don’t really use it intensively due to:

  • The interface
    – I don’t like the popup mode, you can’t move/resize the window, no OK/Cancel possibility…
    – I don’t like to see everything piled up in a small area (I can understand that on small screen there’s no choice, but Qt offers everything to provide responsive interfaces)
    Here’s an example of how it looks at home: scrollbars and small areas…
  • The provided brushes
    – Brushes pack from @RamonM, @Deevad and @Rakurri covers 99% of my needs :slight_smile:

So can’t tell if a nodes based interface is better or not.

Maybe advanced users with brush engine interface like @RamonM, @Deevad or @Rakurri, who produce and provide us good quality packs (sorry for other, I don’t have name of all of you :innocent: :kissing_heart:) can give us a better opinion about this because they’ll probably be the first impacted by this if it’s on day implemented…

Grum999

2 Likes

Thanks for your input! I agree that what I consider “easier” is very much informed from my background. I by no means wanted to come across with “this is my Objective Design Take on how I think this should work.” I definitely appreciate the time and thought put into these counterarguments.

I certainly see the value in maintaining parity with industry, I’m not arguing for “everyone agree with my approach or else”, just “this has been knocking around in my head for months and I want to know what other people think about it too”.

I’m still not entirely turned off by these points though? Specifically I don’t think that the brush editor as-is needs to be removed, and I probably should have been more clear on that. I think Blender’s approach of “a panel of dials and sliders and checkboxes that can then be brought into a node-based setting” is something that could be borrowed from here. This has the easy pick-up of the current system but also has room for extensibility.

I don’t want to downplay the amount of infrastructural work that went into that, nor feel entitled to that from Krita devs.

1 Like

Hi! This is a really argumentative reply, and didn’t add much to the conversation.

1 Like

Thanks for your reply! I definitely think that a node-based system should encompass the system that exists now, rather than remove it entirely. As I mentioned in another reply:

I think Blender’s approach of “a panel of dials and sliders and checkboxes that can then be brought into a node-based setting” is something that could be borrowed from here. This has the easy pick-up of the current system but also has room for extensibility.

I think there’s design space in existing applications that allow for a brush editor as it exists right now to exist, but also be expanded into a more complex system for when you want more complex behaviour.

1 Like

Hey there, thanks for looking into my points.

Ah, i see then i misunderstood you completely, i thought this was proposal to completely remove the current system and add this new node base. Reason why I strongly pointed out how confusing that could be.
The current one has many problems as Grum999 stated and it needs a revamp, especially since its a floating window and that is already causing problems for some people as they can’t resize it to fit their screens and as stated before is far from being friendly. Honestly making it into a docker would probably look better.

I am not sure if I understand what you mean by this. if you could clarify what you mean I would appreciate it.

but this brings into consideration one thing:

  • how to integrate the 2 systems, cause from what i am understanding one would be able to edit the same brush through 2 different interfaces,while one could argue its a different interface that would do the exact same thing I learned to never trust that things will behave the same when coding when you change the interface so much. One might run into problems of having to adapt the code so the interface works the way you want. So this would need to be looked into.

Though this was based on my understanding of what you wrote, since I am a bit confused on part of your answer i might have misunderstood. I am also not familiar with the krita code itself so my concern may be unfunded but still something that I think its good to point out.

2 Likes

@Grum999 I am sorry for my straight forwardness but is kinda how the melting pot goes.

I have changed from engineering school into animation+design and ended up helping teachers teach their students and later I came across people professionals in the field as much in animation as in high profile publicity and strangely enough things don’t change as much as you might think.

I only taught for about 2 years as a “senpai” not faculty but teachers would redirect students to me during their classes. At a certain point you just need to do some simple statistics to see what i mean.

Sorry for my rant I will stop the off-topic.

1 Like

I was referring too

a suggestion to correct a real possible issue and make things easier for new users.

P.S.

1 Like

I think a lot of your points still stand, and I appreciated them nonetheless. I don’t think you misunderstood, I think I should have been clearer.

The main design point that would make this palatable rather than a total paradigm shift is definitely finding ways to make something like the current system be usable but expandable into something like the node-based system that I think would be useful to figure out.

I guess what I mean with blender is that while you can edit a material within a node editor i.e.:

You can also edit a basic, single-node material in-panel:

The comparison isn’t 1-to-1, as what I’m proposing would split up the current editor into a node editor system (again, I don’t think my proposal here is perfect, I just wanted to get it out of my head) but it might be a starting point for an approach?

I think that as long as the way the brushes are interpreted is agnostic of a knobs-and-dials-and-checkboxes interface like we have right now and a node-based one (that is, both of them throw the same format at the engine, or there’s a programmatic way of transforming one format into another) maintaining an interface like we have currently shouldn’t be contradictory with maintaining a node-based one. I might be talking nonsense though.

3 Likes

I missed this reply earlier, but I’d be really interested in any of the design ideas you had for this kind of thing / anything that diverges from what’s been discussed / quirks about the brush engine that might get in the way.

1 Like

Thanks for your clarification, I think I finally understood what you mean. Having the possibility of using sliders and the node system could be interesting.

Indeed that is correct, but there are other problems that might appear in keeping 2 ways to input the same data. For exemple the node system requiring a variable type and the slider view needing another for another this might cause problems. Now I dont know if this would apply to krita as again i dont know the code nor know how Qt handles the interfaces, this is just from my experience as a programmer.

I can think of some things to consider for this. I am not good in designing interfaces, but i can point out some things that need to be considered when changing it.

Firstly lets consider the current system how brushes are built:

  • Brush attributes, brush tip, size, opacity, rotation and so on.
  • Individual parameters (pressure,tilt,drawing angle, speed) that will control the brush attributes, currently we have a list where we can chose more than one and each parameter has a curve that will control it’s behavior. We are also able to have the same curve for all of them or have different curves for each parameter. and we can also change how the curves will be calculated
  • Strength slider that will tell how much that curve will control how strong the curve will affect the brush

Right now all these work together and the interface reflects that.

now lets think of a slider interface, we have the size parameter, what the slider would convey? the strength? the min size? how we would add the curve aspect of it? only in the node editor? and what about how to set the control params on the slider interface? the way krita is built right now what controls the brushes mostly are the curves so this should be taken into account, when decided the interface.

Now for the node editor part, on your proposed mockups you take a very modular approach to them but considering the current way brushes work, that wouldnt be really necessary, as i think almost all parameters would have a curve node, and maybe a mix node to add more params to control an attribute. So i think it would streamline to just have a param having a curve in it.

constant values are interesting but don’t know if they would fit the current brush engine. they might work for things like the airbrush attribute that is just a slider but dont know about other use cases.

another thing worth remembering is the painting mode attribute that is just a radial option how would that fit in a node like environment, or would be a part of the slider interface? There is also the scatter attribute that has checkboxes for x and y would we make a connection to another node specifically for checking x and y?

we also need to be careful to not fragment brush creation, what i mean is “oh this setting is only available on the slider interface, and this one i need to open the node editor.” this will be kinda unavoidable i think, but keeping it into a minimum would be good.

I feel like your initial ideia is fine but, there are things that need a bit more refinement, making everything too modular might overcomplicate things, analyzing how big the code change would be is also important. The slider interface might need some thought on it to adapt to the current scenario and seeing what is feasible with the qt interface, unless we are changing how all the brush engines work, which i think is highly unrealistic and would break all brushes…probably.

well i guess these are all the points i think might cause problems, again i am not familiar to krita code so dont know if i am talking nonsense but hopefully something here might help.

It looks like the brush engine of BlackInk. You could download it and have a try.

2 Likes

The idea of the OP is awesome, and because modular, much easier to handle for users than the current implementation,

Especially newcomers may be scared as hell to discover the verbose abundance of options in the brush interface the first time they open that window.

2 Likes

Freaking awesome stuff.
First time to hear about

3 Likes

I really like the node base system in Blender, I also agree that it would make reading brushes a bit easier because you only see what the brush currently does. Most of the default brushes would only have one node or maybe two. Even my own more complex brushes.

The main difference however is for what nodes are used for in blender which is a very modular and highly customizable back-end. The graphs can get really complex. You can put an image as an input, then put like a low pass filter on some channels put that output into another filter, then add a randomize or whatever and convert it in a height map, then mix it with the input image again. There are really a lot of different node types most of them can be applied several times.

Krita’s brush engine does not really have this flexibility. Currently you can’t put a randomizer in front of a hue mixer and then use it’s output to do something else somewhere else. When I’m not mistaken and from what I remember from a short glance into the code, every brush setting is applied in specific order one after another when the brush is used. The brush engine as it is currently is just does not justifies a node based system. As much as I’d love a node view, I think it would be a better approach to just tidy up the current UI a bit. I for example would appreciate, if I just could hide all the preset options currently not active and add them from a list if needed.

PS: Black Ink looks interesting but from what I see it only uses nodes for layer effects, filters and blending modes (this would also be cool to have in Krita) but not for brushes.

4 Likes

Thanks for bringing this up.

Blackink’s system looks interesting, but that also looks like a node-based system for layers and post-processing and transforms, which is still very cool but different from the brush design system being discussed.

I think the only way you can make a node-based brush editor is by creating a new brush engine. Takiro is right that making a node-based brush editor that creates presets as they are currently defined for the existing brush engines isn’t going to work. The system doesn’t have the flexibility for that.

I think it was Sven Langkamp who started on a node-based brush engine years ago, but that never got very far. Adam Celarek actually made a scriptable brush engine.

4 Likes