This is still work-in-progress (so no new version yet), but i worked on it all night and i want to share :-P.
I added a command to do triplanar projection of whatever image is in the clipboard to the model:
This is mainly useful for easily blendable “detail” materials, so -e.g.- trying to paste something with obvious features (e.g. bricks) would make the blending very obvious (though with some adjustment for the blending smoothness it can be used for some stuff).
I still need to to a GUI for specifying the blending smoothness and image scale and optimize it a bit (Python is very slow for that kind of stuff).
I uploaded a new version (view3d20240229), here is a changelog:
Add a command to paste an image from the clipboard on the 3D model using triplanar blending (as shown in the video i linked previously but now it shows a dialog box with options[0]
Add orthographic projection mode
Allow the use of the current selection as an alpha mask for all commands that create new layers
Hide menu options that affect the camera in newly created image to avoid glitches from wrong projections
Bugfixes:
Messageboxes appearing behind main window in fullscreen mode
Cancelling background color selection dialog now works instead of making it black
[0] The dialog box:
The “hardness” specify how soft/hard the transition will be between the three major axes, the “pixels per unit” specify how many pixels/texels are in a unit in object space (e.g. if you use meters for your 3D models, that’d be how many pixels a meter is) with a default value based on the average ratio between edge lengths in 3D space and texture space, “moat around geometry” is how many pixels around the geometry in texture space (the texture coordinate triangles) to approximate for avoiding seams (4 should be fine for most uses but you may want more if you are editing a texture at a higher resolution than the target one), “per direction blend” allows partial (or no) mixing of the X/Y/Z planes and finally “ignore negative X” can be used to only take into account the positive X axis (useful for when a 3D model has overlapping texture coordinates from mirroring - without it, you’d get artifacts).
EDIT: removed the wrongly copy/pasted entry about the view alignment command, that was added in the previous version
Here is yet another version (view3d20240302), though a somewhat minor one since i only added a feature to create a gradient from a reference point in 3D space:
This can be used for various reasons, like - when combined with Krita’s “levels” adjustment filter - to create masks that isolate areas in 3D space, as shown in the video.
Also fixed a couple of bugs. Here is the changelog:
Add a command to generate a gradient based on the distance from a reference point (camera, top, front, etc) to each texel/pixel in 3D
Fix the moat value in triplanar paste being ignored
Fix potential Qt memory leak for the triplanar paste dialog box
Hi, i finally have time so test this, the view3D appeared black and did not display anything in direct3d so i guess this plugin is functional in opengl only ?
Anyway. ive changed the display mode to opengl and the blackground color seems to be transparent cause it shows what windows/ browsers i have behind the krita windows - is this a bug ?
But despite being written in a compiled language, it was still somewhat slow as it did raytracing against the high poly geometry. With Python it’d be so slow it may even take more than an hour to finish.
And TBH i do not see how this will be helpful for texture painting in Krita as the high poly and the low poly models will need to be done in another program anyway (Blender or whatever) and what you’d end up with (the normalmap) wouldn’t really be that editable in Krita. It feels like 99% of the work here would be done outside of Krita anyway, so might as well use a dedicated program for that task.
I do plan on adding some shader support at some point, but it’d be about stuff you can paint in Krita, so the closest would be painting a bump map and having the plugin generate a normal map out of it.
Generating AO, cavities, etc is something that can be useful for texture painting, so it is something i want to look into at some point - sadly it also needs raytracing against the geometry, meaning that it’d be impractically slow in Python.
There are two potential approaches to avoid Python’s performance issues, one being to try and somehow use OpenCL or OpenGL compute and have the GPU compute things (sadly without any form of hardware accelerated raytracing, that’d be too useful, but no vendor has exposed any OpenGL extension for raytracing - then again, i don’t have a GPU with HW raytracing either :-P), the other being to try and reimplement all compute things in native code (C or whatever) that is loaded via the Python script as a dynamic library. This would also allow using Vulkan (and by extension, hardware raytracing) at some point.
The problem is that both approaches pretty much require rewriting almost the entire plugin - and i’d already need to rewrite half of it to support shaders :-P. The native library approach would allow for the most flexibility, but it’d also be the hardest approach - and i’d also only be able to make builds for Linux and Windows (no macOS).
Another thing is that i remember reading some comments that the Krita developers do want to add a 3D texture painting mode in Krita (which TBH would also bypass a lot of the limitations that currently exist with the Python plugin) at some point, so i’m not sure how much effort is worth putting into this plugin.
I understand the dilemma but this is probably the most important plugin for Krita going forward. I think it serves already as a huge proof of concept already. It does not need to replace substance and generate things in 3d but just using the brush on the surface is already a huge huge win.
If i could make brush strokes programmatically it’d make Alt+Click even more useful, though i’d settle for having some way to do undo/redo from scripts :-P.
I did try to build Krita from source code (thinking i might contribute some functionality for scripting) but while i managed to build it, it always crashed at startup and couldn’t figure out if it was an actual bug or some misconfiguration. I didn’t tried much though, i might redownload and try again in the future.
I’m only using Blender for modelling and a rather old version at that (2.79) as i didn’t like the UI changes (not for any major reason, i just find the old UI more comfortable) and some things take a bit more time to do (e.g. setting up a texture on a model - again nothing major but it is a couple more steps regardless). Blender’s scripting API is also not very stable, there seem to be changes even between minor releases.
I’d also like the plugin to be usable regardless of what 3D modelling program one uses (e.g. IMO Wings3D is easier to use for making some things), the only reason i mention Blender is because the JTF format the plugin loads only has a Blender exporter, though it is possible for someone using another program to use Blender as a conversion tool.
What Python limitations are you worried about now? Are you observing any bottlenecks?
I think if the community sees this plugin as important, it may make sense to make some changes in Krita C++ side to better support such plugins. Another that comes to mind is the Blender Layer.
AFAICT from writing this plugin there are two main limitations:
Performance: even looping through each pixel in an image is slow in Python. There are some ways to improve performance (though note below) but at the end of the day, Python isn’t really made to write fast software in but instead to glue together stuff made in native code. As an example the triplanar blending command that in View3D takes a couple of seconds in Python would be almost instant in C.
Krita’s scripting API: there are various things that would improve functionality, like some way to tell the Krita “i’m going to be modifying a node now” and “i’ve finished modifying the node” so it can make an undo/redo step, being able to programmatically tell “start a brush stroke at X,Y on that node”, “move the brush stroke at X,Y” and “finish the brush stroke at X,Y”, etc. It can also be useful for performance, e.g. BIG benefit, not only for View3D but for other plugins as well, would be some signal to be attached to nodes/documents/projection when they are modified and the area that has been modified to avoid having to repeatedly generate a preview image and process the entirety of it (e.g. editing 2048x2048 or 4096x4096 textures in my PC is impractical because of the Krita projection → QImage → OpenGL texture conversion that needs to happen every “frame”, but if i could have a signal for whenever the document/node/projection was modified - including brush stroke in progress - with the affected area, i could just update that specific area of the texture and only when that change was made instead of all the time)
At least as far as View3D is concerned, for the Python bits i could make a dynamically loaded library that does the heavy lifting and only use Python for interfacing with Krita and Qt, but even that would still be limited by what Krita exposes to Python scripts.
Thanks, this is very useful. I’m not sure when, but I will try to look into this. Python could still be fast if it’s limited to defining work, rather than executing work. For example: “here’s some image memory, now copy it into Krita”, that sort of thing.
But yeah, it makes no sense to me to do heroics in Python if Krita is open source and new capabilities could be added to improve the integration of more advanced plugins.
Ideally new functionality will be part of Krita, but doing things in Python still helps with being able to do things “now” with Krita as-is, especially when sharing things with others. For example View3D can be used right now by anyone in this thread with their existing Krita installation regardless of where they got it from, but if Krita had no plugin support and you could only add things by modifying the source code, it’d need people to wait until the feature was merged in the source code[0] and new builds be made - for Linux users it may also mean waiting until their distro’s repository maintainers upgrade the relevant packages too.
Even for stuff that may make more sense to be in Krita, Python can still help as a way to quickly prototype things and be checked by users who aren’t programmers before being reimplemented in C++ as part of Krita’s codebase - but this prototyping is still limited by what Krita actually exposed via Python.
[0] assuming it was accepted, some things might be considered out of scope for the project
Right, to be clear, I wanted that message to be positive. Meaning, in a hypothetical future version of Krita, the plugin could perform even better if we add these new integration possibilities to the scripting API.
Having the current slower path as fallback is very good too.
On my side, I started few day ago to implement new methods & signals in API
Having signal when active node is changed is in my list, but when I analyzed C++ source classes, I identified hundreds of useful methods that can be added to API: it will took me time to implement everything.
It probably won’t be ready before 5.4 or 5.5
For heavy math computations, python is not the best language and C/C++ (using SIMd if possible) if highly recommended
The thing is, C++ is hard in comparison to python
On my side it took me more time to code the same stuff
Also, a python plugin you can publish your progress as your on rythm and implement it as you want; coding in Krita directly as some advantages (function available for everyone) but also some disadvantage (you’re dependant of MR being accepted or, it can takes time to get MR accepted, there’s delay between releases and then you can’t provide new function/bug fix quickly)
If you guys need to resolve a problem or want to discuss some ideas, I’ll be happy to help. Well, at least within my ability, a lot of Krita code is still a mystery to me.
So, just out of curiosity, i hacked together quickly a C library that i load from Python using the ctypes.cdll module. All the library does is load a JTF mesh and render it instead of the Python code, but it was just to see how it could work. Turned out to not be as hard as i thought and i wrote a script to build both a Linux and a Windows version of the library (using mingw64 as a crosscompiler) and tested it with both my distribution (openSUSE)'s Krita (which is what i normally use) and the latest Windows Krita release running under Wine and worked fine in both cases.
So i think i’ll just try to convert the code from Python to C and see how it goes
(sorry macOS users )
EDIT: yeah, it is worth the effort, while a QImage from the document projection still needs to be generated, i can pass the QImage data directly to the C library, thus avoiding all sorts of extra copies and making updates much faster. Editing 4096x4096 textures is still laggy, but 2048x2048 are as fast as 1024x1024 was previously (so basically a 4x speedup).
