View3D plugin to view/modify 3D mesh with the current image as texture

[badsector]

Hi all,

I was doing some texture painting recently and i was using an old painting program for that (PSP7). However going back and forth from the program and Blender to check out the texture was a PITA and decided to check if Krita (being newer and all) had some sort of “3d mode” in views so i could paint directly on the model, like in Blender but with layers, selections, copy/paste, etc - and sadly learned that this is not the case.

Supposedly this is something the devs (based on an older forum post i found) want to add at some point but in the meanwhile i decided to make a stopgap solution with a plugin written in Python that displays the currently edited document in an OpenGL viewport that you can rotate, pan and zoom so you can at least avoid the whole “export PNG, alt+tab to Blender, reload texture, notice the mistakes, alt+tab to paint app, do modifications while trying to imagine how they’d end up, export PNG, alt+tab to Blender, reload texture, notice your imagination is lacking, etc” cycle.

You can find the plugin here: Krita View3D Plugin

A very brief YouTube video with it in action: https://www.youtube.com/watch?v=PMsI8P2YCnU

The functionality the plugin provides is:

• The aforementioned model viewing

• Draw “blobs” by alt+clicking on the model (it isn’t exactly painting on the model itself but can help find where things are in an image)

• Create a new image with the current render

• Backproject a layer from the previously created image to the current image in a new layer (can help with doing some indirect painting on the model)

Some limitations are that the whole backprojection is slow (everything is done in Python) and not very accurate, especially if you zoom out the model (you’ll get gaps between pixels in the texture). Also no undo for blob painting since Krita does not provide a way to do that (make a new layer to draw blobs in to avoid damaging the existing image).

Note that the plugin is very rough as i primarily made it for my own use but thought about sharing it. I only tested it on my own PC running Linux and it may or may not work with other systems (i think it may not work with Intel GPUs and probably also macOS).

[Michelist]

Hello @badsector, and welcome to the forum!

Thank you for making this plugin!

I haven’t tested it yet, but it sounds like it has at least a few similarities to this one, but the mesh thing does not sound as they were completely similar:

Plugin: Blender Layer - Live 3d View in Krita

Blender Layer is a plugin that allows you to stream 3D View from Blender into Krita, behaving like a normal layer: You can trace over it, change the blending mode and even apply layer styles! [BlenderLayer] [Layer] Direct zip link: https://github.com/Yuntokon/BlenderLayer/archive/refs/heads/main.zip Limitations Poor performance for high resolutions and complicated scenes Blender must be open in the background Transparent background requires Blender 3.6.0 (Still in alpha at time of writing) …

But it is always better to have choices.

Michelist

[EyeOdin]

Thank you so much for sharing it.

I do need to test it properly as I need a model in order to look with eyes to see.

But texturing tools like this is a real god bless.

[badsector]

@Michelist

I haven’t tested it yet, but it sounds like it has at least a few similarities to this one, but the mesh thing does not sound as they were completely similar:

I saw that and there does seem to be a bit of functionality overlap but AFAICT Blender Layer’s main purpose is to use a live Blender viewport as a reference for drawing on it whereas View3D’s main purpose is to show the current document (layers and all) on a mesh as a texture for making textures.

[badsector]

Here is quick example for using this plugin to fix a seam caused by trying to paint across surfaces that have discontinuities in the texture:

[EyeOdin]

Even fixing seams? That is literally amazing.

[badsector]

Well, fixing for the most part :-P. Sadly the way the backprojection is done isn’t very accurate so sometimes you may still need to do some editing/cloning by hand but it’ll only take a few seconds or so (alt+clicking on the model helps to find the exact place you need to retouch).

[EyeOdin]

I was not even using it yet but I got this

AttributeError
Python 3.10.7: C:\Program Files\Krita (x64)\bin\krita.exe
Sat Nov  4 01:09:01 2023

A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.

 C:\Users\EyeOd\AppData\Roaming\krita\pykrita\view3d\__init__.py in initializeGL(self=<view3d.View3DViewport object>)
  300         self._tprog.link()
  301         self._updateTPData = True
  302         gl.glClearColor(0.1, 0.2, 0.3, 1.0)
  303 
  304     def cleanupGL(self):
gl = None
gl.glClearColor undefined
AttributeError: 'NoneType' object has no attribute 'glClearColor'
    __cause__ = None
    __class__ = <class 'AttributeError'>
    __context__ = None
    __delattr__ = <method-wrapper '__delattr__' of AttributeError object>
    __dict__ = {}
    __dir__ = <built-in method __dir__ of AttributeError object>
    __doc__ = 'Attribute not found.'
    __eq__ = <method-wrapper '__eq__' of AttributeError object>
    __format__ = <built-in method __format__ of AttributeError object>
    __ge__ = <method-wrapper '__ge__' of AttributeError object>
    __getattribute__ = <method-wrapper '__getattribute__' of AttributeError object>
    __gt__ = <method-wrapper '__gt__' of AttributeError object>
    __hash__ = <method-wrapper '__hash__' of AttributeError object>
    __init__ = <method-wrapper '__init__' of AttributeError object>
    __init_subclass__ = <built-in method __init_subclass__ of type object>
    __le__ = <method-wrapper '__le__' of AttributeError object>
    __lt__ = <method-wrapper '__lt__' of AttributeError object>
    __ne__ = <method-wrapper '__ne__' of AttributeError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of AttributeError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of AttributeError object>
    __repr__ = <method-wrapper '__repr__' of AttributeError object>
    __setattr__ = <method-wrapper '__setattr__' of AttributeError object>
    __setstate__ = <built-in method __setstate__ of AttributeError object>
    __sizeof__ = <built-in method __sizeof__ of AttributeError object>
    __str__ = <method-wrapper '__str__' of AttributeError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __suppress_context__ = False
    __traceback__ = <traceback object>
    args = ("'NoneType' object has no attribute 'glClearColor'",)
    name = 'glClearColor'
    obj = None
    with_traceback = <built-in method with_traceback of AttributeError object>

The above is a description of an error in a Python program.  Here is
the original traceback:

Traceback (most recent call last):
  File "C:\Users\EyeOd\AppData\Roaming\krita\pykrita\view3d\__init__.py", line 302, in initializeGL
    gl.glClearColor(0.1, 0.2, 0.3, 1.0)
AttributeError: 'NoneType' object has no attribute 'glClearColor'

[badsector]

Seems that context.versionFunctions(profile) returns None. It is possible either the version of Qt your Krita comes with or the Windows drivers for your GPU do not support using OpenGL 2.1 with QOpenGLWidget. What GPU do you have?

I’ll try to reboot to Windows later and download the Windows version of Krita to check if it works there.

[badsector]

I tried Krita and the plugin under Windows and got the same error. The reason is that Krita uses Direct3D 11 / ANGLE out of the box on Windows which is probably not compatible with OpenGL 2.1. However if you switch to OpenGL (via Settings → Configure Krita → Display → Preferred Renderer) it should work:

(note that i moved the dockers around a bit in the image, by default it will appear somewhere at the bottom right corner and will barely be visible)

I’m not sure what drawbacks there might be by using OpenGL instead of ANGLE on Windows. AFAIK ANGLE is an OpenGL implementation on top of Direct3D that was made because of bad OpenGL native drivers, mainly on Intel-based iGPUs and you shouldn’t have any issues with Nvidia or AMD GPUs.

[badsector]

I just tried to run Krita and the plugin crashed it, seems some change in Krita, PyQt or Qt caused the OpenGL profile to not be set with the same parameters as previously. I mistakenly thought that setting the profile parameters in initializeGL would work but apparently that wasn’t the case.

I uploaded a new version that should setup the profile properly now. It shouldn’t affect the Windows version of Krita (probably, i didn’t test it) but if you get crash or errors try to use the new version. I haven’t made any other change so if the plugin works for you, you can keep using the previous version.

[eks]

jtf file? Why not obj, ply or fbx which is more common?

[badsector]

JTF is both incredibly trivial to parse and, being a binary format, very fast to load. The loading code is just 17 lines of Python without needing any 3D utilities code.

OBJ, PLY and especially FBX are way more complicated formats and chances are you’d need at least some utility 3D code to render them with OpenGL. To give you an idea, an OBJ loader i wrote once for the smallest of OBJ subsets (i.e. only triangles and ignoring objects, etc, just assuming you selected a single mesh and exported it in Blender) was more than 200 lines of code and using various utility functions i already had (which i have not available in Python or in Krita and i’d need to rewrite). The JTF format is so simple that it took me less code than that to write an exporter for Blender.

At the end i just wanted to put some triangles with texture coordinates in a viewport and so i went with the simplest solution to do that

[polydogon]

Very cool plugin! Consider also this plugin which automatically reloads the working texture file in blender at a set interval (or with hotkey)
I’m not sure if there is a way to use .kra with blender but the add on does work with .psd

[badsector]

Looks like my last bugfix broke Alt+Click and To Image/From Image due to accidentally enabling multisampling (i thought setting the samples to “1” would mean 1 sample per pixel but apparently i needed to use 0… who knew? :-P).

I uploaded a new version (view3d20240207.7z) that fixes this bug.

[badsector]

Uploaded a new version of the plugin. This reworks the UI a bit to replace the buttons below the viewport with a popup menu so that the docker can be resized to smaller sizes than the combined width of all buttons (and i can add more stuff without having a ton of buttons visible all the time).

Here are two YouTube videos showing some of the new stuff. The first is from yesterday when i did the initial UI change and added a couple of new features and the second is from today when i reworked the UI slightly and tried to fix the precision so that painting across seams works better (the first video has me manually fixing some seams which shouldn’t be needed now):

Video from yesterday with the new stuff:

More Krita View3D plugin features

View from today with the seam painting fix and final UI:

Improved painting across seams in Krita View3D plugin

(and yeah, one of these days i might even finish making the texture for that model :-P)

Overall the new version (view3d20240218.7z) has the following changes:

• Rework the UI to use a popup menu instead of buttons (avoids minimum docker size)
• Add wireframe and backface toggling
• Add background color selection
• Add generation of a mask from view
• Add generation of layers for the texture coordinate triangles (as a filled mask or wireframe)
• When making an “image from view” use the background color and make a layer group to paint on instead of a single layer (allows using multiple layers and layer effects)
• Backprojecting (bringing the painted layer back to a texture image from an image created with “image to view”) should be a bit more accurate across edges/seams
• Allow “locking” the texture to a specific document instead of the active one
• Remember up to 10 last opened meshes (saved in Krita settings file)

It should also fix some bug/error when reopening a Krita document with the plugin active.

Note that i haven’t tested this on Windows though as i didn’t really change much in the rendering side aside from adding wireframe and backface culling, it should work fine.

[quockhanhlk]

Thank you so much, this is an awesome plugin, painting texture with a 2d software’s brush engine is pure bliss, something i cannot find in any other texturing softwares out there so far (substance/ 3dcoat…).
Btw your krita theme looks great ! do you mind sharing it ?

[badsector]

@quockhanhlk

Btw your krita theme looks great ! do you mind sharing it ?

It is CDE-esque2, i think i got it from CDE-esque2 - KDE Store

[EyeOdin]

this is just amazing

[badsector]

@EyeOdin thanks

I uploaded a new version with a couple of minor features i thought of:

• Added a command to align the canvas to the mouse position so that the down direction in both the view and the canvas are the same in screen (helps when the textures coordinates are not aligned with the texture itself and, e.g., are rotated)
• Added a menu with snapshot preview commands to show how the texture would look downscaled at 50%, 25% and 12.5% (note that these stop the live updates) - this is mainly helpful for when making textures at a higher resolution (e.g. 2x or 4x) the “final” size and want to see how they’d look on the model without resizing the image (ideally this’d be done in realtime but since everything is done in Python it’d be very slow for any image larger than a few pixels - perhaps in the future i might rewrite the OpenGL parts so i can use OpenGL for resizing)
• Added an “Update” button to explicitly update the texture if live updates are enabled

A short video that shows these (mainly how the align command can be helpful):

Scaled snapshots and canvas alignment in the View3D Krita plugin