Clicking with scripts

I am a beginner in programming.
Is it possible to create a script that clicks on a specific position in the Krita canvas (for example, the current cursor position)? I would like to automate the process of switching to the continuous selection tool, clicking on an arbitrary position, extending the area, and so on. However, I could not find a function to do the clicking.

Translated at DeepL

It’s not possible to emit arbitrary click events from within Krita and while it is possible to create selections with Krita’s python scripting API manually (als growing and shrinking) I couldn’t find a way to actually use a specific selection algorithm like the Contiguous Selection Tool. So basically you can tell Krita which pixels to select but not “use this selection tool at these coordinates for me” and let Krita handle the creation of the selection. I’m not an expert on the Krita API however, and the documentation is in complete or lacking, perhaps a user with more Krita scripting experience is of better help or at least can confirm whether it’s possible or not.

Outside of Krita however there are ways to automate some tasks. Tools like AutoHotkey (Windows) or xdotool (Linux) can let you send Mouse and Keyboard events to pretty much any application. However it could proof difficult to do what you want with them, depending on the complexity of the task.

Hackish way to generate QTableEvents.

Create new document and run script in Scripter.
Note: pan, rotate and zoom are ignored.

from random import random
from krita import Krita

from PyQt5.QtCore import (
        Qt,
        QPoint,
        QPointF,
        QEvent)

from PyQt5.QtGui import (
        QTabletEvent,)

from PyQt5.QtWidgets import (
        QWidget,
        QApplication)


def find_current_canvas():
    app = Krita.instance()
    q_window = app.activeWindow().qwindow()
    q_stacked_widget = q_window.centralWidget()
    q_mdi_area = q_stacked_widget.currentWidget()
    q_mdi_sub_window = q_mdi_area.currentSubWindow()
    view = q_mdi_sub_window.widget()
    for c in view.children():
        if c.metaObject().className() == 'KisCanvasController':
            # first QWidget child of viewport should be canvas...
            viewport = c.viewport()
            canvas = viewport.findChild(QWidget)  
            return canvas


def click_canvas(pos):
    """
    https://doc.qt.io/qt-5/qtabletevent.html
    """
    canvas = find_current_canvas()
    global_pos = QPointF(canvas.mapToGlobal(pos.toPoint()))  # precision is lost ???
    device = QTabletEvent.Stylus
    pointer_type = QTabletEvent.Pen
    pressure = random()
    x_tilt = 0
    y_tilt = 0
    tangential_pressure = 0.0
    rotation = 0.0
    z_pos = 0
    key_state = Qt.NoModifier
    unique_id = 1234  # ???
    button = Qt.LeftButton
    buttons = Qt.LeftButton

    table_press = QTabletEvent(
            QEvent.TabletPress,
            pos,
            global_pos,
            device,
            pointer_type,
            pressure,
            x_tilt,
            y_tilt,
            tangential_pressure,
            rotation,
            z_pos,
            key_state,
            unique_id,
            button,
            buttons)

    table_release = QTabletEvent(
            QEvent.TabletRelease,
            pos,
            global_pos,
            device,
            pointer_type,
            pressure,
            x_tilt,
            y_tilt,
            tangential_pressure,
            rotation,
            z_pos,
            key_state,
            unique_id,
            button,
            buttons)

    canvas.activateWindow()
    QApplication.sendEvent(canvas, table_press)
    QApplication.sendEvent(canvas, table_release)


# plot some random points...
for i in range(200):
    pos = QPointF(random() * 1000.0, random() * 1000.0)
    click_canvas(pos)

/AkiR

1 Like

I would like to thank the people who replied to my question.

Thanks, I can use PyAutoGui a bit, so I figured that would be easier for me. finding functions from KritaAPI or PyQT is difficult for me.

I was surprised to see so many dots pop up! The code is much more complex than I imagined, but I am very happy to see that it is clickable.
I’m sorry if this is a rudimentary question, but if I want to simply perform the click at the cursor position instead of randomly, how do I write it?

pos = QPointF(???)
click_canvas(pos)

I’ve been tinkering with this for a little while, but it’s not working.

QCursor can be used to query current postion of cursor.
(weird script… user can not press mouse / tablet button, but is expected to move to correct postion?)

from random import random
from krita import Krita

from PyQt5.QtCore import (
        Qt,
        QPoint,
        QPointF,
        QTimer,
        QEvent)

from PyQt5.QtGui import (
        QTabletEvent,
        QCursor)

from PyQt5.QtWidgets import (
        QWidget,
        QApplication)


def find_current_canvas():
    app = Krita.instance()
    q_window = app.activeWindow().qwindow()
    q_stacked_widget = q_window.centralWidget()
    q_mdi_area = q_stacked_widget.currentWidget()
    q_mdi_sub_window = q_mdi_area.currentSubWindow()
    view = q_mdi_sub_window.widget()
    for c in view.children():
        if c.metaObject().className() == 'KisCanvasController':
            # first QWidget child of viewport should be canvas...
            viewport = c.viewport()
            canvas = viewport.findChild(QWidget)
            return canvas


def click_canvas(global_pos=None):
    """
    https://doc.qt.io/qt-5/qtabletevent.html
    """
    canvas = find_current_canvas()
    if global_pos is None:
        global_pos = QCursor.pos()
        global_posF = QPointF(global_pos)
        posF = QPointF(canvas.mapFromGlobal(global_pos))
    else:
        global_pos = QPoint(global_pos)
        global_posF = QPointF(global_pos)
        posF = QPointF(canvas.mapFromGlobal(global_pos))
    device = QTabletEvent.Stylus
    pointer_type = QTabletEvent.Pen
    pressure = random()
    x_tilt = 0
    y_tilt = 0
    tangential_pressure = 0.0
    rotation = 0.0
    z_pos = 0
    key_state = Qt.NoModifier
    unique_id = 1234  # ???
    button = Qt.LeftButton
    buttons = Qt.LeftButton

    table_press = QTabletEvent(
            QEvent.TabletPress,
            posF,
            global_posF,
            device,
            pointer_type,
            pressure,
            x_tilt,
            y_tilt,
            tangential_pressure,
            rotation,
            z_pos,
            key_state,
            unique_id,
            button,
            buttons)

    table_release = QTabletEvent(
            QEvent.TabletRelease,
            posF,
            global_posF,
            device,
            pointer_type,
            pressure,
            x_tilt,
            y_tilt,
            tangential_pressure,
            rotation,
            z_pos,
            key_state,
            unique_id,
            button,
            buttons)

    if not canvas.isActiveWindow():
        canvas.activateWindow()
    QApplication.sendEvent(canvas, table_press)
    QApplication.sendEvent(canvas, table_release)


annoy_timer = QTimer()
annoy_timer.timeout.connect(click_canvas)
annoy_timer.start(500)  # about 500ms

QTimer.singleShot(10000, annoy_timer.stop)  # about 10s
2 Likes

I’m very grateful for your help, but I’m confused by the addition of unexpected elements like QTimer()… :sweat_smile: I simply want to incorporate a single click on the cursor position into the script.

posCur = QCursor.pos()
print(posCur)

pos = QPointF(posCur)
click_canvas(pos)

As you pointed out, I could easily get the current position of the mouse. Thank you very much.
However, when I substitute this into click_canvas(pos) and execute it, the click is made at a point about 500 pixels off to the right. It seems to be caused by the misalignment between the display and the Krita window. Hmmm.

last version of code can be called without pos argument. (note: two versions of code have little bit different def click_canvas(global_pos=None) functions)

click_canvas()

it will use QCursor.pos() as pos if one is not given.
it then converts pos from global to local position at canvas.

QTimer was just used to demo that position is taken from cursor. (hard to press [play] button in script editor and at the same time be above the canvas :smiley: )

/AkiR

So it was that the cursor position was already built in.
I also learned a lot about QTimer().
Thank you very much for your time and patience. I think I can do what I want now.

1 Like

Akir still has a point. Why even try emmiting a click event at a specific position when the user is expected to move the mouse there manually anyway? User just can press the mouse button themselves while they have their hand on the mouse after moving it.

I guess because it sends the signal Qt to wake-up and update properly because many times you need a new signal to do its job and you can’t tell it to do it. I have alot of weird code to try and fix this because I can’t trick Qt to do what it should be doing.