How to "paste" content to a VNC console which does not support copy and paste



In the tutorial, we are going to build a python GUI that automatically types.

My initial use case is for server setup with noVNC/QEMU terminal where copy paste is not available. Check out the YouTube demo here.

How it works

With the Auto Switch function on. You type something into the bottom text box and click the send button.

  • It visually locates the VNC browser windows coordinate by the image feature provided
  • It clicks left mouse button in the middle of the image feature, this switches to the VNC window
  • It types your text into the VNC window.
  • It changes the mouse cursor back to where it used to be


Install dependencies

The App depend on two libraries

pyautogui for simulating keyboard and mouse as well as visually locate windows on the screen

wx is the Python front user interface library.

We install them with Python 3 pip3 command.

pip3 install pyautogui wx

Get the code

Clone or download the source code from my GitHub.

Run the GUI

Once you have the dependencies installed and source code downloaded. Open a terminal window in the TypeApp folder and type.


The first thing we need is an image feature of the VNC windows we are going to type in. Let's capture a screenshot of it and save it as a png file. The tool will later locate this image on your screen and do a single mouse click to switch to this window for you.

Here is an example screenshot of my browser VNC window.


Captured from this window

vnc window

In the App you can choose your image by clicking the Browse button, also be sure to check the Auto Switch check box.


Before we are trying to type something into the "Input" text box, be sure the VNC browser window is visible on the main screen (if you are using multiple monitors).

Highlights of the source code

  • The wx event type and "typing" thread
  • pyautogui functions, where the magic happens

The wx event type and "typing" thread

In order to keep our GUI responsive, we should avoid running any computation intensive or any action take a quite a while in the main GUI thread.

In our case the action takes a while to complete is the typing action. We are going to execute it in another thread parallel to the GUI thread. But how does the second typing thread notify the GUI thread when the typing is done or failed? Here is where the wx event type comes into play.

First, we create an instance of the NewEventType, it will be used to create an event object.

Custom event class TypingEvent will be used to send an event from the typing thread to the main GUI thread.

myEVT_TYPE = wx.NewEventType()
EVT_TYPE = wx.PyEventBinder(myEVT_TYPE, 1)

class TypingEvent(wx.PyCommandEvent):
    """Event to signal that typing is done"""
    def __init__(self, etype, eid, value=None):
        """Creates the event object"""
        wx.PyCommandEvent.__init__(self, etype, eid)
        self._value = value

    def get_value(self):
        """Returns the value from the event.
        @return: the value of this event

        return self._value

Define the typing thread where we will be running our typing code in. At the end of the run method, the thread will generate an event to tell the GUI the typing is finished.

class TypingThread(threading.Thread):
    def __init__(self, parent, value):
                @param parent: The gui object that should recieve the value
                @param value: value to 'type' to
        self._parent = parent
        self._value = value

    def run(self):
        """Overrides Don't call this directly its called internally
        when you call Thread.start().
        time.sleep(0.5)  # our simulated calculation time
        if isinstance(self._value, TypePara):
            cmd = self._value.cmd
            img = self._value.img
            # Typing code
            # . . . .
            evt = TypingEvent(myEVT_TYPE, -1, evt_value)
            wx.PostEvent(self._parent, evt)

Start the thread from the GUI thread when the "Send" click.

worker = TypingThread(self.main_view, TypePara(cmd, img_path))

In the GUI thread listening to the event by "Binding" a "listener" to it, the on_type_over will be called on GUI thread every time the TypingThread posts an event.

self.main_view.Bind(EVT_TYPE, self.on_type_over)

def on_type_over(self, evt):
    evt_value = evt.get_value()
    if evt_value:
        print("Done Typing \"{}\"".format(evt_value))
        pyautogui.alert('Please move the window \"feature\" to current screen and try again.')

pyautogui functions, where the magic happens

pyautogui.position read the current mouse position on the screen.

pyautogui.locateCenterOnScreen takes our screenshot image as an argument and return the coordinate of the image on the screen as its name suggest, click the left mouse button on the coordinate found in previous step

pyautogui.typewrite types the text

pyautogui.moveTo moves the mouse back to it is used to be.

done_typing = False
if img is not None and os.path.exists(img):
    last_coordinate = pyautogui.position()
    coordinate = pyautogui.locateCenterOnScreen(img)
    if coordinate is not None:
        x, y = coordinate, y=y, clicks=1, button='left')
        pyautogui.typewrite(cmd, interval=0.01)
        done_typing = True
        pyautogui.moveTo(last_coordinate[0], last_coordinate[1], duration=.1)

Current rating: 1