code.ivysaur.me

Cardboard Sikuli

IDE and scripting platform to automate and control desktop applications, for GUI application testing or robotic process automation (RPA).

This is a clean-room implementation of the Sikuli application based on RobotGo, Goja, and MIQT.

  • OpenCV image matching
  • Desktop IDE with screen grabber based on Qt 6
  • It uses Javascript as the scripting language instead of Python 2

Compiling

Tested on Debian 12

apt install libopencv-dev qt6-base-dev libx11-dev xorg-dev libxtst-dev build-essential golang-go
go build -ldflags '-s -w'

Container and CI usage

Running the IDE in a container helps isolate your scripts and images from any OS changes.

# Build cardboard-sikuli and the runtime environment
# The resulting Docker image is about 1.1 GiB.
docker build -t cardboard-sikuili:latest -f cardboard-sikuli.Dockerfile .

# Run interactively (IDE)
docker run --rm cardboard-sikuili:latest
# ""Cardboard-sikuli is now running - connect to VNC on port :5901""
# Either use -p to map the IDE VNC port, or, find the container's IP from `docker inspect`

# Run a test (one shot)
docker run -v /path/to/my/script:/src cardboard-sikuili:latest --run /src/my_file.Sikuli

See also

Scripting reference

  • SIKULI_WAIT - Global variable to set the default refresh rate between operations (milliseconds). Default = 200
  • SIKULI_TIMEOUT - Global variable to set the default time to wait for an image to appear (milliseconds). Default = 3000
  • SIKULI_THRESHOLD - Global variable to set the image match accuracy threshold (floating point from 0 to 1). Default = 0.9
  • ClickAt(filepath, offset_x, offset_y) - Wait for image to appear, then move the mouse and click it at the specified offset. Throws a Javascript error if no match is found.