Use xvfb to automate X program

Question

I originally posted this question here, but upon second thoughts I decided here was a better place for this.

I sadly have to use this proprietary program that bundles a camera driver. I want to perform a specific action with it, which involves clicking a few buttons. And there is no CLI.

So, what is the easiest way to automate this? I though of using xvfb to open the program in a fake X environment, but then how would I go about performing input operations? It could be click pixel at (x, y), but it would be even nicer if I could reference buttons and menu items.

If there is a simpler way to do it, please tell! I'm not sure xvfb is the correct method. I don't even know the actual name of what I'm trying to accomplish, so searching is very hard...

xvfb + xdotool seems like good choice. Maybe for first/debugging purposes you can try some simple window manager, and for end solution xvfb or xvnc. xvnc will be also hidden, and advantage is that you can connect to it with some vnc viewer and check application status. — MetNP, Feb 02 '16 at 16:00
You may find inspiration in [Selenium](http://seleniumhq.org/) and [Watir](http://watir.com/). They're frameworks that run web browsers and drive them through faked user input, the same way you're planning to automate that camera application. I have no idea whether Selenium and Watir contain code or libraries that you could reuse. — Gilles 'SO- stop being evil', Feb 02 '16 at 23:18

score 8 · Accepted Answer · edited Jun 16 '16 at 01:20

You can use xdotool which can fake keyboard/mouse input, window management, and more.

See example.sh script for xcalc "4*9=", to view capture-result image:

#!/usr/bin/env bash

save="$DISPLAY"                          # save original X display number
export DISPLAY=:44                       # set random choosen display for xvfb
case "$1" in                             #         and x-programs called below
  start) Xvfb $DISPLAY &   ;;            # starting xvfb server on :44
   calc) xcalc &           ;;            # run x-calculator
    xdo) xdotool mousemove 1 1 click 1   # focus with click on x:y=1:1
         xdotool key 4 asterisk 9        # calc 4*9
         xdotool key equal ;;            #         =
   show) #capture root window ('36' result image) and display it to saved Xno
         xwd -root | xwud -display $save &                                   ;;
   stop) ps auxww | grep "Xvfb $DISPLAY" | awk '{print $2}' | xargs kill     ;;
    all) DISPLAY=$save; $0 start; $0 calc; sleep 1; $0 xdo; $0 show; $0 stop ;;
esac

Note: Individual options can be used for testing, or run all with ./example.sh all.

Any idea how to make the xvfb/xwd capture color? This always makes the image black-and-white, I don't know how to make it in color — Karel Bílek, Aug 26 '17 at 21:01

Use xvfb to automate X program

1 Answers1