Desktop Bluetooth Remote
Controlling the Desktop via S60 Bluetooth Phone
Recently, I bought a new cellular phone (NOKIA E51), which is equipped with Bluetooth. I also had an unused Bluetooth dongle at home, which I'd never had any use for - it had come as a gimmick when I bought my stationary computer. The availability of these gadgets, however, triggered my curiosity and I started wondering whether they could be coerced into doing something useful. This turned out to be easy. In my day job, I frequently give presentations with laptop and beamer, and therefore started to work out how to use the phone as a remote control for my computer.
The first idea was to turn the phone into a Bluetooth mouse, but the phone does not support the required HID Bluetooth profile, so I decided to program a classical server-client application instead. There are other projects that work in a similar way, e.g., the IrDA driver to convert infrared signals to commands, but, as far as I know, it works only with programs that can be remote-controlled. The server on the PC receives commands from the remote, and then itself sends commands via programs like xine-remote to the respective applications (here, the media player xine). I wanted to control the entire desktop, including mouse motion, mouse-click, and cursor or Enter keys. In this way, I could control any application and especially the Acrobat Reader, which does not sport a native remote control. This feature turned out to be the most tricky -- but stay tuned. Just to summarize briefly: I wanted an application on the phone (the client) that reads the phone keys and sends commands via Bluetooth to the PC, where a server is running that receives the messages and causes the mouse to move or click, or causes key-press events.
To keep things orderly, I will describe them in the sequence that the information moves - from your thumb pressing the phone key to the mouse-click on the PC - and start with the application on the phone.
On the Phone
My new cellular phone runs under the Symbian operating system (3rd
generation), which is rather common in modern smart-phones. One attractive
feature is that it can be programmed in Java and also in Python. The latter
is a very attractive language for rapid application development, with a
host of powerful built-in functions. Since I had programmed with Python
before, I started out my project by installing the PyS60 tools (Python
language[1]
and script-shell[2]) from the Sourceforge PyS60 Web site
to the phone. After doing that, you'll have a working Python environment on
your phone, and can test it by running one of the programs (e.g.,
"ball.py") that comes with the installation. The default installation
directory for Python scripts is the folder E:\Python
, which
resides on the memory card inside the phone. You can now write your own
Python scripts, and copy them to the same directory; then you can run them
in the same way you ran the example that came with the installation. To
give you a feel, here is the typical "hello, world" program. Just put the
following line in a file called "hello.py":
print "Hi there, World"
and transfer it to the phone, providing that you have the bluez
Bluetooth stack and the obex_ftp package installed. You transfer
"hello.py" with the obex_ftp
program by executing
obexftp -b 00:21:08:nn:nn:nn -c E:\\Python -p hello.py
where "00:21:08:nn:nn:nn" is the MAC address of the phone. (You can find
this by keying *#bta0# or *#2820# on
Nokia phones.) Another way is to turn on the phone and perform a Bluetooth
scan with hcitool scan
on your PC. The output will tell you
the MAC address of your phone's Bluetooth adapter (and probably those of
your neighbors' phones, as well). There are very nice pages on the Web
about Python programming, and I encourage you to consult them. I found
references [4] through [7] particularly useful. A book that I
particularly like as a reference for Python is Ref. [8].
[ When scanning for Bluetooth devices, make sure they are visible and "discoverable". Due to frequent scanning "attacks", some vendors have disabled the Bluetooth devices' visibility. I've also seen Bluetooth devices in PCs that stay invisible unless the visible mode is activated. This mode may also be called "inquiry mode". hcitool can change the mode for most devices. -- René ]
Now that we have a working Python environment and a way to transfer Python code to the phone, we can start coding the client program. This will monitor the keys on the phone and send appropriate commands to the server (discussed further below). The entire program is linked here, but we discuss the individual parts one at a time.
First, the Keyboard class that is copied straight from the "ball.py" example in Ref. [6]. This class represents a listener for the keyboard on the phone, and returns the key-codes of the pressed keys to the program. Then we have to open a Bluetooth connection to the server by executing
PCBTADDR=00:21:08:nn:nn:nn port=4 btsock = socket(AF_BT, SOCK_STREAM, BTPROTO_RFCOMM) target = (PCBTADDR, port) btsock.connect(target)
where PCBTADDR is the MAC address of the Bluetooth adapter on the PC. Then, we somewhat arbitrarily define port number 4 for use. (Just make sure no other service on your phone uses the port number you select, by executing "sdptool browse MAC_OF_PHONE" on the PC and checking the port numbers.) In the next line, a Bluetooth socket, btsock, is opened, and the connection to the server is established. This code is modeled after the examples in Ref.[9]. We then enter a loop starting with "while running", where the program waits a short while (0.1 s), and checks whether a key is pressed. If that is the case, a short text is displayed on the screen of the phone, and the corresponding text string is sent via the socket. Here is a snippet from the file "remote.py":
running=1 while running: e32.ao_sleep(0.1) e32.ao_yield() handle_redraw(()) if keyboard.is_down(EScancodeLeftArrow): canvas.text((0,32),u"Left key pressed",fill=0xff0000) btsock.send('Left') : # many more if keyboard.is_down(EScancodeHash): canvas.text((0,48),u"Hash key pressed",fill=0xff0000) btsock.send('Hash') running=0
The name of the scan-codes are documented in the PyS60 library reference in Ref. [7]. Please check the file remote.py for the full code listing. Note that apart from the core activities described here, there is also some code to update the user interface on the phone.
This file is all that is needed on the phone. Just copy it via
obex_ftp
, or any other means, to the Python subdirectory on
the phone. Once that is done, we can now progress to discussing the server
application on the PC.
On the PC
On the PC, we need a server to receive the commands sent from the phone, and take appropriate action. I decided to write this in Python as well, and needed to install the Python bindings for the bluez stack called PyBluez from Ref.[10]. The server program is then a very simple matter, and the full listing is linked here. The main part of the program opens the socket:
server_sock=BluetoothSocket( RFCOMM ) server_sock.bind(("",port)) server_sock.listen(backlog) client_sock, client_info = server_sock.accept()
and then enters in a runtime loop that receives the package from the phone and takes suitable action. A snippet is shown here:
running=1 while running: data = client_sock.recv(10) if data=='Hash': running=0 if data=='Left': os.system("xevent -r -10 0") : client_sock.close() server_sock.close()
Note that a hash terminates the runtime loop, after which the last two
commands that close the socket are executed. The action to be performed is
simply written as a string that describes a Unix command which is passed
directly to the operating system via the os.system()
function
call. This makes the code very adaptable to whatever whims strike the
programmer.
Remember the wishlist from the start of this article, where I wanted to control the cursor on the desktop? This feat is actually performed by the program "xevent" that is executed within the runtime loop of the server program. In the above example, it moves the cursor relative to its present position by 10 pixels to the left. How this is done is discussed in detail, in the following section.
Causing X-window events without keyboard or mouse
I wrote the xevent program [11], which is based on code from
Ref.[12] and
the X Window System library libXtst, to cause X-windows events with a command line
program. For example, the command "xevent -a 0 0" will move the mouse
cursor to the absolute position pixel (0,0), which is the top left corner of
the screen. Please inspect the rather short source
of "xevent" for details, but the rough line of execution is as follows:
first the display and default screen are assigned and the root window is
specified; then, depending on the command-line argument, different actions
are taken. If the relative mouse motion (-r) is specified, the
libXtst function XTestFakeRelativeMotionEvent()
is called with appropriate arguments; if absolute motion is specified (-a),
XTestFakeMotionEvent()
is called. If a mouse button event is
called (-b), the function XTestFakeButtonEvent()
is used,
either as a button down (is_down=1) event or button up (is_down=0) event.
The default action is down and up. Similarly, the libXtst function
XTestFakeKeyEvent()
is used to simulate a
key-press event, either down, up, or both. Please check the manual page for
xevent that is available on the Web page specified in Ref.[11]. The
libXtst functions are defined in the header file libXtst.h.
Putting It All Together
Once the "remote.py" file resides on the phone, the "xevent" program is installed on the PC, and the "remote_on_pcx.py" is copied to the PC, we can start the server by entering the following in a command window:
python remote_on_pc.py
which will wait for the phone to make a connection. Then, we start the client on the phone by running the "remote.py" script there. The phone then makes a connection to the server. The server program on the PC will acknowledge the connection if everything went well, and the screen on the phone will turn white. Then you can use the the big navigation button on the phone to steer the mouse on your PC and the big select button as the left mouse click. If you press it twice in rapid sequence, it will behave as a double-click. The initial assignment of the keys is documented in the following table for easy reference, but this can be easily changed by editing the server program "remote_on_pc.py" and just assigning different events to all keys. I picked just those that are most convenient to steer programs.
Key on the Phone |
Action on the PC |
Select Key | Left mouse click |
Left | Move mouse left by 10 pixel on PC screen |
Right | Move mouse right by 10 pixel on PC screen |
Up | Move mouse up by 10 pixel on PC screen |
Down | Move mouse down by 10 pixel on PC screen |
1 | Enter key pressed |
2 | Middle mouse click |
3 | Right mouse click |
4 | Page Up |
5 | Text cursor up |
6 | Page Down |
7 | Text cursor left |
8 | Text cursor down |
9 | Text cursor right |
Asterisk (*) | Tab key pressed |
0 | Escape key pressed |
Hash (#) | Exit the program |
The code in both the client and server applications was deliberately made rather simple, because that makes it easier to understand and adapt to individual wishes. Turning this skeleton into a user-friendly package will certainly require more work.
A note about copyright: the xevent program is based on the code from Ref.[12], which is GPLed, and therefore xevent is also GPL. The client-server construction is so trivial and commonly known that assigning a copyright seems useless. The keyboard class in "remote.py" is a straight copy from Ref. [6], and is licensed by Nokia for personal use.
That being said: I hope you will have fun playing around with the remote control, and adapt it to whatever fun things you come up with. My next project is a remote control for a USB-controlled foam-rocket launcher.
References
- [1] PythonForS60_1_4_n_3rdEd.sis
- [2] PythonScriptShell_1_4_n_3rdEd.sis
- [3] http://sourceforge.net/projects/pys60
- [4] Installing PyS60 on http://wiki.opensource.nokia.com/projects/Installing_PyS60
- [5] Getting Started with Python for Series 60 Platform
- [6] Programming with Python for Series 60 Platform
- [7] PyS60 Library Reference, PythonForS60_1_4_n_doc.pdf available from http://sourceforge.net/projects/pys60
- [8] A. Martelli, Python in a Nutshell, O'Reilly, Cambridge, 2003.
- [9] Socket Programming section on http://pramode.net/articles/lfy/mobile/pramode.html
- [10] PyBluez-0.15.tar.gz from http://code.google.com/p/pybluez/
- [11] http://www3.tsl.uu.se/~ziemann/xevent
- [12] xsendkey-0.01.tar.bz2 from http://www.ltn.lv/~aivils/
Talkback: Discuss this article with The Answer Gang
Volker lives in Uppsala, Sweden and works as a physicist on particle accelerator based projects at CERN in Switzerland and DESY in Germany. He was forced into using computers during his undergraduate thesis in the mid-eighties and has used them with growing enthusiasm ever since. He is an active Linux user since 1995.