Saturday, August 6, 2011

Android Screen Control Without Root

The ability to control the screen of my Android phone has been a major pain since I moved from Windows Mobile to Android quite some time ago.

The only solutions required root permissions. However there are scenarios where that is not an option such as in a corporate environment / providing remote support for a device that is not your own.

Android 2.1 onwards ( as far as I know) supports a API called MonkeyRunner that is to be used for application testing. Using this along with java sources provided by "The Android Open Source Project" I have put together a small application that will capture the screen and allow passing of click and key press events to the device.

It is highly limited to the extent that it allows only for touch events and keystrokes. It currently does not support drag events and typing of special characters but something is better than nothing :)


Download the zip file from here

http://www.mediafire.com/file/a8870ynh151vm9d/ScreenControl.zip

In order to run the files, you need to ensure that the device is connected via USB cable only and debugging has been enabled on the device.

Once this is done, Fire up the screen.bat and it should load up the window. The screen.bat is a single line command calling java.exe along with all the required class and jar files required to make this work.

I have also included adb.exe which is required, you could replace with your own, however mine will probably be more stable.



Again, Consider this only if your device is not rooted. If you have rooted your device I would recommend you use the either of the following better softwares

Android Screencast
http://code.google.com/p/androidscreencast/

This works with the USB connection

Droid VNC Server by José Luis Pereira
http://www.onaips.com/wordpress/?page_id=60

This works with Wifi



I hope this has been helpful for some of you all. Am sure there are more brilliant people out there who can modify the sources to add drag functionality as well. Do let me know !


UPDATE
F.J. Wechselberger has integrated this functionality into MyPhoneExplorer through native TCP-Calls. Its not possible for him to implement Java-Library into the PC-Program but he was able to translate the source of MonkeyRunner. I Would recommend you download this if you are looking for a simple solution for controlling the screen.

www.fjsoft.at


Cheers,
Vikram Sridharan.
at

26 comments:

  1. AnonymousAugust 8, 2011 at 9:33 PM

    I would sure love this to work!!! Exactly what I am looking for. Downloaded zip, usb connected, and 'storage in use', ran screen.bat, but the command window pops up and hangs. Nothing else. In reading some of the files, it mentions not having any other USB devices connected? I ahve sevearl (keyboard, drives, etc) Do I need to disconnect them all?

    ReplyDelete
  2. Vikram SridharanAugust 9, 2011 at 5:45 PM

    Other USB devices can be connected, it should not be an issue. Just make sure storage is not in use and the USB Debugging is checked. This option is in Settings -> Applications -> Development.

    Hope you have java available on your computer.

    ReplyDelete
  3. AnonymousAugust 9, 2011 at 10:31 PM

    This is great work! Will you be adding the drag events or provide some way to scroll using the UI? If that feature is added it could definitely be used to remotely support Android devices.

    ReplyDelete
  4. AnonymousAugust 9, 2011 at 10:31 PM

    Looks like drag is supported by the Money API. Just need to map it to the UI. Creating scroll up/down right/left buttons on the UI may be a good way to implement it at first and then connect the event to what the user is doing with the mouse in a later release.

    Example of someone using Monkey for a drag event.
    http://stackoverflow.com/questions/4396059/how-to-simulate-android-touch-event

    ReplyDelete
  5. Vikram SridharanAugust 10, 2011 at 5:36 PM

    @Anonymous

    Thanks ! Drag is support by Monkryrunner, however my knowledge is programming is quite limited, am trying to get some help on this which is why the zip has all the sources. Am open to anyone to help on this. If I do manage, will definitely update the files here.

    ReplyDelete
  6. AnonymousAugust 11, 2011 at 3:19 AM

    Well you have done well for someone with limited programming knowledge!

    I would help if I could. I'm a C++ and .Net guy so I would have to come up a learning curve with the tools and environment before I could even contribute.

    ReplyDelete
  7. AnonymousAugust 12, 2011 at 3:20 AM

    Great job! I had the first post about it not working... I had java and the SDK, but was missing some python stuff on my computer. It is now up and running. This shows a lot of promise. My three comments: (1) it seems laggy, especially when typing. I don't mind the slow screen refresh, but typing is very stop and go. (2) when typing, the space bar takes you completely back to the home screen. I read that there are problem handling the space bar in a string, but that there are work arounds. (3) my droid is in a desktop dock on the computer, so it is in landscape format. The PC emulator shows it in landscape, but rotated to portrait, so everything is sideways. I suppose I could figure out how to have the droid stay in portrait when docked... but until the spacebar is fixed, its not gonna work for me. Wish I could help you with the coding on it!!! I'll stay tuned.

    ReplyDelete
  8. AnonymousAugust 18, 2011 at 8:43 PM

    Any idea when you'll have an update that allows for scrolling and fixes the space bar issues? I can hardly wait!

    ReplyDelete
  9. AnonymousAugust 27, 2011 at 3:25 AM

    Hi There,

    I am getting the following Exception. Need Help please.

    Exception in thread "main" java.lang.NoClassDefFoundError: com/android/monkeyrun
    ner/controller/MonkeyController
    Caused by: java.lang.ClassNotFoundException: com.android.monkeyrunner.controller
    .MonkeyController
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    Could not find the main class: com.android.monkeyrunner.controller.MonkeyControl
    ler. Program will exit.

    ReplyDelete
  10. Dokta John mo Dokta DianeOctober 14, 2011 at 7:40 PM

    Hi Vikram,
    I am having trouble "firing up Screen.bat"
    Double clicking opens up gedit.
    If I use my terminal I get the following output:
    john@john-775Dual-880Pro:~/ScreenControl/files$ Screen.bat
    Screen.bat: command not found
    john@john-775Dual-880Pro:~/ScreenControl/files$ cd Screen.bat
    bash: cd: Screen.bat: Not a directory
    john@john-775Dual-880Pro:~/ScreenControl/files$ ./Screen.bat
    bash: ./Screen.bat: Permission denied
    john@john-775Dual-880Pro:~/ScreenControl/files$ sudo ./Screen.bat
    [sudo] password for john:
    sudo: ./Screen.bat: command not found
    john@john-775Dual-880Pro:~/ScreenControl/files$
    What am I doing wrong?
    John

    ReplyDelete
  11. Vikram SridharanOctober 15, 2011 at 10:30 PM

    @john

    You are running the files in a linux / unix operating system. The program is meant for windows.

    However considering it is a java based, am sure it should be possible to run on a unix platform but unfortunately am not sure of the right syntax.

    ReplyDelete
  12. AnonymousNovember 4, 2011 at 5:57 PM

    is there actual any way round the hanging after launching the screen.bat file?

    ReplyDelete
  13. AnonymousJanuary 24, 2012 at 3:52 PM

    It also hangs for me (Win7 Pro, HTC Desire S). Someone replied earlier about "...missing some python stuff on my computer.." - how do I find out what might be missing?

    ReplyDelete
  14. AnonymousMarch 22, 2012 at 4:40 PM

    no need of python for me, just had to copy all the content of c:\program files\java\jre6\bin into the unzipped Screencontrol directory and it no longer hangs but perfectly works !
    Awesome work dude !!! You rock !!!!!
    Would be real nice however to add the slide ability (up/down / sideways).

    ReplyDelete
  15. DeadMattMarch 31, 2012 at 5:12 AM

    Hi,

    I get this message and dont know how I can fix this :/

    31.03.2012 01:40:08 com.android.monkeyrunner.MonkeyManager sendMonkeyEventAndGet
    Response
    INFO: Monkey Command: wake.
    31.03.2012 01:40:08 com.android.monkeyrunner.adb.AdbMonkeyDevice createManager
    SCHWERWIEGEND: Timeout while trying to create monkey mananger
    Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:
    182)
    at com.android.monkeyrunner.adb.AdbMonkeyDevice.(AdbMonkeyDevice.j
    ava:68)
    at com.android.monkeyrunner.adb.AdbBackend.waitForConnection(AdbBackend.
    java:100)
    at com.android.monkeyrunner.adb.AdbBackend.waitForConnection(AdbBackend.
    java:91)
    at com.android.monkeyrunner.controller.MonkeyController$1.run(MonkeyCont
    roller.java:43)
    at java.awt.event.InvocationEvent.dispatch(Unknown Source)
    at java.awt.EventQueue.dispatchEvent(Unknown Source)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.awt.EventDispatchThread.run(Unknown Source)


    to copy the java bin does not wort for me :(
    - Win7, Motorola Milestone2

    any ideas? want these functionality!

    ReplyDelete
  16. Vikram SridharanMarch 31, 2012 at 10:31 PM

    @ all

    Please check the post have updated on an easier alternative !

    ReplyDelete
  17. AnonymousApril 3, 2012 at 4:33 AM

    ita say to me:

    C:\Documents and Settings\Shlomi\Desktop\files>java.exe -cp MonkeyRunner.jar;ddm
    lib.jar;sdklib.jar;guavalib.jar com.android.monkeyrunner.controller.MonkeyContro
    ller

    ReplyDelete
  18. AnonymousMay 26, 2012 at 6:16 AM

    Encountered the same problem as right above: launching the command prompt showing the line in Screen.bat only and nothing happened (ie. without having the MonkeyController window). How to resovle?

    ReplyDelete
  19. AnonymousJune 12, 2012 at 4:11 PM

    Excellent, perfect on HTC Sensation XE.

    ReplyDelete
  20. AnonymousJune 12, 2012 at 7:28 PM

    This is awesome, or at least it would if the screen wasn't rotated. The screen taps are correct it's just the image that's wrong.

    Is there any way to rotate the image ?

    ReplyDelete
  21. AnonymousJune 21, 2012 at 2:27 PM

    I've a "socket write error"... What should I do?

    ReplyDelete
  22. AnonymousJune 26, 2012 at 1:44 AM

    It would be perfect it worked because there is no way to open up the program

    ReplyDelete
  23. AnonymousAugust 10, 2012 at 10:20 AM

    You can try this Android app, VMLite VNC Server that does NOT require root.

    https://play.google.com/store/apps/details?id=com.vmlite.vncserver

    ReplyDelete
  24. mark anthonyOctober 17, 2012 at 2:37 PM

    It feels great to feature much revealing and unequalled articles on your websites.
    one click root blog

    ReplyDelete
  25. Sudhir RanjanSeptember 7, 2013 at 4:04 PM

    Thank you very much.it's work for me but the question is same project i want to implement in android so that if you install this Application on your mobile you can run directly while connecting with USB. Please suggest me...

    ReplyDelete
  26. AnonymousApril 20, 2014 at 1:05 PM

    Hi , I run it from linux . I followed following things :


    1. Copy all .jar files except monkeyrunner.jar to a folder named libs in the same directory.

    2. Run the following command :
    java -cp monkeyrunner.jar:libs/* com.android.monkeyrunner.controller.MonkeyController

    ReplyDelete