Tuesday, 7 October 2008

Testing with the Android Framework

The second installment on testing with Android. There is a dearth of decent documentation (or indeed any documentation) out on how to test Android applications.

The best starting point is to have a look at the ApiDemos sample that comes with the SDK. It has a complete and working setup for a variety of tests. The first steps are to make sure that this actually works on your machine.

  1. First of all, build the officially supplied test suite. This does not work out of Eclipse as far as I can tell, so just go to a command line window and go to /samples/ApiDemos/tests and do a build with ant.
  2. You might find that the ApiDemos itself was not built before, so go there first.
  3. Once the ApiDemos is built, try building the test again.
  4. Now you need to install the ApiDemos and the tests. This is done by going into the respective bin directories and run adb install ApiDemos-debug.apk and adb install ApiDemosTest-debug.apk. This requires that the emulator is running. Mine is, as I started it from Eclipse.
  5. Now restart the emulator. This is important as otherwise it might not know that these new apks have been installed. I made the mistake of letting it run, and I always got an error when running with the suggested commands.
  6. Run something like 
    $ adb shell am instrument -w com.example.android.apis.tests/android.test.InstrumentationTestRunner

    The tests should now run (some fail for me, but the main machinery is up and running).

Now ist the time to get my own tests in:
  1. I recreate the directory structure for my tests from the Google template and copy the AndroidManifest.xml and build.xml. Both will need some modification to make sure that my classes will be tested.
  2. In the src directory I adjust the files for the top-level test runners and add a directory that will exercise my own TracklogViewer.
  3. The build with ant fails on a few points. -e option as usual gives a bit more information. Some directories need to be created by hand as they were expected but are not there.
  4. I install my own test apk on the emulator.
  5. I run the instrumentation.
  6. It fails. The reason, it transpires, is that I have not restarted the emulator.
  7. It finally runs. As the tests are pretty empty, there is no error. 
Now it comes to actually implementing a few tests. But that is another story.


Mariano said...

Maybe the tests are not meant by google to be usable just yet?

Anyway, I am pretty new to that and was trying to follow along your instructions (thanks btw.), but I am stuck with something very basic in step 1).

I don't seem to find a build.xml?!

I am using android-sdk-mac_x86-1.0_r1.zip. Do you use another version?

Ludwig said...

You should be able to find the build.xml for the ApiDemos in the samples/ApiDemos directory. Build this first with ant.

After it has built, change into the tests directory below where there is another build.xml.

AFAIR the build.xml files were just in the SDK installation (mine is on 1.0_r1 on vista, but there should be no difference).

For my own tests I first copied the tests/build.xml to my top-level test directory and made a few modifications in the first block.

kingarold said...

There is no build.xml files in both directories in android-sdk-windows-1.0_r1.zip =((
I tried to make them using activityCreator, but no success for tests/build.xml.

Ludwig said...

You are right, I am wrong. There are no build.xml files supplied with the Android sample code.

I am a bit puzzled right now where I actually got my build.xml files from.

Anyway, I pasted the build.xml for the api tests at dpaste

kingarold said...

Thanks a lot!

But when I tried to run ant in tools/ I got great number of errors like this:

Buildfile: build.xml

[echo] Creating output directories if needed...

[echo] Generating R.java / Manifest.java from the resources...

[echo] Compiling aidl files into Java classes...

[javac] Compiling 10 source files to C:\android-sdk-windows-1.0_r1\samples\ApiDemos\tests\bin\classes
[javac] C:\android-sdk-windows-1.0_r1\samples\ApiDemos\tests\src\com\example\android\apis\ApiDemosApplicationTests.java:31: cannot find symbol
[javac] symbol: class ApiDemosApplication
[javac] public class ApiDemosApplicationTests extends ApplicationTestCase_ApiDemosApplication_ {
[javac] ^

Ludwig said...

It requires that the api samples itself is build first.

I dpasted the build.xml for the api samples as well.

I have no recollection how I actually got these files. They are not generated from Eclipse and I certainly did not type them myself. If anyone can remind me...

kingarold said...

I have re-created apidemos from android-sdk-windows-1.0_r1.zip and then use yours build.xml (with using activitycreator --out . --ide intellij com.example.android.ApiDemos before). Ant now succeed with both. Thanks!

Later I will try to install apk to device.

kingarold said...

Oh, I had success in testing ApiDemos! But unfortunately I cant build with my own tests for my application. When I am starting ant build, it tells me "type parameter _name of my main activity class with package_ is not within its bound...

Ludwig said...

Hi, the error message from your build is a bit short to make any sense of.
I assume you have adjusted the build.xml to your directory settings (only the top part should be affected)?
The test build.xml only builds the tests, so it is assumed that all your other code has been build already and is available in the bin directory under your root development directory.

kingarold said...

Hi! It was my mistake - I have no Application class in my project=). So I deleted test-class that extends ApplicationTestCase. Build all with ant, install both apk to emulator (be sure that both are installed by command "adb shell pm list packages"). And eventually everything is working with my one TestCase!
Thanks a lot for your guides!

Bee said...

Hi Ludwig,
I am in the same boat as Kingarold.

I just picked up your build.xml and placed that in the samples/ApiDemos.

One clarification before I run ant.
Do I pickup the 2nd build.xml and place that in the tests folder?

Many thanks. If this works, it will be a great help for me.

Ludwig said...

One of the build.xml files goes into the ApiDemos folder, the other in the tests folder below it.
You need to first build ApiDemos and when that has completed successfully the tests folder.

Don't forget to make the adjustments for your settings in the top of the build.xml files.


Diego Torres Milano said...

I've published a post featuring a simple way to build and run ApiDemos
tests not needing ant or maven. This method can also be used to build
and run tests for other projects as well.
See Android: Testing on the Android platform - ApiDemos tests.

Comments are gladly welcome.

Grace Hu said...

Hi, a same error happened in the project like kingarold. BUt i have build the main project successfully, and have created the applicaion.class. it exites in the but when i build the test build.xml, it still show the errors. I don't what should i do next.

gudujarlson said...

Could you please repost the build.xml's?