android_frameworks_base/docs/html/tools/testing/testing_android.jd

page.title=Testing Concepts
parent.title=Testing
parent.link=index.html
@jd:body

<div id="qv-wrapper">
  <div id="qv">
  <h2>In this document</h2>
  <ol>
    <li><a href="#TestStructure">Test Structure</a></li>
    <li><a href="#TestAPI">Testing APIs</a>
      <ol>
        <li>
          <a href="#JUnit">JUnit</a>
        </li>
        <li>
          <a href="#Instrumentation">Instrumentation</a>
        </li>
        <li>
            <a href="#TestingSupportLibraryClasses">Android Testing Support Library APIs</a>
        </li>
        <li>
          <a href="#AssertionClasses">Assertion classes</a>
        </li>
      </ol>
    </li>
    <li>
        <a href="#Monkeys">Monkey and Monkeyrunner</a>
    </li>
  </ol>
  <h2>See also</h2>
  <ol>
      <li><a href="{@docRoot}training/testing/start/index.html">Getting Started with Testing</a></li>
  </ol>
  </div>
</div>

<p>
This document describes key concepts related to Android app testing. It assumes you have a basic
knowledge of the <a href="http://junit.org/" class="external-link">JUnit</a> testing framework.</p>

<h2 id="TestStructure">Test Structure</h2>
<p>Android testing is based on JUnit. In general, a JUnit test is a method whose statements test a
part of the application. You organize test methods into classes called
<em>test cases</em>. You can further organize these classes into <em>test suites</em>.</p>
<p>In JUnit, you build one or more test classes and use a test runner to
execute them. In Android, you use <a href="{@docRoot}tools/studio/index.html">
Android Studio </a>(or the <a href="{@docRoot}tools/building/plugin-for-gradle.html">
Android Plugin for Gradle</a>) to build one or more test source files into an
<em>Android test app</em>.</p>

<p>From your testing environment, you can run your test in one of the following ways:</p>
<ul>
    <li><strong>On your local machine:</strong> Compile the test classes and
        execute them locally on the Java Virtual Machine (JVM) using the JUnit test runner.</li>
    <li><strong>On a device or emulator</strong>: Install the test app and the app
under test to a physical device or emulator, and then execute your tests using an Android-specific
test runner (such as <a href="{@docRoot}tools/testing-support-library/index.html#AndroidJUnitRunner">
{@code AndroidJUnitRunner}</a>).</li>
</ul>

<p>The structure of your test code and the way you build and run the tests in Android Studio depend
on the type of testing you are performing. The following table summarizes the common testing types
for Android:</p>

<table>
<tr><th>Type</th>
    <th>Subtype</th>
    <th>Description</th></tr>

<tr><td rowspan="3">Unit tests</td>
    <tr><td>Local Unit Tests</td>
        <td>Unit tests that run on your local machine only. These tests are compiled to run locally
on the JVM to minimize execution time. Use this approach to run unit tests
that have no dependencies on the Android framework or have dependencies that mock objects can
satisfy.</td></tr>
    <tr><td>Instrumented unit tests</td>
        <td>Unit tests that run on an Android device or emulator. These tests have access to
{@link android.app.Instrumentation} information, such as the {@link android.content.Context} of the
app under test. Use this approach to run unit tests that have Android dependencies which mock
objects cannot easily satisfy.</td></tr>

<tr><td rowspan="3">Integration Tests</td>
    <tr><td>Components within your app only</td>
        <td>This type of test verifies that the target app behaves as expected when a user performs
a specific action or enters a specific input in its activities. For example, it allows you to check
that the target app returns the correct UI output in response to user interactions in the app’s
activities. UI testing frameworks like
<a href="{@docRoot}tools/testing-support-library/index.html#Espresso">Espresso</a> allow you to
programmatically simulate user actions and test complex intra-app user interactions.</td></tr>
    <tr><td>Cross-app Components</td>
    <td>This type of test verifies the correct behavior of interactions between different user
apps or between user apps and system apps. For example, you might want to test that your app behaves
correctly when the user performs an action in the Android Settings menu. UI testing frameworks
that support cross-app interactions, such as UI Automator, allow you to create tests for such
scenarios.</td></tr>
</table>

<p>Based on the type of test you want to create, you need to configure the test code source
location and the project dependencies in Android Studio as described in
<a href="{@docRoot}training/testing/start/index.html">Getting Started with Testing</a>.</p>

<h2 id="TestAPI">Testing APIs</h2>
<p>The following list summarizes the common APIs related to app testing for Android.</p>

<h3 id="JUnit">JUnit</h3>

<p>You should write your unit or integration test class as a JUnit 4 test class.
<a href="http://junit.org/" class="external-link">JUnit</a> is the most popular
and widely-used unit testing framework for Java. The framework offers a convenient way to perform
common setup, teardown, and assertion operations in your test.</p>

<p>JUnit 4 allows you to write tests in a cleaner and more
flexible way than its predecessor versions. Unlike the previous approach to Android unit testing
based on JUnit 3, with JUnit 4, you do not need to extend the {@code junit.framework.TestCase}
class. You also do not need to prepend the {@code test} keyword to your test method name, or
use any classes in the {@code junit.framework} or {@code junit.extensions} package.</p>

<p>A basic JUnit 4 test class is a Java class that contains one or more test methods.
A test method begins with the {@code &#64;Test} annotation and contains the code to exercise
and verify a single functionality (that is, a logical <em>unit</em>) in the component that you want
to test.</p>
<p>The following snippet shows an example JUnit 4 integration test that uses the Espresso
APIs to perform a click action on a UI element, then checks to see if an expected string is
displayed.</p>
<pre>
&#64;RunWith(AndroidJUnit4.class)
&#64;LargeTest
public class MainActivityInstrumentationTest {

    &#64;Rule
    public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<>(
            MainActivity.class);

    &#64;Test
    public void sayHello(){
        onView(withText("Say hello!")).perform(click());

        onView(withId(R.id.textView)).check(matches(withText("Hello, World!")));
    }
}
</pre>
<p>In your JUnit 4 test class, you can call out sections in your test code for
special processing by using the following annotations:</p>
<ul>
<li>
{@code &#64;Before}: Use this annotation to specify a block of code that contains test setup
operations. The test class invokes this code block before each test. You can have multiple
{@code &#64;Before} methods but the order in which the test class calls these methods
is not guaranteed.
</li>
<li>
{@code &#64;After}: This annotation specifies a block of code that contains test
tear-down operations. The test class calls this code block after every test method. You can define
multiple {@code &#64;After} operations in your test code. Use this annotation to release any
resources from memory.
</li>
<li>
{@code &#64;Test}: Use this annotation to mark a test method. A single test class can contain
multiple test methods, each prefixed with this annotation.
</li>
<li>
{@code &#64;Rule}: Rules allow you to flexibly add or redefine the behavior of each test
method in a reusable way. In Android testing, use this annotation together with
one of the test rule classes that the Android Testing Support Library provides, such as
<a href="{@docRoot}reference/android/support/test/rule/ActivityTestRule.html">
{@code ActivityTestRule}</a> or
<a href="{@docRoot}reference/android/support/test/rule/ServiceTestRule.html">
{@code ServiceTestRule}</a>.
</li>
<li>
{@code &#64;BeforeClass}: Use this annotation to specify static methods for each test class to
invoke only once. This testing step is useful for expensive operations such as connecting to a
database.
</li>
<li>
{@code &#64;AfterClass}: Use this annotation to specify static methods for the test class to invoke
only after all tests in the class have run. This testing step is useful for releasing any
resources allocated in the {@code &#64;BeforeClass} block.
</li>
<li>
{@code &#64;Test(timeout=&lt;milliseconds&gt;)}: Some annotations support the ability to pass in
elements for which you can set values. For example, you can specify a timeout period for the test.
If the test starts but does not complete within the given timeout period, it automatically fails.
You must specify the timeout period in milliseconds, for example: {@code &#64;Test(timeout=5000)}.
</li>
</ul>
<p>For more annotations, see the documentation for
<a href="//junit.sourceforge.net/javadoc/org/junit/package-summary.html" class="external-link">
JUnit annotations</a> and the
<a href="{@docRoot}android/support/test/annotation/package-summary.html">
Android-specific annotations</a>.
<p>
    You use the JUnit {@link junit.framework.Assert} class to verify the correctness of an object's
    state. The assert methods compare values you expect from a test to the actual results and
    throw an exception if the comparison fails. <a href="#AssertionClasses">
    Assertion classes</a> describes these methods in more detail.
</p>

</p>
<h3 id="Instrumentation">Instrumentation</h3>
<p>
    Android instrumentation is a set of control methods or hooks in the Android system. These
    hooks control an Android component independently of its normal lifecycle. They also control how
    Android loads applications.
</p>
<p>
    The following diagram summarizes the testing framework:
</p>
<div style="width: 70%; margin-left:auto; margin-right:auto;">
<a href="{@docRoot}images/testing/test_framework.png">
    <img src="{@docRoot}images/testing/test_framework.png"
        alt="The Android testing framework"/>
</a>
</div>
<p>
Normally, an Android component runs in a lifecycle that the system determines. For example, an
{@link android.app.Activity} object's lifecycle starts when an {@link android.content.Intent}
activates the {@link android.app.Activity}. The system calls the object's <code>onCreate()</code>
method, on then the <code>onResume()</code> method. When the user starts another application, the
system calls the <code>onPause()</code> method. If the {@link android.app.Activity} code calls
the <code>finish()</code> method, the system calls the <code>onDestroy()</code> method. The Android
framework API does not provide a way for your code to invoke these callback methods directly, but
you can do so using instrumentation.
</p>
<p>
The system runs all the components of an application in the same process. You can allow some
components, such as content providers, to run in a separate process,
but you can't force an application to run in the same process as another application that is
already running.
</p>
<p>
Instrumentation can load both a test package and the app under test into the
same process. Since the application components and their tests are in the same process, your
tests can invoke methods in the components, and modify and examine fields in the components.
</p>
<h3 id="TestingSupportLibraryClasses">Android Testing Support Library APIs</h3>
<p>
The <a href="{@docRoot}tools/testing-support-library/index.html">Android Testing Support Library</a>
provides a set of APIs that allow you to quickly build and run test code for your apps, including
JUnit 4 and functional user interface (UI) tests. The library includes the following
instrumentation-based APIs that are useful when you want to automate your tests:</p>
<ul>
    <li><a href="{@docRoot}tools/testing-support-library/index.html#AndroidJUnitRunner">
        {@code AndroidJUnitRunner}</a>:
      JUnit 4-compatible test runner for Android
    </li>

    <li><a href="{@docRoot}tools/testing-support-library/index.html#Espresso">Espresso</a>:
    UI testing framework; suitable for functional UI testing within an app
    </li>

    <li><a href="{@docRoot}tools/testing-support-library/index.html#UIAutomator">UI Automator</a>:
    UI testing framework; suitable for cross-app functional UI testing across system and installed
    apps</li>
  </ul>

<h3 id="AssertionClasses">Assertion classes</h3>
<p>Because Android Testing Support Library APIs extend JUnit, you can use assertion methods to
display the results of tests. An assertion method compares an actual value returned by a test to an
expected value, and throws an AssertionException if the comparison test fails. Using assertions
is more convenient than logging, and provides better test performance.
</p>
<p>To simplify your test development, we recommend that you use the
<a href="https://github.com/hamcrest"
class="external-link">Hamcrest</a> library, which lets you create more flexible tests using the
Hamcrest matcher APIs.</p>

<h2 id="Monkeys">Monkey and Monkeyrunner</h2>
<p>
    The SDK provides two tools for functional-level application testing:
</p>
    <ul>
        <li>
The <a href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a>,
            usually called "monkey", is a command-line tool that sends pseudo-random streams of
            keystrokes, touches, and gestures to a device. You run it with the
            <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge</a> (adb) tool.
            You use it to stress-test your application and report back errors that are encountered.
            You can repeat a stream of events by running the tool each time with the same random
            number seed.
        </li>
        <li>
    The <a href="{@docRoot}tools/help/monkeyrunner_concepts.html">monkeyrunner</a> tool
            is an API and execution environment for test programs written in Python. The API
            includes functions for connecting to a device, installing and uninstalling packages,
            taking screenshots, comparing two images, and running a test package against an
            application. Using the API, you can write a wide range of large, powerful, and complex
            tests. You run programs that use the API with the <code>monkeyrunner</code> command-line
            tool.
        </li>
    </ul>