android_frameworks_base/docs/html/tools/testing/testing_otheride.jd

page.title=Testing from the Command-Line

@jd:body

<div id="qv-wrapper">
    <div id="qv">
        <h2>In this document</h2>
            <ol>
                <li>
                    <a href="#RunTestsCommand">Running Tests</a>
                    <ol>
                        <li>
                            <a href="#RunTestsGradle">Running unit tests with Gradle</a>
                        </li>
                        <li>
                            <a href="#RunTestsDevice">Running tests with adb</a>
                        </li>
                    </ol>
                </li>
                <li>
                    <a href="#AMSyntax">Using the Instrument Command</a>
                    <ol>
                        <li>
                            <a href="#AMOptionsSyntax">Instrument options</a>
                        </li>
                        <li>
                            <a href="#RunTestExamples">Instrument examples</a>
                        </li>
                    </ol>
                </li>
            </ol>
        <h2>See Also</h2>
            <ol>
                <li>
                    <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge</a>
                </li>
            </ol>
    </div>
</div>
<p>
    This document describes how to create and run tests directly from the command line. This
    document assumes that you already know how to create a Android application in your programming
    environment.
</p>

<h2 id="RunTestsCommand">Running Tests</h2>
<p>
    You can run tests from the command-line, either with Gradle or with an
    <a href="{@docRoot}tools/help/adb.html">
    Android Debug Bridge (adb)</a> shell.
</p>
<h3 id="RunTestsGradle">Running unit tests with Gradle</h3>

<p>The <a href="{@docRoot}tools/building/plugin-for-gradle.html">Android Plugin for Gradle</a>
lets you run unit tests from your Gradle project via the command-line. For more information on
how to build unit tests for your app, see
<a href="{@docRoot}training/testing/unit-testing/index.html">Building Effective Unit Tests</a>.</p>

<p>The table below summarizes how to run your unit tests with Gradle:</p>
<table>
    <tr>
        <th>Unit Test Type</th>
        <th>Command To Run</th>
        <th>Test Result Location</th>
    </tr>
    <tr>
        <td>Local unit test</td>
        <td>Call the {@code test} task:
<pre>
./gradlew test
</pre></td>
        <td>
HTML test result files:
{@code &lt;path_to_your_project&gt;/app/build/reports/tests/} directory.
<p>XML test result files:
{@code &lt;path_to_your_project&gt;/app/build/test-results/} directory.
</p></td>
    </tr>
    <tr>
        <td>Instrumented unit test</td>
        <td>Call the {@code connectedAndroidTest} (or {@code cAT}) task:
<pre>
./gradlew cAT
</pre>
        </td>
        <td>
HTML test result files:
{@code &lt;path_to_your_project&gt;/app/build/outputs/reports/androidTests/connected/} directory.
<p>XML test result files:
{@code &lt;path_to_your_project&gt;/app/build/outputs/androidTest-results/connected/} directory.
</p></td>
    </tr>
</table>

<h3 id="RunTestsDevice">Running tests with ADB</h3>
<p>
    When you run tests from the command-line with
    <a href="{@docRoot}tools/help/adb.html">
    Android Debug Bridge (adb)</a>, you get more options for choosing the tests
    to run than with any other method. You can select individual test methods, filter tests
    according to their annotation, or specify testing options. Since the test run is controlled
    entirely from a command-line, you can customize your testing with shell scripts in various ways.
</p>
<p>
    To run a test from the command-line, you run <code>adb shell</code> to start a command-line
    shell on your device or emulator, and then in the shell run the <code>am instrument</code>
    command. You control <code>am</code> and your tests with command-line flags.
</p>
<p>
    As a shortcut, you can start an <code>adb</code> shell, call <code>am instrument</code>, and
    specify command-line flags all on one input line. The shell opens on the device or emulator,
    runs your tests, produces output, and then returns to the command-line on your computer.
</p>
<p>
    To run a test with <code>am instrument</code>:
</p>
<ol>
    <li>
        If necessary, rebuild your main application and test package.
    </li>
    <li>
        Install your test package and main application Android package files
        (<code>.apk</code> files) to your current Android device or emulator</li>
    <li>
        At the command-line, enter:
<pre>
$ adb shell am instrument -w &lt;test_package_name&gt;/&lt;runner_class&gt;
</pre>
        <p>
            where <code>&lt;test_package_name&gt;</code> is the Android package name of your test
            application, and <code>&lt;runner_class&gt;</code> is the name of the Android test
            runner class you are using. The Android package name is the value of the
            <code>package</code> attribute of the <code>manifest</code> element in the manifest file
            (<code>AndroidManifest.xml</code>) of your test package. The Android test runner
            class is usually
<a href="{@docRoot}reference/android/support/test/runner/AndroidJUnitRunner.html">
{@code AndroidJUnitRunner}</a>.
        </p>
        <p>
            Your test results appear in <code>STDOUT</code>.
        </p>
    </li>
</ol>
<p>
    This operation starts an <code>adb</code> shell, then runs <code>am instrument</code>
    with the specified parameters. This particular form of the command will run all of the tests
    in your test package. You can control this behavior with flags that you pass to
    <code>am instrument</code>. These flags are described in the next section.
</p>
<h2 id="AMSyntax">Using the am instrument Command</h2>
<p>
    The general syntax of the <code>am instrument</code> command is:
</p>
<pre>
am instrument [flags] &lt;test_package&gt;/&lt;runner_class&gt;
</pre>
<p>
    The main input parameters to <code>am instrument</code> are described in the following table:
</p>
<table>
    <tr>
        <th>
            Parameter
        </th>
        <th>
            Value
        </th>
        <th>
            Description
        </th>
    </tr>
    <tr>
        <td>
            <code>&lt;test_package&gt;</code>
        </td>
        <td>
            The Android package name of the test package.
        </td>
        <td>
            The value of the <code>package</code> attribute of the <code>manifest</code>
            element in the test package's manifest file.
        </td>
    </tr>
    <tr>
        <td>
            <code>&lt;runner_class&gt;</code>
        </td>
        <td>
            The class name of the instrumented test runner you are using.
        </td>
        <td>
            This is usually
<a href="{@docRoot}reference/android/support/test/runner/AndroidJUnitRunner.html">
{@code AndroidJUnitRunner}</a>.
        </td>
    </tr>
</table>

<p>
    The flags for <code>am instrument</code> are described in the following table:
</p>
<table>
    <tr>
        <th>
            Flag
        </th>
        <th>
            Value
        </th>
        <th>
            Description
        </th>
    </tr>
    <tr>
        <td>
            <code>-w</code>
        </td>
        <td>
            (none)
        </td>
        <td>
            Forces <code>am instrument</code> to wait until the instrumentation terminates
            before terminating itself. The net effect is to keep the shell open until the tests
            have finished. This flag is not required, but if you do not use it, you will not
            see the results of your tests.
        </td>
    </tr>
    <tr>
        <td>
            <code>-r</code>
        </td>
        <td>
            (none)
        </td>
        <td>
            Outputs results in raw format. Use this flag when you want to collect
            performance measurements, so that they are not formatted as test results. This flag is
            designed for use with the flag <code>-e perf true</code> (documented in the section
            <a href="#AMOptionsSyntax">Instrument options</a>).
        </td>
    </tr>
    <tr>
        <td>
            <code>-e</code>
        </td>
        <td>
             &lt;test_options&gt;
        </td>
        <td>
            Provides testing options as key-value pairs. The
            <code>am instrument</code> tool passes these to the specified instrumentation class
            via its <code>onCreate()</code> method. You can specify multiple occurrences of
            <code>-e &lt;test_options&gt;</code>. The keys and values are described in the
            section <a href="#AMOptionsSyntax">am instrument options</a>. You can only use these
            key-value pairs with
<a href="{@docRoot}reference/android/support/test/runner/AndroidJUnitRunner.html">
{@code AndroidJUnitRunner}</a> or with {@link android.test.InstrumentationTestRunner} and its
subclasses. Using them with any other class has no effect.
        </td>
    </tr>
</table>

<h3 id="AMOptionsSyntax">am instrument options</h3>
<p>
    The <code>am instrument</code> tool passes testing options to
<a href="{@docRoot}reference/android/support/test/runner/AndroidJUnitRunner.html">
{@code AndroidJUnitRunner}</a> or {@link android.test.InstrumentationTestRunner} in the form of
key-value pairs, using the <code>-e</code> flag, with this syntax:
</p>
<pre>
-e &lt;key&gt; &lt;value&gt;
</pre>
<p>
    Some keys accept multiple values. You specify multiple values in a comma-separated list.
    For example, this invocation of
<a href="{@docRoot}reference/android/support/test/runner/AndroidJUnitRunner.html">
{@code AndroidJUnitRunner}</a> provides multiple values for the <code>package</code> key:
</p>
<pre>
$ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 \
&gt; com.android.test/android.support.test.runner.AndroidJUnitRunner
</pre>
<p>The following table lists the key-value pairs you can use with your test runner.</p>
<table>
    <tr>
        <th>Key</th>
        <th>Value</th>
        <th>Description</th>
    </tr>
    <tr>
        <td>
            <code>package</code>
        </td>
        <td>
            &lt;Java_package_name&gt;
        </td>
        <td>
            The fully-qualified <em>Java</em> package name for one of the packages in the test
            application. Any test case class that uses this package name is executed. Notice that
            this is not an <em>Android</em> package name; a test package has a single
            Android package name but may have several Java packages within it.
        </td>
    </tr>
    <tr>
        <td rowspan="2"><code>class</code></td>
        <td>&lt;class_name&gt;</td>
        <td>
            The fully-qualified Java class name for one of the test case classes. Only this test
            case class is executed.
        </td>
    </tr>
    <tr>
        <td>&lt;class_name&gt;<strong>#</strong>method name</td>
        <td>
            A fully-qualified test case class name, and one of its methods. Only this method is
            executed. Note the hash mark (#) between the class name and the method name.
        </td>
    </tr>
    <tr>
        <td><code>func</code></td>
        <td><code>true</code></td>
        <td>
            Runs all test classes that extend {@link android.test.InstrumentationTestCase}.
        </td>
    </tr>
    <tr>
        <td><code>unit</code></td>
        <td><code>true</code></td>
        <td>
            Runs all test classes that do <em>not</em> extend either
            {@link android.test.InstrumentationTestCase} or
            {@link android.test.PerformanceTestCase}.
        </td>
    </tr>
    <tr>
        <td><code>size</code></td>
        <td>
            [<code>small</code> | <code>medium</code> | <code>large</code>]
        </td>
        <td>
            Runs a test method annotated by size. The annotations are <code>&#64;SmallTest</code>,
            <code>&#64;MediumTest</code>, and <code>&#64;LargeTest</code>.
        </td>
    </tr>
    <tr>
        <td><code>perf</code></td>
        <td><code>true</code></td>
        <td>
            Runs all test classes that implement {@link android.test.PerformanceTestCase}.
            When you use this option, also specify the <code>-r</code> flag for
            <code>am instrument</code>, so that the output is kept in raw format and not
            re-formatted as test results.
        </td>
    </tr>
    <tr>
        <td><code>debug</code></td>
        <td><code>true</code></td>
        <td>
            Runs tests in debug mode.
        </td>
    </tr>
    <tr>
        <td><code>log</code></td>
        <td><code>true</code></td>
        <td>
            Loads and logs all specified tests, but does not run them. The test
            information appears in <code>STDOUT</code>. Use this to verify combinations of other
            filters and test specifications.
        </td>
    </tr>
    <tr>
        <td><code>emma</code></td>
        <td><code>true</code></td>
        <td>
            Runs an EMMA code coverage analysis and writes the output to
            <code>/data/&lt;app_package&gt;/coverage.ec</code> on the device. To override the
            file location, use the <code>coverageFile</code> key that is described in the
            following entry.
            <p class="note">
                <strong>Note:</strong> This option requires an EMMA-instrumented build of the test
                application, which you can generate with the <code>coverage</code> target.
            </p>
        </td>
    </tr>
    <tr>
        <td><code>coverageFile</code></td>
        <td><code>&lt;filename&gt;</code></td>
        <td>
            Overrides the default location of the EMMA coverage file on the device. Specify this
            value as a path and filename in UNIX format. The default filename is described in the
            entry for the <code>emma</code> key.
        </td>
    </tr>
</table>
<strong><code>-e</code> Flag Usage Notes</strong>
<ul>
    <li>
        <code>am instrument</code> invokes
        {@link android.test.InstrumentationTestRunner#onCreate(Bundle)}
        with a {@link android.os.Bundle} containing the key-value pairs.
    </li>
    <li>
        The <code>package</code> key takes precedence over the <code>class</code> key. If you
        specifiy a package, and then separately specify a class within that package, Android
        will run all the tests in the package and ignore the <code>class</code> key.
    </li>
    <li>
        The <code>func</code> key and <code>unit</code> key are mutually exclusive.
    </li>
</ul>
<h3 id="RunTestExamples">Usage examples</h3>
<p>
The following sections provide examples of using <code>am instrument</code> to run tests.
They are based on the following structure:</p>
<ul>
    <li>
        The test package has the Android package name <code>com.android.demo.app.tests</code>
    </li>
    <li>
        Two instrumented test classes:
        <ul>
            <li>{@code Foo1} which contains the test method {@code bar1}, and</li>
            <li>{@code Foo2} which contains test methods {@code bar2} and {@code bar3}</li>
        </ul>
    </li>
    <li>
        The test runner is
<a href="{@docRoot}reference/android/support/test/runner/AndroidJUnitRunner.html">
{@code AndroidJUnitRunner}</a>.
    </li>
</ul>
<h4>Running the entire test package</h4>
<p>
    To run all of the test classes in the test package, enter:
</p>
<pre>
$ adb shell am instrument -w com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner
</pre>
<h4>Running all tests in a test case class</h4>
<p>
    To run all of the tests in the class <code>UnitTests</code>, enter:
</p>
<pre>
$ adb shell am instrument -w  \
&gt; -e class com.android.demo.app.tests.Foo \
&gt; com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner
</pre>
<p>
  <code>am instrument</code> gets the value of the <code>-e</code> flag, detects the
  <code>class</code> keyword, and runs all the methods in the <code>UnitTests</code> class.
</p>
<h4>Selecting a subset of tests</h4>
<p>
    To run all of the tests in <code>Foo1</code>, and the <code>bar3</code> method in
    <code>Foo2</code>, enter:
</p>
<pre>
$ adb shell am instrument -w \
&gt; -e class com.android.demo.app.tests.Foo1,com.android.demo.app.tests.Foo2#bar3 \
&gt; com.android.demo.app.tests/android.support.test.runner.AndroidJUnitRunner
</pre>