25 Commits

Author SHA1 Message Date
Artur Satayev
44036a7689 Blacklist @TestApis by default.
If the signature has @UnsupportedAppUsage or @SystemApi annotation as well, those take precedence and the API would either be whitelisted or greylisted accordingly. All other, "pure" @TestApi signatures would be blacklisted and only allowed access in instrumented processes.

Test: manual
Bug: 133832325
Change-Id: I4684929caed8be7c42c91fed33ddd2a3b67ae19b
2019-11-15 16:54:14 +00:00
David Brazdil
a4e64da2e4 hiddenapi: Support greylist-max-q
Test: set maxTargetSdk=Q on an API, compiles, check hiddenapi-flags.csv
Change-Id: I990972d652d4f4bc578592a9b3d9954f93b1ab9f
2019-05-01 11:57:05 +01:00
Andrei Onea
a6e09b4273 Automatically greylist code in 3P packages
generate_hidden_api_lists now receives a file containing package names
which need to be greylisted (although it could be made to work with any
api list required).
Also took the opportunity to clean up the tests to reflect the more
strict code.

Bug: 129387816
Test: m appcompat
Test: frameworks/base/tools/hiddenapi/generate_hiddenapi_lists_test.py
Change-Id: I619f8581d166aa48eda572bc0053d8739d6420eb
2019-04-01 15:32:36 +01:00
Andrei Onea
80a5660238 Add extra hidden api related tags
@SystemApi and @TestApi entries in the whitelist can now be
differentiated from the rest of the apis. @TestApi methods
are implicitly greylisted.

Test: m test-art-host-gtest-hiddenapi_test
Change-Id: Id739f04550842f7b3160685e1635ba20efb223cc
2019-03-07 10:40:52 +00:00
Paul Duffin
997fa46197 Switch android.test and junit classes to use UnsupportedAppUsage
The UnsupportedAppUsage annotations could not be added directly to the
java files in src/ as they have to be built against the current api
which does not include the annotation. Instead this uses the same
technique as used for libcore/ojluni files and adds the annotations to
stub files (in hiddenapi/src) which are built as part of the
android.test.base-hiddenapi target. That target is added to a special
whitelist in build/soong/java/config/config.go which causes the
hiddenapi information to be extracted from the target.

Also, updates the preupload check to prevent anymore entries being
added to the config/hiddenapi-greylist.txt for android.test or junit
classes.

Bug: 73711752
Test: m cts-hiddenapi_flags-csv and check that it contained the
        correct entries even though they had been removed from
	config/hiddenapi-greylist.txt
Change-Id: Ifaf15d2751f54cb03f8402b866a0ee4da7acc4d2
2019-03-01 10:39:16 +00:00
David Brazdil
439d3496ed hiddenapi: Support 'core-platform-api' flag
Libcore class members annotated with @CorePlatformApi now generate
a new hiddenapi flag. This is the first of "domain API" flags which
can be used in conjunction with API list flags. Therefore modify
the 'generate_hiddenapi_lists.py' logic to treat them differently.
Specifically, the script marks otherwise unassigned class members
blacklisted. A class member with 'core-platform-api' may still not
be assigned an API list and should be blacklisted.

Bug: 119068555
Test: m appcompat
Change-Id: I2b67e7a619677e853c87bc2da934410458ce4d14
2019-01-19 15:57:36 +00:00
David Brazdil
17d16e84da hiddenapi: Expect public/private API as CSV
Refactor of `hiddenapi` changed the output format of public/private API
lists to a single CSV file. Change API list generation accordingly.

In order to avoid special-casing this CSV file, it is treated the same
as the CSV files produced by `class2greylist`. The merging rules are
relaxed so that signatures in CSV files are not checked against
a pre-initialized set of all signatures (previously generated from the
public/private API files). This should not lead to build errors as the
CSV files are always auto-generated, and a missing/extra signature will
be caught by `hiddenapi`.

API lists in frameworks/base/config are processed later and checked
that they are a subset of the signatures in CSV.

Bug: 119068555
Test: compiles, hiddenapi-flags.csv unchanged
Change-Id: I33f2cbaa15f2d423a75e6ca64abe1c5b0c40c86f
2019-01-19 15:57:36 +00:00
Paul Duffin
352956b07e Switch last c.a.i18n.phonenumbers hiddenapi entries to annotations
Previous changes could not remove these entries as they are implicit
methods, i.e. are not present in the source, and so could not be
annotated. That is no longer true and so these entries can now be
removed.

This was tested by making and then manually checking that the generated
out/target/common/obj/PACKAGING/hiddenapi-light-greylist.txt was the
same (after sorting) before and after this change.

Bug: 117818301
Bug: 119861512
Test: as above
Change-Id: Ic33c693f50cac011332c5ba5a5c0e2b6562e6ef8
2018-12-10 16:05:54 +00:00
David Brazdil
5cd148fc7a hiddenapi: Add constants for 'greylist-max-p'
New category of hidden API has been created. Update the script
generate_hiddenapi_lists.py with the new flag name.

Test: m, phone boots
Change-Id: I79e5478678880939e20e500cb8dad9b2a56fc84f
2018-11-29 15:43:02 +00:00
David Brazdil
89bf0f28bf Turn hidden API lists into a single CSV
Maintaining multiple text files has become too cumbersome as adding
each new category of API requires changes across many projects.
This patch changes generate_hiddenapi_lists.py to produce a single
CSV file in the format:

  <api_signature>,<flag1>,...,<flagN>

It can accept legacy API list files as input (for existing
frameworks/base/config/hiddenapi-*.txt files) as well as per-package
CSVs produced by class2greylist.

Test: m, check lists have not changed
Test: phone boots
Test: tools/hiddenapi/generate_hiddenapi_lists_test.py
Change-Id: Iebcef426ec93ea1d72b662bbff91d4e068fa0a70
2018-11-29 15:43:01 +00:00
Paul Duffin
00537c15e8 Prevent hiddenapi entries being added for libcore related projects
The libcore related projects (see below) have been (mostly) switched
over to use UnsupportedAppUsage annotations, This change will prevent
entries for those projects being added to a config/hiddenapi-* file.

* libcore
* external/bouncycastle
* external/conscrypt
* external/icu
* external/okhttp
* external/libphonenumber - still has a couple of entries in
      config/hiddenapi-light-greylist.txt due to limitations in
      UnsupportedAppUsage/class2greylist.

Tested by attempting to upload the file with entries for libcore
projects and without those entries and checking that the behavior
is expected.

Test: see above
Bug: 117818301
Change-Id: I67a30b307e12e842b28cfb2160fab0029868fa06
2018-11-29 09:41:56 +00:00
Mathew Inwood
5037a7eb93 Merge "Merge hidden API metadata into a single CSV file." 2018-10-17 09:20:04 +00:00
Paul Duffin
c78bea41c6 Fix issue with sort_api.sh
Should use -ne not -neq.

Test: run script manually
Bug: 115609023
Change-Id: I337ed43be1f9250e6c2b2c88d97c68a5c9e8941c
2018-10-16 14:30:01 +01:00
Mathew Inwood
ea14c0c36f Merge hidden API metadata into a single CSV file.
Add a python script to merge the CSV files, dealing with the fact that the
source columns may differ due to the annotation properties present in the
source.

Add this new file and the existing greylist.txt files as dependencies of
the 'dist' target, so that they appear as artifacts on the build server.
This allows them to be processed later by reading them from the build
server.

Test: m out/target/common/obj/PACKAGING/hiddenapi-greylist.csv
Bug: 117314178
Change-Id: Ib9e5da90e4c32333a3f4f02ee7f159f77086ae5e
2018-10-16 12:12:27 +01:00
Mathew Inwood
50dea42729 Add new "max-sdk-p blacklist".
This will contain greylist entries accessible by apps with a
targetSdkVersion of less than Q. Currently empty, will be populated in
follow up CLs.

For now, these APIs are just merged into the light greylist at build time,
pending support in the runtime for implementing the correct runtime
behavior.

Also fix a bug in sort_api.sh so it deals with empty API lists
correctly.

Bug: 115609023
Test: m
Change-Id: I213874062f393f96d120648a934ae5ad7aba93af
2018-10-15 16:38:12 +01:00
Paul Duffin
e9622a3d3a Add serialization methods to whitelist not light greylist
Checked the contents of the hiddenapi lists before and after the change
and the methods were definitely moved from light greylist to the
whitelist.

Flashed the device and checked the log and did not see anything out of
the ordinary.

Test: frameworks/base/tools/hiddenapi/generate_hiddenapi_lists_test.py
Change-Id: I9b4b2426251e99495f65ae02a3c2c32ce6966625
2018-09-21 11:05:06 +01:00
Mathew Inwood
6be0fdde4a Don't fail if there are no comments.
Grep exits with non-zero if it find no matches. But it doesn't matter in
this case, so just carry on.

Test: frameworks/base/tools/hiddenapi/sort_api.sh \
Test:    frameworks/base/config/hiddenapi-light-greylist.txt
Change-Id: I7c219ee617f7dfa3cff5232ab979cc4ba38ebbfd
2018-09-14 15:29:35 +01:00
David Brazdil
4a55eebf26 Revert "Check in P dark greylist, use it for hidden API list generation"
This reverts commit 3cc74c71ef907a76547424aea6597f7b2907e9fb.
We saw app crashes due stricter classification of non-SDK APIs.

Bug: 115284625
Bug: 113881436
Test: phone boots

Change-Id: I689d0fbf66fabeffa5032e13f2f1f314c50b6cc5
2018-09-11 11:43:07 +01:00
David Brazdil
ae88d4e21e Ignore comment lines in hidden api lists
It is sometimes useful to add comments at the top of a source file. This
patch changes hidden API list generation to ignore lines beginning with
a hash.

Note that due to the sorting constraints on hidden API lists, comments
can be only at the top of the files.

Test: m appcompat
Change-Id: I1bc6fd44d1b1f10a5adc45093d7f7ed5a0c5a54f
2018-09-06 15:10:28 +01:00
David Brazdil
3cc74c71ef Check in P dark greylist, use it for hidden API list generation
We want to preserve the same semantics as P wrt dark greylist. This will
eventually become the "blacklist for target SDK >= P".

Bug: 113881436
Test: phone boots
Change-Id: I3bb72102e1d054eb0f2cae089ab633a0bc2bb276
2018-09-06 11:10:51 +01:00
David Brazdil
8503b9045b Rewrite hidden API list generation in Python
Generating hidden API lists has grown in complexity and the original
Makefile rule has become a bottleneck for build times. Rewrite the
logic in Python.

Bug: 113278235
Bug: 73736106
Test: frameworks/base/tools/hiddenapi/generate_hiddenapi_lists_test.py
Change-Id: I63f03133d70260d06c55f482b844a4980dc6f734
2018-08-30 16:19:26 +01:00
Mathew Inwood
f364aedecb Remove class2greylist from frameworks/base.
It's moving to a new home under art/tools, since it is needed by
branches which don't include frameworks/base.

Bug: 110868826
Test: Build master-art branch
Change-Id: Ice8324db52ae1e7e2ac6734e45874a66de935c9a
2018-07-26 15:57:05 +01:00
Mathew Inwood
738a23fbeb Update annotation name in class2greylist.
Annotation was added with name @UnsupportedAppUsage after Android API
council discussion.

Bug: 110868826
Test: m class2greylist
Change-Id: I63ad61e97c5e8fdebfcead9e4ee5b060a60f97c7
2018-07-25 09:45:04 +01:00
Mathew Inwood
6395690ec9 Add new "class2greylist" tool.
This extracts signatures of methods that have the @UsedByApps annotation
for generating the greylist. It will be integrated into the build to
replace many members on greylist.txt.

Test: $ atest class2greylisttest
Bug: 110868826
Change-Id: Ifaf5859b60076c051de6be5a912ef70734330ce7
2018-07-23 14:39:19 +01:00
David Brazdil
2200a1b83b Add presubmit hook for hidden API lists.
Check that they're sorted as expected and contain no duplicates.
The sort order now uses:
$ LC_COLLATE=C sort -f

So that non-alphanumeric characters are not ignored, giving a more
intuitive sort order. the '-f' means ignore case.

Also sort the existing lists accordingly.

Test: repo upload
Bug: 64382372
Merged-In: I52b884da33a9a46455df6747a215683d9d3c3218
Change-Id: Icc124fd8ceb3f001a9c11bbf40e0a111910c6b39
(cherry picked from commit e5dc64d4281dd9f38c2b585202e26ae4278fcfc5)
2018-05-08 10:29:50 +01:00