67 Commits

Author SHA1 Message Date
Ryan Mitchell
db21f09a8e Revert^2 "libandroidfw hardening for IncFs"
55ef6167a2c235bd88c7216238b2001b46795b79

Change-Id: I02d4890d181655dfd0a14c188468db512559d27b
2020-11-17 23:01:35 +00:00
Ryan Mitchell
55ef6167a2 Revert "libandroidfw hardening for IncFs"
Revert "Move map_ptr to incfs namspace"

Revert submission 12787270

Reason for revert: b/173250495
Reverted Changes:
I5cd1bc8a2:libandroidfw hardening for IncFs
Ice5dbcfb2:Move map_ptr to incfs namspace
I29ccdc8ed:Do not cache bag parent stack until requested
I1e9e9acaa:Cache resolved theme values

Change-Id: Ib90ef68339710086df41e9abe0833a542d03a74f
2020-11-13 23:55:20 +00:00
Ryan Mitchell
c75c2e0922 libandroidfw hardening for IncFs
Migrate libandroifw to using incfs::util::map_ptr to prevent processes
from crashing when parsing the resources.arsc, parsing compiled xml,
files, and retrieving resource values.

This change propagates incremental failures to the JNI level where they
are raised as ResourcesNotFoundException.

Performance of ResourcesPerfWorkloads without change (time in
nanoseconds):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.883s)
    youtube_ns_median: 93812805
    youtube_ns_standardDeviation: 4387062
    youtube_ns_mean: 94455597
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.265s)
    maps_ns_standardDeviation: 2997543
    maps_ns_mean: 83480371
    maps_ns_median: 82210941
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.963s)
    gmail_ns_median: 266141091
    gmail_ns_standardDeviation: 3492043
    gmail_ns_mean: 267472765

With change and verification forcibly enabled for all apks
(including the framework-res.apk):
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.646s)
    youtube_ns_median: 101999396
    youtube_ns_standardDeviation: 4625782
    youtube_ns_mean: 102631770
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (11.286s)
    maps_ns_standardDeviation: 2692088
    maps_ns_mean: 91326538
    maps_ns_median: 90519884
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.694s)
    gmail_ns_median: 290284442
    gmail_ns_standardDeviation: 5764632
    gmail_ns_mean: 291660464

With change and verification disabled:
[1/3] com.android.resources.perf.PerfTest#youtube: PASSED (11.748s)
    youtube_ns_median: 95490747
    youtube_ns_standardDeviation: 7282249
    youtube_ns_mean: 98442515
[2/3] com.android.resources.perf.PerfTest#maps: PASSED (10.862s)
    maps_ns_standardDeviation: 4484213
    maps_ns_mean: 87912988
    maps_ns_median: 86325549
[3/3] com.android.resources.perf.PerfTest#gmail: PASSED (24.034s)
    gmail_ns_median: 282175838
    gmail_ns_standardDeviation: 6560876
    gmail_ns_mean: 282869146

These tests were done on a Pixel 3 and with cpu settings configured by
libs/hwui/tests/scripts/prep_generic.sh:

 Locked CPUs 4,5,6,7 to 1459200 / 2803200 KHz
 Disabled CPUs 0,1,2,3

Bug: 160635104
Bug: 169423204
Test: boot device && atest ResourcesPerfWorkloads

Change-Id: I5cd1bc8a2257bffaba6ca4a1c96f4e6640106866
2020-11-12 08:13:44 -08:00
Ryan Mitchell
19e1a5b4c8 Ignore resources from unknown types
When loading binary APKs into aapt2 for dumping purposes, aapt2 allows
the presence of custom resource types in the resources.arsc. Since
apapt2 may not be able to correctly interpret the data of custom types,
ignore resources from custom types completely when constructing the
ResourceTable data structure for the APK.

Bug: 168247188
Bug: 36051266
Test: run aapt2 dump bading/resources on APKs from both bugs
Change-Id: Ia3c0b4e07457eb8634b5a256ef77270e99b8bd72
2020-09-14 12:10:33 -07:00
Zoran Jovanovic
9336d9e13b OMS: Add config_signature policy handling
Alongside SIGNATURE and ACTOR_SIGNATURE policies, add CONFIG_SIGNATURE
policy to overlayable that overlay fulfills if it is signed with the
same certificate as the reference package whose package name is
declared in 'config-signature' tag of SystemConfig and is vetted by
OMS that it's a system pre-installed package.

BUG: 158726924
TEST: regular aapt2, idmap2, OMS tests

Change-Id: I645ee72271496008742886274be0d63a2985201b
2020-07-28 18:30:53 +00:00
Clark DuVall
8f51d6b876 Reorder styleable attributes in shared libraries
This is a similar fix to
https://googleplex-android-review.git.corp.google.com/c/platform/frameworks/base/+/10109386
but deals with styleable attributes instead of style items.

Bug: 147674078
Test: JavaClassGeneratorTest.SortsDynamicAttributesAfterFrameworkAttributes
Change-Id: Ida6572cf07e2b5987e9d8941cf169a37c43578c4
2020-04-22 13:19:28 -07:00
Donald Chai
ab7e0d5a51 Include the name of the resource in an error message.
Bug: 153393440
Tested: N/A
Change-Id: I6979c683978af2670b1823ff8380175673067630
2020-04-15 22:03:56 -07:00
Mohamed Heikal
e0388de6be Enable --collapse-resource-names on bundles
This cl enables aapt2 optimize and convert to handle collapsed resource
names optimization.

Test: make aapt2_test

Change-Id: I160d7e5bbd94580b52c00b648918e47beb4674f1
2020-03-31 14:58:01 -04:00
Winson
f56ade365a Actor signature overlayable policy
There are cases where an app can ship overlays for itself,
but the "signature" policy as described would open up
a vulnerability by allowing the system actor to create
and sign any arbitrary overlay that will apply to the target.

To prevent this, redefine "signature" as target package only,
and introduce "actor" for checking against the actor signature.
Any app that wishes to use both can include both policies.

Bug: 130563563

Test: m aapt2_tests idmapt2_tests and run from host test output
Test: atest libandroidfw_tests

Change-Id: I1c583a5b37f4abbeb18fc6a35c502377d8977a41
2020-02-26 15:59:44 -08:00
Winson
62ac8b56a9 Refactor overlayable policy
To make it easier to add the actor policy in a follow up CL,
move most of the policy handling to a central location.

The strings and transformation between strings and flags is
now handled in libidmap2policies, with libandroidfw
containing the single source of policy flags.

This also extracts all the test resource IDs into an R.h
so they can be swapped without having to edit a dozen files
each time.

Bug: 130563563

Test: m aapt2_tests idmapt2_tests and run from host test output
Test: atest libandroidfw_tests

Change-Id: Ie533c9cebf938215df7586f00c38763ae467e606
2020-02-26 15:59:43 -08:00
Clark DuVall
18b15bb3a6 Reorder style items in shared libraries
AssetManager2.cpp expects style attribute IDs to be in sorted order when
applying a style (see AssetManager2::GetBag). Shared libraries have a
package ID of 0x00, which will mean any attribute defined in a shared
library will be put before all other attributes. Once the attribute ID
is looked up in the dynamic ref table, the package ID is no longer 0x00,
which means this ID is no longer in sorted order. This messes up the
logic in AssetManager2::GetBag, and results in some style attributes
getting dropped from shared libraries.

This change modifies how aapt2 sorts the style entries, sorting entries
with dynamic IDs after entries with the android framework ID. This means
the entries will still be in sorted order when the IDs are looked up.

Bug: 147674078
Test: TableFlattenerTest.FlattenSharedLibraryWithStyle
Change-Id: Ic4f4004b6d9cecde9325dcdb37f71138857f8236
2020-01-16 12:37:36 -08:00
Clark DuVall
e9fedbe223 Fix serializing dynamic references to proto
The is_dynamic bit on references was getting lost when converting to
proto and back. This was preventing bundles from being used as shared
libraries, since layout inflation would fail when it hit a dynamic
reference.

Bug: 146491000
Test: Build a shared library with a layout that has a dynamic reference,
      and attempt to inflate that layout.

Change-Id: Ia0e615670d2ac52f9266e3eec8813af7687d3323
2020-01-13 09:55:53 -08:00
Donald Chai
b99e50922b Fix computation of 'entry_length' in AAPT2 container format
When the protobuf header (or data payload) size are a multiple of 4,
"entry_length" (called "aligned size" in the code) would have counted bytes that
are never actually written.

Bug: 139418052
Change-Id: Ia688a82a67f3807f7feb0be03670bf2827b1d6a1
2019-10-21 15:14:05 -07:00
Ryan Mitchell
8a891d86ab Allow for RRO internal referencing
This change allows RROs to reference their own internal resources as
expected.

Overlays are loaded as shared libraries so they can have their own
resource id space that does not conflict with the resource id space of
the target or other overlays.

References to overlay resources that override target resources now
appear as references to the target resources.

Overlay values that are inlined into the xml file specified using
android:overlayResources are now able to be used at runtime.

See go/rro-references for more information.

Bug: 135943783
Test: idmap2_tests
Test: libandroidfw_tests
Change-Id: Ie349c56d7fd3f7d94b7d595ed6d01dc6b59b6178
2019-10-17 13:09:33 -07:00
Brian Chang
dcef831e61 Improve how we exempt resources from getting their names collapsed.
Removed the --whitelist-path flag, which is poorly named and we can
already specify these resources with --resources-config-path.

Renamed TableFlattenerOptions.whitelisted_resources to keep_resources.
It holds ResourceName instead of std::string. This lets us include type
when specifying what gets exempted, for correctness.

Bug: 111115201
Test: make aapt2_tests
Change-Id: Ifa5df924b5e2265c32cdcf8ca7dfa4a3992a0468
2019-09-16 17:29:46 -07:00
Ryan Mitchell
ef9e688393 Add --exclude-sources to AAPT2 link
Information about where resources are defined can make having
reproducible builds more difficult, makes the generates resources.pb
large, and can exposes details about the machines that compiled the
resources.

The --exclude-sources flags can only be used when building a proto APK
and prevents source information from being included in the generated
resources protobuf.

Bug: 134929532
Test: checked debug string with and without the string
Change-Id: Ia345f067fe781ea82a4bcad37eb55576c72c44d7
2019-08-14 19:28:32 +00:00
Ryan Mitchell
c1676807f4 Retain parsed attribute type
If the value of an attribute enum is defined as a hexadecimal integer,
flatten uses of the attribute as with the
android::Res_value::TYPE_INT_HEX type.

This change adds a "type" field to pb::Attribute::Symbol, which if left
unset, will have a default value of android::Res_value::TYPE_INT_DEC
when deserialized by aapt2.

Bug: 124474141
Test: aapt2_tests and manual compilation of files and inspection using
`aapt2 dump chunks`
Change-Id: Ibf12394284fdbe3a8047f7ecf4fe68517dfc3abb
2019-06-06 14:19:26 +00:00
Winson
5e7370d1d0 AAPT2: Fix --output-to-dir
Without a FinishEntry call, the file_ wasn't being released, which
prevents another StartEntry from succeeding.

Bug: 68033366

Test: manual aapt2 link --output-to-dir -o outputDir
Test: Archive_test.cpp

Change-Id: I3b66266327163307a2e4c7f891f6ee76baf7fc10
2019-05-02 10:30:04 -07:00
Ryan Mitchell
743266643a Merge "Add build number to aapt2 version" into qt-dev 2019-04-18 19:04:40 +00:00
Ryan Mitchell
939df096be Add odm and oem policies
This change adds parsing, encoding, and validating of odm and oem
overlayable policies to aapt2, libandroidfw, and idmap2.

Bug: 121033532
Test: aapt2_tests, idmap2_tests
Change-Id: Ifc0d4b6c9f9c37e06b2988abade69dbb277c50c2
2019-04-17 16:07:57 -07:00
Ryan Mitchell
34039b26f5 Add build number to aapt2 version
This changes adds the build id to the version printed by "aapt2
version". This change also adds a field to the ResourceTable proto that
specifies the fingerprints of tools used to build the table.

Bug: 123663089
Test: manual
Change-Id: Ifaf33c1e506b68e9f1d921fdbeddf36485e65790
2019-04-17 10:59:38 -07:00
TreeHugger Robot
b496d6150c Merge "Add --trace_folder to aapt2" 2019-03-07 16:40:03 +00:00
Ryan Mitchell
1d358ff5bb Fix aapt2 whitespace diffs from aapt(1)
CDATA blocks were being processed differently in aapt2 so this change
fixes aapt2 to not treat cdata blocks differently and still trime
whitespace.

Also, aapt did not process escapes when compiling xml files. This
change removes over-processing of xml text nodes.

All test strings are what aapt(1) would output.

Test: aapt2_tests
Bug: 124470332
Change-Id: I90ee0c1e5e9208f8a5c60cee93e3ba02712c9b2c
2019-03-06 15:06:49 -08:00
Fabien Sanglard
2d34e76dac Add --trace_folder to aapt2
Add a tracing API and instrument key functions in order to profile
aapt2 bottleneck. The API allows to generate systrace fragment files.

Impact on performance is neglibible with each Trace requiring less
than 1us and the final Flush operation at the end of a command
requiring around 40us.

Bug: None
Test: None
Change-Id: I51b564d3694e9384679f43b878b32295527dddf6
2019-03-05 15:09:27 -08:00
TreeHugger Robot
03f311cb75 Merge "Revert "Fix loaded apk string pool order"" 2019-02-16 00:46:04 +00:00
TreeHugger Robot
ed6c556145 Merge "Do not derive source from binary APK" 2019-02-15 22:57:17 +00:00
Ryan Mitchell
90b7a08aaf Revert "Fix loaded apk string pool order"
This reverts commit 4e9a922ede24f7f7bfe793321f7328623ee2a061.

Reason for revert: <b/122518436>

Change-Id: I3650b2c6c9bdfa69a3034f9ca49e95a9698c3cdd
2019-02-15 14:37:45 -08:00
Winson
b2d7f5343d Signature policy for overlayable items
Add encoding/decoding of new policy for overlays. Signature enforces
that an overlay package is signed with the same key as the actor of
the target resource, so that an overlay can be installed by the user
as a normal app but restricted to those built by the author of the
actor (which can be the same as the target).

This also enforces that a valid policy is specified.

This doesn't implement the actors nor the signature check.

Bug: 119402606

Test: ResourceParserTest ParseOverlayablePolicy
Test: ProtoSerializerTest SerializeAndDeserializeOverlayable
Test: aapt2_tests

Change-Id: I8495ad790c2ebd51759bc6eba81149680c209475
2019-02-14 10:07:13 -08:00
Ryan Mitchell
b5b162bfe6 Do not derive source from binary APK
When loading the binary APK, do not set the paths of Source structs to
the path of the binary APK. This causes the merge_zips step of the build
system to fail.

Bug: 124082573
Test: vendor/google/build/build_mainline_modules.sh
Change-Id: I84b8d43c5b6a5835e7ff6bd579c4eb744e02c20f
2019-02-07 20:28:26 -08:00
Ryan Mitchell
02d9c1e931 Remove RRO policy product_services
Since Q will no longer have a product_services partition, remove instances of
the product_services policy across aapt2, androidfw, and idmap2.

Bug:122745343
Test: aapt2_tests and libandroidfw_tests
Change-Id: I97c223a0bf5a2eab95811e5f738b44af6335e0ea
2019-01-16 11:47:33 -08:00
Ryan Mitchell
cf9dd94323 Merge "Add encoding of name and actor overlayable fields" 2019-01-15 19:34:30 +00:00
Mohamed Heikal
149b0d29c6 Merge "Resource Path Obfuscation" 2019-01-15 15:34:06 +00:00
Ryan Mitchell
ef5673a4c4 Add encoding of name and actor overlayable fields
Encode the actor and name attributes of <overlayable> elements in the
overlayable chunks. Two chunks cannot have the same name.

Bug: 110869880
Bug: 119390855
Test: aapt2_tests and libandroidfw_tests
Change-Id: I0ed95efef640b3927046b6155d1840f0d96030dd
2019-01-11 14:45:09 -08:00
Mohamed Heikal
c769403631 Resource Path Obfuscation
This CL allows aapt2 to obfuscate resource paths within the output apk
and move resources to shorter obfuscated paths. This reduces apk size
when there is a large number of resources since the path metadata exists
in 4 places in the apk.

This CL adds two arguments to aapt2, one to enable resource path
obfuscation and one to point to a path to output the path map to (for
later debugging).

Test: make aapt2_tests
Bug: b/75965637

Change-Id: I9cacafe1d17800d673566b2d61b0b88f3fb8d60c
2018-12-20 18:19:25 -05:00
Ryan Mitchell
54237ffed4 Add actor and name parsing for overlayable
Add parsing of two overlayable attributes:
 name : The unnique identifying name of the overlayable set of resources
 actor: The component responsible for enabling and disabling overlays
        targeting the specified set of resources

Bug: 110869880
Bug: 119390855
Test: m -j aapt2_tests
Change-Id: Id42463e2b92b69034fb39cd29bc8606affb61ba7
2018-12-13 16:05:10 -08:00
Ryan Mitchell
1bb1fe068a Refactor policy parsing
This change removes the ability for an overlayable resource to be
defined in multiple policy blocks within the same overlayable. This
change also changes aapt2 to use a bit mask to keep track of the parsed
policies.

Bug: 110869880
Bug: 120298168
Test: aapt2_tests
Change-Id: Ie26cd913f94a16c0b312f222bccfa48f62feceaa
2018-12-11 13:48:45 -08:00
Ryan Mitchell
75e20dda2b Encoding of <overlayable> and <policy>
This change defines two new chunks for encoding overlayable information.
RES_TABLE_OVERLAYABLE_POLICY_TYPE contains flags that represent
restrictions enforced on overlays that try to overlay a specific set of
resource ids. The chunk header is followed by ResTable_ref for each id
that belongs to the policy type. A policy chunk will be created for
every unique combination of policies that are defined in overlayable
declarations.

RES_TABLE_OVERLAYABLE_TYPE holds policy blocks. Since <overlayable>
does not currently have any attributes, only one overlayable block is
encoded in an APK.

This change also removes the SPEC_OVERLAYABLE flag because the runtime
does not use the flag, and the overlayable chunk encoding renders it
obsolete.

Bug: 110869880
Bug: 117545186
Test: libandroidfw_tests and aapt2_tests
Change-Id: I45ae9bf4176699f14c85e2b7a2e8560185d8a0b8
2018-12-04 16:45:26 -08:00
Ryan Mitchell
19954a261c Merge "Do not serialize empty text in manifest proto" 2018-11-19 22:13:35 +00:00
Ryan Mitchell
467b689a4c Do not serialize empty text in manifest proto
When linking an APK in the proto format, the manifest is currently
serializes text nodes that only contain whitespace:

child: {
  text: "\n        "
  source: {
    line_number  : 0x0000000f
    column_number: 0x0000002f
  }
}

These whitespace bloat the proto size unnecessarily. Do not write these
text nodes for proto apks.

Bug: 118800653
Test: make aapt2_tests
Change-Id: Icfaaf88976f81450bbf51610a316b336deeae60c
2018-11-16 15:58:06 -08:00
Ryan Mitchell
4e9a922ede Fix loaded apk string pool order
Loading in an APk changed the ordering of strings in the string pool.
When loading an apk, assign the strings to the same index as they
are in the ResStringPool.

Bug: 118831219
Test: "aapt2 dump strings left.apk" prints in the correct order,
"aapt2 convert left.apk --output-format binary -o left_binary.apk" has
entries in the correct order, and aapt2_tests

Change-Id: I00014c02195f39c1152a110e90083d9b14e9216e
2018-11-15 11:37:01 -08:00
Ryan Mitchell
e4e989ccba RRO: Added partition policies for overlays
<overlayable> tags can now have policy elements that indicate which
partition the overlay apk must reside on in order to be allowed to
overlay a resource. This change only adds parsing of <policy> and
encoding of policy in the proto ResourceTable. A later change will add
the encoding of policy and overlayable in the binary APK.

<overlayable>
  <policy type="system|vendor|product|product_services|public" >
    <item type="string" name="oof" />
  </policy>
</overlayable>

Bug: 110869880
Test: make aapt2_tests
Change-Id: I8d4ed7b0e01f981149c6e3190af1681073b79b03
2018-11-06 00:10:26 +00:00
Mårten Kongstad
24c9aa6541 libandroidfw: move ConfigDescription from aapt2 to libandroidfw
This is to allow idmap2 to access ConfigDescription.

Test: libandroidfw_tests
Test: aapt2_tests
Change-Id: I54210bbbd8dad5903cb7100807df977efa394ad5
2018-10-08 06:50:22 -07:00
Ryan Mitchell
0ce89734d1 Fix failing Windows tests
Some tests were not written to run on Windows correctly. Compile also
has a bug that caused using the --zip flag to fail on Windows.
iswspace does count the non breaking space as Whiespace on Windows but
not on Unix based systems

Bug: 117156986
Change-Id: I999375162bdfdf86fb25992ee88e2962ab90b577
Test: aapt2_tests and wine aapt2_tests.exe
2018-10-03 11:35:37 -07:00
Donald Chai
e0936a1fda Merge "Include values in error message." 2018-08-20 17:11:01 +00:00
Donald Chai
9f1911181d Include values in error message.
Bug: 112554546
Test: N/A
Change-Id: I56c468a9a4f162f6ce8079d6b4b61c5ec4b8f633
2018-08-20 03:48:47 +00:00
Ryan Mitchell
0f7da5eda8 AAPT2: Set array element keys
Many google3 apps using lemon resource filtering had difficulty
converting to aapt2. AAPT2 never set the name of the ResTable_map name
value mapping.

Bug: 78472540
Test: blaze build java/com/google/android/apps/gmail:Gmail_go_release
and checking that all array resources are present
Change-Id: I3d253a4c52c6100085e4c1083f4c36a943f11c76
2018-08-17 14:37:33 -07:00
Ryan Mitchell
83a37adfbb AAPT2: Loosen loading apk format requirements
The Android runtime and AAPT are more lenient of apk format, allowing
for duplicate enty, types, and configs. This change loosens the
ResourceTable's checks on resource uniqueness when apks are loaded; not
when ResourceTables are being created by aapt2.

Bug: 36051266
Test: Tested using apks in bug with allow_duplicates on and off
Change-Id: I9296417bf2dc53e1e891479a53679a0388210d50
2018-08-09 10:57:29 -07:00
Ryan Mitchell
0dcb20c727 AAPT2: Fix quoted text in res/xml assets
AAPT2 aggressively processed xml nodes and regressed from the behavior
of AAPT. This change restores AAPT's processing of xml nodes with
quotations.

Bug: 71805084
Test: Created tests in XmlFlattener_test.cpp and ran the test in the
Android Studio project given in the bug

Change-Id: Idedb4f1002e9fd705ceb5feae35289235b754b02
2018-05-02 17:53:21 +00:00
Michael Wachenschwanz
8b74927e9e Fix fraction and dimension type in Resource.proto
Change fraction and dimnesion type from float to uint32

Change-Id: I641dc42cf7b6bdd8bc16c8cdbd573ec3281a084a
Fixes: 78182701
Test: aapt2_tests
2018-04-20 10:46:47 -07:00
Ryan Mitchell
70414f22dc Check the size of the strings in the StringPool before flattening.
Test: Tested for normal functionality when string does not exceed
maximum length and tests for detection of string that is too lonhg for
UTF8i
Bug: b/74176037

Change-Id: Ic71d3671a069e7012e8ca107e79e071499eebbf6
(cherry picked from commit a15c2a8957b9883cb293fdacaeabd7f2e037a0a5)
2018-04-04 15:54:43 +00:00