47 Commits

Author SHA1 Message Date
Adam Lesinski
4ffea04064 AAPT2: Define intermediate compiled XML proto
This proto format is meant to encapsulate more information
that is specific to Android and allows for easier validation
and manipulation across tools.

Test: make aapt2_tests
Change-Id: I13bc34a460671fc0a36246be0d287a3d37d244d6
2017-08-23 11:54:38 -07:00
Adam Lesinski
6b37299129 AAPT2: Change XmlDom to exclude Namespace as a node
In preparation for exporting an XML proto format for UAM to consume,
this change brings the XML DOM API more in line with other APIs that
do not make the Namespace a separate node.

Treating Namespace declarations as just properties of an Element
node makes the implementation of algorithms much simpler, as
the constraints that Namespace nodes have only one child
are now built in and traversing to find Element nodes
is much simpler.

Also made a bunch of quality of life improvements, like formatting and
comment style.

Test: make aapt2_tests
Change-Id: Ib97ff1c4252b7907e2cc1f13a448dc4ca3b809a4
2017-08-17 17:43:08 +00:00
Adam Lesinski
efeb7af13b AAPT2: Fix windows unicode path issues
Mingw64 was being difficult, so instead of defining a wmain entrypoint,
the command line parameters are parsed manually using built-in Windows
methods that support Unicode. The results are converted to UTF8 and
handled just like the rest of the linux/mac version of the code.

This also removes dependencies on std::istream in favour of a
FileInputStream which calls the appropriate unicode version of
open to read a file.

No speed regressions found on Linux or MacOS.

Bug: 62336414
Bug: 63830502
Test: manual
Change-Id: I597da51e33729ed1b98bf246e7e773337fd3fee8
2017-08-03 17:33:52 -07:00
Adam Lesinski
060b53d028 AAPT2: Ensure style strings are always first in StringPool
Move the styled strings to a separate section of the StringPool so
that sorting can never mess up the order of Styles.

Bug: 63570514
Test: make aapt2_tests
Change-Id: Id2ce1355b92be1bb31ce0daa7e54ae9b5b6c2ffe
2017-07-31 17:46:28 -07:00
Adam Lesinski
fba0cf2950 AAPT2: Fix processing of quotes in XML
When processing attributes in XML, quotes can't be used to mark a
section as whitespace preserving, so the assumption should be that the
entire string is whitespace preserving, which makes quote characters
literals.

Bug: 62840718
Bug: 62840406
Test: make aapt2_tests
Change-Id: I4afff02148b5b8e78833abf1f323c2f5325d6155
2017-07-07 13:23:55 -07:00
Adam Lesinski
a45893a5c5 AAPT2: Clean up tests a bit
Since the latest gtest has fixed support for
explicit bool operators, remvoe AAPT_ASSERT_* and AAPT_EXPECT_*.

Also switch to use NotNull() matchers, which are more legible.

Test: make aapt2_tests
Change-Id: Idce199ca9d567d70f7aae275fee15e04bb914c9e
2017-06-09 14:57:07 -07:00
Adam Lesinski
c744ae8aca AAPT2: Implement attribute compat versioning
This change defines some hardcoded rules to degrade
attributes in newer SDKs to specific older attributes.

An attribute with a degrade rule will generate a new XML for the API
in which the attribute resulting from the degradation was introduced.

Since API 22 (Lollipop MR1), attributes are correctly ignored and do
not need to be versioned. In XML files defined for APIs 22+, the
original and degraded attributes coexist in the same XML file.

One such example is paddingHorizontal, introduced in API 26.
paddingHorizontal degrades to paddingLeft and paddingRight, which
were both introduced in API 1.

Bug: 35763493
Test: make aapt2_tests
Change-Id: I4aa8755a9ee2c0cc5afdc55c3d30093fd3a47f3d
2017-05-24 15:14:29 -07:00
Adam Lesinski
48448e8a31 AAPT2: Fix string escaping
We were processing escaped strings too early, before
parsing of values into types. Now the escaped strings get
processed when they are being flattened.

Bug: 37715376
Test: make aapt2_tests
Change-Id: Ic59aa2e3a20c40756c219752ff74b2a4f8a602ba
2017-04-28 12:51:23 -07:00
Adam Lesinski
4ca56978a9 AAPT2: Add workaround for non-standard package IDs
The dynamic ref table used to map build-time IDs to runtime IDs
is mainly used for shared resource libraries and has a few built-in
mappings (app 0x7f and framework 0x01).

Using a non-standard package ID like 0x80 causes a failure in package ID
lookup. The solution is to ship the dynamic_ref_table with an identity mapping
with any resource table that uses a non-standard package ID.

Adds some tests to ensure this works correctly.

Bug: 37498913
Test: make libandroidfw_tests
Test: make aapt2_tests
Change-Id: Ic3f67942384d34e7fdcbc94ded360e940e3ebc8a
2017-04-26 21:55:31 -07:00
Adam Lesinski
b522f04bc2 AAPT2: Allow truncating of package names
ResTable_package header only allows 127 UTF-16 characters, so AAPT
would truncate the real package name to fit. AAPT2 would error-out
on any package name longer than 127 UTF-16 characters. This strictness
is not required except when building shared libraries, which use the
full package name as a way of identifying the runtime assigned
package ID to package name mapping.

Bug: 36940145
Test: make aapt2_tests
Change-Id: I7d2b7e50c7ab30c6a6c4f15d310e711f68e35091
2017-04-21 17:02:09 -07:00
Adam Lesinski
06460ef0d7 AAPT2: Fix up file IO
This also enables an AAPT behavior that CTS tests have
come to depend on.

Small files that compress negatively (get larger) are stored
uncompressed. Some CTS tests assume this and try to open these
files by mmapping them, which is only possible if they are
uncompressed.

Bug: 35461578
Test: make aapt2_tests
Change-Id: Id622a6150fe72477ad65d67d1bad897a8ee2ffb9
2017-03-20 16:53:46 -07:00
Adam Lesinski
f34b6f4f2b AAPT2: Add --package-id flag for feature-split suppport
Bug: 35928935
Change-Id: Ia8496505e61cfcfdb8f9f62366d2f36e453db111
Test: make aapt2_tests
2017-03-07 11:06:16 -08:00
Adam Lesinski
ac6edc501b AAPT2: Fix escaping sequence processing in XML parsing
Bug: 35483813
Test: make aapt2_tests
Change-Id: I68baba17ab3639c220b734a2a68d86aad0dedf8c
2017-03-02 19:32:32 -08:00
Adam Lesinski
ceb9b2f80f AAPT2: Shared library support
Test: make aapt2_tests
Change-Id: I98dddf1367e6c0ac425bb20be46e6ff05f4f2f45
2017-02-22 11:16:13 -08:00
Adam Lesinski
c8f71aa67e Add ResTable_sparseTypeEntry support
Benchmarks on bullhead-userdebug show that there is a negligent
performance impact when using sparse entries on a 30% loaded
sparse type of 1000 resources.

Benchmark                                             Time           CPU Iterations
-----------------------------------------------------------------------------------
BM_SparseEntryGetResourceSparseLarge                255 ns        254 ns    2751408
BM_SparseEntryGetResourceNotSparseLarge             254 ns        254 ns    2756534

Bug: 27381711
Test: make libandroidfw_tests aapt2_tests
Change-Id: I051ea22f2f6b2bc3696e446adc9e2a34be18009f
2017-02-15 12:40:02 -08:00
Adam Lesinski
d5083f6f6b Move StringPiece to libandroidfw
libandroidfw needs to make use of StringPiece, so
move it to libandroidfw and update all code referencing
StringPiece in aapt2.

Test: make libandroidfw_tests libaapt2_tests
Change-Id: I68d7f0fc7c651b048d9d1f5e7971f10ef5349fa1
2017-01-17 18:55:51 -08:00
Adam Lesinski
ce5e56e243 AAPT2: Rename to match new style
Use Google3 naming style to match new
projects' and open source google projects' style.

Preferred to do this in a massive CL so as to avoid
style inconsistencies that plague legacy code bases.
This is a relatively NEW code base, may as well keep
it up to date.

Test: name/style refactor - existing tests pass
Change-Id: Ie80ecb78d46ec53efdfca2336bb57d96cbb7fb87
2016-10-26 19:30:23 -07:00
Adam Lesinski
b54ef10f55 AAPT2: Ensure string pool ordering is compact
Keep styledstrings at the beginning of the StringPool
to reduce the padding in the StyledString array.

Bug:32336940
Test: manual
Change-Id: Iec820c21a54daac40ecc3b2f87517a0f1efc9d3d
2016-10-21 13:51:59 -07:00
Adam Lesinski
cacb28f2d6 Use Google3 style guide with .clang-format
Test: style change only, builds ok
Change-Id: I885180e24cb2e7b58cfb4967c3bcb40058ce4078
2016-10-19 12:18:14 -07:00
Adam Lesinski
5eeaaddffd AAPT2: Add Inline Complex XML support
See: https://developer.android.com/guide/topics/resources/complex-xml-resources.html

Change-Id: I8274c85e25cabf90423141c228697e873167d136
2016-08-31 18:32:34 -07:00
Adam Lesinski
5c3464c75f AAPT2: Fix merging of styleables the right way
Styleables should only be merged when processing overlays.
This moves the styleable merging code out of ResourceTable
and into TableMerger.

Change-Id: I3aae05cf4dd875cd25ac2ac744b61194409b2fee
2016-08-26 19:40:50 -07:00
Chih-Hung Hsieh
470f8fcb5c Fix clang-tidy warnings in aapt and aapt2.
* Add explicit keyword to conversion constructors.
* Add NOLINT(implicit) comments for implicit conversion constructors.
Bug: 28341362
* Use const reference type for read-only parameters.
Bug: 30407689
* Add NOLINT to suppress wrong clang-tidy warnings on macros.
Test: build with WITH_TIDY=1

Change-Id: Ibfafb23d9be463b4072745a51a6d9f1812dcac45
2016-08-15 12:32:51 -07:00
Chih-Hung Hsieh
9b8528fee4 Fix clang-tidy warnings in aapt and aapt2.
* Add explicit keyword to conversion constructors.
* Add NOLINT(implicit) comments for implicit conversion constructors.
Bug: 28341362
* Use const reference type for read-only parameters.
Bug: 30407689
* Use const reference type to avoid unnecessary copy.
Bug: 30413862
Test: build with WITH_TIDY=1

Change-Id: Id6d21961f313a1ad92b15a37fdaa5be9e8ab48e1
2016-08-11 15:16:41 -07:00
Alexandria Cornwall
a9ff14098b AAPT2: Strip dedicated tools namespace from XML
Android has a dedicated XML namespace dedicated for tools that should not be
included in the final APK. AAPT strips this out, but the feature was missing
from AAPT2. See: http://tools.android.com/tech-docs/tools-attributes

Bug: 29115919
Change-Id: I8f4fc79e6c8592a313a691134e44d16fd91f36ed
2016-08-03 10:02:15 -07:00
Adam Lesinski
ad48bbcbf6 Merge "AAPT2: Change accepted notation for resource names in tests" 2016-07-27 19:52:44 +00:00
Adam Lesinski
58a20a6482 AAPT2: Change accepted notation for resource names in tests
Previously the way to name resources in tests was to use reference
notation (@[package:][type/]name). Now we use name notation (no @).

Change-Id: I68f0a36562d89cc78c582d128f370d9556c58707
2016-07-25 17:56:58 -07:00
Alexandria Cornwall
e0af925c8f AAPT2: Convert StringPool flattening to UTF-8
Convert StringPools in resources.arsc and binary XML files to use UTF-8
instead of UTF-16 to save lots of bytes.

Bug: 30053277
Change-Id: I8eb6c78644e51b15f89edadab71cc86f5a851a45
2016-07-25 09:46:38 -07:00
Adam Lesinski
d0f116b619 AAPT2: Remove usage of u16string
For legacy reasons, we kept around the use of UTF-16 internally
in AAPT2. We don't need this and this CL removes all instances of
std::u16string and StringPiece16. The only places still needed
are when interacting with the ResTable APIs that only operate in
UTF16.

Change-Id: I492475b84bb9014fa13bf992cff447ee7a5fe588
2016-07-13 17:45:28 -07:00
Adam Lesinski
e1fda9aec4 AAPT2: Encode empty attribute string values
Due to another bug, empty strings in XML files
were encoded as NULLs. This was only needed for
encoding missing namespace URIs. Attribute
values should remain empty strings.

Bug:29939875
Bug:29462255
Change-Id: I3897661d85865c88bb2b7cf1495da16c30f7272e
2016-07-01 14:51:51 -07:00
Adam Lesinski
fb6312fe93 AAPT2: Add version collapsing
When an app specifies (or imports) resources with various
configurations for different SDK versions, specifying
a minSdk will make many of those resources unreachable.

Version collapsing will prune out the resources specified
for SDK versions less than the minSdk.

If, however, there is no exact matching resource for the
minSdk version, the next smallest SDK version is kept.

Change-Id: Ic7bcab6c59d65c97c67c8767358abb57cdec60a4
2016-06-29 17:02:19 -07:00
Adam Lesinski
32852a575b AAPT2: Use hash map for StringPool indices
Hash map has better locality and average case performance

Change-Id: Ic2d111c3cff1d5c0e7e9cbe36714669e6508c275
2016-06-18 01:10:34 +00:00
Adam Lesinski
803c7c8079 AAPT2: Enable windows build and make sure it builds
Mingw32 4.8 is kind of picky with macros and some complicated template
stuff. Luckily there was another way to represent the
SFINAE code that works on all platforms. Yay!

Change-Id: Idc2e38f47bfdc57b394550bfa0f53cc0b825df25
2016-04-06 16:24:02 -07:00
Adam Lesinski
64587af817 AAPT2: Support building, linking, and merging static libraries
Android static libraries are like APKs but they contain much more debugging
and source information. We need to treat them differently in 3 ways:

1) When building a static library, we skip things like ID assignment and
   product/config stripping. Source information is emitted as well.
2) When building a static library and linking against another
   static library, we don't want to merge, we want to simply reference.
3) When building an app that uses static libraries, we want to merge
   the static library under the same package with or without mangling.

Bug:25958912
Change-Id: I425e032857936a3e83173c1edc2a6cdc6020b842
2016-03-04 14:39:01 -08:00
Adam Lesinski
e4bb9eb5af AAPT2: Introduce notion of 'product' to ResourceTable
This allows us to preserve the various product definitions during the compile
phase, and allows us to select the product in the link phase.

This allows compiled files to remain product-independent, so that they do not need
to be recompiled when switching targets.

Bug:25958912
Change-Id: Iaa7eed25c834b67a39cdc9be43613e8b5ab6cdd7
2016-02-12 22:21:48 -08:00
Adam Lesinski
59e04c6f92 AAPT2: Switch to protobuf for intermediate format
Without needing to conform to the runtime data format,
it is much easier to add new features such as debugging symbols
and carrying over product data to link time.

This also simplifies the runtime format parser and serializer,
which will change much less frequently than the protobuf intermediate
format.

Change-Id: I209787bbf087db0a58a534cb8511c51d21133e00
2016-02-09 19:59:17 +00:00
Adam Lesinski
24b8ff0faf AAPT2: Fix references to private parent
Change-Id: Id4697551b6c8cb6167f562de593006ae3c6158c0
2015-12-17 11:01:55 -08:00
Adam Lesinski
a6fe345be9 AAPT2: Fix overlay support
Supports the <add-resource> tag and mimics old AAPT behavior of
not allowing new resources defined unless <add-resource> was used
or --auto-add-overlay was specified.

Change-Id: I9b461137357617ade37fd7045b418b8e6450b9c4
2015-12-10 16:24:15 -08:00
Elliott Hughes
e7d2cc308e Track rename of base/ to android-base/.
Change-Id: If08a94025fc12433f7e248b7787cee3c14843097
2015-12-08 08:57:14 -08:00
Adam Lesinski
a40e972fda AAPT2: Support compiling a res/ directory and output to zip
This allows us to compile an entire directory and output to a single
file. This is important to support generated resources in the make
build, since we may not know what resources get generated.

The link step will accept the zip and read the contents of it as if they
were passed in on the command line.

Change-Id: If1a51b0abe772350c24074353eb4989953c2e0cb
2015-11-30 11:14:01 -08:00
Adam Lesinski
28cacf091a AAPT2: Record source/comments for compound values' children
Values like styles or arrays have children which need to have
their source and comments preserved.

Change-Id: I6fc713ba36627e6d66c7930b4080cc1403bac207
2015-11-23 14:22:47 -08:00
Adam Lesinski
a587065721 AAPT2: Verify min/max attr fields
Integers are now checked to see if they fall in the range
of min/max for the attribute they are assigned.

Change-Id: I42c435b15fd3f0bd23691c83efccce4ad5973276
2015-11-23 12:01:15 -08:00
Adam Lesinski
467f171315 AAPT2: Fail compiling when private symbols are referenced
Also moved some XML specific stuff into its own directory,
and refactored ReferenceLinker a bit.

Change-Id: I912247a82023c1bbf72dc191fbdaf62858cbec0c
2015-11-19 14:46:53 -08:00
Adam Lesinski
3b4cd94034 AAPT2: Add support for comments in R.java
Change-Id: Iaa5f3b75bf7de9dbf458fa5c452f7312989f4c4f
2015-11-02 11:53:47 -08:00
Adam Lesinski
e78fd617ec AAPT2: Move comments and source into Value
Values are closely related to where they were defined, so
this information should live inside the Value.

This also enables comments to be attached to nested Values.

Change-Id: Ic7481b5a5f26d0ef248d638e2e29252f88154581
2015-10-22 16:35:47 -07:00
Adam Lesinski
9e10ac7015 AAPT2: Process <java-symbols> and private symbol package
Need to introduce the idea of multiple levels of visibility to support <java-symbol>.

Public, Private, Undefined.

Public means it is accessible from outside and requires an ID assigned.
Private means that we explicitly want this to be a symbol (show up in R.java), but not visible
to other packages. No ID required.

Undefined is any normal resource. When --private-symbols is specified in the link phase,
these resources will not show up in R.java.

Change-Id: Icba89221e08e685dee7683786aa7112baf28c856
2015-10-22 12:52:49 -07:00
Adam Lesinski
9ba47d8130 Filter products during compile phase
Unfortunately there is no good way to deal with products in the link phase.
Products are like preprocessor defines in that they are processed early
and change the composition of the compiled unit.

Change-Id: I6d5e15ef60d29df8e83e059ba857c09333993779
2015-10-16 15:50:58 -07:00
Adam Lesinski
1ab598f46c AAPT2: Separate out the various steps
An early refactor. Some ideas became clearer as
development continued. Now the various phases are much
clearer and more easily reusable.

Also added a ton of tests!

Change-Id: Ic8f0a70c8222370352e63533b329c40457c0903e
2015-10-16 12:13:08 -07:00