5534 Commits

Author SHA1 Message Date
TreeHugger Robot
8910bf95c6 Merge "Fix ValueMetricProducer_test on hwasan" into rvc-dev 2020-03-25 17:32:55 +00:00
Tej Singh
547f46331c Fix ValueMetricProducer_test on hwasan
TestUseZeroDefaultBaseWithPullFailures was failing on hwasan builds
because the test had a reference to an Interval object that was
deleted. The test tried to get another Interval object and assign it to
the reference. However, what this really did is change the Interval
object that the reference referred to. Since that Interval object had
been deleted, hwasan failed the test.

This fixes the issue. Also clean up some usages of auto to make
the test more readable.

Test: bit statsd_test:* on blueline_hwasan
Bug: 152354576
Change-Id: I392740770715391c727d190b1ed8ba22cc01210f
2020-03-25 08:10:35 +00:00
tsaichristine
63143251fe Update the cc_benchmark to use the new socket schema
Also removed unused CreateEvent helper functions in metric_util.cpp

Test: bit statsd_benchmark:*
Bug: 149794614
Change-Id: I14bcc254c44773eb42f749305beadaaf23a19b2b
2020-03-24 16:03:48 -07:00
Christine Tsai
4aa7af0a8f Merge "(Part 3) Use new socket schema with statsd tests" into rvc-dev 2020-03-24 20:55:26 +00:00
Mike Ma
06b3aaee39 Merge "Optimize memory usage in incidentd" into rvc-dev 2020-03-24 20:27:41 +00:00
junyulai
f889330c89 [SM01] Add rat type fields in data usage atoms for mobile data
Test: build
Bug: 129082217
Change-Id: Idfc048d49f98acb805f49046a0b3dc871e001142
Merged-In: Idfc048d49f98acb805f49046a0b3dc871e001142
(cherry picked from commit 071a06635b713a1aa9d2ba9331799ffecd6f1088)
2020-03-24 04:28:48 +00:00
Mike Ma
892ccd9bd5 Optimize memory usage in incidentd
EncodedBuffer is used a lot in incidentd. EncodedBuffer uses malloc
internally to acquire memory. Frequently creating and destroying
EncodedBuffer creates memory fragmentation, leading to high memory
usage after taking an incident report.
Also fixes a few other places with lots of malloc/free operations.

This change:
* Creates a pool of EncodedBuffer in incidentd. The saving is
significant. It reduces EncodedBuffer creation from 3 per section to
3 per report.
* Replaces malloc with mmap inside EncodedBuffer. mmap is guaranteed
to be mem page aligned, so there will be no mem fragmentation after
destroying EncodedBuffer.
* Replaces new with mmap inside TombstoneSection
* Forks a process to execute LogSection, because liblog malloc & free
significant amount of memory

Result:
PSS before taking a report: 1295 KB
PSS after taking a report: 1336 KB

Bug: 150311553
Test: heapprofd
Change-Id: I83bd9c969b751c80b2f42747020799bd85d8aae6
2020-03-23 18:46:58 -07:00
TreeHugger Robot
14f33ae83e Merge "Native API Council asked that we remove the C++ class from the public header." into rvc-dev 2020-03-24 01:35:40 +00:00
tsaichristine
a3d2ed8cb2 (Part 3) Use new socket schema with statsd tests
Update last set of statsd tests to use new socket schema

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I0fe2c219ad75813db54ff0cfbad50f55e29cb626
2020-03-23 12:23:26 -07:00
Greg Kaiser
f0f1d83624 ShellSubscriber: Avoid rollover bug
We initialize 'mToken' instead of leaving it uninitialized, making
it much less likely to rollover.  In additional, we change our
check to inequality, to handle rollover if that were to happen.

There is the (theoretical) possibility of exactly 2^32 other
requests being claimed between our claim and our check.  It's
assumed that is essentially impossible and not a concern.

Test: TreeHugger
Bug: 150619687
Change-Id: Iee303e05082a6b3b31ed546bd62d3afe67c771d8
2020-03-23 16:09:57 +00:00
Joe Onorato
084aa3cb17 Native API Council asked that we remove the C++ class from the public header.
The API is simple enough, so just reimplement everything using the
C API directly.

Bug: 148940365
Test: treehugger
Change-Id: I0a75744e975e8d3c2a557e533eacd03200388ddc
2020-03-21 18:33:28 -07:00
TreeHugger Robot
e13e948d6d Merge "Change sint to int in atoms.proto" into rvc-dev 2020-03-21 07:56:17 +00:00
TreeHugger Robot
1e377304cb Merge "Statsd update for native puller api feedback" into rvc-dev 2020-03-21 03:15:06 +00:00
Howard Ro
8e051743c5 Change sint to int in atoms.proto
sint32 is not supported in statsd

Bug: 151110217
Test: m statsd
Change-Id: Ic711cee06a15a957fd4e25c4c01176cc44598cb2
2020-03-20 15:07:20 -07:00
Muhammad Qureshi
4d57eccd1d Merge "Remove kBytesFieldAtoms from atoms_info" into rvc-dev 2020-03-20 20:34:35 +00:00
Tej Singh
73597dcb9a Statsd update for native puller api feedback
Update statsd to take in times in milliseconds instead of nanoseconds.

Also make appropriate updates for graphics stats, odpm, subsystem sleep
state, and LibStatsPullTests

Test: atest LibStatsPullTests
Test: bit statsd_test:*
Bug: 150788562
Change-Id: I593552d6c50bb4dcb89ca9cc1c737781653e7cc5
2020-03-20 11:16:18 -07:00
Muhammad Qureshi
f77913fbc6 Remove kBytesFieldAtoms from atoms_info
Bug: 150415347
Test: m
Test: bit statsd_test:*
Change-Id: I2e55b445121a39e3bf12164cd9123561e8b5bd4c
2020-03-20 10:55:46 -07:00
TreeHugger Robot
a799cdba86 Merge "Implement new perfd<->statsd ShellSubscriber comm." into rvc-dev 2020-03-20 17:35:58 +00:00
Christine Tsai
a9c3cc288c Merge "(Part 2.6 - Final one!) Use new socket schema in ValueMetricProducer_test" into rvc-dev 2020-03-20 16:37:11 +00:00
tsaichristine
e5405ba55a (Part 2.6 - Final one!) Use new socket schema in ValueMetricProducer_test
First CL at ag/10697747. Sending ValueMetricProducer_test as multiple CLs to retain diff and make
it easier to review.

Updates from TestInvalidBucketWhenMultipleBucketsSkipped to TestSlicedStateWithPrimaryField_WithDimensions

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I682a19ea6fe39dd750ca396c9de479befa658f0d
2020-03-19 20:59:32 -07:00
Mike Ma
707263d99c Merge "TextDumpsysSection memory optimization" into rvc-dev 2020-03-20 03:52:54 +00:00
TreeHugger Robot
591bb7b1fe Merge "Enforce permission on native puller API" into rvc-dev 2020-03-19 21:12:52 +00:00
Mike Ma
87003ad3f6 TextDumpsysSection memory optimization
TextDumpsysSection used to inherent WorkerThreadSection, which
allocates 2x more memory than the dumped content. This change saves
the extra allocation by writing dumpsys content directly to FdBuffer.

Bug: 150311553
Test: Manually run "incident -p EXPLICIT 4000"
Change-Id: I9c0c0db75c8595822ee0711040e8865dd69378b6
2020-03-19 12:31:29 -07:00
Tej Singh
10458eca11 Enforce permission on native puller API
Test: m
Test: no security exceptions on boot
Test: atest LibStatsPullTests
Bug: 148955001
Change-Id: I4b06bfc41be2925270eaddd717f1499d98739dae
2020-03-19 11:54:11 -07:00
tsaichristine
3ca4d78d72 (Part 2.4) Use new socket schema in ValueMetricProducer_test
First CL at ag/10697747. Sending ValueMetricProducer_test as multiple CLs to retain diff and make
it easier to review.

Updates from TestUseZeroDefaultBase to TestEmptyDataResetsBase_onDataPulled

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I91601123e8aaeb92aa6e6618dbd12467949a94e4
2020-03-19 10:28:20 -07:00
Ruchir Rastogi
a5e4bb5aa6 Implement new perfd<->statsd ShellSubscriber comm.
Because we no longer linkToDeath against a binder object to detect if
the cmd process has died, we detect deaths by checking if writes fail.
ag/10476582 proves that writes fail if the cmd process dies.

Test: m statsd
Test: bit statsd_test:ShellSubscriberTest.testPushedSubscription
Test: bit statsd_test:ShellSubscriberTest.testPulledSubscription
Bug: 150619687
Change-Id: I44a777ffff11e5b9298912b2906063c65e9009eb
2020-03-19 09:17:44 -07:00
tsaichristine
5adb1b1461 (Part 2.3) Use new socket schema in ValueMetricProducer_test
First CL at ag/10697747. Sending ValueMetricProducer_test as multiple CLs to retain diff and make
it easier to review.

Updates from TestPushedEventsWithoutCondition to TestSkipZeroDiffOutputMultiValue

Test: bit statsd_test:*
Bug: 149590301
Change-Id: Ie429f36124d475c16652eb52fed41ab88128ffdd
2020-03-18 16:24:54 -07:00
TreeHugger Robot
0c4353a5d6 Merge "Fix incidentd stack use-after-return" into rvc-dev 2020-03-18 20:33:45 +00:00
Jeffrey Huang
4980c0a6cf Merge "Migrate statsd_test to use libstatslog_statsdtest" into rvc-dev 2020-03-18 19:47:14 +00:00
Mike Ma
b328e290f1 Fix incidentd stack use-after-return
The detached thread may live longer than the caller, and then "data"
goes out of scope. Fix it by managing data using a strong pointer.

Fixes: 151335416
Test: turn on hwasan, tweak WorkerThreadSection timeout, verify no
hwasan error

Change-Id: I179204b17c381e4e920b9aee07900150d9497639
2020-03-18 11:43:15 -07:00
Christine Tsai
300ee6ea68 Merge "(Part 2.2) Use new socket schema in ValueMetricProducer_test" into rvc-dev 2020-03-18 17:45:08 +00:00
tsaichristine
b828fe2a72 (Part 2.5) Use new socket schema in ValueMetricProducer_test
First CL at ag/10697747. Sending ValueMetricProducer_test as multiple CLs to retain diff and make
it easier to review.

Updates from TestEmptyDataResetsBase_onConditionChanged to TestInvalidBucketWhenPullFailed

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I915b47e086ee5065eaa71094b83d2b271ce847b7
2020-03-17 18:25:37 -07:00
tsaichristine
f487f2a509 (Part 2.2) Use new socket schema in ValueMetricProducer_test
First CL at ag/10697747. Sending ValueMetricProducer_test as multiple CLs to retain diff and make
it easier to review.

Updates from TestPulledEventsTakeAbsoluteValueOnReset to TestPulledValueWithUpgradeWhileConditionFalse

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I31b4a635262586aedeb23f84cae3d66f48593b3f
2020-03-17 18:07:23 -07:00
Susi Kharraz-Post
3b8389263e Merge "Add sharesheet + generic ranking atoms for statsd." into rvc-dev 2020-03-17 20:14:27 +00:00
Jeffrey Huang
3eb84d4da9 Migrate statsd_test to use libstatslog_statsdtest
This is part of the migration to remove libstatslog from statsd

Since we still depend on libstatsmetadata, we cannot fully mock out
all the atoms. So we will create a new libstatslog_statsdtest to house
atoms used only in the tests.

Bug: 150976524
Test: bit statsd_test:*
Change-Id: I6368305eb89b2c35e670e42907a308afd922e604
2020-03-17 11:59:01 -07:00
TreeHugger Robot
92eb1bbfd4 Merge "(Part 2.1) Use new socket schema in ValueMetricProducer_test" into rvc-dev 2020-03-17 18:21:32 +00:00
Susi Kharraz-Post
1df7b0066a Add sharesheet + generic ranking atoms for statsd.
Bug: 147508495

Change-Id: Ie02cbb84a0ddfe7ca51acf722f9a5632b846f283
Test: m out/soong/.intermediates/frameworks/base/framework-statslog-gen/gen/android/util/StatsLogInternal.java out/soong/.intermediates/frameworks/base/tools/stats_log_api_gen/statslog.cpp/gen/statslog.cpp
Android Metrics Design Review: https://eldar.corp.google.com/assessments/369729150/revisions/1?jsmode=du

Change-Id: I49391a786840febb6b7b67ac91e029d8d742e956
2020-03-17 18:10:37 +00:00
tsaichristine
bb33b803ef (Part 2.1) Use new socket schema in ValueMetricProducer_test
Sending ValueMetricProducer_test as multiple CLs to retain diff and make
it easier to review.

I added a few different CreateLogEvent functions that account for tests
that need events with 0, 2, and 3 values.

The old #createEvent function took in one value but added it twice to the
LogEvent. I replaced this with #CreateRepeatedValueLogEvent. Any events
made with #createEvent or that took the form of:
event.write(tagId)
event.write(value)

was replaced with a call to #CreateRepeatedValueLogEvent. I assumed
that the tagId was used as the first field because an arbitrary number
was needed.

Test: bit statsd_test:*
Bug: 149590301
Change-Id: I26de8074a66babcf3ab7ee395be0598d88b4ba59
2020-03-17 09:17:27 -07:00
Stanislav Zholnin
86e27477a7 Merge "Delay initialization of RarelyUsedPackages and package sampling." into rvc-dev 2020-03-16 09:23:17 +00:00
Stanislav Zholnin
48811ee600 Add logging operation ids.
Add operation ids which are used only for logging. They will
allow to reduce bandwidth required for logging. Mapping between
operation string ids (OPSTR_, public API) and Logging ids
is enforced by statsd CTS test.

Bug:143519689
Test: android.cts.statsd.atom.UidAtomTests#testAppOps
Test: android.cts.statsd.atom.UidAtomTests#testForegroundServiceAccessAppOp
Exempt-From-Owner-Approval: discussed  with moltmann@ in chat.

Change-Id: I2f85d6889e946219557d26b28334d4bacde06b12
Merged-In: I2f85d6889e946219557d26b28334d4bacde06b12
2020-03-15 14:29:32 +00:00
Stanislav Zholnin
4c32385bf6 Delay initialization of RarelyUsedPackages and package sampling.
Delay uniform sampling of packages.
Introduce boot time sampling of packages - simplified sampling designed
to save time during start up sequence.

Bug: 150239020
Test: manual verification
Change-Id: I5911b663cfefe324a443b0ba3f97c46e0984a0f3
2020-03-15 11:23:53 +00:00
TreeHugger Robot
e9133b9351 Merge "Parse annotations within LogEvent" into rvc-dev 2020-03-14 01:29:41 +00:00
Ruchir Rastogi
84eb44bb4c Parse annotations within LogEvent
This is a preliminary CL that parses annotations within LogEvent, but
does nothing with the annotations provided. Annotation information will
be exposed from LogEvent/FieldValue in a future CL. (That CL will also
contain tests for parsing annotations.)

Test: m statsd
Bug: 150414043
Change-Id: Iae79644ea8455280a654076bfd5819927d4183d3
2020-03-13 16:40:14 -07:00
Philip P. Moltmann
12ac3f406f Rename featureId -> attributionTag
In the core functionality this changes everything including aidl's and
field names:
- Context
- ContentProvider
- AppOps*
- Package parsing

For the rest, this is a shallow change to only change to the changed
APIs. This keeps the change small-ish

Exempt-From-Owner-Approval: Rename
Fixes: 148792795
Test: TH
Change-Id: I2a2245fe76e09e62cb13d5785d2efb4a304ba54a
Merged-In: I2a2245fe76e09e62cb13d5785d2efb4a304ba54a
2020-03-12 19:26:10 +00:00
Chi Zhang
91af57e5c1 Merge "Telephony statsd atoms." into rvc-dev 2020-03-12 17:53:55 +00:00
Jooyung Han
9432967df0 Merge "Use aidl::nullable for nullable type in C++" into rvc-dev 2020-03-12 11:37:07 +00:00
Jooyung Han
66c567a8c4 Use aidl::nullable for nullable type in C++
To reduce the discrepance between old code which still uses
std::unique_ptr and new code using std::optional.

This might help to avoid merge-conflicts between branches.

Bug: 144773267
Test: m
Merged-In: Ie3196ee5cce17d77950eea9479d2cc1406e9e674
Merged-In: I33822bc76ef87637d5408849f64a0607e121792e
Change-Id: I33822bc76ef87637d5408849f64a0607e121792e
(cherry picked from commit ad62e8cbf5cf6083568f6f230ef7d73ad8776971)

Exempt-From-Owner-Approval: cherry-pick from master with owner's approval
2020-03-12 11:36:33 +00:00
TreeHugger Robot
94425b919b Merge "Update path for SnapshotMergeReported." into rvc-dev 2020-03-12 10:38:43 +00:00
Chi Zhang
bb9ed92165 Telephony statsd atoms.
Test: build
Bug: 146066107
Bug: 141631489
Bug: 122371089
Bug: 149880090
Bug: 127666858

Change-Id: I124cf2da59ed4c5e92406081aac0df0f347ace74
2020-03-11 15:10:36 -07:00
Yifan Hong
36137f09e9 Update path for SnapshotMergeReported.
Test: none
Bug: 147696014
Change-Id: I125691cfdda1e42946fd23879390fe5fafe1fbc5
2020-03-11 10:35:33 -07:00