816 Commits

Author SHA1 Message Date
TreeHugger Robot
d11df1d76f Merge "Atom: BootSequenceReported" 2018-01-31 04:24:54 +00:00
TreeHugger Robot
b78bb44a87 Merge "Atom: Shutdown sequence event" 2018-01-31 02:05:26 +00:00
Yao Chen
6d6619a44e Make statsd dogfood app to be a system app.
And update the baseline config.

Bug: 72710440
Test: manual
Change-Id: Icc37d1d688d3788e9c9e98a594b9532889ffeea9
2018-01-30 15:53:15 -08:00
Yangster-mac
9369446f0b Support dimension in condition in metric producers.
Test: added e2e tests for count/duration metrics sliced by fields in condition and with/without links.

Change-Id: Ie34deba68e6780abdde458be3f0ce5284e76a1a2
2018-01-30 13:33:57 -08:00
TreeHugger Robot
641e29df46 Merge "Extend gauge metric to support memory metric." 2018-01-30 19:08:41 +00:00
TreeHugger Robot
357b63b172 Merge "Cpu usage optimization: 1/ Avoid unnecessary field/dimension proto construction. 2/ use unordered_map for slicing. 3/ Use dimension fields to compare dimension keys." 2018-01-30 17:37:14 +00:00
Yangster-mac
34ea1103a0 Extend gauge metric to support memory metric.
Test: statd unit test passed.

Test: statsd unit test passed
Change-Id: I2e3f26563678ae77d44afe168454b6d1ea449f3a
2018-01-30 09:14:54 -08:00
Tej Singh
6483ea4e0a Atom: BootSequenceReported
Update to atoms.proto to include boot sequence reported

Test: manual
Change-Id: Ie6b7021528ef81b95969ae90f130f5f0ad0eb9a5
2018-01-29 19:26:57 -08:00
yro
9b97976e76 Re-enable statsd from droidfooders by default
Test: the team-wide bug bash to capture any issues in advance but did
not find any statsd crash

Change-Id: I253e0067a710c37bf10667e3a8dbde99d0b289c7
2018-01-29 16:23:13 -08:00
Rajeev Kumar
9023599fc3 Split usage_in_bytes into following separate fields:
1. rss_in_bytes
  2. cache_in_bytes
  3. swap_in_bytes

See: go/android-p-memory-metrics

Bug: 72177881
Test: Manual
Change-Id: I8d88aa24aac500b9e5cb4e40b9af9dab3af7113b
2018-01-29 11:06:48 -08:00
Tej Singh
1ea4289b4d Atom: Shutdown sequence event
Records start and duration of shutdown, along with reason and if it was
a reboot.

Test: manually verified statsd received atom. CTS test for this will be
difficult, and I will investigate further later.

Change-Id: I0f6b595e0e251fd0a8b38127182d055885460a55
2018-01-29 10:35:51 -08:00
Yao Chen
ba6644d6be Merge "Add more statsd's debugging info to dumpsys." 2018-01-29 18:32:45 +00:00
Rajeev Kumar
8a9fa058a4 Refine atom definitions for memory metrics.
Defines following atoms:
 -- AppStartMemoryStateChanged
 -- ProcessMemoryState
 -- LmkStateChanged
 -- LmkProcessKilled

These changes are based on the discussion with WestWorld team.
See: go/android-p-memory-metrics
Bug: 72177881
Test: Manual

Change-Id: I884f23f3d7627440af5faa25f9479ba0d8763bec
2018-01-26 23:08:47 -08:00
TreeHugger Robot
18ffea7130 Merge "Add Bluetooth atoms" 2018-01-27 04:57:19 +00:00
TreeHugger Robot
ce01bd906e Merge "Proto enums for DeviceIdleMode" 2018-01-27 01:18:20 +00:00
Yangster-mac
7ba8fc357e Cpu usage optimization:
1/ Avoid unnecessary field/dimension proto construction.
2/ use unordered_map for slicing.
3/ Use dimension fields to compare dimension keys.

Test: all statsd tests passed.
Change-Id: I2f74f78589b7f6ecd0803a2ead822b8d0399f334
2018-01-26 23:17:02 +00:00
Chenjie Yu
9d7720b5b4 Add Bluetooth atoms
+ refactor pullers in StatsCompanionService to be more modular
+ rename CpuSuspendTime and CpuIdleTime to SystemElapsedRealtime
  and SystemUptime

Test: will add cts test
Change-Id: I463103fb271511cef4e0f877c20fd167fe8b173b
2018-01-26 23:15:51 +00:00
TreeHugger Robot
d2e162890e Merge "Proto enum reference for App Transitions" 2018-01-26 22:12:26 +00:00
Yao Chen
884c8c130f Add more statsd's debugging info to dumpsys.
+ Bugreport will use the non-verbose mode
+ Reuse the log_msg object in LogReader
+ Add logd errors to StatsdStats

Bug: 72383073

Test: manual + statsd_test

Change-Id: Id5a8b103074d034f5ece3c9831c740d44a5df9cd
2018-01-26 12:03:58 -08:00
Bookatz
8bdae8d682 Proto enums for DeviceIdleMode
Created an enums.proto that contains the device idle modes,
which is referenced by batterystats.

Note that, currently, batterystats is the origin of these
constants in the java code, and I have kept that as is.
Thus, batterystats references the enum.
Nevertheless, because the actual control of device idle
mode is DeviceIdleController.java, where these constants
will likely one day be moved, I have put the constants
in server/enums.proto (since DeviceIdleController is
in server, not os like BatteryStats).

Bug: 69478930
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.HostAtomTests#testDeviceIdleModeStateChangedAtom
Change-Id: I6e66801ae8256aab423067f9a4b852a194564a8d
2018-01-26 17:26:20 +00:00
TreeHugger Robot
b0db811fe9 Merge "Statsd and frameworks reference proto enums" 2018-01-26 05:02:35 +00:00
Bookatz
80d11a00eb Proto enum reference for App Transitions
App transitions, from ActivityManagerInternal, are now referenced via a
proto enum. This is also referenced by statsd's atoms.proto.

Bug: 69478930
Test: still compiles
Change-Id: Ifb5cb0120d4cd4e2464ce3b5a7455842cb55259c
2018-01-25 17:12:31 -08:00
TreeHugger Robot
a024f21852 Merge "Fix a bug of reading files with "_" separator" 2018-01-26 00:01:27 +00:00
Bookatz
1a1b0464cb Statsd and frameworks reference proto enums
For frameworks constants that don't have intrinsic meaning (i.e. their actual
value and order don't matter), so that it is unlikely that their values
will be changed:
This cl introduces proto enums representing some constants found in
the Android codebase, and connects the two.

By using the Proto enum as the source-of-truth, it means that Java and
proto can be kept in sync. Otherwise, when the Java frameworks code
changes, it silently breaks the protos from working properly, since the enums
are wrong. By having the Java code reference the proto enums, it ensures
that everything is in sync. The values of the constants are unchanged.
But future changes to these constants will need to be done in the proto
file, which the Java file merely references.

The protos are necessary for incidentd and statsd and, in the future,
possibly dumpsys. In this way, the logging mechanism is much less likely
to get broken when new constants are added, and we can be ensured that
the logging accurately reflects the underlying codebase.

Bug: 69478930
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases
Test: cts-tradefed run cts-dev -m CtsIncidentHostTestCases
Change-Id: If79032c34b2799db1e3e70cb47b1312fd72092b9
2018-01-25 15:47:57 -08:00
TreeHugger Robot
b93b797199 Merge "Update to the ActivityManager proto" 2018-01-25 22:21:30 +00:00
yro
50d23f1ed5 Fix a bug of reading files with "_" separator
Test: manual testing
Change-Id: Ibb19336122684712b3eb73088553acd9438b7611
2018-01-25 13:52:20 -08:00
yro
181c835f81 Disable statsd that was accidentally enabled in ag/3480570
Test: manual test

Change-Id: I57176203a1ca7d0b5639119642335e3e9a1e4e4c
2018-01-25 10:55:57 -08:00
TreeHugger Robot
53d179d246 Merge "Fix Perfetto.cpp to silence SELinux warnings" 2018-01-25 12:43:34 +00:00
TreeHugger Robot
d0c260ff41 Merge "Adding guardrails on writing to disk from statsd" 2018-01-25 06:47:29 +00:00
TreeHugger Robot
73a331f524 Merge "Statsd broadcast subscriber" 2018-01-25 06:23:36 +00:00
Howard Ro
540aa206e2 Merge "Revert "Revert "Revert "Enable init.rc for statsd to start statsd by default in boot time"""" 2018-01-25 04:32:50 +00:00
TreeHugger Robot
b5f12bbd2f Merge "Add statsd microbenchmark and fix a crash in LogEvent" 2018-01-25 02:04:11 +00:00
Howard Ro
62ac686168 Revert "Revert "Revert "Enable init.rc for statsd to start statsd by default in boot time"""
This reverts commit b00cce84d9ddba04a3ebd5ed45457c1e154be535.

Reason for revert: b/72459151 

Change-Id: I71593b4b323d69edb548dcb1bb0e1f4eb7cddbc9
2018-01-25 02:00:35 +00:00
Howard Ro
b2b547454b Merge "Fix adb shell cmd config remove command when no (uid, id) pair is specified." 2018-01-24 23:00:14 +00:00
TreeHugger Robot
82c2173b67 Merge "Statsd always includes snapshot of uid map." 2018-01-24 22:31:24 +00:00
Bookatz
c697797d43 Statsd broadcast subscriber
Allows a uid that uploads a statsd config to additionally
register a BroadcastSubscriber with statsd. If statsd
detects an anomaly (according to the config's Alert),
statsd can inform a BroadcastSubscriber provided in the config.
The config uses a subscriberId (just an int) to identify the
BroadcastSubscriber. It then uses StatsManager.setBroadcastSubscriber
to associate that subscriberId with a given PendingIntent.
Then, when the anomaly is detected, statsd sends a broadcast
using that PendingIntent, alerting whoever was specified by
the config/setBroadcastSubscriber.

Bug: 70356901
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.alert.BroadcastSubscriberTests
Change-Id: I4d9ea9a6c8a85e61fadfd99c1513c55abbadd5e9
2018-01-24 13:08:19 -08:00
Bookatz
db026a28aa Update to the ActivityManager proto
Rename the activitymanager.proto to enums.proto, in accordance with the
new convention on naming these protos.
Also, changes processStateAmToProto from simplying multiplying by 100 to
a switch statement, to be more future-proof.
Also, changes the values of the proto's process state, to reflect that
the values will not be kept in sync with ActivityManager.

Note that this cl was originally started in ag/3008434, and partially fulfilled in ag/3114337.

Bug: 69478930
Test: run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.ProcStateAtomTests
Change-Id: I2b3bf2552b879c74d8985338df5a57c01850cb91
2018-01-24 12:07:35 -08:00
Primiano Tucci
a74556061a Fix Perfetto.cpp to silence SELinux warnings
Turns out that statsd leaves some file descriptors opened
without the O_CLOEXEC flag. This CL mass-closes file descriptors
up to a realistic number of FDs. This is to avoid SELinux complaining
about perfetto accessing these files from the statsd domain.

With this change perfetto works with statsd without disabling SELinux.
Relevant SELinux CL:
https://android-review.googlesource.com/c/platform/system/sepolicy/+/598774

Change-Id: I745d621937fbc9b20a4c733948cd0dd24dd6e8b2
2018-01-24 19:56:44 +00:00
TreeHugger Robot
3f9a1a5426 Merge "Fix deadlock for write-disk cmd." 2018-01-24 19:51:22 +00:00
yro
98a28501fe Adding guardrails on writing to disk from statsd
- Limit total number of files to 1000
- Limit total size of files to 5MB
- Remove idle files to be deleted after 30 days

Bug: 69854160
Test: manual testing, statsd, statsd_test
Change-Id: I33148a3b7ca11d413ec2495d5c0659f1ba4485c3
2018-01-24 10:33:54 -08:00
TreeHugger Robot
adb4fc73b0 Merge "Avoid processing log event when there is no uid field." 2018-01-24 18:18:24 +00:00
Yao Chen
48d75189cc Add statsd microbenchmark and fix a crash in LogEvent
+ We should benchmark the core functions in the critical path.
     1. LogEvent parsing
     2. Log matching
     3. Event processing in various metrics
     ....

+ Using microbenchmark, we can quantify the optimization we do in the future.

    ---------------------------------------------------------
    Benchmark                  Time           CPU Iterations
    ---------------------------------------------------------
    BM_LogEventCreation       4772 ns       4705 ns     145738

Test: make -j64 statsd_benchmark

Bug: 72242322

Change-Id: I43aa704ffdc7cd21e02ef9038eff66ca7022dbfb
2018-01-24 10:02:36 -08:00
TreeHugger Robot
1a68be0f0d Merge "Simplify the memory metrics atoms." 2018-01-24 16:49:15 +00:00
Yangster-mac
635b4b3721 Do not output empty metric reports.
Test: statsd unit test passed.

Change-Id: I296edbe621eaea72ad2e1dfe656698302614c0f3
2018-01-23 20:30:46 -08:00
TreeHugger Robot
a65f508748 Merge "Fix the crash when parsing empty config." 2018-01-24 03:13:12 +00:00
David Chen
cfc311d2f0 Statsd always includes snapshot of uid map.
Statsd will contain at least one snapshot of the uid map. The
previous design was not very robust in case a snapshot was missing.

Also fixes subtle bug with updating the isolated uid mapping since
this should always be kept up to date even if there are no metrics
being used (since metrics may be added later after the isolated uid
was created).

Test: Checked that unit-tests pass on marlin-eng.
Change-Id: I99754ed9016d980564e409b0946a46b398fd12b7
2018-01-23 18:04:03 -08:00
Rajeev Kumar
27572b9a19 Simplify the memory metrics atoms.
Implement suggestions made in go/android-p-memory-metrics

Bug: 72177881
Change-Id: I8260a7053620d143f16600a852d808a3dbbe56f2
2018-01-23 17:39:08 -08:00
Yangster-mac
8617950962 Fix deadlock for write-disk cmd.
Test: manual tested.
Change-Id: I6c1e1f10bbb3830c932b3d7b57df8d4960c13977
2018-01-23 15:51:17 -08:00
TreeHugger Robot
7e004f96dd Merge "Atom: WifiMulticastLockStateChanged" 2018-01-23 18:09:20 +00:00
Yangster-mac
68985805f2 Avoid processing log event when there is no uid field.
Test: all statsd unit test passed

Change-Id: Id434d86586950a485b30a244f3c030e8202c1c6d
2018-01-23 16:43:07 +00:00