816 Commits

Author SHA1 Message Date
TreeHugger Robot
77f45d4d5c Merge "Log deferred job stats." 2018-09-07 03:31:52 +00:00
Yangster-mac
96353008d9 Log deferred job stats.
Test: statsd tests

BUG: b/113353829
Change-Id: If0c07f5c1baccf831fea39982b25b64b6fc88001
2018-09-05 17:33:07 -07:00
Josh Gao
3077025818 Merge "statsd: fix double close." am: 0b8f17b3ba am: 6547dad5ac
am: 86b0140271

Change-Id: I0377db4c1e5343355882e1cfea94d7e4d04159d1
2018-09-05 13:12:54 -07:00
Marcin Oczeretko
f1d4563acd Merge "Collect telemetry data for System Server Loopers." 2018-09-05 19:12:45 +00:00
Josh Gao
3330d1208f statsd: fix double close.
Previously, fdopen was being called on a file descriptor that was owned
by a unique_fd without releasing. This leads to a double close, since
both fclose and the unique_fd destructor will try to close the fd.

Bug: http://b/113880863
Test: treehugger
Change-Id: I6f6f48d304861b5e4d7efee0d3ad0e30178a95a4
2018-09-04 11:12:04 -07:00
Marcin Oczeretko
d8cc8593c0 Collect telemetry data for System Server Loopers.
Adds LooperStats and LooperStatsService which set a static
Looper.Observer to record the count of dispatched messages and
cpu/latency information. The aggregated stats are collected by
Westworld as pulled atoms.

The collection is disabled by default and requires a manual call to
enable:

adb shell cmd looper_stats enable

Test: Unit tests and tested manually that the collected data seems
reasonable.

Change-Id: I7162fbca4e324f62887f95df3405205cbe4416ca
2018-09-03 16:06:06 +01:00
TreeHugger Robot
f75320d841 Merge "Add an enum file for stats event" 2018-09-01 04:21:20 +00:00
Adam Bookatz
cb0fba36aa Merge "statsd: add Vibrator atom" 2018-08-31 23:31:05 +00:00
Howard Ro
0546d54fb4 Add an enum file for stats event
The list of events in this file is meant to be used with Generic Atom
defined in atoms.proto in statsd. Instead of unique int, we would like
to log it with an enum that shows semantic meaning of events. Also, this
change re-sorts imports and removes unused imports and remove the
previous logging line to test GenericAtom.

Test: statsd_test
Change-Id: Ia9335e48ada68bee6d716ad4d7ac50238932da95
2018-08-31 16:21:28 -07:00
Howard Ro
e60992b873 Fix a typo in StatsService
Test: I can promise that this change doesn not cause further breakage.

Change-Id: I7964fa2bfa6ad6b7e56fce0867ab6fdda9a24922
2018-08-30 14:38:46 -07:00
Bookatz
a7020bd9d0 statsd: add Vibrator atom
Adds VibratorStateChanged atom.

Test: run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.UidAtomTests#testVibratorState
Bug: 113352975
Change-Id: I7b2b80d6f13306d5b60e17c5c143e0c687718369
2018-08-29 12:47:11 -07:00
Tej Singh
f156ea7353 Merge "Revert "DiskStats Westworld Migration"" 2018-08-29 17:15:17 +00:00
Tej Singh
6188aa3294 Revert "DiskStats Westworld Migration"
This reverts commit f154cf018994bc8a814996e4e0345c6388507054.

Reason for revert: Temporary rollback to allow ag/4819964 to go into pi-dev (and into master).

Change-Id: I607ec64aa9da3b0b1adf49bd361f9ed4927dba05
2018-08-29 00:21:07 +00:00
Tej Singh
008ecd3de0 Merge "DiskStats Westworld Migration" 2018-08-27 18:58:06 +00:00
Yao Chen
0f5d661803 Add more accurate logging for stats log loss.
+ Only record eventual failure after the retries to get a accurate log loss count.
+ Record all types of failures which lead to log loss
+ Change the timestamp from elapsedRealtime to wallclock time for easier debugging server side
+ Also log the count too.

Bug: 80538532
Test: manually tested

Change-Id: I4fcccae3fa39c9e280a842e27c6432bb0a090b85
2018-08-22 14:21:42 -07:00
Tej Singh
f154cf0189 DiskStats Westworld Migration
Migrates the existing information collected by diskstats to westworld.
Creates 4 pulled atoms, 2 of which read from a cached file produced by
DiskStatsLoggingService. Most of the logic is taken from
DiskStatsService.java.

Test: Manually verified information is the same as dumpsys. Will look
into cts as well.

Change-Id: I9818ac787de46514bc09ab3887cbfaec6cff273c
2018-08-21 10:35:39 -07:00
Yangster-mac
48b3d62bfe Create log event from key value maps.
BUG: b/112816333
Test: statsd test.
Change-Id: Ib66f06186abfacd77807436379e1e142a5b87c99
2018-08-19 22:37:59 +00:00
TreeHugger Robot
a5f28237e4 Merge "allow statsd pull based on event trigger" 2018-08-11 20:09:22 +00:00
Chenjie Yu
8858897205 allow statsd pull based on event trigger
Several restrictions:
1) This is only with GaugeMetric. We don't have use case for ValueMetric
to pull on event trigger.
2) trigger_event is set in the config. It can be generic atom matcher. But
we limit the number of atoms referenced in it to be 1. So we don't allow
multiple atoms to form a complex trigger.
3) This has to go with ALL_CONDITION_CHANGES sampling type.

+ also specify atom id of GaugeMetric output.

Bug: 111937835
Test: unit test
Change-Id: Ia15b1f209945f022edffb9ec5d673317d55d9e4f
2018-08-10 20:49:52 -07:00
TreeHugger Robot
632b39288d Merge "Remove the obsolete code for logd and add statsd socket log loss detection." 2018-08-11 00:30:47 +00:00
Yangster
8a34384797 Return unknown for combination condition eval when operation is NOT and
there is no child.

Test: added unit test and rerun the statsd tests.

BUG: b/112311529

Change-Id: I0c5829e3cb26474b7dbcc05f20c4311e9f801d97
2018-08-10 04:30:11 +00:00
Yao Chen
3ff3a490e4 Remove the obsolete code for logd and add statsd socket log loss detection.
+ Remove dead code
+ Add a simple log loss detection as a starter to see if there is any log loss
  detected at all.

TODO: If we do see log loss, we can add more sophisticated logging and reset mechanism.

Bug: 80538532
Test: statsd_test
Change-Id: Iff150c9d8f9f936dbd4586161a3484bef7035c28
2018-08-06 16:24:49 -07:00
Howard Ro
21a039cb8b Move GenericAtom from pulled atom group to pushed atom group
Test: no test needed
Change-Id: I67fb88e6707a8196d379d37d170aa1199abf849a
2018-08-06 14:55:47 -07:00
TreeHugger Robot
4c3b325d7b Merge "make proto change backwards compatible" 2018-07-30 16:28:08 +00:00
TreeHugger Robot
4481c7c2f5 Merge "Adjust 1st bucket start time" 2018-07-27 21:28:38 +00:00
TreeHugger Robot
f94dcd9818 Merge "Update GenericAtom definition to decouple from metrics_constants proto" 2018-07-27 19:26:46 +00:00
Howard Ro
de7130bbbb Update GenericAtom definition to decouple from metrics_constants proto
Bug: 110537998
Test: statsd, statsd_test
Change-Id: Ib82b90731f7c49a9e27ceff5d5fdb8e2cd6fb4e6
2018-07-27 11:01:55 -07:00
Chenjie Yu
e1361ed422 Adjust 1st bucket start time
adjust 1st bucket start time for a partial bucket
also make valuemetric and gauge metric pull on first bucket

Bug: 111607838
Bug: 111660710
Bug: 111842941

Test: unit test
Change-Id: I5932c2258f8deac57e7abbf26f3214f87914a964
2018-07-27 10:53:38 -07:00
Chenjie Yu
c3f0077cfc make proto change backwards compatible
use [deprecated] rather than reserved as this change is for Q and we
expect data from P to come in for a long time.

Bug: 111889247
Test: cts tests that rely on this proto works correctly
Change-Id: Id64b40ab1566f02fc0e19f617ac58bab3adafab5
2018-07-26 13:54:38 -07:00
Olivier Gaillard
6f52d1572d Pull more data from binder calls.
Exceptions counts by class name and few new fields for binder calls.

> adb shell cmd stats pull-source 10023
Pull from 10023: { 1532528725000000000 49754068496 (10023)0x10000->java.lang.SecurityException[S] 0x20000->2[I]  }
Pull from 10023: { 1532528725000000000 49754068496 (10023)0x10000->java.lang.IllegalArgumentException[S] 0x20000->16[I]  }

Test: manual
Change-Id: I4d24528a7df8edde87f629837fb3117a0504d09e
2018-07-25 23:43:19 +01:00
Olivier Gaillard
9ea238d2a4 Update BinderCalls and add BinderCallsExceptions.
Add a new field recorded_call_count since binder calls now supports
random sampling.

Add BinderCallsExceptions to track which exception classes are thrown by
binder calls.

Test: n/a (proto changes only)
Change-Id: I8f9f9a027fd950706112a9deef39ac01cacaa24d
2018-07-24 18:31:37 +01:00
Chenjie Yu
bd1a28f45c Atoms for ProcStats duration and pss metrics
atom and logging for statsd to mimic procstats process duration metrics.

ActivityManagerSleepStateChanged
MemoryFactorStateChanged
ExcessiveCpuUsageReported
CachedKillReported
ProcessStateChanged
ProcessMemoryStatReported

Some of the enums will need to use frameworks proto. It is blocked by
other cls that are cleaning namespace, etc.

Bug: 110784286
Bug: 110538804
Bug: 110539297
Bug: 111561536
Bug: 111685402
Bug: 111128767

Test: cts test
Change-Id: I7502eb0bb36b90f43493b147a35bee48bad1b000
2018-07-23 14:22:45 -07:00
Primiano Tucci
65c72fc53c Remove unnecessary perfetto/perfprofd messages from statsd_config proto
This change simplifies the on-device statsd_config.proto when
it comes to handling perfetto and perfprofd configs.
In both cases statsd doesn't need to deserialize or know the schema
of the nested perfetto/perfprofd configs, because it just passes
the binary-encoded message to the corresponding daemons.
This change replaces the submessage with a generic "bytes" field.

As per https://developers.google.com/protocol-buffers/docs/proto#updating
"Embedded messages are compatible with bytes if the bytes contain an
encoded version of the message."

Advantages:
* One less copy of perfetto/perfprofd config around, reducing the risk
  of getting that out of sync and the corresponding maintenance cost.
* Reduce the risk of failures within statsd if trying to parse an invalid
  or outdated config proto.
* Reduce the CPU usage of statsd, by avoiding deserialization and
  re-encoding of the config protos.
* Avoid bugs like b/111448265 where the binary size of statsd inflates
  if the config protos are too big.
* Reflect what happens in the statsd code, specifically the fact that
  statsd only sees the bytes of the config but doesn't touch/alter its
  contents.

This change depends on: aosp/718808 and ag/4569627

Bug: 111448265
Test: Manual + CTS (atest AnomalyDetectionTests#testPerfetto)
Change-Id: I6ede5fa07c3ab3f71d29d38f2a40a90e88a16fd6
2018-07-19 22:46:23 +00:00
TreeHugger Robot
a6b034c093 Merge "Fix timestamp error in GaugeMetric" 2018-07-17 00:38:36 +00:00
Yao Chen
f4be88f780 Fix timestamp error in GaugeMetric
Bug: 111516208
Test: unit tests pass, manually tested too
Change-Id: Ia268d3a38c331daab0f96c33461bb7dc794ce610
2018-07-16 15:39:18 -07:00
TreeHugger Robot
b4fc58bab7 Merge "Definition of GenericAtom" 2018-07-14 09:45:12 +00:00
Howard Ro
cb767f659c Definition of GenericAtom
Test: no test is necessary as it only adds a field to atoms.proto
Change-Id: If4e7c9497d1a4a8ba0fda3e8fb1ef67c525d6e64
Bug: 110537998
2018-07-13 14:41:26 -07:00
Chenjie Yu
a0f0224906 ValueMetric supports multiple aggregation types
1. Add support for MIN, MAX, AVG
2. ValueMetric also allow floats now, in addition to long data type.
AnomalyDetection still takes long only. I am not sure if it makes
sense to do anomaly on AVG. I will leave that for later.
3. ValueMetric supports sliced condition change for pushed events.
I don't think it makes sense for pulled events to have sliced condition
changes so leave it for now.

Test: unit test
Change-Id: I8bc510d98ea9b8a6eb16d04ff99dce6b574249cd
2018-07-13 10:24:41 -07:00
Olivier Gaillard
00bfb1b95a Collects binder call stats data through WestWorld.
We require binder calls detailed tracking to be enabled to collect the
stats (in addition to enabling it in WestWorld).

Test: unit test + manual

adb shell cmd stats pull-source 10022
Pull from 10022: { 1531240941000000000 25807560798 (10022)0x10000->0[I]
0x20000->com.android.server.StorageManagerService$3[S]
0x30000->onVolumePathChanged[S] 0x40000->1[L] 0x50000->0[L]
0x60000->18490[L] 0x70000->18490[L] 0x80000->2611[L] 0x90000->2611[L]
0xa0000->0[L]  } ...

Change-Id: I07cad5d8678426cdac45872cda028ea7a85d7d81
2018-07-11 10:44:49 +01:00
TreeHugger Robot
3e2e1ae482 Merge "Clean up TODOs in statsd" 2018-06-26 17:40:06 +00:00
Yao Chen
5bfffb54da Clean up TODOs in statsd
+ Created bugs for those TODOs that are still relevant.
+ Remove obsolete TODOs.

Test: no code change.
Change-Id: I41c2a89a882f087817ee6cbc3f095e1d80e1928e
2018-06-25 11:08:04 -07:00
Andreas Gampe
9e4e689f8f Statsd: Update perfprofd config proto location
Link to new location.

Bug: 110555909
Test: m
Change-Id: I306e1dc740886d1227787604a48158d9f0c3c40a
2018-06-25 09:59:26 -07:00
Chenjie Yu
e22192071d StatsPullerManager not use singleton
This is to be consistent with other patterns such as UidMap.
This also makes unit test simpler.

Change-Id: I1558cd609e470481f269ecf2ae616277a95cfbf0
Bug: 72722120
Test: unit test
2018-06-14 15:46:54 -07:00
Yao Chen
a11cbf368b Merge "Disable statsd and make StatsLog no-op if ro.statsd.enable=false" into pi-dev am: 38b92986df
am: 39c8fe12e4

Change-Id: Ie27a0d43da9fa0eb0c751cafa3f04220a5f8f18a
2018-06-07 15:50:46 -07:00
Yao Chen
38b92986df Merge "Disable statsd and make StatsLog no-op if ro.statsd.enable=false" into pi-dev 2018-06-07 21:10:16 +00:00
Yao Chen
cf3829a696 Disable statsd and make StatsLog no-op if ro.statsd.enable=false
Bug: 79324611
Test: manually tested on marlin_svelte
Change-Id: Ic7fbf86d7c295cb6b2f20e7f9a957d1ed74c42c0
2018-06-06 17:25:08 -07:00
TreeHugger Robot
6976b764ad Merge "Wean Statsd off BatteryStats: Network Power" 2018-06-05 19:57:10 +00:00
TreeHugger Robot
4e0524bf32 Merge changes Iebe457a5,If32da8e6
* changes:
  Statsd: Add perfprofd call
  Statsd: Add perfprofd_config proto
2018-06-05 16:40:19 +00:00
Bookatz
0b028b1dc0 Wean Statsd off BatteryStats: Network Power
Statsd calls were sometimes made from BatteryStats, instead of directly
from the source, as a way of getting it running quickly in P. These must
now be done properly and separately.
Note that Statsd needs to be independent of BatteryStats, in preparation
for deprecating large parts of BatteryStats.

Bug: 80308558
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases
Change-Id: I54fa54a2dc59dfa68f0e063d7063fac423bf0bf4
2018-06-01 10:52:36 -07:00
TreeHugger Robot
f74c983280 Merge "Allow stats log printing from userdebug builds too." 2018-05-30 17:07:15 +00:00