739 Commits

Author SHA1 Message Date
Yangster-mac
e2cd6d509b 1/ Duration anomaly tracker with alarm.
2/ Init anomaly from config based on the public language.
3/ Unit tests for anomaly detection in count/gauge producer.
4/ Revisit the duration tracker logic.

Test: unit test passed.
Change-Id: I2423c0e0f05b1e37626954de9e749303423963f2
2017-11-20 15:37:24 -08:00
yro
29f69e8f04 Remove IStatsCallback aidl from master
Test: statsd, statsd_test

Change-Id: Idd0fb945b62f3a139a0db07f1014a3c4a4e2f5f8
2017-11-20 11:13:19 -08:00
TreeHugger Robot
b92f1eed1b Merge "2 updates to storing configs on disk" 2017-11-20 03:44:51 +00:00
yro
9c98c05b6f 2 updates to storing configs on disk
1. When an existing config is updated, remove the outdated config from
disk to stop accumulating configs.
2. Check disk to remove configs to make sure we delete lingering files
that does not live on memory.

Test: statsd, statsd_test, manual
Change-Id: Iedce4b6eb99a3d36bb5e1d1ccc0d88c84859e8f5
2017-11-19 14:33:56 -08:00
Stefan Lafon
0afe084965 Merge "Update statsd_config.proto." 2017-11-19 05:04:25 +00:00
TreeHugger Robot
e6f7ed873c Merge "Fix a memory leak in LogEvent" 2017-11-19 04:58:53 +00:00
Yao Chen
8d9989bb37 The first step towards adding a guardrail.
Added a malloc debug cmd to find memory leak and where
the most heap allocations are with backtrace.

More info: malloc_debug/README.md

The util class is borrowed from audio server.

Usage:

adb shell stop
adb shell setprop libc.debug.malloc.program statsd
adb shell setprop libc.debug.malloc.options backtrace
adb shell start
adb shell cmd stats meminfo

Also,

adb shell dumpsys meminfo [statsd_pid]

also gives you the overall summary of the memory usage.

Test: manual

Change-Id: I4eb4167466a748ea0c3d9fd39e45d9900bc00fc2
2017-11-18 19:26:07 -08:00
Yao Chen
ae6a83a1cd Fix a memory leak in LogEvent
My previous understanding of the liblog api was wrong. Luckily it
is caught in time. Also added a memory profiling tool in another CL.

Test: statsd_test
      adb shell dumpsys meminfo
      adb shell cmd stats meminfo

Change-Id: I742af2a38e52e35edf0d4c77ccc35ede7019a73b
2017-11-18 18:56:52 -08:00
Stefan Lafon
cfed20bce8 Update statsd_config.proto.
Test: Ran statsd tests.

Change-Id: I3a83bdfb38be82d70ca547825152370df8bca9cc
2017-11-18 15:58:10 -08:00
TreeHugger Robot
43699661dd Merge "Don't just clear mPastBuckets when onDumpReport is called." 2017-11-18 23:46:35 +00:00
Yao Chen
30b6a20c99 Don't just clear mPastBuckets when onDumpReport is called.
This needs to be done properly. Left a TODO.

Test: statsd_test
Change-Id: I8537f0a870ab82039e0822c69096784403ee1651
2017-11-18 14:14:38 -08:00
TreeHugger Robot
744988cdce Merge "Fix the unit test failure." 2017-11-18 21:34:16 +00:00
Yao Chen
ff2632829f Fix the unit test failure.
Test: statsd_test
Change-Id: I7ab57d4eb8789523a2b435413fd35057602cc988
2017-11-18 12:03:42 -08:00
Howard Ro
fe945d189a Merge "Resolve segfault when running send-broadcast command while statscompanion service is not connected" 2017-11-18 18:39:16 +00:00
TreeHugger Robot
3f9278b7da Merge "Use ProtoOutputStream's bytesWritten to properly estimate the size of EventMetricsData on memory" 2017-11-18 03:50:01 +00:00
yro
75f0a4d394 Use ProtoOutputStream's bytesWritten to properly estimate the size of
EventMetricsData on memory

Test: statsd, statsd_test
Change-Id: I6fc90c5d9303fe6cab582afb03c480969a6dc5a5
2017-11-17 17:48:36 -08:00
TreeHugger Robot
4aa167736b Merge "One more feature in DurationMetric -- nesting on top of Or and Max" 2017-11-18 01:42:24 +00:00
yro
4d889e635d Resolve segfault when running send-broadcast command while
statscompanion service is not connected

Test: statsd, statsd_test

Change-Id: Ie98de1884f8d852c31c14db146d3da44cfc5aa8b
2017-11-17 17:36:07 -08:00
TreeHugger Robot
fe7a60e9ae Merge "Support StatsD sending broadcasts." 2017-11-18 00:48:34 +00:00
TreeHugger Robot
aedbfdfa48 Merge "Fix incorrect adb shell cmd help command for pull-source" 2017-11-17 23:06:50 +00:00
David Chen
1d7b0cd6b5 Support StatsD sending broadcasts.
StatsD will send a broadcast when we're 90% of the way to our
allocated memory limit for the configuration. If the memory usage
goes over the limit, we just lose all the data for this config.

Also modifies the adb shell commands to facilitate debugging of the
broadcasts.

Test: Manually tested on marlin-eng with custom gmscore code.

Change-Id: I517a15bd4c959aa221802f84a51f13141a725102
2017-11-17 14:57:50 -08:00
TreeHugger Robot
e8f0660c0b Merge "statsd: parse the new format of stats log" 2017-11-17 22:16:22 +00:00
yro
3fca5ba3a7 Fix incorrect adb shell cmd help command for pull-source
Test: statsd should pass, but it's not required as it's just an update
in help instruction.

Change-Id: I36f4e4794ae8066abdeaac98db5e679a3bcf0a06
2017-11-17 13:22:52 -08:00
TreeHugger Robot
13378dd01a Merge "fix the permission check crash in statsd." 2017-11-17 20:14:01 +00:00
Yao Chen
80235403d2 statsd: parse the new format of stats log
+ Changed how we construct LogEvent, now it's based on the context from log_msg
  without making a copy of the list.

+ All stats logs now have the same event tag, the atom id is the first elem in the log.

Test: statsd_test
Change-Id: I4419380f2ee1c2b2155d427b9f2adb78883b337f
2017-11-17 11:53:15 -08:00
Yao Chen
7ee94150ab fix the permission check crash in statsd.
And change the fake config uid to be 1000, so we can dump it from the test app.

Test: manual
Change-Id: I99b4cd28b351add8cf3e315e1dccf8cf511a0be8
2017-11-17 10:20:30 -08:00
Howard Ro
04dacb54cc Merge "Write statsd configuration to disk and add cmd to clear it" 2017-11-17 02:16:07 +00:00
yro
87d983cf6f Write statsd configuration to disk and add cmd to clear it
Test: statsd, statsd_test
Change-Id: Iba37a7f295256d24969185bdde6cbf28f9b89a55
2017-11-16 16:25:34 -08:00
TreeHugger Robot
5b968dbe29 Merge "Fix some bugs found in statsd" 2017-11-16 21:28:10 +00:00
Yao Chen
a81a0ffa52 Sync proto from google3
Default initial value for Condition changed to false

Test: statsd_test

Change-Id: I73ad80d873f2ff1264d18b1eec6d882ad0e9bb6f
2017-11-16 10:15:46 -08:00
TreeHugger Robot
62462871b5 Merge "Update statsd proto comment" 2017-11-16 18:01:22 +00:00
Yao Chen
d41c4220d0 Fix some bugs found in statsd
+ in log matcher, condition tracker and duration metric

Test: added unit test
Change-Id: Id633e856ba5453842487321d7ddc0c64100e4bb8
2017-11-16 09:25:53 -08:00
yro
de4ca10e10 Update statsd proto comment
Test: not needed.
Change-Id: I2f0c16f2e305624128eaca4658d5ccb9f2a20fe5
2017-11-15 22:58:01 -08:00
TreeHugger Robot
42753a512e Merge "Fix AOSP copyright years in statsd" 2017-11-16 03:34:55 +00:00
Yao Chen
0ea19901ef One more feature in DurationMetric -- nesting on top of Or and Max
+ Added counters to count the starts.

Test: added some unit test in statsd_test.

TODO: ADD MORE UNIT TESTS

Change-Id: I3575e921a4abab27c4ea50fffde19cc1d4564030
2017-11-15 16:45:59 -08:00
yro
0feae94bab Fix AOSP copyright years in statsd
Test: no test needed

Change-Id: I230b7d51eb00dafb16913d038475c3ac14b44096
2017-11-15 14:39:22 -08:00
Yao Chen
5c925ad04f Sync the DurationMetric proto from google3.
There are other statsd_config changes for Anomaly detection, but requires
code change. So will sync later.

Test: statsd_test
Change-Id: I927e99bd0763cb949fe339993b232f53a4eefe7a
2017-11-15 14:22:37 -08:00
Stefan Lafon
c783baca1e Merge "Rename "stats_events" as "atom"." 2017-11-14 22:03:54 +00:00
TreeHugger Robot
341775bfef Merge "Adds client API for interacting with statsd." 2017-11-14 20:25:09 +00:00
Stefan Lafon
ae2df01aae Rename "stats_events" as "atom".
Also make atom WakelockStateChanged use the "Type" enum instead of int32.

Test: Unittests are passing. Statsd is working.

Change-Id: I0909e2d97297f78996a81366d66aae62d5bf5ce1
2017-11-14 11:56:05 -08:00
Yangster-mac
d1815dc7b8 Sync statsd to the public proto changes:
1/ use string id/name for all metrics/configs/matchers.
2/ move alerts out of metric proto.

Test: unit test passed.
Change-Id: I08269f47be30f83f6dbadf873e7d62213449932b
2017-11-13 22:09:08 -08:00
David Chen
adaf8b344e Adds client API for interacting with statsd.
This API will primarily be used by GmsCore to send updated configs.
Also, sending a config will implicitly notify the StatsD that this
client wants to know when it should request data for this config.

We send a broadcast so that all interested subscribers can know if
data needs to be pulled.

Test: Manually tested that sending broadcast works via new adb
command added in StatsService.

Change-Id: I23cdd1df706036e14b32c3d01af30c3d4af819fa
2017-11-13 19:37:12 -08:00
Chenjie Yu
6736c893a7 Unit tests for ValueMetricProducer
StatsPullerManager is refactored so that we can mock it.
It may need more refactor pass to make is safer for longer runs.

Test: unit test
Change-Id: Ief0c99710e4d06e1454678f8b749c9599467d114
2017-11-13 13:42:17 -08:00
Chenjie Yu
e33bc3b967 add CpuTimePerUidPuller and CpuTimePerUidFreqPuller
Note: these pullers return monotonically increasing numbers.
The uids could include isolated uids that needs further translation in
statsd.

Test: manually tested on device and check print outs
Change-Id: I7097aec0417a3ac567b9b86149b757ac27cd58d8
2017-11-11 15:40:06 -08:00
Chenjie Yu
7f8def9d5a pull cpu time per freq
also adjust lock for ValueMetricProducer.cpp

Test: manual test
Change-Id: Ib96e3011d7bcf44ac92346d880196c425623a86f
2017-11-11 10:49:56 -08:00
TreeHugger Robot
87244f1f55 Merge "make statsd less chatty." 2017-11-11 01:03:59 +00:00
TreeHugger Robot
01528f4f2b Merge "1. Add FIELD_COUNT_REPEATED bit masks to repeated fields that were previously missing 2. Resolve segfault when serializing proto into vector of uint8_t 3. Add a local byte size variable for EventMetric data as we cannot call size() of ProtoOutputStream before we finish writing. 4. Replace hardcoded conversion of second to nano second with NS_PER_SEC" 2017-11-11 00:46:54 +00:00
Yao Chen
4e6364ac1c make statsd less chatty.
Test: builds and adb logcat
Change-Id: I06ee9cce1844b21ed3960bbaf76262fca3357bf4
2017-11-10 14:35:58 -08:00
yro
b0378b093d 1. Add FIELD_COUNT_REPEATED bit masks to repeated fields that were previously missing
2. Resolve segfault when serializing proto into vector of uint8_t
3. Add a local byte size variable for EventMetric data as we cannot call
size() of ProtoOutputStream before we finish writing.
4. Replace hardcoded conversion of second to nano second with NS_PER_SEC

Test: statsd, statsd_test

Change-Id: I47a2ce2b05e6191c18596489682118edcb41e945
2017-11-10 11:09:20 -08:00
TreeHugger Robot
51f29ce476 Merge "Make SimpleCondition satisfactorily complicated." 2017-11-10 18:32:57 +00:00