16 Commits

Author SHA1 Message Date
Yi Jin
ad3e6e5d5a This cl tries to fix cts tests IncidentdTest
1. Disable BatteryType section which is device-specific
2. Make timeout longer since meminfo section timedout in test
3. make some negative values sint
4. varint can be 64 bits, there is a bug implicitly convert it to 32
which loses values.
5. Found another bug which failed to read 64 bits varint, create a
native test to make sure it works.

Bug: 77291057
Test: atest CtsIncidentHostTestCases:com.android.server.cts.IncidentdTest
Change-Id: I04cc730741f7901f37ac57a11af7777d57118a23
2018-04-04 16:29:55 -07:00
TreeHugger Robot
56051569ab Merge "Use modern c++ code style for incidentd." into pi-dev 2018-04-02 18:01:15 +00:00
Yi Jin
6cacbcbf43 Use modern c++ code style for incidentd.
This cl does not contain code logic changes.

Bug: 77333635
Test: manual and incidentd_test
Change-Id: Iea0a402b1051defd45159ca267e6dd705f9ffa49
2018-03-30 17:36:49 -07:00
Yi Jin
00a754499e Correctly use int types and fix uint64_t print.
Bug: 74118023
Test: atest incidentd_test and atest incident_helper
Change-Id: I951021c12e3d0dc5c83a1bbb03fa0d64d99c0920
2018-03-30 17:06:46 -07:00
Yi Jin
5ee0787024 Use uint64_t instead of long long as API type for consistent reason.
Bug: 74118023
Test: manual
Change-Id: Icd5f506c76d3a008a79cb6c9d2061962ca7fdd40
2018-03-05 18:18:27 -08:00
Yi Jin
1a11fa1097 Implement a new section to attach LAST_KMSG to incident report
This section simply gzip a large file and stores result in GZippedFileProto

This greatly improves the size, before gzip, the last kmsg size ~500KB,
after gzip the proto size is ~60KB.

Bug: 73354384
Test: atest incidentd_test and manual on device test
Change-Id: I9bfc2cf07384487671edbffb5f0bd8495608fea6
2018-02-27 11:36:29 -08:00
Yi Jin
7f9e63b632 Add clear method to ProtoOutputStream so it is reusable by just
rewinding its internal pointer.

Bug: 72319159
Test: atest incidentd_test and atest statsd_test
Change-Id: Id046b987a7460d68300e5d4d46ae56aec1eba2d0
2018-02-09 14:20:48 -08:00
Yi Jin
295d9b1d43 Fix the tag size in token of ProtoOutputStream.cpp, the java version has
the correct tag size computed.

Bug: 72755317
Test: incident -p A 3005 | aprotoc --decode_raw
Change-Id: I0275dbecb566101b562b649c2ae3d58fbfb9188b
2018-02-02 14:33:20 -08:00
Yi Jin
0abdfb0889 Expose how many bytes are written in ProtoOutputStream
Bug: 69428467
Test: N/A
Change-Id: If1405885949635cebd1c6d89e220c210ef9fa210
2017-11-16 15:32:27 -08:00
Yi Jin
f9ed04b3d9 Remove the extra mapping of field types in ProtoOutputStream
It is very unlikely the protobuf changes the value in descriptor.h,
and if defines an extra mapping, there are several places to maintain:
1. java-stream,
2. cpp-stream,
3. ProtoOutputStream.java
4. ProtoOutputStream.cpp
5. Privacy.h (GetFieldId)
6. StatsLog to generate field id (type << 32 + field number)

Therefore use the current value in descriptor.h seems reasonable unless
they change that, very very unlikely, they probably will just add new
types, and deprect the existing ones like Group.

Test: test output of dumpsys proto
Change-Id: I6e150ab427851dd3b5dd55d3b273deeed7a0963c
2017-11-01 17:49:42 +00:00
Yi Jin
04625ad488 Refactor incident_helper to use protoutil and cppstream plugin.
1. Split the parsers to its own file to prevent all the parsers in one
gaint file.

2. Completely get rid of protobuf-cpp-full in incident_helper, use
ProtoOutputStream and cppstream instead, the incident_helper binary is
reduced from ~500K to ~113K.

3. Write data to protobuf even its values are zero/default, the reason
is for example we have a repeated int32 orders = 1; and people
explicitly append 0 so the total repeated field has 10 values, if zero
is not written to serialized data, this repeated field will only have 9
values which is not what we want at first place. This also aligns with
the default protobuf serialization behavior in incident_helper_test.

4. Use Android.bp for protoutil lib since it is not able to depend on
libs compiled by .mk file, it works the other way.

5. Add a new custom message option for streaming_proto, if specified,
the cppstream will create extra metadata to get field ids by field name.
A Table class is created in incident_helper to use it.

Bug: 67860303
Test: unit tested as well as on device test
Change-Id: I8e136fd15f343a4a623d20910ec64b622b478a3e
2017-10-31 16:54:38 -07:00
Yi Jin
8ad193870b Create a new API allowing callers to concat a serialized message field.
Test: N/A
Change-Id: I90570ea62e6c406d40c399cee6cbb8dcb5172e21
2017-10-30 17:54:00 -07:00
Yi Jin
e08333067a Fix several nits
1. Change the APIs of ProtoOutputStream to be able to write bytes
2. Fix the tests in incidentd, stdout is closed so can't capture the
value, use temporaryFile instead.

Test: N/A
Change-Id: Ibc31f2efd068afc6c06188d92f57ca5a754c3683
2017-10-23 16:36:06 -07:00
Yi Jin
42711a0b4f Use ProtoOutputStream to strip Pii sensetive fields in incidentd
The in-place algorithm in ProtoOutputStream will have at most 2 copies
of possible nested message values incidentd is getting which makes it
more ram-efficient, and minimize allocations.

Bug: 65641021
Test: the units pass
Change-Id: Ic3fe99b7e27895bd613a129ba0f12ccfca4af317
2017-10-13 15:44:26 -07:00
Yi Jin
974a9c2885 Implement go/streaming-proto native libs, part 2
Implement ProtoOutputStream.h to write protobuf format data.

Usage of dumping proto:
ProtoOutputStream proto(fd);

proto.write(fieldId, value1); // dump a single value

// start to dump a message:
long long token = proto.start(messageFieldId);
proto.write(nestedField1, nestedValue1);
...
proto.end(token);

fieldId will be generated by protoc-cpp plugin(TBD). It is an encoded uint64_t
value, with 0 - 32 bits as its proto number, 33 - 40 bits reserved for
field type, int32, bool, string, message, etc. and 41 - 43 bits for
single, repeated or packed type. Currently packed field is not
supported.

Bug: 65641021
Test: N/A, need to wait for protoc-cpp plugin and will test in
incident_helper
Change-Id: Ic188615b950235aae0edeee4876b78d31feb5619
2017-10-10 17:16:34 -07:00
Yi Jin
c23fad2f90 Implement c++ native lib for streaming proto, part 1
Extract protobuf class out and creates EncodedBuffer class
which holds protobuf data.
Next step is to create a ProtoOutputStream and let incident helper
adapt the change as well.
please see frameworks/base/core/java/android/util/proto

Bug: 65641021
Test: unit tested
Change-Id: I0dd343b2e62d60f091c8f857fae3452ec8da6b96
2017-09-27 12:23:51 -07:00