88 Commits

Author SHA1 Message Date
Jeff Sharkey
1f8ea2dcd1 Send broadcast when changing restrict background.
When changing global restrict background status, send connectivity
change broadcast, since it radically changes DISCONNECTED/BLOCKED
status system-wide.

Also reduce verbose stats logging.

Bug: 5854466
Change-Id: I3b612c520f50cc3000a3a569b7e0ab5f691cc2bd
2012-02-07 12:08:58 -08:00
Jeff Sharkey
706498d387 List rotated netstats files during --checkin dump.
Also disable verbose logging.

Change-Id: I4c51e05d695eac43f0d9c28f9699f1655c642a99
2012-02-06 17:35:11 -08:00
Jeff Sharkey
241dde2306 Unify shorthand for byte-based units.
Change-Id: If990859dee3f0973e1d4c48f05312c84071b3328
2012-02-03 14:50:13 -08:00
Jeff Sharkey
0e2e5f8b56 Support snoozing of data warning notification.
Introduce second snooze timestamp to support dismissal of data usage
warning notification.  Also explicitly set notification "when" to
avoid fighting with other notifications, such as ongoing downloads.

Bug: 5443756, 5262414
Change-Id: I03342c25b0410b1b2db84de9a40884f04cb1d8ae
2012-02-02 16:22:33 -08:00
Jeff Sharkey
25ce9ed57f Avoid NDC deadlock when re-arming network alert.
When NDC delivers limitReached() event, NMS re-arms the alert, which
makes another call back to NDC.  If NDC already started processing
another call, this causes deadlock.

Fix this by re-arming on a handler thread.

Bug: 5879848
Change-Id: I46fa78fe1f1122c5f31dd6020483e5ae89a9f39d
2012-02-02 13:09:52 -08:00
Jeff Sharkey
e630f7b31b Clear identity when updating network stats.
Bug: 5945461
Change-Id: Ie51ea3f107a705c395b65684f21bf4c28c7f4d33
2012-01-31 17:13:09 -08:00
Jeff Sharkey
63abc37356 Move network stats to FileRotator pattern.
Split existing network stats into two separate classes: a recorder
which generates historical data based on periodic counter snapshots,
and a collection of historical data with persistance logic.

Recorder keeps a pending history in memory until outstanding data
crosses a specific threshold.  Persisting is handled through a given
FileRotator.  This pattern significantly reduces disk churn and
memory overhead.  Separate UID data from UID tag data, enabling a
shorter rotation cycle.  Migrate existing stats into new structure.

Remove "xt" stats until iptables hooks are ready.  Avoid consuming
Entry values when recording into NetworkStatsHistory.  Assign
operation counts to default route interface.

Introduce "Rewriter" interface in FileRotator with methods to enable
rewriteAll().  Introduce IndentingPrintWriter to handle indenting in
dump() methods.

Bug: 5386531
Change-Id: Ibe086230a17999a197206ca62d45f266225fdff1
2012-01-24 11:13:14 -08:00
Jeff Sharkey
5a7bcf31a4 Move non-monotonic reporting to interface.
Report non-monotonic NetworkStats through an observer interface
instead of throwing, since those events are still recoverable.

Change-Id: Ic0749f4634b0ac05dbe90e95ca490957ec8b2f23
2012-01-10 17:24:44 -08:00
Jeff Sharkey
f60d0afd1e Restrict app data on metered networks.
This separates the definition of "metered network" and "network with
limit."  For now, all mobile networks are considered metered.

Bug: 5571454
Change-Id: I394cd385bd33add75e53bfc9cf2fefd06a00208a
2011-12-15 14:30:04 -08:00
Jeff Sharkey
4529bb66c4 Move NMS to CONNECTIVITY_INTERNAL permission.
Change-Id: I6043c1b9cb10aeeafb93d1b5c60da571d2fec5c9
2011-12-14 17:51:13 -08:00
Jeff Sharkey
17b2890487 Merge "Avoid reading network stats when disabled." 2011-12-14 10:12:17 -08:00
Jeff Sharkey
418d12dc8f Avoid reading network stats when disabled.
Bug: 5518868
Change-Id: Idb5c588b018bf735597ad7ff904ecc69224dd0a0
2011-12-13 15:43:47 -08:00
Jeff Sharkey
d04ff11d6f Avoid NPE when missing network stats.
Bug: 5750339
Change-Id: I4b739dbd076b270a5a2b2e2a1e0f6e445d8c124e
2011-12-12 18:51:26 -08:00
Jeff Sharkey
c506ff6150 Only write network stats when already loaded.
Bug: 5584564
Change-Id: Ife2a68804bf3691e9bccdc167807204bce8a85b2
2011-11-17 12:05:46 -08:00
Jeff Sharkey
684c54a2eb Update NTP outside locks, log large trim events.
Move NTP updates outside locks to avoid blocking ShutdownThread.  Add
logging around trim events that remove substantial history.  Include
history start in sample events.

Bug: 5627247, 5584564
Change-Id: If9bbd93842c710efcdec94e68da87e9e42a5f961
2011-11-16 17:58:47 -08:00
Jeff Sharkey
7ee8658453 Conservatively trim data usage stats.
Instead of trusting NTP time alone, use the most-conservative of
system clock and NTP.

Bug: 5584564
Change-Id: I5dd87fc009959b1cf0a7d660e385a0b1a8be238b
2011-11-14 20:11:38 -08:00
Jeff Sharkey
d4ef8c8fc9 Clamp non-monotonic stats instead of dropping.
When encountering non-monotonic stats rows, recover remaining data by
clamping to 0.  In particular, this avoids edge-case where persisting
threshold checks would never trigger.  Also recover when tethering
snapshots are missing.

Bug: 5600785, 5433871, 5600678
Change-Id: I1871954ce3955cc4ac8846f9841bae0066176ffe
2011-11-10 17:54:23 -08:00
Jeff Sharkey
630a13bdd8 Merge "Bootstrap stats before registering listeners." into ics-mr1 2011-11-08 23:43:58 -08:00
Jeff Sharkey
4c48d2ff69 Merge "Make operation counts monotonically increase." into ics-mr1 2011-11-08 23:43:38 -08:00
Jeff Sharkey
3359aca765 Bootstrap stats before registering listeners.
Also move ISE to handled NonMonotonicException.

Bug: 5570872
Change-Id: I793f349287ef5589185f117dfbec6f1fad73a661
2011-11-08 18:24:12 -08:00
Jeff Sharkey
7e25b0ea88 Dispatch alerts through background thread.
Bug: 5557659
Change-Id: Icc3ad476a98ec3430b33f915723ffcc36ccbfae7
2011-11-08 17:41:26 -08:00
Jeff Sharkey
4abb1b8ef6 Make operation counts monotonically increase.
Bug: 5585704
Change-Id: I5247a68acdb209a4ef0f5aac18944595c295e6ca
2011-11-08 17:35:28 -08:00
Jeff Sharkey
3f3115b277 Make policy data enabled flag static.
For devices with both CDMA and GSM stack, ConnectivityService only
connects with the GSM variant.  Making this flag static communicates
the policy state between all DCT.

Bug: 5586935
Change-Id: Iff0384027303470dd382d5173558d2d091ce4bf6
2011-11-08 17:15:46 -08:00
Jeff Sharkey
163e6443f2 Correct proc file reader, optimizations.
Moved away from BufferedReader, which only reads the first 8KB of
some proc files because it aggresively fills its buffer.  Optimized
proc parsing, now double the speed.  Tests to cover.

Log when NetworkStats counters roll backwards when subtracting, and
optimizations around findIndex().  When system removes UID, also
remove from last stats snapshot to avoid xt counters from rolling
backwards.

Bug: 5472949, 5458380
Change-Id: I07c08fe5233156fac2b84450f6291868bf9bfaf2
2011-11-02 15:14:11 -07:00
Jeff Sharkey
6f7af03cf1 Process AMS events in NetworkPolicy handler.
When a process changes foreground status or dies, NetworkPolicy
updates its internal state with a lock held.  In cases where there
is contention, this can block the AMS handler and prevent other
events, such as broadcasts, from being dispatched.

This change moves the incoming AMS events to an existing internal
NetworkPolicy handler thread, where they can execute without
blocking AMS.

Bug: 5497544
Change-Id: Ie0c620a620fd9f0f4eb02af510bd819efa4deb6a
2011-11-01 19:37:34 -07:00
Jeff Sharkey
50e7e51d68 Over-limit within handful of MTU's, update assets.
Consider a network over-limit when it's measured usage is within a
handful of MTU's, since kernel completely drops packets that trigger
limit alert.

Update notification drawables.

Bug: 5433359, 5224629
Change-Id: I10b19cc34c34b47775904229829a25208cd85d25
2011-10-10 16:57:21 -07:00
Jeff Sharkey
293779f9c6 Refine random stats generation.
Offer parameters to pick out specific list of packages, and to change
duration and total data.

Bug: 5416742
Change-Id: I6970a5e76f3198ef23a400c079037212b000ca6b
2011-10-05 23:31:57 -07:00
Jeff Sharkey
905b5891d2 Track xtables summary, move tether stats, time.
Begin tracking xtables summary of data usage to compare with values
reported from /proc/net/dev.  Roll tethering directly into UID stats
to trigger UID stats persisting when crossing threshold.

Include xtables summary and authoritative time in samples.

Bug: 5373561, 5397882, 5381980
Change-Id: Ib7945522caadfbe0864fdf391582dc820f4f371e
2011-10-03 17:21:05 -07:00
Jeff Sharkey
367d15ab1a Watch network subtype, tethering teardown, empty.
Watch for changes to telephony network subtype, and update iface
mapping to persist stats under correct type.  Update network stats
before removing tethering NAT rules.

Skip recording that would create empty historical buckets.  Query UID
stats before iface stats to always skew positive when counters are
actively rolling forward.

Bug: 5360042, 5359860, 5335674, 5334448
Change-Id: I8aa37b568e8ffb70647218aa1aff5195d3e44d5a
2011-09-22 16:19:06 -07:00
Jeff Sharkey
92cefaeb31 Upload correct txPackets count.
Bug: 5321340
Change-Id: Ie4dc7a91bf1cdc4d64c75ff09ae0cdd0fb141b5d
2011-09-19 18:37:11 -07:00
Jeff Sharkey
1f0b13b9d6 DO NOT MERGE: Sample atomic network stats buckets, full poll.
When sampling network stats, always use atomic buckets instead of
interpolating.  Always poll iface and UID together so we distribute
into buckets equally.  Move stale bucket trimming to just before
writing stats.

Bug: 5321340
Change-Id: I78a2226778a79c875f3668336e39ea24a7b4d5c4
2011-09-18 16:17:51 -07:00
Jeff Sharkey
cdd02c5d76 Collect and persist tethering stats.
Use new "gettetherstats" netd command to retrieve statistics for
active tethering connections.  Keep tethering poll events separate
from UID poll, even though they end up same historical structures.

Bug: 5244846
Change-Id: Ia0c5165f6712c12b51586f86c331a2aad4ad6afb
2011-09-16 16:05:51 -07:00
Jeff Sharkey
fc0ab4c065 Merge "Network stats flag to force complete poll." 2011-09-14 21:44:19 -07:00
Jeff Sharkey
8ca953da93 Change data usage warning notification wording.
Bug: 5308710
Change-Id: Ic807640a031a3edb03897634e638473d5a039ecc
2011-09-14 19:56:11 -07:00
Jeff Sharkey
991d1b1b3f Network stats flag to force complete poll.
Devices currently delay collecting UID stats during common operations
like global alerts to reduce parsing load.  Here we introduce a flag
to always collect UID and iface stats together to aid debugging.

Bug: 5321340
Change-Id: Ia8e1daf1a6f9d5f5cf1f64a04b6b6766ee965ab6
2011-09-14 19:47:05 -07:00
Jeff Sharkey
b3d5957604 Poll UID stats during iface changes, enforce idx.
When ifaces change, poll UID stats without persisting, since they
depend on knowing active iface to store correctly.  Log dropped UID
stats when iface is unknown.  Switch to using flags when calling
performPoll().

Enforce that "idx" values are consistent from xt_qtaguid.  Transition
to using Log.wtf() for important checks, mostly around file I/O,
kernel stats parsing, and kernel module control.  Increase stats
persist threshold to 2MB to reduce churn.

Bug: 5269476, 5270106
Change-Id: I721215bfb65127f95775c71cf135e907cd567e92
2011-09-07 18:06:30 -07:00
Jeff Sharkey
07b0dd9a09 Sample network statistics for sanity check.
After detailed poll events, sample high-level network statistics to
sanity check iface totals against UID totals.

Bug: 5248382
Change-Id: I1c1fbb7c094c9ff0c9dde416467c563dda68f478
2011-09-01 15:07:19 -07:00
Jeff Sharkey
961e304583 Introduce immediate CONNECTIVITY_ACTION variant.
New broadcast that is dispatched immediately after connectivity
changes have been handled by ConnectivityService, bypassing any
applicable CONNECTIVITY_CHANGE_DELAY.

Also protect CONNECTIVITY_CHANGE broadcasts, since they should only
be sent by system.

Bug: 5198167
Change-Id: I75f1fb44b21da1879f0ab960bcaa481126d70fde
2011-08-29 16:10:43 -07:00
Jeff Sharkey
47eb102b40 Always splice historical data stats, debug info.
When reading network counters, always splice in xt_qtaguid values
to avoid counting backwards. Test to verify.

Remove verbose logging around global alerts, and add dumpsys debug
info for NMS and MDST. Also fix subtle bug around stats persisting
and dumping.

Bug: 5212893, 5211028
Change-Id: I783e5286637a67ee2dd2b09878198711a926d0cb
2011-08-26 14:19:54 -07:00
Jeff Sharkey
8e9992ae50 Register for kernel global data usage alerts.
Instead of polling every 15 minutes, register for alerts that trigger
when system-wide traffic passes a threshold.  Still mixed with polling
to persist UID stats, but relaxed to 30 minutes.  Currently watches
for every 512kB.

Make persistence decision separately for network versus UID, and use
total delta bytes when making decision.  Use light bootstrap during
systemReady() instead of heavy poll, which had been force-loading all
UID data unnecessarily.

Bug: 5023631
Change-Id: I04b723d6c4bf872fb1028071122dba66a8e1b576
2011-08-23 18:46:14 -07:00
Jeff Sharkey
8e28b7d782 Proactively disable data when over policy limit.
Add policy controls to NetworkStateTracker which are combined with
other user preference and internal flags to decide if data connection
should be established.  Better locking around enabled flags.

When data network would be over limit, proactively disable data on
that network.  Enable when policy is snoozed or when cycle resets.

Track and dismiss notifications from now-stale policies.

Bug: 4587023, 5178147
Change-Id: Ibfcc9f73cda7c369209af701b46eddd3d1943f2d
2011-08-23 18:45:44 -07:00
Jeff Sharkey
3a844fcf5a Background data notification, API clean up.
When restricting background data, show ongoing notification to give
easy access to re-enable. Deprecate getBackgroundDataSetting() API
to always return true, since NetworkInfo.isConnected() is new source
of truth. Handle upgrade path by reading from existing secure value,
and kick one last broadcast when changing value. Remove background
data code from ConnectivityService.

Remove warning alerts, since they push ifaces into restricted list;
should only happen when iface has limit.

Bug: 5163559, 5129421
Change-Id: I0064d9d643656a4d32aaae51d4a58bce49fe295f
2011-08-17 14:12:42 -07:00
Jeff Sharkey
b5d55e302d Foreground/background network stats using sets.
Teach NetworkStats about "counter sets" coming from kernel, and use
them to track usage in foreground/background.  Add AID_NET_BW_ACCT to
system_server so it can control counter sets.

Move to composite key of NetworkIdentitySet, UID, set, and tag when
recording historical usage.  Persisting still clusters by identity,
since that is heaviest object.

Request async stats poll during systemReady() to bootstrap later
delta calculations. Reset kernel counters when UID removed. Update
various tests.

Bug: 5105592, 5146067
Change-Id: Idabec9e3ffcaf212879821515602ecde0a03de8c
2011-08-15 17:30:25 -07:00
Jeff Sharkey
63d27a9233 Network stats with varint, omit parcel fields.
Persist NetworkStatsHistory using variable-length encoding; since
most buckets have small numbers, we can encode them tighter. Initial
test showed 44% space savings. Also persist packet and operation
counters.

Let NetworkStatsHistory consumers request which fields they actually
need to reduce parcel overhead.

Tests for verify varint and history field requests, also verify end-
to-end by persisting history into byte[] and restoring. Expose
bandwidth control enabled state. Extend random generation to create
packet and operation counts. Moved operation counts to long.

Fix bug that miscalculated bytes since last persist, which would
cause partial stats loss when battery pulled.

Bug: 4581977, 5023706, 5023635, 5096903
Change-Id: If61e89f681ffa11fe5711471fd9f7c238d3d37b0
2011-08-03 17:21:07 -07:00
Jeff Sharkey
f0ceede8ff Expose quota status for active network.
Create API to expose quota status derived from underlying network
policy.  This is designed to support applications making informed
decisions when performing network requests.

Fix bug with random stats generation, and write policy when changing
restrict background data flag.  Deprecate EXTRA_NETWORK_INFO, since
it varies based on UID.

Bug: 4517283, 5088603
Change-Id: Ic6893a8967f69937e466be226ba7bb86ef5a5d2d
2011-08-02 18:19:54 -07:00
Jeff Sharkey
a63ba59260 Add operation counts to network statistics.
Provide API to increment "operation counts" for a UID and tag, used
to eventually derive bytes/operation stats.  Internally is stored at
network layer, but should belong at data layer.  Switch profiling
to use data layer stats, which are emulated by summarizing network
layer details.

Read packet counts from new /proc/ columns and collect them into
NetworkStatsHistory.  Prevent double-counting by ignoring values from
first snapshot.  Watch for duplicate /proc/ entries.  Update tests
to verify packet and operation counters.

Bug: 5052136, 5097392
Change-Id: I1832f65a2b8a9188f8088f253474a667c21a2f09
2011-07-29 22:24:01 -07:00
Jeff Sharkey
4664500de0 Maintain separate "restrict background data" flag.
Since "restrict background" depends on active networks, separate its
definition from setBackgroundDataSetting().

Bug: 4979025
Change-Id: I12bfe3a2e606375b39c67706270caa7a1bb0214e
2011-07-27 21:11:32 -07:00
Jeff Sharkey
41ff7ec824 Revise data limit notifs, watch kernel alerts.
Teach NetworkPolicy limits to "snooze" when requested by user, and
notify with both dialog and notification.  Register for network alerts
through NMS to trigger updates immediately instead of waiting for
next stats update.

Enforce that all NetworkPolicy are unique on a template basis, and
move SCREEN_ON/OFF broadcasts to background thread.  Launch SystemUI
and Settings directly instead of using actions, and include full
NetworkTemplate in extras.

Tests to verify notification and snooze behavior.

Bug: 5057979, 5023579, 4723336, 5045721
Change-Id: I03724beff94a7c0547cb5220431ba8d4cd44d077
2011-07-27 09:31:39 -07:00
Jeff Sharkey
02e21d6a5b Data usage notifs, newly installed apps, defaults.
Only show warning/limit notifications for active networks, since they
aren't actionable for inactive IMSI.  Include IMSI in debug output on
engineering builds.

Move default policy warning to be configured through overlay.  Watch
for newly installed packages to enforce global policy.

Bug: 5038729, 5038541, 4979026, 5023634
Change-Id: I8fc8ab4c23c440a3091504ea16133ed6ababf58e
2011-07-17 16:28:02 -07:00
Jeff Sharkey
6248926e19 Acquire wakelock during network stats collection.
Bug: 4517275
Change-Id: I01dfe3b13d7aca84605446e9e9927d0648623552
2011-07-17 13:03:29 -07:00