An accessibility service may register to observe the interactive windows
on the primary display. These windows are the one that has input focus and
ones a sighted user can touch. It is sometimes beneficial for an
accessibility service to overlay a window to intercept user interaction
and based on that introspect and perform an action on the windows that
are on the screen. This is problematic as overlaying a full screen window
that is touchable prevents the accessibility service to introspect the
content under this window.
This change adds a special type of window that only an accessibility service
can place which does not affect what an accessibility service can "see" on
the screen. Hence, even putting such a window full screen the service will
be able to interact with the other interactive windows it covers.
Change-Id: I053ccc3a5c6360a98dc40bdb172b54dab35d8b31
Updating the accessibility layer behavior to reflect the new
model where accessibility no longer overrides strong encryption.
Now enabling an accessibility service lowers the encryption
level but the user can bump it up in settings if desired.
bug:17881324
Change-Id: Ic60d760c267d3f934040a42e1963b179bd8b9f5f
An app can send an accessibility event by calling the send methods
on view or directly asking the accessibility manager to do that.
While the recommened way to send such events is calling the methods
on view a legacy app or app whose developer did not read the docs
carefully may be calling the accessibility manager APIs directly.
In such a case the event does not have assigned window id and does
not get send. Since events fired by using the accessibility manager
directly lack context to determine whether thier source is important
for accessibility we assume they come from an important view to
avoid breaking backwards compatibility.
bug:18001711
Change-Id: Ie1c298fa5a0670cbeaedfcd64f820961c296b6ca
If there is accessibilty focus and the user touch explores
location that does not change accessibility focus that is
not in the app window, e.g. system bar, double tap does not
click on the system UI affordance.
bug:17588024
Change-Id: I6c8c0f65b208ae1d3f31d7f06b0721dc223ec19f
This reverts commit 851a5059a47cbf76e530c9d050a677cb6e3f8657 as
it creates a regression. Let us revert this and correctly fix the
issue the original change was trying to address.
bug:17789608
Change-Id: I8abb1a61d5310430e839e4ef60e7ca5cc0cbdd80
Accessibility introspection APIs are meant to query only the state of
the current user. There are command line tools that run as the shell
user and want to be able to intropspect the screen. When resolving
the calling user we were using the calling user id instead of the
special constant for the current user.
Now when resolving the calling user for intrspection we are using
the current user constant and consequentially only the current
user or a profile of the current user or the root or the shell or
the system or an app with cross user permission can introspect the
screen.
bug:17674631
Change-Id: I36d1d7b65441d04c3b4204123c4b6d036ff032c0
We allow a fake accessibility service to connect to the system for UI
automation. If the process hosting the fake service dies right after
registering it, the accessibility layer ends up in a bad state and
subsequent attempts to connect UI automation service fail.
bug:17725904
Change-Id: Idad288eab41bbdd486d95e1e5803220640653fb7
When device is encrypted the user has to authenticate in order to decrypt
the data partition which is required for running accessibility services
and Text-To-Speech. In order to address this issue we are falling back
to use the default password if there is an enabled accessibility service
and the user has secure lock. This will enable the user to authenticate
when accessibility layer is completely functional.
bug:17671790
Change-Id: Iafffe7bcd234008cf91ffb5011b21b803dca227a
If accessibility is enabled and a test tool based on the accessibility APIs
connects to the system we suspend the current accessibility services while
the tool is running (to avoid inteference as the tool is such a service) and
after the tool disconnects we restore the accessibility state. The issue is
that when clearing the accessibility state we were also wrongly clearing the
active window. We are now careful to not clear the active window in such a
case.
It is also possible that the active window was never initilaized before the
tool is run so now it is lazily loaded such that if we do not know which one
it is, we get the one the has input focus. The definition of an active window
is the one that has input focus or the user is touching.
bug:17663432
Change-Id: I8868866a5126c590d3bddad099ababb97978227a
The UiAutormator tool is a part of the SDK. If it is run while
accessibility is enabled it stops all accessibility services
as it is an accessibility service itself to avoid interference
and when done restores back the accessibility state. The issue
was that the accessibility state is not restored leaving the
device in a bad state.
bug:17662770
Change-Id: I3c4f46fa05c76b874eeffdeb867ef433c3fedf2e
If all accessibility services are disabled we turn off event firing
to avoid a lot of work for nothing. The logic to do that had a flaw
and was not turning event firing off in every case it should. We were
tuning event firing if it is on and no service is enabled but it
is possible that a package on the system image is disabled so the
service is enabled just not bound. Arguably we can remove the
serivice from the enabled list but it is technically on the device.
Note that we correctly handle the case of a package being removed.
bug:17332506
Change-Id: I6d6d9cb9cc271116a3d22ed9fd8d7f533dec9a0b
If there is accessibilty focus and the user touch explores
location that does not change accessibility focus that is
not in the app window, e.g. system bar, double tap does not
click on the system UI affordance. This is due to obsolete
logic from the time where accessibility focus was only in
the active window at the time of double clicking.
bug:17588024
Change-Id: Ib780103e873d8a2afd3b35de3227d54116f1a1b0
We added new APIs to allow accessibility services to query all
windows a user can touch. Sometimes the window state change
event arrives before the window manager sent over the new window
state which leads to a case that the app gets the event and
asks for the window and the window is not there. To address this
if we do not have the window, we hold on to the event and
fire it as soon as the window arrives. This logic is correct
except we were wrongly expecting that the window should have
input focus.
bug:17464645
Change-Id: I1ef50ebddeb4416a6c0776b096bb16aee703700c
In touch exploration mode an accessibility service can move
accessibility focus in response to user gestures. In this case
when the user double-taps the system is sending down and up
events at the center of the acessibility focused view. This
works fine until the clicked view's center is covered by another
clickable view. In such a scenario the user thinks he is clicking
on one view but the click is handled by another. Terrible.
This change solves the problem of clicking on the wrong view
and also solves the problem of clicking on the wrong window.
The key idea is that when the system detects a double tap or
a double tap and hold it asks the accessibility focused node
(if such) to compute a point at which a click can be performed.
In respinse to that the node is asking the source view to
compute this.
If a view is partially covered by siblings or siblings of
predecessors that are clickable, the click point will be
properly computed to ensure the click occurs on the desired
view. The click point is also bounded in the interactive
part of the host window.
The current approach has rare edge cases that may produce false
positives or false negatives. For example, a portion of the
view may be covered by an interactive descendant of a
predecessor, which we do not compute (we check only siblings of
predecessors). Also a view may be handling raw touch events
instead of registering click listeners, which we cannot compute.
Despite these limitations this approach will work most of the
time and it is a huge improvement over just blindly sending
the down and up events in the center of the view.
Note that the additional computational complexity is incurred
only when the user wants to click on the accessibility focused
view which is very a rare event and this is a good tradeoff.
bug:15696993
Change-Id: I85927a77d6c24f7550b0d5f9f762722a8230830f
We get calls for window changes from the window manager and
as a result send accessibility events for these changes. It
is possible that the call for a windows change from the window
manager comes as a result of an unprivileged client calling
into the window manager. However, the event sending code
is performing security checks.
bug:17364244
Change-Id: Ief016f9dafd13ac35418676817848b3ea3ed4d66
The isVisibleToUser property of an AccessibilityNodeInfo specifies
whether the user can see the source view. It is used by accessibility
services to figure out whether to focus on a view. This property
was giving a wrong value if the view is covered by another window
such as the keyboard. As a result the user hears one thing but when
double taps interacts with the overlaid window which is another thing.
bug:15938254
Change-Id: Ib9feb20ea422a24a512c47ed1234961ae0386a7f
1. There was a bug in the touch explorer which was crashing almost
every time after accessibility was enabled via the gesture. The
problem was that in dragging state when a finger goes up we were
not transitioning to touch exploring state.
2. The global actions dialog was not going away after enabling
accessibility while it should as the user brought it up to
turn accessibility on rather to interact with global actions.
bug:15254250
Change-Id: Iaa45f758e09566822775b53e87d2980138e85ef9
1. An external contribution changed the ordering of views for
accessibility. While it attempted to fix a platform issue
for a comparator breaking transitivity, it changed the way
we order views and results in very unnatural accessibility
traversal order. It also broke CTS tets. This change tweaks
the comparator which fixes the tests and improves traversal
order.
2. If there is at least one accessibility service which cares
about windows we register a callback in the window manager
for window change notifications. We are updating the window
list on this callback. There was a case where if the service
requests window updates and immediately asks for the windows
it gets none as we have not received a callback from the
window manager yet. Now this call returns after we get the
callback in a timed fashion. This is consistent with how the
other introspection APIs work.
3. Window info objects are cached in the accessibility service
process. When putting them in the cache a cloning call was
missing resulting in some cases of clobbering windows given
to the client. For example, we get some windows, cache them,
and return these windows to the client. Now a call to clear
the cache arrives while the user processes the windows and
the client windows get clobbered.
4. Added API for checking if a window has accessiblity focus
to be consistent to the API we have to check whether this
window has input focus.
5. Removed some obsolete code.
bug:16402352
Change-Id: Ided6da4a82cc0fc703008c58a2dff0119a3ff317
Fixed a bug caused by a missing temporary when swapping two
matrices. This could produce undefined results in the case where
both display inversion and the greyscale daltonizer were enabled
simultaneously.
Tightened up the code a little to make this kind of error
less likely by removing redundant state.
Removed the native prefix on some methods since they are not
native at all.
Change-Id: I716ffc8fbe76a304c60d45870074340c0121059f
We are caching the window data in the accessibility service process.
When windows change we were sending the dalta of the windows the
service knows about. To make this work when the app asked for all
windows we had to call into the system as new windows may have
appeared. This was slow.
Now we are telling the service some windows change and if it gets
the windows we cache them. We call into the system only on a cache
miss and evict all windows from the cache on window change event.
We do not evict the nodes of the window as the former may have
just moved. If views in a window change they fire accessibility
events that trigger the correct eviction.
Change-Id: I586a72a2497b0d44a75288fa758e7e88817f3300
b/14624452
Adds a feature which draws all text (in the HW accelerated standard
path) in a high contrast mode. Text is drawn at full alpha, and either
white or black (depending on its original color) with a starkly
contrasted outline beneath it.
Change-Id: I943f624b6367de35367cced3b2a8298f2bc62377
Also fixes an infinite recursion bug in the WindowManagerService
implementation of WindowManagerInternal.
BUG: 16129909
Change-Id: I4f9d32f4e6c3ad460652c5e5271540fa5032a1f5
1. The APIs for introspecting interactive windows were reporting only
the touchable windows but were missing the focused window. The user
can interact with the latter by typing, hence it should always be
reported. Also this was breaking backwards compatibility as if the
focused window is covered by a modal one, the focused window was not
reporeted and this was putting the active window in a bad state as
the latter is either the focused window or the one the user is touching.
2. Window change events are too frequent as on window transition things
are chanign a lot. Now we are trottling the windows changed events
at the standard recurring accessibility event interval.
3. Fixed a wrong flag comparison and removed some unneded code.
buy:15434666
bug:15432989
Change-Id: I825b33067e8cbf26396a4d38642bde4907b6427a
When the position and size of a window changes we have to report that
to the accessibility layer if the window introspection is enabled.
bug:15569915
Change-Id: I3f869e0a582592bfa5f3743d5c2133ee8cb39b34
1. AccessibiltiyAction was incorectly throwing an exception when
a custom action was constructed.
2. AccessibilityManagerService should no longer enforce only standard
actions as we allow custom ones too.
bug:15110963
Change-Id: Iea57e0a6449b87bd8d103c55ca255e80705f2565
Makes the inversion bit public so apps can be smarter about rendering
images non-inverted (if they are so inclined).
BUG: 14680114
Change-Id: I8ca2b2517af2ffc13446d7e0b4e859c1171e2d0e
For user profiles the accessibility layer will use the accessibility
state of the their owner. A profile cannot have its own plugins or
settings. The rationale behind this is that if a user with impaired
vision is using the device with different profiles linked to his
main one, then he/she should be able to interact with the device no
matter what profile the app is being run with. Hence, The behavior
of the accessibility features should be consistent across apps run
with different profile.
bug:14564491
Change-Id: Id19ce81e1a2dfa48015c7ab3053d66c058be3e46
Now that we have APIs to query all interactive windows and allow
an accessibility service to put accessibility focus in each of
them we have to guarantee that there is a single accessibility
focus. This is required for correct operation of the touch
explorer as on double tap in clicks in the center of the focused
area, hence having more that one focus is an issue. Also the
system is maintaining a single input focus so now accessibility
focus behaves consistently with that.
bug:13965563
Change-Id: I0b5c26dadfabbf80dbed8dc4602073aa575ac179
The accessibility manager service was not handling the focused
window properly which as a result was breaking the active window
tracking.
bug:13788022
Change-Id: Iaa3fadfbf7b9545f5151e3603beeda2662fe7618
This change achieves a couple of things:
- Let Keyguard be a library, so we can use it in SystemUI.
- Introduce FLAG_KEYGUARD for windows and deprecate TYPE_KEYGUARD. Make
all the TYPE_KEYGUARD behaviour dependant on the flag.
- Implement a new KeyguardService in SystemUI, and bind that service
from PhoneWindowManager.
- Introduce BaseStatusBar.setKeyguardState and inflate
KeyguardSimpleHostView there and use FLAG_KEYGUARD for the window, such
that the status bar window essentially gets the Keyguard.
Bug: 13635952
Change-Id: I059d80d8b9b9818a778ab685f4672ea2694def63
1. The old introspection model was allowing querying only the active window
which is the one the user is touching or the focused one if no window is
touched. This was limiting as auto completion drop downs were not inspectable,
there was not way to know when the IME toggles, non-focusable windows were
not inspectable if the user taps them as until a screen-reader starts
introspecting the users finger is up, accessibility focus was limited to
only one window and the user couldn't use gestures to visit the whole UI,
and other things I can't remember right now.
The new APIs allow getting all interactive windows, i.e. ones that a
sighted user can interact with. This prevents an accessibility service
from interacting with content a sighter user cannot. The list of windows
can be obtained from an accessibility service or the host window from an
accessibility node info. Introspecting windows obey the same rules for
introspecting node, i.e. the service has to declare this capability
in its manifest.
When some windows change accessibility services receive a new type
of event. Initially the types of windows is very limited. We provide
the bounds in screen, layer, and some other properties which are
enough for a client to determined the spacial and hierarchical
relationship of the windows.
2. Update the documentation in AccessibilityService for newer event types.
3. LongArray was not removing elements properly.
4. Composite accessibility node ids were not properly constructed as they
are composed of two ints, each taking 32 bits. However, the values for
undefined were -1 so composing a 64 long from -1, -1 prevents from getting
back these values when unpacking.
5. Some apps were generating inconsistent AccessibilityNodeInfo tree. Added
a check that enforces such trees to be well formed on dev builds.
6. Removed an necessary code for piping the touch exploration state to
the policy as it should just use the AccessibilityManager from context.
7. When view's visibility changed it was not firing an event to notify
clients it disappeared/appeared. Also ViewGroup was sending accessibility
events for changes if the view is included for accessibility but this is
wrong as there may be a service that want all nodes, hence events from them.
The accessibility manager service takes care of delivering events from
not important for accessibility nodes only to services that want such.
8. Several places were asking for prefetching of sibling but not predecessor
nodes which resulted in prefetching of unconnected subtrees.
9. The local AccessibilityManager implementation was relying on the backing
service being ready when it is created but it can be fetched from a context
before that. If that happens the local manager was in a broken state forever.
Now it is more robust and starts working properly once the backing service
is up. Several places were lacking locking.
bug:13331285
Change-Id: Ie51166d4875d5f3def8d29d77973da4b9251f5c8
