1. This change enforces an accessibility service to require the system
defined BIND_ACCESSIBILITY_SERVICE permission.
bug:6507771
Change-Id: If5e16bb4fa97891be0ccbb35e343773712e33b98
1. The way for computing the magnified region was simplistic and
incorrect. It was ignoring window layering resulting in broken
behavior. For example, if the IME is up, then the everything else
is magnifed and the IME not. Now the keyguard appears and covers
the IME but the magnified region does not expand while it would
since the keyguard completely covers the not magnified IME window.
bug:7138937
Change-Id: I21414635aefab700ce75d40f3e913c1472cba202
1. When the screen goes off the user will be in a completely
different context upon turning the screen on. Therefore,
if magnification auto update is enabled magnification
will be disengaged on screen off.
bug:7139088
Change-Id: I790cfa5b3cf31d34e95fc9548e6246a84096c37b
1. If screen magnification is enabled the user has to triple tap
and lift or triple tap and hold to engage magnification. Hence,
we delay the touch events until we are sure that it is no longer
possible for the user to perform a multi-tap to engage
magnification. While such a delay is unavoidable it feels a
bit longer than it should be. This change reduces the delay
between taps to be considered a multi-tap, essentially making
the click delay shorter.
bug:7139918
Change-Id: I2100945171fff99600766193f0effdaef1f1db8f
1. If the user changes the magnification level while moving the
viewport the magnification is locked. The gesture handle has
to put device back into a viewport moving state if this was
the last state.
bug:7139363
Change-Id: I24992b973bb15624580114353b004efdb35c2faa
1. Before in magnified state the user was able to only scale or
pan. Based on user input this change allows performing pan
or scale or both. If the user scales more than a threshold
we are performing a scale and independently of that if the
use pans more than a threshold we are performing a pan.
bug:7138928
Change-Id: Ic1511500ba3369091dcfd070669d3e4f0286fbe5
1. Due to frequent changes of the behavior of ScaleGestureDetector
this patch rolls in a gesture detector used for changing the
screen magnification level. It has an improved algorithm which
uses the diameter of min circle around the points as the span, the
center of this circle as the focal point, and the average slop
of the lines from each pointer to the center to determine the
angle of the diameter used when computing the span x and y.
Change-Id: I5cee8dba84032a0702016b8f9632f78139024bbe
1. If screen magnification is disabled when the screen is in a
magnified state we have to zoom out since otherwise the user
is stuck in a magnified state without ability to pan/zoom/
toggle magnification which renders the device useless.
bug:7131030
Change-Id: I8f3339f31310448ec8742f3101c1fdc61a6a5f83
1. If screen magnification is disabled when the screen is in a
magnified state we have to zoom out since otherwise the user
is stuck in a magnified state without ability to pan/zoom/
toggle magnification which renders the device useless.
bug:7131030
Change-Id: Ia620954fbd594e7cd470e43b89d9ed04c0397c3c
This change is the initial check in of the screen magnification
feature. This feature enables magnification of the screen via
global gestures (assuming it has been enabled from settings)
to allow a low vision user to efficiently use an Android device.
Interaction model:
1. Triple tap toggles permanent screen magnification which is magnifying
the area around the location of the triple tap. One can think of the
location of the triple tap as the center of the magnified viewport.
For example, a triple tap when not magnified would magnify the screen
and leave it in a magnified state. A triple tapping when magnified would
clear magnification and leave the screen in a not magnified state.
2. Triple tap and hold would magnify the screen if not magnified and enable
viewport dragging mode until the finger goes up. One can think of this
mode as a way to move the magnified viewport since the area around the
moving finger will be magnified to fit the screen. For example, if the
screen was not magnified and the user triple taps and holds the screen
would magnify and the viewport will follow the user's finger. When the
finger goes up the screen will clear zoom out. If the same user interaction
is performed when the screen is magnified, the viewport movement will
be the same but when the finger goes up the screen will stay magnified.
In other words, the initial magnified state is sticky.
3. Pinching with any number of additional fingers when viewport dragging
is enabled, i.e. the user triple tapped and holds, would adjust the
magnification scale which will become the current default magnification
scale. The next time the user magnifies the same magnification scale
would be used.
4. When in a permanent magnified state the user can use two or more fingers
to pan the viewport. Note that in this mode the content is panned as
opposed to the viewport dragging mode in which the viewport is moved.
5. When in a permanent magnified state the user can use three or more
fingers to change the magnification scale which will become the current
default magnification scale. The next time the user magnifies the same
magnification scale would be used.
6. The magnification scale will be persisted in settings and in the cloud.
Note: Since two fingers are used to pan the content in a permanently magnified
state no other two finger gestures in touch exploration or applications
will work unless the uses zooms out to normal state where all gestures
works as expected. This is an intentional tradeoff to allow efficient
panning since in a permanently magnified state this would be the dominant
action to be performed.
Design:
1. The window manager exposes APIs for setting accessibility transformation
which is a scale and offsets for X and Y axis. The window manager queries
the window policy for which windows will not be magnified. For example,
the IME windows and the navigation bar are not magnified including windows
that are attached to them.
2. The accessibility features such a screen magnification and touch
exploration are now impemented as a sequence of transformations on the
event stream. The accessibility manager service may request each
of these features or both. The behavior of the features is not changed
based on the fact that another one is enabled.
3. The screen magnifier keeps a viewport of the content that is magnified
which is surrounded by a glow in a magnified state. Interactions outside
of the viewport are delegated directly to the application without
interpretation. For example, a triple tap on the letter 'a' of the IME
would type three letters instead of toggling magnified state. The viewport
is updated on screen rotation and on window transitions. For example,
when the IME pops up the viewport shrinks.
4. The glow around the viewport is implemented as a special type of window
that does not take input focus, cannot be touched, is laid out in the
screen coordiates with width and height matching these of the screen.
When the magnified region changes the root view of the window draws the
hightlight but the size of the window does not change - unless a rotation
happens. All changes in the viewport size or showing or hiding it are
animated.
5. The viewport is encapsulated in a class that knows how to show,
hide, and resize the viewport - potentially animating that.
This class uses the new animation framework for animations.
6. The magnification is handled by a magnification controller that
keeps track of the current trnasformation to be applied to the screen
content and the desired such. If these two are not the same it is
responsibility of the magnification controller to reconcile them by
potentially animating the transition from one to the other.
7. A dipslay content observer wathces for winodw transitions, screen
rotations, and when a rectange on the screen has been reqeusted. This
class is responsible for handling interesting state changes such
as changing the viewport bounds on IME pop up or screen rotation,
panning the content to make a requested rectangle visible on the
screen, etc.
8. To implement viewport updates the window manger was updated with APIs
to watch for window transitions and when a rectangle has been requested
on the screen. These APIs are protected by a signature level permission.
Also a parcelable and poolable window info class has been added with
APIs for getting the window info given the window token. This enables
getting some useful information about a window. There APIs are also
signature protected.
bug:6795382
Change-Id: Iec93da8bf6376beebbd4f5167ab7723dc7d9bd00
1. The window manager was not notifying a window when the latter
has been moved. This was causing incorrect coordinates of the
nodes reported to accessibility services. To workaround that
we have carried the correct window location when making a
call from the accessibility layer into a window. Now the
window manager notifies the window when it is moved and the
workaround is no longer needed. This change takes it out.
2. The left and right in the attach info were not updated properly
after a report that the window has moved.
3. The accessibility manager service was calling directly methods
on the window manager service without going through the interface
of the latter. This leads to unnecessary coupling and in the
long rung increases system complexity and reduces maintability.
bug:6623031
Change-Id: Iacb734b1bf337a47fad02c827ece45bb2f53a79d
1. There was a misspelled duplicate member in the accessibility service
class which was causing inconsistent behavior because one field was
updated and another checked.
2. When the set of services that can put the device in explore by touch
mode changes we were disconnecting and reconnecting all services
and this is not correct. Now only the state of explore by touch is
updated appropriately.
bug:6798860
Change-Id: Ib3c119cef8e71c3458d56e4ce6fbde2c2f750dcd
This fix adjusts the sensitivity of the gesture recognizer by
eliminating gesture rotation in the recognition process.
Bug:6697119
Change-Id: Ic767f513c05210b27e583338c4f0adcaa1c4c625
1. Accessibility allows querying only of the active window.
The active window is the one that has input focus or the
one the user is touching. Hence, if the user is touching
a window that does not have input focus this window is
the active one and as soon as the user stops touching
it the active window becomes the one that has input
focus. Currently the active window is not updated properly
when the user lifts his finger. This leads to a scenario
of traversal actions sent to the wrong window and the user
being stuck.
The reason is that the last touch explored event that is
used to determine where to click is cleared when accessibility
focus moves but this event is also used to determine when to
send the hover exit and touch exploration gesture end events.
The problem is that the last hover event is cleared before
it is used for sending the right exit events, thus the event
stream is inconsistent and the accessibility manager service
relies on this stream to update the active window. Now we
are keeping separate copies of the last touch event - one
for clicking and one for determining the which events to
inject to ensure consistent stream.
bug:6666041
Change-Id: Ie9961e562a42ef8a9463afacfff2246adcb66303
1. We are deciding whether the user is performing a gesture or an exploration based
on the gesture velocity. If we are detecting gesture we do the recognition at the
gesture end which is when the finger goes up. This is better than having a mode
toggle gesture for exploring and gestures detection. However, it is possible that
the user really wanted to perform an exploration but was moving too fast and
unless he lifts his finger the device is in gesture detection mode. This is
frustrating since the user has no feedback and assumes exploration does not
work.
We want to perform gesture detection only for a maximal time frame and if the
user did not lift his finger we transition into touch exploration state.
bug:6663173
Change-Id: I954ff937cca902e31b51325d1e1dfce84d239624
1. The touch explorer was notified for accessibility events from
a binder thread which was poking the internal state of the
latter which by design is not tread safe. Since the touch
explorer is expected to be running only on the main thread
the accessibility manager service delivers the accessibility
events to the explorer on that thread.
bug:6635496
Change-Id: Ifdc5329e4be8e485d7f77f0fb472184494fa0d15
1. AccessibilityInput filter was not checking whether the touch
explorer instance is not null before passing it an accessibility
event. If the accessibility event is dispatched before the input
filter is installed but after it is created we runt into this
case.
2. Added a missing null check in accessibility node info.
bug:6635089
Change-Id: Ia389dc1f427427eb73794f6331ccb870e0b44c55
1. When typing into an auto completion edit field a list of completions pops up and if
the user touch explores the list and tries to double tap to select the touched
completion the latter is not selected.
The auto completion is a popup that does not take input focus and is overlaid on
top of the window that has input focus. The touch explorer was clicking on the
location of the accessibility focus if the last touch explored location is within
the bounds of the active window. In this case this was the window with the edit
text into which the user is typing. The check performed by the touch explorer
was missing the case when the last touch explored location was within the bounds
of the active window but it actually was deloverd to another overlaid window.
Now we are poking on the accessibility focus location if the last explored
location is within the active window and was delivered to it.
bug:6629535
Change-Id: Ie66d5bb81ab021f2bb0414339b7de26d96826191
1. If the last touch explored location is within the active window we
used to click on exact location if it is within the accessibility
focus otherwise in the accessibility focus center. If the last touch
explored location is not within the active window we used to just
click there. This breaks in the case were one has touch explored
at a given place in the current window and now a dialog opens *not*
covering the touch explored location. If one uses swipes to move
accessibility focus i.e. to traverse the dialog without touching
it one cannot activate anything because the touch explorer is using
the last touch explored location that is outside of the active
window e.g the dialog.
The solution is to clear the last touch explored location when a
window opens or accessibility focus moves. If the last touch
explored location is null we are clicking in the accessibility
focus location.
bug:6620911
2. There is a bug in the window manager that does not notify a
window that its location has changed (bug:6623031). This breaks
accessibility interaction with dialogs that have input because
when the IME is up the dialog is moved but not notified. Now
the accessibility layer gets incorrect location for the
accessibility focus and the window bounds.
The soluion is when the accessibility manager service calls
into the remove thress to obtain some accessibility node infos
it passes the window left and top which it gets from the
window manager. These values are used to update the attach info
window left and top so all accessibility node infos emitted
from that window had correct bounds in screen coordinates.
bug:6620796
Change-Id: I18914f2095c55cfc826acf5277bd94b776bda0c8
1. Now after setting the content description on a view we mark is as
important for accessibility of the current important for accessibility
mode of that view is auto.
2. Minor tweak to a touch explorer coefficient to make performing double
tapping easier.
bug:6615353
Change-Id: I3b477f533a3ebde85d425caf32ace5e851240f88
1. The global action to open recent apps shows the old dialog style rent apps
panel. Apparently the key code to open recent apps is not opening the new
UI so the AccessibilityManagerService is calling directly the method on
the IStatusBarSerivce to do so.
bug:6607664
Change-Id: I94c1963b07947776bf1c2448903b26f3603f9a59
1. Touch exploration gestures are demarcated by start and end
events. Due to a bug in the AccessibilityManagerService
the gesture end event was not dispatched. This caused the
AccessibilityNodeInfoCache to be off sync since it relies
on getting such events not to mention that the clients were
not getting the end but only the start event. The issue
was that the notified service types variable was not reset
after every event so when the manager sends the last hover
exit it flags that the service type is already notified
resulting in dropping on the floor the following gesture
end event.
bug:6539306
Change-Id: I2b96bcecea3b2240199d67f01afa6a033afce1de
1. If the runnable for performing a long press is not
removed when all pointers are up and it is executed
the explorer gets into delegating mode with no pointer
down and the next down crashes the explorer. Added
code to remove the long press runnable in a few places
it was missing and also added a safety in the runnable
to avoid executing it in case there are no active pointers.
bug:6557183
Change-Id: I9dab3de88fd08d8e2b38af18249ac551837c0736
1. The long press routine was using the coordintates of the
accessibility focused item in the input focused window.
As a result double tap and hold did not work in a window
that does not take input focus such as the system bar.
Now the routine is using the last touch explored location
if it cannot find accessibility focus in the last touched
window.
bug:6584438
Change-Id: Ifd43adb20a066f389a9d4bd5716dd7ad834dd574
1. Now we are asking the user to grant permission to the service to enable
touch exploration only the first time this service is enabled. If the
service was uninstalled and then later installed we ask the user again.
This avoids the scenario in which rebooting the device or upgrading an
accessibility service leaves the device in a state in which the user
cannot interact with.
bug:6582088
Change-Id: I51d24e4892b3b48c9fb11dfb09ec1118502ba526
1. If a UI test automation accessibility service is connected to the
system we pospone state updates in the AccessibilityManagerService
for the moment the UI automations service dies or is disconnected.
bug:6540522
Change-Id: I48ddf603b53d2158a00edcf8ad05cfe2575d4d75
1. We are passing the interrogating process id in the remote
accessibility requests to catch the query from the same
thread. While all other methods were doing this correctly
somehow the perform action is using the incorrect process id.
bug:6534935
Change-Id: Icef50833903c562758d51ef316b60c53c7a336c0
1. The internal service instance created by AccessibilityManagerService
was getting the looper of the current thread when created. This works
for real accessibility services but since UI automation service is
registered via an IPC the binder thread has no looper. Now we explicitly
get the correct looper.
bug:6535435
Change-Id: I63a2ada1b65c4b3c71c3d1e6deb3dfdeb7a3d6d6
1. Now the user have to double tap to activate the last
item. If the last touched window is not active because
it does not take input focus the click on the last
touch explored location. Othewise the click is on the
accessibility focus location.
bug:5932640
Change-Id: Ibb7b97262a7c5f2f94abef429e02790fdc91a8dd
1. Every accessibility services targeting JellyBean or higher has
to request a special permission for the system to bind to it.
Change-Id: I6e579326bdf3597f148d6c67317455701ec8af68
1. During gesture detection we are not injecting the events we receive
since we do not want the accessibility focus to move as a result of
the hover event of the gesture. Because of that it was possible that
we consume all events since the user performs only gesture to navigate
resulting in the screen being off while the user is actively interacting
with the device. Now we are poking the user activity in the power
manager to keep the screen on.
bug:6485171
Change-Id: I06a09c5297f01bef5e20d471cee76fa7aae0c4fe
1. Since the API version has been finalized this change
updates the SDk version checks to use the JellyBean
verson number.
bug:5947249
Change-Id: Ie22fa7e18a7ea7b0c7077d80246a26c17f327ceb
1. The initial design was to have some accessibility gestures
being handled by the system if the gesture handling access
service does not consume the gesture. However, we are not
sure what a good default is and once we add a default handler
we cannot remove it since people may rely on it. Thus, we
take the simples approach and let the accessibility service
handle the gestures. If no gestures are handled the system
will work in explore by touch as before.
bug:5932640
Change-Id: I865a83549fa03b0141d27ce9713e9b7bb45a57b4
1. Scrolling actions are crucial for enabling a gesture based
traversal of the UI and specifically scrollable containers
especially lists and anything backed by an adapter. Since
accessibility focus can land only attached views, it cannot
visit views for adapter items not shown on the screen.
Auto scrolling the list as a result of putting access focus
ot a list item does not work well since the user may get
trapped in a long list. Adding an accessibility node provider
to emit virtual views for one view before the first and one
after the last is complex and suffers the limitation of trapping
the user. Accessibility service need an explicit scroll actions
which may be performed upon an explicit user action. Hence,
the user is informed for the start/end of the visible part of
the list and he makes a deliberate choice to scroll. This will
benefit also people developing Braille devices since they can
scroll the content without telling the user to stop using the
Braille controller and take the device out of his pocket to scroll
and go back to the Braille controller.
NOTE: Without these action large portions of the screen will be
hard to access since users will have to touch and explore to
find and scroll the list.
Change-Id: Iafcf54d4967893205872b3649025a4e347a299ed
1. Delegating activation gestures has several issues that we should
decide how to handle if possible before allowing an accessibility
service to take over them:
A) It is needed that every view than can be clicked or long pressed on
reacts to such as a response to calling performClick and performLongPress
which is not necessary true since the view may watch the touch
events and do its own click long click detection. As a result it may
be possible that there are view a user cannot interact with in
touch exploration mode but can if not in that mode.
B) Clicking or long pressing on a different location in a view may yield
different results, for example NumberPicker. Ideally such views have
to implement AccessibilityNodeProvide which provider handles correctly
the request for click long press on virtual nodes. Some apps however
just fire different hover accessibility events when the user is over
a specific semantic portion of the view but do not provide virtual
nodes. Hence, a user will not be able to interact with such semantic
regions but the system can achieve that by sending the click/long click
at the precise location in the view that was last touch explored.
2. Adding a flag on accessibility service info to request explore by touch
mode. There is no need to put the device in this mode if node of the currently
enabled accessibility services supports it. Now the problem is inverted and
the service has to explicitly state its capability.
3. Fixing a bug where includeImportantViews was ignored for automation
services.
Change-Id: I3b29a19f24ab5e26ee29f974bbac2197614c9e2a
This problem was introduced in I74df9c24. The intention of the
change was still let UiTestAutomationBridge see the
non-important views, but there were bugs in the implementation:
1. AccessibilityManagerService was not really updating
mIncludeNotImportantViews when mIsAutomation is true
2. Wrong constant is used to set the flag
Change-Id: Ia0a2e9ed9720bd0ea3a563e0b492e870a6ec1586
1. Since we are using a stateless proxy accessibility service to
perform default accessibility gesture handling it shuld not
operate against not important views.
bug:6422069
Change-Id: I74df9c2415ab3b164d9ac5873f7004c0459e2bfa
1. Changed all references to granularity to movement
granularity. BTW, to be more precise it should be
text movement granularity.
bug:6435232
Change-Id: If6366b002ca3390f74918995b342baff2cbcfd01
