Compare commits

...

23 Commits

Author SHA1 Message Date
Romain Vimont
11337c4a2a Reassign -K and -M to UHID keyboard and mouse
The options were deprecated, but for convenience, reassign them to
aliases for --keyboard=uhid and --mouse=uhid respectively.

Their long version (--hid-keyboard and --hid-mouse) remain deprecated.
2024-02-26 20:29:38 +01:00
Romain Vimont
b614cc144c Add UHID mouse support
Use the following command:

    scrcpy --mouse=uhid
2024-02-26 20:29:38 +01:00
Romain Vimont
b51da9a278 Add shortcut to open keyboard settings
The keyboard settings can be opened by:

    adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS

Add a shortcut (MOD+k) for convenience if the current keyboard is HID.
2024-02-26 20:29:37 +01:00
Romain Vimont
abc64b46c4 Fix startActivity() for supporting API < 30
Call the older startActivityAsUser() instead of
startActivityAsUserWithFeature() so that it also works on older Android
versions.

Fixes #4704 <https://github.com/Genymobile/scrcpy/issues/4704>
2024-02-26 20:27:19 +01:00
Romain Vimont
630adf829c Create UhidManager only on first use
There is no need to create a UhidManager instance (with its thread) if
no UHID is used.
2024-02-26 20:27:19 +01:00
Simon Chan
dbb2dc8ad2 Handle UHID output
Use UHID output reports to synchronize CapsLock and VerrNum states.

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2024-02-26 20:27:19 +01:00
Romain Vimont
70f9e1c94d Refactor DeviceMessageSender
Refactor DeviceMessage as a queue of message. This will allow to add
other message types.
2024-02-26 20:27:19 +01:00
Simon Chan
ece0831062 Add UHID keyboard support
Use the following command:

    scrcpy --keyboard=uhid

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2024-02-26 20:27:19 +01:00
Romain Vimont
4b3069ce83 Log controller handling errors
On close, the controller is expected to throw an IOException because the
socket is closed, so the exception was ignored.

However, message handling actions may also throw IOException, and they
must not be silently ignored.
2024-02-26 20:27:19 +01:00
Romain Vimont
f4fc339820 Initialize controller before keyboards
The UHID keyboard initializer will need the controller.
2024-02-26 20:27:19 +01:00
Romain Vimont
94686ad08d Initialize controller in two steps
There is a dependency cycle in the initialization order:
 - keyboard depends on controller
 - controller depends on acksync
 - acksync depends on keyboard initialization

To break this cycle, bind the async instance to the controller in a
second step.
2024-02-26 20:27:19 +01:00
Romain Vimont
077496cf8d Extract binary to hex string conversion 2024-02-26 20:27:19 +01:00
Romain Vimont
5058f99cc3 Rename default keyboard implementation to "sdk"
Rename {keyboard,mouse}_inject to {keyboard,mouse}_sdk.

All implementations "inject" key events and mouse events, what differs
is the mechanism. For these implementations, the Android SDK API is used
to inject events.

Note that the input mode enum variants were already renamed
(SC_KEYBOARD_INPUT_MODE_SDK and SC_MOUSE_INPUT_MODE_SDK).
2024-02-26 20:27:19 +01:00
Romain Vimont
673d87a63a Extract mouse HID handling
Split the mouse implementation using AOA and the code handling HID
events, so that HID events can be reused for another protocol (UHID).
2024-02-26 20:27:19 +01:00
Romain Vimont
597b78686a Extract keyboard HID handling
Split the keyboard implementation using AOA and the code handling HID
events, so that HID events can be reused for another protocol (UHID).
2024-02-26 20:27:19 +01:00
Romain Vimont
8a0490d5a7 Extract HID events struct
An event contained several fields:
 - the accessory id
 - the HID event data
 - a field ack_to_wait specific to the AOA implementation.

Extract the HID event part to prepare the factorization of HID event
creation.
2024-02-26 20:27:19 +01:00
Romain Vimont
c6666b2d95 Embed HID event data
In the implementation, an HID event is at most 8 bytes. Embed the data
in the HID event structure to avoid allocations and simplify the code.
2024-02-26 20:27:19 +01:00
Romain Vimont
6dae71ff77 Rename hid event "buffer" to "data"
This fields contains the HID event data (there is no "bufferization").
2024-02-26 20:27:19 +01:00
Romain Vimont
c2d25421df Rename "buffer" to "data"
The variable name is intended to match the parameter name of
libusb_control_transfer().
2024-02-26 20:27:19 +01:00
Romain Vimont
e3077723e7 Fix HID mouse documentation
The size of a mouse HID event is 4 bytes.
2024-02-26 20:27:19 +01:00
Simon Chan
807d48ed25 Introduce --keyboard and --mouse
Until now, there was two modes for keyboard and mouse:
 - event injection using the Android system API (default)
 - HID/AOA over USB

For this reason, the options were exposed as simple flags:
 - -K or --hid-keyboard to enable physical keyboard simulation (AOA)
 - -M or --hid-mouse to enable physical mouse simulation (AOA)

Replace them by explicit --keyboard and --mouse options, with 3 possible
values:
 - disabled
 - sdk (default)
 - aoa

This will allow to add a new mode (uhid).

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
2024-02-26 20:27:19 +01:00
Romain Vimont
585b0c245a Accept disabled keyboard or mouse
The input manager assumed that if a controller was present, then both a
key processor and a mouse processor were present.

Remove this assumption, to support disabling keyboard and mouse
separately. This prepares the introduction of new command line options
--keyboard and --mouse.
2024-02-26 20:27:19 +01:00
Romain Vimont
4ad9a7d469 Always pass input manager instance
Some functions in input_manager.c only have access to a sub-object (for
example the controller). For consistency, always pass the whole
input manager instance.

This will allow to add assertions when keyboard and mouse could be
disabled separately.
2024-02-26 20:27:18 +01:00
59 changed files with 2269 additions and 898 deletions

View File

@ -27,8 +27,9 @@ _scrcpy() {
--force-adb-forward
--forward-all-clicks
-h --help
-K
--keyboard
--kill-adb-on-close
-K --hid-keyboard
--legacy-paste
--list-camera-sizes
--list-cameras
@ -37,8 +38,9 @@ _scrcpy() {
--lock-video-orientation
--lock-video-orientation=
-m --max-size=
-M --hid-mouse
-M
--max-fps=
--mouse=
-n --no-control
-N --no-playback
--no-audio
@ -115,6 +117,14 @@ _scrcpy() {
COMPREPLY=($(compgen -W 'front back external' -- "$cur"))
return
;;
--keyboard)
COMPREPLY=($(compgen -W 'disabled sdk aoa uhid' -- "$cur"))
return
;;
--mouse)
COMPREPLY=($(compgen -W 'disabled sdk aoa uhid' -- "$cur"))
return
;;
--orientation|--display-orientation)
COMPREPLY=($(compgen -W '0 90 180 270 flip0 flip90 flip180 flip270' -- "$cur"))
return

View File

@ -34,8 +34,9 @@ arguments=(
'--force-adb-forward[Do not attempt to use \"adb reverse\" to connect to the device]'
'--forward-all-clicks[Forward clicks to device]'
{-h,--help}'[Print the help]'
'-K[Use UHID keyboard (same as --keyboard=uhid)]'
'--keyboard[Set the keyboard input mode]:mode:(disabled sdk aoa uhid)'
'--kill-adb-on-close[Kill adb when scrcpy terminates]'
{-K,--hid-keyboard}'[Simulate a physical keyboard by using HID over AOAv2]'
'--legacy-paste[Inject computer clipboard text as a sequence of key events on Ctrl+v]'
'--list-camera-sizes[List the valid camera capture sizes]'
'--list-cameras[List cameras available on the device]'
@ -43,8 +44,9 @@ arguments=(
'--list-encoders[List video and audio encoders available on the device]'
'--lock-video-orientation=[Lock video orientation]:orientation:(unlocked initial 0 90 180 270)'
{-m,--max-size=}'[Limit both the width and height of the video to value]'
{-M,--hid-mouse}'[Simulate a physical mouse by using HID over AOAv2]'
'-M[Use UHID mouse (same as --mouse=uhid)]'
'--max-fps=[Limit the frame rate of screen capture]'
'--mouse[Set the mouse input mode]:mode:(disabled sdk aoa uhid)'
{-n,--no-control}'[Disable device control \(mirror the device in read only\)]'
{-N,--no-playback}'[Disable video and audio playback]'
'--no-audio[Disable audio forwarding]'

View File

@ -20,8 +20,8 @@ src = [
'src/fps_counter.c',
'src/frame_buffer.c',
'src/input_manager.c',
'src/keyboard_inject.c',
'src/mouse_inject.c',
'src/keyboard_sdk.c',
'src/mouse_sdk.c',
'src/opengl.c',
'src/options.c',
'src/packet_merger.c',
@ -31,8 +31,13 @@ src = [
'src/screen.c',
'src/server.c',
'src/version.c',
'src/hid/hid_keyboard.c',
'src/hid/hid_mouse.c',
'src/trait/frame_source.c',
'src/trait/packet_source.c',
'src/uhid/keyboard_uhid.c',
'src/uhid/mouse_uhid.c',
'src/uhid/uhid_output.c',
'src/util/acksync.c',
'src/util/audiobuf.c',
'src/util/average.c',
@ -88,8 +93,8 @@ usb_support = get_option('usb')
if usb_support
src += [
'src/usb/aoa_hid.c',
'src/usb/hid_keyboard.c',
'src/usb/hid_mouse.c',
'src/usb/keyboard_aoa.c',
'src/usb/mouse_aoa.c',
'src/usb/scrcpy_otg.c',
'src/usb/screen_otg.c',
'src/usb/usb.c',

View File

@ -172,24 +172,33 @@ By default, right-click triggers BACK (or POWER on) and middle-click triggers HO
Print this help.
.TP
.B \-\-kill\-adb\-on\-close
Kill adb when scrcpy terminates.
.B \-K
Same as \fB\-\-keyboard=uhid\fR.
.TP
.B \-K, \-\-hid\-keyboard
Simulate a physical keyboard by using HID over AOAv2.
.BI "\-\-keyboard " mode
Select how to send keyboard inputs to the device.
This provides a better experience for IME users, and allows to generate non-ASCII characters, contrary to the default injection method.
Possible values are "disabled", "sdk" and "aoa":
It may only work over USB.
- "disabled" does not send keyboard inputs to the device.
- "sdk" uses the Android system API to deliver keyboard events to applications.
- "aoa" simulates a physical HID keyboard using the AOAv2 protocol. It may only work over USB.
- "uhid" simulates a physical HID keyboard using the Linux HID kernel module on the device.
The keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly:
For "aoa" and "uhid", the keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly using the shortcut MOD+k (except in OTG mode) or by executing:
adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
However, the option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
If mirroring is enabled, the shortcot MOD+k opens it.
Also see \fB\-\-hid\-mouse\fR.
This option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
Also see \fB\-\-mouse\fR.
.TP
.B \-\-kill\-adb\-on\-close
Kill adb when scrcpy terminates.
.TP
.B \-\-legacy\-paste
@ -230,21 +239,31 @@ Limit both the width and height of the video to \fIvalue\fR. The other dimension
Default is 0 (unlimited).
.TP
.B \-M, \-\-hid\-mouse
Simulate a physical mouse by using HID over AOAv2.
In this mode, the computer mouse is captured to control the device directly (relative mouse mode).
LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
It may only work over USB.
Also see \fB\-\-hid\-keyboard\fR.
.B \-M
Same as \fB\-\-mouse=uhid\fR.
.TP
.BI "\-\-max\-fps " value
Limit the framerate of screen capture (officially supported since Android 10, but may work on earlier versions).
.TP
.BI "\-\-mouse " mode
Select how to send mouse inputs to the device.
Possible values are "disabled", "sdk" and "aoa":
- "disabled" does not send mouse inputs to the device.
- "sdk" uses the Android system API to deliver mouse events to applications.
- "aoa" simulates a physical mouse using the AOAv2 protocol. It may only work over USB.
- "uhid" simulates a physical HID mouse using the Linux HID kernel module on the device.
In "aoa" and "uhid" modes, the computer mouse is captured to control the device directly (relative mouse mode).
LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
Also see \fB\-\-keyboard\fR.
.TP
.B \-n, \-\-no\-control
Disable device control (mirror the device in read\-only).
@ -636,6 +655,10 @@ Copy computer clipboard to device, then paste (inject PASTE keycode, Android >=
.B MOD+Shift+v
Inject computer clipboard text as a sequence of key events
.TP
.B MOD+k
Open keyboard settings on the device (for HID keyboard only)
.TP
.B MOD+i
Enable/disable FPS counter (print frames/second in logs)

View File

@ -93,6 +93,10 @@ enum {
OPT_DISPLAY_ORIENTATION,
OPT_RECORD_ORIENTATION,
OPT_ORIENTATION,
OPT_KEYBOARD,
OPT_MOUSE,
OPT_HID_KEYBOARD_DEPRECATED,
OPT_HID_MOUSE_DEPRECATED,
};
struct sc_option {
@ -358,27 +362,44 @@ static const struct sc_option options[] = {
.longopt = "help",
.text = "Print this help.",
},
{
.shortopt = 'K',
.text = "Same as --keyboard=uhid.",
},
{
.longopt_id = OPT_KEYBOARD,
.longopt = "keyboard",
.argdesc = "mode",
.text = "Select how to send keyboard inputs to the device.\n"
"Possible values are \"disabled\", \"sdk\", \"aoa\" and "
"\"uhid\".\n"
"\"disabled\" does not send keyboard inputs to the device.\n"
"\"sdk\" uses the Android system API to deliver keyboard\n"
"events to applications.\n"
"\"aoa\" simulates a physical HID keyboard using the AOAv2\n"
"protocol. It may only work over USB.\n"
"\"uhid\" simulates a physical HID keyboard using the Linux "
"UHID kernel module on the device."
"For \"aoa\" and \"uhid\", the keyboard layout must be "
"configured (once and for all) on the device, via Settings -> "
"System -> Languages and input -> Physical keyboard. This "
"settings page can be started directly using the shortcut "
"MOD+k (except in OTG mode) or by executing: `adb shell am "
"start -a android.settings.HARD_KEYBOARD_SETTINGS`.\n"
"This option is only available when a HID keyboard is enabled "
"enabled (or a physical keyboard is connected).\n"
"Also see --mouse.",
},
{
.longopt_id = OPT_KILL_ADB_ON_CLOSE,
.longopt = "kill-adb-on-close",
.text = "Kill adb when scrcpy terminates.",
},
{
.shortopt = 'K',
// deprecated
//.shortopt = 'K', // old, reassigned
.longopt_id = OPT_HID_KEYBOARD_DEPRECATED,
.longopt = "hid-keyboard",
.text = "Simulate a physical keyboard by using HID over AOAv2.\n"
"It provides a better experience for IME users, and allows to "
"generate non-ASCII characters, contrary to the default "
"injection method.\n"
"It may only work over USB.\n"
"The keyboard layout must be configured (once and for all) on "
"the device, via Settings -> System -> Languages and input -> "
"Physical keyboard. This settings page can be started "
"directly: `adb shell am start -a "
"android.settings.HARD_KEYBOARD_SETTINGS`.\n"
"However, the option is only available when the HID keyboard "
"is enabled (or a physical keyboard is connected).\n"
"Also see --hid-mouse.",
},
{
.longopt_id = OPT_LEGACY_PASTE,
@ -432,15 +453,14 @@ static const struct sc_option options[] = {
"Default is 0 (unlimited).",
},
{
.shortopt = 'M',
// deprecated
//.shortopt = 'M', // old, reassigned
.longopt_id = OPT_HID_MOUSE_DEPRECATED,
.longopt = "hid-mouse",
.text = "Simulate a physical mouse by using HID over AOAv2.\n"
"In this mode, the computer mouse is captured to control the "
"device directly (relative mouse mode).\n"
"LAlt, LSuper or RSuper toggle the capture mode, to give "
"control of the mouse back to the computer.\n"
"It may only work over USB.\n"
"Also see --hid-keyboard.",
},
{
.shortopt = 'M',
.text = "Same as --mouse=uhid.",
},
{
.longopt_id = OPT_MAX_FPS,
@ -449,6 +469,25 @@ static const struct sc_option options[] = {
.text = "Limit the frame rate of screen capture (officially supported "
"since Android 10, but may work on earlier versions).",
},
{
.longopt_id = OPT_MOUSE,
.longopt = "mouse",
.argdesc = "mode",
.text = "Select how to send mouse inputs to the device.\n"
"Possible values are \"disabled\", \"sdk\" and \"aoa\".\n"
"\"disabled\" does not send mouse inputs to the device.\n"
"\"sdk\" uses the Android system API to deliver mouse\n"
"events to applications.\n"
"\"aoa\" simulates a physical mouse using the AOAv2 protocol\n"
"It may only work over USB.\n"
"\"uhid\" simulates a physical HID mouse using the Linux UHID "
"kernel module on the device."
"In \"aoa\" and \"uhid\" modes, the computer mouse is captured "
"to control the device directly (relative mouse mode).\n"
"LAlt, LSuper or RSuper toggle the capture mode, to give "
"control of the mouse back to the computer.\n"
"Also see --keyboard.",
},
{
.shortopt = 'n',
.longopt = "no-control",
@ -543,10 +582,10 @@ static const struct sc_option options[] = {
"mirroring is disabled.\n"
"LAlt, LSuper or RSuper toggle the mouse capture mode, to give "
"control of the mouse back to the computer.\n"
"If any of --hid-keyboard or --hid-mouse is set, only enable "
"keyboard or mouse respectively, otherwise enable both.\n"
"Keyboard and mouse may be disabled separately using\n"
"--keyboard=disabled and --mouse=disabled.\n"
"It may only work over USB.\n"
"See --hid-keyboard and --hid-mouse.",
"See --keyboard and --mouse.",
},
{
.shortopt = 'p',
@ -941,6 +980,10 @@ static const struct sc_shortcut shortcuts[] = {
.shortcuts = { "MOD+Shift+v" },
.text = "Inject computer clipboard text as a sequence of key events",
},
{
.shortcuts = { "MOD+k" },
.text = "Open keyboard settings on the device (for HID keyboard only)",
},
{
.shortcuts = { "MOD+i" },
.text = "Enable/disable FPS counter (print frames/second in logs)",
@ -1906,6 +1949,69 @@ parse_camera_fps(const char *s, uint16_t *camera_fps) {
return true;
}
static bool
parse_keyboard(const char *optarg, enum sc_keyboard_input_mode *mode) {
if (!strcmp(optarg, "disabled")) {
*mode = SC_KEYBOARD_INPUT_MODE_DISABLED;
return true;
}
if (!strcmp(optarg, "sdk")) {
*mode = SC_KEYBOARD_INPUT_MODE_SDK;
return true;
}
if (!strcmp(optarg, "aoa")) {
#ifdef HAVE_USB
*mode = SC_KEYBOARD_INPUT_MODE_AOA;
return true;
#else
LOGE("--keyboard=aoa is disabled.");
return false;
#endif
}
if (!strcmp(optarg, "uhid")) {
*mode = SC_KEYBOARD_INPUT_MODE_UHID;
return true;
}
LOGE("Unsupported keyboard: %s (expected disabled, sdk, aoa or uhid)",
optarg);
return false;
}
static bool
parse_mouse(const char *optarg, enum sc_mouse_input_mode *mode) {
if (!strcmp(optarg, "disabled")) {
*mode = SC_MOUSE_INPUT_MODE_DISABLED;
return true;
}
if (!strcmp(optarg, "sdk")) {
*mode = SC_MOUSE_INPUT_MODE_SDK;
return true;
}
if (!strcmp(optarg, "aoa")) {
#ifdef HAVE_USB
*mode = SC_MOUSE_INPUT_MODE_AOA;
return true;
#else
LOGE("--mouse=aoa is disabled.");
return false;
#endif
}
if (!strcmp(optarg, "uhid")) {
*mode = SC_MOUSE_INPUT_MODE_UHID;
return true;
}
LOGE("Unsupported mouse: %s (expected disabled, sdk, aoa or uhid)", optarg);
return false;
}
static bool
parse_time_limit(const char *s, sc_tick *tick) {
long value;
@ -1994,13 +2100,17 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
args->help = true;
break;
case 'K':
#ifdef HAVE_USB
opts->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_HID;
opts->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_UHID;
break;
#else
LOGE("HID over AOA (-K/--hid-keyboard) is disabled.");
case OPT_KEYBOARD:
if (!parse_keyboard(optarg, &opts->keyboard_input_mode)) {
return false;
}
break;
case OPT_HID_KEYBOARD_DEPRECATED:
LOGE("--hid-keyboard has been removed, use --keyboard=aoa or "
"--keyboard=uhid instead.");
return false;
#endif
case OPT_MAX_FPS:
if (!parse_max_fps(optarg, &opts->max_fps)) {
return false;
@ -2012,13 +2122,17 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
}
break;
case 'M':
#ifdef HAVE_USB
opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_HID;
opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_UHID;
break;
#else
LOGE("HID over AOA (-M/--hid-mouse) is disabled.");
case OPT_MOUSE:
if (!parse_mouse(optarg, &opts->mouse_input_mode)) {
return false;
}
break;
case OPT_HID_MOUSE_DEPRECATED:
LOGE("--hid-mouse has been removed, use --mouse=aoa or "
"--mouse=uhid instead.");
return false;
#endif
case OPT_LOCK_VIDEO_ORIENTATION:
if (!parse_lock_video_orientation(optarg,
&opts->lock_video_orientation)) {
@ -2465,6 +2579,31 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
}
#endif
if (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AUTO) {
opts->keyboard_input_mode = otg ? SC_KEYBOARD_INPUT_MODE_AOA
: SC_KEYBOARD_INPUT_MODE_SDK;
}
if (opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_AUTO) {
opts->mouse_input_mode = otg ? SC_MOUSE_INPUT_MODE_AOA
: SC_MOUSE_INPUT_MODE_SDK;
}
if (otg) {
enum sc_keyboard_input_mode kmode = opts->keyboard_input_mode;
if (kmode != SC_KEYBOARD_INPUT_MODE_AOA
&& kmode != SC_KEYBOARD_INPUT_MODE_DISABLED) {
LOGE("In OTG mode, --keyboard only supports aoa or disabled.");
return false;
}
enum sc_mouse_input_mode mmode = opts->mouse_input_mode;
if (mmode != SC_MOUSE_INPUT_MODE_AOA
&& mmode != SC_MOUSE_INPUT_MODE_DISABLED) {
LOGE("In OTG mode, --mouse only supports aoa or disabled.");
return false;
}
}
if ((opts->tunnel_host || opts->tunnel_port) && !opts->force_adb_forward) {
LOGI("Tunnel host/port is set, "
"--force-adb-forward automatically enabled.");
@ -2625,12 +2764,12 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
}
# ifdef _WIN32
if (!otg && (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID
|| opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID)) {
if (!otg && (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA
|| opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA)) {
LOGE("On Windows, it is not possible to open a USB device already open "
"by another process (like adb).");
LOGE("Therefore, -K/--hid-keyboard and -M/--hid-mouse may only work in "
"OTG mode (--otg).");
LOGE("Therefore, --keyboard=aoa and --mouse=aoa may only work in OTG"
"mode (--otg).");
return false;
}
# endif

View File

@ -146,10 +146,22 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf) {
case SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE:
buf[1] = msg->set_screen_power_mode.mode;
return 2;
case SC_CONTROL_MSG_TYPE_UHID_CREATE:
sc_write16be(&buf[1], msg->uhid_create.id);
sc_write16be(&buf[3], msg->uhid_create.report_desc_size);
memcpy(&buf[5], msg->uhid_create.report_desc,
msg->uhid_create.report_desc_size);
return 5 + msg->uhid_create.report_desc_size;
case SC_CONTROL_MSG_TYPE_UHID_INPUT:
sc_write16be(&buf[1], msg->uhid_input.id);
sc_write16be(&buf[3], msg->uhid_input.size);
memcpy(&buf[5], msg->uhid_input.data, msg->uhid_input.size);
return 5 + msg->uhid_input.size;
case SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL:
case SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL:
case SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS:
case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
// no additional data
return 1;
default:
@ -242,6 +254,26 @@ sc_control_msg_log(const struct sc_control_msg *msg) {
case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
LOG_CMSG("rotate device");
break;
case SC_CONTROL_MSG_TYPE_UHID_CREATE:
LOG_CMSG("UHID create [%" PRIu16 "] report_desc_size=%" PRIu16,
msg->uhid_create.id, msg->uhid_create.report_desc_size);
break;
case SC_CONTROL_MSG_TYPE_UHID_INPUT: {
char *hex = sc_str_to_hex_string(msg->uhid_input.data,
msg->uhid_input.size);
if (hex) {
LOG_CMSG("UHID input [%" PRIu16 "] %s",
msg->uhid_input.id, hex);
free(hex);
} else {
LOG_CMSG("UHID input [%" PRIu16 "] size=%" PRIu16,
msg->uhid_input.id, msg->uhid_input.size);
}
break;
}
case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
LOG_CMSG("open hard keyboard settings");
break;
default:
LOG_CMSG("unknown type: %u", (unsigned) msg->type);
break;

View File

@ -10,6 +10,7 @@
#include "android/input.h"
#include "android/keycodes.h"
#include "coords.h"
#include "hid/hid_event.h"
#define SC_CONTROL_MSG_MAX_SIZE (1 << 18) // 256k
@ -37,6 +38,9 @@ enum sc_control_msg_type {
SC_CONTROL_MSG_TYPE_SET_CLIPBOARD,
SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE,
SC_CONTROL_MSG_TYPE_ROTATE_DEVICE,
SC_CONTROL_MSG_TYPE_UHID_CREATE,
SC_CONTROL_MSG_TYPE_UHID_INPUT,
SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
};
enum sc_screen_power_mode {
@ -92,6 +96,16 @@ struct sc_control_msg {
struct {
enum sc_screen_power_mode mode;
} set_screen_power_mode;
struct {
uint16_t id;
uint16_t report_desc_size;
const uint8_t *report_desc; // pointer to static data
} uhid_create;
struct {
uint16_t id;
uint16_t size;
uint8_t data[SC_HID_MAX_SIZE];
} uhid_input;
};
};

View File

@ -7,8 +7,7 @@
#define SC_CONTROL_MSG_QUEUE_MAX 64
bool
sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
struct sc_acksync *acksync) {
sc_controller_init(struct sc_controller *controller, sc_socket control_socket) {
sc_vecdeque_init(&controller->queue);
bool ok = sc_vecdeque_reserve(&controller->queue, SC_CONTROL_MSG_QUEUE_MAX);
@ -16,7 +15,7 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
return false;
}
ok = sc_receiver_init(&controller->receiver, control_socket, acksync);
ok = sc_receiver_init(&controller->receiver, control_socket);
if (!ok) {
sc_vecdeque_destroy(&controller->queue);
return false;
@ -43,6 +42,14 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
return true;
}
void
sc_controller_configure(struct sc_controller *controller,
struct sc_acksync *acksync,
struct sc_uhid_devices *uhid_devices) {
controller->receiver.acksync = acksync;
controller->receiver.uhid_devices = uhid_devices;
}
void
sc_controller_destroy(struct sc_controller *controller) {
sc_cond_destroy(&controller->msg_cond);

View File

@ -25,8 +25,12 @@ struct sc_controller {
};
bool
sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
struct sc_acksync *acksync);
sc_controller_init(struct sc_controller *controller, sc_socket control_socket);
void
sc_controller_configure(struct sc_controller *controller,
struct sc_acksync *acksync,
struct sc_uhid_devices *uhid_devices);
void
sc_controller_destroy(struct sc_controller *controller);

View File

@ -46,6 +46,31 @@ sc_device_msg_deserialize(const uint8_t *buf, size_t len,
msg->ack_clipboard.sequence = sequence;
return 9;
}
case DEVICE_MSG_TYPE_UHID_OUTPUT: {
if (len < 5) {
// at least id + size
return 0; // not available
}
uint16_t id = sc_read16be(&buf[1]);
size_t size = sc_read16be(&buf[3]);
if (size < len - 5) {
return 0; // not available
}
uint8_t *data = malloc(size);
if (!data) {
LOG_OOM();
return -1;
}
if (size) {
memcpy(data, &buf[5], size);
}
msg->uhid_output.id = id;
msg->uhid_output.size = size;
msg->uhid_output.data = data;
return 5 + size;
}
default:
LOGW("Unknown device message type: %d", (int) msg->type);
return -1; // error, we cannot recover
@ -54,7 +79,15 @@ sc_device_msg_deserialize(const uint8_t *buf, size_t len,
void
sc_device_msg_destroy(struct sc_device_msg *msg) {
if (msg->type == DEVICE_MSG_TYPE_CLIPBOARD) {
switch (msg->type) {
case DEVICE_MSG_TYPE_CLIPBOARD:
free(msg->clipboard.text);
break;
case DEVICE_MSG_TYPE_UHID_OUTPUT:
free(msg->uhid_output.data);
break;
default:
// nothing to do
break;
}
}

View File

@ -14,6 +14,7 @@
enum sc_device_msg_type {
DEVICE_MSG_TYPE_CLIPBOARD,
DEVICE_MSG_TYPE_ACK_CLIPBOARD,
DEVICE_MSG_TYPE_UHID_OUTPUT,
};
struct sc_device_msg {
@ -25,6 +26,11 @@ struct sc_device_msg {
struct {
uint64_t sequence;
} ack_clipboard;
struct {
uint16_t id;
uint16_t size;
uint8_t *data; // owned, to be freed by free()
} uhid_output;
};
};

18
app/src/hid/hid_event.h Normal file
View File

@ -0,0 +1,18 @@
#ifndef SC_HID_EVENT_H
#define SC_HID_EVENT_H
#include "common.h"
#include <stdint.h>
#define SC_HID_MAX_SIZE 8
struct sc_hid_event {
uint8_t data[SC_HID_MAX_SIZE];
uint8_t size;
};
char *
sc_hid_event_to_string(const struct sc_hid_event *event, size_t max_data_bytes);
#endif

View File

@ -1,40 +1,34 @@
#include "hid_keyboard.h"
#include <assert.h>
#include <string.h>
#include "input_events.h"
#include "util/log.h"
/** Downcast key processor to hid_keyboard */
#define DOWNCAST(KP) container_of(KP, struct sc_hid_keyboard, key_processor)
#define SC_HID_MOD_NONE 0x00
#define SC_HID_MOD_LEFT_CONTROL (1 << 0)
#define SC_HID_MOD_LEFT_SHIFT (1 << 1)
#define SC_HID_MOD_LEFT_ALT (1 << 2)
#define SC_HID_MOD_LEFT_GUI (1 << 3)
#define SC_HID_MOD_RIGHT_CONTROL (1 << 4)
#define SC_HID_MOD_RIGHT_SHIFT (1 << 5)
#define SC_HID_MOD_RIGHT_ALT (1 << 6)
#define SC_HID_MOD_RIGHT_GUI (1 << 7)
#define HID_KEYBOARD_ACCESSORY_ID 1
#define HID_MODIFIER_NONE 0x00
#define HID_MODIFIER_LEFT_CONTROL (1 << 0)
#define HID_MODIFIER_LEFT_SHIFT (1 << 1)
#define HID_MODIFIER_LEFT_ALT (1 << 2)
#define HID_MODIFIER_LEFT_GUI (1 << 3)
#define HID_MODIFIER_RIGHT_CONTROL (1 << 4)
#define HID_MODIFIER_RIGHT_SHIFT (1 << 5)
#define HID_MODIFIER_RIGHT_ALT (1 << 6)
#define HID_MODIFIER_RIGHT_GUI (1 << 7)
#define HID_KEYBOARD_INDEX_MODIFIER 0
#define HID_KEYBOARD_INDEX_KEYS 2
#define SC_HID_KEYBOARD_INDEX_MODS 0
#define SC_HID_KEYBOARD_INDEX_KEYS 2
// USB HID protocol says 6 keys in an event is the requirement for BIOS
// keyboard support, though OS could support more keys via modifying the report
// desc. 6 should be enough for scrcpy.
#define HID_KEYBOARD_MAX_KEYS 6
#define HID_KEYBOARD_EVENT_SIZE \
(HID_KEYBOARD_INDEX_KEYS + HID_KEYBOARD_MAX_KEYS)
#define SC_HID_KEYBOARD_MAX_KEYS 6
#define SC_HID_KEYBOARD_EVENT_SIZE \
(SC_HID_KEYBOARD_INDEX_KEYS + SC_HID_KEYBOARD_MAX_KEYS)
#define HID_RESERVED 0x00
#define HID_ERROR_ROLL_OVER 0x01
#define SC_HID_RESERVED 0x00
#define SC_HID_ERROR_ROLL_OVER 0x01
/**
* For HID over AOAv2, only report descriptor is needed.
* For HID, only report descriptor is needed.
*
* The specification is available here:
* <https://www.usb.org/sites/default/files/hid1_11.pdf>
@ -53,7 +47,7 @@
*
* (change vid:pid' to your device's vendor ID and product ID).
*/
static const unsigned char keyboard_report_desc[] = {
const uint8_t SC_HID_KEYBOARD_REPORT_DESC[] = {
// Usage Page (Generic Desktop)
0x05, 0x01,
// Usage (Keyboard)
@ -119,7 +113,7 @@ static const unsigned char keyboard_report_desc[] = {
// Report Size (8)
0x75, 0x08,
// Report Count (6)
0x95, HID_KEYBOARD_MAX_KEYS,
0x95, SC_HID_KEYBOARD_MAX_KEYS,
// Input (Data, Array): Keys
0x81, 0x00,
@ -127,6 +121,9 @@ static const unsigned char keyboard_report_desc[] = {
0xC0
};
const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN =
sizeof(SC_HID_KEYBOARD_REPORT_DESC);
/**
* A keyboard HID event is 8 bytes long:
*
@ -201,51 +198,50 @@ static const unsigned char keyboard_report_desc[] = {
* +---------------+
*/
static unsigned char
sdl_keymod_to_hid_modifiers(uint16_t mod) {
unsigned char modifiers = HID_MODIFIER_NONE;
if (mod & SC_MOD_LCTRL) {
modifiers |= HID_MODIFIER_LEFT_CONTROL;
}
if (mod & SC_MOD_LSHIFT) {
modifiers |= HID_MODIFIER_LEFT_SHIFT;
}
if (mod & SC_MOD_LALT) {
modifiers |= HID_MODIFIER_LEFT_ALT;
}
if (mod & SC_MOD_LGUI) {
modifiers |= HID_MODIFIER_LEFT_GUI;
}
if (mod & SC_MOD_RCTRL) {
modifiers |= HID_MODIFIER_RIGHT_CONTROL;
}
if (mod & SC_MOD_RSHIFT) {
modifiers |= HID_MODIFIER_RIGHT_SHIFT;
}
if (mod & SC_MOD_RALT) {
modifiers |= HID_MODIFIER_RIGHT_ALT;
}
if (mod & SC_MOD_RGUI) {
modifiers |= HID_MODIFIER_RIGHT_GUI;
}
return modifiers;
static void
sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
hid_event->size = SC_HID_KEYBOARD_EVENT_SIZE;
uint8_t *data = hid_event->data;
data[SC_HID_KEYBOARD_INDEX_MODS] = SC_HID_MOD_NONE;
data[1] = SC_HID_RESERVED;
memset(&data[SC_HID_KEYBOARD_INDEX_KEYS], 0, SC_HID_KEYBOARD_MAX_KEYS);
}
static bool
sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
unsigned char *buffer = malloc(HID_KEYBOARD_EVENT_SIZE);
if (!buffer) {
LOG_OOM();
return false;
static uint16_t
sc_hid_mod_from_sdl_keymod(uint16_t mod) {
uint16_t mods = SC_HID_MOD_NONE;
if (mod & SC_MOD_LCTRL) {
mods |= SC_HID_MOD_LEFT_CONTROL;
}
if (mod & SC_MOD_LSHIFT) {
mods |= SC_HID_MOD_LEFT_SHIFT;
}
if (mod & SC_MOD_LALT) {
mods |= SC_HID_MOD_LEFT_ALT;
}
if (mod & SC_MOD_LGUI) {
mods |= SC_HID_MOD_LEFT_GUI;
}
if (mod & SC_MOD_RCTRL) {
mods |= SC_HID_MOD_RIGHT_CONTROL;
}
if (mod & SC_MOD_RSHIFT) {
mods |= SC_HID_MOD_RIGHT_SHIFT;
}
if (mod & SC_MOD_RALT) {
mods |= SC_HID_MOD_RIGHT_ALT;
}
if (mod & SC_MOD_RGUI) {
mods |= SC_HID_MOD_RIGHT_GUI;
}
return mods;
}
buffer[HID_KEYBOARD_INDEX_MODIFIER] = HID_MODIFIER_NONE;
buffer[1] = HID_RESERVED;
memset(&buffer[HID_KEYBOARD_INDEX_KEYS], 0, HID_KEYBOARD_MAX_KEYS);
sc_hid_event_init(hid_event, HID_KEYBOARD_ACCESSORY_ID, buffer,
HID_KEYBOARD_EVENT_SIZE);
return true;
void
sc_hid_keyboard_init(struct sc_hid_keyboard *hid) {
memset(hid->keys, false, SC_HID_KEYBOARD_KEYS);
}
static inline bool
@ -253,8 +249,8 @@ scancode_is_modifier(enum sc_scancode scancode) {
return scancode >= SC_SCANCODE_LCTRL && scancode <= SC_SCANCODE_RGUI;
}
static bool
convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
bool
sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
struct sc_hid_event *hid_event,
const struct sc_key_event *event) {
enum sc_scancode scancode = event->scancode;
@ -268,39 +264,37 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
return false;
}
if (!sc_hid_keyboard_event_init(hid_event)) {
LOGW("Could not initialize HID keyboard event");
return false;
}
sc_hid_keyboard_event_init(hid_event);
unsigned char modifiers = sdl_keymod_to_hid_modifiers(event->mods_state);
uint16_t mods = sc_hid_mod_from_sdl_keymod(event->mods_state);
if (scancode < SC_HID_KEYBOARD_KEYS) {
// Pressed is true and released is false
kb->keys[scancode] = (event->action == SC_ACTION_DOWN);
hid->keys[scancode] = (event->action == SC_ACTION_DOWN);
LOGV("keys[%02x] = %s", scancode,
kb->keys[scancode] ? "true" : "false");
hid->keys[scancode] ? "true" : "false");
}
hid_event->buffer[HID_KEYBOARD_INDEX_MODIFIER] = modifiers;
hid_event->data[SC_HID_KEYBOARD_INDEX_MODS] = mods;
unsigned char *keys_buffer = &hid_event->buffer[HID_KEYBOARD_INDEX_KEYS];
uint8_t *keys_data = &hid_event->data[SC_HID_KEYBOARD_INDEX_KEYS];
// Re-calculate pressed keys every time
int keys_pressed_count = 0;
for (int i = 0; i < SC_HID_KEYBOARD_KEYS; ++i) {
if (kb->keys[i]) {
if (hid->keys[i]) {
// USB HID protocol says that if keys exceeds report count, a
// phantom state should be reported
if (keys_pressed_count >= HID_KEYBOARD_MAX_KEYS) {
if (keys_pressed_count >= SC_HID_KEYBOARD_MAX_KEYS) {
// Phantom state:
// - Modifiers
// - Reserved
// - ErrorRollOver * HID_MAX_KEYS
memset(keys_buffer, HID_ERROR_ROLL_OVER, HID_KEYBOARD_MAX_KEYS);
memset(keys_data, SC_HID_ERROR_ROLL_OVER,
SC_HID_KEYBOARD_MAX_KEYS);
goto end;
}
keys_buffer[keys_pressed_count] = i;
keys_data[keys_pressed_count] = i;
++keys_pressed_count;
}
}
@ -308,124 +302,32 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
end:
LOGV("hid keyboard: key %-4s scancode=%02x (%u) mod=%02x",
event->action == SC_ACTION_DOWN ? "down" : "up", event->scancode,
event->scancode, modifiers);
event->scancode, mods);
return true;
}
static bool
push_mod_lock_state(struct sc_hid_keyboard *kb, uint16_t mods_state) {
bool
sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
uint16_t mods_state) {
bool capslock = mods_state & SC_MOD_CAPS;
bool numlock = mods_state & SC_MOD_NUM;
if (!capslock && !numlock) {
// Nothing to do
return true;
}
struct sc_hid_event hid_event;
if (!sc_hid_keyboard_event_init(&hid_event)) {
LOGW("Could not initialize HID keyboard event");
return false;
}
sc_hid_keyboard_event_init(event);
unsigned i = 0;
if (capslock) {
hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
++i;
}
if (numlock) {
hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
++i;
}
if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
sc_hid_event_destroy(&hid_event);
LOGW("Could not request HID event (mod lock state)");
return false;
}
LOGD("HID keyboard state synchronized");
return true;
}
static void
sc_key_processor_process_key(struct sc_key_processor *kp,
const struct sc_key_event *event,
uint64_t ack_to_wait) {
if (event->repeat) {
// In USB HID protocol, key repeat is handled by the host (Android), so
// just ignore key repeat here.
return;
}
struct sc_hid_keyboard *kb = DOWNCAST(kp);
struct sc_hid_event hid_event;
// Not all keys are supported, just ignore unsupported keys
if (convert_hid_keyboard_event(kb, &hid_event, event)) {
if (!kb->mod_lock_synchronized) {
// Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
// keyboard state
if (push_mod_lock_state(kb, event->mods_state)) {
kb->mod_lock_synchronized = true;
}
}
if (ack_to_wait) {
// Ctrl+v is pressed, so clipboard synchronization has been
// requested. Wait until clipboard synchronization is acknowledged
// by the server, otherwise it could paste the old clipboard
// content.
hid_event.ack_to_wait = ack_to_wait;
}
if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
sc_hid_event_destroy(&hid_event);
LOGW("Could not request HID event (key)");
}
}
}
bool
sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa) {
kb->aoa = aoa;
bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
keyboard_report_desc,
ARRAY_LEN(keyboard_report_desc));
if (!ok) {
LOGW("Register HID keyboard failed");
return false;
}
// Reset all states
memset(kb->keys, false, SC_HID_KEYBOARD_KEYS);
kb->mod_lock_synchronized = false;
static const struct sc_key_processor_ops ops = {
.process_key = sc_key_processor_process_key,
// Never forward text input via HID (all the keys are injected
// separately)
.process_text = NULL,
};
// Clipboard synchronization is requested over the control socket, while HID
// events are sent over AOA, so it must wait for clipboard synchronization
// to be acknowledged by the device before injecting Ctrl+v.
kb->key_processor.async_paste = true;
kb->key_processor.ops = &ops;
return true;
}
void
sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb) {
// Unregister HID keyboard so the soft keyboard shows again on Android
bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
if (!ok) {
LOGW("Could not unregister HID keyboard");
}
}

View File

@ -5,8 +5,8 @@
#include <stdbool.h>
#include "aoa_hid.h"
#include "trait/key_processor.h"
#include "hid/hid_event.h"
#include "input_events.h"
// See "SDL2/SDL_scancode.h".
// Maybe SDL_Keycode is used by most people, but SDL_Scancode is taken from USB
@ -14,6 +14,9 @@
// 0x65 is Application, typically AT-101 Keyboard ends here.
#define SC_HID_KEYBOARD_KEYS 0x66
extern const uint8_t SC_HID_KEYBOARD_REPORT_DESC[];
extern const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN;
/**
* HID keyboard events are sequence-based, every time keyboard state changes
* it sends an array of currently pressed keys, the host is responsible for
@ -27,18 +30,19 @@
* phantom state.
*/
struct sc_hid_keyboard {
struct sc_key_processor key_processor; // key processor trait
struct sc_aoa *aoa;
bool keys[SC_HID_KEYBOARD_KEYS];
bool mod_lock_synchronized;
};
bool
sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa);
void
sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb);
sc_hid_keyboard_init(struct sc_hid_keyboard *hid);
bool
sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
struct sc_hid_event *hid_event,
const struct sc_key_event *event);
bool
sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
uint16_t mods_state);
#endif

192
app/src/hid/hid_mouse.c Normal file
View File

@ -0,0 +1,192 @@
#include "hid_mouse.h"
// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position,
// 1 byte for wheel motion
#define HID_MOUSE_EVENT_SIZE 4
/**
* Mouse descriptor from the specification:
* <https://www.usb.org/sites/default/files/hid1_11.pdf>
*
* Appendix E (p71): §E.10 Report Descriptor (Mouse)
*
* The usage tags (like Wheel) are listed in "HID Usage Tables":
* <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
* §4 Generic Desktop Page (0x01) (p26)
*/
const uint8_t SC_HID_MOUSE_REPORT_DESC[] = {
// Usage Page (Generic Desktop)
0x05, 0x01,
// Usage (Mouse)
0x09, 0x02,
// Collection (Application)
0xA1, 0x01,
// Usage (Pointer)
0x09, 0x01,
// Collection (Physical)
0xA1, 0x00,
// Usage Page (Buttons)
0x05, 0x09,
// Usage Minimum (1)
0x19, 0x01,
// Usage Maximum (5)
0x29, 0x05,
// Logical Minimum (0)
0x15, 0x00,
// Logical Maximum (1)
0x25, 0x01,
// Report Count (5)
0x95, 0x05,
// Report Size (1)
0x75, 0x01,
// Input (Data, Variable, Absolute): 5 buttons bits
0x81, 0x02,
// Report Count (1)
0x95, 0x01,
// Report Size (3)
0x75, 0x03,
// Input (Constant): 3 bits padding
0x81, 0x01,
// Usage Page (Generic Desktop)
0x05, 0x01,
// Usage (X)
0x09, 0x30,
// Usage (Y)
0x09, 0x31,
// Usage (Wheel)
0x09, 0x38,
// Local Minimum (-127)
0x15, 0x81,
// Local Maximum (127)
0x25, 0x7F,
// Report Size (8)
0x75, 0x08,
// Report Count (3)
0x95, 0x03,
// Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
0x81, 0x06,
// End Collection
0xC0,
// End Collection
0xC0,
};
const size_t SC_HID_MOUSE_REPORT_DESC_LEN =
sizeof(SC_HID_MOUSE_REPORT_DESC);
/**
* A mouse HID event is 4 bytes long:
*
* - byte 0: buttons state
* - byte 1: relative x motion (signed byte from -127 to 127)
* - byte 2: relative y motion (signed byte from -127 to 127)
* - byte 3: wheel motion (-1, 0 or 1)
*
* 7 6 5 4 3 2 1 0
* +---------------+
* byte 0: |0 0 0 . . . . .| buttons state
* +---------------+
* ^ ^ ^ ^ ^
* | | | | `- left button
* | | | `--- right button
* | | `----- middle button
* | `------- button 4
* `--------- button 5
*
* +---------------+
* byte 1: |. . . . . . . .| relative x motion
* +---------------+
* byte 2: |. . . . . . . .| relative y motion
* +---------------+
* byte 3: |. . . . . . . .| wheel motion
* +---------------+
*
* As an example, here is the report for a motion of (x=5, y=-4) with left
* button pressed:
*
* +---------------+
* |0 0 0 0 0 0 0 1| left button pressed
* +---------------+
* |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
* +---------------+
* |1 1 1 1 1 1 0 0| relative y motion (y = -4)
* +---------------+
* |0 0 0 0 0 0 0 0| wheel motion
* +---------------+
*/
static void
sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
hid_event->size = HID_MOUSE_EVENT_SIZE;
// Leave hid_event->data uninitialized, it will be fully initialized by
// callers
}
static uint8_t
sc_hid_buttons_from_buttons_state(uint8_t buttons_state) {
uint8_t c = 0;
if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
c |= 1 << 0;
}
if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
c |= 1 << 1;
}
if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
c |= 1 << 2;
}
if (buttons_state & SC_MOUSE_BUTTON_X1) {
c |= 1 << 3;
}
if (buttons_state & SC_MOUSE_BUTTON_X2) {
c |= 1 << 4;
}
return c;
}
void
sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
const struct sc_mouse_motion_event *event) {
sc_hid_mouse_event_init(hid_event);
uint8_t *data = hid_event->data;
data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
data[1] = CLAMP(event->xrel, -127, 127);
data[2] = CLAMP(event->yrel, -127, 127);
data[3] = 0; // wheel coordinates only used for scrolling
}
void
sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
const struct sc_mouse_click_event *event) {
sc_hid_mouse_event_init(hid_event);
uint8_t *data = hid_event->data;
data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
data[1] = 0; // no x motion
data[2] = 0; // no y motion
data[3] = 0; // wheel coordinates only used for scrolling
}
void
sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
const struct sc_mouse_scroll_event *event) {
sc_hid_mouse_event_init(hid_event);
uint8_t *data = hid_event->data;
data[0] = 0; // buttons state irrelevant (and unknown)
data[1] = 0; // no x motion
data[2] = 0; // no y motion
// In practice, vscroll is always -1, 0 or 1, but in theory other values
// are possible
data[3] = CLAMP(event->vscroll, -127, 127);
// Horizontal scrolling ignored
}

26
app/src/hid/hid_mouse.h Normal file
View File

@ -0,0 +1,26 @@
#ifndef SC_HID_MOUSE_H
#define SC_HID_MOUSE_H
#endif
#include "common.h"
#include <stdbool.h>
#include "hid/hid_event.h"
#include "input_events.h"
extern const uint8_t SC_HID_MOUSE_REPORT_DESC[];
extern const size_t SC_HID_MOUSE_REPORT_DESC_LEN;
void
sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
const struct sc_mouse_motion_event *event);
void
sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
const struct sc_mouse_click_event *event);
void
sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
const struct sc_mouse_scroll_event *event);

View File

@ -52,8 +52,11 @@ is_shortcut_mod(struct sc_input_manager *im, uint16_t sdl_mod) {
void
sc_input_manager_init(struct sc_input_manager *im,
const struct sc_input_manager_params *params) {
assert(!params->controller || (params->kp && params->kp->ops));
assert(!params->controller || (params->mp && params->mp->ops));
// A key/mouse processor may not be present if there is no controller
assert((!params->kp && !params->mp) || params->controller);
// A processor must have ops initialized
assert(!params->kp || params->kp->ops);
assert(!params->mp || params->mp->ops);
im->controller = params->controller;
im->fp = params->fp;
@ -87,8 +90,10 @@ sc_input_manager_init(struct sc_input_manager *im,
}
static void
send_keycode(struct sc_controller *controller, enum android_keycode keycode,
send_keycode(struct sc_input_manager *im, enum android_keycode keycode,
enum sc_action action, const char *name) {
assert(im->controller && im->kp);
// send DOWN event
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_INJECT_KEYCODE;
@ -99,100 +104,109 @@ send_keycode(struct sc_controller *controller, enum android_keycode keycode,
msg.inject_keycode.metastate = 0;
msg.inject_keycode.repeat = 0;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request 'inject %s'", name);
}
}
static inline void
action_home(struct sc_controller *controller, enum sc_action action) {
send_keycode(controller, AKEYCODE_HOME, action, "HOME");
action_home(struct sc_input_manager *im, enum sc_action action) {
send_keycode(im, AKEYCODE_HOME, action, "HOME");
}
static inline void
action_back(struct sc_controller *controller, enum sc_action action) {
send_keycode(controller, AKEYCODE_BACK, action, "BACK");
action_back(struct sc_input_manager *im, enum sc_action action) {
send_keycode(im, AKEYCODE_BACK, action, "BACK");
}
static inline void
action_app_switch(struct sc_controller *controller, enum sc_action action) {
send_keycode(controller, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
action_app_switch(struct sc_input_manager *im, enum sc_action action) {
send_keycode(im, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
}
static inline void
action_power(struct sc_controller *controller, enum sc_action action) {
send_keycode(controller, AKEYCODE_POWER, action, "POWER");
action_power(struct sc_input_manager *im, enum sc_action action) {
send_keycode(im, AKEYCODE_POWER, action, "POWER");
}
static inline void
action_volume_up(struct sc_controller *controller, enum sc_action action) {
send_keycode(controller, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
action_volume_up(struct sc_input_manager *im, enum sc_action action) {
send_keycode(im, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
}
static inline void
action_volume_down(struct sc_controller *controller, enum sc_action action) {
send_keycode(controller, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
action_volume_down(struct sc_input_manager *im, enum sc_action action) {
send_keycode(im, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
}
static inline void
action_menu(struct sc_controller *controller, enum sc_action action) {
send_keycode(controller, AKEYCODE_MENU, action, "MENU");
action_menu(struct sc_input_manager *im, enum sc_action action) {
send_keycode(im, AKEYCODE_MENU, action, "MENU");
}
// turn the screen on if it was off, press BACK otherwise
// If the screen is off, it is turned on only on ACTION_DOWN
static void
press_back_or_turn_screen_on(struct sc_controller *controller,
press_back_or_turn_screen_on(struct sc_input_manager *im,
enum sc_action action) {
assert(im->controller && im->kp);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_BACK_OR_SCREEN_ON;
msg.back_or_screen_on.action = action == SC_ACTION_DOWN
? AKEY_EVENT_ACTION_DOWN
: AKEY_EVENT_ACTION_UP;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request 'press back or turn screen on'");
}
}
static void
expand_notification_panel(struct sc_controller *controller) {
expand_notification_panel(struct sc_input_manager *im) {
assert(im->controller);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request 'expand notification panel'");
}
}
static void
expand_settings_panel(struct sc_controller *controller) {
expand_settings_panel(struct sc_input_manager *im) {
assert(im->controller);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request 'expand settings panel'");
}
}
static void
collapse_panels(struct sc_controller *controller) {
collapse_panels(struct sc_input_manager *im) {
assert(im->controller);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request 'collapse notification panel'");
}
}
static bool
get_device_clipboard(struct sc_controller *controller,
enum sc_copy_key copy_key) {
get_device_clipboard(struct sc_input_manager *im, enum sc_copy_key copy_key) {
assert(im->controller && im->kp);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_GET_CLIPBOARD;
msg.get_clipboard.copy_key = copy_key;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request 'get device clipboard'");
return false;
}
@ -201,8 +215,10 @@ get_device_clipboard(struct sc_controller *controller,
}
static bool
set_device_clipboard(struct sc_controller *controller, bool paste,
set_device_clipboard(struct sc_input_manager *im, bool paste,
uint64_t sequence) {
assert(im->controller && im->kp);
char *text = SDL_GetClipboardText();
if (!text) {
LOGW("Could not get clipboard text: %s", SDL_GetError());
@ -222,7 +238,7 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
msg.set_clipboard.text = text_dup;
msg.set_clipboard.paste = paste;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
free(text_dup);
LOGW("Could not request 'set device clipboard'");
return false;
@ -232,19 +248,23 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
}
static void
set_screen_power_mode(struct sc_controller *controller,
set_screen_power_mode(struct sc_input_manager *im,
enum sc_screen_power_mode mode) {
assert(im->controller);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
msg.set_screen_power_mode.mode = mode;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request 'set screen power mode'");
}
}
static void
switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
switch_fps_counter_state(struct sc_input_manager *im) {
struct sc_fps_counter *fps_counter = &im->screen->fps_counter;
// the started state can only be written from the current thread, so there
// is no ToCToU issue
if (sc_fps_counter_is_started(fps_counter)) {
@ -256,7 +276,9 @@ switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
}
static void
clipboard_paste(struct sc_controller *controller) {
clipboard_paste(struct sc_input_manager *im) {
assert(im->controller && im->kp);
char *text = SDL_GetClipboardText();
if (!text) {
LOGW("Could not get clipboard text: %s", SDL_GetError());
@ -278,25 +300,40 @@ clipboard_paste(struct sc_controller *controller) {
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_INJECT_TEXT;
msg.inject_text.text = text_dup;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
free(text_dup);
LOGW("Could not request 'paste clipboard'");
}
}
static void
rotate_device(struct sc_controller *controller) {
rotate_device(struct sc_input_manager *im) {
assert(im->controller);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_ROTATE_DEVICE;
if (!sc_controller_push_msg(controller, &msg)) {
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request device rotation");
}
}
static void
apply_orientation_transform(struct sc_screen *screen,
open_hard_keyboard_settings(struct sc_input_manager *im) {
assert(im->controller);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS;
if (!sc_controller_push_msg(im->controller, &msg)) {
LOGW("Could not request opening hard keyboard settings");
}
}
static void
apply_orientation_transform(struct sc_input_manager *im,
enum sc_orientation transform) {
struct sc_screen *screen = im->screen;
enum sc_orientation new_orientation =
sc_orientation_apply(screen->orientation, transform);
sc_screen_set_orientation(screen, new_orientation);
@ -364,7 +401,7 @@ static void
sc_input_manager_process_key(struct sc_input_manager *im,
const SDL_KeyboardEvent *event) {
// controller is NULL if --no-control is requested
struct sc_controller *controller = im->controller;
bool control = im->controller;
SDL_Keycode keycode = event->keysym.sym;
uint16_t mod = event->keysym.mod;
@ -390,68 +427,68 @@ sc_input_manager_process_key(struct sc_input_manager *im,
enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
switch (keycode) {
case SDLK_h:
if (controller && !shift && !repeat) {
action_home(controller, action);
if (im->kp && !shift && !repeat) {
action_home(im, action);
}
return;
case SDLK_b: // fall-through
case SDLK_BACKSPACE:
if (controller && !shift && !repeat) {
action_back(controller, action);
if (im->kp && !shift && !repeat) {
action_back(im, action);
}
return;
case SDLK_s:
if (controller && !shift && !repeat) {
action_app_switch(controller, action);
if (im->kp && !shift && !repeat) {
action_app_switch(im, action);
}
return;
case SDLK_m:
if (controller && !shift && !repeat) {
action_menu(controller, action);
if (im->kp && !shift && !repeat) {
action_menu(im, action);
}
return;
case SDLK_p:
if (controller && !shift && !repeat) {
action_power(controller, action);
if (im->kp && !shift && !repeat) {
action_power(im, action);
}
return;
case SDLK_o:
if (controller && !repeat && down) {
if (control && !repeat && down) {
enum sc_screen_power_mode mode = shift
? SC_SCREEN_POWER_MODE_NORMAL
: SC_SCREEN_POWER_MODE_OFF;
set_screen_power_mode(controller, mode);
set_screen_power_mode(im, mode);
}
return;
case SDLK_DOWN:
if (shift) {
if (!repeat & down) {
apply_orientation_transform(im->screen,
apply_orientation_transform(im,
SC_ORIENTATION_FLIP_180);
}
} else if (controller) {
} else if (im->kp) {
// forward repeated events
action_volume_down(controller, action);
action_volume_down(im, action);
}
return;
case SDLK_UP:
if (shift) {
if (!repeat & down) {
apply_orientation_transform(im->screen,
apply_orientation_transform(im,
SC_ORIENTATION_FLIP_180);
}
} else if (controller) {
} else if (im->kp) {
// forward repeated events
action_volume_up(controller, action);
action_volume_up(im, action);
}
return;
case SDLK_LEFT:
if (!repeat && down) {
if (shift) {
apply_orientation_transform(im->screen,
apply_orientation_transform(im,
SC_ORIENTATION_FLIP_0);
} else {
apply_orientation_transform(im->screen,
apply_orientation_transform(im,
SC_ORIENTATION_270);
}
}
@ -459,34 +496,33 @@ sc_input_manager_process_key(struct sc_input_manager *im,
case SDLK_RIGHT:
if (!repeat && down) {
if (shift) {
apply_orientation_transform(im->screen,
apply_orientation_transform(im,
SC_ORIENTATION_FLIP_0);
} else {
apply_orientation_transform(im->screen,
apply_orientation_transform(im,
SC_ORIENTATION_90);
}
}
return;
case SDLK_c:
if (controller && !shift && !repeat && down) {
get_device_clipboard(controller, SC_COPY_KEY_COPY);
if (im->kp && !shift && !repeat && down) {
get_device_clipboard(im, SC_COPY_KEY_COPY);
}
return;
case SDLK_x:
if (controller && !shift && !repeat && down) {
get_device_clipboard(controller, SC_COPY_KEY_CUT);
if (im->kp && !shift && !repeat && down) {
get_device_clipboard(im, SC_COPY_KEY_CUT);
}
return;
case SDLK_v:
if (controller && !repeat && down) {
if (im->kp && !repeat && down) {
if (shift || im->legacy_paste) {
// inject the text as input events
clipboard_paste(controller);
clipboard_paste(im);
} else {
// store the text in the device clipboard and paste,
// without requesting an acknowledgment
set_device_clipboard(controller, true,
SC_SEQUENCE_INVALID);
set_device_clipboard(im, true, SC_SEQUENCE_INVALID);
}
}
return;
@ -507,23 +543,30 @@ sc_input_manager_process_key(struct sc_input_manager *im,
return;
case SDLK_i:
if (!shift && !repeat && down) {
switch_fps_counter_state(&im->screen->fps_counter);
switch_fps_counter_state(im);
}
return;
case SDLK_n:
if (controller && !repeat && down) {
if (control && !repeat && down) {
if (shift) {
collapse_panels(controller);
collapse_panels(im);
} else if (im->key_repeat == 0) {
expand_notification_panel(controller);
expand_notification_panel(im);
} else {
expand_settings_panel(controller);
expand_settings_panel(im);
}
}
return;
case SDLK_r:
if (controller && !shift && !repeat && down) {
rotate_device(controller);
if (control && !shift && !repeat && down) {
rotate_device(im);
}
return;
case SDLK_k:
if (control && !shift && !repeat && down
&& im->kp && im->kp->hid) {
// Only if the current keyboard is hid
open_hard_keyboard_settings(im);
}
return;
}
@ -531,7 +574,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
return;
}
if (!controller) {
if (!im->kp) {
return;
}
@ -540,7 +583,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
if (im->clipboard_autosync && is_ctrl_v) {
if (im->legacy_paste) {
// inject the text as input events
clipboard_paste(controller);
clipboard_paste(im);
return;
}
@ -550,7 +593,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
// Synchronize the computer clipboard to the device clipboard before
// sending Ctrl+v, to allow seamless copy-paste.
bool ok = set_device_clipboard(controller, false, sequence);
bool ok = set_device_clipboard(im, false, sequence);
if (!ok) {
LOGW("Clipboard could not be synchronized, Ctrl+v not injected");
return;
@ -652,7 +695,7 @@ sc_input_manager_process_touch(struct sc_input_manager *im,
static void
sc_input_manager_process_mouse_button(struct sc_input_manager *im,
const SDL_MouseButtonEvent *event) {
struct sc_controller *controller = im->controller;
bool control = im->controller;
if (event->which == SDL_TOUCH_MOUSEID) {
// simulated from touch events, so it's a duplicate
@ -661,27 +704,27 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
bool down = event->type == SDL_MOUSEBUTTONDOWN;
if (!im->forward_all_clicks) {
if (controller) {
if (control) {
enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
if (event->button == SDL_BUTTON_X1) {
action_app_switch(controller, action);
if (im->kp && event->button == SDL_BUTTON_X1) {
action_app_switch(im, action);
return;
}
if (event->button == SDL_BUTTON_X2 && down) {
if (event->clicks < 2) {
expand_notification_panel(controller);
expand_notification_panel(im);
} else {
expand_settings_panel(controller);
expand_settings_panel(im);
}
return;
}
if (event->button == SDL_BUTTON_RIGHT) {
press_back_or_turn_screen_on(controller, action);
if (im->kp && event->button == SDL_BUTTON_RIGHT) {
press_back_or_turn_screen_on(im, action);
return;
}
if (event->button == SDL_BUTTON_MIDDLE) {
action_home(controller, action);
if (im->kp && event->button == SDL_BUTTON_MIDDLE) {
action_home(im, action);
return;
}
}
@ -704,7 +747,7 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
// otherwise, send the click event to the device
}
if (!controller) {
if (!im->mp) {
return;
}
@ -844,7 +887,7 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
bool control = im->controller;
switch (event->type) {
case SDL_TEXTINPUT:
if (!control) {
if (!im->kp) {
break;
}
sc_input_manager_process_text_input(im, &event->text);
@ -856,13 +899,13 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
sc_input_manager_process_key(im, &event->key);
break;
case SDL_MOUSEMOTION:
if (!control) {
if (!im->mp) {
break;
}
sc_input_manager_process_mouse_motion(im, &event->motion);
break;
case SDL_MOUSEWHEEL:
if (!control) {
if (!im->mp) {
break;
}
sc_input_manager_process_mouse_wheel(im, &event->wheel);
@ -876,7 +919,7 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
case SDL_FINGERMOTION:
case SDL_FINGERDOWN:
case SDL_FINGERUP:
if (!control) {
if (!im->mp) {
break;
}
sc_input_manager_process_touch(im, &event->tfinger);

View File

@ -1,4 +1,4 @@
#include "keyboard_inject.h"
#include "keyboard_sdk.h"
#include <assert.h>
@ -9,8 +9,8 @@
#include "util/intmap.h"
#include "util/log.h"
/** Downcast key processor to sc_keyboard_inject */
#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_inject, key_processor)
/** Downcast key processor to sc_keyboard_sdk */
#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_sdk, key_processor)
static enum android_keyevent_action
convert_keycode_action(enum sc_action action) {
@ -271,20 +271,20 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
// is set before injecting Ctrl+v.
(void) ack_to_wait;
struct sc_keyboard_inject *ki = DOWNCAST(kp);
struct sc_keyboard_sdk *kb = DOWNCAST(kp);
if (event->repeat) {
if (!ki->forward_key_repeat) {
if (!kb->forward_key_repeat) {
return;
}
++ki->repeat;
++kb->repeat;
} else {
ki->repeat = 0;
kb->repeat = 0;
}
struct sc_control_msg msg;
if (convert_input_key(event, &msg, ki->key_inject_mode, ki->repeat)) {
if (!sc_controller_push_msg(ki->controller, &msg)) {
if (convert_input_key(event, &msg, kb->key_inject_mode, kb->repeat)) {
if (!sc_controller_push_msg(kb->controller, &msg)) {
LOGW("Could not request 'inject keycode'");
}
}
@ -293,14 +293,14 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
static void
sc_key_processor_process_text(struct sc_key_processor *kp,
const struct sc_text_event *event) {
struct sc_keyboard_inject *ki = DOWNCAST(kp);
struct sc_keyboard_sdk *kb = DOWNCAST(kp);
if (ki->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
if (kb->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
// Never inject text events
return;
}
if (ki->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
if (kb->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
char c = event->text[0];
if (isalpha(c) || c == ' ') {
assert(event->text[1] == '\0');
@ -316,22 +316,22 @@ sc_key_processor_process_text(struct sc_key_processor *kp,
LOGW("Could not strdup input text");
return;
}
if (!sc_controller_push_msg(ki->controller, &msg)) {
if (!sc_controller_push_msg(kb->controller, &msg)) {
free(msg.inject_text.text);
LOGW("Could not request 'inject text'");
}
}
void
sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
struct sc_controller *controller,
enum sc_key_inject_mode key_inject_mode,
bool forward_key_repeat) {
ki->controller = controller;
ki->key_inject_mode = key_inject_mode;
ki->forward_key_repeat = forward_key_repeat;
kb->controller = controller;
kb->key_inject_mode = key_inject_mode;
kb->forward_key_repeat = forward_key_repeat;
ki->repeat = 0;
kb->repeat = 0;
static const struct sc_key_processor_ops ops = {
.process_key = sc_key_processor_process_key,
@ -339,6 +339,7 @@ sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
};
// Key injection and clipboard synchronization are serialized
ki->key_processor.async_paste = false;
ki->key_processor.ops = &ops;
kb->key_processor.async_paste = false;
kb->key_processor.hid = false;
kb->key_processor.ops = &ops;
}

View File

@ -1,5 +1,5 @@
#ifndef SC_KEYBOARD_INJECT_H
#define SC_KEYBOARD_INJECT_H
#ifndef SC_KEYBOARD_SDK_H
#define SC_KEYBOARD_SDK_H
#include "common.h"
@ -9,7 +9,7 @@
#include "options.h"
#include "trait/key_processor.h"
struct sc_keyboard_inject {
struct sc_keyboard_sdk {
struct sc_key_processor key_processor; // key processor trait
struct sc_controller *controller;
@ -23,7 +23,7 @@ struct sc_keyboard_inject {
};
void
sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
struct sc_controller *controller,
enum sc_key_inject_mode key_inject_mode,
bool forward_key_repeat);

View File

@ -1,4 +1,4 @@
#include "mouse_inject.h"
#include "mouse_sdk.h"
#include <assert.h>
@ -9,8 +9,8 @@
#include "util/intmap.h"
#include "util/log.h"
/** Downcast mouse processor to sc_mouse_inject */
#define DOWNCAST(MP) container_of(MP, struct sc_mouse_inject, mouse_processor)
/** Downcast mouse processor to sc_mouse_sdk */
#define DOWNCAST(MP) container_of(MP, struct sc_mouse_sdk, mouse_processor)
static enum android_motionevent_buttons
convert_mouse_buttons(uint32_t state) {
@ -63,7 +63,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
return;
}
struct sc_mouse_inject *mi = DOWNCAST(mp);
struct sc_mouse_sdk *m = DOWNCAST(mp);
struct sc_control_msg msg = {
.type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@ -76,7 +76,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
},
};
if (!sc_controller_push_msg(mi->controller, &msg)) {
if (!sc_controller_push_msg(m->controller, &msg)) {
LOGW("Could not request 'inject mouse motion event'");
}
}
@ -84,7 +84,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
static void
sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
const struct sc_mouse_click_event *event) {
struct sc_mouse_inject *mi = DOWNCAST(mp);
struct sc_mouse_sdk *m = DOWNCAST(mp);
struct sc_control_msg msg = {
.type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@ -98,7 +98,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
},
};
if (!sc_controller_push_msg(mi->controller, &msg)) {
if (!sc_controller_push_msg(m->controller, &msg)) {
LOGW("Could not request 'inject mouse click event'");
}
}
@ -106,7 +106,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
static void
sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
const struct sc_mouse_scroll_event *event) {
struct sc_mouse_inject *mi = DOWNCAST(mp);
struct sc_mouse_sdk *m = DOWNCAST(mp);
struct sc_control_msg msg = {
.type = SC_CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT,
@ -118,7 +118,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
},
};
if (!sc_controller_push_msg(mi->controller, &msg)) {
if (!sc_controller_push_msg(m->controller, &msg)) {
LOGW("Could not request 'inject mouse scroll event'");
}
}
@ -126,7 +126,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
static void
sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
const struct sc_touch_event *event) {
struct sc_mouse_inject *mi = DOWNCAST(mp);
struct sc_mouse_sdk *m = DOWNCAST(mp);
struct sc_control_msg msg = {
.type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@ -139,15 +139,14 @@ sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
},
};
if (!sc_controller_push_msg(mi->controller, &msg)) {
if (!sc_controller_push_msg(m->controller, &msg)) {
LOGW("Could not request 'inject touch event'");
}
}
void
sc_mouse_inject_init(struct sc_mouse_inject *mi,
struct sc_controller *controller) {
mi->controller = controller;
sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller) {
m->controller = controller;
static const struct sc_mouse_processor_ops ops = {
.process_mouse_motion = sc_mouse_processor_process_mouse_motion,
@ -156,7 +155,7 @@ sc_mouse_inject_init(struct sc_mouse_inject *mi,
.process_touch = sc_mouse_processor_process_touch,
};
mi->mouse_processor.ops = &ops;
m->mouse_processor.ops = &ops;
mi->mouse_processor.relative_mode = false;
m->mouse_processor.relative_mode = false;
}

View File

@ -1,5 +1,5 @@
#ifndef SC_MOUSE_INJECT_H
#define SC_MOUSE_INJECT_H
#ifndef SC_MOUSE_SDK_H
#define SC_MOUSE_SDK_H
#include "common.h"
@ -9,14 +9,13 @@
#include "screen.h"
#include "trait/mouse_processor.h"
struct sc_mouse_inject {
struct sc_mouse_sdk {
struct sc_mouse_processor mouse_processor; // mouse processor trait
struct sc_controller *controller;
};
void
sc_mouse_inject_init(struct sc_mouse_inject *mi,
struct sc_controller *controller);
sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller);
#endif

View File

@ -21,8 +21,8 @@ const struct scrcpy_options scrcpy_options_default = {
.video_source = SC_VIDEO_SOURCE_DISPLAY,
.audio_source = SC_AUDIO_SOURCE_AUTO,
.record_format = SC_RECORD_FORMAT_AUTO,
.keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT,
.mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT,
.keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_AUTO,
.mouse_input_mode = SC_MOUSE_INPUT_MODE_AUTO,
.camera_facing = SC_CAMERA_FACING_ANY,
.port_range = {
.first = DEFAULT_LOCAL_PORT_RANGE_FIRST,

View File

@ -140,13 +140,19 @@ enum sc_lock_video_orientation {
};
enum sc_keyboard_input_mode {
SC_KEYBOARD_INPUT_MODE_INJECT,
SC_KEYBOARD_INPUT_MODE_HID,
SC_KEYBOARD_INPUT_MODE_AUTO,
SC_KEYBOARD_INPUT_MODE_DISABLED,
SC_KEYBOARD_INPUT_MODE_SDK,
SC_KEYBOARD_INPUT_MODE_AOA,
SC_KEYBOARD_INPUT_MODE_UHID,
};
enum sc_mouse_input_mode {
SC_MOUSE_INPUT_MODE_INJECT,
SC_MOUSE_INPUT_MODE_HID,
SC_MOUSE_INPUT_MODE_AUTO,
SC_MOUSE_INPUT_MODE_DISABLED,
SC_MOUSE_INPUT_MODE_SDK,
SC_MOUSE_INPUT_MODE_AOA,
SC_MOUSE_INPUT_MODE_UHID,
};
enum sc_key_inject_mode {

View File

@ -1,22 +1,24 @@
#include "receiver.h"
#include <assert.h>
#include <inttypes.h>
#include <stdint.h>
#include <SDL2/SDL_clipboard.h>
#include "device_msg.h"
#include "util/log.h"
#include "util/str.h"
bool
sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
struct sc_acksync *acksync) {
sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket) {
bool ok = sc_mutex_init(&receiver->mutex);
if (!ok) {
return false;
}
receiver->control_socket = control_socket;
receiver->acksync = acksync;
receiver->acksync = NULL;
receiver->uhid_devices = NULL;
return true;
}
@ -48,6 +50,31 @@ process_msg(struct sc_receiver *receiver, struct sc_device_msg *msg) {
msg->ack_clipboard.sequence);
sc_acksync_ack(receiver->acksync, msg->ack_clipboard.sequence);
break;
case DEVICE_MSG_TYPE_UHID_OUTPUT:
if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
char *hex = sc_str_to_hex_string(msg->uhid_output.data,
msg->uhid_output.size);
if (hex) {
LOGV("UHID output [%" PRIu16 "] %s",
msg->uhid_output.id, hex);
free(hex);
} else {
LOGV("UHID output [%" PRIu16 "] size=%" PRIu16,
msg->uhid_output.id, msg->uhid_output.size);
}
}
assert(receiver->uhid_devices);
struct sc_uhid_receiver *uhid_receiver =
sc_uhid_devices_get_receiver(receiver->uhid_devices,
msg->uhid_output.id);
if (uhid_receiver) {
uhid_receiver->ops->process_output(uhid_receiver,
msg->uhid_output.data,
msg->uhid_output.size);
} else {
LOGW("No UHID receiver for id %" PRIu16, msg->uhid_output.id);
}
break;
}
}

View File

@ -5,6 +5,7 @@
#include <stdbool.h>
#include "uhid/uhid_output.h"
#include "util/acksync.h"
#include "util/net.h"
#include "util/thread.h"
@ -17,11 +18,11 @@ struct sc_receiver {
sc_mutex mutex;
struct sc_acksync *acksync;
struct sc_uhid_devices *uhid_devices;
};
bool
sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
struct sc_acksync *acksync);
sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket);
void
sc_receiver_destroy(struct sc_receiver *receiver);

View File

@ -20,15 +20,17 @@
#include "demuxer.h"
#include "events.h"
#include "file_pusher.h"
#include "keyboard_inject.h"
#include "mouse_inject.h"
#include "keyboard_sdk.h"
#include "mouse_sdk.h"
#include "recorder.h"
#include "screen.h"
#include "server.h"
#include "uhid/keyboard_uhid.h"
#include "uhid/mouse_uhid.h"
#ifdef HAVE_USB
# include "usb/aoa_hid.h"
# include "usb/hid_keyboard.h"
# include "usb/hid_mouse.h"
# include "usb/keyboard_aoa.h"
# include "usb/mouse_aoa.h"
# include "usb/usb.h"
#endif
#include "util/acksync.h"
@ -61,18 +63,21 @@ struct scrcpy {
struct sc_aoa aoa;
// sequence/ack helper to synchronize clipboard and Ctrl+v via HID
struct sc_acksync acksync;
struct sc_uhid_devices uhid_devices;
#endif
union {
struct sc_keyboard_inject keyboard_inject;
struct sc_keyboard_sdk keyboard_sdk;
#ifdef HAVE_USB
struct sc_hid_keyboard keyboard_hid;
struct sc_keyboard_aoa keyboard_aoa;
#endif
struct sc_keyboard_uhid keyboard_uhid;
};
union {
struct sc_mouse_inject mouse_inject;
struct sc_mouse_sdk mouse_sdk;
#ifdef HAVE_USB
struct sc_hid_mouse mouse_hid;
struct sc_mouse_aoa mouse_aoa;
#endif
struct sc_mouse_uhid mouse_uhid;
};
struct sc_timeout timeout;
};
@ -330,8 +335,8 @@ scrcpy(struct scrcpy_options *options) {
bool audio_demuxer_started = false;
#ifdef HAVE_USB
bool aoa_hid_initialized = false;
bool hid_keyboard_initialized = false;
bool hid_mouse_initialized = false;
bool keyboard_aoa_initialized = false;
bool mouse_aoa_initialized = false;
#endif
bool controller_initialized = false;
bool controller_started = false;
@ -340,6 +345,7 @@ scrcpy(struct scrcpy_options *options) {
bool timeout_started = false;
struct sc_acksync *acksync = NULL;
struct sc_uhid_devices *uhid_devices = NULL;
uint32_t scid = scrcpy_generate_scid();
@ -542,12 +548,19 @@ scrcpy(struct scrcpy_options *options) {
struct sc_mouse_processor *mp = NULL;
if (options->control) {
if (!sc_controller_init(&s->controller, s->server.control_socket)) {
goto end;
}
controller_initialized = true;
controller = &s->controller;
#ifdef HAVE_USB
bool use_hid_keyboard =
options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
bool use_hid_mouse =
options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
if (use_hid_keyboard || use_hid_mouse) {
bool use_keyboard_aoa =
options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
bool use_mouse_aoa =
options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
if (use_keyboard_aoa || use_mouse_aoa) {
bool ok = sc_acksync_init(&s->acksync);
if (!ok) {
goto end;
@ -590,25 +603,25 @@ scrcpy(struct scrcpy_options *options) {
goto aoa_hid_end;
}
if (use_hid_keyboard) {
if (sc_hid_keyboard_init(&s->keyboard_hid, &s->aoa)) {
hid_keyboard_initialized = true;
kp = &s->keyboard_hid.key_processor;
if (use_keyboard_aoa) {
if (sc_keyboard_aoa_init(&s->keyboard_aoa, &s->aoa)) {
keyboard_aoa_initialized = true;
kp = &s->keyboard_aoa.key_processor;
} else {
LOGE("Could not initialize HID keyboard");
}
}
if (use_hid_mouse) {
if (sc_hid_mouse_init(&s->mouse_hid, &s->aoa)) {
hid_mouse_initialized = true;
mp = &s->mouse_hid.mouse_processor;
if (use_mouse_aoa) {
if (sc_mouse_aoa_init(&s->mouse_aoa, &s->aoa)) {
mouse_aoa_initialized = true;
mp = &s->mouse_aoa.mouse_processor;
} else {
LOGE("Could not initialized HID mouse");
}
}
bool need_aoa = hid_keyboard_initialized || hid_mouse_initialized;
bool need_aoa = keyboard_aoa_initialized || mouse_aoa_initialized;
if (!need_aoa || !sc_aoa_start(&s->aoa)) {
sc_acksync_destroy(&s->acksync);
@ -624,58 +637,66 @@ scrcpy(struct scrcpy_options *options) {
aoa_hid_end:
if (!aoa_hid_initialized) {
if (hid_keyboard_initialized) {
sc_hid_keyboard_destroy(&s->keyboard_hid);
hid_keyboard_initialized = false;
if (keyboard_aoa_initialized) {
sc_keyboard_aoa_destroy(&s->keyboard_aoa);
keyboard_aoa_initialized = false;
}
if (hid_mouse_initialized) {
sc_hid_mouse_destroy(&s->mouse_hid);
hid_mouse_initialized = false;
if (mouse_aoa_initialized) {
sc_mouse_aoa_destroy(&s->mouse_aoa);
mouse_aoa_initialized = false;
}
}
if (use_hid_keyboard && !hid_keyboard_initialized) {
LOGE("Fallback to default keyboard injection method "
"(-K/--hid-keyboard ignored)");
options->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT;
if (use_keyboard_aoa && !keyboard_aoa_initialized) {
LOGE("Fallback to --keyboard=sdk (--keyboard=aoa ignored)");
options->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_SDK;
}
if (use_hid_mouse && !hid_mouse_initialized) {
LOGE("Fallback to default mouse injection method "
"(-M/--hid-mouse ignored)");
options->mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT;
if (use_mouse_aoa && !mouse_aoa_initialized) {
LOGE("Fallback to --keyboard=sdk (--keyboard=aoa ignored)");
options->mouse_input_mode = SC_MOUSE_INPUT_MODE_SDK;
}
}
#else
assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_HID);
assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_HID);
assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_AOA);
assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_AOA);
#endif
// keyboard_input_mode may have been reset if HID mode failed
if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_INJECT) {
sc_keyboard_inject_init(&s->keyboard_inject, &s->controller,
// keyboard_input_mode may have been reset if AOA mode failed
if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_SDK) {
sc_keyboard_sdk_init(&s->keyboard_sdk, &s->controller,
options->key_inject_mode,
options->forward_key_repeat);
kp = &s->keyboard_inject.key_processor;
}
// mouse_input_mode may have been reset if HID mode failed
if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_INJECT) {
sc_mouse_inject_init(&s->mouse_inject, &s->controller);
mp = &s->mouse_inject.mouse_processor;
}
if (!sc_controller_init(&s->controller, s->server.control_socket,
acksync)) {
kp = &s->keyboard_sdk.key_processor;
} else if (options->keyboard_input_mode
== SC_KEYBOARD_INPUT_MODE_UHID) {
bool ok = sc_keyboard_uhid_init(&s->keyboard_uhid, &s->controller,
&s->uhid_devices);
if (!ok) {
goto end;
}
controller_initialized = true;
uhid_devices = &s->uhid_devices;
kp = &s->keyboard_uhid.key_processor;
}
// mouse_input_mode may have been reset if AOA mode failed
if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_SDK) {
sc_mouse_sdk_init(&s->mouse_sdk, &s->controller);
mp = &s->mouse_sdk.mouse_processor;
} else if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_UHID) {
bool ok = sc_mouse_uhid_init(&s->mouse_uhid, &s->controller);
if (!ok) {
goto end;
}
mp = &s->mouse_uhid.mouse_processor;
}
sc_controller_configure(&s->controller, acksync, uhid_devices);
if (!sc_controller_start(&s->controller)) {
goto end;
}
controller_started = true;
controller = &s->controller;
}
// There is a controller if and only if control is enabled
@ -815,11 +836,11 @@ end:
// end-of-stream
#ifdef HAVE_USB
if (aoa_hid_initialized) {
if (hid_keyboard_initialized) {
sc_hid_keyboard_destroy(&s->keyboard_hid);
if (keyboard_aoa_initialized) {
sc_keyboard_aoa_destroy(&s->keyboard_aoa);
}
if (hid_mouse_initialized) {
sc_hid_mouse_destroy(&s->mouse_hid);
if (mouse_aoa_initialized) {
sc_mouse_aoa_destroy(&s->mouse_aoa);
}
sc_aoa_stop(&s->aoa);
sc_usb_stop(&s->usb);

View File

@ -23,6 +23,12 @@ struct sc_key_processor {
*/
bool async_paste;
/** Set by the implementation to indicate that the keyboard is HID. In
* practice, it is used to react on a shortcut to open the hard keyboard
* settings only if the keyboard is HID.
*/
bool hid;
const struct sc_key_processor_ops *ops;
};

View File

@ -0,0 +1,155 @@
#include "keyboard_uhid.h"
#include "util/log.h"
/** Downcast key processor to keyboard_uhid */
#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_uhid, key_processor)
/** Downcast uhid_receiver to keyboard_uhid */
#define DOWNCAST_RECEIVER(UR) \
container_of(UR, struct sc_keyboard_uhid, uhid_receiver)
#define UHID_KEYBOARD_ID 1
static void
sc_keyboard_uhid_send_input(struct sc_keyboard_uhid *kb,
const struct sc_hid_event *event) {
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
msg.uhid_input.id = UHID_KEYBOARD_ID;
assert(event->size <= SC_HID_MAX_SIZE);
memcpy(msg.uhid_input.data, event->data, event->size);
msg.uhid_input.size = event->size;
if (!sc_controller_push_msg(kb->controller, &msg)) {
LOGE("Could not send UHID_INPUT message (key)");
}
}
static void
sc_keyboard_uhid_synchronize_mod(struct sc_keyboard_uhid *kb) {
SDL_Keymod sdl_mod = SDL_GetModState();
uint16_t mod = sc_mods_state_from_sdl(sdl_mod) & (SC_MOD_CAPS | SC_MOD_NUM);
uint16_t device_mod =
atomic_load_explicit(&kb->device_mod, memory_order_relaxed);
uint16_t diff = mod ^ device_mod;
if (diff) {
// Inherently racy (the HID output reports arrive asynchronously in
// response to key presses), but will re-synchronize on next key press
// or HID output anyway
atomic_store_explicit(&kb->device_mod, mod, memory_order_relaxed);
struct sc_hid_event hid_event;
sc_hid_keyboard_event_from_mods(&hid_event, diff);
LOGV("HID keyboard state synchronized");
sc_keyboard_uhid_send_input(kb, &hid_event);
}
}
static void
sc_key_processor_process_key(struct sc_key_processor *kp,
const struct sc_key_event *event,
uint64_t ack_to_wait) {
(void) ack_to_wait;
if (event->repeat) {
// In USB HID protocol, key repeat is handled by the host (Android), so
// just ignore key repeat here.
return;
}
struct sc_keyboard_uhid *kb = DOWNCAST(kp);
struct sc_hid_event hid_event;
// Not all keys are supported, just ignore unsupported keys
if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
if (event->scancode == SC_SCANCODE_CAPSLOCK) {
atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_CAPS,
memory_order_relaxed);
} else if (event->scancode == SC_SCANCODE_NUMLOCK) {
atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_NUM,
memory_order_relaxed);
} else {
// Synchronize modifiers (only if the scancode itself does not
// change the modifiers)
sc_keyboard_uhid_synchronize_mod(kb);
}
sc_keyboard_uhid_send_input(kb, &hid_event);
}
}
static unsigned
sc_keyboard_uhid_to_sc_mod(uint8_t hid_led) {
// <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
// (chapter 11: LED page)
unsigned mod = 0;
if (hid_led & 0x01) {
mod |= SC_MOD_NUM;
}
if (hid_led & 0x02) {
mod |= SC_MOD_CAPS;
}
return mod;
}
static void
sc_uhid_receiver_process_output(struct sc_uhid_receiver *receiver,
const uint8_t *data, size_t len) {
// Called from the thread receiving device messages
assert(len);
(void) len;
struct sc_keyboard_uhid *kb = DOWNCAST_RECEIVER(receiver);
uint8_t hid_led = data[0];
uint16_t device_mod = sc_keyboard_uhid_to_sc_mod(hid_led);
atomic_store_explicit(&kb->device_mod, device_mod, memory_order_relaxed);
}
bool
sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
struct sc_controller *controller,
struct sc_uhid_devices *uhid_devices) {
sc_hid_keyboard_init(&kb->hid);
kb->controller = controller;
atomic_init(&kb->device_mod, 0);
static const struct sc_key_processor_ops ops = {
.process_key = sc_key_processor_process_key,
// Never forward text input via HID (all the keys are injected
// separately)
.process_text = NULL,
};
// Clipboard synchronization is requested over the same control socket, so
// there is no need for a specific synchronization mechanism
kb->key_processor.async_paste = false;
kb->key_processor.hid = true;
kb->key_processor.ops = &ops;
static const struct sc_uhid_receiver_ops uhid_receiver_ops = {
.process_output = sc_uhid_receiver_process_output,
};
kb->uhid_receiver.id = UHID_KEYBOARD_ID;
kb->uhid_receiver.ops = &uhid_receiver_ops;
sc_uhid_devices_add_receiver(uhid_devices, &kb->uhid_receiver);
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
msg.uhid_create.id = UHID_KEYBOARD_ID;
msg.uhid_create.report_desc = SC_HID_KEYBOARD_REPORT_DESC;
msg.uhid_create.report_desc_size = SC_HID_KEYBOARD_REPORT_DESC_LEN;
if (!sc_controller_push_msg(controller, &msg)) {
LOGE("Could not send UHID_CREATE message (keyboard)");
return false;
}
return true;
}

View File

@ -0,0 +1,27 @@
#ifndef SC_KEYBOARD_UHID_H
#define SC_KEYBOARD_UHID_H
#include "common.h"
#include <stdbool.h>
#include "controller.h"
#include "hid/hid_keyboard.h"
#include "uhid/uhid_output.h"
#include "trait/key_processor.h"
struct sc_keyboard_uhid {
struct sc_key_processor key_processor; // key processor trait
struct sc_uhid_receiver uhid_receiver;
struct sc_hid_keyboard hid;
struct sc_controller *controller;
atomic_uint_least16_t device_mod;
};
bool
sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
struct sc_controller *controller,
struct sc_uhid_devices *uhid_devices);
#endif

89
app/src/uhid/mouse_uhid.c Normal file
View File

@ -0,0 +1,89 @@
#include "mouse_uhid.h"
#include "hid/hid_mouse.h"
#include "input_events.h"
#include "util/log.h"
/** Downcast mouse processor to mouse_uhid */
#define DOWNCAST(MP) container_of(MP, struct sc_mouse_uhid, mouse_processor)
#define UHID_MOUSE_ID 2
static void
sc_mouse_uhid_send_input(struct sc_mouse_uhid *kb,
const struct sc_hid_event *event, const char *name) {
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
msg.uhid_input.id = UHID_MOUSE_ID;
assert(event->size <= SC_HID_MAX_SIZE);
memcpy(msg.uhid_input.data, event->data, event->size);
msg.uhid_input.size = event->size;
if (!sc_controller_push_msg(kb->controller, &msg)) {
LOGE("Could not send UHID_INPUT message (%s)", name);
}
}
static void
sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
const struct sc_mouse_motion_event *event) {
struct sc_mouse_uhid *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
sc_hid_mouse_event_from_motion(&hid_event, event);
sc_mouse_uhid_send_input(mouse, &hid_event, "mouse motion");
}
static void
sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
const struct sc_mouse_click_event *event) {
struct sc_mouse_uhid *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
sc_hid_mouse_event_from_click(&hid_event, event);
sc_mouse_uhid_send_input(mouse, &hid_event, "mouse click");
}
static void
sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
const struct sc_mouse_scroll_event *event) {
struct sc_mouse_uhid *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
sc_hid_mouse_event_from_scroll(&hid_event, event);
sc_mouse_uhid_send_input(mouse, &hid_event, "mouse scroll");
}
bool
sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
struct sc_controller *controller) {
mouse->controller = controller;
static const struct sc_mouse_processor_ops ops = {
.process_mouse_motion = sc_mouse_processor_process_mouse_motion,
.process_mouse_click = sc_mouse_processor_process_mouse_click,
.process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
// Touch events not supported (coordinates are not relative)
.process_touch = NULL,
};
mouse->mouse_processor.ops = &ops;
mouse->mouse_processor.relative_mode = true;
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
msg.uhid_create.id = UHID_MOUSE_ID;
msg.uhid_create.report_desc = SC_HID_MOUSE_REPORT_DESC;
msg.uhid_create.report_desc_size = SC_HID_MOUSE_REPORT_DESC_LEN;
if (!sc_controller_push_msg(controller, &msg)) {
LOGE("Could not send UHID_CREATE message (mouse)");
return false;
}
return true;
}

19
app/src/uhid/mouse_uhid.h Normal file
View File

@ -0,0 +1,19 @@
#ifndef SC_MOUSE_UHID_H
#define SC_MOUSE_UHID_H
#include <stdbool.h>
#include "controller.h"
#include "trait/mouse_processor.h"
struct sc_mouse_uhid {
struct sc_mouse_processor mouse_processor; // mouse processor trait
struct sc_controller *controller;
};
bool
sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
struct sc_controller *controller);
#endif

View File

@ -0,0 +1,25 @@
#include "uhid_output.h"
#include <assert.h>
void
sc_uhid_devices_init(struct sc_uhid_devices *devices) {
devices->count = 0;
}
void
sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
struct sc_uhid_receiver *receiver) {
assert(devices->count < SC_UHID_MAX_RECEIVERS);
devices->receivers[devices->count++] = receiver;
}
struct sc_uhid_receiver *
sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id) {
for (size_t i = 0; i < devices->count; ++i) {
if (devices->receivers[i]->id == id) {
return devices->receivers[i];
}
}
return NULL;
}

View File

@ -0,0 +1,45 @@
#ifndef SC_UHID_OUTPUT_H
#define SC_UHID_OUTPUT_H
#include "common.h"
#include <stdbool.h>
#include <stdint.h>
/**
* The communication with UHID devices is bidirectional.
*
* This component manages the registration of receivers to handle UHID output
* messages (sent from the device to the computer).
*/
struct sc_uhid_receiver {
uint16_t id;
const struct sc_uhid_receiver_ops *ops;
};
struct sc_uhid_receiver_ops {
void
(*process_output)(struct sc_uhid_receiver *receiver,
const uint8_t *data, size_t len);
};
#define SC_UHID_MAX_RECEIVERS 1
struct sc_uhid_devices {
struct sc_uhid_receiver *receivers[SC_UHID_MAX_RECEIVERS];
unsigned count;
};
void
sc_uhid_devices_init(struct sc_uhid_devices *devices);
void
sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
struct sc_uhid_receiver *receiver);
struct sc_uhid_receiver *
sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id);
#endif

View File

@ -5,6 +5,7 @@
#include "aoa_hid.h"
#include "util/log.h"
#include "util/str.h"
// See <https://source.android.com/devices/accessories/aoa2#hid-support>.
#define ACCESSORY_REGISTER_HID 54
@ -14,37 +15,18 @@
#define DEFAULT_TIMEOUT 1000
#define SC_HID_EVENT_QUEUE_MAX 64
#define SC_AOA_EVENT_QUEUE_MAX 64
static void
sc_hid_event_log(const struct sc_hid_event *event) {
sc_hid_event_log(uint16_t accessory_id, const struct sc_hid_event *event) {
// HID Event: [00] FF FF FF FF...
assert(event->size);
unsigned buffer_size = event->size * 3 + 1;
char *buffer = malloc(buffer_size);
if (!buffer) {
LOG_OOM();
char *hex = sc_str_to_hex_string(event->data, event->size);
if (!hex) {
return;
}
for (unsigned i = 0; i < event->size; ++i) {
snprintf(buffer + i * 3, 4, " %02x", event->buffer[i]);
}
LOGV("HID Event: [%d]%s", event->accessory_id, buffer);
free(buffer);
}
void
sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
unsigned char *buffer, uint16_t buffer_size) {
hid_event->accessory_id = accessory_id;
hid_event->buffer = buffer;
hid_event->size = buffer_size;
hid_event->ack_to_wait = SC_SEQUENCE_INVALID;
}
void
sc_hid_event_destroy(struct sc_hid_event *hid_event) {
free(hid_event->buffer);
LOGV("HID Event: [%d] %s", accessory_id, hex);
free(hex);
}
bool
@ -52,7 +34,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
struct sc_acksync *acksync) {
sc_vecdeque_init(&aoa->queue);
if (!sc_vecdeque_reserve(&aoa->queue, SC_HID_EVENT_QUEUE_MAX)) {
if (!sc_vecdeque_reserve(&aoa->queue, SC_AOA_EVENT_QUEUE_MAX)) {
return false;
}
@ -76,12 +58,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
void
sc_aoa_destroy(struct sc_aoa *aoa) {
// Destroy remaining events
while (!sc_vecdeque_is_empty(&aoa->queue)) {
struct sc_hid_event *event = sc_vecdeque_popref(&aoa->queue);
assert(event);
sc_hid_event_destroy(event);
}
sc_vecdeque_destroy(&aoa->queue);
sc_cond_destroy(&aoa->event_cond);
sc_mutex_destroy(&aoa->mutex);
@ -97,10 +74,10 @@ sc_aoa_register_hid(struct sc_aoa *aoa, uint16_t accessory_id,
// index (arg1): total length of the HID report descriptor
uint16_t value = accessory_id;
uint16_t index = report_desc_size;
unsigned char *buffer = NULL;
unsigned char *data = NULL;
uint16_t length = 0;
int result = libusb_control_transfer(aoa->usb->handle, request_type,
request, value, index, buffer, length,
request, value, index, data, length,
DEFAULT_TIMEOUT);
if (result < 0) {
LOGE("REGISTER_HID: libusb error: %s", libusb_strerror(result));
@ -130,14 +107,14 @@ sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
* See <https://libusb.sourceforge.io/api-1.0/libusb_packetoverflow.html>
*/
// value (arg0): accessory assigned ID for the HID device
// index (arg1): offset of data (buffer) in descriptor
// index (arg1): offset of data in descriptor
uint16_t value = accessory_id;
uint16_t index = 0;
// libusb_control_transfer expects a pointer to non-const
unsigned char *buffer = (unsigned char *) report_desc;
unsigned char *data = (unsigned char *) report_desc;
uint16_t length = report_desc_size;
int result = libusb_control_transfer(aoa->usb->handle, request_type,
request, value, index, buffer, length,
request, value, index, data, length,
DEFAULT_TIMEOUT);
if (result < 0) {
LOGE("SET_HID_REPORT_DESC: libusb error: %s", libusb_strerror(result));
@ -169,18 +146,19 @@ sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
}
static bool
sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
sc_aoa_send_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
const struct sc_hid_event *event) {
uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
uint8_t request = ACCESSORY_SEND_HID_EVENT;
// <https://source.android.com/devices/accessories/aoa2.html#hid-support>
// value (arg0): accessory assigned ID for the HID device
// index (arg1): 0 (unused)
uint16_t value = event->accessory_id;
uint16_t value = accessory_id;
uint16_t index = 0;
unsigned char *buffer = event->buffer;
unsigned char *data = (uint8_t *) event->data; // discard const
uint16_t length = event->size;
int result = libusb_control_transfer(aoa->usb->handle, request_type,
request, value, index, buffer, length,
request, value, index, data, length,
DEFAULT_TIMEOUT);
if (result < 0) {
LOGE("SEND_HID_EVENT: libusb error: %s", libusb_strerror(result));
@ -192,7 +170,7 @@ sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
}
bool
sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id) {
uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
uint8_t request = ACCESSORY_UNREGISTER_HID;
// <https://source.android.com/devices/accessories/aoa2.html#hid-support>
@ -200,10 +178,10 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
// index (arg1): 0
uint16_t value = accessory_id;
uint16_t index = 0;
unsigned char *buffer = NULL;
unsigned char *data = NULL;
uint16_t length = 0;
int result = libusb_control_transfer(aoa->usb->handle, request_type,
request, value, index, buffer, length,
request, value, index, data, length,
DEFAULT_TIMEOUT);
if (result < 0) {
LOGE("UNREGISTER_HID: libusb error: %s", libusb_strerror(result));
@ -215,16 +193,25 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
}
bool
sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
uint16_t accessory_id,
const struct sc_hid_event *event,
uint64_t ack_to_wait) {
if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
sc_hid_event_log(event);
sc_hid_event_log(accessory_id, event);
}
sc_mutex_lock(&aoa->mutex);
bool full = sc_vecdeque_is_full(&aoa->queue);
if (!full) {
bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
sc_vecdeque_push_noresize(&aoa->queue, *event);
struct sc_aoa_event *aoa_event =
sc_vecdeque_push_hole_noresize(&aoa->queue);
aoa_event->hid = *event;
aoa_event->accessory_id = accessory_id;
aoa_event->ack_to_wait = ack_to_wait;
if (was_empty) {
sc_cond_signal(&aoa->event_cond);
}
@ -252,7 +239,7 @@ run_aoa_thread(void *data) {
}
assert(!sc_vecdeque_is_empty(&aoa->queue));
struct sc_hid_event event = sc_vecdeque_pop(&aoa->queue);
struct sc_aoa_event event = sc_vecdeque_pop(&aoa->queue);
uint64_t ack_to_wait = event.ack_to_wait;
sc_mutex_unlock(&aoa->mutex);
@ -271,17 +258,14 @@ run_aoa_thread(void *data) {
if (result == SC_ACKSYNC_WAIT_TIMEOUT) {
LOGW("Ack not received after 500ms, discarding HID event");
sc_hid_event_destroy(&event);
continue;
} else if (result == SC_ACKSYNC_WAIT_INTR) {
// stopped
sc_hid_event_destroy(&event);
break;
}
}
bool ok = sc_aoa_send_hid_event(aoa, &event);
sc_hid_event_destroy(&event);
bool ok = sc_aoa_send_hid_event(aoa, event.accessory_id, &event.hid);
if (!ok) {
LOGW("Could not send HID event to USB device");
}

View File

@ -6,28 +6,22 @@
#include <libusb-1.0/libusb.h>
#include "hid/hid_event.h"
#include "usb.h"
#include "util/acksync.h"
#include "util/thread.h"
#include "util/tick.h"
#include "util/vecdeque.h"
struct sc_hid_event {
#define SC_HID_MAX_SIZE 8
struct sc_aoa_event {
struct sc_hid_event hid;
uint16_t accessory_id;
unsigned char *buffer;
uint16_t size;
uint64_t ack_to_wait;
};
// Takes ownership of buffer
void
sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
unsigned char *buffer, uint16_t buffer_size);
void
sc_hid_event_destroy(struct sc_hid_event *hid_event);
struct sc_hid_event_queue SC_VECDEQUE(struct sc_hid_event);
struct sc_aoa_event_queue SC_VECDEQUE(struct sc_aoa_event);
struct sc_aoa {
struct sc_usb *usb;
@ -35,7 +29,7 @@ struct sc_aoa {
sc_mutex mutex;
sc_cond event_cond;
bool stopped;
struct sc_hid_event_queue queue;
struct sc_aoa_event_queue queue;
struct sc_acksync *acksync;
};
@ -63,6 +57,16 @@ bool
sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id);
bool
sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event);
sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
uint16_t accessory_id,
const struct sc_hid_event *event,
uint64_t ack_to_wait);
static inline bool
sc_aoa_push_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
const struct sc_hid_event *event) {
return sc_aoa_push_hid_event_with_ack_to_wait(aoa, accessory_id, event,
SC_SEQUENCE_INVALID);
}
#endif

View File

@ -1,267 +0,0 @@
#include "hid_mouse.h"
#include <assert.h>
#include "input_events.h"
#include "util/log.h"
/** Downcast mouse processor to hid_mouse */
#define DOWNCAST(MP) container_of(MP, struct sc_hid_mouse, mouse_processor)
#define HID_MOUSE_ACCESSORY_ID 2
// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position
#define HID_MOUSE_EVENT_SIZE 4
/**
* Mouse descriptor from the specification:
* <https://www.usb.org/sites/default/files/hid1_11.pdf>
*
* Appendix E (p71): §E.10 Report Descriptor (Mouse)
*
* The usage tags (like Wheel) are listed in "HID Usage Tables":
* <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
* §4 Generic Desktop Page (0x01) (p26)
*/
static const unsigned char mouse_report_desc[] = {
// Usage Page (Generic Desktop)
0x05, 0x01,
// Usage (Mouse)
0x09, 0x02,
// Collection (Application)
0xA1, 0x01,
// Usage (Pointer)
0x09, 0x01,
// Collection (Physical)
0xA1, 0x00,
// Usage Page (Buttons)
0x05, 0x09,
// Usage Minimum (1)
0x19, 0x01,
// Usage Maximum (5)
0x29, 0x05,
// Logical Minimum (0)
0x15, 0x00,
// Logical Maximum (1)
0x25, 0x01,
// Report Count (5)
0x95, 0x05,
// Report Size (1)
0x75, 0x01,
// Input (Data, Variable, Absolute): 5 buttons bits
0x81, 0x02,
// Report Count (1)
0x95, 0x01,
// Report Size (3)
0x75, 0x03,
// Input (Constant): 3 bits padding
0x81, 0x01,
// Usage Page (Generic Desktop)
0x05, 0x01,
// Usage (X)
0x09, 0x30,
// Usage (Y)
0x09, 0x31,
// Usage (Wheel)
0x09, 0x38,
// Local Minimum (-127)
0x15, 0x81,
// Local Maximum (127)
0x25, 0x7F,
// Report Size (8)
0x75, 0x08,
// Report Count (3)
0x95, 0x03,
// Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
0x81, 0x06,
// End Collection
0xC0,
// End Collection
0xC0,
};
/**
* A mouse HID event is 3 bytes long:
*
* - byte 0: buttons state
* - byte 1: relative x motion (signed byte from -127 to 127)
* - byte 2: relative y motion (signed byte from -127 to 127)
*
* 7 6 5 4 3 2 1 0
* +---------------+
* byte 0: |0 0 0 . . . . .| buttons state
* +---------------+
* ^ ^ ^ ^ ^
* | | | | `- left button
* | | | `--- right button
* | | `----- middle button
* | `------- button 4
* `--------- button 5
*
* +---------------+
* byte 1: |. . . . . . . .| relative x motion
* +---------------+
* byte 2: |. . . . . . . .| relative y motion
* +---------------+
* byte 3: |. . . . . . . .| wheel motion (-1, 0 or 1)
* +---------------+
*
* As an example, here is the report for a motion of (x=5, y=-4) with left
* button pressed:
*
* +---------------+
* |0 0 0 0 0 0 0 1| left button pressed
* +---------------+
* |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
* +---------------+
* |1 1 1 1 1 1 0 0| relative y motion (y = -4)
* +---------------+
* |0 0 0 0 0 0 0 0| wheel motion
* +---------------+
*/
static bool
sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
unsigned char *buffer = calloc(1, HID_MOUSE_EVENT_SIZE);
if (!buffer) {
LOG_OOM();
return false;
}
sc_hid_event_init(hid_event, HID_MOUSE_ACCESSORY_ID, buffer,
HID_MOUSE_EVENT_SIZE);
return true;
}
static unsigned char
buttons_state_to_hid_buttons(uint8_t buttons_state) {
unsigned char c = 0;
if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
c |= 1 << 0;
}
if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
c |= 1 << 1;
}
if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
c |= 1 << 2;
}
if (buttons_state & SC_MOUSE_BUTTON_X1) {
c |= 1 << 3;
}
if (buttons_state & SC_MOUSE_BUTTON_X2) {
c |= 1 << 4;
}
return c;
}
static void
sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
const struct sc_mouse_motion_event *event) {
struct sc_hid_mouse *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
if (!sc_hid_mouse_event_init(&hid_event)) {
return;
}
unsigned char *buffer = hid_event.buffer;
buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
buffer[1] = CLAMP(event->xrel, -127, 127);
buffer[2] = CLAMP(event->yrel, -127, 127);
buffer[3] = 0; // wheel coordinates only used for scrolling
if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
sc_hid_event_destroy(&hid_event);
LOGW("Could not request HID event (mouse motion)");
}
}
static void
sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
const struct sc_mouse_click_event *event) {
struct sc_hid_mouse *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
if (!sc_hid_mouse_event_init(&hid_event)) {
return;
}
unsigned char *buffer = hid_event.buffer;
buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
buffer[1] = 0; // no x motion
buffer[2] = 0; // no y motion
buffer[3] = 0; // wheel coordinates only used for scrolling
if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
sc_hid_event_destroy(&hid_event);
LOGW("Could not request HID event (mouse click)");
}
}
static void
sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
const struct sc_mouse_scroll_event *event) {
struct sc_hid_mouse *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
if (!sc_hid_mouse_event_init(&hid_event)) {
return;
}
unsigned char *buffer = hid_event.buffer;
buffer[0] = 0; // buttons state irrelevant (and unknown)
buffer[1] = 0; // no x motion
buffer[2] = 0; // no y motion
// In practice, vscroll is always -1, 0 or 1, but in theory other values
// are possible
buffer[3] = CLAMP(event->vscroll, -127, 127);
// Horizontal scrolling ignored
if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
sc_hid_event_destroy(&hid_event);
LOGW("Could not request HID event (mouse scroll)");
}
}
bool
sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa) {
mouse->aoa = aoa;
bool ok = sc_aoa_setup_hid(aoa, HID_MOUSE_ACCESSORY_ID, mouse_report_desc,
ARRAY_LEN(mouse_report_desc));
if (!ok) {
LOGW("Register HID mouse failed");
return false;
}
static const struct sc_mouse_processor_ops ops = {
.process_mouse_motion = sc_mouse_processor_process_mouse_motion,
.process_mouse_click = sc_mouse_processor_process_mouse_click,
.process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
// Touch events not supported (coordinates are not relative)
.process_touch = NULL,
};
mouse->mouse_processor.ops = &ops;
mouse->mouse_processor.relative_mode = true;
return true;
}
void
sc_hid_mouse_destroy(struct sc_hid_mouse *mouse) {
bool ok = sc_aoa_unregister_hid(mouse->aoa, HID_MOUSE_ACCESSORY_ID);
if (!ok) {
LOGW("Could not unregister HID mouse");
}
}

110
app/src/usb/keyboard_aoa.c Normal file
View File

@ -0,0 +1,110 @@
#include "keyboard_aoa.h"
#include <assert.h>
#include "input_events.h"
#include "util/log.h"
/** Downcast key processor to keyboard_aoa */
#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_aoa, key_processor)
#define HID_KEYBOARD_ACCESSORY_ID 1
static bool
push_mod_lock_state(struct sc_keyboard_aoa *kb, uint16_t mods_state) {
struct sc_hid_event hid_event;
if (!sc_hid_keyboard_event_from_mods(&hid_event, mods_state)) {
// Nothing to do
return true;
}
if (!sc_aoa_push_hid_event(kb->aoa, HID_KEYBOARD_ACCESSORY_ID,
&hid_event)) {
LOGW("Could not request HID event (mod lock state)");
return false;
}
LOGD("HID keyboard state synchronized");
return true;
}
static void
sc_key_processor_process_key(struct sc_key_processor *kp,
const struct sc_key_event *event,
uint64_t ack_to_wait) {
if (event->repeat) {
// In USB HID protocol, key repeat is handled by the host (Android), so
// just ignore key repeat here.
return;
}
struct sc_keyboard_aoa *kb = DOWNCAST(kp);
struct sc_hid_event hid_event;
// Not all keys are supported, just ignore unsupported keys
if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
if (!kb->mod_lock_synchronized) {
// Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
// keyboard state
if (push_mod_lock_state(kb, event->mods_state)) {
kb->mod_lock_synchronized = true;
}
}
// If ack_to_wait is != SC_SEQUENCE_INVALID, then Ctrl+v is pressed, so
// clipboard synchronization has been requested. Wait until clipboard
// synchronization is acknowledged by the server, otherwise it could
// paste the old clipboard content.
if (!sc_aoa_push_hid_event_with_ack_to_wait(kb->aoa,
HID_KEYBOARD_ACCESSORY_ID,
&hid_event,
ack_to_wait)) {
LOGW("Could not request HID event (key)");
}
}
}
bool
sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa) {
kb->aoa = aoa;
bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
SC_HID_KEYBOARD_REPORT_DESC,
SC_HID_KEYBOARD_REPORT_DESC_LEN);
if (!ok) {
LOGW("Register HID keyboard failed");
return false;
}
sc_hid_keyboard_init(&kb->hid);
kb->mod_lock_synchronized = false;
static const struct sc_key_processor_ops ops = {
.process_key = sc_key_processor_process_key,
// Never forward text input via HID (all the keys are injected
// separately)
.process_text = NULL,
};
// Clipboard synchronization is requested over the control socket, while HID
// events are sent over AOA, so it must wait for clipboard synchronization
// to be acknowledged by the device before injecting Ctrl+v.
kb->key_processor.async_paste = true;
kb->key_processor.hid = true;
kb->key_processor.ops = &ops;
return true;
}
void
sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb) {
// Unregister HID keyboard so the soft keyboard shows again on Android
bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
if (!ok) {
LOGW("Could not unregister HID keyboard");
}
}

View File

@ -0,0 +1,27 @@
#ifndef SC_KEYBOARD_AOA_H
#define SC_KEYBOARD_AOA_H
#include "common.h"
#include <stdbool.h>
#include "aoa_hid.h"
#include "hid/hid_keyboard.h"
#include "trait/key_processor.h"
struct sc_keyboard_aoa {
struct sc_key_processor key_processor; // key processor trait
struct sc_hid_keyboard hid;
struct sc_aoa *aoa;
bool mod_lock_synchronized;
};
bool
sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa);
void
sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb);
#endif

89
app/src/usb/mouse_aoa.c Normal file
View File

@ -0,0 +1,89 @@
#include "mouse_aoa.h"
#include <assert.h>
#include "hid/hid_mouse.h"
#include "input_events.h"
#include "util/log.h"
/** Downcast mouse processor to mouse_aoa */
#define DOWNCAST(MP) container_of(MP, struct sc_mouse_aoa, mouse_processor)
#define HID_MOUSE_ACCESSORY_ID 2
static void
sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
const struct sc_mouse_motion_event *event) {
struct sc_mouse_aoa *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
sc_hid_mouse_event_from_motion(&hid_event, event);
if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
&hid_event)) {
LOGW("Could not request HID event (mouse motion)");
}
}
static void
sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
const struct sc_mouse_click_event *event) {
struct sc_mouse_aoa *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
sc_hid_mouse_event_from_click(&hid_event, event);
if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
&hid_event)) {
LOGW("Could not request HID event (mouse click)");
}
}
static void
sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
const struct sc_mouse_scroll_event *event) {
struct sc_mouse_aoa *mouse = DOWNCAST(mp);
struct sc_hid_event hid_event;
sc_hid_mouse_event_from_scroll(&hid_event, event);
if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
&hid_event)) {
LOGW("Could not request HID event (mouse scroll)");
}
}
bool
sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa) {
mouse->aoa = aoa;
bool ok = sc_aoa_setup_hid(aoa, HID_MOUSE_ACCESSORY_ID,
SC_HID_MOUSE_REPORT_DESC,
SC_HID_MOUSE_REPORT_DESC_LEN);
if (!ok) {
LOGW("Register HID mouse failed");
return false;
}
static const struct sc_mouse_processor_ops ops = {
.process_mouse_motion = sc_mouse_processor_process_mouse_motion,
.process_mouse_click = sc_mouse_processor_process_mouse_click,
.process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
// Touch events not supported (coordinates are not relative)
.process_touch = NULL,
};
mouse->mouse_processor.ops = &ops;
mouse->mouse_processor.relative_mode = true;
return true;
}
void
sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse) {
bool ok = sc_aoa_unregister_hid(mouse->aoa, HID_MOUSE_ACCESSORY_ID);
if (!ok) {
LOGW("Could not unregister HID mouse");
}
}

View File

@ -1,5 +1,5 @@
#ifndef SC_HID_MOUSE_H
#define SC_HID_MOUSE_H
#ifndef SC_MOUSE_AOA_H
#define SC_MOUSE_AOA_H
#include "common.h"
@ -8,16 +8,16 @@
#include "aoa_hid.h"
#include "trait/mouse_processor.h"
struct sc_hid_mouse {
struct sc_mouse_aoa {
struct sc_mouse_processor mouse_processor; // mouse processor trait
struct sc_aoa *aoa;
};
bool
sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa);
sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa);
void
sc_hid_mouse_destroy(struct sc_hid_mouse *mouse);
sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse);
#endif

View File

@ -10,8 +10,8 @@
struct scrcpy_otg {
struct sc_usb usb;
struct sc_aoa aoa;
struct sc_hid_keyboard keyboard;
struct sc_hid_mouse mouse;
struct sc_keyboard_aoa keyboard;
struct sc_mouse_aoa mouse;
struct sc_screen_otg screen_otg;
};
@ -73,8 +73,8 @@ scrcpy_otg(struct scrcpy_options *options) {
enum scrcpy_exit_code ret = SCRCPY_EXIT_FAILURE;
struct sc_hid_keyboard *keyboard = NULL;
struct sc_hid_mouse *mouse = NULL;
struct sc_keyboard_aoa *keyboard = NULL;
struct sc_mouse_aoa *mouse = NULL;
bool usb_device_initialized = false;
bool usb_connected = false;
bool aoa_started = false;
@ -117,10 +117,15 @@ scrcpy_otg(struct scrcpy_options *options) {
}
aoa_initialized = true;
assert(options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA
|| options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_DISABLED);
assert(options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA
|| options->mouse_input_mode == SC_MOUSE_INPUT_MODE_DISABLED);
bool enable_keyboard =
options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
bool enable_mouse =
options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
// If neither --hid-keyboard or --hid-mouse is passed, enable both
if (!enable_keyboard && !enable_mouse) {
@ -129,7 +134,7 @@ scrcpy_otg(struct scrcpy_options *options) {
}
if (enable_keyboard) {
ok = sc_hid_keyboard_init(&s->keyboard, &s->aoa);
ok = sc_keyboard_aoa_init(&s->keyboard, &s->aoa);
if (!ok) {
goto end;
}
@ -137,7 +142,7 @@ scrcpy_otg(struct scrcpy_options *options) {
}
if (enable_mouse) {
ok = sc_hid_mouse_init(&s->mouse, &s->aoa);
ok = sc_mouse_aoa_init(&s->mouse, &s->aoa);
if (!ok) {
goto end;
}
@ -186,10 +191,10 @@ end:
sc_usb_stop(&s->usb);
if (mouse) {
sc_hid_mouse_destroy(&s->mouse);
sc_mouse_aoa_destroy(&s->mouse);
}
if (keyboard) {
sc_hid_keyboard_destroy(&s->keyboard);
sc_keyboard_aoa_destroy(&s->keyboard);
}
if (aoa_initialized) {

View File

@ -6,12 +6,12 @@
#include <stdbool.h>
#include <SDL2/SDL.h>
#include "hid_keyboard.h"
#include "hid_mouse.h"
#include "keyboard_aoa.h"
#include "mouse_aoa.h"
struct sc_screen_otg {
struct sc_hid_keyboard *keyboard;
struct sc_hid_mouse *mouse;
struct sc_keyboard_aoa *keyboard;
struct sc_mouse_aoa *mouse;
SDL_Window *window;
SDL_Renderer *renderer;
@ -22,8 +22,8 @@ struct sc_screen_otg {
};
struct sc_screen_otg_params {
struct sc_hid_keyboard *keyboard;
struct sc_hid_mouse *mouse;
struct sc_keyboard_aoa *keyboard;
struct sc_mouse_aoa *mouse;
const char *window_title;
bool always_on_top;

View File

@ -3,6 +3,7 @@
#include <assert.h>
#include <errno.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@ -333,3 +334,22 @@ sc_str_remove_trailing_cr(char *s, size_t len) {
}
return len;
}
char *
sc_str_to_hex_string(const uint8_t *data, size_t size) {
size_t buffer_size = size * 3 + 1;
char *buffer = malloc(buffer_size);
if (!buffer) {
LOG_OOM();
return NULL;
}
for (size_t i = 0; i < size; ++i) {
snprintf(buffer + i * 3, 4, "%02X ", data[i]);
}
// Remove the final space
buffer[size * 3] = '\0';
return buffer;
}

View File

@ -138,4 +138,10 @@ sc_str_index_of_column(const char *s, unsigned col, const char *seps);
size_t
sc_str_remove_trailing_cr(char *s, size_t len);
/**
* Convert binary data to hexadecimal string
*/
char *
sc_str_to_hex_string(const uint8_t *data, size_t len);
#endif

View File

@ -323,6 +323,68 @@ static void test_serialize_rotate_device(void) {
assert(!memcmp(buf, expected, sizeof(expected)));
}
static void test_serialize_uhid_create(void) {
const uint8_t report_desc[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11};
struct sc_control_msg msg = {
.type = SC_CONTROL_MSG_TYPE_UHID_CREATE,
.uhid_create = {
.id = 42,
.report_desc_size = sizeof(report_desc),
.report_desc = report_desc,
},
};
uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
size_t size = sc_control_msg_serialize(&msg, buf);
assert(size == 16);
const uint8_t expected[] = {
SC_CONTROL_MSG_TYPE_UHID_CREATE,
0, 42, // id
0, 11, // size
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
};
assert(!memcmp(buf, expected, sizeof(expected)));
}
static void test_serialize_uhid_input(void) {
struct sc_control_msg msg = {
.type = SC_CONTROL_MSG_TYPE_UHID_INPUT,
.uhid_input = {
.id = 42,
.size = 5,
.data = {1, 2, 3, 4, 5},
},
};
uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
size_t size = sc_control_msg_serialize(&msg, buf);
assert(size == 10);
const uint8_t expected[] = {
SC_CONTROL_MSG_TYPE_UHID_INPUT,
0, 42, // id
0, 5, // size
1, 2, 3, 4, 5,
};
assert(!memcmp(buf, expected, sizeof(expected)));
}
static void test_serialize_open_hard_keyboard(void) {
struct sc_control_msg msg = {
.type = SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
};
uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
size_t size = sc_control_msg_serialize(&msg, buf);
assert(size == 1);
const uint8_t expected[] = {
SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
};
assert(!memcmp(buf, expected, sizeof(expected)));
}
int main(int argc, char *argv[]) {
(void) argc;
(void) argv;
@ -341,5 +403,8 @@ int main(int argc, char *argv[]) {
test_serialize_set_clipboard_long();
test_serialize_set_screen_power_mode();
test_serialize_rotate_device();
test_serialize_uhid_create();
test_serialize_uhid_input();
test_serialize_open_hard_keyboard();
return 0;
}

View File

@ -61,6 +61,28 @@ static void test_deserialize_ack_set_clipboard(void) {
assert(msg.ack_clipboard.sequence == UINT64_C(0x0102030405060708));
}
static void test_deserialize_uhid_output(void) {
const uint8_t input[] = {
DEVICE_MSG_TYPE_UHID_OUTPUT,
0, 42, // id
0, 5, // size
0x01, 0x02, 0x03, 0x04, 0x05, // data
};
struct sc_device_msg msg;
ssize_t r = sc_device_msg_deserialize(input, sizeof(input), &msg);
assert(r == 10);
assert(msg.type == DEVICE_MSG_TYPE_UHID_OUTPUT);
assert(msg.uhid_output.id == 42);
assert(msg.uhid_output.size == 5);
uint8_t expected[] = {1, 2, 3, 4, 5};
assert(!memcmp(msg.uhid_output.data, expected, sizeof(expected)));
sc_device_msg_destroy(&msg);
}
int main(int argc, char *argv[]) {
(void) argc;
(void) argv;
@ -68,5 +90,6 @@ int main(int argc, char *argv[]) {
test_deserialize_clipboard();
test_deserialize_clipboard_big();
test_deserialize_ack_set_clipboard();
test_deserialize_uhid_output();
return 0;
}

View File

@ -48,6 +48,7 @@ _<kbd>[Super]</kbd> is typically the <kbd>Windows</kbd> or <kbd>Cmd</kbd> key._
| Cut to clipboard⁵ | <kbd>MOD</kbd>+<kbd>x</kbd>
| Synchronize clipboards and paste⁵ | <kbd>MOD</kbd>+<kbd>v</kbd>
| Inject computer clipboard text | <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>v</kbd>
| Open keyboard settings (HID keyboard only) | <kbd>MOD</kbd>+<kbd>k</kbd>
| Enable/disable FPS counter (on stdout) | <kbd>MOD</kbd>+<kbd>i</kbd>
| Pinch-to-zoom/rotate | <kbd>Ctrl</kbd>+_click-and-move_
| Tilt (slide vertically with 2 fingers) | <kbd>Shift</kbd>+_click-and-move_

View File

@ -79,7 +79,7 @@ public final class AudioCapture {
intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
intent.addCategory(Intent.CATEGORY_LAUNCHER);
intent.setComponent(new ComponentName(FakeContext.PACKAGE_NAME, "com.android.shell.HeapDumpActivity"));
ServiceManager.getActivityManager().startActivityAsUserWithFeature(intent);
ServiceManager.getActivityManager().startActivity(intent);
}
private static void stopWorkaroundAndroid11() {

View File

@ -17,6 +17,9 @@ public final class ControlMessage {
public static final int TYPE_SET_CLIPBOARD = 9;
public static final int TYPE_SET_SCREEN_POWER_MODE = 10;
public static final int TYPE_ROTATE_DEVICE = 11;
public static final int TYPE_UHID_CREATE = 12;
public static final int TYPE_UHID_INPUT = 13;
public static final int TYPE_OPEN_HARD_KEYBOARD_SETTINGS = 14;
public static final long SEQUENCE_INVALID = 0;
@ -40,6 +43,8 @@ public final class ControlMessage {
private boolean paste;
private int repeat;
private long sequence;
private int id;
private byte[] data;
private ControlMessage() {
}
@ -123,6 +128,22 @@ public final class ControlMessage {
return msg;
}
public static ControlMessage createUhidCreate(int id, byte[] reportDesc) {
ControlMessage msg = new ControlMessage();
msg.type = TYPE_UHID_CREATE;
msg.id = id;
msg.data = reportDesc;
return msg;
}
public static ControlMessage createUhidInput(int id, byte[] data) {
ControlMessage msg = new ControlMessage();
msg.type = TYPE_UHID_INPUT;
msg.id = id;
msg.data = data;
return msg;
}
public int getType() {
return type;
}
@ -186,4 +207,12 @@ public final class ControlMessage {
public long getSequence() {
return sequence;
}
public int getId() {
return id;
}
public byte[] getData() {
return data;
}
}

View File

@ -15,6 +15,8 @@ public class ControlMessageReader {
static final int SET_SCREEN_POWER_MODE_PAYLOAD_LENGTH = 1;
static final int GET_CLIPBOARD_LENGTH = 1;
static final int SET_CLIPBOARD_FIXED_PAYLOAD_LENGTH = 9;
static final int UHID_CREATE_FIXED_PAYLOAD_LENGTH = 4;
static final int UHID_INPUT_FIXED_PAYLOAD_LENGTH = 4;
private static final int MESSAGE_MAX_SIZE = 1 << 18; // 256k
@ -84,8 +86,15 @@ public class ControlMessageReader {
case ControlMessage.TYPE_EXPAND_SETTINGS_PANEL:
case ControlMessage.TYPE_COLLAPSE_PANELS:
case ControlMessage.TYPE_ROTATE_DEVICE:
case ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
msg = ControlMessage.createEmpty(type);
break;
case ControlMessage.TYPE_UHID_CREATE:
msg = parseUhidCreate();
break;
case ControlMessage.TYPE_UHID_INPUT:
msg = parseUhidInput();
break;
default:
Ln.w("Unknown event type: " + type);
msg = null;
@ -110,12 +119,21 @@ public class ControlMessageReader {
return ControlMessage.createInjectKeycode(action, keycode, repeat, metaState);
}
private String parseString() {
if (buffer.remaining() < 4) {
return null;
private int parseBufferLength(int sizeBytes) {
assert sizeBytes > 0 && sizeBytes <= 4;
if (buffer.remaining() < sizeBytes) {
return -1;
}
int len = buffer.getInt();
if (buffer.remaining() < len) {
int value = 0;
for (int i = 0; i < sizeBytes; ++i) {
value = (value << 8) | (buffer.get() & 0xFF);
}
return value;
}
private String parseString() {
int len = parseBufferLength(4);
if (len == -1 || buffer.remaining() < len) {
return null;
}
int position = buffer.position();
@ -124,6 +142,16 @@ public class ControlMessageReader {
return new String(rawBuffer, position, len, StandardCharsets.UTF_8);
}
private byte[] parseByteArray(int sizeBytes) {
int len = parseBufferLength(sizeBytes);
if (len == -1 || buffer.remaining() < len) {
return null;
}
byte[] data = new byte[len];
buffer.get(data);
return data;
}
private ControlMessage parseInjectText() {
String text = parseString();
if (text == null) {
@ -193,6 +221,30 @@ public class ControlMessageReader {
return ControlMessage.createSetScreenPowerMode(mode);
}
private ControlMessage parseUhidCreate() {
if (buffer.remaining() < UHID_CREATE_FIXED_PAYLOAD_LENGTH) {
return null;
}
int id = buffer.getShort();
byte[] data = parseByteArray(2);
if (data == null) {
return null;
}
return ControlMessage.createUhidCreate(id, data);
}
private ControlMessage parseUhidInput() {
if (buffer.remaining() < UHID_INPUT_FIXED_PAYLOAD_LENGTH) {
return null;
}
int id = buffer.getShort();
byte[] data = parseByteArray(2);
if (data == null) {
return null;
}
return ControlMessage.createUhidInput(id, data);
}
private static Position readPosition(ByteBuffer buffer) {
int x = buffer.getInt();
int y = buffer.getInt();

View File

@ -1,7 +1,9 @@
package com.genymobile.scrcpy;
import com.genymobile.scrcpy.wrappers.InputManager;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import android.content.Intent;
import android.os.Build;
import android.os.SystemClock;
import android.view.InputDevice;
@ -26,6 +28,8 @@ public class Controller implements AsyncProcessor {
private Thread thread;
private UhidManager uhidManager;
private final Device device;
private final ControlChannel controlChannel;
private final CleanUp cleanUp;
@ -52,6 +56,13 @@ public class Controller implements AsyncProcessor {
sender = new DeviceMessageSender(controlChannel);
}
private UhidManager getUhidManager() {
if (uhidManager == null) {
uhidManager = new UhidManager(sender);
}
return uhidManager;
}
private void initPointers() {
for (int i = 0; i < PointersState.MAX_POINTERS; ++i) {
MotionEvent.PointerProperties props = new MotionEvent.PointerProperties();
@ -81,8 +92,9 @@ public class Controller implements AsyncProcessor {
SystemClock.sleep(500);
}
while (!Thread.currentThread().isInterrupted()) {
handleEvent();
boolean alive = true;
while (!Thread.currentThread().isInterrupted() && alive) {
alive = handleEvent();
}
}
@ -92,9 +104,12 @@ public class Controller implements AsyncProcessor {
try {
control();
} catch (IOException e) {
// this is expected on close
Ln.e("Controller error", e);
} finally {
Ln.d("Controller stopped");
if (uhidManager != null) {
uhidManager.closeAll();
}
listener.onTerminated(true);
}
}, "control-recv");
@ -122,8 +137,15 @@ public class Controller implements AsyncProcessor {
return sender;
}
private void handleEvent() throws IOException {
ControlMessage msg = controlChannel.recv();
private boolean handleEvent() throws IOException {
ControlMessage msg;
try {
msg = controlChannel.recv();
} catch (IOException e) {
// this is expected on close
return false;
}
switch (msg.getType()) {
case ControlMessage.TYPE_INJECT_KEYCODE:
if (device.supportsInputEvents()) {
@ -182,9 +204,20 @@ public class Controller implements AsyncProcessor {
case ControlMessage.TYPE_ROTATE_DEVICE:
device.rotateDevice();
break;
case ControlMessage.TYPE_UHID_CREATE:
getUhidManager().open(msg.getId(), msg.getData());
break;
case ControlMessage.TYPE_UHID_INPUT:
getUhidManager().writeInput(msg.getId(), msg.getData());
break;
case ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
openHardKeyboardSettings();
break;
default:
// do nothing
}
return true;
}
private boolean injectKeycode(int action, int keycode, int repeat, int metaState) {
@ -393,7 +426,8 @@ public class Controller implements AsyncProcessor {
if (!clipboardAutosync) {
String clipboardText = Device.getClipboardText();
if (clipboardText != null) {
sender.pushClipboardText(clipboardText);
DeviceMessage msg = DeviceMessage.createClipboard(clipboardText);
sender.send(msg);
}
}
}
@ -411,9 +445,15 @@ public class Controller implements AsyncProcessor {
if (sequence != ControlMessage.SEQUENCE_INVALID) {
// Acknowledgement requested
sender.pushAckClipboard(sequence);
DeviceMessage msg = DeviceMessage.createAckClipboard(sequence);
sender.send(msg);
}
return ok;
}
private void openHardKeyboardSettings() {
Intent intent = new Intent("android.settings.HARD_KEYBOARD_SETTINGS");
ServiceManager.getActivityManager().startActivity(intent);
}
}

View File

@ -4,12 +4,13 @@ public final class DeviceMessage {
public static final int TYPE_CLIPBOARD = 0;
public static final int TYPE_ACK_CLIPBOARD = 1;
public static final long SEQUENCE_INVALID = ControlMessage.SEQUENCE_INVALID;
public static final int TYPE_UHID_OUTPUT = 2;
private int type;
private String text;
private long sequence;
private int id;
private byte[] data;
private DeviceMessage() {
}
@ -28,6 +29,14 @@ public final class DeviceMessage {
return event;
}
public static DeviceMessage createUhidOutput(int id, byte[] data) {
DeviceMessage event = new DeviceMessage();
event.type = TYPE_UHID_OUTPUT;
event.id = id;
event.data = data;
return event;
}
public int getType() {
return type;
}
@ -39,4 +48,12 @@ public final class DeviceMessage {
public long getSequence() {
return sequence;
}
public int getId() {
return id;
}
public byte[] getData() {
return data;
}
}

View File

@ -1,54 +1,30 @@
package com.genymobile.scrcpy;
import java.io.IOException;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
public final class DeviceMessageSender {
private final ControlChannel controlChannel;
private Thread thread;
private String clipboardText;
private long ack;
private final BlockingQueue<DeviceMessage> queue = new ArrayBlockingQueue<>(16);
public DeviceMessageSender(ControlChannel controlChannel) {
this.controlChannel = controlChannel;
}
public synchronized void pushClipboardText(String text) {
clipboardText = text;
notify();
public void send(DeviceMessage msg) {
if (!queue.offer(msg)) {
Ln.w("Device message dropped: " + msg.getType());
}
public synchronized void pushAckClipboard(long sequence) {
ack = sequence;
notify();
}
private void loop() throws IOException, InterruptedException {
while (!Thread.currentThread().isInterrupted()) {
String text;
long sequence;
synchronized (this) {
while (ack == DeviceMessage.SEQUENCE_INVALID && clipboardText == null) {
wait();
}
text = clipboardText;
clipboardText = null;
sequence = ack;
ack = DeviceMessage.SEQUENCE_INVALID;
}
if (sequence != DeviceMessage.SEQUENCE_INVALID) {
DeviceMessage event = DeviceMessage.createAckClipboard(sequence);
controlChannel.send(event);
}
if (text != null) {
DeviceMessage event = DeviceMessage.createClipboard(text);
controlChannel.send(event);
}
DeviceMessage msg = queue.take();
controlChannel.send(msg);
}
}

View File

@ -29,6 +29,13 @@ public class DeviceMessageWriter {
buffer.putLong(msg.getSequence());
output.write(rawBuffer, 0, buffer.position());
break;
case DeviceMessage.TYPE_UHID_OUTPUT:
buffer.putShort((short) msg.getId());
byte[] data = msg.getData();
buffer.putShort((short) data.length);
buffer.put(data);
output.write(rawBuffer, 0, buffer.position());
break;
default:
Ln.w("Unknown device message: " + msg.getType());
break;

View File

@ -133,7 +133,10 @@ public final class Server {
if (control) {
ControlChannel controlChannel = connection.getControlChannel();
Controller controller = new Controller(device, controlChannel, cleanUp, options.getClipboardAutosync(), options.getPowerOn());
device.setClipboardListener(text -> controller.getSender().pushClipboardText(text));
device.setClipboardListener(text -> {
DeviceMessage msg = DeviceMessage.createClipboard(text);
controller.getSender().send(msg);
});
asyncProcessors.add(controller);
}

View File

@ -0,0 +1,218 @@
package com.genymobile.scrcpy;
import android.os.Build;
import android.os.HandlerThread;
import android.os.MessageQueue;
import android.system.ErrnoException;
import android.system.Os;
import android.system.OsConstants;
import android.util.ArrayMap;
import java.io.FileDescriptor;
import java.io.IOException;
import java.io.InterruptedIOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
public final class UhidManager {
// Linux: include/uapi/linux/uhid.h
private static final int UHID_OUTPUT = 6;
private static final int UHID_CREATE2 = 11;
private static final int UHID_INPUT2 = 12;
// Linux: include/uapi/linux/input.h
private static final short BUS_VIRTUAL = 0x06;
private static final int SIZE_OF_UHID_EVENT = 4380; // sizeof(struct uhid_event)
private final ArrayMap<Integer, FileDescriptor> fds = new ArrayMap<>();
private final ByteBuffer buffer = ByteBuffer.allocate(SIZE_OF_UHID_EVENT).order(ByteOrder.nativeOrder());
private final DeviceMessageSender sender;
private final HandlerThread thread = new HandlerThread("UHidManager");
private final MessageQueue queue;
public UhidManager(DeviceMessageSender sender) {
this.sender = sender;
thread.start();
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
queue = thread.getLooper().getQueue();
} else {
queue = null;
}
}
public void open(int id, byte[] reportDesc) throws IOException {
try {
FileDescriptor fd = Os.open("/dev/uhid", OsConstants.O_RDWR, 0);
try {
FileDescriptor old = fds.put(id, fd);
if (old != null) {
Ln.w("Duplicate UHID id: " + id);
close(old);
}
byte[] req = buildUhidCreate2Req(reportDesc);
Os.write(fd, req, 0, req.length);
registerUhidListener(id, fd);
} catch (Exception e) {
close(fd);
throw e;
}
} catch (ErrnoException e) {
throw new IOException(e);
}
}
private void registerUhidListener(int id, FileDescriptor fd) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
queue.addOnFileDescriptorEventListener(fd, MessageQueue.OnFileDescriptorEventListener.EVENT_INPUT, (fd2, events) -> {
try {
buffer.clear();
int r = Os.read(fd2, buffer);
buffer.flip();
if (r > 0) {
int type = buffer.getInt();
if (type == UHID_OUTPUT) {
byte[] data = extractHidOutputData(buffer);
if (data != null) {
DeviceMessage msg = DeviceMessage.createUhidOutput(id, data);
sender.send(msg);
}
}
}
} catch (ErrnoException | InterruptedIOException e) {
Ln.e("Failed to read UHID output", e);
return 0;
}
return events;
});
}
}
private static byte[] extractHidOutputData(ByteBuffer buffer) {
/*
* #define UHID_DATA_MAX 4096
* struct uhid_event {
* uint32_t type;
* union {
* // ...
* struct uhid_output_req {
* __u8 data[UHID_DATA_MAX];
* __u16 size;
* __u8 rtype;
* };
* };
* } __attribute__((__packed__));
*/
if (buffer.remaining() < 4099) {
Ln.w("Incomplete HID output");
return null;
}
int size = buffer.getShort(buffer.position() + 4096) & 0xFFFF;
if (size > 4096) {
Ln.w("Incorrect HID output size: " + size);
return null;
}
byte[] data = new byte[size];
buffer.get(data);
return data;
}
public void writeInput(int id, byte[] data) throws IOException {
FileDescriptor fd = fds.get(id);
if (fd == null) {
Ln.w("Unknown UHID id: " + id);
return;
}
try {
byte[] req = buildUhidInput2Req(data);
Os.write(fd, req, 0, req.length);
} catch (ErrnoException e) {
throw new IOException(e);
}
}
private static byte[] buildUhidCreate2Req(byte[] reportDesc) {
/*
* struct uhid_event {
* uint32_t type;
* union {
* // ...
* struct uhid_create2_req {
* uint8_t name[128];
* uint8_t phys[64];
* uint8_t uniq[64];
* uint16_t rd_size;
* uint16_t bus;
* uint32_t vendor;
* uint32_t product;
* uint32_t version;
* uint32_t country;
* uint8_t rd_data[HID_MAX_DESCRIPTOR_SIZE];
* };
* };
* } __attribute__((__packed__));
*/
byte[] empty = new byte[256];
ByteBuffer buf = ByteBuffer.allocate(280 + reportDesc.length).order(ByteOrder.nativeOrder());
buf.putInt(UHID_CREATE2);
buf.put("scrcpy".getBytes(StandardCharsets.US_ASCII));
buf.put(empty, 0, 256 - "scrcpy".length());
buf.putShort((short) reportDesc.length);
buf.putShort(BUS_VIRTUAL);
buf.putInt(0); // vendor id
buf.putInt(0); // product id
buf.putInt(0); // version
buf.putInt(0); // country;
buf.put(reportDesc);
return buf.array();
}
private static byte[] buildUhidInput2Req(byte[] data) {
/*
* struct uhid_event {
* uint32_t type;
* union {
* // ...
* struct uhid_input2_req {
* uint16_t size;
* uint8_t data[UHID_DATA_MAX];
* };
* };
* } __attribute__((__packed__));
*/
ByteBuffer buf = ByteBuffer.allocate(6 + data.length).order(ByteOrder.nativeOrder());
buf.putInt(UHID_INPUT2);
buf.putShort((short) data.length);
buf.put(data);
return buf.array();
}
public void close(int id) {
FileDescriptor fd = fds.get(id);
assert fd != null;
close(fd);
}
public void closeAll() {
for (FileDescriptor fd : fds.values()) {
close(fd);
}
}
private static void close(FileDescriptor fd) {
try {
Os.close(fd);
} catch (ErrnoException e) {
Ln.e("Failed to close uhid: " + e.getMessage());
}
}
}

View File

@ -22,7 +22,7 @@ public final class ActivityManager {
private Method getContentProviderExternalMethod;
private boolean getContentProviderExternalMethodNewVersion = true;
private Method removeContentProviderExternalMethod;
private Method startActivityAsUserWithFeatureMethod;
private Method startActivityAsUserMethod;
private Method forceStopPackageMethod;
static ActivityManager create() {
@ -107,26 +107,25 @@ public final class ActivityManager {
return getContentProviderExternal("settings", new Binder());
}
private Method getStartActivityAsUserWithFeatureMethod() throws NoSuchMethodException, ClassNotFoundException {
if (startActivityAsUserWithFeatureMethod == null) {
private Method getStartActivityAsUserMethod() throws NoSuchMethodException, ClassNotFoundException {
if (startActivityAsUserMethod == null) {
Class<?> iApplicationThreadClass = Class.forName("android.app.IApplicationThread");
Class<?> profilerInfo = Class.forName("android.app.ProfilerInfo");
startActivityAsUserWithFeatureMethod = manager.getClass()
.getMethod("startActivityAsUserWithFeature", iApplicationThreadClass, String.class, String.class, Intent.class, String.class,
IBinder.class, String.class, int.class, int.class, profilerInfo, Bundle.class, int.class);
startActivityAsUserMethod = manager.getClass()
.getMethod("startActivityAsUser", iApplicationThreadClass, String.class, Intent.class, String.class, IBinder.class, String.class,
int.class, int.class, profilerInfo, Bundle.class, int.class);
}
return startActivityAsUserWithFeatureMethod;
return startActivityAsUserMethod;
}
@SuppressWarnings("ConstantConditions")
public int startActivityAsUserWithFeature(Intent intent) {
public int startActivity(Intent intent) {
try {
Method method = getStartActivityAsUserWithFeatureMethod();
Method method = getStartActivityAsUserMethod();
return (int) method.invoke(
/* this */ manager,
/* caller */ null,
/* callingPackage */ FakeContext.PACKAGE_NAME,
/* callingFeatureId */ null,
/* intent */ intent,
/* resolvedType */ null,
/* resultTo */ null,

View File

@ -322,6 +322,66 @@ public class ControlMessageReaderTest {
Assert.assertEquals(ControlMessage.TYPE_ROTATE_DEVICE, event.getType());
}
@Test
public void testParseUhidCreate() throws IOException {
ControlMessageReader reader = new ControlMessageReader();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(bos);
dos.writeByte(ControlMessage.TYPE_UHID_CREATE);
dos.writeShort(42); // id
byte[] data = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11};
dos.writeShort(data.length); // size
dos.write(data);
byte[] packet = bos.toByteArray();
reader.readFrom(new ByteArrayInputStream(packet));
ControlMessage event = reader.next();
Assert.assertEquals(ControlMessage.TYPE_UHID_CREATE, event.getType());
Assert.assertEquals(42, event.getId());
Assert.assertArrayEquals(data, event.getData());
}
@Test
public void testParseUhidInput() throws IOException {
ControlMessageReader reader = new ControlMessageReader();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(bos);
dos.writeByte(ControlMessage.TYPE_UHID_INPUT);
dos.writeShort(42); // id
byte[] data = {1, 2, 3, 4, 5};
dos.writeShort(data.length); // size
dos.write(data);
byte[] packet = bos.toByteArray();
reader.readFrom(new ByteArrayInputStream(packet));
ControlMessage event = reader.next();
Assert.assertEquals(ControlMessage.TYPE_UHID_INPUT, event.getType());
Assert.assertEquals(42, event.getId());
Assert.assertArrayEquals(data, event.getData());
}
@Test
public void testParseOpenHardKeyboardSettings() throws IOException {
ControlMessageReader reader = new ControlMessageReader();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(bos);
dos.writeByte(ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS);
byte[] packet = bos.toByteArray();
reader.readFrom(new ByteArrayInputStream(packet));
ControlMessage event = reader.next();
Assert.assertEquals(ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS, event.getType());
}
@Test
public void testMultiEvents() throws IOException {
ControlMessageReader reader = new ControlMessageReader();

View File

@ -52,4 +52,27 @@ public class DeviceMessageWriterTest {
Assert.assertArrayEquals(expected, actual);
}
@Test
public void testSerializeUhidOutput() throws IOException {
DeviceMessageWriter writer = new DeviceMessageWriter();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(bos);
dos.writeByte(DeviceMessage.TYPE_UHID_OUTPUT);
dos.writeShort(42); // id
byte[] data = {1, 2, 3, 4, 5};
dos.writeShort(data.length);
dos.write(data);
byte[] expected = bos.toByteArray();
DeviceMessage msg = DeviceMessage.createUhidOutput(42, data);
bos = new ByteArrayOutputStream();
writer.writeTo(msg, bos);
byte[] actual = bos.toByteArray();
Assert.assertArrayEquals(expected, actual);
}
}