Detect missing initializations

Write invalid data in memory to easily detect missing initializations in
debug mode.

FAQ.md

@@ -133,9 +133,9 @@ Try with another USB cable or plug it into another USB port. See [#281] and
 [#283]: https://github.com/Genymobile/scrcpy/issues/283
 
 
-## HID/OTG issues on Windows
+## OTG issues on Windows
 
-On Windows, if `scrcpy --otg` (or `--hid-keyboard`/`--hid-mouse`) results in:
+On Windows, if `scrcpy --otg` (or `--keyboard=aoa`/`--mouse=aoa`) results in:
 
 >     ERROR: Could not find any USB device
 
@@ -170,12 +170,13 @@ The default text injection method is [limited to ASCII characters][text-input].
 A trick allows to also inject some [accented characters][accented-characters],
 but that's all. See [#37].
 
-It is also possible to simulate a [physical keyboard][hid] (HID).
+To avoid the problem, [change the keyboard mode to simulate a physical
+keyboard][hid].
 
 [text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
 [accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
 [#37]: https://github.com/Genymobile/scrcpy/issues/37
-[hid]: doc/hid-otg.md
+[hid]: doc/keyboard.md#physical-keyboard-simulation
 
 
 ## Client issues

README.md

@@ -32,10 +32,13 @@ Its features include:
  - [configurable quality](doc/video.md)
  - [camera mirroring](doc/camera.md) (Android 12+)
  - [mirroring as a webcam (V4L2)](doc/v4l2.md) (Linux-only)
- - [physical keyboard/mouse simulation (HID)](doc/hid-otg.md)
- - [OTG mode](doc/hid-otg.md#otg)
+ - physical [keyboard][hid-keyboard] and [mouse][hid-mouse] simulation (HID)
+ - [OTG mode](doc/otg.md)
  - and more…
 
+[hid-keyboard]: doc/keyboard.md#physical-keyboard-simulation
+[hid-mouse]: doc/mouse.md#physical-mouse-simulation
+
 ## Prerequisites
 
 The Android device requires at least API 21 (Android 5.0).
@@ -53,8 +56,7 @@ this option is set.
 
 [control]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
 
-Note that USB debugging is not required to run scrcpy in [OTG
-mode](doc/hid-otg.md#otg).
+Note that USB debugging is not required to run scrcpy in [OTG mode](doc/otg.md).
 
 
 ## Get the app
@@ -64,6 +66,39 @@ mode](doc/hid-otg.md#otg).
  - [macOS](doc/macos.md)
 
 
+## Usage examples
+
+There are a lot of options, [documented](#user-documentation) in separate pages.
+Here are just some common examples.
+
+ - Capture the screen in H.265 (better quality), limit the size to 1920, limit
+   the frame rate to 60fps, disable audio, and control the device by simulating
+   a physical keyboard:
+
+    ```bash
+    scrcpy --video-codec=h265 --max-size=1920 --max-fps=60 --no-audio --keyboard=uhid
+    scrcpy --video-codec=h265 -m1920 --max-fps=60 --no-audio -K  # short version
+    ```
+
+ - Record the device camera in H.265 at 1920x1080 (and microphone) to an MP4 file:
+
+    ```bash
+    scrcpy --video-source=camera --video-codec=h265 --camera-size=1920x1080 --record=file.mp4
+    ```
+
+ - Capture the device front camera and expose it as a webcam on the computer (on Linux):
+
+    ```bash
+    scrcpy --video-source=camera --camera-size=1920x1080 --camera-facing=front --v4l2-sink=/dev/video2 --no-playback
+    ```
+
+ - Control the device without mirroring by simulating a physical keyboard and mouse
+(USB debugging not required):
+
+    ```bash
+    scrcpy --otg
+    ```
+
 ## User documentation
 
 The application provides a lot of features and configuration options. They are
@@ -73,11 +108,13 @@ documented in the following pages:
  - [Video](doc/video.md)
  - [Audio](doc/audio.md)
  - [Control](doc/control.md)
+ - [Keyboard](doc/keyboard.md)
+ - [Mouse](doc/mouse.md)
  - [Device](doc/device.md)
  - [Window](doc/window.md)
  - [Recording](doc/recording.md)
  - [Tunnels](doc/tunnels.md)
- - [HID/OTG](doc/hid-otg.md)
+ - [OTG](doc/otg.md)
  - [Camera](doc/camera.md)
  - [Video4Linux](doc/v4l2.md)
  - [Shortcuts](doc/shortcuts.md)

app/data/bash-completion/scrcpy

@@ -27,8 +27,9 @@ _scrcpy() {
         --force-adb-forward
         --forward-all-clicks
         -h --help
+        -K
+        --keyboard=
         --kill-adb-on-close
-        -K --hid-keyboard
         --legacy-paste
         --list-camera-sizes
         --list-cameras
@@ -37,8 +38,9 @@ _scrcpy() {
         --lock-video-orientation
         --lock-video-orientation=
         -m --max-size=
-        -M --hid-mouse
+        -M
         --max-fps=
+        --mouse=
         -n --no-control
         -N --no-playback
         --no-audio
@@ -115,6 +117,14 @@ _scrcpy() {
             COMPREPLY=($(compgen -W 'front back external' -- "$cur"))
             return
             ;;
+        --keyboard)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
+            return
+            ;;
+        --mouse)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
+            return
+            ;;
         --orientation|--display-orientation)
             COMPREPLY=($(compgen -W '0 90 180 270 flip0 flip90 flip180 flip270' -- "$cur"))
             return

app/data/zsh-completion/_scrcpy

@@ -34,8 +34,9 @@ arguments=(
     '--force-adb-forward[Do not attempt to use \"adb reverse\" to connect to the device]'
     '--forward-all-clicks[Forward clicks to device]'
     {-h,--help}'[Print the help]'
+    '-K[Use UHID keyboard (same as --keyboard=uhid)]'
+    '--keyboard[Set the keyboard input mode]:mode:(disabled sdk uhid aoa)'
     '--kill-adb-on-close[Kill adb when scrcpy terminates]'
-    {-K,--hid-keyboard}'[Simulate a physical keyboard by using HID over AOAv2]'
     '--legacy-paste[Inject computer clipboard text as a sequence of key events on Ctrl+v]'
     '--list-camera-sizes[List the valid camera capture sizes]'
     '--list-cameras[List cameras available on the device]'
@@ -43,8 +44,9 @@ arguments=(
     '--list-encoders[List video and audio encoders available on the device]'
     '--lock-video-orientation=[Lock video orientation]:orientation:(unlocked initial 0 90 180 270)'
     {-m,--max-size=}'[Limit both the width and height of the video to value]'
-    {-M,--hid-mouse}'[Simulate a physical mouse by using HID over AOAv2]'
+    '-M[Use UHID mouse (same as --mouse=uhid)]'
     '--max-fps=[Limit the frame rate of screen capture]'
+    '--mouse[Set the mouse input mode]:mode:(disabled sdk uhid aoa)'
     {-n,--no-control}'[Disable device control \(mirror the device in read only\)]'
     {-N,--no-playback}'[Disable video and audio playback]'
     '--no-audio[Disable audio forwarding]'

app/meson.build

@@ -20,8 +20,8 @@ src = [
     'src/fps_counter.c',
     'src/frame_buffer.c',
     'src/input_manager.c',
-    'src/keyboard_inject.c',
-    'src/mouse_inject.c',
+    'src/keyboard_sdk.c',
+    'src/mouse_sdk.c',
     'src/opengl.c',
     'src/options.c',
     'src/packet_merger.c',
@@ -31,8 +31,13 @@ src = [
     'src/screen.c',
     'src/server.c',
     'src/version.c',
+    'src/hid/hid_keyboard.c',
+    'src/hid/hid_mouse.c',
     'src/trait/frame_source.c',
     'src/trait/packet_source.c',
+    'src/uhid/keyboard_uhid.c',
+    'src/uhid/mouse_uhid.c',
+    'src/uhid/uhid_output.c',
     'src/util/acksync.c',
     'src/util/audiobuf.c',
     'src/util/average.c',
@@ -88,8 +93,8 @@ usb_support = get_option('usb')
 if usb_support
     src += [
         'src/usb/aoa_hid.c',
-        'src/usb/hid_keyboard.c',
-        'src/usb/hid_mouse.c',
+        'src/usb/keyboard_aoa.c',
+        'src/usb/mouse_aoa.c',
         'src/usb/scrcpy_otg.c',
         'src/usb/screen_otg.c',
         'src/usb/usb.c',

app/scrcpy.1

@@ -172,24 +172,31 @@ By default, right-click triggers BACK (or POWER on) and middle-click triggers HO
 Print this help.
 
 .TP
-.B \-\-kill\-adb\-on\-close
-Kill adb when scrcpy terminates.
+.B \-K
+Same as \fB\-\-keyboard=uhid\fR.
 
 .TP
-.B \-K, \-\-hid\-keyboard
-Simulate a physical keyboard by using HID over AOAv2.
+.BI "\-\-keyboard " mode
+Select how to send keyboard inputs to the device.
 
-This provides a better experience for IME users, and allows to generate non-ASCII characters, contrary to the default injection method.
+Possible values are "disabled", "sdk", "uhid" and "aoa":
 
-It may only work over USB.
+ - "disabled" does not send keyboard inputs to the device.
+ - "sdk" uses the Android system API to deliver keyboard events to applications.
+ - "uhid" simulates a physical HID keyboard using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical HID keyboard using the AOAv2 protocol. It may only work over USB.
 
-The keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly:
+For "uhid" and "aoa", the keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly using the shortcut MOD+k (except in OTG mode), or by executing:
 
     adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
 
-However, the option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
+This option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
 
-Also see \fB\-\-hid\-mouse\fR.
+Also see \fB\-\-mouse\fR.
+
+.TP
+.B \-\-kill\-adb\-on\-close
+Kill adb when scrcpy terminates.
 
 .TP
 .B \-\-legacy\-paste
@@ -230,21 +237,31 @@ Limit both the width and height of the video to \fIvalue\fR. The other dimension
 Default is 0 (unlimited).
 
 .TP
-.B \-M, \-\-hid\-mouse
-Simulate a physical mouse by using HID over AOAv2.
-
-In this mode, the computer mouse is captured to control the device directly (relative mouse mode).
-
-LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
-
-It may only work over USB.
-
-Also see \fB\-\-hid\-keyboard\fR.
+.B \-M
+Same as \fB\-\-mouse=uhid\fR.
 
 .TP
 .BI "\-\-max\-fps " value
 Limit the framerate of screen capture (officially supported since Android 10, but may work on earlier versions).
 
+.TP
+.BI "\-\-mouse " mode
+Select how to send mouse inputs to the device.
+
+Possible values are "disabled", "sdk", "uhid" and "aoa":
+
+ - "disabled" does not send mouse inputs to the device.
+ - "sdk" uses the Android system API to deliver mouse events to applications.
+ - "uhid" simulates a physical HID mouse using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical mouse using the AOAv2 protocol. It may only work over USB.
+
+In "uhid" and "aoa" modes, the computer mouse is captured to control the device directly (relative mouse mode).
+
+LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
+
+Also see \fB\-\-keyboard\fR.
+
+
 .TP
 .B \-n, \-\-no\-control
 Disable device control (mirror the device in read\-only).
@@ -636,6 +653,10 @@ Copy computer clipboard to device, then paste (inject PASTE keycode, Android >=
 .B MOD+Shift+v
 Inject computer clipboard text as a sequence of key events
 
+.TP
+.B MOD+k
+Open keyboard settings on the device (for HID keyboard only)
+
 .TP
 .B MOD+i
 Enable/disable FPS counter (print frames/second in logs)

app/src/cli.c

@@ -93,6 +93,10 @@ enum {
     OPT_DISPLAY_ORIENTATION,
     OPT_RECORD_ORIENTATION,
     OPT_ORIENTATION,
+    OPT_KEYBOARD,
+    OPT_MOUSE,
+    OPT_HID_KEYBOARD_DEPRECATED,
+    OPT_HID_MOUSE_DEPRECATED,
 };
 
 struct sc_option {
@@ -358,27 +362,44 @@ static const struct sc_option options[] = {
         .longopt = "help",
         .text = "Print this help.",
     },
+    {
+        .shortopt = 'K',
+        .text = "Same as --keyboard=uhid.",
+    },
+    {
+        .longopt_id = OPT_KEYBOARD,
+        .longopt = "keyboard",
+        .argdesc = "mode",
+        .text = "Select how to send keyboard inputs to the device.\n"
+                "Possible values are \"disabled\", \"sdk\", \"uhid\" and "
+                "\"aoa\".\n"
+                "\"disabled\" does not send keyboard inputs to the device.\n"
+                "\"sdk\" uses the Android system API to deliver keyboard "
+                "events to applications.\n"
+                "\"uhid\" simulates a physical HID keyboard using the Linux "
+                "UHID kernel module on the device.\n"
+                "\"aoa\" simulates a physical keyboard using the AOAv2 "
+                "protocol. It may only work over USB.\n"
+                "For \"uhid\" and \"aoa\", the keyboard layout must be "
+                "configured (once and for all) on the device, via Settings -> "
+                "System -> Languages and input -> Physical keyboard. This "
+                "settings page can be started directly using the shortcut "
+                "MOD+k (except in OTG mode) or by executing: `adb shell am "
+                "start -a android.settings.HARD_KEYBOARD_SETTINGS`.\n"
+                "This option is only available when a HID keyboard is enabled "
+                "(or a physical keyboard is connected).\n"
+                "Also see --mouse.",
+    },
     {
         .longopt_id = OPT_KILL_ADB_ON_CLOSE,
         .longopt = "kill-adb-on-close",
         .text = "Kill adb when scrcpy terminates.",
     },
     {
-        .shortopt = 'K',
+        // deprecated
+        //.shortopt = 'K', // old, reassigned
+        .longopt_id = OPT_HID_KEYBOARD_DEPRECATED,
         .longopt = "hid-keyboard",
-        .text = "Simulate a physical keyboard by using HID over AOAv2.\n"
-                "It provides a better experience for IME users, and allows to "
-                "generate non-ASCII characters, contrary to the default "
-                "injection method.\n"
-                "It may only work over USB.\n"
-                "The keyboard layout must be configured (once and for all) on "
-                "the device, via Settings -> System -> Languages and input -> "
-                "Physical keyboard. This settings page can be started "
-                "directly: `adb shell am start -a "
-                "android.settings.HARD_KEYBOARD_SETTINGS`.\n"
-                "However, the option is only available when the HID keyboard "
-                "is enabled (or a physical keyboard is connected).\n"
-                "Also see --hid-mouse.",
     },
     {
         .longopt_id = OPT_LEGACY_PASTE,
@@ -432,15 +453,14 @@ static const struct sc_option options[] = {
                 "Default is 0 (unlimited).",
     },
     {
-        .shortopt = 'M',
+        // deprecated
+        //.shortopt = 'M', // old, reassigned
+        .longopt_id = OPT_HID_MOUSE_DEPRECATED,
         .longopt = "hid-mouse",
-        .text = "Simulate a physical mouse by using HID over AOAv2.\n"
-                "In this mode, the computer mouse is captured to control the "
-                "device directly (relative mouse mode).\n"
-                "LAlt, LSuper or RSuper toggle the capture mode, to give "
-                "control of the mouse back to the computer.\n"
-                "It may only work over USB.\n"
-                "Also see --hid-keyboard.",
+    },
+    {
+        .shortopt = 'M',
+        .text = "Same as --mouse=uhid.",
     },
     {
         .longopt_id = OPT_MAX_FPS,
@@ -449,6 +469,26 @@ static const struct sc_option options[] = {
         .text = "Limit the frame rate of screen capture (officially supported "
                 "since Android 10, but may work on earlier versions).",
     },
+    {
+        .longopt_id = OPT_MOUSE,
+        .longopt = "mouse",
+        .argdesc = "mode",
+        .text = "Select how to send mouse inputs to the device.\n"
+                "Possible values are \"disabled\", \"sdk\", \"uhid\" and "
+                "\"aoa\".\n"
+                "\"disabled\" does not send mouse inputs to the device.\n"
+                "\"sdk\" uses the Android system API to deliver mouse events"
+                "to applications.\n"
+                "\"uhid\" simulates a physical HID mouse using the Linux UHID "
+                "kernel module on the device.\n"
+                "\"aoa\" simulates a physical mouse using the AOAv2 protocol. "
+                "It may only work over USB.\n"
+                "In \"uhid\" and \"aoa\" modes, the computer mouse is captured "
+                "to control the device directly (relative mouse mode).\n"
+                "LAlt, LSuper or RSuper toggle the capture mode, to give "
+                "control of the mouse back to the computer.\n"
+                "Also see --keyboard.",
+    },
     {
         .shortopt = 'n',
         .longopt = "no-control",
@@ -543,10 +583,10 @@ static const struct sc_option options[] = {
                 "mirroring is disabled.\n"
                 "LAlt, LSuper or RSuper toggle the mouse capture mode, to give "
                 "control of the mouse back to the computer.\n"
-                "If any of --hid-keyboard or --hid-mouse is set, only enable "
-                "keyboard or mouse respectively, otherwise enable both.\n"
+                "Keyboard and mouse may be disabled separately using"
+                "--keyboard=disabled and --mouse=disabled.\n"
                 "It may only work over USB.\n"
-                "See --hid-keyboard and --hid-mouse.",
+                "See --keyboard and --mouse.",
     },
     {
         .shortopt = 'p',
@@ -941,6 +981,10 @@ static const struct sc_shortcut shortcuts[] = {
         .shortcuts = { "MOD+Shift+v" },
         .text = "Inject computer clipboard text as a sequence of key events",
     },
+    {
+        .shortcuts = { "MOD+k" },
+        .text = "Open keyboard settings on the device (for HID keyboard only)",
+    },
     {
         .shortcuts = { "MOD+i" },
         .text = "Enable/disable FPS counter (print frames/second in logs)",
@@ -1906,6 +1950,69 @@ parse_camera_fps(const char *s, uint16_t *camera_fps) {
     return true;
 }
 
+static bool
+parse_keyboard(const char *optarg, enum sc_keyboard_input_mode *mode) {
+    if (!strcmp(optarg, "disabled")) {
+        *mode = SC_KEYBOARD_INPUT_MODE_DISABLED;
+        return true;
+    }
+
+    if (!strcmp(optarg, "sdk")) {
+        *mode = SC_KEYBOARD_INPUT_MODE_SDK;
+        return true;
+    }
+
+    if (!strcmp(optarg, "uhid")) {
+        *mode = SC_KEYBOARD_INPUT_MODE_UHID;
+        return true;
+    }
+
+    if (!strcmp(optarg, "aoa")) {
+#ifdef HAVE_USB
+        *mode = SC_KEYBOARD_INPUT_MODE_AOA;
+        return true;
+#else
+        LOGE("--keyboard=aoa is disabled.");
+        return false;
+#endif
+    }
+
+    LOGE("Unsupported keyboard: %s (expected disabled, sdk, uhid and aoa)",
+         optarg);
+    return false;
+}
+
+static bool
+parse_mouse(const char *optarg, enum sc_mouse_input_mode *mode) {
+    if (!strcmp(optarg, "disabled")) {
+        *mode = SC_MOUSE_INPUT_MODE_DISABLED;
+        return true;
+    }
+
+    if (!strcmp(optarg, "sdk")) {
+        *mode = SC_MOUSE_INPUT_MODE_SDK;
+        return true;
+    }
+
+    if (!strcmp(optarg, "uhid")) {
+        *mode = SC_MOUSE_INPUT_MODE_UHID;
+        return true;
+    }
+
+    if (!strcmp(optarg, "aoa")) {
+#ifdef HAVE_USB
+        *mode = SC_MOUSE_INPUT_MODE_AOA;
+        return true;
+#else
+        LOGE("--mouse=aoa is disabled.");
+        return false;
+#endif
+    }
+
+    LOGE("Unsupported mouse: %s (expected disabled, sdk, uhid or aoa)", optarg);
+    return false;
+}
+
 static bool
 parse_time_limit(const char *s, sc_tick *tick) {
     long value;
@@ -1994,13 +2101,17 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
                 args->help = true;
                 break;
             case 'K':
-#ifdef HAVE_USB
-                opts->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_HID;
+                opts->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_UHID;
                 break;
-#else
-                LOGE("HID over AOA (-K/--hid-keyboard) is disabled.");
+            case OPT_KEYBOARD:
+                if (!parse_keyboard(optarg, &opts->keyboard_input_mode)) {
+                    return false;
+                }
+                break;
+            case OPT_HID_KEYBOARD_DEPRECATED:
+                LOGE("--hid-keyboard has been removed, use --keyboard=aoa or "
+                     "--keyboard=uhid instead.");
                 return false;
-#endif
             case OPT_MAX_FPS:
                 if (!parse_max_fps(optarg, &opts->max_fps)) {
                     return false;
@@ -2012,13 +2123,17 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
                 }
                 break;
             case 'M':
-#ifdef HAVE_USB
-                opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_HID;
+                opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_UHID;
                 break;
-#else
-                LOGE("HID over AOA (-M/--hid-mouse) is disabled.");
+            case OPT_MOUSE:
+                if (!parse_mouse(optarg, &opts->mouse_input_mode)) {
+                    return false;
+                }
+                break;
+            case OPT_HID_MOUSE_DEPRECATED:
+                LOGE("--hid-mouse has been removed, use --mouse=aoa or "
+                     "--mouse=uhid instead.");
                 return false;
-#endif
             case OPT_LOCK_VIDEO_ORIENTATION:
                 if (!parse_lock_video_orientation(optarg,
                         &opts->lock_video_orientation)) {
@@ -2465,6 +2580,54 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
     }
 #endif
 
+    if (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AUTO) {
+        opts->keyboard_input_mode = otg ? SC_KEYBOARD_INPUT_MODE_AOA
+                                        : SC_KEYBOARD_INPUT_MODE_SDK;
+    }
+    if (opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_AUTO) {
+        opts->mouse_input_mode = otg ? SC_MOUSE_INPUT_MODE_AOA
+                                     : SC_MOUSE_INPUT_MODE_SDK;
+    }
+
+    if (otg) {
+        enum sc_keyboard_input_mode kmode = opts->keyboard_input_mode;
+        if (kmode != SC_KEYBOARD_INPUT_MODE_AOA
+                && kmode != SC_KEYBOARD_INPUT_MODE_DISABLED) {
+            LOGE("In OTG mode, --keyboard only supports aoa or disabled.");
+            return false;
+        }
+
+        enum sc_mouse_input_mode mmode = opts->mouse_input_mode;
+        if (mmode != SC_MOUSE_INPUT_MODE_AOA
+                && mmode != SC_MOUSE_INPUT_MODE_DISABLED) {
+            LOGE("In OTG mode, --mouse only supports aoa or disabled.");
+            return false;
+        }
+
+        if (kmode == SC_KEYBOARD_INPUT_MODE_DISABLED
+                && mmode == SC_MOUSE_INPUT_MODE_DISABLED) {
+            LOGE("Could not disable both keyboard and mouse in OTG mode.");
+            return false;
+        }
+    }
+
+    if (opts->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_SDK) {
+        if (opts->key_inject_mode == SC_KEY_INJECT_MODE_TEXT) {
+            LOGE("--prefer-text is specific to --keyboard=sdk");
+            return false;
+        }
+
+        if (opts->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
+            LOGE("--raw-key-events is specific to --keyboard=sdk");
+            return false;
+        }
+
+        if (!opts->forward_key_repeat) {
+            LOGE("--no-key-repeat is specific to --keyboard=sdk");
+            return false;
+        }
+    }
+
     if ((opts->tunnel_host || opts->tunnel_port) && !opts->force_adb_forward) {
         LOGI("Tunnel host/port is set, "
              "--force-adb-forward automatically enabled.");
@@ -2625,12 +2788,12 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
     }
 
 # ifdef _WIN32
-    if (!otg && (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID
-                || opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID)) {
+    if (!otg && (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA
+                || opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA)) {
         LOGE("On Windows, it is not possible to open a USB device already open "
              "by another process (like adb).");
-        LOGE("Therefore, -K/--hid-keyboard and -M/--hid-mouse may only work in "
-             "OTG mode (--otg).");
+        LOGE("Therefore, --keyboard=aoa and --mouse=aoa may only work in OTG"
+             "mode (--otg).");
         return false;
     }
 # endif

app/src/control_msg.c

@@ -146,10 +146,22 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf) {
         case SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE:
             buf[1] = msg->set_screen_power_mode.mode;
             return 2;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE:
+            sc_write16be(&buf[1], msg->uhid_create.id);
+            sc_write16be(&buf[3], msg->uhid_create.report_desc_size);
+            memcpy(&buf[5], msg->uhid_create.report_desc,
+                            msg->uhid_create.report_desc_size);
+            return 5 + msg->uhid_create.report_desc_size;
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT:
+            sc_write16be(&buf[1], msg->uhid_input.id);
+            sc_write16be(&buf[3], msg->uhid_input.size);
+            memcpy(&buf[5], msg->uhid_input.data, msg->uhid_input.size);
+            return 5 + msg->uhid_input.size;
         case SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL:
         case SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL:
         case SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS:
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
             // no additional data
             return 1;
         default:
@@ -242,6 +254,26 @@ sc_control_msg_log(const struct sc_control_msg *msg) {
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
             LOG_CMSG("rotate device");
             break;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE:
+            LOG_CMSG("UHID create [%" PRIu16 "] report_desc_size=%" PRIu16,
+                     msg->uhid_create.id, msg->uhid_create.report_desc_size);
+            break;
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT: {
+            char *hex = sc_str_to_hex_string(msg->uhid_input.data,
+                                             msg->uhid_input.size);
+            if (hex) {
+                LOG_CMSG("UHID input [%" PRIu16 "] %s",
+                         msg->uhid_input.id, hex);
+                free(hex);
+            } else {
+                LOG_CMSG("UHID input [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_input.id, msg->uhid_input.size);
+            }
+            break;
+        }
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
+            LOG_CMSG("open hard keyboard settings");
+            break;
         default:
             LOG_CMSG("unknown type: %u", (unsigned) msg->type);
             break;

app/src/control_msg.h

@@ -10,6 +10,7 @@
 #include "android/input.h"
 #include "android/keycodes.h"
 #include "coords.h"
+#include "hid/hid_event.h"
 
 #define SC_CONTROL_MSG_MAX_SIZE (1 << 18) // 256k
 
@@ -37,6 +38,9 @@ enum sc_control_msg_type {
     SC_CONTROL_MSG_TYPE_SET_CLIPBOARD,
     SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE,
     SC_CONTROL_MSG_TYPE_ROTATE_DEVICE,
+    SC_CONTROL_MSG_TYPE_UHID_CREATE,
+    SC_CONTROL_MSG_TYPE_UHID_INPUT,
+    SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
 };
 
 enum sc_screen_power_mode {
@@ -92,6 +96,16 @@ struct sc_control_msg {
         struct {
             enum sc_screen_power_mode mode;
         } set_screen_power_mode;
+        struct {
+            uint16_t id;
+            uint16_t report_desc_size;
+            const uint8_t *report_desc; // pointer to static data
+        } uhid_create;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t data[SC_HID_MAX_SIZE];
+        } uhid_input;
     };
 };
 

app/src/controller.c

@@ -7,8 +7,7 @@
 #define SC_CONTROL_MSG_QUEUE_MAX 64
 
 bool
-sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync) {
+sc_controller_init(struct sc_controller *controller, sc_socket control_socket) {
     sc_vecdeque_init(&controller->queue);
 
     bool ok = sc_vecdeque_reserve(&controller->queue, SC_CONTROL_MSG_QUEUE_MAX);
@@ -16,7 +15,7 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
         return false;
     }
 
-    ok = sc_receiver_init(&controller->receiver, control_socket, acksync);
+    ok = sc_receiver_init(&controller->receiver, control_socket);
     if (!ok) {
         sc_vecdeque_destroy(&controller->queue);
         return false;
@@ -43,6 +42,14 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
     return true;
 }
 
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices) {
+    controller->receiver.acksync = acksync;
+    controller->receiver.uhid_devices = uhid_devices;
+}
+
 void
 sc_controller_destroy(struct sc_controller *controller) {
     sc_cond_destroy(&controller->msg_cond);

app/src/controller.h

@@ -25,8 +25,12 @@ struct sc_controller {
 };
 
 bool
-sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync);
+sc_controller_init(struct sc_controller *controller, sc_socket control_socket);
+
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices);
 
 void
 sc_controller_destroy(struct sc_controller *controller);

app/src/device_msg.c

@@ -46,6 +46,31 @@ sc_device_msg_deserialize(const uint8_t *buf, size_t len,
             msg->ack_clipboard.sequence = sequence;
             return 9;
         }
+        case DEVICE_MSG_TYPE_UHID_OUTPUT: {
+            if (len < 5) {
+                // at least id + size
+                return 0; // not available
+            }
+            uint16_t id = sc_read16be(&buf[1]);
+            size_t size = sc_read16be(&buf[3]);
+            if (size < len - 5) {
+                return 0; // not available
+            }
+            uint8_t *data = malloc(size);
+            if (!data) {
+                LOG_OOM();
+                return -1;
+            }
+            if (size) {
+                memcpy(data, &buf[5], size);
+            }
+
+            msg->uhid_output.id = id;
+            msg->uhid_output.size = size;
+            msg->uhid_output.data = data;
+
+            return 5 + size;
+        }
         default:
             LOGW("Unknown device message type: %d", (int) msg->type);
             return -1; // error, we cannot recover
@@ -54,7 +79,15 @@ sc_device_msg_deserialize(const uint8_t *buf, size_t len,
 
 void
 sc_device_msg_destroy(struct sc_device_msg *msg) {
-    if (msg->type == DEVICE_MSG_TYPE_CLIPBOARD) {
-        free(msg->clipboard.text);
+    switch (msg->type) {
+        case DEVICE_MSG_TYPE_CLIPBOARD:
+            free(msg->clipboard.text);
+            break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            free(msg->uhid_output.data);
+            break;
+        default:
+            // nothing to do
+            break;
     }
 }

app/src/device_msg.h

@@ -14,6 +14,7 @@
 enum sc_device_msg_type {
     DEVICE_MSG_TYPE_CLIPBOARD,
     DEVICE_MSG_TYPE_ACK_CLIPBOARD,
+    DEVICE_MSG_TYPE_UHID_OUTPUT,
 };
 
 struct sc_device_msg {
@@ -25,6 +26,11 @@ struct sc_device_msg {
         struct {
             uint64_t sequence;
         } ack_clipboard;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t *data; // owned, to be freed by free()
+        } uhid_output;
     };
 };
 

app/src/display.c

@@ -62,6 +62,7 @@ sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
         LOGD("Trilinear filtering disabled (not an OpenGL renderer");
     }
 
+    display->texture = NULL;
     display->pending.flags = 0;
     display->pending.frame = NULL;
 

app/src/hid/hid_event.h

@@ -0,0 +1,15 @@
+#ifndef SC_HID_EVENT_H
+#define SC_HID_EVENT_H
+
+#include "common.h"
+
+#include <stdint.h>
+
+#define SC_HID_MAX_SIZE 8
+
+struct sc_hid_event {
+    uint8_t data[SC_HID_MAX_SIZE];
+    uint8_t size;
+};
+
+#endif

app/src/hid/hid_keyboard.c

@@ -1,40 +1,34 @@
 #include "hid_keyboard.h"
 
-#include <assert.h>
+#include <string.h>
 
-#include "input_events.h"
 #include "util/log.h"
 
-/** Downcast key processor to hid_keyboard */
-#define DOWNCAST(KP) container_of(KP, struct sc_hid_keyboard, key_processor)
+#define SC_HID_MOD_NONE 0x00
+#define SC_HID_MOD_LEFT_CONTROL (1 << 0)
+#define SC_HID_MOD_LEFT_SHIFT (1 << 1)
+#define SC_HID_MOD_LEFT_ALT (1 << 2)
+#define SC_HID_MOD_LEFT_GUI (1 << 3)
+#define SC_HID_MOD_RIGHT_CONTROL (1 << 4)
+#define SC_HID_MOD_RIGHT_SHIFT (1 << 5)
+#define SC_HID_MOD_RIGHT_ALT (1 << 6)
+#define SC_HID_MOD_RIGHT_GUI (1 << 7)
 
-#define HID_KEYBOARD_ACCESSORY_ID 1
-
-#define HID_MODIFIER_NONE 0x00
-#define HID_MODIFIER_LEFT_CONTROL (1 << 0)
-#define HID_MODIFIER_LEFT_SHIFT (1 << 1)
-#define HID_MODIFIER_LEFT_ALT (1 << 2)
-#define HID_MODIFIER_LEFT_GUI (1 << 3)
-#define HID_MODIFIER_RIGHT_CONTROL (1 << 4)
-#define HID_MODIFIER_RIGHT_SHIFT (1 << 5)
-#define HID_MODIFIER_RIGHT_ALT (1 << 6)
-#define HID_MODIFIER_RIGHT_GUI (1 << 7)
-
-#define HID_KEYBOARD_INDEX_MODIFIER 0
-#define HID_KEYBOARD_INDEX_KEYS 2
+#define SC_HID_KEYBOARD_INDEX_MODS 0
+#define SC_HID_KEYBOARD_INDEX_KEYS 2
 
 // USB HID protocol says 6 keys in an event is the requirement for BIOS
 // keyboard support, though OS could support more keys via modifying the report
 // desc. 6 should be enough for scrcpy.
-#define HID_KEYBOARD_MAX_KEYS 6
-#define HID_KEYBOARD_EVENT_SIZE \
-    (HID_KEYBOARD_INDEX_KEYS + HID_KEYBOARD_MAX_KEYS)
+#define SC_HID_KEYBOARD_MAX_KEYS 6
+#define SC_HID_KEYBOARD_EVENT_SIZE \
+    (SC_HID_KEYBOARD_INDEX_KEYS + SC_HID_KEYBOARD_MAX_KEYS)
 
-#define HID_RESERVED 0x00
-#define HID_ERROR_ROLL_OVER 0x01
+#define SC_HID_RESERVED 0x00
+#define SC_HID_ERROR_ROLL_OVER 0x01
 
 /**
- * For HID over AOAv2, only report descriptor is needed.
+ * For HID, only report descriptor is needed.
  *
  * The specification is available here:
  * <https://www.usb.org/sites/default/files/hid1_11.pdf>
@@ -53,7 +47,7 @@
  *
  * (change vid:pid' to your device's vendor ID and product ID).
  */
-static const unsigned char keyboard_report_desc[]  = {
+const uint8_t SC_HID_KEYBOARD_REPORT_DESC[] = {
     // Usage Page (Generic Desktop)
     0x05, 0x01,
     // Usage (Keyboard)
@@ -119,7 +113,7 @@ static const unsigned char keyboard_report_desc[]  = {
     // Report Size (8)
     0x75, 0x08,
     // Report Count (6)
-    0x95, HID_KEYBOARD_MAX_KEYS,
+    0x95, SC_HID_KEYBOARD_MAX_KEYS,
     // Input (Data, Array): Keys
     0x81, 0x00,
 
@@ -127,6 +121,9 @@ static const unsigned char keyboard_report_desc[]  = {
     0xC0
 };
 
+const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN =
+    sizeof(SC_HID_KEYBOARD_REPORT_DESC);
+
 /**
  * A keyboard HID event is 8 bytes long:
  *
@@ -201,51 +198,50 @@ static const unsigned char keyboard_report_desc[]  = {
  *                  +---------------+
  */
 
-static unsigned char
-sdl_keymod_to_hid_modifiers(uint16_t mod) {
-    unsigned char modifiers = HID_MODIFIER_NONE;
-    if (mod & SC_MOD_LCTRL) {
-        modifiers |= HID_MODIFIER_LEFT_CONTROL;
-    }
-    if (mod & SC_MOD_LSHIFT) {
-        modifiers |= HID_MODIFIER_LEFT_SHIFT;
-    }
-    if (mod & SC_MOD_LALT) {
-        modifiers |= HID_MODIFIER_LEFT_ALT;
-    }
-    if (mod & SC_MOD_LGUI) {
-        modifiers |= HID_MODIFIER_LEFT_GUI;
-    }
-    if (mod & SC_MOD_RCTRL) {
-        modifiers |= HID_MODIFIER_RIGHT_CONTROL;
-    }
-    if (mod & SC_MOD_RSHIFT) {
-        modifiers |= HID_MODIFIER_RIGHT_SHIFT;
-    }
-    if (mod & SC_MOD_RALT) {
-        modifiers |= HID_MODIFIER_RIGHT_ALT;
-    }
-    if (mod & SC_MOD_RGUI) {
-        modifiers |= HID_MODIFIER_RIGHT_GUI;
-    }
-    return modifiers;
+static void
+sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
+    hid_event->size = SC_HID_KEYBOARD_EVENT_SIZE;
+
+    uint8_t *data = hid_event->data;
+
+    data[SC_HID_KEYBOARD_INDEX_MODS] = SC_HID_MOD_NONE;
+    data[1] = SC_HID_RESERVED;
+    memset(&data[SC_HID_KEYBOARD_INDEX_KEYS], 0, SC_HID_KEYBOARD_MAX_KEYS);
 }
 
-static bool
-sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
-    unsigned char *buffer = malloc(HID_KEYBOARD_EVENT_SIZE);
-    if (!buffer) {
-        LOG_OOM();
-        return false;
+static uint16_t
+sc_hid_mod_from_sdl_keymod(uint16_t mod) {
+    uint16_t mods = SC_HID_MOD_NONE;
+    if (mod & SC_MOD_LCTRL) {
+        mods |= SC_HID_MOD_LEFT_CONTROL;
     }
+    if (mod & SC_MOD_LSHIFT) {
+        mods |= SC_HID_MOD_LEFT_SHIFT;
+    }
+    if (mod & SC_MOD_LALT) {
+        mods |= SC_HID_MOD_LEFT_ALT;
+    }
+    if (mod & SC_MOD_LGUI) {
+        mods |= SC_HID_MOD_LEFT_GUI;
+    }
+    if (mod & SC_MOD_RCTRL) {
+        mods |= SC_HID_MOD_RIGHT_CONTROL;
+    }
+    if (mod & SC_MOD_RSHIFT) {
+        mods |= SC_HID_MOD_RIGHT_SHIFT;
+    }
+    if (mod & SC_MOD_RALT) {
+        mods |= SC_HID_MOD_RIGHT_ALT;
+    }
+    if (mod & SC_MOD_RGUI) {
+        mods |= SC_HID_MOD_RIGHT_GUI;
+    }
+    return mods;
+}
 
-    buffer[HID_KEYBOARD_INDEX_MODIFIER] = HID_MODIFIER_NONE;
-    buffer[1] = HID_RESERVED;
-    memset(&buffer[HID_KEYBOARD_INDEX_KEYS], 0, HID_KEYBOARD_MAX_KEYS);
-
-    sc_hid_event_init(hid_event, HID_KEYBOARD_ACCESSORY_ID, buffer,
-                      HID_KEYBOARD_EVENT_SIZE);
-    return true;
+void
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid) {
+    memset(hid->keys, false, SC_HID_KEYBOARD_KEYS);
 }
 
 static inline bool
@@ -253,10 +249,10 @@ scancode_is_modifier(enum sc_scancode scancode) {
     return scancode >= SC_SCANCODE_LCTRL && scancode <= SC_SCANCODE_RGUI;
 }
 
-static bool
-convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
-                           struct sc_hid_event *hid_event,
-                           const struct sc_key_event *event) {
+bool
+sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
+                               struct sc_hid_event *hid_event,
+                               const struct sc_key_event *event) {
     enum sc_scancode scancode = event->scancode;
     assert(scancode >= 0);
 
@@ -268,39 +264,37 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
         return false;
     }
 
-    if (!sc_hid_keyboard_event_init(hid_event)) {
-        LOGW("Could not initialize HID keyboard event");
-        return false;
-    }
+    sc_hid_keyboard_event_init(hid_event);
 
-    unsigned char modifiers = sdl_keymod_to_hid_modifiers(event->mods_state);
+    uint16_t mods = sc_hid_mod_from_sdl_keymod(event->mods_state);
 
     if (scancode < SC_HID_KEYBOARD_KEYS) {
         // Pressed is true and released is false
-        kb->keys[scancode] = (event->action == SC_ACTION_DOWN);
+        hid->keys[scancode] = (event->action == SC_ACTION_DOWN);
         LOGV("keys[%02x] = %s", scancode,
-             kb->keys[scancode] ? "true" : "false");
+             hid->keys[scancode] ? "true" : "false");
     }
 
-    hid_event->buffer[HID_KEYBOARD_INDEX_MODIFIER] = modifiers;
+    hid_event->data[SC_HID_KEYBOARD_INDEX_MODS] = mods;
 
-    unsigned char *keys_buffer = &hid_event->buffer[HID_KEYBOARD_INDEX_KEYS];
+    uint8_t *keys_data = &hid_event->data[SC_HID_KEYBOARD_INDEX_KEYS];
     // Re-calculate pressed keys every time
     int keys_pressed_count = 0;
     for (int i = 0; i < SC_HID_KEYBOARD_KEYS; ++i) {
-        if (kb->keys[i]) {
+        if (hid->keys[i]) {
             // USB HID protocol says that if keys exceeds report count, a
             // phantom state should be reported
-            if (keys_pressed_count >= HID_KEYBOARD_MAX_KEYS) {
+            if (keys_pressed_count >= SC_HID_KEYBOARD_MAX_KEYS) {
                 // Phantom state:
                 //  - Modifiers
                 //  - Reserved
                 //  - ErrorRollOver * HID_MAX_KEYS
-                memset(keys_buffer, HID_ERROR_ROLL_OVER, HID_KEYBOARD_MAX_KEYS);
+                memset(keys_data, SC_HID_ERROR_ROLL_OVER,
+                       SC_HID_KEYBOARD_MAX_KEYS);
                 goto end;
             }
 
-            keys_buffer[keys_pressed_count] = i;
+            keys_data[keys_pressed_count] = i;
             ++keys_pressed_count;
         }
     }
@@ -308,124 +302,32 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
 end:
     LOGV("hid keyboard: key %-4s scancode=%02x (%u) mod=%02x",
          event->action == SC_ACTION_DOWN ? "down" : "up", event->scancode,
-         event->scancode, modifiers);
+         event->scancode, mods);
 
     return true;
 }
 
-
-static bool
-push_mod_lock_state(struct sc_hid_keyboard *kb, uint16_t mods_state) {
+bool
+sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
+                                uint16_t mods_state) {
     bool capslock = mods_state & SC_MOD_CAPS;
     bool numlock = mods_state & SC_MOD_NUM;
     if (!capslock && !numlock) {
         // Nothing to do
-        return true;
-    }
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_keyboard_event_init(&hid_event)) {
-        LOGW("Could not initialize HID keyboard event");
         return false;
     }
 
+    sc_hid_keyboard_event_init(event);
+
     unsigned i = 0;
     if (capslock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
+        event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
         ++i;
     }
     if (numlock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
+        event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
         ++i;
     }
 
-    if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mod lock state)");
-        return false;
-    }
-
-    LOGD("HID keyboard state synchronized");
-
     return true;
 }
-
-static void
-sc_key_processor_process_key(struct sc_key_processor *kp,
-                             const struct sc_key_event *event,
-                             uint64_t ack_to_wait) {
-    if (event->repeat) {
-        // In USB HID protocol, key repeat is handled by the host (Android), so
-        // just ignore key repeat here.
-        return;
-    }
-
-    struct sc_hid_keyboard *kb = DOWNCAST(kp);
-
-    struct sc_hid_event hid_event;
-    // Not all keys are supported, just ignore unsupported keys
-    if (convert_hid_keyboard_event(kb, &hid_event, event)) {
-        if (!kb->mod_lock_synchronized) {
-            // Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
-            // keyboard state
-            if (push_mod_lock_state(kb, event->mods_state)) {
-                kb->mod_lock_synchronized = true;
-            }
-        }
-
-        if (ack_to_wait) {
-            // Ctrl+v is pressed, so clipboard synchronization has been
-            // requested. Wait until clipboard synchronization is acknowledged
-            // by the server, otherwise it could paste the old clipboard
-            // content.
-            hid_event.ack_to_wait = ack_to_wait;
-        }
-
-        if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-            sc_hid_event_destroy(&hid_event);
-            LOGW("Could not request HID event (key)");
-        }
-    }
-}
-
-bool
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa) {
-    kb->aoa = aoa;
-
-    bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
-                               keyboard_report_desc,
-                               ARRAY_LEN(keyboard_report_desc));
-    if (!ok) {
-        LOGW("Register HID keyboard failed");
-        return false;
-    }
-
-    // Reset all states
-    memset(kb->keys, false, SC_HID_KEYBOARD_KEYS);
-
-    kb->mod_lock_synchronized = false;
-
-    static const struct sc_key_processor_ops ops = {
-        .process_key = sc_key_processor_process_key,
-        // Never forward text input via HID (all the keys are injected
-        // separately)
-        .process_text = NULL,
-    };
-
-    // Clipboard synchronization is requested over the control socket, while HID
-    // events are sent over AOA, so it must wait for clipboard synchronization
-    // to be acknowledged by the device before injecting Ctrl+v.
-    kb->key_processor.async_paste = true;
-    kb->key_processor.ops = &ops;
-
-    return true;
-}
-
-void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb) {
-    // Unregister HID keyboard so the soft keyboard shows again on Android
-    bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
-    if (!ok) {
-        LOGW("Could not unregister HID keyboard");
-    }
-}

app/src/hid/hid_keyboard.h

@@ -5,8 +5,8 @@
 
 #include <stdbool.h>
 
-#include "aoa_hid.h"
-#include "trait/key_processor.h"
+#include "hid/hid_event.h"
+#include "input_events.h"
 
 // See "SDL2/SDL_scancode.h".
 // Maybe SDL_Keycode is used by most people, but SDL_Scancode is taken from USB
@@ -14,6 +14,9 @@
 // 0x65 is Application, typically AT-101 Keyboard ends here.
 #define SC_HID_KEYBOARD_KEYS 0x66
 
+extern const uint8_t SC_HID_KEYBOARD_REPORT_DESC[];
+extern const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN;
+
 /**
  * HID keyboard events are sequence-based, every time keyboard state changes
  * it sends an array of currently pressed keys, the host is responsible for
@@ -27,18 +30,19 @@
  * phantom state.
  */
 struct sc_hid_keyboard {
-    struct sc_key_processor key_processor; // key processor trait
-
-    struct sc_aoa *aoa;
     bool keys[SC_HID_KEYBOARD_KEYS];
-
-    bool mod_lock_synchronized;
 };
 
-bool
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa);
-
 void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb);
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid);
+
+bool
+sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
+                               struct sc_hid_event *hid_event,
+                               const struct sc_key_event *event);
+
+bool
+sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
+                                uint16_t mods_state);
 
 #endif

app/src/hid/hid_mouse.c

@@ -0,0 +1,192 @@
+#include "hid_mouse.h"
+
+// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position,
+// 1 byte for wheel motion
+#define HID_MOUSE_EVENT_SIZE 4
+
+/**
+ * Mouse descriptor from the specification:
+ * <https://www.usb.org/sites/default/files/hid1_11.pdf>
+ *
+ * Appendix E (p71): §E.10 Report Descriptor (Mouse)
+ *
+ * The usage tags (like Wheel) are listed in "HID Usage Tables":
+ * <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
+ * §4 Generic Desktop Page (0x01) (p26)
+ */
+const uint8_t SC_HID_MOUSE_REPORT_DESC[] = {
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (Mouse)
+    0x09, 0x02,
+
+    // Collection (Application)
+    0xA1, 0x01,
+
+    // Usage (Pointer)
+    0x09, 0x01,
+
+    // Collection (Physical)
+    0xA1, 0x00,
+
+    // Usage Page (Buttons)
+    0x05, 0x09,
+
+    // Usage Minimum (1)
+    0x19, 0x01,
+     // Usage Maximum (5)
+    0x29, 0x05,
+    // Logical Minimum (0)
+    0x15, 0x00,
+    // Logical Maximum (1)
+    0x25, 0x01,
+    // Report Count (5)
+    0x95, 0x05,
+    // Report Size (1)
+    0x75, 0x01,
+    // Input (Data, Variable, Absolute): 5 buttons bits
+    0x81, 0x02,
+
+    // Report Count (1)
+    0x95, 0x01,
+    // Report Size (3)
+    0x75, 0x03,
+    // Input (Constant): 3 bits padding
+    0x81, 0x01,
+
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (X)
+    0x09, 0x30,
+    // Usage (Y)
+    0x09, 0x31,
+    // Usage (Wheel)
+    0x09, 0x38,
+    // Local Minimum (-127)
+    0x15, 0x81,
+    // Local Maximum (127)
+    0x25, 0x7F,
+    // Report Size (8)
+    0x75, 0x08,
+    // Report Count (3)
+    0x95, 0x03,
+    // Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
+    0x81, 0x06,
+
+    // End Collection
+    0xC0,
+
+    // End Collection
+    0xC0,
+};
+
+const size_t SC_HID_MOUSE_REPORT_DESC_LEN =
+    sizeof(SC_HID_MOUSE_REPORT_DESC);
+
+/**
+ * A mouse HID event is 4 bytes long:
+ *
+ *  - byte 0: buttons state
+ *  - byte 1: relative x motion (signed byte from -127 to 127)
+ *  - byte 2: relative y motion (signed byte from -127 to 127)
+ *  - byte 3: wheel motion (-1, 0 or 1)
+ *
+ *                   7 6 5 4 3 2 1 0
+ *                  +---------------+
+ *         byte 0:  |0 0 0 . . . . .| buttons state
+ *                  +---------------+
+ *                         ^ ^ ^ ^ ^
+ *                         | | | | `- left button
+ *                         | | | `--- right button
+ *                         | | `----- middle button
+ *                         | `------- button 4
+ *                         `--------- button 5
+ *
+ *                  +---------------+
+ *         byte 1:  |. . . . . . . .| relative x motion
+ *                  +---------------+
+ *         byte 2:  |. . . . . . . .| relative y motion
+ *                  +---------------+
+ *         byte 3:  |. . . . . . . .| wheel motion
+ *                  +---------------+
+ *
+ * As an example, here is the report for a motion of (x=5, y=-4) with left
+ * button pressed:
+ *
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 1| left button pressed
+ *                  +---------------+
+ *                  |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
+ *                  +---------------+
+ *                  |1 1 1 1 1 1 0 0| relative y motion (y = -4)
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 0| wheel motion
+ *                  +---------------+
+ */
+
+static void
+sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
+    hid_event->size = HID_MOUSE_EVENT_SIZE;
+    // Leave hid_event->data uninitialized, it will be fully initialized by
+    // callers
+}
+
+static uint8_t
+sc_hid_buttons_from_buttons_state(uint8_t buttons_state) {
+    uint8_t c = 0;
+    if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
+        c |= 1 << 0;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
+        c |= 1 << 1;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
+        c |= 1 << 2;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X1) {
+        c |= 1 << 3;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X2) {
+        c |= 1 << 4;
+    }
+    return c;
+}
+
+void
+sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_motion_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = CLAMP(event->xrel, -127, 127);
+    data[2] = CLAMP(event->yrel, -127, 127);
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
+                              const struct sc_mouse_click_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_scroll_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = 0; // buttons state irrelevant (and unknown)
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    // In practice, vscroll is always -1, 0 or 1, but in theory other values
+    // are possible
+    data[3] = CLAMP(event->vscroll, -127, 127);
+    // Horizontal scrolling ignored
+}

app/src/hid/hid_mouse.h

@@ -0,0 +1,26 @@
+#ifndef SC_HID_MOUSE_H
+#define SC_HID_MOUSE_H
+
+#endif
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "hid/hid_event.h"
+#include "input_events.h"
+
+extern const uint8_t SC_HID_MOUSE_REPORT_DESC[];
+extern const size_t SC_HID_MOUSE_REPORT_DESC_LEN;
+
+void
+sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_motion_event *event);
+
+void
+sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
+                              const struct sc_mouse_click_event *event);
+
+void
+sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_scroll_event *event);

app/src/input_manager.c

@@ -52,8 +52,11 @@ is_shortcut_mod(struct sc_input_manager *im, uint16_t sdl_mod) {
 void
 sc_input_manager_init(struct sc_input_manager *im,
                       const struct sc_input_manager_params *params) {
-    assert(!params->controller || (params->kp && params->kp->ops));
-    assert(!params->controller || (params->mp && params->mp->ops));
+    // A key/mouse processor may not be present if there is no controller
+    assert((!params->kp && !params->mp) || params->controller);
+    // A processor must have ops initialized
+    assert(!params->kp || params->kp->ops);
+    assert(!params->mp || params->mp->ops);
 
     im->controller = params->controller;
     im->fp = params->fp;
@@ -87,8 +90,10 @@ sc_input_manager_init(struct sc_input_manager *im,
 }
 
 static void
-send_keycode(struct sc_controller *controller, enum android_keycode keycode,
+send_keycode(struct sc_input_manager *im, enum android_keycode keycode,
              enum sc_action action, const char *name) {
+    assert(im->controller && im->kp);
+
     // send DOWN event
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_INJECT_KEYCODE;
@@ -99,100 +104,109 @@ send_keycode(struct sc_controller *controller, enum android_keycode keycode,
     msg.inject_keycode.metastate = 0;
     msg.inject_keycode.repeat = 0;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'inject %s'", name);
     }
 }
 
 static inline void
-action_home(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_HOME, action, "HOME");
+action_home(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_HOME, action, "HOME");
 }
 
 static inline void
-action_back(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_BACK, action, "BACK");
+action_back(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_BACK, action, "BACK");
 }
 
 static inline void
-action_app_switch(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
+action_app_switch(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
 }
 
 static inline void
-action_power(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_POWER, action, "POWER");
+action_power(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_POWER, action, "POWER");
 }
 
 static inline void
-action_volume_up(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
+action_volume_up(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
 }
 
 static inline void
-action_volume_down(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
+action_volume_down(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
 }
 
 static inline void
-action_menu(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_MENU, action, "MENU");
+action_menu(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_MENU, action, "MENU");
 }
 
 // turn the screen on if it was off, press BACK otherwise
 // If the screen is off, it is turned on only on ACTION_DOWN
 static void
-press_back_or_turn_screen_on(struct sc_controller *controller,
+press_back_or_turn_screen_on(struct sc_input_manager *im,
                              enum sc_action action) {
+    assert(im->controller && im->kp);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_BACK_OR_SCREEN_ON;
     msg.back_or_screen_on.action = action == SC_ACTION_DOWN
                                  ? AKEY_EVENT_ACTION_DOWN
                                  : AKEY_EVENT_ACTION_UP;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'press back or turn screen on'");
     }
 }
 
 static void
-expand_notification_panel(struct sc_controller *controller) {
+expand_notification_panel(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'expand notification panel'");
     }
 }
 
 static void
-expand_settings_panel(struct sc_controller *controller) {
+expand_settings_panel(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'expand settings panel'");
     }
 }
 
 static void
-collapse_panels(struct sc_controller *controller) {
+collapse_panels(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'collapse notification panel'");
     }
 }
 
 static bool
-get_device_clipboard(struct sc_controller *controller,
-                     enum sc_copy_key copy_key) {
+get_device_clipboard(struct sc_input_manager *im, enum sc_copy_key copy_key) {
+    assert(im->controller && im->kp);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_GET_CLIPBOARD;
     msg.get_clipboard.copy_key = copy_key;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'get device clipboard'");
         return false;
     }
@@ -201,8 +215,10 @@ get_device_clipboard(struct sc_controller *controller,
 }
 
 static bool
-set_device_clipboard(struct sc_controller *controller, bool paste,
+set_device_clipboard(struct sc_input_manager *im, bool paste,
                      uint64_t sequence) {
+    assert(im->controller && im->kp);
+
     char *text = SDL_GetClipboardText();
     if (!text) {
         LOGW("Could not get clipboard text: %s", SDL_GetError());
@@ -222,7 +238,7 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
     msg.set_clipboard.text = text_dup;
     msg.set_clipboard.paste = paste;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         free(text_dup);
         LOGW("Could not request 'set device clipboard'");
         return false;
@@ -232,19 +248,23 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
 }
 
 static void
-set_screen_power_mode(struct sc_controller *controller,
+set_screen_power_mode(struct sc_input_manager *im,
                       enum sc_screen_power_mode mode) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
     msg.set_screen_power_mode.mode = mode;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'set screen power mode'");
     }
 }
 
 static void
-switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
+switch_fps_counter_state(struct sc_input_manager *im) {
+    struct sc_fps_counter *fps_counter = &im->screen->fps_counter;
+
     // the started state can only be written from the current thread, so there
     // is no ToCToU issue
     if (sc_fps_counter_is_started(fps_counter)) {
@@ -256,7 +276,9 @@ switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
 }
 
 static void
-clipboard_paste(struct sc_controller *controller) {
+clipboard_paste(struct sc_input_manager *im) {
+    assert(im->controller && im->kp);
+
     char *text = SDL_GetClipboardText();
     if (!text) {
         LOGW("Could not get clipboard text: %s", SDL_GetError());
@@ -278,25 +300,40 @@ clipboard_paste(struct sc_controller *controller) {
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_INJECT_TEXT;
     msg.inject_text.text = text_dup;
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         free(text_dup);
         LOGW("Could not request 'paste clipboard'");
     }
 }
 
 static void
-rotate_device(struct sc_controller *controller) {
+rotate_device(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_ROTATE_DEVICE;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request device rotation");
     }
 }
 
 static void
-apply_orientation_transform(struct sc_screen *screen,
+open_hard_keyboard_settings(struct sc_input_manager *im) {
+    assert(im->controller);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS;
+
+    if (!sc_controller_push_msg(im->controller, &msg)) {
+        LOGW("Could not request opening hard keyboard settings");
+    }
+}
+
+static void
+apply_orientation_transform(struct sc_input_manager *im,
                             enum sc_orientation transform) {
+    struct sc_screen *screen = im->screen;
     enum sc_orientation new_orientation =
         sc_orientation_apply(screen->orientation, transform);
     sc_screen_set_orientation(screen, new_orientation);
@@ -364,7 +401,7 @@ static void
 sc_input_manager_process_key(struct sc_input_manager *im,
                              const SDL_KeyboardEvent *event) {
     // controller is NULL if --no-control is requested
-    struct sc_controller *controller = im->controller;
+    bool control = im->controller;
 
     SDL_Keycode keycode = event->keysym.sym;
     uint16_t mod = event->keysym.mod;
@@ -390,68 +427,68 @@ sc_input_manager_process_key(struct sc_input_manager *im,
         enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
         switch (keycode) {
             case SDLK_h:
-                if (controller && !shift && !repeat) {
-                    action_home(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_home(im, action);
                 }
                 return;
             case SDLK_b: // fall-through
             case SDLK_BACKSPACE:
-                if (controller && !shift && !repeat) {
-                    action_back(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_back(im, action);
                 }
                 return;
             case SDLK_s:
-                if (controller && !shift && !repeat) {
-                    action_app_switch(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_app_switch(im, action);
                 }
                 return;
             case SDLK_m:
-                if (controller && !shift && !repeat) {
-                    action_menu(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_menu(im, action);
                 }
                 return;
             case SDLK_p:
-                if (controller && !shift && !repeat) {
-                    action_power(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_power(im, action);
                 }
                 return;
             case SDLK_o:
-                if (controller && !repeat && down) {
+                if (control && !repeat && down) {
                     enum sc_screen_power_mode mode = shift
                                                    ? SC_SCREEN_POWER_MODE_NORMAL
                                                    : SC_SCREEN_POWER_MODE_OFF;
-                    set_screen_power_mode(controller, mode);
+                    set_screen_power_mode(im, mode);
                 }
                 return;
             case SDLK_DOWN:
                 if (shift) {
                     if (!repeat & down) {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_180);
                     }
-                } else if (controller) {
+                } else if (im->kp) {
                     // forward repeated events
-                    action_volume_down(controller, action);
+                    action_volume_down(im, action);
                 }
                 return;
             case SDLK_UP:
                 if (shift) {
                     if (!repeat & down) {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_180);
                     }
-                } else if (controller) {
+                } else if (im->kp) {
                     // forward repeated events
-                    action_volume_up(controller, action);
+                    action_volume_up(im, action);
                 }
                 return;
             case SDLK_LEFT:
                 if (!repeat && down) {
                     if (shift) {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_0);
                     } else {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_270);
                     }
                 }
@@ -459,34 +496,33 @@ sc_input_manager_process_key(struct sc_input_manager *im,
             case SDLK_RIGHT:
                 if (!repeat && down) {
                     if (shift) {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_0);
                     } else {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_90);
                     }
                 }
                 return;
             case SDLK_c:
-                if (controller && !shift && !repeat && down) {
-                    get_device_clipboard(controller, SC_COPY_KEY_COPY);
+                if (im->kp && !shift && !repeat && down) {
+                    get_device_clipboard(im, SC_COPY_KEY_COPY);
                 }
                 return;
             case SDLK_x:
-                if (controller && !shift && !repeat && down) {
-                    get_device_clipboard(controller, SC_COPY_KEY_CUT);
+                if (im->kp && !shift && !repeat && down) {
+                    get_device_clipboard(im, SC_COPY_KEY_CUT);
                 }
                 return;
             case SDLK_v:
-                if (controller && !repeat && down) {
+                if (im->kp && !repeat && down) {
                     if (shift || im->legacy_paste) {
                         // inject the text as input events
-                        clipboard_paste(controller);
+                        clipboard_paste(im);
                     } else {
                         // store the text in the device clipboard and paste,
                         // without requesting an acknowledgment
-                        set_device_clipboard(controller, true,
-                                             SC_SEQUENCE_INVALID);
+                        set_device_clipboard(im, true, SC_SEQUENCE_INVALID);
                     }
                 }
                 return;
@@ -507,23 +543,30 @@ sc_input_manager_process_key(struct sc_input_manager *im,
                 return;
             case SDLK_i:
                 if (!shift && !repeat && down) {
-                    switch_fps_counter_state(&im->screen->fps_counter);
+                    switch_fps_counter_state(im);
                 }
                 return;
             case SDLK_n:
-                if (controller && !repeat && down) {
+                if (control && !repeat && down) {
                     if (shift) {
-                        collapse_panels(controller);
+                        collapse_panels(im);
                     } else if (im->key_repeat == 0) {
-                        expand_notification_panel(controller);
+                        expand_notification_panel(im);
                     } else {
-                        expand_settings_panel(controller);
+                        expand_settings_panel(im);
                     }
                 }
                 return;
             case SDLK_r:
-                if (controller && !shift && !repeat && down) {
-                    rotate_device(controller);
+                if (control && !shift && !repeat && down) {
+                    rotate_device(im);
+                }
+                return;
+            case SDLK_k:
+                if (control && !shift && !repeat && down
+                        && im->kp && im->kp->hid) {
+                    // Only if the current keyboard is hid
+                    open_hard_keyboard_settings(im);
                 }
                 return;
         }
@@ -531,7 +574,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
         return;
     }
 
-    if (!controller) {
+    if (!im->kp) {
         return;
     }
 
@@ -540,7 +583,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
     if (im->clipboard_autosync && is_ctrl_v) {
         if (im->legacy_paste) {
             // inject the text as input events
-            clipboard_paste(controller);
+            clipboard_paste(im);
             return;
         }
 
@@ -550,7 +593,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
 
         // Synchronize the computer clipboard to the device clipboard before
         // sending Ctrl+v, to allow seamless copy-paste.
-        bool ok = set_device_clipboard(controller, false, sequence);
+        bool ok = set_device_clipboard(im, false, sequence);
         if (!ok) {
             LOGW("Clipboard could not be synchronized, Ctrl+v not injected");
             return;
@@ -652,7 +695,7 @@ sc_input_manager_process_touch(struct sc_input_manager *im,
 static void
 sc_input_manager_process_mouse_button(struct sc_input_manager *im,
                                       const SDL_MouseButtonEvent *event) {
-    struct sc_controller *controller = im->controller;
+    bool control = im->controller;
 
     if (event->which == SDL_TOUCH_MOUSEID) {
         // simulated from touch events, so it's a duplicate
@@ -661,27 +704,27 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
 
     bool down = event->type == SDL_MOUSEBUTTONDOWN;
     if (!im->forward_all_clicks) {
-        if (controller) {
+        if (control) {
             enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
 
-            if (event->button == SDL_BUTTON_X1) {
-                action_app_switch(controller, action);
+            if (im->kp && event->button == SDL_BUTTON_X1) {
+                action_app_switch(im, action);
                 return;
             }
             if (event->button == SDL_BUTTON_X2 && down) {
                 if (event->clicks < 2) {
-                    expand_notification_panel(controller);
+                    expand_notification_panel(im);
                 } else {
-                    expand_settings_panel(controller);
+                    expand_settings_panel(im);
                 }
                 return;
             }
-            if (event->button == SDL_BUTTON_RIGHT) {
-                press_back_or_turn_screen_on(controller, action);
+            if (im->kp && event->button == SDL_BUTTON_RIGHT) {
+                press_back_or_turn_screen_on(im, action);
                 return;
             }
-            if (event->button == SDL_BUTTON_MIDDLE) {
-                action_home(controller, action);
+            if (im->kp && event->button == SDL_BUTTON_MIDDLE) {
+                action_home(im, action);
                 return;
             }
         }
@@ -704,7 +747,7 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
         // otherwise, send the click event to the device
     }
 
-    if (!controller) {
+    if (!im->mp) {
         return;
     }
 
@@ -844,7 +887,7 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
     bool control = im->controller;
     switch (event->type) {
         case SDL_TEXTINPUT:
-            if (!control) {
+            if (!im->kp) {
                 break;
             }
             sc_input_manager_process_text_input(im, &event->text);
@@ -856,13 +899,13 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
             sc_input_manager_process_key(im, &event->key);
             break;
         case SDL_MOUSEMOTION:
-            if (!control) {
+            if (!im->mp) {
                 break;
             }
             sc_input_manager_process_mouse_motion(im, &event->motion);
             break;
         case SDL_MOUSEWHEEL:
-            if (!control) {
+            if (!im->mp) {
                 break;
             }
             sc_input_manager_process_mouse_wheel(im, &event->wheel);
@@ -876,7 +919,7 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
         case SDL_FINGERMOTION:
         case SDL_FINGERDOWN:
         case SDL_FINGERUP:
-            if (!control) {
+            if (!im->mp) {
                 break;
             }
             sc_input_manager_process_touch(im, &event->tfinger);

app/src/keyboard_sdk.c

@@ -1,4 +1,4 @@
-#include "keyboard_inject.h"
+#include "keyboard_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast key processor to sc_keyboard_inject */
-#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_inject, key_processor)
+/** Downcast key processor to sc_keyboard_sdk */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_sdk, key_processor)
 
 static enum android_keyevent_action
 convert_keycode_action(enum sc_action action) {
@@ -271,20 +271,20 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
     // is set before injecting Ctrl+v.
     (void) ack_to_wait;
 
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
     if (event->repeat) {
-        if (!ki->forward_key_repeat) {
+        if (!kb->forward_key_repeat) {
             return;
         }
-        ++ki->repeat;
+        ++kb->repeat;
     } else {
-        ki->repeat = 0;
+        kb->repeat = 0;
     }
 
     struct sc_control_msg msg;
-    if (convert_input_key(event, &msg, ki->key_inject_mode, ki->repeat)) {
-        if (!sc_controller_push_msg(ki->controller, &msg)) {
+    if (convert_input_key(event, &msg, kb->key_inject_mode, kb->repeat)) {
+        if (!sc_controller_push_msg(kb->controller, &msg)) {
             LOGW("Could not request 'inject keycode'");
         }
     }
@@ -293,14 +293,14 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
 static void
 sc_key_processor_process_text(struct sc_key_processor *kp,
                               const struct sc_text_event *event) {
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
         // Never inject text events
         return;
     }
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
         char c = event->text[0];
         if (isalpha(c) || c == ' ') {
             assert(event->text[1] == '\0');
@@ -316,22 +316,22 @@ sc_key_processor_process_text(struct sc_key_processor *kp,
         LOGW("Could not strdup input text");
         return;
     }
-    if (!sc_controller_push_msg(ki->controller, &msg)) {
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
         free(msg.inject_text.text);
         LOGW("Could not request 'inject text'");
     }
 }
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
-                        struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat) {
-    ki->controller = controller;
-    ki->key_inject_mode = key_inject_mode;
-    ki->forward_key_repeat = forward_key_repeat;
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
+                     struct sc_controller *controller,
+                     enum sc_key_inject_mode key_inject_mode,
+                     bool forward_key_repeat) {
+    kb->controller = controller;
+    kb->key_inject_mode = key_inject_mode;
+    kb->forward_key_repeat = forward_key_repeat;
 
-    ki->repeat = 0;
+    kb->repeat = 0;
 
     static const struct sc_key_processor_ops ops = {
         .process_key = sc_key_processor_process_key,
@@ -339,6 +339,7 @@ sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
     };
 
     // Key injection and clipboard synchronization are serialized
-    ki->key_processor.async_paste = false;
-    ki->key_processor.ops = &ops;
+    kb->key_processor.async_paste = false;
+    kb->key_processor.hid = false;
+    kb->key_processor.ops = &ops;
 }

app/src/keyboard_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_KEYBOARD_INJECT_H
-#define SC_KEYBOARD_INJECT_H
+#ifndef SC_KEYBOARD_SDK_H
+#define SC_KEYBOARD_SDK_H
 
 #include "common.h"
 
@@ -9,7 +9,7 @@
 #include "options.h"
 #include "trait/key_processor.h"
 
-struct sc_keyboard_inject {
+struct sc_keyboard_sdk {
     struct sc_key_processor key_processor; // key processor trait
 
     struct sc_controller *controller;
@@ -23,9 +23,9 @@ struct sc_keyboard_inject {
 };
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
-                        struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat);
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
+                     struct sc_controller *controller,
+                     enum sc_key_inject_mode key_inject_mode,
+                     bool forward_key_repeat);
 
 #endif

app/src/mouse_sdk.c

@@ -1,4 +1,4 @@
-#include "mouse_inject.h"
+#include "mouse_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast mouse processor to sc_mouse_inject */
-#define DOWNCAST(MP) container_of(MP, struct sc_mouse_inject, mouse_processor)
+/** Downcast mouse processor to sc_mouse_sdk */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_sdk, mouse_processor)
 
 static enum android_motionevent_buttons
 convert_mouse_buttons(uint32_t state) {
@@ -63,7 +63,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
         return;
     }
 
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -76,7 +76,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse motion event'");
     }
 }
@@ -84,7 +84,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
                                     const struct sc_mouse_click_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -98,7 +98,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse click event'");
     }
 }
@@ -106,7 +106,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
                                    const struct sc_mouse_scroll_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT,
@@ -118,7 +118,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse scroll event'");
     }
 }
@@ -126,7 +126,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
                                  const struct sc_touch_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -139,15 +139,14 @@ sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject touch event'");
     }
 }
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
-                     struct sc_controller *controller) {
-    mi->controller = controller;
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller) {
+    m->controller = controller;
 
     static const struct sc_mouse_processor_ops ops = {
         .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
@@ -156,7 +155,7 @@ sc_mouse_inject_init(struct sc_mouse_inject *mi,
         .process_touch = sc_mouse_processor_process_touch,
     };
 
-    mi->mouse_processor.ops = &ops;
+    m->mouse_processor.ops = &ops;
 
-    mi->mouse_processor.relative_mode = false;
+    m->mouse_processor.relative_mode = false;
 }

app/src/mouse_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_MOUSE_INJECT_H
-#define SC_MOUSE_INJECT_H
+#ifndef SC_MOUSE_SDK_H
+#define SC_MOUSE_SDK_H
 
 #include "common.h"
 
@@ -9,14 +9,13 @@
 #include "screen.h"
 #include "trait/mouse_processor.h"
 
-struct sc_mouse_inject {
+struct sc_mouse_sdk {
     struct sc_mouse_processor mouse_processor; // mouse processor trait
 
     struct sc_controller *controller;
 };
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
-                     struct sc_controller *controller);
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller);
 
 #endif

app/src/options.c

@@ -21,8 +21,8 @@ const struct scrcpy_options scrcpy_options_default = {
     .video_source = SC_VIDEO_SOURCE_DISPLAY,
     .audio_source = SC_AUDIO_SOURCE_AUTO,
     .record_format = SC_RECORD_FORMAT_AUTO,
-    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT,
-    .mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT,
+    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_AUTO,
+    .mouse_input_mode = SC_MOUSE_INPUT_MODE_AUTO,
     .camera_facing = SC_CAMERA_FACING_ANY,
     .port_range = {
         .first = DEFAULT_LOCAL_PORT_RANGE_FIRST,

app/src/options.h

@@ -140,13 +140,19 @@ enum sc_lock_video_orientation {
 };
 
 enum sc_keyboard_input_mode {
-    SC_KEYBOARD_INPUT_MODE_INJECT,
-    SC_KEYBOARD_INPUT_MODE_HID,
+    SC_KEYBOARD_INPUT_MODE_AUTO,
+    SC_KEYBOARD_INPUT_MODE_DISABLED,
+    SC_KEYBOARD_INPUT_MODE_SDK,
+    SC_KEYBOARD_INPUT_MODE_UHID,
+    SC_KEYBOARD_INPUT_MODE_AOA,
 };
 
 enum sc_mouse_input_mode {
-    SC_MOUSE_INPUT_MODE_INJECT,
-    SC_MOUSE_INPUT_MODE_HID,
+    SC_MOUSE_INPUT_MODE_AUTO,
+    SC_MOUSE_INPUT_MODE_DISABLED,
+    SC_MOUSE_INPUT_MODE_SDK,
+    SC_MOUSE_INPUT_MODE_UHID,
+    SC_MOUSE_INPUT_MODE_AOA,
 };
 
 enum sc_key_inject_mode {

app/src/receiver.c

@@ -1,22 +1,24 @@
 #include "receiver.h"
 
 #include <assert.h>
+#include <inttypes.h>
 #include <stdint.h>
 #include <SDL2/SDL_clipboard.h>
 
 #include "device_msg.h"
 #include "util/log.h"
+#include "util/str.h"
 
 bool
-sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
-              struct sc_acksync *acksync) {
+sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket) {
     bool ok = sc_mutex_init(&receiver->mutex);
     if (!ok) {
         return false;
     }
 
     receiver->control_socket = control_socket;
-    receiver->acksync = acksync;
+    receiver->acksync = NULL;
+    receiver->uhid_devices = NULL;
 
     return true;
 }
@@ -58,6 +60,41 @@ process_msg(struct sc_receiver *receiver, struct sc_device_msg *msg) {
 
             sc_acksync_ack(receiver->acksync, msg->ack_clipboard.sequence);
             break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
+                char *hex = sc_str_to_hex_string(msg->uhid_output.data,
+                                                 msg->uhid_output.size);
+                if (hex) {
+                    LOGV("UHID output [%" PRIu16 "] %s",
+                         msg->uhid_output.id, hex);
+                    free(hex);
+                } else {
+                    LOGV("UHID output [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_output.id, msg->uhid_output.size);
+                }
+            }
+
+            // This is a programming error to receive this message if there is
+            // no uhid_devices instance
+            assert(receiver->uhid_devices);
+
+            // Also check at runtime (do not trust the server)
+            if (!receiver->uhid_devices) {
+                LOGE("Received unexpected HID output message");
+                return;
+            }
+
+            struct sc_uhid_receiver *uhid_receiver =
+                sc_uhid_devices_get_receiver(receiver->uhid_devices,
+                                             msg->uhid_output.id);
+            if (uhid_receiver) {
+                uhid_receiver->ops->process_output(uhid_receiver,
+                                                   msg->uhid_output.data,
+                                                   msg->uhid_output.size);
+            } else {
+                LOGW("No UHID receiver for id %" PRIu16, msg->uhid_output.id);
+            }
+            break;
     }
 }
 

app/src/receiver.h

@@ -5,6 +5,7 @@
 
 #include <stdbool.h>
 
+#include "uhid/uhid_output.h"
 #include "util/acksync.h"
 #include "util/net.h"
 #include "util/thread.h"
@@ -17,11 +18,11 @@ struct sc_receiver {
     sc_mutex mutex;
 
     struct sc_acksync *acksync;
+    struct sc_uhid_devices *uhid_devices;
 };
 
 bool
-sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
-                 struct sc_acksync *acksync);
+sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket);
 
 void
 sc_receiver_destroy(struct sc_receiver *receiver);

app/src/scrcpy.c

@@ -20,15 +20,17 @@
 #include "demuxer.h"
 #include "events.h"
 #include "file_pusher.h"
-#include "keyboard_inject.h"
-#include "mouse_inject.h"
+#include "keyboard_sdk.h"
+#include "mouse_sdk.h"
 #include "recorder.h"
 #include "screen.h"
 #include "server.h"
+#include "uhid/keyboard_uhid.h"
+#include "uhid/mouse_uhid.h"
 #ifdef HAVE_USB
 # include "usb/aoa_hid.h"
-# include "usb/hid_keyboard.h"
-# include "usb/hid_mouse.h"
+# include "usb/keyboard_aoa.h"
+# include "usb/mouse_aoa.h"
 # include "usb/usb.h"
 #endif
 #include "util/acksync.h"
@@ -61,17 +63,20 @@ struct scrcpy {
     struct sc_aoa aoa;
     // sequence/ack helper to synchronize clipboard and Ctrl+v via HID
     struct sc_acksync acksync;
+    struct sc_uhid_devices uhid_devices;
 #endif
     union {
-        struct sc_keyboard_inject keyboard_inject;
+        struct sc_keyboard_sdk keyboard_sdk;
+        struct sc_keyboard_uhid keyboard_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_keyboard keyboard_hid;
+        struct sc_keyboard_aoa keyboard_aoa;
 #endif
     };
     union {
-        struct sc_mouse_inject mouse_inject;
+        struct sc_mouse_sdk mouse_sdk;
+        struct sc_mouse_uhid mouse_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_mouse mouse_hid;
+        struct sc_mouse_aoa mouse_aoa;
 #endif
     };
     struct sc_timeout timeout;
@@ -307,6 +312,10 @@ scrcpy_generate_scid(void) {
 enum scrcpy_exit_code
 scrcpy(struct scrcpy_options *options) {
     static struct scrcpy scrcpy;
+#ifndef NDEBUG
+    // Detect missing initializations
+    memset(&scrcpy, 42, sizeof(scrcpy));
+#endif
     struct scrcpy *s = &scrcpy;
 
     // Minimal SDL initialization
@@ -330,8 +339,8 @@ scrcpy(struct scrcpy_options *options) {
     bool audio_demuxer_started = false;
 #ifdef HAVE_USB
     bool aoa_hid_initialized = false;
-    bool hid_keyboard_initialized = false;
-    bool hid_mouse_initialized = false;
+    bool keyboard_aoa_initialized = false;
+    bool mouse_aoa_initialized = false;
 #endif
     bool controller_initialized = false;
     bool controller_started = false;
@@ -340,6 +349,7 @@ scrcpy(struct scrcpy_options *options) {
     bool timeout_started = false;
 
     struct sc_acksync *acksync = NULL;
+    struct sc_uhid_devices *uhid_devices = NULL;
 
     uint32_t scid = scrcpy_generate_scid();
 
@@ -542,12 +552,19 @@ scrcpy(struct scrcpy_options *options) {
     struct sc_mouse_processor *mp = NULL;
 
     if (options->control) {
+        if (!sc_controller_init(&s->controller, s->server.control_socket)) {
+            goto end;
+        }
+        controller_initialized = true;
+
+        controller = &s->controller;
+
 #ifdef HAVE_USB
-        bool use_hid_keyboard =
-            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
-        bool use_hid_mouse =
-            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
-        if (use_hid_keyboard || use_hid_mouse) {
+        bool use_keyboard_aoa =
+            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
+        bool use_mouse_aoa =
+            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
+        if (use_keyboard_aoa || use_mouse_aoa) {
             bool ok = sc_acksync_init(&s->acksync);
             if (!ok) {
                 goto end;
@@ -557,7 +574,7 @@ scrcpy(struct scrcpy_options *options) {
             if (!ok) {
                 LOGE("Failed to initialize USB");
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             assert(serial);
@@ -565,7 +582,7 @@ scrcpy(struct scrcpy_options *options) {
             ok = sc_usb_select_device(&s->usb, serial, &usb_device);
             if (!ok) {
                 sc_usb_destroy(&s->usb);
-                goto aoa_hid_end;
+                goto end;
             }
 
             LOGI("USB device: %s (%04" PRIx16 ":%04" PRIx16 ") %s %s",
@@ -578,7 +595,7 @@ scrcpy(struct scrcpy_options *options) {
                 LOGE("Failed to connect to USB device %s", serial);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             ok = sc_aoa_init(&s->aoa, &s->usb, &s->acksync);
@@ -587,95 +604,80 @@ scrcpy(struct scrcpy_options *options) {
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
-            if (use_hid_keyboard) {
-                if (sc_hid_keyboard_init(&s->keyboard_hid, &s->aoa)) {
-                    hid_keyboard_initialized = true;
-                    kp = &s->keyboard_hid.key_processor;
+            if (use_keyboard_aoa) {
+                if (sc_keyboard_aoa_init(&s->keyboard_aoa, &s->aoa)) {
+                    keyboard_aoa_initialized = true;
+                    kp = &s->keyboard_aoa.key_processor;
                 } else {
                     LOGE("Could not initialize HID keyboard");
                 }
             }
 
-            if (use_hid_mouse) {
-                if (sc_hid_mouse_init(&s->mouse_hid, &s->aoa)) {
-                    hid_mouse_initialized = true;
-                    mp = &s->mouse_hid.mouse_processor;
+            if (use_mouse_aoa) {
+                if (sc_mouse_aoa_init(&s->mouse_aoa, &s->aoa)) {
+                    mouse_aoa_initialized = true;
+                    mp = &s->mouse_aoa.mouse_processor;
                 } else {
                     LOGE("Could not initialized HID mouse");
                 }
             }
 
-            bool need_aoa = hid_keyboard_initialized || hid_mouse_initialized;
+            bool need_aoa = keyboard_aoa_initialized || mouse_aoa_initialized;
 
             if (!need_aoa || !sc_aoa_start(&s->aoa)) {
                 sc_acksync_destroy(&s->acksync);
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_aoa_destroy(&s->aoa);
-                goto aoa_hid_end;
+                goto end;
             }
 
             acksync = &s->acksync;
 
             aoa_hid_initialized = true;
-
-aoa_hid_end:
-            if (!aoa_hid_initialized) {
-                if (hid_keyboard_initialized) {
-                    sc_hid_keyboard_destroy(&s->keyboard_hid);
-                    hid_keyboard_initialized = false;
-                }
-                if (hid_mouse_initialized) {
-                    sc_hid_mouse_destroy(&s->mouse_hid);
-                    hid_mouse_initialized = false;
-                }
-            }
-
-            if (use_hid_keyboard && !hid_keyboard_initialized) {
-                LOGE("Fallback to default keyboard injection method "
-                     "(-K/--hid-keyboard ignored)");
-                options->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT;
-            }
-
-            if (use_hid_mouse && !hid_mouse_initialized) {
-                LOGE("Fallback to default mouse injection method "
-                     "(-M/--hid-mouse ignored)");
-                options->mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT;
-            }
         }
 #else
-        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_HID);
-        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_HID);
+        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_AOA);
+        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_AOA);
 #endif
 
-        // keyboard_input_mode may have been reset if HID mode failed
-        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_INJECT) {
-            sc_keyboard_inject_init(&s->keyboard_inject, &s->controller,
-                                    options->key_inject_mode,
-                                    options->forward_key_repeat);
-            kp = &s->keyboard_inject.key_processor;
+        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_SDK) {
+            sc_keyboard_sdk_init(&s->keyboard_sdk, &s->controller,
+                                 options->key_inject_mode,
+                                 options->forward_key_repeat);
+            kp = &s->keyboard_sdk.key_processor;
+        } else if (options->keyboard_input_mode
+                == SC_KEYBOARD_INPUT_MODE_UHID) {
+            sc_uhid_devices_init(&s->uhid_devices);
+            bool ok = sc_keyboard_uhid_init(&s->keyboard_uhid, &s->controller,
+                                            &s->uhid_devices);
+            if (!ok) {
+                goto end;
+            }
+            uhid_devices = &s->uhid_devices;
+            kp = &s->keyboard_uhid.key_processor;
         }
 
-        // mouse_input_mode may have been reset if HID mode failed
-        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_INJECT) {
-            sc_mouse_inject_init(&s->mouse_inject, &s->controller);
-            mp = &s->mouse_inject.mouse_processor;
+        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_SDK) {
+            sc_mouse_sdk_init(&s->mouse_sdk, &s->controller);
+            mp = &s->mouse_sdk.mouse_processor;
+        } else if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_UHID) {
+            bool ok = sc_mouse_uhid_init(&s->mouse_uhid, &s->controller);
+            if (!ok) {
+                goto end;
+            }
+            mp = &s->mouse_uhid.mouse_processor;
         }
 
-        if (!sc_controller_init(&s->controller, s->server.control_socket,
-                                acksync)) {
-            goto end;
-        }
-        controller_initialized = true;
+        sc_controller_configure(&s->controller, acksync, uhid_devices);
 
         if (!sc_controller_start(&s->controller)) {
             goto end;
         }
         controller_started = true;
-        controller = &s->controller;
     }
 
     // There is a controller if and only if control is enabled
@@ -815,11 +817,11 @@ end:
     // end-of-stream
 #ifdef HAVE_USB
     if (aoa_hid_initialized) {
-        if (hid_keyboard_initialized) {
-            sc_hid_keyboard_destroy(&s->keyboard_hid);
+        if (keyboard_aoa_initialized) {
+            sc_keyboard_aoa_destroy(&s->keyboard_aoa);
         }
-        if (hid_mouse_initialized) {
-            sc_hid_mouse_destroy(&s->mouse_hid);
+        if (mouse_aoa_initialized) {
+            sc_mouse_aoa_destroy(&s->mouse_aoa);
         }
         sc_aoa_stop(&s->aoa);
         sc_usb_stop(&s->usb);

app/src/trait/key_processor.h

@@ -23,6 +23,13 @@ struct sc_key_processor {
      */
     bool async_paste;
 
+    /**
+     * Set by the implementation to indicate that the keyboard is HID. In
+     * practice, it is used to react on a shortcut to open the hard keyboard
+     * settings only if the keyboard is HID.
+     */
+    bool hid;
+
     const struct sc_key_processor_ops *ops;
 };
 

app/src/uhid/keyboard_uhid.c

@@ -0,0 +1,162 @@
+#include "keyboard_uhid.h"
+
+#include "util/log.h"
+
+/** Downcast key processor to keyboard_uhid */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_uhid, key_processor)
+
+/** Downcast uhid_receiver to keyboard_uhid */
+#define DOWNCAST_RECEIVER(UR) \
+    container_of(UR, struct sc_keyboard_uhid, uhid_receiver)
+
+#define UHID_KEYBOARD_ID 1
+
+static void
+sc_keyboard_uhid_send_input(struct sc_keyboard_uhid *kb,
+                            const struct sc_hid_event *event) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = UHID_KEYBOARD_ID;
+
+    assert(event->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, event->data, event->size);
+    msg.uhid_input.size = event->size;
+
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
+        LOGE("Could not send UHID_INPUT message (key)");
+    }
+}
+
+static void
+sc_keyboard_uhid_synchronize_mod(struct sc_keyboard_uhid *kb) {
+    SDL_Keymod sdl_mod = SDL_GetModState();
+    uint16_t mod = sc_mods_state_from_sdl(sdl_mod) & (SC_MOD_CAPS | SC_MOD_NUM);
+
+    uint16_t device_mod =
+        atomic_load_explicit(&kb->device_mod, memory_order_relaxed);
+    uint16_t diff = mod ^ device_mod;
+
+    if (diff) {
+        // Inherently racy (the HID output reports arrive asynchronously in
+        // response to key presses), but will re-synchronize on next key press
+        // or HID output anyway
+        atomic_store_explicit(&kb->device_mod, mod, memory_order_relaxed);
+
+        struct sc_hid_event hid_event;
+        sc_hid_keyboard_event_from_mods(&hid_event, diff);
+
+        LOGV("HID keyboard state synchronized");
+
+        sc_keyboard_uhid_send_input(kb, &hid_event);
+    }
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const struct sc_key_event *event,
+                             uint64_t ack_to_wait) {
+    (void) ack_to_wait;
+
+    if (event->repeat) {
+        // In USB HID protocol, key repeat is handled by the host (Android), so
+        // just ignore key repeat here.
+        return;
+    }
+
+    struct sc_keyboard_uhid *kb = DOWNCAST(kp);
+
+    struct sc_hid_event hid_event;
+
+    // Not all keys are supported, just ignore unsupported keys
+    if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
+        if (event->scancode == SC_SCANCODE_CAPSLOCK) {
+            atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_CAPS,
+                                      memory_order_relaxed);
+        } else if (event->scancode == SC_SCANCODE_NUMLOCK) {
+            atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_NUM,
+                                      memory_order_relaxed);
+        } else {
+            // Synchronize modifiers (only if the scancode itself does not
+            // change the modifiers)
+            sc_keyboard_uhid_synchronize_mod(kb);
+        }
+        sc_keyboard_uhid_send_input(kb, &hid_event);
+    }
+}
+
+static unsigned
+sc_keyboard_uhid_to_sc_mod(uint8_t hid_led) {
+    // <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
+    // (chapter 11: LED page)
+    unsigned mod = 0;
+    if (hid_led & 0x01) {
+        mod |= SC_MOD_NUM;
+    }
+    if (hid_led & 0x02) {
+        mod |= SC_MOD_CAPS;
+    }
+    return mod;
+}
+
+static void
+sc_uhid_receiver_process_output(struct sc_uhid_receiver *receiver,
+                                const uint8_t *data, size_t len) {
+    // Called from the thread receiving device messages
+
+    assert(len);
+
+    // Also check at runtime (do not trust the server)
+    if (!len) {
+        LOGE("Unexpected empty HID output message");
+        return;
+    }
+
+    struct sc_keyboard_uhid *kb = DOWNCAST_RECEIVER(receiver);
+
+    uint8_t hid_led = data[0];
+    uint16_t device_mod = sc_keyboard_uhid_to_sc_mod(hid_led);
+    atomic_store_explicit(&kb->device_mod, device_mod, memory_order_relaxed);
+}
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller,
+                      struct sc_uhid_devices *uhid_devices) {
+    sc_hid_keyboard_init(&kb->hid);
+
+    kb->controller = controller;
+    atomic_init(&kb->device_mod, 0);
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        // Never forward text input via HID (all the keys are injected
+        // separately)
+        .process_text = NULL,
+    };
+
+    // Clipboard synchronization is requested over the same control socket, so
+    // there is no need for a specific synchronization mechanism
+    kb->key_processor.async_paste = false;
+    kb->key_processor.hid = true;
+    kb->key_processor.ops = &ops;
+
+    static const struct sc_uhid_receiver_ops uhid_receiver_ops = {
+        .process_output = sc_uhid_receiver_process_output,
+    };
+
+    kb->uhid_receiver.id = UHID_KEYBOARD_ID;
+    kb->uhid_receiver.ops = &uhid_receiver_ops;
+    sc_uhid_devices_add_receiver(uhid_devices, &kb->uhid_receiver);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = UHID_KEYBOARD_ID;
+    msg.uhid_create.report_desc = SC_HID_KEYBOARD_REPORT_DESC;
+    msg.uhid_create.report_desc_size = SC_HID_KEYBOARD_REPORT_DESC_LEN;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not send UHID_CREATE message (keyboard)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/keyboard_uhid.h

@@ -0,0 +1,27 @@
+#ifndef SC_KEYBOARD_UHID_H
+#define SC_KEYBOARD_UHID_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "hid/hid_keyboard.h"
+#include "uhid/uhid_output.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_uhid {
+    struct sc_key_processor key_processor; // key processor trait
+    struct sc_uhid_receiver uhid_receiver;
+
+    struct sc_hid_keyboard hid;
+    struct sc_controller *controller;
+    atomic_uint_least16_t device_mod;
+};
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller,
+                      struct sc_uhid_devices *uhid_devices);
+
+#endif

app/src/uhid/mouse_uhid.c

@@ -0,0 +1,89 @@
+#include "mouse_uhid.h"
+
+#include "hid/hid_mouse.h"
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to mouse_uhid */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_uhid, mouse_processor)
+
+#define UHID_MOUSE_ID 2
+
+static void
+sc_mouse_uhid_send_input(struct sc_mouse_uhid *mouse,
+                         const struct sc_hid_event *event, const char *name) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = UHID_MOUSE_ID;
+
+    assert(event->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, event->data, event->size);
+    msg.uhid_input.size = event->size;
+
+    if (!sc_controller_push_msg(mouse->controller, &msg)) {
+        LOGE("Could not send UHID_INPUT message (%s)", name);
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_motion_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_motion(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse motion");
+}
+
+static void
+sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
+                                   const struct sc_mouse_click_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_click(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse click");
+}
+
+static void
+sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_scroll_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_scroll(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse scroll");
+}
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller) {
+    mouse->controller = controller;
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_mouse_click = sc_mouse_processor_process_mouse_click,
+        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
+        // Touch events not supported (coordinates are not relative)
+        .process_touch = NULL,
+    };
+
+    mouse->mouse_processor.ops = &ops;
+
+    mouse->mouse_processor.relative_mode = true;
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = UHID_MOUSE_ID;
+    msg.uhid_create.report_desc = SC_HID_MOUSE_REPORT_DESC;
+    msg.uhid_create.report_desc_size = SC_HID_MOUSE_REPORT_DESC_LEN;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not send UHID_CREATE message (mouse)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/mouse_uhid.h

@@ -0,0 +1,19 @@
+#ifndef SC_MOUSE_UHID_H
+#define SC_MOUSE_UHID_H
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "trait/mouse_processor.h"
+
+struct sc_mouse_uhid {
+    struct sc_mouse_processor mouse_processor; // mouse processor trait
+
+    struct sc_controller *controller;
+};
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller);
+
+#endif

app/src/uhid/uhid_output.c

@@ -0,0 +1,25 @@
+#include "uhid_output.h"
+
+#include <assert.h>
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices) {
+    devices->count = 0;
+}
+
+void
+sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
+                             struct sc_uhid_receiver *receiver) {
+    assert(devices->count < SC_UHID_MAX_RECEIVERS);
+    devices->receivers[devices->count++] = receiver;
+}
+
+struct sc_uhid_receiver *
+sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id) {
+    for (size_t i = 0; i < devices->count; ++i) {
+        if (devices->receivers[i]->id == id) {
+            return devices->receivers[i];
+        }
+    }
+    return NULL;
+}

app/src/uhid/uhid_output.h

@@ -0,0 +1,45 @@
+#ifndef SC_UHID_OUTPUT_H
+#define SC_UHID_OUTPUT_H
+
+#include "common.h"
+
+#include <stdbool.h>
+#include <stdint.h>
+
+/**
+ * The communication with UHID devices is bidirectional.
+ *
+ * This component manages the registration of receivers to handle UHID output
+ * messages (sent from the device to the computer).
+ */
+
+struct sc_uhid_receiver {
+    uint16_t id;
+
+    const struct sc_uhid_receiver_ops *ops;
+};
+
+struct sc_uhid_receiver_ops {
+    void
+    (*process_output)(struct sc_uhid_receiver *receiver,
+                      const uint8_t *data, size_t len);
+};
+
+#define SC_UHID_MAX_RECEIVERS 1
+
+struct sc_uhid_devices {
+    struct sc_uhid_receiver *receivers[SC_UHID_MAX_RECEIVERS];
+    unsigned count;
+};
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices);
+
+void
+sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
+                             struct sc_uhid_receiver *receiver);
+
+struct sc_uhid_receiver *
+sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id);
+
+#endif

app/src/usb/aoa_hid.c

@@ -5,6 +5,7 @@
 
 #include "aoa_hid.h"
 #include "util/log.h"
+#include "util/str.h"
 
 // See <https://source.android.com/devices/accessories/aoa2#hid-support>.
 #define ACCESSORY_REGISTER_HID 54
@@ -14,37 +15,18 @@
 
 #define DEFAULT_TIMEOUT 1000
 
-#define SC_HID_EVENT_QUEUE_MAX 64
+#define SC_AOA_EVENT_QUEUE_MAX 64
 
 static void
-sc_hid_event_log(const struct sc_hid_event *event) {
+sc_hid_event_log(uint16_t accessory_id, const struct sc_hid_event *event) {
     // HID Event: [00] FF FF FF FF...
     assert(event->size);
-    unsigned buffer_size = event->size * 3 + 1;
-    char *buffer = malloc(buffer_size);
-    if (!buffer) {
-        LOG_OOM();
+    char *hex = sc_str_to_hex_string(event->data, event->size);
+    if (!hex) {
         return;
     }
-    for (unsigned i = 0; i < event->size; ++i) {
-        snprintf(buffer + i * 3, 4, " %02x", event->buffer[i]);
-    }
-    LOGV("HID Event: [%d]%s", event->accessory_id, buffer);
-    free(buffer);
-}
-
-void
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
-                  unsigned char *buffer, uint16_t buffer_size) {
-    hid_event->accessory_id = accessory_id;
-    hid_event->buffer = buffer;
-    hid_event->size = buffer_size;
-    hid_event->ack_to_wait = SC_SEQUENCE_INVALID;
-}
-
-void
-sc_hid_event_destroy(struct sc_hid_event *hid_event) {
-    free(hid_event->buffer);
+    LOGV("HID Event: [%d] %s", accessory_id, hex);
+    free(hex);
 }
 
 bool
@@ -52,7 +34,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
             struct sc_acksync *acksync) {
     sc_vecdeque_init(&aoa->queue);
 
-    if (!sc_vecdeque_reserve(&aoa->queue, SC_HID_EVENT_QUEUE_MAX)) {
+    if (!sc_vecdeque_reserve(&aoa->queue, SC_AOA_EVENT_QUEUE_MAX)) {
         return false;
     }
 
@@ -76,12 +58,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
 
 void
 sc_aoa_destroy(struct sc_aoa *aoa) {
-    // Destroy remaining events
-    while (!sc_vecdeque_is_empty(&aoa->queue)) {
-        struct sc_hid_event *event = sc_vecdeque_popref(&aoa->queue);
-        assert(event);
-        sc_hid_event_destroy(event);
-    }
+    sc_vecdeque_destroy(&aoa->queue);
 
     sc_cond_destroy(&aoa->event_cond);
     sc_mutex_destroy(&aoa->mutex);
@@ -97,10 +74,10 @@ sc_aoa_register_hid(struct sc_aoa *aoa, uint16_t accessory_id,
     // index (arg1): total length of the HID report descriptor
     uint16_t value = accessory_id;
     uint16_t index = report_desc_size;
-    unsigned char *buffer = NULL;
+    unsigned char *data = NULL;
     uint16_t length = 0;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("REGISTER_HID: libusb error: %s", libusb_strerror(result));
@@ -130,14 +107,14 @@ sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
      * See <https://libusb.sourceforge.io/api-1.0/libusb_packetoverflow.html>
      */
     // value (arg0): accessory assigned ID for the HID device
-    // index (arg1): offset of data (buffer) in descriptor
+    // index (arg1): offset of data in descriptor
     uint16_t value = accessory_id;
     uint16_t index = 0;
     // libusb_control_transfer expects a pointer to non-const
-    unsigned char *buffer = (unsigned char *) report_desc;
+    unsigned char *data = (unsigned char *) report_desc;
     uint16_t length = report_desc_size;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("SET_HID_REPORT_DESC: libusb error: %s", libusb_strerror(result));
@@ -169,18 +146,19 @@ sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
 }
 
 static bool
-sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
+sc_aoa_send_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
+                      const struct sc_hid_event *event) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_SEND_HID_EVENT;
     // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
     // value (arg0): accessory assigned ID for the HID device
     // index (arg1): 0 (unused)
-    uint16_t value = event->accessory_id;
+    uint16_t value = accessory_id;
     uint16_t index = 0;
-    unsigned char *buffer = event->buffer;
+    unsigned char *data = (uint8_t *) event->data; // discard const
     uint16_t length = event->size;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("SEND_HID_EVENT: libusb error: %s", libusb_strerror(result));
@@ -192,7 +170,7 @@ sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
 }
 
 bool
-sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
+sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_UNREGISTER_HID;
     // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
@@ -200,10 +178,10 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
     // index (arg1): 0
     uint16_t value = accessory_id;
     uint16_t index = 0;
-    unsigned char *buffer = NULL;
+    unsigned char *data = NULL;
     uint16_t length = 0;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("UNREGISTER_HID: libusb error: %s", libusb_strerror(result));
@@ -215,16 +193,25 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
 }
 
 bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
+sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
+                                       uint16_t accessory_id,
+                                       const struct sc_hid_event *event,
+                                       uint64_t ack_to_wait) {
     if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
-        sc_hid_event_log(event);
+        sc_hid_event_log(accessory_id, event);
     }
 
     sc_mutex_lock(&aoa->mutex);
     bool full = sc_vecdeque_is_full(&aoa->queue);
     if (!full) {
         bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
-        sc_vecdeque_push_noresize(&aoa->queue, *event);
+
+        struct sc_aoa_event *aoa_event =
+            sc_vecdeque_push_hole_noresize(&aoa->queue);
+        aoa_event->hid = *event;
+        aoa_event->accessory_id = accessory_id;
+        aoa_event->ack_to_wait = ack_to_wait;
+
         if (was_empty) {
             sc_cond_signal(&aoa->event_cond);
         }
@@ -252,7 +239,7 @@ run_aoa_thread(void *data) {
         }
 
         assert(!sc_vecdeque_is_empty(&aoa->queue));
-        struct sc_hid_event event = sc_vecdeque_pop(&aoa->queue);
+        struct sc_aoa_event event = sc_vecdeque_pop(&aoa->queue);
         uint64_t ack_to_wait = event.ack_to_wait;
         sc_mutex_unlock(&aoa->mutex);
 
@@ -271,17 +258,14 @@ run_aoa_thread(void *data) {
 
             if (result == SC_ACKSYNC_WAIT_TIMEOUT) {
                 LOGW("Ack not received after 500ms, discarding HID event");
-                sc_hid_event_destroy(&event);
                 continue;
             } else if (result == SC_ACKSYNC_WAIT_INTR) {
                 // stopped
-                sc_hid_event_destroy(&event);
                 break;
             }
         }
 
-        bool ok = sc_aoa_send_hid_event(aoa, &event);
-        sc_hid_event_destroy(&event);
+        bool ok = sc_aoa_send_hid_event(aoa, event.accessory_id, &event.hid);
         if (!ok) {
             LOGW("Could not send HID event to USB device");
         }

app/src/usb/aoa_hid.h

@@ -6,28 +6,22 @@
 
 #include <libusb-1.0/libusb.h>
 
+#include "hid/hid_event.h"
 #include "usb.h"
 #include "util/acksync.h"
 #include "util/thread.h"
 #include "util/tick.h"
 #include "util/vecdeque.h"
 
-struct sc_hid_event {
+#define SC_HID_MAX_SIZE 8
+
+struct sc_aoa_event {
+    struct sc_hid_event hid;
     uint16_t accessory_id;
-    unsigned char *buffer;
-    uint16_t size;
     uint64_t ack_to_wait;
 };
 
-// Takes ownership of buffer
-void
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
-                  unsigned char *buffer, uint16_t buffer_size);
-
-void
-sc_hid_event_destroy(struct sc_hid_event *hid_event);
-
-struct sc_hid_event_queue SC_VECDEQUE(struct sc_hid_event);
+struct sc_aoa_event_queue SC_VECDEQUE(struct sc_aoa_event);
 
 struct sc_aoa {
     struct sc_usb *usb;
@@ -35,7 +29,7 @@ struct sc_aoa {
     sc_mutex mutex;
     sc_cond event_cond;
     bool stopped;
-    struct sc_hid_event_queue queue;
+    struct sc_aoa_event_queue queue;
 
     struct sc_acksync *acksync;
 };
@@ -63,6 +57,16 @@ bool
 sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id);
 
 bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event);
+sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
+                                       uint16_t accessory_id,
+                                       const struct sc_hid_event *event,
+                                       uint64_t ack_to_wait);
+
+static inline bool
+sc_aoa_push_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
+                      const struct sc_hid_event *event) {
+    return sc_aoa_push_hid_event_with_ack_to_wait(aoa, accessory_id, event,
+                                                  SC_SEQUENCE_INVALID);
+}
 
 #endif

app/src/usb/hid_mouse.c

@@ -1,267 +0,0 @@
-#include "hid_mouse.h"
-
-#include <assert.h>
-
-#include "input_events.h"
-#include "util/log.h"
-
-/** Downcast mouse processor to hid_mouse */
-#define DOWNCAST(MP) container_of(MP, struct sc_hid_mouse, mouse_processor)
-
-#define HID_MOUSE_ACCESSORY_ID 2
-
-// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position
-#define HID_MOUSE_EVENT_SIZE 4
-
-/**
- * Mouse descriptor from the specification:
- * <https://www.usb.org/sites/default/files/hid1_11.pdf>
- *
- * Appendix E (p71): §E.10 Report Descriptor (Mouse)
- *
- * The usage tags (like Wheel) are listed in "HID Usage Tables":
- * <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
- * §4 Generic Desktop Page (0x01) (p26)
- */
-static const unsigned char mouse_report_desc[]  = {
-    // Usage Page (Generic Desktop)
-    0x05, 0x01,
-    // Usage (Mouse)
-    0x09, 0x02,
-
-    // Collection (Application)
-    0xA1, 0x01,
-
-    // Usage (Pointer)
-    0x09, 0x01,
-
-    // Collection (Physical)
-    0xA1, 0x00,
-
-    // Usage Page (Buttons)
-    0x05, 0x09,
-
-    // Usage Minimum (1)
-    0x19, 0x01,
-     // Usage Maximum (5)
-    0x29, 0x05,
-    // Logical Minimum (0)
-    0x15, 0x00,
-    // Logical Maximum (1)
-    0x25, 0x01,
-    // Report Count (5)
-    0x95, 0x05,
-    // Report Size (1)
-    0x75, 0x01,
-    // Input (Data, Variable, Absolute): 5 buttons bits
-    0x81, 0x02,
-
-    // Report Count (1)
-    0x95, 0x01,
-    // Report Size (3)
-    0x75, 0x03,
-    // Input (Constant): 3 bits padding
-    0x81, 0x01,
-
-    // Usage Page (Generic Desktop)
-    0x05, 0x01,
-    // Usage (X)
-    0x09, 0x30,
-    // Usage (Y)
-    0x09, 0x31,
-    // Usage (Wheel)
-    0x09, 0x38,
-    // Local Minimum (-127)
-    0x15, 0x81,
-    // Local Maximum (127)
-    0x25, 0x7F,
-    // Report Size (8)
-    0x75, 0x08,
-    // Report Count (3)
-    0x95, 0x03,
-    // Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
-    0x81, 0x06,
-
-    // End Collection
-    0xC0,
-
-    // End Collection
-    0xC0,
-};
-
-/**
- * A mouse HID event is 3 bytes long:
- *
- *  - byte 0: buttons state
- *  - byte 1: relative x motion (signed byte from -127 to 127)
- *  - byte 2: relative y motion (signed byte from -127 to 127)
- *
- *                   7 6 5 4 3 2 1 0
- *                  +---------------+
- *         byte 0:  |0 0 0 . . . . .| buttons state
- *                  +---------------+
- *                         ^ ^ ^ ^ ^
- *                         | | | | `- left button
- *                         | | | `--- right button
- *                         | | `----- middle button
- *                         | `------- button 4
- *                         `--------- button 5
- *
- *                  +---------------+
- *         byte 1:  |. . . . . . . .| relative x motion
- *                  +---------------+
- *         byte 2:  |. . . . . . . .| relative y motion
- *                  +---------------+
- *         byte 3:  |. . . . . . . .| wheel motion (-1, 0 or 1)
- *                  +---------------+
- *
- * As an example, here is the report for a motion of (x=5, y=-4) with left
- * button pressed:
- *
- *                  +---------------+
- *                  |0 0 0 0 0 0 0 1| left button pressed
- *                  +---------------+
- *                  |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
- *                  +---------------+
- *                  |1 1 1 1 1 1 0 0| relative y motion (y = -4)
- *                  +---------------+
- *                  |0 0 0 0 0 0 0 0| wheel motion
- *                  +---------------+
- */
-
-static bool
-sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
-    unsigned char *buffer = calloc(1, HID_MOUSE_EVENT_SIZE);
-    if (!buffer) {
-        LOG_OOM();
-        return false;
-    }
-
-    sc_hid_event_init(hid_event, HID_MOUSE_ACCESSORY_ID, buffer,
-                      HID_MOUSE_EVENT_SIZE);
-    return true;
-}
-
-static unsigned char
-buttons_state_to_hid_buttons(uint8_t buttons_state) {
-    unsigned char c = 0;
-    if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
-        c |= 1 << 0;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
-        c |= 1 << 1;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
-        c |= 1 << 2;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_X1) {
-        c |= 1 << 3;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_X2) {
-        c |= 1 << 4;
-    }
-    return c;
-}
-
-static void
-sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
-                                    const struct sc_mouse_motion_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
-    buffer[1] = CLAMP(event->xrel, -127, 127);
-    buffer[2] = CLAMP(event->yrel, -127, 127);
-    buffer[3] = 0; // wheel coordinates only used for scrolling
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse motion)");
-    }
-}
-
-static void
-sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
-                                   const struct sc_mouse_click_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
-    buffer[1] = 0; // no x motion
-    buffer[2] = 0; // no y motion
-    buffer[3] = 0; // wheel coordinates only used for scrolling
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse click)");
-    }
-}
-
-static void
-sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
-                                    const struct sc_mouse_scroll_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = 0; // buttons state irrelevant (and unknown)
-    buffer[1] = 0; // no x motion
-    buffer[2] = 0; // no y motion
-    // In practice, vscroll is always -1, 0 or 1, but in theory other values
-    // are possible
-    buffer[3] = CLAMP(event->vscroll, -127, 127);
-    // Horizontal scrolling ignored
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse scroll)");
-    }
-}
-
-bool
-sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa) {
-    mouse->aoa = aoa;
-
-    bool ok = sc_aoa_setup_hid(aoa, HID_MOUSE_ACCESSORY_ID, mouse_report_desc,
-                               ARRAY_LEN(mouse_report_desc));
-    if (!ok) {
-        LOGW("Register HID mouse failed");
-        return false;
-    }
-
-    static const struct sc_mouse_processor_ops ops = {
-        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
-        .process_mouse_click = sc_mouse_processor_process_mouse_click,
-        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
-        // Touch events not supported (coordinates are not relative)
-        .process_touch = NULL,
-    };
-
-    mouse->mouse_processor.ops = &ops;
-
-    mouse->mouse_processor.relative_mode = true;
-
-    return true;
-}
-
-void
-sc_hid_mouse_destroy(struct sc_hid_mouse *mouse) {
-    bool ok = sc_aoa_unregister_hid(mouse->aoa, HID_MOUSE_ACCESSORY_ID);
-    if (!ok) {
-        LOGW("Could not unregister HID mouse");
-    }
-}

app/src/usb/keyboard_aoa.c

@@ -0,0 +1,110 @@
+#include "keyboard_aoa.h"
+
+#include <assert.h>
+
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast key processor to keyboard_aoa */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_aoa, key_processor)
+
+#define HID_KEYBOARD_ACCESSORY_ID 1
+
+static bool
+push_mod_lock_state(struct sc_keyboard_aoa *kb, uint16_t mods_state) {
+    struct sc_hid_event hid_event;
+    if (!sc_hid_keyboard_event_from_mods(&hid_event, mods_state)) {
+        // Nothing to do
+        return true;
+    }
+
+    if (!sc_aoa_push_hid_event(kb->aoa, HID_KEYBOARD_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mod lock state)");
+        return false;
+    }
+
+    LOGD("HID keyboard state synchronized");
+
+    return true;
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const struct sc_key_event *event,
+                             uint64_t ack_to_wait) {
+    if (event->repeat) {
+        // In USB HID protocol, key repeat is handled by the host (Android), so
+        // just ignore key repeat here.
+        return;
+    }
+
+    struct sc_keyboard_aoa *kb = DOWNCAST(kp);
+
+    struct sc_hid_event hid_event;
+
+    // Not all keys are supported, just ignore unsupported keys
+    if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
+        if (!kb->mod_lock_synchronized) {
+            // Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
+            // keyboard state
+            if (push_mod_lock_state(kb, event->mods_state)) {
+                kb->mod_lock_synchronized = true;
+            }
+        }
+
+        // If ack_to_wait is != SC_SEQUENCE_INVALID, then Ctrl+v is pressed, so
+        // clipboard synchronization has been requested. Wait until clipboard
+        // synchronization is acknowledged by the server, otherwise it could
+        // paste the old clipboard content.
+
+        if (!sc_aoa_push_hid_event_with_ack_to_wait(kb->aoa,
+                                                    HID_KEYBOARD_ACCESSORY_ID,
+                                                    &hid_event,
+                                                    ack_to_wait)) {
+            LOGW("Could not request HID event (key)");
+        }
+    }
+}
+
+bool
+sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa) {
+    kb->aoa = aoa;
+
+    bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
+                               SC_HID_KEYBOARD_REPORT_DESC,
+                               SC_HID_KEYBOARD_REPORT_DESC_LEN);
+    if (!ok) {
+        LOGW("Register HID keyboard failed");
+        return false;
+    }
+
+    sc_hid_keyboard_init(&kb->hid);
+
+    kb->mod_lock_synchronized = false;
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        // Never forward text input via HID (all the keys are injected
+        // separately)
+        .process_text = NULL,
+    };
+
+    // Clipboard synchronization is requested over the control socket, while HID
+    // events are sent over AOA, so it must wait for clipboard synchronization
+    // to be acknowledged by the device before injecting Ctrl+v.
+    kb->key_processor.async_paste = true;
+    kb->key_processor.hid = true;
+    kb->key_processor.ops = &ops;
+
+    return true;
+}
+
+void
+sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb) {
+    // Unregister HID keyboard so the soft keyboard shows again on Android
+    bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
+    if (!ok) {
+        LOGW("Could not unregister HID keyboard");
+    }
+}

app/src/usb/keyboard_aoa.h

@@ -0,0 +1,27 @@
+#ifndef SC_KEYBOARD_AOA_H
+#define SC_KEYBOARD_AOA_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "aoa_hid.h"
+#include "hid/hid_keyboard.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_aoa {
+    struct sc_key_processor key_processor; // key processor trait
+
+    struct sc_hid_keyboard hid;
+    struct sc_aoa *aoa;
+
+    bool mod_lock_synchronized;
+};
+
+bool
+sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa);
+
+void
+sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb);
+
+#endif

app/src/usb/mouse_aoa.c

@@ -0,0 +1,89 @@
+#include "mouse_aoa.h"
+
+#include <assert.h>
+
+#include "hid/hid_mouse.h"
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to mouse_aoa */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_aoa, mouse_processor)
+
+#define HID_MOUSE_ACCESSORY_ID 2
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_motion_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_motion(&hid_event, event);
+
+    if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mouse motion)");
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
+                                   const struct sc_mouse_click_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_click(&hid_event, event);
+
+    if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mouse click)");
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_scroll_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_scroll(&hid_event, event);
+
+    if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mouse scroll)");
+    }
+}
+
+bool
+sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa) {
+    mouse->aoa = aoa;
+
+    bool ok = sc_aoa_setup_hid(aoa, HID_MOUSE_ACCESSORY_ID,
+                               SC_HID_MOUSE_REPORT_DESC,
+                               SC_HID_MOUSE_REPORT_DESC_LEN);
+    if (!ok) {
+        LOGW("Register HID mouse failed");
+        return false;
+    }
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_mouse_click = sc_mouse_processor_process_mouse_click,
+        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
+        // Touch events not supported (coordinates are not relative)
+        .process_touch = NULL,
+    };
+
+    mouse->mouse_processor.ops = &ops;
+
+    mouse->mouse_processor.relative_mode = true;
+
+    return true;
+}
+
+void
+sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse) {
+    bool ok = sc_aoa_unregister_hid(mouse->aoa, HID_MOUSE_ACCESSORY_ID);
+    if (!ok) {
+        LOGW("Could not unregister HID mouse");
+    }
+}

app/src/usb/mouse_aoa.h

@@ -1,5 +1,5 @@
-#ifndef SC_HID_MOUSE_H
-#define SC_HID_MOUSE_H
+#ifndef SC_MOUSE_AOA_H
+#define SC_MOUSE_AOA_H
 
 #include "common.h"
 
@@ -8,16 +8,16 @@
 #include "aoa_hid.h"
 #include "trait/mouse_processor.h"
 
-struct sc_hid_mouse {
+struct sc_mouse_aoa {
     struct sc_mouse_processor mouse_processor; // mouse processor trait
 
     struct sc_aoa *aoa;
 };
 
 bool
-sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa);
+sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa);
 
 void
-sc_hid_mouse_destroy(struct sc_hid_mouse *mouse);
+sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse);
 
 #endif

app/src/usb/scrcpy_otg.c

@@ -10,8 +10,8 @@
 struct scrcpy_otg {
     struct sc_usb usb;
     struct sc_aoa aoa;
-    struct sc_hid_keyboard keyboard;
-    struct sc_hid_mouse mouse;
+    struct sc_keyboard_aoa keyboard;
+    struct sc_mouse_aoa mouse;
 
     struct sc_screen_otg screen_otg;
 };
@@ -73,8 +73,8 @@ scrcpy_otg(struct scrcpy_options *options) {
 
     enum scrcpy_exit_code ret = SCRCPY_EXIT_FAILURE;
 
-    struct sc_hid_keyboard *keyboard = NULL;
-    struct sc_hid_mouse *mouse = NULL;
+    struct sc_keyboard_aoa *keyboard = NULL;
+    struct sc_mouse_aoa *mouse = NULL;
     bool usb_device_initialized = false;
     bool usb_connected = false;
     bool aoa_started = false;
@@ -117,19 +117,18 @@ scrcpy_otg(struct scrcpy_options *options) {
     }
     aoa_initialized = true;
 
-    bool enable_keyboard =
-        options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
-    bool enable_mouse =
-        options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
+    assert(options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA
+        || options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_DISABLED);
+    assert(options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA
+        || options->mouse_input_mode == SC_MOUSE_INPUT_MODE_DISABLED);
 
-    // If neither --hid-keyboard or --hid-mouse is passed, enable both
-    if (!enable_keyboard && !enable_mouse) {
-        enable_keyboard = true;
-        enable_mouse = true;
-    }
+    bool enable_keyboard =
+        options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
+    bool enable_mouse =
+        options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
 
     if (enable_keyboard) {
-        ok = sc_hid_keyboard_init(&s->keyboard, &s->aoa);
+        ok = sc_keyboard_aoa_init(&s->keyboard, &s->aoa);
         if (!ok) {
             goto end;
         }
@@ -137,7 +136,7 @@ scrcpy_otg(struct scrcpy_options *options) {
     }
 
     if (enable_mouse) {
-        ok = sc_hid_mouse_init(&s->mouse, &s->aoa);
+        ok = sc_mouse_aoa_init(&s->mouse, &s->aoa);
         if (!ok) {
             goto end;
         }
@@ -186,10 +185,10 @@ end:
     sc_usb_stop(&s->usb);
 
     if (mouse) {
-        sc_hid_mouse_destroy(&s->mouse);
+        sc_mouse_aoa_destroy(&s->mouse);
     }
     if (keyboard) {
-        sc_hid_keyboard_destroy(&s->keyboard);
+        sc_keyboard_aoa_destroy(&s->keyboard);
     }
 
     if (aoa_initialized) {

app/src/usb/screen_otg.h

@@ -6,12 +6,12 @@
 #include <stdbool.h>
 #include <SDL2/SDL.h>
 
-#include "hid_keyboard.h"
-#include "hid_mouse.h"
+#include "keyboard_aoa.h"
+#include "mouse_aoa.h"
 
 struct sc_screen_otg {
-    struct sc_hid_keyboard *keyboard;
-    struct sc_hid_mouse *mouse;
+    struct sc_keyboard_aoa *keyboard;
+    struct sc_mouse_aoa *mouse;
 
     SDL_Window *window;
     SDL_Renderer *renderer;
@@ -22,8 +22,8 @@ struct sc_screen_otg {
 };
 
 struct sc_screen_otg_params {
-    struct sc_hid_keyboard *keyboard;
-    struct sc_hid_mouse *mouse;
+    struct sc_keyboard_aoa *keyboard;
+    struct sc_mouse_aoa *mouse;
 
     const char *window_title;
     bool always_on_top;

app/src/util/str.c

@@ -3,6 +3,7 @@
 #include <assert.h>
 #include <errno.h>
 #include <limits.h>
+#include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 
@@ -333,3 +334,22 @@ sc_str_remove_trailing_cr(char *s, size_t len) {
     }
     return len;
 }
+
+char *
+sc_str_to_hex_string(const uint8_t *data, size_t size) {
+    size_t buffer_size = size * 3 + 1;
+    char *buffer = malloc(buffer_size);
+    if (!buffer) {
+        LOG_OOM();
+        return NULL;
+    }
+
+    for (size_t i = 0; i < size; ++i) {
+        snprintf(buffer + i * 3, 4, "%02X ", data[i]);
+    }
+
+    // Remove the final space
+    buffer[size * 3] = '\0';
+
+    return buffer;
+}

app/src/util/str.h

@@ -138,4 +138,10 @@ sc_str_index_of_column(const char *s, unsigned col, const char *seps);
 size_t
 sc_str_remove_trailing_cr(char *s, size_t len);
 
+/**
+ * Convert binary data to hexadecimal string
+ */
+char *
+sc_str_to_hex_string(const uint8_t *data, size_t len);
+
 #endif

app/tests/test_control_msg_serialize.c

@@ -323,6 +323,68 @@ static void test_serialize_rotate_device(void) {
     assert(!memcmp(buf, expected, sizeof(expected)));
 }
 
+static void test_serialize_uhid_create(void) {
+    const uint8_t report_desc[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11};
+    struct sc_control_msg msg = {
+        .type = SC_CONTROL_MSG_TYPE_UHID_CREATE,
+        .uhid_create = {
+            .id = 42,
+            .report_desc_size = sizeof(report_desc),
+            .report_desc = report_desc,
+        },
+    };
+
+    uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
+    size_t size = sc_control_msg_serialize(&msg, buf);
+    assert(size == 16);
+
+    const uint8_t expected[] = {
+        SC_CONTROL_MSG_TYPE_UHID_CREATE,
+        0, 42, // id
+        0, 11, // size
+        1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
+    };
+    assert(!memcmp(buf, expected, sizeof(expected)));
+}
+
+static void test_serialize_uhid_input(void) {
+    struct sc_control_msg msg = {
+        .type = SC_CONTROL_MSG_TYPE_UHID_INPUT,
+        .uhid_input = {
+            .id = 42,
+            .size = 5,
+            .data = {1, 2, 3, 4, 5},
+        },
+    };
+
+    uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
+    size_t size = sc_control_msg_serialize(&msg, buf);
+    assert(size == 10);
+
+    const uint8_t expected[] = {
+        SC_CONTROL_MSG_TYPE_UHID_INPUT,
+        0, 42, // id
+        0, 5, // size
+        1, 2, 3, 4, 5,
+    };
+    assert(!memcmp(buf, expected, sizeof(expected)));
+}
+
+static void test_serialize_open_hard_keyboard(void) {
+    struct sc_control_msg msg = {
+        .type = SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
+    };
+
+    uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
+    size_t size = sc_control_msg_serialize(&msg, buf);
+    assert(size == 1);
+
+    const uint8_t expected[] = {
+        SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
+    };
+    assert(!memcmp(buf, expected, sizeof(expected)));
+}
+
 int main(int argc, char *argv[]) {
     (void) argc;
     (void) argv;
@@ -341,5 +403,8 @@ int main(int argc, char *argv[]) {
     test_serialize_set_clipboard_long();
     test_serialize_set_screen_power_mode();
     test_serialize_rotate_device();
+    test_serialize_uhid_create();
+    test_serialize_uhid_input();
+    test_serialize_open_hard_keyboard();
     return 0;
 }

app/tests/test_device_msg_deserialize.c

@@ -61,6 +61,28 @@ static void test_deserialize_ack_set_clipboard(void) {
     assert(msg.ack_clipboard.sequence == UINT64_C(0x0102030405060708));
 }
 
+static void test_deserialize_uhid_output(void) {
+    const uint8_t input[] = {
+        DEVICE_MSG_TYPE_UHID_OUTPUT,
+        0, 42, // id
+        0, 5, // size
+        0x01, 0x02, 0x03, 0x04, 0x05, // data
+    };
+
+    struct sc_device_msg msg;
+    ssize_t r = sc_device_msg_deserialize(input, sizeof(input), &msg);
+    assert(r == 10);
+
+    assert(msg.type == DEVICE_MSG_TYPE_UHID_OUTPUT);
+    assert(msg.uhid_output.id == 42);
+    assert(msg.uhid_output.size == 5);
+
+    uint8_t expected[] = {1, 2, 3, 4, 5};
+    assert(!memcmp(msg.uhid_output.data, expected, sizeof(expected)));
+
+    sc_device_msg_destroy(&msg);
+}
+
 int main(int argc, char *argv[]) {
     (void) argc;
     (void) argv;
@@ -68,5 +90,6 @@ int main(int argc, char *argv[]) {
     test_deserialize_clipboard();
     test_deserialize_clipboard_big();
     test_deserialize_ack_set_clipboard();
+    test_deserialize_uhid_output();
     return 0;
 }

doc/control.md

@@ -10,36 +10,9 @@ scrcpy --no-control
 scrcpy -n   # short version
 ```
 
+## Keyboard and mouse
 
-## Text injection preference
-
-Two kinds of [events][textevents] are generated when typing text:
- - _key events_, signaling that a key is pressed or released;
- - _text events_, signaling that a text has been entered.
-
-By default, letters are injected using key events, so that the keyboard behaves
-as expected in games (typically for WASD keys).
-
-But this may [cause issues][prefertext]. If you encounter such a problem, you
-can avoid it by:
-
-```bash
-scrcpy --prefer-text
-```
-
-(but this will break keyboard behavior in games)
-
-On the contrary, you could force to always inject raw key events:
-
-```bash
-scrcpy --raw-key-events
-```
-
-These options have no effect on HID keyboard (all key events are sent as
-scancodes in this mode).
-
-[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
-[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
+Read [keyboard](keyboard.md) and [mouse](mouse.md).
 
 
 ## Copy-paste
@@ -85,6 +58,7 @@ way as <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>v</kbd>).
 To disable automatic clipboard synchronization, use
 `--no-clipboard-autosync`.
 
+
 ## Pinch-to-zoom, rotate and tilt simulation
 
 To simulate "pinch-to-zoom": <kbd>Ctrl</kbd>+_click-and-move_.
@@ -100,20 +74,7 @@ at a location inverted through the center of the screen. When pressing
 <kbd>Ctrl</kbd> the x and y coordinates are inverted. Using <kbd>Shift</kbd>
 only inverts x.
 
-
-## Key repeat
-
-By default, holding a key down generates repeated key events. This can cause
-performance problems in some games, where these events are useless anyway.
-
-To avoid forwarding repeated key events:
-
-```bash
-scrcpy --no-key-repeat
-```
-
-This option has no effect on HID keyboard (key repeat is handled by Android
-directly in this mode).
+This only works for the default mouse mode (`--mouse=sdk`).
 
 
 ## Right-click and middle-click
@@ -147,7 +108,3 @@ The target directory can be changed on start:
 ```bash
 scrcpy --push-target=/sdcard/Movies/
 ```
-
-## Physical keyboard and mouse simulation
-
-See the dedicated [HID/OTG](hid-otg.md) page.

doc/develop.md

@@ -234,7 +234,7 @@ The video and audio streams are decoded by [FFmpeg].
 The client parses the command line arguments, then [runs one of two code
 paths][run]:
  - scrcpy in "normal" mode ([`scrcpy.c`])
- - scrcpy in [OTG mode](hid-otg.md) ([`scrcpy_otg.c`])
+ - scrcpy in [OTG mode](otg.md) ([`scrcpy_otg.c`])
 
 [run]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/main.c#L81-L82
 [`scrcpy.c`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/scrcpy.c#L292-L293

doc/hid-otg.md

@@ -1,112 +0,0 @@
-# HID/OTG
-
-By default, _scrcpy_ injects input events at the Android API level. As an
-alternative, when connected over USB, it is possible to send HID events, so that
-scrcpy behaves as if it was a physical keyboard and/or mouse connected to the
-Android device.
-
-A special [OTG](#otg) mode allows to control the device without mirroring (and
-without USB debugging).
-
-
-## Physical keyboard simulation
-
-By default, _scrcpy_ uses Android key or text injection. It works everywhere,
-but is limited to ASCII.
-
-Instead, it can simulate a physical USB keyboard on Android to provide a better
-input experience (using [USB HID over AOAv2][hid-aoav2]): the virtual keyboard
-is disabled and it works for all characters and IME.
-
-[hid-aoav2]: https://source.android.com/devices/accessories/aoa2#hid-support
-
-However, it only works if the device is connected via USB.
-
-Note: On Windows, it may only work in [OTG mode](#otg), not while mirroring (it
-is not possible to open a USB device if it is already open by another process
-like the _adb daemon_).
-
-To enable this mode:
-
-```bash
-scrcpy --hid-keyboard
-scrcpy -K   # short version
-```
-
-If it fails for some reason (for example because the device is not connected via
-USB), it automatically fallbacks to the default mode (with a log in the
-console). This allows using the same command line options when connected over
-USB and TCP/IP.
-
-In this mode, raw key events (scancodes) are sent to the device, independently
-of the host key mapping. Therefore, if your keyboard layout does not match, it
-must be configured on the Android device, in Settings → System → Languages and
-input → [Physical keyboard].
-
-This settings page can be started directly:
-
-```bash
-adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
-```
-
-However, the option is only available when the HID keyboard is enabled (or when
-a physical keyboard is connected).
-
-[Physical keyboard]: https://github.com/Genymobile/scrcpy/pull/2632#issuecomment-923756915
-
-
-## Physical mouse simulation
-
-By default, _scrcpy_ uses Android mouse events injection with absolute
-coordinates. By simulating a physical mouse, a mouse pointer appears on the
-Android device, and relative mouse motion, clicks and scrolls are injected.
-
-To enable this mode:
-
-```bash
-scrcpy --hid-mouse
-scrcpy -M   # short version
-```
-
-When this mode is enabled, the computer mouse is "captured" (the mouse pointer
-disappears from the computer and appears on the Android device instead).
-
-Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
-(disable or enable) the mouse capture. Use one of them to give the control of
-the mouse back to the computer.
-
-
-## OTG
-
-It is possible to run _scrcpy_ with only physical keyboard and mouse simulation
-(HID), as if the computer keyboard and mouse were plugged directly to the device
-via an OTG cable.
-
-In this mode, `adb` (USB debugging) is not necessary, and mirroring is disabled.
-
-This is similar to `--hid-keyboard --hid-mouse`, but without mirroring.
-
-To enable OTG mode:
-
-```bash
-scrcpy --otg
-# Pass the serial if several USB devices are available
-scrcpy --otg -s 0123456789abcdef
-```
-
-It is possible to enable only HID keyboard or HID mouse:
-
-```bash
-scrcpy --otg --hid-keyboard              # keyboard only
-scrcpy --otg --hid-mouse                 # mouse only
-scrcpy --otg --hid-keyboard --hid-mouse  # keyboard and mouse
-# for convenience, enable both by default
-scrcpy --otg                             # keyboard and mouse
-```
-
-Like `--hid-keyboard` and `--hid-mouse`, it only works if the device is
-connected over USB.
-
-## HID/OTG issues on Windows
-
-See [FAQ](/FAQ.md#hidotg-issues-on-windows).

doc/keyboard.md

@@ -0,0 +1,136 @@
+# Keyboard
+
+Several keyboard input modes are available:
+
+ - `--keyboard=sdk` (default)
+ - `--keyboard=uhid` (or `-K`): simulates a physical HID keyboard using the UHID
+   kernel module on the device
+ - `--keyboard=aoa`: simulates a physical HID keyboard using the AOAv2 protocol
+ - `--keyboard=disabled`
+
+By default, `sdk` is used, but if you use scrcpy regularly, it is recommended to
+use [`uhid`](#uhid) and configure the keyboard layout once and for all.
+
+
+## SDK keyboard
+
+In this mode (`--keyboard=sdk`, or if the parameter is omitted), keyboard input
+events are injected at the Android API level. It works everywhere, but it is
+limited to ASCII and some other characters.
+
+Note that on some devices, an additional option must be enabled in developer
+options for this keyboard mode to work. See
+[prerequisites](/README.md#prerequisites).
+
+Additional parameters (specific to `--keyboard=sdk`) described below allow to
+customize the behavior.
+
+
+### Text injection preference
+
+Two kinds of [events][textevents] are generated when typing text:
+ - _key events_, signaling that a key is pressed or released;
+ - _text events_, signaling that a text has been entered.
+
+By default, numbers and "special characters" are inserted using text events, but
+letters are injected using key events, so that the keyboard behaves as expected
+in games (typically for WASD keys).
+
+But this may [cause issues][prefertext]. If you encounter such a problem, you
+can inject letters as text (or just switch to [UHID](#uhid)):
+
+```bash
+scrcpy --prefer-text
+```
+
+(but this will break keyboard behavior in games)
+
+On the contrary, you could force to always inject raw key events:
+
+```bash
+scrcpy --raw-key-events
+```
+
+[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
+[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
+
+
+### Key repeat
+
+By default, holding a key down generates repeated key events. Ths can cause
+performance problems in some games, where these events are useless anyway.
+
+To avoid forwarding repeated key events:
+
+```bash
+scrcpy --no-key-repeat
+```
+
+
+## Physical keyboard simulation
+
+Two modes allow to simulate a physical HID keyboard on the device.
+
+To work properly, it is necessary to configure (once and for all) the keyboard
+layout on the device to match that of the computer.
+
+The configuration page can be opened in one of the following ways:
+ - from the scrcpy window (when `uhid` or `aoa` is used), by pressing
+   <kbd>MOD</kbd>+<kbd>k</kbd> (see [shortcuts](shortcuts.md))
+ - from the device, in Settings → System → Languages and input → Physical
+   devices
+ - from a terminal on the computer, by executing `adb shell am start -a
+   android.settings.HARD_KEYBOARD_SETTINGS`
+
+From this configuration page, it is also possible to enable or disable on-screen
+keyboard.
+
+
+### UHID
+
+This mode simulates a physical HID keyboard using the [UHID] kernel module on the
+device.
+
+[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
+
+To enable UHID keyboard, use:
+
+```bash
+scrcpy --keyboard=uhid
+scrcpy -K  # short version
+```
+
+Once the keyboard layout is configured (see above), it is the best mode for
+using the keyboard while mirroring:
+
+ - it works for all characters and IME (contrary to `--keyboard=sdk`)
+ - the on-screen keyboard can be disabled (contrary to `--keyboard=sdk`)
+ - it works over TCP/IP (wirelessly) (contrary to `--keyboard=aoa`)
+ - there are no issues on Windows (contrary to `--keyboard=aoa`)
+
+One drawback is that it may not work on old Android versions due to permission
+errors.
+
+
+### AOA
+
+This mode simulates a physical HID keyboard using the [AOAv2] protocol.
+
+[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
+
+To enable AOA keyboard, use:
+
+```bash
+scrcpy --keyboard=aoa
+```
+
+Contrary to the other modes, it works at the USB level directly (so it only
+works over USB).
+
+It does not use the scrcpy server, and does not require `adb` (USB debugging).
+Therefore, it is possible to control the device (but not mirror) even with USB
+debugging disabled (see [OTG](otg.md)).
+
+Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
+(it is not possible to open a USB device if it is already open by another
+process like the _adb daemon_).

doc/mouse.md

@@ -0,0 +1,70 @@
+# Mouse
+
+Several mouse input modes are available:
+
+ - `--mouse=sdk` (default)
+ - `--mouse=uhid` (or `-M`): simulates a physical HID mouse using the UHID
+   kernel module on the device
+ - `--mouse=aoa`: simulates a physical HID mouse using the AOAv2 protocol
+ - `--mouse=disabled`
+
+
+## SDK mouse
+
+In this mode (`--mouse=sdk`, or if the parameter is omitted), mouse input events
+are injected at the Android API level with absolute coordinates.
+
+Note that on some devices, an additional option must be enabled in developer
+options for this mouse mode to work. See
+[prerequisites](/README.md#prerequisites).
+
+
+## Physical mouse simulation
+
+Two modes allow to simulate a physical HID mouse on the device.
+
+In these modes, the computer mouse is "captured": the mouse pointer disapeears
+from the computer and appears on the Android device instead.
+
+Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
+(disable or enable) the mouse capture. Use one of them to give the control of
+the mouse back to the computer.
+
+
+### UHID
+
+This mode simulates a physical HID mouse using the [UHID] kernel module on the
+device.
+
+[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
+
+To enable UHID mouse, use:
+
+```bash
+scrcpy --mouse=uhid
+scrcpy -M  # short version
+```
+
+
+### AOA
+
+This mode simulates a physical HID mouse using the [AOAv2] protocol.
+
+[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
+
+To enable AOA mouse, use:
+
+```bash
+scrcpy --mouse=aoa
+```
+
+Contrary to the other modes, it works at the USB level directly (so it only
+works over USB).
+
+It does not use the scrcpy server, and does not require `adb` (USB debugging).
+Therefore, it is possible to control the device (but not mirror) even with USB
+debugging disabled (see [OTG](otg.md)).
+
+Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
+(it is not possible to open a USB device if it is already open by another
+process like the _adb daemon_).

doc/otg.md

@@ -0,0 +1,37 @@
+# OTG
+
+By default, _scrcpy_ injects input events at the Android API level. As an
+alternative, when connected over USB, it is possible to send HID events, so that
+scrcpy behaves as if it was a physical keyboard and/or mouse connected to the
+Android device.
+
+A special mode allows to control the device without mirroring, using AOA
+[keyboard](keyboard.md#aoa) and [mouse](mouse.md#aoa). Therefore, it is possible
+to run _scrcpy_ with only physical keyboard and mouse simulation (HID), as if
+the computer keyboard and mouse were plugged directly to the device via an OTG
+cable.
+
+In this mode, `adb` (USB debugging) is not necessary, and mirroring is disabled.
+
+This is similar to `--keyboard=aoa --mouse=aoa`, but without mirroring.
+
+To enable OTG mode:
+
+```bash
+scrcpy --otg
+# Pass the serial if several USB devices are available
+scrcpy --otg -s 0123456789abcdef
+```
+
+It is possible to disable HID keyboard or HID mouse:
+
+```bash
+scrcpy --otg --keyboard=disabled
+scrcpy --otg --mouse=disabled
+```
+
+It only works if the device is connected over USB.
+
+## OTG issues on Windows
+
+See [FAQ](/FAQ.md#hidotg-issues-on-windows).

doc/shortcuts.md

@@ -48,6 +48,7 @@ _<kbd>[Super]</kbd> is typically the <kbd>Windows</kbd> or <kbd>Cmd</kbd> key._
  | Cut to clipboard⁵                           | <kbd>MOD</kbd>+<kbd>x</kbd>
  | Synchronize clipboards and paste⁵           | <kbd>MOD</kbd>+<kbd>v</kbd>
  | Inject computer clipboard text              | <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>v</kbd>
+ | Open keyboard settings (HID keyboard only)  | <kbd>MOD</kbd>+<kbd>k</kbd>
  | Enable/disable FPS counter (on stdout)      | <kbd>MOD</kbd>+<kbd>i</kbd>
  | Pinch-to-zoom/rotate                        | <kbd>Ctrl</kbd>+_click-and-move_
  | Tilt (slide vertically with 2 fingers)      | <kbd>Shift</kbd>+_click-and-move_

server/src/main/java/com/genymobile/scrcpy/AudioCapture.java

@@ -79,7 +79,7 @@ public final class AudioCapture {
         intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
         intent.addCategory(Intent.CATEGORY_LAUNCHER);
         intent.setComponent(new ComponentName(FakeContext.PACKAGE_NAME, "com.android.shell.HeapDumpActivity"));
-        ServiceManager.getActivityManager().startActivityAsUserWithFeature(intent);
+        ServiceManager.getActivityManager().startActivity(intent);
     }
 
     private static void stopWorkaroundAndroid11() {

server/src/main/java/com/genymobile/scrcpy/ControlMessage.java

@@ -17,6 +17,9 @@ public final class ControlMessage {
     public static final int TYPE_SET_CLIPBOARD = 9;
     public static final int TYPE_SET_SCREEN_POWER_MODE = 10;
     public static final int TYPE_ROTATE_DEVICE = 11;
+    public static final int TYPE_UHID_CREATE = 12;
+    public static final int TYPE_UHID_INPUT = 13;
+    public static final int TYPE_OPEN_HARD_KEYBOARD_SETTINGS = 14;
 
     public static final long SEQUENCE_INVALID = 0;
 
@@ -40,6 +43,8 @@ public final class ControlMessage {
     private boolean paste;
     private int repeat;
     private long sequence;
+    private int id;
+    private byte[] data;
 
     private ControlMessage() {
     }
@@ -123,6 +128,22 @@ public final class ControlMessage {
         return msg;
     }
 
+    public static ControlMessage createUhidCreate(int id, byte[] reportDesc) {
+        ControlMessage msg = new ControlMessage();
+        msg.type = TYPE_UHID_CREATE;
+        msg.id = id;
+        msg.data = reportDesc;
+        return msg;
+    }
+
+    public static ControlMessage createUhidInput(int id, byte[] data) {
+        ControlMessage msg = new ControlMessage();
+        msg.type = TYPE_UHID_INPUT;
+        msg.id = id;
+        msg.data = data;
+        return msg;
+    }
+
     public int getType() {
         return type;
     }
@@ -186,4 +207,12 @@ public final class ControlMessage {
     public long getSequence() {
         return sequence;
     }
+
+    public int getId() {
+        return id;
+    }
+
+    public byte[] getData() {
+        return data;
+    }
 }

server/src/main/java/com/genymobile/scrcpy/ControlMessageReader.java

@@ -15,6 +15,8 @@ public class ControlMessageReader {
     static final int SET_SCREEN_POWER_MODE_PAYLOAD_LENGTH = 1;
     static final int GET_CLIPBOARD_LENGTH = 1;
     static final int SET_CLIPBOARD_FIXED_PAYLOAD_LENGTH = 9;
+    static final int UHID_CREATE_FIXED_PAYLOAD_LENGTH = 4;
+    static final int UHID_INPUT_FIXED_PAYLOAD_LENGTH = 4;
 
     private static final int MESSAGE_MAX_SIZE = 1 << 18; // 256k
 
@@ -84,8 +86,15 @@ public class ControlMessageReader {
             case ControlMessage.TYPE_EXPAND_SETTINGS_PANEL:
             case ControlMessage.TYPE_COLLAPSE_PANELS:
             case ControlMessage.TYPE_ROTATE_DEVICE:
+            case ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
                 msg = ControlMessage.createEmpty(type);
                 break;
+            case ControlMessage.TYPE_UHID_CREATE:
+                msg = parseUhidCreate();
+                break;
+            case ControlMessage.TYPE_UHID_INPUT:
+                msg = parseUhidInput();
+                break;
             default:
                 Ln.w("Unknown event type: " + type);
                 msg = null;
@@ -110,12 +119,21 @@ public class ControlMessageReader {
         return ControlMessage.createInjectKeycode(action, keycode, repeat, metaState);
     }
 
-    private String parseString() {
-        if (buffer.remaining() < 4) {
-            return null;
+    private int parseBufferLength(int sizeBytes) {
+        assert sizeBytes > 0 && sizeBytes <= 4;
+        if (buffer.remaining() < sizeBytes) {
+            return -1;
         }
-        int len = buffer.getInt();
-        if (buffer.remaining() < len) {
+        int value = 0;
+        for (int i = 0; i < sizeBytes; ++i) {
+            value = (value << 8) | (buffer.get() & 0xFF);
+        }
+        return value;
+    }
+
+    private String parseString() {
+        int len = parseBufferLength(4);
+        if (len == -1 || buffer.remaining() < len) {
             return null;
         }
         int position = buffer.position();
@@ -124,6 +142,16 @@ public class ControlMessageReader {
         return new String(rawBuffer, position, len, StandardCharsets.UTF_8);
     }
 
+    private byte[] parseByteArray(int sizeBytes) {
+        int len = parseBufferLength(sizeBytes);
+        if (len == -1 || buffer.remaining() < len) {
+            return null;
+        }
+        byte[] data = new byte[len];
+        buffer.get(data);
+        return data;
+    }
+
     private ControlMessage parseInjectText() {
         String text = parseString();
         if (text == null) {
@@ -193,6 +221,30 @@ public class ControlMessageReader {
         return ControlMessage.createSetScreenPowerMode(mode);
     }
 
+    private ControlMessage parseUhidCreate() {
+        if (buffer.remaining() < UHID_CREATE_FIXED_PAYLOAD_LENGTH) {
+            return null;
+        }
+        int id = buffer.getShort();
+        byte[] data = parseByteArray(2);
+        if (data == null) {
+            return null;
+        }
+        return ControlMessage.createUhidCreate(id, data);
+    }
+
+    private ControlMessage parseUhidInput() {
+        if (buffer.remaining() < UHID_INPUT_FIXED_PAYLOAD_LENGTH) {
+            return null;
+        }
+        int id = buffer.getShort();
+        byte[] data = parseByteArray(2);
+        if (data == null) {
+            return null;
+        }
+        return ControlMessage.createUhidInput(id, data);
+    }
+
     private static Position readPosition(ByteBuffer buffer) {
         int x = buffer.getInt();
         int y = buffer.getInt();

server/src/main/java/com/genymobile/scrcpy/Controller.java

@@ -1,7 +1,9 @@
 package com.genymobile.scrcpy;
 
 import com.genymobile.scrcpy.wrappers.InputManager;
+import com.genymobile.scrcpy.wrappers.ServiceManager;
 
+import android.content.Intent;
 import android.os.Build;
 import android.os.SystemClock;
 import android.view.InputDevice;
@@ -26,6 +28,8 @@ public class Controller implements AsyncProcessor {
 
     private Thread thread;
 
+    private UhidManager uhidManager;
+
     private final Device device;
     private final ControlChannel controlChannel;
     private final CleanUp cleanUp;
@@ -52,6 +56,13 @@ public class Controller implements AsyncProcessor {
         sender = new DeviceMessageSender(controlChannel);
     }
 
+    private UhidManager getUhidManager() {
+        if (uhidManager == null) {
+            uhidManager = new UhidManager(sender);
+        }
+        return uhidManager;
+    }
+
     private void initPointers() {
         for (int i = 0; i < PointersState.MAX_POINTERS; ++i) {
             MotionEvent.PointerProperties props = new MotionEvent.PointerProperties();
@@ -81,8 +92,9 @@ public class Controller implements AsyncProcessor {
             SystemClock.sleep(500);
         }
 
-        while (!Thread.currentThread().isInterrupted()) {
-            handleEvent();
+        boolean alive = true;
+        while (!Thread.currentThread().isInterrupted() && alive) {
+            alive = handleEvent();
         }
     }
 
@@ -92,9 +104,12 @@ public class Controller implements AsyncProcessor {
             try {
                 control();
             } catch (IOException e) {
-                // this is expected on close
+                Ln.e("Controller error", e);
             } finally {
                 Ln.d("Controller stopped");
+                if (uhidManager != null) {
+                    uhidManager.closeAll();
+                }
                 listener.onTerminated(true);
             }
         }, "control-recv");
@@ -122,8 +137,15 @@ public class Controller implements AsyncProcessor {
         return sender;
     }
 
-    private void handleEvent() throws IOException {
-        ControlMessage msg = controlChannel.recv();
+    private boolean handleEvent() throws IOException {
+        ControlMessage msg;
+        try {
+            msg = controlChannel.recv();
+        } catch (IOException e) {
+            // this is expected on close
+            return false;
+        }
+
         switch (msg.getType()) {
             case ControlMessage.TYPE_INJECT_KEYCODE:
                 if (device.supportsInputEvents()) {
@@ -182,9 +204,20 @@ public class Controller implements AsyncProcessor {
             case ControlMessage.TYPE_ROTATE_DEVICE:
                 device.rotateDevice();
                 break;
+            case ControlMessage.TYPE_UHID_CREATE:
+                getUhidManager().open(msg.getId(), msg.getData());
+                break;
+            case ControlMessage.TYPE_UHID_INPUT:
+                getUhidManager().writeInput(msg.getId(), msg.getData());
+                break;
+            case ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
+                openHardKeyboardSettings();
+                break;
             default:
                 // do nothing
         }
+
+        return true;
     }
 
     private boolean injectKeycode(int action, int keycode, int repeat, int metaState) {
@@ -393,7 +426,8 @@ public class Controller implements AsyncProcessor {
         if (!clipboardAutosync) {
             String clipboardText = Device.getClipboardText();
             if (clipboardText != null) {
-                sender.pushClipboardText(clipboardText);
+                DeviceMessage msg = DeviceMessage.createClipboard(clipboardText);
+                sender.send(msg);
             }
         }
     }
@@ -411,9 +445,15 @@ public class Controller implements AsyncProcessor {
 
         if (sequence != ControlMessage.SEQUENCE_INVALID) {
             // Acknowledgement requested
-            sender.pushAckClipboard(sequence);
+            DeviceMessage msg = DeviceMessage.createAckClipboard(sequence);
+            sender.send(msg);
         }
 
         return ok;
     }
+
+    private void openHardKeyboardSettings() {
+        Intent intent = new Intent("android.settings.HARD_KEYBOARD_SETTINGS");
+        ServiceManager.getActivityManager().startActivity(intent);
+    }
 }

server/src/main/java/com/genymobile/scrcpy/DeviceMessage.java

@@ -4,12 +4,13 @@ public final class DeviceMessage {
 
     public static final int TYPE_CLIPBOARD = 0;
     public static final int TYPE_ACK_CLIPBOARD = 1;
-
-    public static final long SEQUENCE_INVALID = ControlMessage.SEQUENCE_INVALID;
+    public static final int TYPE_UHID_OUTPUT = 2;
 
     private int type;
     private String text;
     private long sequence;
+    private int id;
+    private byte[] data;
 
     private DeviceMessage() {
     }
@@ -28,6 +29,14 @@ public final class DeviceMessage {
         return event;
     }
 
+    public static DeviceMessage createUhidOutput(int id, byte[] data) {
+        DeviceMessage event = new DeviceMessage();
+        event.type = TYPE_UHID_OUTPUT;
+        event.id = id;
+        event.data = data;
+        return event;
+    }
+
     public int getType() {
         return type;
     }
@@ -39,4 +48,12 @@ public final class DeviceMessage {
     public long getSequence() {
         return sequence;
     }
+
+    public int getId() {
+        return id;
+    }
+
+    public byte[] getData() {
+        return data;
+    }
 }

server/src/main/java/com/genymobile/scrcpy/DeviceMessageSender.java

@@ -1,54 +1,30 @@
 package com.genymobile.scrcpy;
 
 import java.io.IOException;
+import java.util.concurrent.ArrayBlockingQueue;
+import java.util.concurrent.BlockingQueue;
 
 public final class DeviceMessageSender {
 
     private final ControlChannel controlChannel;
 
     private Thread thread;
-
-    private String clipboardText;
-
-    private long ack;
+    private final BlockingQueue<DeviceMessage> queue = new ArrayBlockingQueue<>(16);
 
     public DeviceMessageSender(ControlChannel controlChannel) {
         this.controlChannel = controlChannel;
     }
 
-    public synchronized void pushClipboardText(String text) {
-        clipboardText = text;
-        notify();
-    }
-
-    public synchronized void pushAckClipboard(long sequence) {
-        ack = sequence;
-        notify();
+    public void send(DeviceMessage msg) {
+        if (!queue.offer(msg)) {
+            Ln.w("Device message dropped: " + msg.getType());
+        }
     }
 
     private void loop() throws IOException, InterruptedException {
         while (!Thread.currentThread().isInterrupted()) {
-            String text;
-            long sequence;
-            synchronized (this) {
-                while (ack == DeviceMessage.SEQUENCE_INVALID && clipboardText == null) {
-                    wait();
-                }
-                text = clipboardText;
-                clipboardText = null;
-
-                sequence = ack;
-                ack = DeviceMessage.SEQUENCE_INVALID;
-            }
-
-            if (sequence != DeviceMessage.SEQUENCE_INVALID) {
-                DeviceMessage event = DeviceMessage.createAckClipboard(sequence);
-                controlChannel.send(event);
-            }
-            if (text != null) {
-                DeviceMessage event = DeviceMessage.createClipboard(text);
-                controlChannel.send(event);
-            }
+            DeviceMessage msg = queue.take();
+            controlChannel.send(msg);
         }
     }
 

server/src/main/java/com/genymobile/scrcpy/DeviceMessageWriter.java

@@ -29,6 +29,13 @@ public class DeviceMessageWriter {
                 buffer.putLong(msg.getSequence());
                 output.write(rawBuffer, 0, buffer.position());
                 break;
+            case DeviceMessage.TYPE_UHID_OUTPUT:
+                buffer.putShort((short) msg.getId());
+                byte[] data = msg.getData();
+                buffer.putShort((short) data.length);
+                buffer.put(data);
+                output.write(rawBuffer, 0, buffer.position());
+                break;
             default:
                 Ln.w("Unknown device message: " + msg.getType());
                 break;

server/src/main/java/com/genymobile/scrcpy/Server.java

@@ -133,7 +133,10 @@ public final class Server {
             if (control) {
                 ControlChannel controlChannel = connection.getControlChannel();
                 Controller controller = new Controller(device, controlChannel, cleanUp, options.getClipboardAutosync(), options.getPowerOn());
-                device.setClipboardListener(text -> controller.getSender().pushClipboardText(text));
+                device.setClipboardListener(text -> {
+                    DeviceMessage msg = DeviceMessage.createClipboard(text);
+                    controller.getSender().send(msg);
+                });
                 asyncProcessors.add(controller);
             }
 

server/src/main/java/com/genymobile/scrcpy/UhidManager.java

@@ -0,0 +1,218 @@
+package com.genymobile.scrcpy;
+
+import android.os.Build;
+import android.os.HandlerThread;
+import android.os.MessageQueue;
+import android.system.ErrnoException;
+import android.system.Os;
+import android.system.OsConstants;
+import android.util.ArrayMap;
+
+import java.io.FileDescriptor;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.charset.StandardCharsets;
+
+public final class UhidManager {
+
+    // Linux: include/uapi/linux/uhid.h
+    private static final int UHID_OUTPUT = 6;
+    private static final int UHID_CREATE2 = 11;
+    private static final int UHID_INPUT2 = 12;
+
+    // Linux: include/uapi/linux/input.h
+    private static final short BUS_VIRTUAL = 0x06;
+
+    private static final int SIZE_OF_UHID_EVENT = 4380; // sizeof(struct uhid_event)
+
+    private final ArrayMap<Integer, FileDescriptor> fds = new ArrayMap<>();
+    private final ByteBuffer buffer = ByteBuffer.allocate(SIZE_OF_UHID_EVENT).order(ByteOrder.nativeOrder());
+
+    private final DeviceMessageSender sender;
+    private final HandlerThread thread = new HandlerThread("UHidManager");
+    private final MessageQueue queue;
+
+    public UhidManager(DeviceMessageSender sender) {
+        this.sender = sender;
+        thread.start();
+        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
+            queue = thread.getLooper().getQueue();
+        } else {
+            queue = null;
+        }
+    }
+
+    public void open(int id, byte[] reportDesc) throws IOException {
+        try {
+            FileDescriptor fd = Os.open("/dev/uhid", OsConstants.O_RDWR, 0);
+            try {
+                FileDescriptor old = fds.put(id, fd);
+                if (old != null) {
+                    Ln.w("Duplicate UHID id: " + id);
+                    close(old);
+                }
+
+                byte[] req = buildUhidCreate2Req(reportDesc);
+                Os.write(fd, req, 0, req.length);
+
+                registerUhidListener(id, fd);
+            } catch (Exception e) {
+                close(fd);
+                throw e;
+            }
+        } catch (ErrnoException e) {
+            throw new IOException(e);
+        }
+    }
+
+    private void registerUhidListener(int id, FileDescriptor fd) {
+        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
+            queue.addOnFileDescriptorEventListener(fd, MessageQueue.OnFileDescriptorEventListener.EVENT_INPUT, (fd2, events) -> {
+                try {
+                    buffer.clear();
+                    int r = Os.read(fd2, buffer);
+                    buffer.flip();
+                    if (r > 0) {
+                        int type = buffer.getInt();
+                        if (type == UHID_OUTPUT) {
+                            byte[] data = extractHidOutputData(buffer);
+                            if (data != null) {
+                                DeviceMessage msg = DeviceMessage.createUhidOutput(id, data);
+                                sender.send(msg);
+                            }
+                        }
+                    }
+                } catch (ErrnoException | InterruptedIOException e) {
+                    Ln.e("Failed to read UHID output", e);
+                    return 0;
+                }
+                return events;
+            });
+        }
+    }
+
+    private static byte[] extractHidOutputData(ByteBuffer buffer) {
+        /*
+         * #define UHID_DATA_MAX 4096
+         * struct uhid_event {
+         *     uint32_t type;
+         *     union {
+         *         // ...
+         *         struct uhid_output_req {
+         *             __u8 data[UHID_DATA_MAX];
+         *             __u16 size;
+         *             __u8 rtype;
+         *         };
+         *     };
+         * } __attribute__((__packed__));
+         */
+
+        if (buffer.remaining() < 4099) {
+            Ln.w("Incomplete HID output");
+            return null;
+        }
+        int size = buffer.getShort(buffer.position() + 4096) & 0xFFFF;
+        if (size > 4096) {
+            Ln.w("Incorrect HID output size: " + size);
+            return null;
+        }
+        byte[] data = new byte[size];
+        buffer.get(data);
+        return data;
+    }
+
+    public void writeInput(int id, byte[] data) throws IOException {
+        FileDescriptor fd = fds.get(id);
+        if (fd == null) {
+            Ln.w("Unknown UHID id: " + id);
+            return;
+        }
+
+        try {
+            byte[] req = buildUhidInput2Req(data);
+            Os.write(fd, req, 0, req.length);
+        } catch (ErrnoException e) {
+            throw new IOException(e);
+        }
+    }
+
+    private static byte[] buildUhidCreate2Req(byte[] reportDesc) {
+        /*
+         * struct uhid_event {
+         *     uint32_t type;
+         *     union {
+         *         // ...
+         *         struct uhid_create2_req {
+         *             uint8_t name[128];
+         *             uint8_t phys[64];
+         *             uint8_t uniq[64];
+         *             uint16_t rd_size;
+         *             uint16_t bus;
+         *             uint32_t vendor;
+         *             uint32_t product;
+         *             uint32_t version;
+         *             uint32_t country;
+         *             uint8_t rd_data[HID_MAX_DESCRIPTOR_SIZE];
+         *         };
+         *     };
+         * } __attribute__((__packed__));
+         */
+
+        byte[] empty = new byte[256];
+        ByteBuffer buf = ByteBuffer.allocate(280 + reportDesc.length).order(ByteOrder.nativeOrder());
+        buf.putInt(UHID_CREATE2);
+        buf.put("scrcpy".getBytes(StandardCharsets.US_ASCII));
+        buf.put(empty, 0, 256 - "scrcpy".length());
+        buf.putShort((short) reportDesc.length);
+        buf.putShort(BUS_VIRTUAL);
+        buf.putInt(0); // vendor id
+        buf.putInt(0); // product id
+        buf.putInt(0); // version
+        buf.putInt(0); // country;
+        buf.put(reportDesc);
+        return buf.array();
+    }
+
+    private static byte[] buildUhidInput2Req(byte[] data) {
+        /*
+         * struct uhid_event {
+         *     uint32_t type;
+         *     union {
+         *         // ...
+         *         struct uhid_input2_req {
+         *             uint16_t size;
+         *             uint8_t data[UHID_DATA_MAX];
+         *         };
+         *     };
+         * } __attribute__((__packed__));
+         */
+
+        ByteBuffer buf = ByteBuffer.allocate(6 + data.length).order(ByteOrder.nativeOrder());
+        buf.putInt(UHID_INPUT2);
+        buf.putShort((short) data.length);
+        buf.put(data);
+        return buf.array();
+    }
+
+    public void close(int id) {
+        FileDescriptor fd = fds.get(id);
+        assert fd != null;
+        close(fd);
+    }
+
+    public void closeAll() {
+        for (FileDescriptor fd : fds.values()) {
+            close(fd);
+        }
+    }
+
+    private static void close(FileDescriptor fd) {
+        try {
+            Os.close(fd);
+        } catch (ErrnoException e) {
+            Ln.e("Failed to close uhid: " + e.getMessage());
+        }
+    }
+}

server/src/main/java/com/genymobile/scrcpy/wrappers/ActivityManager.java

@@ -22,7 +22,7 @@ public final class ActivityManager {
     private Method getContentProviderExternalMethod;
     private boolean getContentProviderExternalMethodNewVersion = true;
     private Method removeContentProviderExternalMethod;
-    private Method startActivityAsUserWithFeatureMethod;
+    private Method startActivityAsUserMethod;
     private Method forceStopPackageMethod;
 
     static ActivityManager create() {
@@ -107,26 +107,25 @@ public final class ActivityManager {
         return getContentProviderExternal("settings", new Binder());
     }
 
-    private Method getStartActivityAsUserWithFeatureMethod() throws NoSuchMethodException, ClassNotFoundException {
-        if (startActivityAsUserWithFeatureMethod == null) {
+    private Method getStartActivityAsUserMethod() throws NoSuchMethodException, ClassNotFoundException {
+        if (startActivityAsUserMethod == null) {
             Class<?> iApplicationThreadClass = Class.forName("android.app.IApplicationThread");
             Class<?> profilerInfo = Class.forName("android.app.ProfilerInfo");
-            startActivityAsUserWithFeatureMethod = manager.getClass()
-                    .getMethod("startActivityAsUserWithFeature", iApplicationThreadClass, String.class, String.class, Intent.class, String.class,
-                            IBinder.class, String.class, int.class, int.class, profilerInfo, Bundle.class, int.class);
+            startActivityAsUserMethod = manager.getClass()
+                    .getMethod("startActivityAsUser", iApplicationThreadClass, String.class, Intent.class, String.class, IBinder.class, String.class,
+                            int.class, int.class, profilerInfo, Bundle.class, int.class);
         }
-        return startActivityAsUserWithFeatureMethod;
+        return startActivityAsUserMethod;
     }
 
     @SuppressWarnings("ConstantConditions")
-    public int startActivityAsUserWithFeature(Intent intent) {
+    public int startActivity(Intent intent) {
         try {
-            Method method = getStartActivityAsUserWithFeatureMethod();
+            Method method = getStartActivityAsUserMethod();
             return (int) method.invoke(
                     /* this */ manager,
                     /* caller */ null,
                     /* callingPackage */ FakeContext.PACKAGE_NAME,
-                    /* callingFeatureId */ null,
                     /* intent */ intent,
                     /* resolvedType */ null,
                     /* resultTo */ null,

server/src/test/java/com/genymobile/scrcpy/ControlMessageReaderTest.java

@@ -322,6 +322,66 @@ public class ControlMessageReaderTest {
         Assert.assertEquals(ControlMessage.TYPE_ROTATE_DEVICE, event.getType());
     }
 
+    @Test
+    public void testParseUhidCreate() throws IOException {
+        ControlMessageReader reader = new ControlMessageReader();
+
+        ByteArrayOutputStream bos = new ByteArrayOutputStream();
+        DataOutputStream dos = new DataOutputStream(bos);
+        dos.writeByte(ControlMessage.TYPE_UHID_CREATE);
+        dos.writeShort(42); // id
+        byte[] data = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11};
+        dos.writeShort(data.length); // size
+        dos.write(data);
+
+        byte[] packet = bos.toByteArray();
+
+        reader.readFrom(new ByteArrayInputStream(packet));
+        ControlMessage event = reader.next();
+
+        Assert.assertEquals(ControlMessage.TYPE_UHID_CREATE, event.getType());
+        Assert.assertEquals(42, event.getId());
+        Assert.assertArrayEquals(data, event.getData());
+    }
+
+    @Test
+    public void testParseUhidInput() throws IOException {
+        ControlMessageReader reader = new ControlMessageReader();
+
+        ByteArrayOutputStream bos = new ByteArrayOutputStream();
+        DataOutputStream dos = new DataOutputStream(bos);
+        dos.writeByte(ControlMessage.TYPE_UHID_INPUT);
+        dos.writeShort(42); // id
+        byte[] data = {1, 2, 3, 4, 5};
+        dos.writeShort(data.length); // size
+        dos.write(data);
+
+        byte[] packet = bos.toByteArray();
+
+        reader.readFrom(new ByteArrayInputStream(packet));
+        ControlMessage event = reader.next();
+
+        Assert.assertEquals(ControlMessage.TYPE_UHID_INPUT, event.getType());
+        Assert.assertEquals(42, event.getId());
+        Assert.assertArrayEquals(data, event.getData());
+    }
+
+    @Test
+    public void testParseOpenHardKeyboardSettings() throws IOException {
+        ControlMessageReader reader = new ControlMessageReader();
+
+        ByteArrayOutputStream bos = new ByteArrayOutputStream();
+        DataOutputStream dos = new DataOutputStream(bos);
+        dos.writeByte(ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS);
+
+        byte[] packet = bos.toByteArray();
+
+        reader.readFrom(new ByteArrayInputStream(packet));
+        ControlMessage event = reader.next();
+
+        Assert.assertEquals(ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS, event.getType());
+    }
+
     @Test
     public void testMultiEvents() throws IOException {
         ControlMessageReader reader = new ControlMessageReader();

server/src/test/java/com/genymobile/scrcpy/DeviceMessageWriterTest.java

@@ -52,4 +52,27 @@ public class DeviceMessageWriterTest {
 
         Assert.assertArrayEquals(expected, actual);
     }
+
+    @Test
+    public void testSerializeUhidOutput() throws IOException {
+        DeviceMessageWriter writer = new DeviceMessageWriter();
+
+        ByteArrayOutputStream bos = new ByteArrayOutputStream();
+        DataOutputStream dos = new DataOutputStream(bos);
+        dos.writeByte(DeviceMessage.TYPE_UHID_OUTPUT);
+        dos.writeShort(42); // id
+        byte[] data = {1, 2, 3, 4, 5};
+        dos.writeShort(data.length);
+        dos.write(data);
+
+        byte[] expected = bos.toByteArray();
+
+        DeviceMessage msg = DeviceMessage.createUhidOutput(42, data);
+        bos = new ByteArrayOutputStream();
+        writer.writeTo(msg, bos);
+
+        byte[] actual = bos.toByteArray();
+
+        Assert.assertArrayEquals(expected, actual);
+    }
 }