Forward mouse hover events

Also add an option --no-mouse-hover to get the old behavior.

Fixes #2743 <https://github.com/Genymobile/scrcpy/issues/2743>
Fixes #3070 <https://github.com/Genymobile/scrcpy/issues/3070>
PR #5039 <https://github.com/Genymobile/scrcpy/pull/5039>

FAQ.md

@@ -133,9 +133,9 @@ Try with another USB cable or plug it into another USB port. See [#281] and
 [#283]: https://github.com/Genymobile/scrcpy/issues/283
 
 
-## HID/OTG issues on Windows
+## OTG issues on Windows
 
-On Windows, if `scrcpy --otg` (or `--hid-keyboard`/`--hid-mouse`) results in:
+On Windows, if `scrcpy --otg` (or `--keyboard=aoa`/`--mouse=aoa`) results in:
 
 >     ERROR: Could not find any USB device
 
@@ -170,12 +170,13 @@ The default text injection method is [limited to ASCII characters][text-input].
 A trick allows to also inject some [accented characters][accented-characters],
 but that's all. See [#37].
 
-It is also possible to simulate a [physical keyboard][hid] (HID).
+To avoid the problem, [change the keyboard mode to simulate a physical
+keyboard][hid].
 
 [text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
 [accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
 [#37]: https://github.com/Genymobile/scrcpy/issues/37
-[hid]: doc/hid-otg.md
+[hid]: doc/keyboard.md#physical-keyboard-simulation
 
 
 ## Client issues

LICENSE

@@ -188,7 +188,7 @@
       identification within third-party archives.
 
    Copyright (C) 2018 Genymobile
-   Copyright (C) 2018-2023 Romain Vimont
+   Copyright (C) 2018-2024 Romain Vimont
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1,4 +1,8 @@
-# scrcpy (v2.3.1)
+**This GitHub repo (<https://github.com/Genymobile/scrcpy>) is the only official
+source for the project. Do not download releases from random websites, even if
+their name contains `scrcpy`.**
+
+# scrcpy (v2.4)
 
 <img src="app/data/icon.svg" width="128" height="128" alt="scrcpy" align="right" />
 
@@ -32,10 +36,13 @@ Its features include:
  - [configurable quality](doc/video.md)
  - [camera mirroring](doc/camera.md) (Android 12+)
  - [mirroring as a webcam (V4L2)](doc/v4l2.md) (Linux-only)
- - [physical keyboard/mouse simulation (HID)](doc/hid-otg.md)
- - [OTG mode](doc/hid-otg.md#otg)
+ - physical [keyboard][hid-keyboard] and [mouse][hid-mouse] simulation (HID)
+ - [OTG mode](doc/otg.md)
  - and more…
 
+[hid-keyboard]: doc/keyboard.md#physical-keyboard-simulation
+[hid-mouse]: doc/mouse.md#physical-mouse-simulation
+
 ## Prerequisites
 
 The Android device requires at least API 21 (Android 5.0).
@@ -53,8 +60,7 @@ this option is set.
 
 [control]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
 
-Note that USB debugging is not required to run scrcpy in [OTG
-mode](doc/hid-otg.md#otg).
+Note that USB debugging is not required to run scrcpy in [OTG mode](doc/otg.md).
 
 
 ## Get the app
@@ -64,6 +70,41 @@ mode](doc/hid-otg.md#otg).
  - [macOS](doc/macos.md)
 
 
+## Usage examples
+
+There are a lot of options, [documented](#user-documentation) in separate pages.
+Here are just some common examples.
+
+ - Capture the screen in H.265 (better quality), limit the size to 1920, limit
+   the frame rate to 60fps, disable audio, and control the device by simulating
+   a physical keyboard:
+
+    ```bash
+    scrcpy --video-codec=h265 --max-size=1920 --max-fps=60 --no-audio --keyboard=uhid
+    scrcpy --video-codec=h265 -m1920 --max-fps=60 --no-audio -K  # short version
+    ```
+
+ - Record the device camera in H.265 at 1920x1080 (and microphone) to an MP4
+   file:
+
+    ```bash
+    scrcpy --video-source=camera --video-codec=h265 --camera-size=1920x1080 --record=file.mp4
+    ```
+
+ - Capture the device front camera and expose it as a webcam on the computer (on
+   Linux):
+
+    ```bash
+    scrcpy --video-source=camera --camera-size=1920x1080 --camera-facing=front --v4l2-sink=/dev/video2 --no-playback
+    ```
+
+ - Control the device without mirroring by simulating a physical keyboard and
+   mouse (USB debugging not required):
+
+    ```bash
+    scrcpy --otg
+    ```
+
 ## User documentation
 
 The application provides a lot of features and configuration options. They are
@@ -73,11 +114,13 @@ documented in the following pages:
  - [Video](doc/video.md)
  - [Audio](doc/audio.md)
  - [Control](doc/control.md)
+ - [Keyboard](doc/keyboard.md)
+ - [Mouse](doc/mouse.md)
  - [Device](doc/device.md)
  - [Window](doc/window.md)
  - [Recording](doc/recording.md)
  - [Tunnels](doc/tunnels.md)
- - [HID/OTG](doc/hid-otg.md)
+ - [OTG](doc/otg.md)
  - [Camera](doc/camera.md)
  - [Video4Linux](doc/v4l2.md)
  - [Shortcuts](doc/shortcuts.md)
@@ -130,7 +173,7 @@ work][donate]:
 ## Licence
 
     Copyright (C) 2018 Genymobile
-    Copyright (C) 2018-2023 Romain Vimont
+    Copyright (C) 2018-2024 Romain Vimont
 
     Licensed under the Apache License, Version 2.0 (the "License");
     you may not use this file except in compliance with the License.

app/data/bash-completion/scrcpy

@@ -25,10 +25,10 @@ _scrcpy() {
         -e --select-tcpip
         -f --fullscreen
         --force-adb-forward
-        --forward-all-clicks
         -h --help
+        -K
+        --keyboard=
         --kill-adb-on-close
-        -K --hid-keyboard
         --legacy-paste
         --list-camera-sizes
         --list-cameras
@@ -37,8 +37,10 @@ _scrcpy() {
         --lock-video-orientation
         --lock-video-orientation=
         -m --max-size=
-        -M --hid-mouse
+        -M
         --max-fps=
+        --mouse=
+        --mouse-bind=
         -n --no-control
         -N --no-playback
         --no-audio
@@ -48,6 +50,7 @@ _scrcpy() {
         --no-downsize-on-error
         --no-key-repeat
         --no-mipmaps
+        --no-mouse-hover
         --no-power-on
         --no-video
         --no-video-playback
@@ -115,6 +118,14 @@ _scrcpy() {
             COMPREPLY=($(compgen -W 'front back external' -- "$cur"))
             return
             ;;
+        --keyboard)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
+            return
+            ;;
+        --mouse)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
+            return
+            ;;
         --orientation|--display-orientation)
             COMPREPLY=($(compgen -W '0 90 180 270 flip0 flip90 flip180 flip270' -- "$cur"))
             return

app/data/zsh-completion/_scrcpy

@@ -32,10 +32,10 @@ arguments=(
     {-e,--select-tcpip}'[Use TCP/IP device]'
     {-f,--fullscreen}'[Start in fullscreen]'
     '--force-adb-forward[Do not attempt to use \"adb reverse\" to connect to the device]'
-    '--forward-all-clicks[Forward clicks to device]'
     {-h,--help}'[Print the help]'
+    '-K[Use UHID keyboard (same as --keyboard=uhid)]'
+    '--keyboard=[Set the keyboard input mode]:mode:(disabled sdk uhid aoa)'
     '--kill-adb-on-close[Kill adb when scrcpy terminates]'
-    {-K,--hid-keyboard}'[Simulate a physical keyboard by using HID over AOAv2]'
     '--legacy-paste[Inject computer clipboard text as a sequence of key events on Ctrl+v]'
     '--list-camera-sizes[List the valid camera capture sizes]'
     '--list-cameras[List cameras available on the device]'
@@ -43,8 +43,10 @@ arguments=(
     '--list-encoders[List video and audio encoders available on the device]'
     '--lock-video-orientation=[Lock video orientation]:orientation:(unlocked initial 0 90 180 270)'
     {-m,--max-size=}'[Limit both the width and height of the video to value]'
-    {-M,--hid-mouse}'[Simulate a physical mouse by using HID over AOAv2]'
+    '-M[Use UHID mouse (same as --mouse=uhid)]'
     '--max-fps=[Limit the frame rate of screen capture]'
+    '--mouse=[Set the mouse input mode]:mode:(disabled sdk uhid aoa)'
+    '--mouse-bind=[Configure bindings of secondary clicks]'
     {-n,--no-control}'[Disable device control \(mirror the device in read only\)]'
     {-N,--no-playback}'[Disable video and audio playback]'
     '--no-audio[Disable audio forwarding]'
@@ -54,6 +56,7 @@ arguments=(
     '--no-downsize-on-error[Disable lowering definition on MediaCodec error]'
     '--no-key-repeat[Do not forward repeated key events when a key is held down]'
     '--no-mipmaps[Disable the generation of mipmaps]'
+    '--no-mouse-hover[Do not forward mouse hover events]'
     '--no-power-on[Do not power on the device on start]'
     '--no-video[Disable video forwarding]'
     '--no-video-playback[Disable video playback]'

app/deps/.gitignore

@@ -0,0 +1 @@
+/work

app/deps/README

@@ -0,0 +1,27 @@
+This directory (app/deps/) contains:
+
+*.sh          : shell scripts to download and build dependencies
+
+patches/      : patches to fix dependencies (used by scripts)
+
+work/sources/ : downloaded tarballs and extracted folders
+                    ffmpeg-6.1.1.tar.xz
+                    ffmpeg-6.1.1/
+                    libusb-1.0.27.tar.gz
+                    libusb-1.0.27/
+                    ...
+work/build/   : build dirs for each dependency/version/architecture
+                    ffmpeg-6.1.1/win32/
+                    ffmpeg-6.1.1/win64/
+                    libusb-1.0.27/win32/
+                    libusb-1.0.27/win64/
+                    ...
+work/install/ : install dirs for each architexture
+                    win32/bin/
+                    win32/include/
+                    win32/lib/
+                    win32/share/
+                    win64/bin/
+                    win64/include/
+                    win64/lib/
+                    win64/share/

app/deps/adb.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=34.0.5
+FILENAME=platform-tools_r$VERSION-windows.zip
+PROJECT_DIR=platform-tools-$VERSION
+SHA256SUM=3f8320152704377de150418a3c4c9d07d16d80a6c0d0d8f7289c22c499e33571
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://dl.google.com/android/repository/$FILENAME" "$FILENAME" "$SHA256SUM"
+    mkdir -p "$PROJECT_DIR"
+    cd "$PROJECT_DIR"
+    ZIP_PREFIX=platform-tools
+    unzip "../$FILENAME" \
+        "$ZIP_PREFIX"/AdbWinApi.dll \
+        "$ZIP_PREFIX"/AdbWinUsbApi.dll \
+        "$ZIP_PREFIX"/adb.exe
+    mv "$ZIP_PREFIX"/* .
+    rmdir "$ZIP_PREFIX"
+fi
+
+mkdir -p "$INSTALL_DIR/$HOST/bin"
+cd "$INSTALL_DIR/$HOST/bin"
+cp -r "$SOURCES_DIR/$PROJECT_DIR"/. "$INSTALL_DIR/$HOST/bin/"

app/deps/common

@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# This file is intended to be sourced by other scripts, not executed
+
+if [[ $# != 1 ]]
+then
+    # <host>: win32 or win64
+    echo "Syntax: $0 <host>" >&2
+    exit 1
+fi
+
+HOST="$1"
+
+if [[ "$HOST" = win32 ]]
+then
+    HOST_TRIPLET=i686-w64-mingw32
+elif [[ "$HOST" = win64 ]]
+then
+    HOST_TRIPLET=x86_64-w64-mingw32
+else
+    echo "Unsupported host: $HOST" >&2
+    exit 1
+fi
+
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+
+PATCHES_DIR="$PWD/patches"
+
+WORK_DIR="$PWD/work"
+SOURCES_DIR="$WORK_DIR/sources"
+BUILD_DIR="$WORK_DIR/build"
+INSTALL_DIR="$WORK_DIR/install"
+
+mkdir -p "$INSTALL_DIR" "$SOURCES_DIR" "$WORK_DIR"
+
+checksum() {
+    local file="$1"
+    local sum="$2"
+    echo "$file: verifying checksum..."
+    echo "$sum  $file" | sha256sum -c
+}
+
+get_file() {
+    local url="$1"
+    local file="$2"
+    local sum="$3"
+    if [[ -f "$file" ]]
+    then
+        echo "$file: found"
+    else
+        echo "$file: not found, downloading..."
+        wget "$url" -O "$file"
+    fi
+    checksum "$file" "$sum"
+}

app/deps/ffmpeg.sh

@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=6.1.1
+FILENAME=ffmpeg-$VERSION.tar.xz
+PROJECT_DIR=ffmpeg-$VERSION
+SHA256SUM=8684f4b00f94b85461884c3719382f1261f0d9eb3d59640a1f4ac0873616f968
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://ffmpeg.org/releases/$FILENAME" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+    patch -d "$PROJECT_DIR" -p1 < "$PATCHES_DIR"/ffmpeg-6.1-fix-build.patch
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+if [[ "$HOST" = win32 ]]
+then
+    ARCH=x86
+elif [[ "$HOST" = win64 ]]
+then
+    ARCH=x86_64
+else
+    echo "Unsupported host: $HOST" >&2
+    exit 1
+fi
+
+# -static-libgcc to avoid missing libgcc_s_dw2-1.dll
+# -static to avoid dynamic dependency to zlib
+export CFLAGS='-static-libgcc -static'
+export CXXFLAGS="$CFLAGS"
+export LDFLAGS='-static-libgcc -static'
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --enable-cross-compile \
+        --target-os=mingw32 \
+        --arch="$ARCH" \
+        --cross-prefix="${HOST_TRIPLET}-" \
+        --cc="${HOST_TRIPLET}-gcc" \
+        --extra-cflags="-O2 -fPIC" \
+        --enable-shared \
+        --disable-static \
+        --disable-programs \
+        --disable-doc \
+        --disable-swscale \
+        --disable-postproc \
+        --disable-avfilter \
+        --disable-avdevice \
+        --disable-network \
+        --disable-everything \
+        --enable-swresample \
+        --enable-decoder=h264 \
+        --enable-decoder=hevc \
+        --enable-decoder=av1 \
+        --enable-decoder=pcm_s16le \
+        --enable-decoder=opus \
+        --enable-decoder=aac \
+        --enable-decoder=flac \
+        --enable-decoder=png \
+        --enable-protocol=file \
+        --enable-demuxer=image2 \
+        --enable-parser=png \
+        --enable-zlib \
+        --enable-muxer=matroska \
+        --enable-muxer=mp4 \
+        --enable-muxer=opus \
+        --enable-muxer=flac \
+        --enable-muxer=wav \
+        --disable-vulkan
+fi
+
+make -j
+make install

app/deps/libusb.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=1.0.27
+FILENAME=libusb-$VERSION.tar.gz
+PROJECT_DIR=libusb-$VERSION
+SHA256SUM=e8f18a7a36ecbb11fb820bd71540350d8f61bcd9db0d2e8c18a6fb80b214a3de
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://github.com/libusb/libusb/archive/refs/tags/v$VERSION.tar.gz" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+export CFLAGS='-O2'
+export CXXFLAGS="$CFLAGS"
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/bootstrap.sh
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --host="$HOST_TRIPLET" \
+        --enable-shared \
+        --disable-static
+fi
+
+make -j
+make install-strip

app/deps/patches/ffmpeg-6.1-fix-build.patch

@@ -0,0 +1,27 @@
+From 03c80197afb324da38c9b70254231e3fdcfa68fc Mon Sep 17 00:00:00 2001
+From: Romain Vimont <rom@rom1v.com>
+Date: Sun, 12 Nov 2023 17:58:50 +0100
+Subject: [PATCH] Fix FFmpeg 6.1 build
+
+Build failed on tag n6.1 With --enable-decoder=av1 but without
+--enable-muxer=av1.
+---
+ libavcodec/Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/libavcodec/Makefile b/libavcodec/Makefile
+index 580a8d6b54..aff19b670c 100644
+--- a/libavcodec/Makefile
++++ b/libavcodec/Makefile
+@@ -249,7 +249,7 @@ OBJS-$(CONFIG_ATRAC3PAL_DECODER)       += atrac3plusdec.o atrac3plus.o \
+ OBJS-$(CONFIG_ATRAC9_DECODER)          += atrac9dec.o
+ OBJS-$(CONFIG_AURA_DECODER)            += cyuv.o
+ OBJS-$(CONFIG_AURA2_DECODER)           += aura.o
+-OBJS-$(CONFIG_AV1_DECODER)             += av1dec.o
++OBJS-$(CONFIG_AV1_DECODER)             += av1dec.o av1_parse.o
+ OBJS-$(CONFIG_AV1_CUVID_DECODER)       += cuviddec.o
+ OBJS-$(CONFIG_AV1_MEDIACODEC_DECODER)  += mediacodecdec.o
+ OBJS-$(CONFIG_AV1_MEDIACODEC_ENCODER)  += mediacodecenc.o
+-- 
+2.42.0
+

app/deps/sdl.sh

@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=2.28.5
+FILENAME=SDL-$VERSION.tar.gz
+PROJECT_DIR=SDL-release-$VERSION
+SHA256SUM=9f0556e4a24ef5b267010038ad9e9948b62f236d5bcc4b22179f95ef62d84023
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://github.com/libsdl-org/SDL/archive/refs/tags/release-$VERSION.tar.gz" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+export CFLAGS='-O2'
+export CXXFLAGS="$CFLAGS"
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --host="$HOST_TRIPLET" \
+        --enable-shared \
+        --disable-static
+fi
+
+make -j
+# There is no "make install-strip"
+make install
+# Strip manually
+${HOST_TRIPLET}-strip "$INSTALL_DIR/$HOST/bin/SDL2.dll"

app/meson.build

@@ -20,8 +20,8 @@ src = [
     'src/fps_counter.c',
     'src/frame_buffer.c',
     'src/input_manager.c',
-    'src/keyboard_inject.c',
-    'src/mouse_inject.c',
+    'src/keyboard_sdk.c',
+    'src/mouse_sdk.c',
     'src/opengl.c',
     'src/options.c',
     'src/packet_merger.c',
@@ -31,8 +31,13 @@ src = [
     'src/screen.c',
     'src/server.c',
     'src/version.c',
+    'src/hid/hid_keyboard.c',
+    'src/hid/hid_mouse.c',
     'src/trait/frame_source.c',
     'src/trait/packet_source.c',
+    'src/uhid/keyboard_uhid.c',
+    'src/uhid/mouse_uhid.c',
+    'src/uhid/uhid_output.c',
     'src/util/acksync.c',
     'src/util/audiobuf.c',
     'src/util/average.c',
@@ -88,8 +93,8 @@ usb_support = get_option('usb')
 if usb_support
     src += [
         'src/usb/aoa_hid.c',
-        'src/usb/hid_keyboard.c',
-        'src/usb/hid_mouse.c',
+        'src/usb/keyboard_aoa.c',
+        'src/usb/mouse_aoa.c',
         'src/usb/scrcpy_otg.c',
         'src/usb/screen_otg.c',
         'src/usb/usb.c',

app/prebuilt-deps/.gitignore

@@ -1 +0,0 @@
-/data

app/prebuilt-deps/common

@@ -1,22 +0,0 @@
-PREBUILT_DATA_DIR=data
-
-checksum() {
-    local file="$1"
-    local sum="$2"
-    echo "$file: verifying checksum..."
-    echo "$sum  $file" | sha256sum -c
-}
-
-get_file() {
-    local url="$1"
-    local file="$2"
-    local sum="$3"
-    if [[ -f "$file" ]]
-    then
-        echo "$file: found"
-    else
-        echo "$file: not found, downloading..."
-        wget "$url" -O "$file"
-    fi
-    checksum "$file" "$sum"
-}

app/prebuilt-deps/prepare-adb.sh

@@ -1,32 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-DEP_DIR=platform-tools-34.0.5
-
-FILENAME=platform-tools_r34.0.5-windows.zip
-SHA256SUM=3f8320152704377de150418a3c4c9d07d16d80a6c0d0d8f7289c22c499e33571
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://dl.google.com/android/repository/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-ZIP_PREFIX=platform-tools
-unzip "../$FILENAME" \
-    "$ZIP_PREFIX"/AdbWinApi.dll \
-    "$ZIP_PREFIX"/AdbWinUsbApi.dll \
-    "$ZIP_PREFIX"/adb.exe
-mv "$ZIP_PREFIX"/* .
-rmdir "$ZIP_PREFIX"

app/prebuilt-deps/prepare-ffmpeg.sh

@@ -1,30 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-VERSION=6.1-scrcpy-3
-DEP_DIR="ffmpeg-$VERSION"
-
-FILENAME="$DEP_DIR".7z
-SHA256SUM=b646d18a3d543a4e4c46881568213499f22e4454a464e1552f03f2ac9cc3a05a
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/rom1v/scrcpy-deps/releases/download/$VERSION/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-ZIP_PREFIX=ffmpeg
-7z x "../$FILENAME"
-mv "$ZIP_PREFIX"/* .
-rmdir "$ZIP_PREFIX"

app/prebuilt-deps/prepare-libusb.sh

@@ -1,37 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-VERSION=1.0.26
-DEP_DIR="libusb-$VERSION"
-
-FILENAME="libusb-$VERSION-binaries.7z"
-SHA256SUM=9c242696342dbde9cdc47239391f71833939bf9f7aa2bbb28cdaabe890465ec5
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/libusb/libusb/releases/download/v$VERSION/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-7z x "../$FILENAME" \
-    "libusb-$VERSION-binaries/libusb-MinGW-Win32/" \
-    "libusb-$VERSION-binaries/libusb-MinGW-x64/"
-
-mv "libusb-$VERSION-binaries/libusb-MinGW-Win32" .
-mv "libusb-$VERSION-binaries/libusb-MinGW-x64" .
-rm -rf "libusb-$VERSION-binaries"
-
-# Rename the dll to get the same library name on all platforms
-mv libusb-MinGW-Win32/bin/msys-usb-1.0.dll libusb-MinGW-Win32/bin/libusb-1.0.dll
-mv libusb-MinGW-x64/bin/msys-usb-1.0.dll libusb-MinGW-x64/bin/libusb-1.0.dll

app/prebuilt-deps/prepare-sdl.sh

@@ -1,34 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-VERSION=2.28.5
-DEP_DIR="SDL2-$VERSION"
-
-FILENAME="SDL2-devel-$VERSION-mingw.tar.gz"
-SHA256SUM=3c0c655c2ebf67cad48fead72761d1601740ded30808952c3274ba223d226c21
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/libsdl-org/SDL/releases/download/release-$VERSION/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-TAR_PREFIX="$DEP_DIR" # root directory inside the tar has the same name
-tar xf "../$FILENAME" --strip-components=1 \
-    "$TAR_PREFIX"/i686-w64-mingw32/bin/SDL2.dll \
-    "$TAR_PREFIX"/i686-w64-mingw32/include/ \
-    "$TAR_PREFIX"/i686-w64-mingw32/lib/ \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/bin/SDL2.dll \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/include/ \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/lib/ \

app/scrcpy-windows.rc

@@ -13,7 +13,7 @@ BEGIN
       VALUE "LegalCopyright", "Romain Vimont, Genymobile"
       VALUE "OriginalFilename", "scrcpy.exe"
       VALUE "ProductName", "scrcpy"
-      VALUE "ProductVersion", "2.3.1"
+      VALUE "ProductVersion", "2.4"
     END
   END
   BLOCK "VarFileInfo"

app/scrcpy.1

@@ -163,33 +163,36 @@ Start in fullscreen.
 .B \-\-force\-adb\-forward
 Do not attempt to use "adb reverse" to connect to the device.
 
-.TP
-.B \-\-forward\-all\-clicks
-By default, right-click triggers BACK (or POWER on) and middle-click triggers HOME. This option disables these shortcuts and forward the clicks to the device instead.
-
 .TP
 .B \-h, \-\-help
 Print this help.
 
 .TP
-.B \-\-kill\-adb\-on\-close
-Kill adb when scrcpy terminates.
+.B \-K
+Same as \fB\-\-keyboard=uhid\fR.
 
 .TP
-.B \-K, \-\-hid\-keyboard
-Simulate a physical keyboard by using HID over AOAv2.
+.BI "\-\-keyboard " mode
+Select how to send keyboard inputs to the device.
 
-This provides a better experience for IME users, and allows to generate non-ASCII characters, contrary to the default injection method.
+Possible values are "disabled", "sdk", "uhid" and "aoa":
 
-It may only work over USB.
+ - "disabled" does not send keyboard inputs to the device.
+ - "sdk" uses the Android system API to deliver keyboard events to applications.
+ - "uhid" simulates a physical HID keyboard using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical HID keyboard using the AOAv2 protocol. It may only work over USB.
 
-The keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly:
+For "uhid" and "aoa", the keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly using the shortcut MOD+k (except in OTG mode), or by executing:
 
     adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
 
-However, the option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
+This option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
 
-Also see \fB\-\-hid\-mouse\fR.
+Also see \fB\-\-mouse\fR.
+
+.TP
+.B \-\-kill\-adb\-on\-close
+Kill adb when scrcpy terminates.
 
 .TP
 .B \-\-legacy\-paste
@@ -230,21 +233,48 @@ Limit both the width and height of the video to \fIvalue\fR. The other dimension
 Default is 0 (unlimited).
 
 .TP
-.B \-M, \-\-hid\-mouse
-Simulate a physical mouse by using HID over AOAv2.
-
-In this mode, the computer mouse is captured to control the device directly (relative mouse mode).
-
-LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
-
-It may only work over USB.
-
-Also see \fB\-\-hid\-keyboard\fR.
+.B \-M
+Same as \fB\-\-mouse=uhid\fR.
 
 .TP
 .BI "\-\-max\-fps " value
 Limit the framerate of screen capture (officially supported since Android 10, but may work on earlier versions).
 
+.TP
+.BI "\-\-mouse " mode
+Select how to send mouse inputs to the device.
+
+Possible values are "disabled", "sdk", "uhid" and "aoa":
+
+ - "disabled" does not send mouse inputs to the device.
+ - "sdk" uses the Android system API to deliver mouse events to applications.
+ - "uhid" simulates a physical HID mouse using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical mouse using the AOAv2 protocol. It may only work over USB.
+
+In "uhid" and "aoa" modes, the computer mouse is captured to control the device directly (relative mouse mode).
+
+LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
+
+Also see \fB\-\-keyboard\fR.
+
+.TP
+.BI "\-\-mouse\-bind " xxxx
+Configure bindings of secondary clicks.
+
+The argument must be exactly 4 characters, one for each secondary click (in order: right click, middle click, 4th click, 5th click).
+
+Each character must be one of the following:
+
+ - '+': forward the click to the device
+ - '-': ignore the click
+ - 'b': trigger shortcut BACK (or turn screen on if off)
+ - 'h': trigger shortcut HOME
+ - 's': trigger shortcut APP_SWITCH
+ - 'n': trigger shortcut "expand notification panel"
+
+Default is 'bhsn' for SDK mouse, and '++++' for AOA and UHID.
+
+
 .TP
 .B \-n, \-\-no\-control
 Disable device control (mirror the device in read\-only).
@@ -287,6 +317,10 @@ Do not forward repeated key events when a key is held down.
 .B \-\-no\-mipmaps
 If the renderer is OpenGL 3.0+ or OpenGL ES 2.0+, then mipmaps are automatically generated to improve downscaling quality. This option disables the generation of mipmaps.
 
+.TP
+.B \-\-no\-mouse\-hover
+Do not forward mouse hover (mouse motion without any clicks) events.
+
 .TP
 .B \-\-no\-power\-on
 Do not power on the device on start.
@@ -299,6 +333,10 @@ Disable video forwarding.
 .B \-\-no\-video\-playback
 Disable video playback on the computer.
 
+.TP
+.B \-\-no\-window
+Disable scrcpy window. Implies --no-video-playback and --no-control.
+
 .TP
 .BI "\-\-orientation " value
 Same as --display-orientation=value --record-orientation=value.
@@ -403,9 +441,9 @@ Turn the device screen off immediately.
 .BI "\-\-shortcut\-mod " key\fR[+...]][,...]
 Specify the modifiers to use for scrcpy shortcuts. Possible keys are "lctrl", "rctrl", "lalt", "ralt", "lsuper" and "rsuper".
 
-A shortcut can consist in several keys, separated by '+'. Several shortcuts can be specified, separated by ','.
+Several shortcut modifiers can be specified, separated by ','.
 
-For example, to use either LCtrl+LAlt or LSuper for scrcpy shortcuts, pass "lctrl+lalt,lsuper".
+For example, to use either LCtrl or LSuper for scrcpy shortcuts, pass "lctrl,lsuper".
 
 Default is "lalt,lsuper" (left-Alt or left-Super).
 
@@ -560,6 +598,14 @@ Flip display horizontally
 .B MOD+Shift+Up, MOD+Shift+Down
 Flip display vertically
 
+.TP
+.B MOD+z
+Pause or re-pause display
+
+.TP
+.B MOD+Shift+z
+Unpause display
+
 .TP
 .B MOD+g
 Resize window to 1:1 (pixel\-perfect)
@@ -636,6 +682,10 @@ Copy computer clipboard to device, then paste (inject PASTE keycode, Android >=
 .B MOD+Shift+v
 Inject computer clipboard text as a sequence of key events
 
+.TP
+.B MOD+k
+Open keyboard settings on the device (for HID keyboard only)
+
 .TP
 .B MOD+i
 Enable/disable FPS counter (print frames/second in logs)
@@ -693,7 +743,7 @@ Report bugs to <https://github.com/Genymobile/scrcpy/issues>.
 .SH COPYRIGHT
 Copyright \(co 2018 Genymobile <https://www.genymobile.com>
 
-Copyright \(co 2018\-2023 Romain Vimont <rom@rom1v.com>
+Copyright \(co 2018\-2024 Romain Vimont <rom@rom1v.com>
 
 Licensed under the Apache License, Version 2.0.
 

app/src/audio_player.c

@@ -194,7 +194,11 @@ sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
             // Still insufficient, drop old samples to make space
             skipped_samples = sc_audiobuf_read(&ap->buf, NULL, remaining);
             assert(skipped_samples == remaining);
+        }
 
+        SDL_UnlockAudioDevice(ap->device);
+
+        if (written < samples) {
             // Now there is enough space
             uint32_t w = sc_audiobuf_write(&ap->buf,
                                            swr_buf + TO_BYTES(written),
@@ -202,8 +206,6 @@ sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
             assert(w == remaining);
             (void) w;
         }
-
-        SDL_UnlockAudioDevice(ap->device);
     }
 
     uint32_t underflow = 0;

app/src/cli.c

@@ -93,6 +93,13 @@ enum {
     OPT_DISPLAY_ORIENTATION,
     OPT_RECORD_ORIENTATION,
     OPT_ORIENTATION,
+    OPT_KEYBOARD,
+    OPT_MOUSE,
+    OPT_HID_KEYBOARD_DEPRECATED,
+    OPT_HID_MOUSE_DEPRECATED,
+    OPT_NO_WINDOW,
+    OPT_MOUSE_BIND,
+    OPT_NO_MOUSE_HOVER,
 };
 
 struct sc_option {
@@ -347,38 +354,53 @@ static const struct sc_option options[] = {
                 "device.",
     },
     {
+        // deprecated
         .longopt_id = OPT_FORWARD_ALL_CLICKS,
         .longopt = "forward-all-clicks",
-        .text = "By default, right-click triggers BACK (or POWER on) and "
-                "middle-click triggers HOME. This option disables these "
-                "shortcuts and forwards the clicks to the device instead.",
     },
     {
         .shortopt = 'h',
         .longopt = "help",
         .text = "Print this help.",
     },
+    {
+        .shortopt = 'K',
+        .text = "Same as --keyboard=uhid.",
+    },
+    {
+        .longopt_id = OPT_KEYBOARD,
+        .longopt = "keyboard",
+        .argdesc = "mode",
+        .text = "Select how to send keyboard inputs to the device.\n"
+                "Possible values are \"disabled\", \"sdk\", \"uhid\" and "
+                "\"aoa\".\n"
+                "\"disabled\" does not send keyboard inputs to the device.\n"
+                "\"sdk\" uses the Android system API to deliver keyboard "
+                "events to applications.\n"
+                "\"uhid\" simulates a physical HID keyboard using the Linux "
+                "UHID kernel module on the device.\n"
+                "\"aoa\" simulates a physical keyboard using the AOAv2 "
+                "protocol. It may only work over USB.\n"
+                "For \"uhid\" and \"aoa\", the keyboard layout must be "
+                "configured (once and for all) on the device, via Settings -> "
+                "System -> Languages and input -> Physical keyboard. This "
+                "settings page can be started directly using the shortcut "
+                "MOD+k (except in OTG mode) or by executing: `adb shell am "
+                "start -a android.settings.HARD_KEYBOARD_SETTINGS`.\n"
+                "This option is only available when a HID keyboard is enabled "
+                "(or a physical keyboard is connected).\n"
+                "Also see --mouse.",
+    },
     {
         .longopt_id = OPT_KILL_ADB_ON_CLOSE,
         .longopt = "kill-adb-on-close",
         .text = "Kill adb when scrcpy terminates.",
     },
     {
-        .shortopt = 'K',
+        // deprecated
+        //.shortopt = 'K', // old, reassigned
+        .longopt_id = OPT_HID_KEYBOARD_DEPRECATED,
         .longopt = "hid-keyboard",
-        .text = "Simulate a physical keyboard by using HID over AOAv2.\n"
-                "It provides a better experience for IME users, and allows to "
-                "generate non-ASCII characters, contrary to the default "
-                "injection method.\n"
-                "It may only work over USB.\n"
-                "The keyboard layout must be configured (once and for all) on "
-                "the device, via Settings -> System -> Languages and input -> "
-                "Physical keyboard. This settings page can be started "
-                "directly: `adb shell am start -a "
-                "android.settings.HARD_KEYBOARD_SETTINGS`.\n"
-                "However, the option is only available when the HID keyboard "
-                "is enabled (or a physical keyboard is connected).\n"
-                "Also see --hid-mouse.",
     },
     {
         .longopt_id = OPT_LEGACY_PASTE,
@@ -432,15 +454,14 @@ static const struct sc_option options[] = {
                 "Default is 0 (unlimited).",
     },
     {
-        .shortopt = 'M',
+        // deprecated
+        //.shortopt = 'M', // old, reassigned
+        .longopt_id = OPT_HID_MOUSE_DEPRECATED,
         .longopt = "hid-mouse",
-        .text = "Simulate a physical mouse by using HID over AOAv2.\n"
-                "In this mode, the computer mouse is captured to control the "
-                "device directly (relative mouse mode).\n"
-                "LAlt, LSuper or RSuper toggle the capture mode, to give "
-                "control of the mouse back to the computer.\n"
-                "It may only work over USB.\n"
-                "Also see --hid-keyboard.",
+    },
+    {
+        .shortopt = 'M',
+        .text = "Same as --mouse=uhid.",
     },
     {
         .longopt_id = OPT_MAX_FPS,
@@ -449,6 +470,43 @@ static const struct sc_option options[] = {
         .text = "Limit the frame rate of screen capture (officially supported "
                 "since Android 10, but may work on earlier versions).",
     },
+    {
+        .longopt_id = OPT_MOUSE,
+        .longopt = "mouse",
+        .argdesc = "mode",
+        .text = "Select how to send mouse inputs to the device.\n"
+                "Possible values are \"disabled\", \"sdk\", \"uhid\" and "
+                "\"aoa\".\n"
+                "\"disabled\" does not send mouse inputs to the device.\n"
+                "\"sdk\" uses the Android system API to deliver mouse events"
+                "to applications.\n"
+                "\"uhid\" simulates a physical HID mouse using the Linux UHID "
+                "kernel module on the device.\n"
+                "\"aoa\" simulates a physical mouse using the AOAv2 protocol. "
+                "It may only work over USB.\n"
+                "In \"uhid\" and \"aoa\" modes, the computer mouse is captured "
+                "to control the device directly (relative mouse mode).\n"
+                "LAlt, LSuper or RSuper toggle the capture mode, to give "
+                "control of the mouse back to the computer.\n"
+                "Also see --keyboard.",
+    },
+    {
+        .longopt_id = OPT_MOUSE_BIND,
+        .longopt = "mouse-bind",
+        .argdesc = "xxxx",
+        .text = "Configure bindings of secondary clicks.\n"
+                "The argument must be exactly 4 characters, one for each "
+                "secondary click (in order: right click, middle click, 4th "
+                "click, 5th click).\n"
+                "Each character must be one of the following:\n"
+                " '+': forward the click to the device\n"
+                " '-': ignore the click\n"
+                " 'b': trigger shortcut BACK (or turn screen on if off)\n"
+                " 'h': trigger shortcut HOME\n"
+                " 's': trigger shortcut APP_SWITCH\n"
+                " 'n': trigger shortcut \"expand notification panel\"\n"
+                "Default is 'bhsn' for SDK mouse, and '++++' for AOA and UHID.",
+    },
     {
         .shortopt = 'n',
         .longopt = "no-control",
@@ -511,6 +569,12 @@ static const struct sc_option options[] = {
                 "mipmaps are automatically generated to improve downscaling "
                 "quality. This option disables the generation of mipmaps.",
     },
+    {
+        .longopt_id = OPT_NO_MOUSE_HOVER,
+        .longopt = "no-mouse-hover",
+        .text = "Do not forward mouse hover (mouse motion without any clicks) "
+                "events.",
+    },
     {
         .longopt_id = OPT_NO_POWER_ON,
         .longopt = "no-power-on",
@@ -526,6 +590,12 @@ static const struct sc_option options[] = {
         .longopt = "no-video-playback",
         .text = "Disable video playback on the computer.",
     },
+    {
+        .longopt_id = OPT_NO_WINDOW,
+        .longopt = "no-window",
+        .text = "Disable scrcpy window. Implies --no-video-playback and "
+                "--no-control.",
+    },
     {
         .longopt_id = OPT_ORIENTATION,
         .longopt = "orientation",
@@ -543,10 +613,10 @@ static const struct sc_option options[] = {
                 "mirroring is disabled.\n"
                 "LAlt, LSuper or RSuper toggle the mouse capture mode, to give "
                 "control of the mouse back to the computer.\n"
-                "If any of --hid-keyboard or --hid-mouse is set, only enable "
-                "keyboard or mouse respectively, otherwise enable both.\n"
+                "Keyboard and mouse may be disabled separately using"
+                "--keyboard=disabled and --mouse=disabled.\n"
                 "It may only work over USB.\n"
-                "See --hid-keyboard and --hid-mouse.",
+                "See --keyboard and --mouse.",
     },
     {
         .shortopt = 'p',
@@ -669,10 +739,10 @@ static const struct sc_option options[] = {
         .text = "Specify the modifiers to use for scrcpy shortcuts.\n"
                 "Possible keys are \"lctrl\", \"rctrl\", \"lalt\", \"ralt\", "
                 "\"lsuper\" and \"rsuper\".\n"
-                "A shortcut can consist in several keys, separated by '+'. "
-                "Several shortcuts can be specified, separated by ','.\n"
-                "For example, to use either LCtrl+LAlt or LSuper for scrcpy "
-                "shortcuts, pass \"lctrl+lalt,lsuper\".\n"
+                "Several shortcut modifiers can be specified, separated by "
+                "','.\n"
+                "For example, to use either LCtrl or LSuper for scrcpy "
+                "shortcuts, pass \"lctrl,lsuper\".\n"
                 "Default is \"lalt,lsuper\" (left-Alt or left-Super).",
     },
     {
@@ -860,6 +930,14 @@ static const struct sc_shortcut shortcuts[] = {
         .shortcuts = { "MOD+Shift+Up", "MOD+Shift+Down" },
         .text = "Flip display vertically",
     },
+    {
+        .shortcuts = { "MOD+z" },
+        .text = "Pause or re-pause display",
+    },
+    {
+        .shortcuts = { "MOD+Shift+z" },
+        .text = "Unpause display",
+    },
     {
         .shortcuts = { "MOD+g" },
         .text = "Resize window to 1:1 (pixel-perfect)",
@@ -941,6 +1019,10 @@ static const struct sc_shortcut shortcuts[] = {
         .shortcuts = { "MOD+Shift+v" },
         .text = "Inject computer clipboard text as a sequence of key events",
     },
+    {
+        .shortcuts = { "MOD+k" },
+        .text = "Open keyboard settings on the device (for HID keyboard only)",
+    },
     {
         .shortcuts = { "MOD+i" },
         .text = "Enable/disable FPS counter (print frames/second in logs)",
@@ -1628,82 +1710,62 @@ parse_log_level(const char *s, enum sc_log_level *log_level) {
     return false;
 }
 
-// item is a list of mod keys separated by '+' (e.g. "lctrl+lalt")
-// returns a bitwise-or of SC_SHORTCUT_MOD_* constants (or 0 on error)
-static unsigned
+static enum sc_shortcut_mod
 parse_shortcut_mods_item(const char *item, size_t len) {
-    unsigned mod = 0;
-
-    for (;;) {
-        char *plus = strchr(item, '+');
-        // strchr() does not consider the "len" parameter, to it could find an
-        // occurrence too far in the string (there is no strnchr())
-        bool has_plus = plus && plus < item + len;
-
-        assert(!has_plus || plus > item);
-        size_t key_len = has_plus ? (size_t) (plus - item) : len;
-
 #define STREQ(literal, s, len) \
     ((sizeof(literal)-1 == len) && !memcmp(literal, s, len))
 
-        if (STREQ("lctrl", item, key_len)) {
-            mod |= SC_SHORTCUT_MOD_LCTRL;
-        } else if (STREQ("rctrl", item, key_len)) {
-            mod |= SC_SHORTCUT_MOD_RCTRL;
-        } else if (STREQ("lalt", item, key_len)) {
-            mod |= SC_SHORTCUT_MOD_LALT;
-        } else if (STREQ("ralt", item, key_len)) {
-            mod |= SC_SHORTCUT_MOD_RALT;
-        } else if (STREQ("lsuper", item, key_len)) {
-            mod |= SC_SHORTCUT_MOD_LSUPER;
-        } else if (STREQ("rsuper", item, key_len)) {
-            mod |= SC_SHORTCUT_MOD_RSUPER;
-        } else {
-            LOGE("Unknown modifier key: %.*s "
-                 "(must be one of: lctrl, rctrl, lalt, ralt, lsuper, rsuper)",
-                 (int) key_len, item);
-            return 0;
-        }
+    if (STREQ("lctrl", item, len)) {
+        return SC_SHORTCUT_MOD_LCTRL;
+    }
+    if (STREQ("rctrl", item, len)) {
+        return SC_SHORTCUT_MOD_RCTRL;
+    }
+    if (STREQ("lalt", item, len)) {
+        return SC_SHORTCUT_MOD_LALT;
+    }
+    if (STREQ("ralt", item, len)) {
+        return SC_SHORTCUT_MOD_RALT;
+    }
+    if (STREQ("lsuper", item, len)) {
+        return SC_SHORTCUT_MOD_LSUPER;
+    }
+    if (STREQ("rsuper", item, len)) {
+        return SC_SHORTCUT_MOD_RSUPER;
+    }
 #undef STREQ
 
-        if (!has_plus) {
-            break;
-        }
-
-        item = plus + 1;
-        assert(len >= key_len + 1);
-        len -= key_len + 1;
+    bool has_plus = strchr(item, '+');
+    if (has_plus) {
+        LOGE("Shortcut mod combination with '+' is not supported anymore: "
+             "'%.*s' (see #4741)", (int) len, item);
+        return 0;
     }
 
-    return mod;
+    LOGE("Unknown modifier key: %.*s "
+         "(must be one of: lctrl, rctrl, lalt, ralt, lsuper, rsuper)",
+         (int) len, item);
+
+    return 0;
 }
 
 static bool
-parse_shortcut_mods(const char *s, struct sc_shortcut_mods *mods) {
-    unsigned count = 0;
-    unsigned current = 0;
+parse_shortcut_mods(const char *s, uint8_t *shortcut_mods) {
+    uint8_t mods = 0;
 
-    // LCtrl+LAlt or RCtrl or LCtrl+RSuper: "lctrl+lalt,rctrl,lctrl+rsuper"
+    // A list of shortcut modifiers, for example "lctrl,rctrl,rsuper"
 
     for (;;) {
         char *comma = strchr(s, ',');
-        if (comma && count == SC_MAX_SHORTCUT_MODS - 1) {
-            assert(count < SC_MAX_SHORTCUT_MODS);
-            LOGW("Too many shortcut modifiers alternatives");
-            return false;
-        }
-
         assert(!comma || comma > s);
         size_t limit = comma ? (size_t) (comma - s) : strlen(s);
 
-        unsigned mod = parse_shortcut_mods_item(s, limit);
+        enum sc_shortcut_mod mod = parse_shortcut_mods_item(s, limit);
         if (!mod) {
-            LOGE("Invalid modifier keys: %.*s", (int) limit, s);
             return false;
         }
 
-        mods->data[current++] = mod;
-        ++count;
+        mods |= mod;
 
         if (!comma) {
             break;
@@ -1712,7 +1774,7 @@ parse_shortcut_mods(const char *s, struct sc_shortcut_mods *mods) {
         s = comma + 1;
     }
 
-    mods->count = count;
+    *shortcut_mods = mods;
 
     return true;
 }
@@ -1720,7 +1782,7 @@ parse_shortcut_mods(const char *s, struct sc_shortcut_mods *mods) {
 #ifdef SC_TEST
 // expose the function to unit-tests
 bool
-sc_parse_shortcut_mods(const char *s, struct sc_shortcut_mods *mods) {
+sc_parse_shortcut_mods(const char *s, uint8_t *mods) {
     return parse_shortcut_mods(s, mods);
 }
 #endif
@@ -1906,6 +1968,69 @@ parse_camera_fps(const char *s, uint16_t *camera_fps) {
     return true;
 }
 
+static bool
+parse_keyboard(const char *optarg, enum sc_keyboard_input_mode *mode) {
+    if (!strcmp(optarg, "disabled")) {
+        *mode = SC_KEYBOARD_INPUT_MODE_DISABLED;
+        return true;
+    }
+
+    if (!strcmp(optarg, "sdk")) {
+        *mode = SC_KEYBOARD_INPUT_MODE_SDK;
+        return true;
+    }
+
+    if (!strcmp(optarg, "uhid")) {
+        *mode = SC_KEYBOARD_INPUT_MODE_UHID;
+        return true;
+    }
+
+    if (!strcmp(optarg, "aoa")) {
+#ifdef HAVE_USB
+        *mode = SC_KEYBOARD_INPUT_MODE_AOA;
+        return true;
+#else
+        LOGE("--keyboard=aoa is disabled.");
+        return false;
+#endif
+    }
+
+    LOGE("Unsupported keyboard: %s (expected disabled, sdk, uhid and aoa)",
+         optarg);
+    return false;
+}
+
+static bool
+parse_mouse(const char *optarg, enum sc_mouse_input_mode *mode) {
+    if (!strcmp(optarg, "disabled")) {
+        *mode = SC_MOUSE_INPUT_MODE_DISABLED;
+        return true;
+    }
+
+    if (!strcmp(optarg, "sdk")) {
+        *mode = SC_MOUSE_INPUT_MODE_SDK;
+        return true;
+    }
+
+    if (!strcmp(optarg, "uhid")) {
+        *mode = SC_MOUSE_INPUT_MODE_UHID;
+        return true;
+    }
+
+    if (!strcmp(optarg, "aoa")) {
+#ifdef HAVE_USB
+        *mode = SC_MOUSE_INPUT_MODE_AOA;
+        return true;
+#else
+        LOGE("--mouse=aoa is disabled.");
+        return false;
+#endif
+    }
+
+    LOGE("Unsupported mouse: %s (expected disabled, sdk, uhid or aoa)", optarg);
+    return false;
+}
+
 static bool
 parse_time_limit(const char *s, sc_tick *tick) {
     long value;
@@ -1936,11 +2061,63 @@ parse_pause_on_exit(const char *s, enum sc_pause_on_exit *pause_on_exit) {
     }
 
     LOGE("Unsupported pause on exit mode: %s "
-         "(expected true, false or if-error)", optarg);
+         "(expected true, false or if-error)", s);
     return false;
 
 }
 
+static bool
+parse_mouse_binding(char c, enum sc_mouse_binding *b) {
+    switch (c) {
+        case '+':
+            *b = SC_MOUSE_BINDING_CLICK;
+            return true;
+        case '-':
+            *b = SC_MOUSE_BINDING_DISABLED;
+            return true;
+        case 'b':
+            *b = SC_MOUSE_BINDING_BACK;
+            return true;
+        case 'h':
+            *b = SC_MOUSE_BINDING_HOME;
+            return true;
+        case 's':
+            *b = SC_MOUSE_BINDING_APP_SWITCH;
+            return true;
+        case 'n':
+            *b = SC_MOUSE_BINDING_EXPAND_NOTIFICATION_PANEL;
+            return true;
+        default:
+            LOGE("Invalid mouse binding: '%c' "
+                 "(expected '+', '-', 'b', 'h', 's' or 'n')", c);
+            return false;
+    }
+}
+
+static bool
+parse_mouse_bindings(const char *s, struct sc_mouse_bindings *mb) {
+    if (strlen(s) != 4) {
+        LOGE("Invalid mouse bindings: '%s' (expected exactly 4 characters from "
+             "{'+', '-', 'b', 'h', 's', 'n'})", s);
+        return false;
+    }
+
+    if (!parse_mouse_binding(s[0], &mb->right_click)) {
+        return false;
+    }
+    if (!parse_mouse_binding(s[1], &mb->middle_click)) {
+        return false;
+    }
+    if (!parse_mouse_binding(s[2], &mb->click4)) {
+        return false;
+    }
+    if (!parse_mouse_binding(s[3], &mb->click5)) {
+        return false;
+    }
+
+    return true;
+}
+
 static bool
 parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
                        const char *optstring, const struct option *longopts) {
@@ -1994,13 +2171,17 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
                 args->help = true;
                 break;
             case 'K':
-#ifdef HAVE_USB
-                opts->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_HID;
+                opts->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_UHID;
                 break;
-#else
-                LOGE("HID over AOA (-K/--hid-keyboard) is disabled.");
+            case OPT_KEYBOARD:
+                if (!parse_keyboard(optarg, &opts->keyboard_input_mode)) {
+                    return false;
+                }
+                break;
+            case OPT_HID_KEYBOARD_DEPRECATED:
+                LOGE("--hid-keyboard has been removed, use --keyboard=aoa or "
+                     "--keyboard=uhid instead.");
                 return false;
-#endif
             case OPT_MAX_FPS:
                 if (!parse_max_fps(optarg, &opts->max_fps)) {
                     return false;
@@ -2012,13 +2193,25 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
                 }
                 break;
             case 'M':
-#ifdef HAVE_USB
-                opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_HID;
+                opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_UHID;
                 break;
-#else
-                LOGE("HID over AOA (-M/--hid-mouse) is disabled.");
+            case OPT_MOUSE:
+                if (!parse_mouse(optarg, &opts->mouse_input_mode)) {
+                    return false;
+                }
+                break;
+            case OPT_MOUSE_BIND:
+                if (!parse_mouse_bindings(optarg, &opts->mouse_bindings)) {
+                    return false;
+                }
+                break;
+            case OPT_NO_MOUSE_HOVER:
+                opts->mouse_hover = false;
+                break;
+            case OPT_HID_MOUSE_DEPRECATED:
+                LOGE("--hid-mouse has been removed, use --mouse=aoa or "
+                     "--mouse=uhid instead.");
                 return false;
-#endif
             case OPT_LOCK_VIDEO_ORIENTATION:
                 if (!parse_lock_video_orientation(optarg,
                         &opts->lock_video_orientation)) {
@@ -2212,7 +2405,14 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
                 }
                 break;
             case OPT_FORWARD_ALL_CLICKS:
-                opts->forward_all_clicks = true;
+                LOGW("--forward-all-clicks is deprecated, "
+                     "use --mouse-bind=++++ instead.");
+                opts->mouse_bindings = (struct sc_mouse_bindings) {
+                    .right_click = SC_MOUSE_BINDING_CLICK,
+                    .middle_click = SC_MOUSE_BINDING_CLICK,
+                    .click4 = SC_MOUSE_BINDING_CLICK,
+                    .click5 = SC_MOUSE_BINDING_CLICK,
+                };
                 break;
             case OPT_LEGACY_PASTE:
                 opts->legacy_paste = true;
@@ -2363,6 +2563,9 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
             case OPT_CAMERA_HIGH_SPEED:
                 opts->camera_high_speed = true;
                 break;
+            case OPT_NO_WINDOW:
+                opts->window = false;
+                break;
             default:
                 // getopt prints the error message on stderr
                 return false;
@@ -2400,6 +2603,12 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
     v4l2 = !!opts->v4l2_device;
 #endif
 
+    if (!opts->window) {
+        // Without window, there cannot be any video playback or control
+        opts->video_playback = false;
+        opts->control = false;
+    }
+
     if (!opts->video) {
         opts->video_playback = false;
         // Do not power on the device on start if video capture is disabled
@@ -2421,8 +2630,8 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
         opts->audio = false;
     }
 
-    if (!opts->video && !opts->audio && !otg) {
-        LOGE("No video, no audio, no OTG: nothing to do");
+    if (!opts->video && !opts->audio && !opts->control && !otg) {
+        LOGE("No video, no audio, no control, no OTG: nothing to do");
         return false;
     }
 
@@ -2433,9 +2642,9 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
 
     if (opts->audio_playback && opts->audio_buffer == -1) {
         if (opts->audio_codec == SC_CODEC_FLAC) {
-            // Use 50 ms audio buffer by default, but use a higher value for FLAC,
-            // which is not low latency (the default encoder produces blocks of
-            // 4096 samples, which represent ~85.333ms).
+            // Use 50 ms audio buffer by default, but use a higher value for
+            // FLAC, which is not low latency (the default encoder produces
+            // blocks of 4096 samples, which represent ~85.333ms).
             LOGI("FLAC audio: audio buffer increased to 120 ms (use "
                  "--audio-buffer to set a custom value)");
             opts->audio_buffer = SC_TICK_FROM_MS(120);
@@ -2446,6 +2655,11 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
 
 #ifdef HAVE_V4L2
     if (v4l2) {
+        if (!opts->video) {
+            LOGE("V4L2 sink requires video capture, but --no-video was set.");
+            return false;
+        }
+
         if (opts->lock_video_orientation ==
                 SC_LOCK_VIDEO_ORIENTATION_UNLOCKED) {
             LOGI("Video orientation is locked for v4l2 sink. "
@@ -2465,6 +2679,101 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
     }
 #endif
 
+    if (opts->control) {
+        if (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AUTO) {
+            opts->keyboard_input_mode = otg ? SC_KEYBOARD_INPUT_MODE_AOA
+                                            : SC_KEYBOARD_INPUT_MODE_SDK;
+        }
+        if (opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_AUTO) {
+            if (otg) {
+                opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_AOA;
+            } else if (!opts->video_playback) {
+                LOGI("No video mirroring, mouse mode switched to UHID");
+                opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_UHID;
+            } else {
+                opts->mouse_input_mode = SC_MOUSE_INPUT_MODE_SDK;
+            }
+        } else if (opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_SDK
+                    && !opts->video_playback) {
+            LOGE("SDK mouse mode requires video playback. Try --mouse=uhid.");
+            return false;
+        }
+    }
+
+    // If mouse bindings are not explictly set, configure default bindings
+    if (opts->mouse_bindings.right_click == SC_MOUSE_BINDING_AUTO) {
+        assert(opts->mouse_bindings.middle_click == SC_MOUSE_BINDING_AUTO);
+        assert(opts->mouse_bindings.click4 == SC_MOUSE_BINDING_AUTO);
+        assert(opts->mouse_bindings.click5 == SC_MOUSE_BINDING_AUTO);
+
+        // By default, forward all clicks only for UHID and AOA
+        if (opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_SDK) {
+            opts->mouse_bindings = (struct sc_mouse_bindings) {
+                .right_click = SC_MOUSE_BINDING_BACK,
+                .middle_click = SC_MOUSE_BINDING_HOME,
+                .click4 = SC_MOUSE_BINDING_APP_SWITCH,
+                .click5 = SC_MOUSE_BINDING_EXPAND_NOTIFICATION_PANEL,
+            };
+        } else {
+            opts->mouse_bindings = (struct sc_mouse_bindings) {
+                .right_click = SC_MOUSE_BINDING_CLICK,
+                .middle_click = SC_MOUSE_BINDING_CLICK,
+                .click4 = SC_MOUSE_BINDING_CLICK,
+                .click5 = SC_MOUSE_BINDING_CLICK,
+            };
+        }
+    }
+
+    if (otg) {
+        if (!opts->control) {
+            LOGE("--no-control is not allowed in OTG mode");
+            return false;
+        }
+
+        enum sc_keyboard_input_mode kmode = opts->keyboard_input_mode;
+        if (kmode != SC_KEYBOARD_INPUT_MODE_AOA
+                && kmode != SC_KEYBOARD_INPUT_MODE_DISABLED) {
+            LOGE("In OTG mode, --keyboard only supports aoa or disabled.");
+            return false;
+        }
+
+        enum sc_mouse_input_mode mmode = opts->mouse_input_mode;
+        if (mmode != SC_MOUSE_INPUT_MODE_AOA
+                && mmode != SC_MOUSE_INPUT_MODE_DISABLED) {
+            LOGE("In OTG mode, --mouse only supports aoa or disabled.");
+            return false;
+        }
+
+        if (kmode == SC_KEYBOARD_INPUT_MODE_DISABLED
+                && mmode == SC_MOUSE_INPUT_MODE_DISABLED) {
+            LOGE("Could not disable both keyboard and mouse in OTG mode.");
+            return false;
+        }
+    }
+
+    if (opts->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_SDK) {
+        if (opts->key_inject_mode == SC_KEY_INJECT_MODE_TEXT) {
+            LOGE("--prefer-text is specific to --keyboard=sdk");
+            return false;
+        }
+
+        if (opts->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
+            LOGE("--raw-key-events is specific to --keyboard=sdk");
+            return false;
+        }
+
+        if (!opts->forward_key_repeat) {
+            LOGE("--no-key-repeat is specific to --keyboard=sdk");
+            return false;
+        }
+    }
+
+    if (opts->mouse_input_mode != SC_MOUSE_INPUT_MODE_SDK
+            && !opts->mouse_hover) {
+        LOGE("--no-mouse-over is specific to --mouse=sdk");
+        return false;
+    }
+
     if ((opts->tunnel_host || opts->tunnel_port) && !opts->force_adb_forward) {
         LOGI("Tunnel host/port is set, "
              "--force-adb-forward automatically enabled.");
@@ -2529,6 +2838,11 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
     }
 
     if (opts->record_filename) {
+        if (!opts->video && !opts->audio) {
+            LOGE("Video and audio disabled, nothing to record");
+            return false;
+        }
+
         if (!opts->record_format) {
             opts->record_format = guess_record_format(opts->record_filename);
             if (!opts->record_format) {
@@ -2625,16 +2939,21 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
     }
 
 # ifdef _WIN32
-    if (!otg && (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID
-                || opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID)) {
+    if (!otg && (opts->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA
+                || opts->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA)) {
         LOGE("On Windows, it is not possible to open a USB device already open "
              "by another process (like adb).");
-        LOGE("Therefore, -K/--hid-keyboard and -M/--hid-mouse may only work in "
-             "OTG mode (--otg).");
+        LOGE("Therefore, --keyboard=aoa and --mouse=aoa may only work in OTG"
+             "mode (--otg).");
         return false;
     }
 # endif
 
+    if (opts->start_fps_counter && !opts->video_playback) {
+        LOGW("--print-fps has no effect without video playback");
+        opts->start_fps_counter = false;
+    }
+
     if (otg) {
         // OTG mode is compatible with only very few options.
         // Only report obvious errors.

app/src/cli.h

@@ -28,7 +28,7 @@ scrcpy_parse_args(struct scrcpy_cli_args *args, int argc, char *argv[]);
 
 #ifdef SC_TEST
 bool
-sc_parse_shortcut_mods(const char *s, struct sc_shortcut_mods *mods);
+sc_parse_shortcut_mods(const char *s, uint8_t *shortcut_mods);
 #endif
 
 #endif

app/src/control_msg.c

@@ -146,10 +146,22 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf) {
         case SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE:
             buf[1] = msg->set_screen_power_mode.mode;
             return 2;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE:
+            sc_write16be(&buf[1], msg->uhid_create.id);
+            sc_write16be(&buf[3], msg->uhid_create.report_desc_size);
+            memcpy(&buf[5], msg->uhid_create.report_desc,
+                            msg->uhid_create.report_desc_size);
+            return 5 + msg->uhid_create.report_desc_size;
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT:
+            sc_write16be(&buf[1], msg->uhid_input.id);
+            sc_write16be(&buf[3], msg->uhid_input.size);
+            memcpy(&buf[5], msg->uhid_input.data, msg->uhid_input.size);
+            return 5 + msg->uhid_input.size;
         case SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL:
         case SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL:
         case SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS:
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
             // no additional data
             return 1;
         default:
@@ -242,6 +254,26 @@ sc_control_msg_log(const struct sc_control_msg *msg) {
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
             LOG_CMSG("rotate device");
             break;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE:
+            LOG_CMSG("UHID create [%" PRIu16 "] report_desc_size=%" PRIu16,
+                     msg->uhid_create.id, msg->uhid_create.report_desc_size);
+            break;
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT: {
+            char *hex = sc_str_to_hex_string(msg->uhid_input.data,
+                                             msg->uhid_input.size);
+            if (hex) {
+                LOG_CMSG("UHID input [%" PRIu16 "] %s",
+                         msg->uhid_input.id, hex);
+                free(hex);
+            } else {
+                LOG_CMSG("UHID input [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_input.id, msg->uhid_input.size);
+            }
+            break;
+        }
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
+            LOG_CMSG("open hard keyboard settings");
+            break;
         default:
             LOG_CMSG("unknown type: %u", (unsigned) msg->type);
             break;

app/src/control_msg.h

@@ -10,6 +10,7 @@
 #include "android/input.h"
 #include "android/keycodes.h"
 #include "coords.h"
+#include "hid/hid_event.h"
 
 #define SC_CONTROL_MSG_MAX_SIZE (1 << 18) // 256k
 
@@ -37,6 +38,9 @@ enum sc_control_msg_type {
     SC_CONTROL_MSG_TYPE_SET_CLIPBOARD,
     SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE,
     SC_CONTROL_MSG_TYPE_ROTATE_DEVICE,
+    SC_CONTROL_MSG_TYPE_UHID_CREATE,
+    SC_CONTROL_MSG_TYPE_UHID_INPUT,
+    SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
 };
 
 enum sc_screen_power_mode {
@@ -92,6 +96,16 @@ struct sc_control_msg {
         struct {
             enum sc_screen_power_mode mode;
         } set_screen_power_mode;
+        struct {
+            uint16_t id;
+            uint16_t report_desc_size;
+            const uint8_t *report_desc; // pointer to static data
+        } uhid_create;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t data[SC_HID_MAX_SIZE];
+        } uhid_input;
     };
 };
 

app/src/controller.c

@@ -6,9 +6,19 @@
 
 #define SC_CONTROL_MSG_QUEUE_MAX 64
 
+static void
+sc_controller_receiver_on_error(struct sc_receiver *receiver, void *userdata) {
+    (void) receiver;
+
+    struct sc_controller *controller = userdata;
+    // Forward the event to the controller listener
+    controller->cbs->on_error(controller, controller->cbs_userdata);
+}
+
 bool
 sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync) {
+                   const struct sc_controller_callbacks *cbs,
+                   void *cbs_userdata) {
     sc_vecdeque_init(&controller->queue);
 
     bool ok = sc_vecdeque_reserve(&controller->queue, SC_CONTROL_MSG_QUEUE_MAX);
@@ -16,7 +26,12 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
         return false;
     }
 
-    ok = sc_receiver_init(&controller->receiver, control_socket, acksync);
+    static const struct sc_receiver_callbacks receiver_cbs = {
+        .on_error = sc_controller_receiver_on_error,
+    };
+
+    ok = sc_receiver_init(&controller->receiver, control_socket, &receiver_cbs,
+                          controller);
     if (!ok) {
         sc_vecdeque_destroy(&controller->queue);
         return false;
@@ -40,9 +55,21 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
     controller->control_socket = control_socket;
     controller->stopped = false;
 
+    assert(cbs && cbs->on_error);
+    controller->cbs = cbs;
+    controller->cbs_userdata = cbs_userdata;
+
     return true;
 }
 
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices) {
+    controller->receiver.acksync = acksync;
+    controller->receiver.uhid_devices = uhid_devices;
+}
+
 void
 sc_controller_destroy(struct sc_controller *controller) {
     sc_cond_destroy(&controller->msg_cond);
@@ -118,10 +145,16 @@ run_controller(void *data) {
         sc_control_msg_destroy(&msg);
         if (!ok) {
             LOGD("Could not write msg to socket");
-            break;
+            goto error;
         }
     }
+
     return 0;
+
+error:
+    controller->cbs->on_error(controller, controller->cbs_userdata);
+
+    return 1; // ignored
 }
 
 bool

app/src/controller.h

@@ -22,11 +22,24 @@ struct sc_controller {
     bool stopped;
     struct sc_control_msg_queue queue;
     struct sc_receiver receiver;
+
+    const struct sc_controller_callbacks *cbs;
+    void *cbs_userdata;
+};
+
+struct sc_controller_callbacks {
+    void (*on_error)(struct sc_controller *controller, void *userdata);
 };
 
 bool
 sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync);
+                   const struct sc_controller_callbacks *cbs,
+                   void *cbs_userdata);
+
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices);
 
 void
 sc_controller_destroy(struct sc_controller *controller);

app/src/device_msg.c

@@ -46,6 +46,31 @@ sc_device_msg_deserialize(const uint8_t *buf, size_t len,
             msg->ack_clipboard.sequence = sequence;
             return 9;
         }
+        case DEVICE_MSG_TYPE_UHID_OUTPUT: {
+            if (len < 5) {
+                // at least id + size
+                return 0; // not available
+            }
+            uint16_t id = sc_read16be(&buf[1]);
+            size_t size = sc_read16be(&buf[3]);
+            if (size < len - 5) {
+                return 0; // not available
+            }
+            uint8_t *data = malloc(size);
+            if (!data) {
+                LOG_OOM();
+                return -1;
+            }
+            if (size) {
+                memcpy(data, &buf[5], size);
+            }
+
+            msg->uhid_output.id = id;
+            msg->uhid_output.size = size;
+            msg->uhid_output.data = data;
+
+            return 5 + size;
+        }
         default:
             LOGW("Unknown device message type: %d", (int) msg->type);
             return -1; // error, we cannot recover
@@ -54,7 +79,15 @@ sc_device_msg_deserialize(const uint8_t *buf, size_t len,
 
 void
 sc_device_msg_destroy(struct sc_device_msg *msg) {
-    if (msg->type == DEVICE_MSG_TYPE_CLIPBOARD) {
-        free(msg->clipboard.text);
+    switch (msg->type) {
+        case DEVICE_MSG_TYPE_CLIPBOARD:
+            free(msg->clipboard.text);
+            break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            free(msg->uhid_output.data);
+            break;
+        default:
+            // nothing to do
+            break;
     }
 }

app/src/device_msg.h

@@ -14,6 +14,7 @@
 enum sc_device_msg_type {
     DEVICE_MSG_TYPE_CLIPBOARD,
     DEVICE_MSG_TYPE_ACK_CLIPBOARD,
+    DEVICE_MSG_TYPE_UHID_OUTPUT,
 };
 
 struct sc_device_msg {
@@ -25,6 +26,11 @@ struct sc_device_msg {
         struct {
             uint64_t sequence;
         } ack_clipboard;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t *data; // owned, to be freed by free()
+        } uhid_output;
     };
 };
 

app/src/display.c

@@ -1,11 +1,34 @@
 #include "display.h"
 
 #include <assert.h>
+#include <libavutil/pixfmt.h>
 
 #include "util/log.h"
 
+static bool
+sc_display_init_novideo_icon(struct sc_display *display,
+                             SDL_Surface *icon_novideo) {
+    assert(icon_novideo);
+
+    if (SDL_RenderSetLogicalSize(display->renderer,
+                                 icon_novideo->w, icon_novideo->h)) {
+        LOGW("Could not set renderer logical size: %s", SDL_GetError());
+        // don't fail
+    }
+
+    display->texture = SDL_CreateTextureFromSurface(display->renderer,
+                                                    icon_novideo);
+    if (!display->texture) {
+        LOGE("Could not create texture: %s", SDL_GetError());
+        return false;
+    }
+
+    return true;
+}
+
 bool
-sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
+sc_display_init(struct sc_display *display, SDL_Window *window,
+                SDL_Surface *icon_novideo, bool mipmaps) {
     display->renderer =
         SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
     if (!display->renderer) {
@@ -62,8 +85,22 @@ sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
         LOGD("Trilinear filtering disabled (not an OpenGL renderer)");
     }
 
+    display->texture = NULL;
     display->pending.flags = 0;
     display->pending.frame = NULL;
+    display->has_frame = false;
+
+    if (icon_novideo) {
+        // Without video, set a static scrcpy icon as window content
+        bool ok = sc_display_init_novideo_icon(display, icon_novideo);
+        if (!ok) {
+#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
+            SDL_GL_DeleteContext(display->gl_context);
+#endif
+            SDL_DestroyRenderer(display->renderer);
+            return false;
+        }
+    }
 
     return true;
 }
@@ -195,9 +232,25 @@ sc_display_set_texture_size(struct sc_display *display, struct sc_size size) {
     return SC_DISPLAY_RESULT_OK;
 }
 
+static SDL_YUV_CONVERSION_MODE
+sc_display_to_sdl_color_range(enum AVColorRange color_range) {
+    return color_range == AVCOL_RANGE_JPEG ? SDL_YUV_CONVERSION_JPEG
+                                           : SDL_YUV_CONVERSION_AUTOMATIC;
+}
+
 static bool
 sc_display_update_texture_internal(struct sc_display *display,
                                    const AVFrame *frame) {
+    if (!display->has_frame) {
+        // First frame
+        display->has_frame = true;
+
+        // Configure YUV color range conversion
+        SDL_YUV_CONVERSION_MODE sdl_color_range =
+            sc_display_to_sdl_color_range(frame->color_range);
+        SDL_SetYUVConversionMode(sdl_color_range);
+    }
+
     int ret = SDL_UpdateYUVTexture(display->texture, NULL,
                                    frame->data[0], frame->linesize[0],
                                    frame->data[1], frame->linesize[1],

app/src/display.h

@@ -33,6 +33,8 @@ struct sc_display {
         struct sc_size size;
         AVFrame *frame;
     } pending;
+
+    bool has_frame;
 };
 
 enum sc_display_result {
@@ -42,7 +44,8 @@ enum sc_display_result {
 };
 
 bool
-sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps);
+sc_display_init(struct sc_display *display, SDL_Window *window,
+                SDL_Surface *icon_novideo, bool mipmaps);
 
 void
 sc_display_destroy(struct sc_display *display);

app/src/events.h

@@ -7,3 +7,4 @@
 #define SC_EVENT_RECORDER_ERROR           (SDL_USEREVENT + 6)
 #define SC_EVENT_SCREEN_INIT_SIZE         (SDL_USEREVENT + 7)
 #define SC_EVENT_TIME_LIMIT_REACHED       (SDL_USEREVENT + 8)
+#define SC_EVENT_CONTROLLER_ERROR         (SDL_USEREVENT + 9)

app/src/hid/hid_event.h

@@ -0,0 +1,15 @@
+#ifndef SC_HID_EVENT_H
+#define SC_HID_EVENT_H
+
+#include "common.h"
+
+#include <stdint.h>
+
+#define SC_HID_MAX_SIZE 8
+
+struct sc_hid_event {
+    uint8_t data[SC_HID_MAX_SIZE];
+    uint8_t size;
+};
+
+#endif

app/src/hid/hid_keyboard.c

@@ -1,40 +1,34 @@
 #include "hid_keyboard.h"
 
-#include <assert.h>
+#include <string.h>
 
-#include "input_events.h"
 #include "util/log.h"
 
-/** Downcast key processor to hid_keyboard */
-#define DOWNCAST(KP) container_of(KP, struct sc_hid_keyboard, key_processor)
+#define SC_HID_MOD_NONE 0x00
+#define SC_HID_MOD_LEFT_CONTROL (1 << 0)
+#define SC_HID_MOD_LEFT_SHIFT (1 << 1)
+#define SC_HID_MOD_LEFT_ALT (1 << 2)
+#define SC_HID_MOD_LEFT_GUI (1 << 3)
+#define SC_HID_MOD_RIGHT_CONTROL (1 << 4)
+#define SC_HID_MOD_RIGHT_SHIFT (1 << 5)
+#define SC_HID_MOD_RIGHT_ALT (1 << 6)
+#define SC_HID_MOD_RIGHT_GUI (1 << 7)
 
-#define HID_KEYBOARD_ACCESSORY_ID 1
-
-#define HID_MODIFIER_NONE 0x00
-#define HID_MODIFIER_LEFT_CONTROL (1 << 0)
-#define HID_MODIFIER_LEFT_SHIFT (1 << 1)
-#define HID_MODIFIER_LEFT_ALT (1 << 2)
-#define HID_MODIFIER_LEFT_GUI (1 << 3)
-#define HID_MODIFIER_RIGHT_CONTROL (1 << 4)
-#define HID_MODIFIER_RIGHT_SHIFT (1 << 5)
-#define HID_MODIFIER_RIGHT_ALT (1 << 6)
-#define HID_MODIFIER_RIGHT_GUI (1 << 7)
-
-#define HID_KEYBOARD_INDEX_MODIFIER 0
-#define HID_KEYBOARD_INDEX_KEYS 2
+#define SC_HID_KEYBOARD_INDEX_MODS 0
+#define SC_HID_KEYBOARD_INDEX_KEYS 2
 
 // USB HID protocol says 6 keys in an event is the requirement for BIOS
 // keyboard support, though OS could support more keys via modifying the report
 // desc. 6 should be enough for scrcpy.
-#define HID_KEYBOARD_MAX_KEYS 6
-#define HID_KEYBOARD_EVENT_SIZE \
-    (HID_KEYBOARD_INDEX_KEYS + HID_KEYBOARD_MAX_KEYS)
+#define SC_HID_KEYBOARD_MAX_KEYS 6
+#define SC_HID_KEYBOARD_EVENT_SIZE \
+    (SC_HID_KEYBOARD_INDEX_KEYS + SC_HID_KEYBOARD_MAX_KEYS)
 
-#define HID_RESERVED 0x00
-#define HID_ERROR_ROLL_OVER 0x01
+#define SC_HID_RESERVED 0x00
+#define SC_HID_ERROR_ROLL_OVER 0x01
 
 /**
- * For HID over AOAv2, only report descriptor is needed.
+ * For HID, only report descriptor is needed.
  *
  * The specification is available here:
  * <https://www.usb.org/sites/default/files/hid1_11.pdf>
@@ -53,7 +47,7 @@
  *
  * (change vid:pid' to your device's vendor ID and product ID).
  */
-static const unsigned char keyboard_report_desc[]  = {
+const uint8_t SC_HID_KEYBOARD_REPORT_DESC[] = {
     // Usage Page (Generic Desktop)
     0x05, 0x01,
     // Usage (Keyboard)
@@ -119,7 +113,7 @@ static const unsigned char keyboard_report_desc[]  = {
     // Report Size (8)
     0x75, 0x08,
     // Report Count (6)
-    0x95, HID_KEYBOARD_MAX_KEYS,
+    0x95, SC_HID_KEYBOARD_MAX_KEYS,
     // Input (Data, Array): Keys
     0x81, 0x00,
 
@@ -127,6 +121,9 @@ static const unsigned char keyboard_report_desc[]  = {
     0xC0
 };
 
+const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN =
+    sizeof(SC_HID_KEYBOARD_REPORT_DESC);
+
 /**
  * A keyboard HID event is 8 bytes long:
  *
@@ -201,51 +198,50 @@ static const unsigned char keyboard_report_desc[]  = {
  *                  +---------------+
  */
 
-static unsigned char
-sdl_keymod_to_hid_modifiers(uint16_t mod) {
-    unsigned char modifiers = HID_MODIFIER_NONE;
-    if (mod & SC_MOD_LCTRL) {
-        modifiers |= HID_MODIFIER_LEFT_CONTROL;
-    }
-    if (mod & SC_MOD_LSHIFT) {
-        modifiers |= HID_MODIFIER_LEFT_SHIFT;
-    }
-    if (mod & SC_MOD_LALT) {
-        modifiers |= HID_MODIFIER_LEFT_ALT;
-    }
-    if (mod & SC_MOD_LGUI) {
-        modifiers |= HID_MODIFIER_LEFT_GUI;
-    }
-    if (mod & SC_MOD_RCTRL) {
-        modifiers |= HID_MODIFIER_RIGHT_CONTROL;
-    }
-    if (mod & SC_MOD_RSHIFT) {
-        modifiers |= HID_MODIFIER_RIGHT_SHIFT;
-    }
-    if (mod & SC_MOD_RALT) {
-        modifiers |= HID_MODIFIER_RIGHT_ALT;
-    }
-    if (mod & SC_MOD_RGUI) {
-        modifiers |= HID_MODIFIER_RIGHT_GUI;
-    }
-    return modifiers;
+static void
+sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
+    hid_event->size = SC_HID_KEYBOARD_EVENT_SIZE;
+
+    uint8_t *data = hid_event->data;
+
+    data[SC_HID_KEYBOARD_INDEX_MODS] = SC_HID_MOD_NONE;
+    data[1] = SC_HID_RESERVED;
+    memset(&data[SC_HID_KEYBOARD_INDEX_KEYS], 0, SC_HID_KEYBOARD_MAX_KEYS);
 }
 
-static bool
-sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
-    unsigned char *buffer = malloc(HID_KEYBOARD_EVENT_SIZE);
-    if (!buffer) {
-        LOG_OOM();
-        return false;
+static uint16_t
+sc_hid_mod_from_sdl_keymod(uint16_t mod) {
+    uint16_t mods = SC_HID_MOD_NONE;
+    if (mod & SC_MOD_LCTRL) {
+        mods |= SC_HID_MOD_LEFT_CONTROL;
     }
+    if (mod & SC_MOD_LSHIFT) {
+        mods |= SC_HID_MOD_LEFT_SHIFT;
+    }
+    if (mod & SC_MOD_LALT) {
+        mods |= SC_HID_MOD_LEFT_ALT;
+    }
+    if (mod & SC_MOD_LGUI) {
+        mods |= SC_HID_MOD_LEFT_GUI;
+    }
+    if (mod & SC_MOD_RCTRL) {
+        mods |= SC_HID_MOD_RIGHT_CONTROL;
+    }
+    if (mod & SC_MOD_RSHIFT) {
+        mods |= SC_HID_MOD_RIGHT_SHIFT;
+    }
+    if (mod & SC_MOD_RALT) {
+        mods |= SC_HID_MOD_RIGHT_ALT;
+    }
+    if (mod & SC_MOD_RGUI) {
+        mods |= SC_HID_MOD_RIGHT_GUI;
+    }
+    return mods;
+}
 
-    buffer[HID_KEYBOARD_INDEX_MODIFIER] = HID_MODIFIER_NONE;
-    buffer[1] = HID_RESERVED;
-    memset(&buffer[HID_KEYBOARD_INDEX_KEYS], 0, HID_KEYBOARD_MAX_KEYS);
-
-    sc_hid_event_init(hid_event, HID_KEYBOARD_ACCESSORY_ID, buffer,
-                      HID_KEYBOARD_EVENT_SIZE);
-    return true;
+void
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid) {
+    memset(hid->keys, false, SC_HID_KEYBOARD_KEYS);
 }
 
 static inline bool
@@ -253,10 +249,10 @@ scancode_is_modifier(enum sc_scancode scancode) {
     return scancode >= SC_SCANCODE_LCTRL && scancode <= SC_SCANCODE_RGUI;
 }
 
-static bool
-convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
-                           struct sc_hid_event *hid_event,
-                           const struct sc_key_event *event) {
+bool
+sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
+                               struct sc_hid_event *hid_event,
+                               const struct sc_key_event *event) {
     enum sc_scancode scancode = event->scancode;
     assert(scancode >= 0);
 
@@ -268,39 +264,37 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
         return false;
     }
 
-    if (!sc_hid_keyboard_event_init(hid_event)) {
-        LOGW("Could not initialize HID keyboard event");
-        return false;
-    }
+    sc_hid_keyboard_event_init(hid_event);
 
-    unsigned char modifiers = sdl_keymod_to_hid_modifiers(event->mods_state);
+    uint16_t mods = sc_hid_mod_from_sdl_keymod(event->mods_state);
 
     if (scancode < SC_HID_KEYBOARD_KEYS) {
         // Pressed is true and released is false
-        kb->keys[scancode] = (event->action == SC_ACTION_DOWN);
+        hid->keys[scancode] = (event->action == SC_ACTION_DOWN);
         LOGV("keys[%02x] = %s", scancode,
-             kb->keys[scancode] ? "true" : "false");
+             hid->keys[scancode] ? "true" : "false");
     }
 
-    hid_event->buffer[HID_KEYBOARD_INDEX_MODIFIER] = modifiers;
+    hid_event->data[SC_HID_KEYBOARD_INDEX_MODS] = mods;
 
-    unsigned char *keys_buffer = &hid_event->buffer[HID_KEYBOARD_INDEX_KEYS];
+    uint8_t *keys_data = &hid_event->data[SC_HID_KEYBOARD_INDEX_KEYS];
     // Re-calculate pressed keys every time
     int keys_pressed_count = 0;
     for (int i = 0; i < SC_HID_KEYBOARD_KEYS; ++i) {
-        if (kb->keys[i]) {
+        if (hid->keys[i]) {
             // USB HID protocol says that if keys exceeds report count, a
             // phantom state should be reported
-            if (keys_pressed_count >= HID_KEYBOARD_MAX_KEYS) {
+            if (keys_pressed_count >= SC_HID_KEYBOARD_MAX_KEYS) {
                 // Phantom state:
                 //  - Modifiers
                 //  - Reserved
                 //  - ErrorRollOver * HID_MAX_KEYS
-                memset(keys_buffer, HID_ERROR_ROLL_OVER, HID_KEYBOARD_MAX_KEYS);
+                memset(keys_data, SC_HID_ERROR_ROLL_OVER,
+                       SC_HID_KEYBOARD_MAX_KEYS);
                 goto end;
             }
 
-            keys_buffer[keys_pressed_count] = i;
+            keys_data[keys_pressed_count] = i;
             ++keys_pressed_count;
         }
     }
@@ -308,124 +302,32 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
 end:
     LOGV("hid keyboard: key %-4s scancode=%02x (%u) mod=%02x",
          event->action == SC_ACTION_DOWN ? "down" : "up", event->scancode,
-         event->scancode, modifiers);
+         event->scancode, mods);
 
     return true;
 }
 
-
-static bool
-push_mod_lock_state(struct sc_hid_keyboard *kb, uint16_t mods_state) {
+bool
+sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
+                                uint16_t mods_state) {
     bool capslock = mods_state & SC_MOD_CAPS;
     bool numlock = mods_state & SC_MOD_NUM;
     if (!capslock && !numlock) {
         // Nothing to do
-        return true;
-    }
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_keyboard_event_init(&hid_event)) {
-        LOGW("Could not initialize HID keyboard event");
         return false;
     }
 
+    sc_hid_keyboard_event_init(event);
+
     unsigned i = 0;
     if (capslock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
+        event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
         ++i;
     }
     if (numlock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
+        event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
         ++i;
     }
 
-    if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mod lock state)");
-        return false;
-    }
-
-    LOGD("HID keyboard state synchronized");
-
     return true;
 }
-
-static void
-sc_key_processor_process_key(struct sc_key_processor *kp,
-                             const struct sc_key_event *event,
-                             uint64_t ack_to_wait) {
-    if (event->repeat) {
-        // In USB HID protocol, key repeat is handled by the host (Android), so
-        // just ignore key repeat here.
-        return;
-    }
-
-    struct sc_hid_keyboard *kb = DOWNCAST(kp);
-
-    struct sc_hid_event hid_event;
-    // Not all keys are supported, just ignore unsupported keys
-    if (convert_hid_keyboard_event(kb, &hid_event, event)) {
-        if (!kb->mod_lock_synchronized) {
-            // Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
-            // keyboard state
-            if (push_mod_lock_state(kb, event->mods_state)) {
-                kb->mod_lock_synchronized = true;
-            }
-        }
-
-        if (ack_to_wait) {
-            // Ctrl+v is pressed, so clipboard synchronization has been
-            // requested. Wait until clipboard synchronization is acknowledged
-            // by the server, otherwise it could paste the old clipboard
-            // content.
-            hid_event.ack_to_wait = ack_to_wait;
-        }
-
-        if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-            sc_hid_event_destroy(&hid_event);
-            LOGW("Could not request HID event (key)");
-        }
-    }
-}
-
-bool
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa) {
-    kb->aoa = aoa;
-
-    bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
-                               keyboard_report_desc,
-                               ARRAY_LEN(keyboard_report_desc));
-    if (!ok) {
-        LOGW("Register HID keyboard failed");
-        return false;
-    }
-
-    // Reset all states
-    memset(kb->keys, false, SC_HID_KEYBOARD_KEYS);
-
-    kb->mod_lock_synchronized = false;
-
-    static const struct sc_key_processor_ops ops = {
-        .process_key = sc_key_processor_process_key,
-        // Never forward text input via HID (all the keys are injected
-        // separately)
-        .process_text = NULL,
-    };
-
-    // Clipboard synchronization is requested over the control socket, while HID
-    // events are sent over AOA, so it must wait for clipboard synchronization
-    // to be acknowledged by the device before injecting Ctrl+v.
-    kb->key_processor.async_paste = true;
-    kb->key_processor.ops = &ops;
-
-    return true;
-}
-
-void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb) {
-    // Unregister HID keyboard so the soft keyboard shows again on Android
-    bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
-    if (!ok) {
-        LOGW("Could not unregister HID keyboard");
-    }
-}

app/src/hid/hid_keyboard.h

@@ -5,8 +5,8 @@
 
 #include <stdbool.h>
 
-#include "aoa_hid.h"
-#include "trait/key_processor.h"
+#include "hid/hid_event.h"
+#include "input_events.h"
 
 // See "SDL2/SDL_scancode.h".
 // Maybe SDL_Keycode is used by most people, but SDL_Scancode is taken from USB
@@ -14,6 +14,9 @@
 // 0x65 is Application, typically AT-101 Keyboard ends here.
 #define SC_HID_KEYBOARD_KEYS 0x66
 
+extern const uint8_t SC_HID_KEYBOARD_REPORT_DESC[];
+extern const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN;
+
 /**
  * HID keyboard events are sequence-based, every time keyboard state changes
  * it sends an array of currently pressed keys, the host is responsible for
@@ -27,18 +30,19 @@
  * phantom state.
  */
 struct sc_hid_keyboard {
-    struct sc_key_processor key_processor; // key processor trait
-
-    struct sc_aoa *aoa;
     bool keys[SC_HID_KEYBOARD_KEYS];
-
-    bool mod_lock_synchronized;
 };
 
-bool
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa);
-
 void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb);
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid);
+
+bool
+sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
+                               struct sc_hid_event *hid_event,
+                               const struct sc_key_event *event);
+
+bool
+sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
+                                uint16_t mods_state);
 
 #endif

app/src/hid/hid_mouse.c

@@ -0,0 +1,192 @@
+#include "hid_mouse.h"
+
+// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position,
+// 1 byte for wheel motion
+#define HID_MOUSE_EVENT_SIZE 4
+
+/**
+ * Mouse descriptor from the specification:
+ * <https://www.usb.org/sites/default/files/hid1_11.pdf>
+ *
+ * Appendix E (p71): §E.10 Report Descriptor (Mouse)
+ *
+ * The usage tags (like Wheel) are listed in "HID Usage Tables":
+ * <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
+ * §4 Generic Desktop Page (0x01) (p26)
+ */
+const uint8_t SC_HID_MOUSE_REPORT_DESC[] = {
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (Mouse)
+    0x09, 0x02,
+
+    // Collection (Application)
+    0xA1, 0x01,
+
+    // Usage (Pointer)
+    0x09, 0x01,
+
+    // Collection (Physical)
+    0xA1, 0x00,
+
+    // Usage Page (Buttons)
+    0x05, 0x09,
+
+    // Usage Minimum (1)
+    0x19, 0x01,
+     // Usage Maximum (5)
+    0x29, 0x05,
+    // Logical Minimum (0)
+    0x15, 0x00,
+    // Logical Maximum (1)
+    0x25, 0x01,
+    // Report Count (5)
+    0x95, 0x05,
+    // Report Size (1)
+    0x75, 0x01,
+    // Input (Data, Variable, Absolute): 5 buttons bits
+    0x81, 0x02,
+
+    // Report Count (1)
+    0x95, 0x01,
+    // Report Size (3)
+    0x75, 0x03,
+    // Input (Constant): 3 bits padding
+    0x81, 0x01,
+
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (X)
+    0x09, 0x30,
+    // Usage (Y)
+    0x09, 0x31,
+    // Usage (Wheel)
+    0x09, 0x38,
+    // Local Minimum (-127)
+    0x15, 0x81,
+    // Local Maximum (127)
+    0x25, 0x7F,
+    // Report Size (8)
+    0x75, 0x08,
+    // Report Count (3)
+    0x95, 0x03,
+    // Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
+    0x81, 0x06,
+
+    // End Collection
+    0xC0,
+
+    // End Collection
+    0xC0,
+};
+
+const size_t SC_HID_MOUSE_REPORT_DESC_LEN =
+    sizeof(SC_HID_MOUSE_REPORT_DESC);
+
+/**
+ * A mouse HID event is 4 bytes long:
+ *
+ *  - byte 0: buttons state
+ *  - byte 1: relative x motion (signed byte from -127 to 127)
+ *  - byte 2: relative y motion (signed byte from -127 to 127)
+ *  - byte 3: wheel motion (-1, 0 or 1)
+ *
+ *                   7 6 5 4 3 2 1 0
+ *                  +---------------+
+ *         byte 0:  |0 0 0 . . . . .| buttons state
+ *                  +---------------+
+ *                         ^ ^ ^ ^ ^
+ *                         | | | | `- left button
+ *                         | | | `--- right button
+ *                         | | `----- middle button
+ *                         | `------- button 4
+ *                         `--------- button 5
+ *
+ *                  +---------------+
+ *         byte 1:  |. . . . . . . .| relative x motion
+ *                  +---------------+
+ *         byte 2:  |. . . . . . . .| relative y motion
+ *                  +---------------+
+ *         byte 3:  |. . . . . . . .| wheel motion
+ *                  +---------------+
+ *
+ * As an example, here is the report for a motion of (x=5, y=-4) with left
+ * button pressed:
+ *
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 1| left button pressed
+ *                  +---------------+
+ *                  |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
+ *                  +---------------+
+ *                  |1 1 1 1 1 1 0 0| relative y motion (y = -4)
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 0| wheel motion
+ *                  +---------------+
+ */
+
+static void
+sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
+    hid_event->size = HID_MOUSE_EVENT_SIZE;
+    // Leave hid_event->data uninitialized, it will be fully initialized by
+    // callers
+}
+
+static uint8_t
+sc_hid_buttons_from_buttons_state(uint8_t buttons_state) {
+    uint8_t c = 0;
+    if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
+        c |= 1 << 0;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
+        c |= 1 << 1;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
+        c |= 1 << 2;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X1) {
+        c |= 1 << 3;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X2) {
+        c |= 1 << 4;
+    }
+    return c;
+}
+
+void
+sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_motion_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = CLAMP(event->xrel, -127, 127);
+    data[2] = CLAMP(event->yrel, -127, 127);
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
+                              const struct sc_mouse_click_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_scroll_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = 0; // buttons state irrelevant (and unknown)
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    // In practice, vscroll is always -1, 0 or 1, but in theory other values
+    // are possible
+    data[3] = CLAMP(event->vscroll, -127, 127);
+    // Horizontal scrolling ignored
+}

app/src/hid/hid_mouse.h

@@ -0,0 +1,26 @@
+#ifndef SC_HID_MOUSE_H
+#define SC_HID_MOUSE_H
+
+#endif
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "hid/hid_event.h"
+#include "input_events.h"
+
+extern const uint8_t SC_HID_MOUSE_REPORT_DESC[];
+extern const size_t SC_HID_MOUSE_REPORT_DESC_LEN;
+
+void
+sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_motion_event *event);
+
+void
+sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
+                              const struct sc_mouse_click_event *event);
+
+void
+sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_scroll_event *event);

app/src/icon.c

@@ -78,7 +78,19 @@ decode_image(const char *path) {
         goto close_input;
     }
 
-    int stream = av_find_best_stream(ctx, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
+
+// In ffmpeg/doc/APIchanges:
+// 2021-04-27 - 46dac8cf3d - lavf 59.0.100 - avformat.h
+//   av_find_best_stream now uses a const AVCodec ** parameter
+//   for the returned decoder.
+#if LIBAVFORMAT_VERSION_INT >= AV_VERSION_INT(59, 0, 100)
+    const AVCodec *codec;
+#else
+    AVCodec *codec;
+#endif
+
+    int stream =
+        av_find_best_stream(ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &codec, 0);
     if (stream < 0 ) {
         LOGE("Could not find best image stream");
         goto close_input;
@@ -86,12 +98,6 @@ decode_image(const char *path) {
 
     AVCodecParameters *params = ctx->streams[stream]->codecpar;
 
-    const AVCodec *codec = avcodec_find_decoder(params->codec_id);
-    if (!codec) {
-        LOGE("Could not find image decoder");
-        goto close_input;
-    }
-
     AVCodecContext *codec_ctx = avcodec_alloc_context3(codec);
     if (!codec_ctx) {
         LOG_OOM();

app/src/input_events.h

@@ -9,6 +9,7 @@
 #include <SDL2/SDL_events.h>
 
 #include "coords.h"
+#include "options.h"
 
 /* The representation of input events in scrcpy is very close to the SDL API,
  * for simplicity.
@@ -437,15 +438,21 @@ sc_mouse_button_from_sdl(uint8_t button) {
 
 static inline uint8_t
 sc_mouse_buttons_state_from_sdl(uint32_t buttons_state,
-                                bool forward_all_clicks) {
+                                const struct sc_mouse_bindings *mb) {
     assert(buttons_state < 0x100); // fits in uint8_t
 
     uint8_t mask = SC_MOUSE_BUTTON_LEFT;
-    if (forward_all_clicks) {
-        mask |= SC_MOUSE_BUTTON_RIGHT
-              | SC_MOUSE_BUTTON_MIDDLE
-              | SC_MOUSE_BUTTON_X1
-              | SC_MOUSE_BUTTON_X2;
+    if (!mb || mb->right_click == SC_MOUSE_BINDING_CLICK) {
+        mask |= SC_MOUSE_BUTTON_RIGHT;
+    }
+    if (!mb || mb->middle_click == SC_MOUSE_BINDING_CLICK) {
+        mask |= SC_MOUSE_BUTTON_MIDDLE;
+    }
+    if (!mb || mb->click4 == SC_MOUSE_BINDING_CLICK) {
+        mask |= SC_MOUSE_BUTTON_X1;
+    }
+    if (!mb || mb->click5 == SC_MOUSE_BINDING_CLICK) {
+        mask |= SC_MOUSE_BUTTON_X2;
     }
 
     return buttons_state & mask;

app/src/input_manager.c

@@ -10,7 +10,7 @@
 #define SC_SDL_SHORTCUT_MODS_MASK (KMOD_CTRL | KMOD_ALT | KMOD_GUI)
 
 static inline uint16_t
-to_sdl_mod(unsigned shortcut_mod) {
+to_sdl_mod(uint8_t shortcut_mod) {
     uint16_t sdl_mod = 0;
     if (shortcut_mod & SC_SHORTCUT_MOD_LCTRL) {
         sdl_mod |= KMOD_LCTRL;
@@ -38,22 +38,36 @@ is_shortcut_mod(struct sc_input_manager *im, uint16_t sdl_mod) {
     // keep only the relevant modifier keys
     sdl_mod &= SC_SDL_SHORTCUT_MODS_MASK;
 
-    assert(im->sdl_shortcut_mods.count);
-    assert(im->sdl_shortcut_mods.count < SC_MAX_SHORTCUT_MODS);
-    for (unsigned i = 0; i < im->sdl_shortcut_mods.count; ++i) {
-        if (im->sdl_shortcut_mods.data[i] == sdl_mod) {
-            return true;
-        }
-    }
+    // at least one shortcut mod pressed?
+    return sdl_mod & im->sdl_shortcut_mods;
+}
 
-    return false;
+static bool
+is_shortcut_key(struct sc_input_manager *im, SDL_Keycode keycode) {
+    return (im->sdl_shortcut_mods & KMOD_LCTRL && keycode == SDLK_LCTRL)
+        || (im->sdl_shortcut_mods & KMOD_RCTRL && keycode == SDLK_RCTRL)
+        || (im->sdl_shortcut_mods & KMOD_LALT  && keycode == SDLK_LALT)
+        || (im->sdl_shortcut_mods & KMOD_RALT  && keycode == SDLK_RALT)
+        || (im->sdl_shortcut_mods & KMOD_LGUI  && keycode == SDLK_LGUI)
+        || (im->sdl_shortcut_mods & KMOD_RGUI  && keycode == SDLK_RGUI);
+}
+
+static inline bool
+mouse_bindings_has_secondary_click(const struct sc_mouse_bindings *mb) {
+    return mb->right_click == SC_MOUSE_BINDING_CLICK
+        || mb->middle_click == SC_MOUSE_BINDING_CLICK
+        || mb->click4 == SC_MOUSE_BINDING_CLICK
+        || mb->click5 == SC_MOUSE_BINDING_CLICK;
 }
 
 void
 sc_input_manager_init(struct sc_input_manager *im,
                       const struct sc_input_manager_params *params) {
-    assert(!params->controller || (params->kp && params->kp->ops));
-    assert(!params->controller || (params->mp && params->mp->ops));
+    // A key/mouse processor may not be present if there is no controller
+    assert((!params->kp && !params->mp) || params->controller);
+    // A processor must have ops initialized
+    assert(!params->kp || params->kp->ops);
+    assert(!params->mp || params->mp->ops);
 
     im->controller = params->controller;
     im->fp = params->fp;
@@ -61,19 +75,13 @@ sc_input_manager_init(struct sc_input_manager *im,
     im->kp = params->kp;
     im->mp = params->mp;
 
-    im->forward_all_clicks = params->forward_all_clicks;
+    im->mouse_bindings = params->mouse_bindings;
+    im->has_secondary_click =
+        mouse_bindings_has_secondary_click(&im->mouse_bindings);
     im->legacy_paste = params->legacy_paste;
     im->clipboard_autosync = params->clipboard_autosync;
 
-    const struct sc_shortcut_mods *shortcut_mods = params->shortcut_mods;
-    assert(shortcut_mods->count);
-    assert(shortcut_mods->count < SC_MAX_SHORTCUT_MODS);
-    for (unsigned i = 0; i < shortcut_mods->count; ++i) {
-        uint16_t sdl_mod = to_sdl_mod(shortcut_mods->data[i]);
-        assert(sdl_mod);
-        im->sdl_shortcut_mods.data[i] = sdl_mod;
-    }
-    im->sdl_shortcut_mods.count = shortcut_mods->count;
+    im->sdl_shortcut_mods = to_sdl_mod(params->shortcut_mods);
 
     im->vfinger_down = false;
     im->vfinger_invert_x = false;
@@ -87,8 +95,10 @@ sc_input_manager_init(struct sc_input_manager *im,
 }
 
 static void
-send_keycode(struct sc_controller *controller, enum android_keycode keycode,
+send_keycode(struct sc_input_manager *im, enum android_keycode keycode,
              enum sc_action action, const char *name) {
+    assert(im->controller && im->kp);
+
     // send DOWN event
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_INJECT_KEYCODE;
@@ -99,100 +109,109 @@ send_keycode(struct sc_controller *controller, enum android_keycode keycode,
     msg.inject_keycode.metastate = 0;
     msg.inject_keycode.repeat = 0;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'inject %s'", name);
     }
 }
 
 static inline void
-action_home(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_HOME, action, "HOME");
+action_home(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_HOME, action, "HOME");
 }
 
 static inline void
-action_back(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_BACK, action, "BACK");
+action_back(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_BACK, action, "BACK");
 }
 
 static inline void
-action_app_switch(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
+action_app_switch(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
 }
 
 static inline void
-action_power(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_POWER, action, "POWER");
+action_power(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_POWER, action, "POWER");
 }
 
 static inline void
-action_volume_up(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
+action_volume_up(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
 }
 
 static inline void
-action_volume_down(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
+action_volume_down(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
 }
 
 static inline void
-action_menu(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_MENU, action, "MENU");
+action_menu(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_MENU, action, "MENU");
 }
 
 // turn the screen on if it was off, press BACK otherwise
 // If the screen is off, it is turned on only on ACTION_DOWN
 static void
-press_back_or_turn_screen_on(struct sc_controller *controller,
+press_back_or_turn_screen_on(struct sc_input_manager *im,
                              enum sc_action action) {
+    assert(im->controller && im->kp);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_BACK_OR_SCREEN_ON;
     msg.back_or_screen_on.action = action == SC_ACTION_DOWN
                                  ? AKEY_EVENT_ACTION_DOWN
                                  : AKEY_EVENT_ACTION_UP;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'press back or turn screen on'");
     }
 }
 
 static void
-expand_notification_panel(struct sc_controller *controller) {
+expand_notification_panel(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'expand notification panel'");
     }
 }
 
 static void
-expand_settings_panel(struct sc_controller *controller) {
+expand_settings_panel(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'expand settings panel'");
     }
 }
 
 static void
-collapse_panels(struct sc_controller *controller) {
+collapse_panels(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'collapse notification panel'");
     }
 }
 
 static bool
-get_device_clipboard(struct sc_controller *controller,
-                     enum sc_copy_key copy_key) {
+get_device_clipboard(struct sc_input_manager *im, enum sc_copy_key copy_key) {
+    assert(im->controller && im->kp);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_GET_CLIPBOARD;
     msg.get_clipboard.copy_key = copy_key;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'get device clipboard'");
         return false;
     }
@@ -201,8 +220,10 @@ get_device_clipboard(struct sc_controller *controller,
 }
 
 static bool
-set_device_clipboard(struct sc_controller *controller, bool paste,
+set_device_clipboard(struct sc_input_manager *im, bool paste,
                      uint64_t sequence) {
+    assert(im->controller && im->kp);
+
     char *text = SDL_GetClipboardText();
     if (!text) {
         LOGW("Could not get clipboard text: %s", SDL_GetError());
@@ -222,7 +243,7 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
     msg.set_clipboard.text = text_dup;
     msg.set_clipboard.paste = paste;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         free(text_dup);
         LOGW("Could not request 'set device clipboard'");
         return false;
@@ -232,19 +253,23 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
 }
 
 static void
-set_screen_power_mode(struct sc_controller *controller,
+set_screen_power_mode(struct sc_input_manager *im,
                       enum sc_screen_power_mode mode) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
     msg.set_screen_power_mode.mode = mode;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'set screen power mode'");
     }
 }
 
 static void
-switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
+switch_fps_counter_state(struct sc_input_manager *im) {
+    struct sc_fps_counter *fps_counter = &im->screen->fps_counter;
+
     // the started state can only be written from the current thread, so there
     // is no ToCToU issue
     if (sc_fps_counter_is_started(fps_counter)) {
@@ -256,7 +281,9 @@ switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
 }
 
 static void
-clipboard_paste(struct sc_controller *controller) {
+clipboard_paste(struct sc_input_manager *im) {
+    assert(im->controller && im->kp);
+
     char *text = SDL_GetClipboardText();
     if (!text) {
         LOGW("Could not get clipboard text: %s", SDL_GetError());
@@ -278,25 +305,40 @@ clipboard_paste(struct sc_controller *controller) {
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_INJECT_TEXT;
     msg.inject_text.text = text_dup;
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         free(text_dup);
         LOGW("Could not request 'paste clipboard'");
     }
 }
 
 static void
-rotate_device(struct sc_controller *controller) {
+rotate_device(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_ROTATE_DEVICE;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request device rotation");
     }
 }
 
 static void
-apply_orientation_transform(struct sc_screen *screen,
+open_hard_keyboard_settings(struct sc_input_manager *im) {
+    assert(im->controller);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS;
+
+    if (!sc_controller_push_msg(im->controller, &msg)) {
+        LOGW("Could not request opening hard keyboard settings");
+    }
+}
+
+static void
+apply_orientation_transform(struct sc_input_manager *im,
                             enum sc_orientation transform) {
+    struct sc_screen *screen = im->screen;
     enum sc_orientation new_orientation =
         sc_orientation_apply(screen->orientation, transform);
     sc_screen_set_orientation(screen, new_orientation);
@@ -334,8 +376,8 @@ simulate_virtual_finger(struct sc_input_manager *im,
     msg.inject_touch_event.position.screen_size = im->screen->frame_size;
     msg.inject_touch_event.position.point = point;
     msg.inject_touch_event.pointer_id =
-        im->forward_all_clicks ? POINTER_ID_VIRTUAL_MOUSE
-                               : POINTER_ID_VIRTUAL_FINGER;
+        im->has_secondary_click ? POINTER_ID_VIRTUAL_MOUSE
+                                : POINTER_ID_VIRTUAL_FINGER;
     msg.inject_touch_event.pressure = up ? 0.0f : 1.0f;
     msg.inject_touch_event.action_button = 0;
     msg.inject_touch_event.buttons = 0;
@@ -364,7 +406,9 @@ static void
 sc_input_manager_process_key(struct sc_input_manager *im,
                              const SDL_KeyboardEvent *event) {
     // controller is NULL if --no-control is requested
-    struct sc_controller *controller = im->controller;
+    bool control = im->controller;
+    bool paused = im->screen->paused;
+    bool video = im->screen->video;
 
     SDL_Keycode keycode = event->keysym.sym;
     uint16_t mod = event->keysym.mod;
@@ -373,7 +417,12 @@ sc_input_manager_process_key(struct sc_input_manager *im,
     bool shift = event->keysym.mod & KMOD_SHIFT;
     bool repeat = event->repeat;
 
-    bool smod = is_shortcut_mod(im, mod);
+    // Either the modifier includes a shortcut modifier, or the key
+    // press/release is a modifier key.
+    // The second condition is necessary to ignore the release of the modifier
+    // key (because in this case mod is 0).
+    bool is_shortcut = is_shortcut_mod(im, mod)
+                    || is_shortcut_key(im, keycode);
 
     if (down && !repeat) {
         if (keycode == im->last_keycode && mod == im->last_mod) {
@@ -385,145 +434,155 @@ sc_input_manager_process_key(struct sc_input_manager *im,
         }
     }
 
-    // The shortcut modifier is pressed
-    if (smod) {
+    if (is_shortcut) {
         enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
         switch (keycode) {
             case SDLK_h:
-                if (controller && !shift && !repeat) {
-                    action_home(controller, action);
+                if (im->kp && !shift && !repeat && !paused) {
+                    action_home(im, action);
                 }
                 return;
             case SDLK_b: // fall-through
             case SDLK_BACKSPACE:
-                if (controller && !shift && !repeat) {
-                    action_back(controller, action);
+                if (im->kp && !shift && !repeat && !paused) {
+                    action_back(im, action);
                 }
                 return;
             case SDLK_s:
-                if (controller && !shift && !repeat) {
-                    action_app_switch(controller, action);
+                if (im->kp && !shift && !repeat && !paused) {
+                    action_app_switch(im, action);
                 }
                 return;
             case SDLK_m:
-                if (controller && !shift && !repeat) {
-                    action_menu(controller, action);
+                if (im->kp && !shift && !repeat && !paused) {
+                    action_menu(im, action);
                 }
                 return;
             case SDLK_p:
-                if (controller && !shift && !repeat) {
-                    action_power(controller, action);
+                if (im->kp && !shift && !repeat && !paused) {
+                    action_power(im, action);
                 }
                 return;
             case SDLK_o:
-                if (controller && !repeat && down) {
+                if (control && !repeat && down && !paused) {
                     enum sc_screen_power_mode mode = shift
                                                    ? SC_SCREEN_POWER_MODE_NORMAL
                                                    : SC_SCREEN_POWER_MODE_OFF;
-                    set_screen_power_mode(controller, mode);
+                    set_screen_power_mode(im, mode);
+                }
+                return;
+            case SDLK_z:
+                if (video && down && !repeat) {
+                    sc_screen_set_paused(im->screen, !shift);
                 }
                 return;
             case SDLK_DOWN:
                 if (shift) {
-                    if (!repeat & down) {
-                        apply_orientation_transform(im->screen,
+                    if (video && !repeat && down) {
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_180);
                     }
-                } else if (controller) {
+                } else if (im->kp && !paused) {
                     // forward repeated events
-                    action_volume_down(controller, action);
+                    action_volume_down(im, action);
                 }
                 return;
             case SDLK_UP:
                 if (shift) {
-                    if (!repeat & down) {
-                        apply_orientation_transform(im->screen,
+                    if (video && !repeat && down) {
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_180);
                     }
-                } else if (controller) {
+                } else if (im->kp && !paused) {
                     // forward repeated events
-                    action_volume_up(controller, action);
+                    action_volume_up(im, action);
                 }
                 return;
             case SDLK_LEFT:
-                if (!repeat && down) {
+                if (video && !repeat && down) {
                     if (shift) {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_0);
                     } else {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_270);
                     }
                 }
                 return;
             case SDLK_RIGHT:
-                if (!repeat && down) {
+                if (video && !repeat && down) {
                     if (shift) {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_FLIP_0);
                     } else {
-                        apply_orientation_transform(im->screen,
+                        apply_orientation_transform(im,
                                                     SC_ORIENTATION_90);
                     }
                 }
                 return;
             case SDLK_c:
-                if (controller && !shift && !repeat && down) {
-                    get_device_clipboard(controller, SC_COPY_KEY_COPY);
+                if (im->kp && !shift && !repeat && down && !paused) {
+                    get_device_clipboard(im, SC_COPY_KEY_COPY);
                 }
                 return;
             case SDLK_x:
-                if (controller && !shift && !repeat && down) {
-                    get_device_clipboard(controller, SC_COPY_KEY_CUT);
+                if (im->kp && !shift && !repeat && down && !paused) {
+                    get_device_clipboard(im, SC_COPY_KEY_CUT);
                 }
                 return;
             case SDLK_v:
-                if (controller && !repeat && down) {
+                if (im->kp && !repeat && down && !paused) {
                     if (shift || im->legacy_paste) {
                         // inject the text as input events
-                        clipboard_paste(controller);
+                        clipboard_paste(im);
                     } else {
                         // store the text in the device clipboard and paste,
                         // without requesting an acknowledgment
-                        set_device_clipboard(controller, true,
-                                             SC_SEQUENCE_INVALID);
+                        set_device_clipboard(im, true, SC_SEQUENCE_INVALID);
                     }
                 }
                 return;
             case SDLK_f:
-                if (!shift && !repeat && down) {
+                if (video && !shift && !repeat && down) {
                     sc_screen_switch_fullscreen(im->screen);
                 }
                 return;
             case SDLK_w:
-                if (!shift && !repeat && down) {
+                if (video && !shift && !repeat && down) {
                     sc_screen_resize_to_fit(im->screen);
                 }
                 return;
             case SDLK_g:
-                if (!shift && !repeat && down) {
+                if (video && !shift && !repeat && down) {
                     sc_screen_resize_to_pixel_perfect(im->screen);
                 }
                 return;
             case SDLK_i:
-                if (!shift && !repeat && down) {
-                    switch_fps_counter_state(&im->screen->fps_counter);
+                if (video && !shift && !repeat && down) {
+                    switch_fps_counter_state(im);
                 }
                 return;
             case SDLK_n:
-                if (controller && !repeat && down) {
+                if (control && !repeat && down && !paused) {
                     if (shift) {
-                        collapse_panels(controller);
+                        collapse_panels(im);
                     } else if (im->key_repeat == 0) {
-                        expand_notification_panel(controller);
+                        expand_notification_panel(im);
                     } else {
-                        expand_settings_panel(controller);
+                        expand_settings_panel(im);
                     }
                 }
                 return;
             case SDLK_r:
-                if (controller && !shift && !repeat && down) {
-                    rotate_device(controller);
+                if (control && !shift && !repeat && down && !paused) {
+                    rotate_device(im);
+                }
+                return;
+            case SDLK_k:
+                if (control && !shift && !repeat && down && !paused
+                        && im->kp && im->kp->hid) {
+                    // Only if the current keyboard is hid
+                    open_hard_keyboard_settings(im);
                 }
                 return;
         }
@@ -531,7 +590,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
         return;
     }
 
-    if (!controller) {
+    if (!im->kp || paused) {
         return;
     }
 
@@ -540,7 +599,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
     if (im->clipboard_autosync && is_ctrl_v) {
         if (im->legacy_paste) {
             // inject the text as input events
-            clipboard_paste(controller);
+            clipboard_paste(im);
             return;
         }
 
@@ -550,7 +609,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
 
         // Synchronize the computer clipboard to the device clipboard before
         // sending Ctrl+v, to allow seamless copy-paste.
-        bool ok = set_device_clipboard(controller, false, sequence);
+        bool ok = set_device_clipboard(im, false, sequence);
         if (!ok) {
             LOGW("Clipboard could not be synchronized, Ctrl+v not injected");
             return;
@@ -576,29 +635,39 @@ sc_input_manager_process_key(struct sc_input_manager *im,
     im->kp->ops->process_key(im->kp, &evt, ack_to_wait);
 }
 
+static struct sc_position
+sc_input_manager_get_position(struct sc_input_manager *im, int32_t x,
+                                                           int32_t y) {
+    if (im->mp->relative_mode) {
+        // No absolute position
+        return (struct sc_position) {
+            .screen_size = {0, 0},
+            .point = {0, 0},
+        };
+    }
+
+    return (struct sc_position) {
+        .screen_size = im->screen->frame_size,
+        .point = sc_screen_convert_window_to_frame_coords(im->screen, x, y),
+    };
+}
+
 static void
 sc_input_manager_process_mouse_motion(struct sc_input_manager *im,
                                       const SDL_MouseMotionEvent *event) {
-
     if (event->which == SDL_TOUCH_MOUSEID) {
         // simulated from touch events, so it's a duplicate
         return;
     }
 
     struct sc_mouse_motion_event evt = {
-        .position = {
-            .screen_size = im->screen->frame_size,
-            .point = sc_screen_convert_window_to_frame_coords(im->screen,
-                                                              event->x,
-                                                              event->y),
-        },
-        .pointer_id = im->forward_all_clicks ? POINTER_ID_MOUSE
-                                             : POINTER_ID_GENERIC_FINGER,
+        .position = sc_input_manager_get_position(im, event->x, event->y),
+        .pointer_id = im->has_secondary_click ? POINTER_ID_MOUSE
+                                              : POINTER_ID_GENERIC_FINGER,
         .xrel = event->xrel,
         .yrel = event->yrel,
         .buttons_state =
-            sc_mouse_buttons_state_from_sdl(event->state,
-                                            im->forward_all_clicks),
+            sc_mouse_buttons_state_from_sdl(event->state, &im->mouse_bindings),
     };
 
     assert(im->mp->ops->process_mouse_motion);
@@ -649,81 +718,109 @@ sc_input_manager_process_touch(struct sc_input_manager *im,
     im->mp->ops->process_touch(im->mp, &evt);
 }
 
+static enum sc_mouse_binding
+sc_input_manager_get_binding(const struct sc_mouse_bindings *bindings,
+                             uint8_t sdl_button) {
+    switch (sdl_button) {
+        case SDL_BUTTON_LEFT:
+            return SC_MOUSE_BINDING_CLICK;
+        case SDL_BUTTON_RIGHT:
+            return bindings->right_click;
+        case SDL_BUTTON_MIDDLE:
+            return bindings->middle_click;
+        case SDL_BUTTON_X1:
+            return bindings->click4;
+        case SDL_BUTTON_X2:
+            return bindings->click5;
+        default:
+            return SC_MOUSE_BINDING_DISABLED;
+    }
+}
+
 static void
 sc_input_manager_process_mouse_button(struct sc_input_manager *im,
                                       const SDL_MouseButtonEvent *event) {
-    struct sc_controller *controller = im->controller;
-
     if (event->which == SDL_TOUCH_MOUSEID) {
         // simulated from touch events, so it's a duplicate
         return;
     }
 
+    bool control = im->controller;
+    bool paused = im->screen->paused;
     bool down = event->type == SDL_MOUSEBUTTONDOWN;
-    if (!im->forward_all_clicks) {
-        if (controller) {
-            enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
+    if (control && !paused) {
+        enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
 
-            if (event->button == SDL_BUTTON_X1) {
-                action_app_switch(controller, action);
+        enum sc_mouse_binding binding =
+            sc_input_manager_get_binding(&im->mouse_bindings, event->button);
+        assert(binding != SC_MOUSE_BINDING_AUTO);
+        switch (binding) {
+            case SC_MOUSE_BINDING_DISABLED:
+                // ignore click
                 return;
-            }
-            if (event->button == SDL_BUTTON_X2 && down) {
-                if (event->clicks < 2) {
-                    expand_notification_panel(controller);
-                } else {
-                    expand_settings_panel(controller);
+            case SC_MOUSE_BINDING_BACK:
+                if (im->kp) {
+                    press_back_or_turn_screen_on(im, action);
                 }
                 return;
-            }
-            if (event->button == SDL_BUTTON_RIGHT) {
-                press_back_or_turn_screen_on(controller, action);
+            case SC_MOUSE_BINDING_HOME:
+                if (im->kp) {
+                    action_home(im, action);
+                }
                 return;
-            }
-            if (event->button == SDL_BUTTON_MIDDLE) {
-                action_home(controller, action);
+            case SC_MOUSE_BINDING_APP_SWITCH:
+                if (im->kp) {
+                    action_app_switch(im, action);
+                }
                 return;
-            }
-        }
-
-        // double-click on black borders resize to fit the device screen
-        if (event->button == SDL_BUTTON_LEFT && event->clicks == 2) {
-            int32_t x = event->x;
-            int32_t y = event->y;
-            sc_screen_hidpi_scale_coords(im->screen, &x, &y);
-            SDL_Rect *r = &im->screen->rect;
-            bool outside = x < r->x || x >= r->x + r->w
-                        || y < r->y || y >= r->y + r->h;
-            if (outside) {
+            case SC_MOUSE_BINDING_EXPAND_NOTIFICATION_PANEL:
                 if (down) {
-                    sc_screen_resize_to_fit(im->screen);
+                    if (event->clicks < 2) {
+                        expand_notification_panel(im);
+                    } else {
+                        expand_settings_panel(im);
+                    }
                 }
                 return;
-            }
+            default:
+                assert(binding == SC_MOUSE_BINDING_CLICK);
+                break;
         }
-        // otherwise, send the click event to the device
     }
 
-    if (!controller) {
+    // double-click on black borders resizes to fit the device screen
+    bool video = im->screen->video;
+    bool mouse_relative_mode = im->mp && im->mp->relative_mode;
+    if (video && !mouse_relative_mode && event->button == SDL_BUTTON_LEFT
+            && event->clicks == 2) {
+        int32_t x = event->x;
+        int32_t y = event->y;
+        sc_screen_hidpi_scale_coords(im->screen, &x, &y);
+        SDL_Rect *r = &im->screen->rect;
+        bool outside = x < r->x || x >= r->x + r->w
+                    || y < r->y || y >= r->y + r->h;
+        if (outside) {
+            if (down) {
+                sc_screen_resize_to_fit(im->screen);
+            }
+            return;
+        }
+    }
+
+    if (!im->mp || paused) {
         return;
     }
 
     uint32_t sdl_buttons_state = SDL_GetMouseState(NULL, NULL);
 
     struct sc_mouse_click_event evt = {
-        .position = {
-            .screen_size = im->screen->frame_size,
-            .point = sc_screen_convert_window_to_frame_coords(im->screen,
-                                                              event->x,
-                                                              event->y),
-        },
+        .position = sc_input_manager_get_position(im, event->x, event->y),
         .action = sc_action_from_sdl_mousebutton_type(event->type),
         .button = sc_mouse_button_from_sdl(event->button),
-        .pointer_id = im->forward_all_clicks ? POINTER_ID_MOUSE
-                                             : POINTER_ID_GENERIC_FINGER,
-        .buttons_state =
-            sc_mouse_buttons_state_from_sdl(sdl_buttons_state,
-                                            im->forward_all_clicks),
+        .pointer_id = im->has_secondary_click ? POINTER_ID_MOUSE
+                                              : POINTER_ID_GENERIC_FINGER,
+        .buttons_state = sc_mouse_buttons_state_from_sdl(sdl_buttons_state,
+                                                         &im->mouse_bindings),
     };
 
     assert(im->mp->ops->process_mouse_click);
@@ -791,11 +888,7 @@ sc_input_manager_process_mouse_wheel(struct sc_input_manager *im,
     uint32_t buttons = SDL_GetMouseState(&mouse_x, &mouse_y);
 
     struct sc_mouse_scroll_event evt = {
-        .position = {
-            .screen_size = im->screen->frame_size,
-            .point = sc_screen_convert_window_to_frame_coords(im->screen,
-                                                              mouse_x, mouse_y),
-        },
+        .position = sc_input_manager_get_position(im, mouse_x, mouse_y),
 #if SDL_VERSION_ATLEAST(2, 0, 18)
         .hscroll = CLAMP(event->preciseX, -1.0f, 1.0f),
         .vscroll = CLAMP(event->preciseY, -1.0f, 1.0f),
@@ -803,8 +896,8 @@ sc_input_manager_process_mouse_wheel(struct sc_input_manager *im,
         .hscroll = CLAMP(event->x, -1, 1),
         .vscroll = CLAMP(event->y, -1, 1),
 #endif
-        .buttons_state =
-            sc_mouse_buttons_state_from_sdl(buttons, im->forward_all_clicks),
+        .buttons_state = sc_mouse_buttons_state_from_sdl(buttons,
+                                                         &im->mouse_bindings),
     };
 
     im->mp->ops->process_mouse_scroll(im->mp, &evt);
@@ -842,9 +935,10 @@ void
 sc_input_manager_handle_event(struct sc_input_manager *im,
                               const SDL_Event *event) {
     bool control = im->controller;
+    bool paused = im->screen->paused;
     switch (event->type) {
         case SDL_TEXTINPUT:
-            if (!control) {
+            if (!im->kp || paused) {
                 break;
             }
             sc_input_manager_process_text_input(im, &event->text);
@@ -856,13 +950,13 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
             sc_input_manager_process_key(im, &event->key);
             break;
         case SDL_MOUSEMOTION:
-            if (!control) {
+            if (!im->mp || paused) {
                 break;
             }
             sc_input_manager_process_mouse_motion(im, &event->motion);
             break;
         case SDL_MOUSEWHEEL:
-            if (!control) {
+            if (!im->mp || paused) {
                 break;
             }
             sc_input_manager_process_mouse_wheel(im, &event->wheel);
@@ -876,7 +970,7 @@ sc_input_manager_handle_event(struct sc_input_manager *im,
         case SDL_FINGERMOTION:
         case SDL_FINGERDOWN:
         case SDL_FINGERUP:
-            if (!control) {
+            if (!im->mp || paused) {
                 break;
             }
             sc_input_manager_process_touch(im, &event->tfinger);

app/src/input_manager.h

@@ -22,14 +22,12 @@ struct sc_input_manager {
     struct sc_key_processor *kp;
     struct sc_mouse_processor *mp;
 
-    bool forward_all_clicks;
+    struct sc_mouse_bindings mouse_bindings;
+    bool has_secondary_click;
     bool legacy_paste;
     bool clipboard_autosync;
 
-    struct {
-        unsigned data[SC_MAX_SHORTCUT_MODS];
-        unsigned count;
-    } sdl_shortcut_mods;
+    uint16_t sdl_shortcut_mods;
 
     bool vfinger_down;
     bool vfinger_invert_x;
@@ -52,10 +50,10 @@ struct sc_input_manager_params {
     struct sc_key_processor *kp;
     struct sc_mouse_processor *mp;
 
-    bool forward_all_clicks;
+    struct sc_mouse_bindings mouse_bindings;
     bool legacy_paste;
     bool clipboard_autosync;
-    const struct sc_shortcut_mods *shortcut_mods;
+    uint8_t shortcut_mods; // OR of enum sc_shortcut_mod values
 };
 
 void

app/src/keyboard_sdk.c

@@ -1,4 +1,4 @@
-#include "keyboard_inject.h"
+#include "keyboard_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast key processor to sc_keyboard_inject */
-#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_inject, key_processor)
+/** Downcast key processor to sc_keyboard_sdk */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_sdk, key_processor)
 
 static enum android_keyevent_action
 convert_keycode_action(enum sc_action action) {
@@ -271,20 +271,20 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
     // is set before injecting Ctrl+v.
     (void) ack_to_wait;
 
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
     if (event->repeat) {
-        if (!ki->forward_key_repeat) {
+        if (!kb->forward_key_repeat) {
             return;
         }
-        ++ki->repeat;
+        ++kb->repeat;
     } else {
-        ki->repeat = 0;
+        kb->repeat = 0;
     }
 
     struct sc_control_msg msg;
-    if (convert_input_key(event, &msg, ki->key_inject_mode, ki->repeat)) {
-        if (!sc_controller_push_msg(ki->controller, &msg)) {
+    if (convert_input_key(event, &msg, kb->key_inject_mode, kb->repeat)) {
+        if (!sc_controller_push_msg(kb->controller, &msg)) {
             LOGW("Could not request 'inject keycode'");
         }
     }
@@ -293,14 +293,14 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
 static void
 sc_key_processor_process_text(struct sc_key_processor *kp,
                               const struct sc_text_event *event) {
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
         // Never inject text events
         return;
     }
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
         char c = event->text[0];
         if (isalpha(c) || c == ' ') {
             assert(event->text[1] == '\0');
@@ -316,22 +316,22 @@ sc_key_processor_process_text(struct sc_key_processor *kp,
         LOGW("Could not strdup input text");
         return;
     }
-    if (!sc_controller_push_msg(ki->controller, &msg)) {
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
         free(msg.inject_text.text);
         LOGW("Could not request 'inject text'");
     }
 }
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
-                        struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat) {
-    ki->controller = controller;
-    ki->key_inject_mode = key_inject_mode;
-    ki->forward_key_repeat = forward_key_repeat;
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
+                     struct sc_controller *controller,
+                     enum sc_key_inject_mode key_inject_mode,
+                     bool forward_key_repeat) {
+    kb->controller = controller;
+    kb->key_inject_mode = key_inject_mode;
+    kb->forward_key_repeat = forward_key_repeat;
 
-    ki->repeat = 0;
+    kb->repeat = 0;
 
     static const struct sc_key_processor_ops ops = {
         .process_key = sc_key_processor_process_key,
@@ -339,6 +339,7 @@ sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
     };
 
     // Key injection and clipboard synchronization are serialized
-    ki->key_processor.async_paste = false;
-    ki->key_processor.ops = &ops;
+    kb->key_processor.async_paste = false;
+    kb->key_processor.hid = false;
+    kb->key_processor.ops = &ops;
 }

app/src/keyboard_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_KEYBOARD_INJECT_H
-#define SC_KEYBOARD_INJECT_H
+#ifndef SC_KEYBOARD_SDK_H
+#define SC_KEYBOARD_SDK_H
 
 #include "common.h"
 
@@ -9,7 +9,7 @@
 #include "options.h"
 #include "trait/key_processor.h"
 
-struct sc_keyboard_inject {
+struct sc_keyboard_sdk {
     struct sc_key_processor key_processor; // key processor trait
 
     struct sc_controller *controller;
@@ -23,9 +23,9 @@ struct sc_keyboard_inject {
 };
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
-                        struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat);
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
+                     struct sc_controller *controller,
+                     enum sc_key_inject_mode key_inject_mode,
+                     bool forward_key_repeat);
 
 #endif

app/src/mouse_sdk.c

@@ -1,4 +1,4 @@
-#include "mouse_inject.h"
+#include "mouse_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast mouse processor to sc_mouse_inject */
-#define DOWNCAST(MP) container_of(MP, struct sc_mouse_inject, mouse_processor)
+/** Downcast mouse processor to sc_mouse_sdk */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_sdk, mouse_processor)
 
 static enum android_motionevent_buttons
 convert_mouse_buttons(uint32_t state) {
@@ -58,17 +58,18 @@ convert_touch_action(enum sc_touch_action action) {
 static void
 sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
                                     const struct sc_mouse_motion_event *event) {
-    if (!event->buttons_state) {
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
+
+    if (!m->mouse_hover && !event->buttons_state) {
         // Do not send motion events when no click is pressed
         return;
     }
 
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
-
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
         .inject_touch_event = {
-            .action = AMOTION_EVENT_ACTION_MOVE,
+            .action = event->buttons_state ? AMOTION_EVENT_ACTION_MOVE
+                                           : AMOTION_EVENT_ACTION_HOVER_MOVE,
             .pointer_id = event->pointer_id,
             .position = event->position,
             .pressure = 1.f,
@@ -76,7 +77,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse motion event'");
     }
 }
@@ -84,7 +85,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
                                     const struct sc_mouse_click_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -98,7 +99,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse click event'");
     }
 }
@@ -106,7 +107,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
                                    const struct sc_mouse_scroll_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT,
@@ -118,7 +119,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse scroll event'");
     }
 }
@@ -126,7 +127,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
                                  const struct sc_touch_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -139,15 +140,16 @@ sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject touch event'");
     }
 }
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
-                     struct sc_controller *controller) {
-    mi->controller = controller;
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller,
+                  bool mouse_hover) {
+    m->controller = controller;
+    m->mouse_hover = mouse_hover;
 
     static const struct sc_mouse_processor_ops ops = {
         .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
@@ -156,7 +158,7 @@ sc_mouse_inject_init(struct sc_mouse_inject *mi,
         .process_touch = sc_mouse_processor_process_touch,
     };
 
-    mi->mouse_processor.ops = &ops;
+    m->mouse_processor.ops = &ops;
 
-    mi->mouse_processor.relative_mode = false;
+    m->mouse_processor.relative_mode = false;
 }

app/src/mouse_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_MOUSE_INJECT_H
-#define SC_MOUSE_INJECT_H
+#ifndef SC_MOUSE_SDK_H
+#define SC_MOUSE_SDK_H
 
 #include "common.h"
 
@@ -9,14 +9,15 @@
 #include "screen.h"
 #include "trait/mouse_processor.h"
 
-struct sc_mouse_inject {
+struct sc_mouse_sdk {
     struct sc_mouse_processor mouse_processor; // mouse processor trait
 
     struct sc_controller *controller;
+    bool mouse_hover;
 };
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
-                     struct sc_controller *controller);
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller,
+                  bool mouse_hover);
 
 #endif

app/src/options.c

@@ -21,8 +21,14 @@ const struct scrcpy_options scrcpy_options_default = {
     .video_source = SC_VIDEO_SOURCE_DISPLAY,
     .audio_source = SC_AUDIO_SOURCE_AUTO,
     .record_format = SC_RECORD_FORMAT_AUTO,
-    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT,
-    .mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT,
+    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_AUTO,
+    .mouse_input_mode = SC_MOUSE_INPUT_MODE_AUTO,
+    .mouse_bindings = {
+        .right_click = SC_MOUSE_BINDING_AUTO,
+        .middle_click = SC_MOUSE_BINDING_AUTO,
+        .click4 = SC_MOUSE_BINDING_AUTO,
+        .click5 = SC_MOUSE_BINDING_AUTO,
+    },
     .camera_facing = SC_CAMERA_FACING_ANY,
     .port_range = {
         .first = DEFAULT_LOCAL_PORT_RANGE_FIRST,
@@ -30,10 +36,7 @@ const struct scrcpy_options scrcpy_options_default = {
     },
     .tunnel_host = 0,
     .tunnel_port = 0,
-    .shortcut_mods = {
-        .data = {SC_SHORTCUT_MOD_LALT, SC_SHORTCUT_MOD_LSUPER},
-        .count = 2,
-    },
+    .shortcut_mods = SC_SHORTCUT_MOD_LALT | SC_SHORTCUT_MOD_LSUPER,
     .max_size = 0,
     .video_bit_rate = 0,
     .audio_bit_rate = 0,
@@ -71,7 +74,6 @@ const struct scrcpy_options scrcpy_options_default = {
     .force_adb_forward = false,
     .disable_screensaver = false,
     .forward_key_repeat = true,
-    .forward_all_clicks = false,
     .legacy_paste = false,
     .power_off_on_close = false,
     .clipboard_autosync = true,
@@ -89,6 +91,8 @@ const struct scrcpy_options scrcpy_options_default = {
     .kill_adb_on_close = false,
     .camera_high_speed = false,
     .list = 0,
+    .window = true,
+    .mouse_hover = true,
 };
 
 enum sc_orientation

app/src/options.h

@@ -140,13 +140,36 @@ enum sc_lock_video_orientation {
 };
 
 enum sc_keyboard_input_mode {
-    SC_KEYBOARD_INPUT_MODE_INJECT,
-    SC_KEYBOARD_INPUT_MODE_HID,
+    SC_KEYBOARD_INPUT_MODE_AUTO,
+    SC_KEYBOARD_INPUT_MODE_DISABLED,
+    SC_KEYBOARD_INPUT_MODE_SDK,
+    SC_KEYBOARD_INPUT_MODE_UHID,
+    SC_KEYBOARD_INPUT_MODE_AOA,
 };
 
 enum sc_mouse_input_mode {
-    SC_MOUSE_INPUT_MODE_INJECT,
-    SC_MOUSE_INPUT_MODE_HID,
+    SC_MOUSE_INPUT_MODE_AUTO,
+    SC_MOUSE_INPUT_MODE_DISABLED,
+    SC_MOUSE_INPUT_MODE_SDK,
+    SC_MOUSE_INPUT_MODE_UHID,
+    SC_MOUSE_INPUT_MODE_AOA,
+};
+
+enum sc_mouse_binding {
+    SC_MOUSE_BINDING_AUTO,
+    SC_MOUSE_BINDING_DISABLED,
+    SC_MOUSE_BINDING_CLICK,
+    SC_MOUSE_BINDING_BACK,
+    SC_MOUSE_BINDING_HOME,
+    SC_MOUSE_BINDING_APP_SWITCH,
+    SC_MOUSE_BINDING_EXPAND_NOTIFICATION_PANEL,
+};
+
+struct sc_mouse_bindings {
+    enum sc_mouse_binding right_click;
+    enum sc_mouse_binding middle_click;
+    enum sc_mouse_binding click4;
+    enum sc_mouse_binding click5;
 };
 
 enum sc_key_inject_mode {
@@ -163,8 +186,6 @@ enum sc_key_inject_mode {
     SC_KEY_INJECT_MODE_RAW,
 };
 
-#define SC_MAX_SHORTCUT_MODS 8
-
 enum sc_shortcut_mod {
     SC_SHORTCUT_MOD_LCTRL = 1 << 0,
     SC_SHORTCUT_MOD_RCTRL = 1 << 1,
@@ -174,11 +195,6 @@ enum sc_shortcut_mod {
     SC_SHORTCUT_MOD_RSUPER = 1 << 5,
 };
 
-struct sc_shortcut_mods {
-    unsigned data[SC_MAX_SHORTCUT_MODS];
-    unsigned count;
-};
-
 struct sc_port_range {
     uint16_t first;
     uint16_t last;
@@ -209,11 +225,12 @@ struct scrcpy_options {
     enum sc_record_format record_format;
     enum sc_keyboard_input_mode keyboard_input_mode;
     enum sc_mouse_input_mode mouse_input_mode;
+    struct sc_mouse_bindings mouse_bindings;
     enum sc_camera_facing camera_facing;
     struct sc_port_range port_range;
     uint32_t tunnel_host;
     uint16_t tunnel_port;
-    struct sc_shortcut_mods shortcut_mods;
+    uint8_t shortcut_mods; // OR of enum sc_shortcut_mod values
     uint16_t max_size;
     uint32_t video_bit_rate;
     uint32_t audio_bit_rate;
@@ -251,7 +268,6 @@ struct scrcpy_options {
     bool force_adb_forward;
     bool disable_screensaver;
     bool forward_key_repeat;
-    bool forward_all_clicks;
     bool legacy_paste;
     bool power_off_on_close;
     bool clipboard_autosync;
@@ -273,6 +289,8 @@ struct scrcpy_options {
 #define SC_OPTION_LIST_CAMERAS 0x4
 #define SC_OPTION_LIST_CAMERA_SIZES 0x8
     uint8_t list;
+    bool window;
+    bool mouse_hover;
 };
 
 extern const struct scrcpy_options scrcpy_options_default;

app/src/receiver.c

@@ -1,22 +1,29 @@
 #include "receiver.h"
 
 #include <assert.h>
+#include <inttypes.h>
 #include <stdint.h>
 #include <SDL2/SDL_clipboard.h>
 
 #include "device_msg.h"
 #include "util/log.h"
+#include "util/str.h"
 
 bool
 sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
-              struct sc_acksync *acksync) {
+                 const struct sc_receiver_callbacks *cbs, void *cbs_userdata) {
     bool ok = sc_mutex_init(&receiver->mutex);
     if (!ok) {
         return false;
     }
 
     receiver->control_socket = control_socket;
-    receiver->acksync = acksync;
+    receiver->acksync = NULL;
+    receiver->uhid_devices = NULL;
+
+    assert(cbs && cbs->on_error);
+    receiver->cbs = cbs;
+    receiver->cbs_userdata = cbs_userdata;
 
     return true;
 }
@@ -58,6 +65,41 @@ process_msg(struct sc_receiver *receiver, struct sc_device_msg *msg) {
 
             sc_acksync_ack(receiver->acksync, msg->ack_clipboard.sequence);
             break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
+                char *hex = sc_str_to_hex_string(msg->uhid_output.data,
+                                                 msg->uhid_output.size);
+                if (hex) {
+                    LOGV("UHID output [%" PRIu16 "] %s",
+                         msg->uhid_output.id, hex);
+                    free(hex);
+                } else {
+                    LOGV("UHID output [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_output.id, msg->uhid_output.size);
+                }
+            }
+
+            // This is a programming error to receive this message if there is
+            // no uhid_devices instance
+            assert(receiver->uhid_devices);
+
+            // Also check at runtime (do not trust the server)
+            if (!receiver->uhid_devices) {
+                LOGE("Received unexpected HID output message");
+                return;
+            }
+
+            struct sc_uhid_receiver *uhid_receiver =
+                sc_uhid_devices_get_receiver(receiver->uhid_devices,
+                                             msg->uhid_output.id);
+            if (uhid_receiver) {
+                uhid_receiver->ops->process_output(uhid_receiver,
+                                                   msg->uhid_output.data,
+                                                   msg->uhid_output.size);
+            } else {
+                LOGW("No UHID receiver for id %" PRIu16, msg->uhid_output.id);
+            }
+            break;
     }
 }
 
@@ -115,6 +157,8 @@ run_receiver(void *data) {
         }
     }
 
+    receiver->cbs->on_error(receiver, receiver->cbs_userdata);
+
     return 0;
 }
 

app/src/receiver.h

@@ -5,6 +5,7 @@
 
 #include <stdbool.h>
 
+#include "uhid/uhid_output.h"
 #include "util/acksync.h"
 #include "util/net.h"
 #include "util/thread.h"
@@ -17,11 +18,19 @@ struct sc_receiver {
     sc_mutex mutex;
 
     struct sc_acksync *acksync;
+    struct sc_uhid_devices *uhid_devices;
+
+    const struct sc_receiver_callbacks *cbs;
+    void *cbs_userdata;
+};
+
+struct sc_receiver_callbacks {
+    void (*on_error)(struct sc_receiver *receiver, void *userdata);
 };
 
 bool
 sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
-                 struct sc_acksync *acksync);
+                 const struct sc_receiver_callbacks *cbs, void *cbs_userdata);
 
 void
 sc_receiver_destroy(struct sc_receiver *receiver);

app/src/scrcpy.c

@@ -20,15 +20,17 @@
 #include "demuxer.h"
 #include "events.h"
 #include "file_pusher.h"
-#include "keyboard_inject.h"
-#include "mouse_inject.h"
+#include "keyboard_sdk.h"
+#include "mouse_sdk.h"
 #include "recorder.h"
 #include "screen.h"
 #include "server.h"
+#include "uhid/keyboard_uhid.h"
+#include "uhid/mouse_uhid.h"
 #ifdef HAVE_USB
 # include "usb/aoa_hid.h"
-# include "usb/hid_keyboard.h"
-# include "usb/hid_mouse.h"
+# include "usb/keyboard_aoa.h"
+# include "usb/mouse_aoa.h"
 # include "usb/usb.h"
 #endif
 #include "util/acksync.h"
@@ -61,17 +63,20 @@ struct scrcpy {
     struct sc_aoa aoa;
     // sequence/ack helper to synchronize clipboard and Ctrl+v via HID
     struct sc_acksync acksync;
+    struct sc_uhid_devices uhid_devices;
 #endif
     union {
-        struct sc_keyboard_inject keyboard_inject;
+        struct sc_keyboard_sdk keyboard_sdk;
+        struct sc_keyboard_uhid keyboard_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_keyboard keyboard_hid;
+        struct sc_keyboard_aoa keyboard_aoa;
 #endif
     };
     union {
-        struct sc_mouse_inject mouse_inject;
+        struct sc_mouse_sdk mouse_sdk;
+        struct sc_mouse_uhid mouse_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_mouse mouse_hid;
+        struct sc_mouse_aoa mouse_aoa;
 #endif
     };
     struct sc_timeout timeout;
@@ -101,7 +106,6 @@ static BOOL WINAPI windows_ctrl_handler(DWORD ctrl_type) {
 
 static void
 sdl_set_hints(const char *render_driver) {
-
     if (render_driver && !SDL_SetHint(SDL_HINT_RENDER_DRIVER, render_driver)) {
         LOGW("Could not set render driver");
     }
@@ -170,6 +174,9 @@ event_loop(struct scrcpy *s) {
             case SC_EVENT_DEMUXER_ERROR:
                 LOGE("Demuxer error");
                 return SCRCPY_EXIT_FAILURE;
+            case SC_EVENT_CONTROLLER_ERROR:
+                LOGE("Controller error");
+                return SCRCPY_EXIT_FAILURE;
             case SC_EVENT_RECORDER_ERROR:
                 LOGE("Recorder error");
                 return SCRCPY_EXIT_FAILURE;
@@ -261,6 +268,16 @@ sc_audio_demuxer_on_ended(struct sc_demuxer *demuxer,
     }
 }
 
+static void
+sc_controller_on_error(struct sc_controller *controller, void *userdata) {
+    // Note: this function may be called twice, once from the controller thread
+    // and once from the receiver thread
+    (void) controller;
+    (void) userdata;
+
+    PUSH_EVENT(SC_EVENT_CONTROLLER_ERROR);
+}
+
 static void
 sc_server_on_connection_failed(struct sc_server *server, void *userdata) {
     (void) server;
@@ -307,6 +324,10 @@ scrcpy_generate_scid(void) {
 enum scrcpy_exit_code
 scrcpy(struct scrcpy_options *options) {
     static struct scrcpy scrcpy;
+#ifndef NDEBUG
+    // Detect missing initializations
+    memset(&scrcpy, 42, sizeof(scrcpy));
+#endif
     struct scrcpy *s = &scrcpy;
 
     // Minimal SDL initialization
@@ -330,8 +351,8 @@ scrcpy(struct scrcpy_options *options) {
     bool audio_demuxer_started = false;
 #ifdef HAVE_USB
     bool aoa_hid_initialized = false;
-    bool hid_keyboard_initialized = false;
-    bool hid_mouse_initialized = false;
+    bool keyboard_aoa_initialized = false;
+    bool mouse_aoa_initialized = false;
 #endif
     bool controller_initialized = false;
     bool controller_started = false;
@@ -340,6 +361,7 @@ scrcpy(struct scrcpy_options *options) {
     bool timeout_started = false;
 
     struct sc_acksync *acksync = NULL;
+    struct sc_uhid_devices *uhid_devices = NULL;
 
     uint32_t scid = scrcpy_generate_scid();
 
@@ -399,6 +421,12 @@ scrcpy(struct scrcpy_options *options) {
         return SCRCPY_EXIT_FAILURE;
     }
 
+    if (options->window) {
+        // Set hints before starting the server thread to avoid race conditions
+        // in SDL
+        sdl_set_hints(options->render_driver);
+    }
+
     if (!sc_server_start(&s->server)) {
         goto end;
     }
@@ -415,11 +443,7 @@ scrcpy(struct scrcpy_options *options) {
     assert(!options->video_playback || options->video);
     assert(!options->audio_playback || options->audio);
 
-    if (options->video_playback) {
-        sdl_set_hints(options->render_driver);
-    }
-
-    if (options->video_playback ||
+    if (options->window ||
             (options->control && options->clipboard_autosync)) {
         // Initialize the video subsystem even if --no-video or
         // --no-video-playback is passed so that clipboard synchronization
@@ -542,12 +566,24 @@ scrcpy(struct scrcpy_options *options) {
     struct sc_mouse_processor *mp = NULL;
 
     if (options->control) {
+        static const struct sc_controller_callbacks controller_cbs = {
+            .on_error = sc_controller_on_error,
+        };
+
+        if (!sc_controller_init(&s->controller, s->server.control_socket,
+            &controller_cbs, NULL)) {
+            goto end;
+        }
+        controller_initialized = true;
+
+        controller = &s->controller;
+
 #ifdef HAVE_USB
-        bool use_hid_keyboard =
-            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
-        bool use_hid_mouse =
-            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
-        if (use_hid_keyboard || use_hid_mouse) {
+        bool use_keyboard_aoa =
+            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
+        bool use_mouse_aoa =
+            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
+        if (use_keyboard_aoa || use_mouse_aoa) {
             bool ok = sc_acksync_init(&s->acksync);
             if (!ok) {
                 goto end;
@@ -557,7 +593,7 @@ scrcpy(struct scrcpy_options *options) {
             if (!ok) {
                 LOGE("Failed to initialize USB");
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             assert(serial);
@@ -565,7 +601,7 @@ scrcpy(struct scrcpy_options *options) {
             ok = sc_usb_select_device(&s->usb, serial, &usb_device);
             if (!ok) {
                 sc_usb_destroy(&s->usb);
-                goto aoa_hid_end;
+                goto end;
             }
 
             LOGI("USB device: %s (%04" PRIx16 ":%04" PRIx16 ") %s %s",
@@ -578,7 +614,7 @@ scrcpy(struct scrcpy_options *options) {
                 LOGE("Failed to connect to USB device %s", serial);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             ok = sc_aoa_init(&s->aoa, &s->usb, &s->acksync);
@@ -587,113 +623,100 @@ scrcpy(struct scrcpy_options *options) {
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
-            if (use_hid_keyboard) {
-                if (sc_hid_keyboard_init(&s->keyboard_hid, &s->aoa)) {
-                    hid_keyboard_initialized = true;
-                    kp = &s->keyboard_hid.key_processor;
+            if (use_keyboard_aoa) {
+                if (sc_keyboard_aoa_init(&s->keyboard_aoa, &s->aoa)) {
+                    keyboard_aoa_initialized = true;
+                    kp = &s->keyboard_aoa.key_processor;
                 } else {
                     LOGE("Could not initialize HID keyboard");
                 }
             }
 
-            if (use_hid_mouse) {
-                if (sc_hid_mouse_init(&s->mouse_hid, &s->aoa)) {
-                    hid_mouse_initialized = true;
-                    mp = &s->mouse_hid.mouse_processor;
+            if (use_mouse_aoa) {
+                if (sc_mouse_aoa_init(&s->mouse_aoa, &s->aoa)) {
+                    mouse_aoa_initialized = true;
+                    mp = &s->mouse_aoa.mouse_processor;
                 } else {
                     LOGE("Could not initialized HID mouse");
                 }
             }
 
-            bool need_aoa = hid_keyboard_initialized || hid_mouse_initialized;
+            bool need_aoa = keyboard_aoa_initialized || mouse_aoa_initialized;
 
             if (!need_aoa || !sc_aoa_start(&s->aoa)) {
                 sc_acksync_destroy(&s->acksync);
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_aoa_destroy(&s->aoa);
-                goto aoa_hid_end;
+                goto end;
             }
 
             acksync = &s->acksync;
 
             aoa_hid_initialized = true;
-
-aoa_hid_end:
-            if (!aoa_hid_initialized) {
-                if (hid_keyboard_initialized) {
-                    sc_hid_keyboard_destroy(&s->keyboard_hid);
-                    hid_keyboard_initialized = false;
-                }
-                if (hid_mouse_initialized) {
-                    sc_hid_mouse_destroy(&s->mouse_hid);
-                    hid_mouse_initialized = false;
-                }
-            }
-
-            if (use_hid_keyboard && !hid_keyboard_initialized) {
-                LOGE("Fallback to default keyboard injection method "
-                     "(-K/--hid-keyboard ignored)");
-                options->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT;
-            }
-
-            if (use_hid_mouse && !hid_mouse_initialized) {
-                LOGE("Fallback to default mouse injection method "
-                     "(-M/--hid-mouse ignored)");
-                options->mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT;
-            }
         }
 #else
-        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_HID);
-        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_HID);
+        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_AOA);
+        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_AOA);
 #endif
 
-        // keyboard_input_mode may have been reset if HID mode failed
-        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_INJECT) {
-            sc_keyboard_inject_init(&s->keyboard_inject, &s->controller,
-                                    options->key_inject_mode,
-                                    options->forward_key_repeat);
-            kp = &s->keyboard_inject.key_processor;
+        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_SDK) {
+            sc_keyboard_sdk_init(&s->keyboard_sdk, &s->controller,
+                                 options->key_inject_mode,
+                                 options->forward_key_repeat);
+            kp = &s->keyboard_sdk.key_processor;
+        } else if (options->keyboard_input_mode
+                == SC_KEYBOARD_INPUT_MODE_UHID) {
+            sc_uhid_devices_init(&s->uhid_devices);
+            bool ok = sc_keyboard_uhid_init(&s->keyboard_uhid, &s->controller,
+                                            &s->uhid_devices);
+            if (!ok) {
+                goto end;
+            }
+            uhid_devices = &s->uhid_devices;
+            kp = &s->keyboard_uhid.key_processor;
         }
 
-        // mouse_input_mode may have been reset if HID mode failed
-        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_INJECT) {
-            sc_mouse_inject_init(&s->mouse_inject, &s->controller);
-            mp = &s->mouse_inject.mouse_processor;
+        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_SDK) {
+            sc_mouse_sdk_init(&s->mouse_sdk, &s->controller,
+                              options->mouse_hover);
+            mp = &s->mouse_sdk.mouse_processor;
+        } else if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_UHID) {
+            bool ok = sc_mouse_uhid_init(&s->mouse_uhid, &s->controller);
+            if (!ok) {
+                goto end;
+            }
+            mp = &s->mouse_uhid.mouse_processor;
         }
 
-        if (!sc_controller_init(&s->controller, s->server.control_socket,
-                                acksync)) {
-            goto end;
-        }
-        controller_initialized = true;
+        sc_controller_configure(&s->controller, acksync, uhid_devices);
 
         if (!sc_controller_start(&s->controller)) {
             goto end;
         }
         controller_started = true;
-        controller = &s->controller;
     }
 
     // There is a controller if and only if control is enabled
     assert(options->control == !!controller);
 
-    if (options->video_playback) {
+    if (options->window) {
         const char *window_title =
             options->window_title ? options->window_title : info->device_name;
 
         struct sc_screen_params screen_params = {
+            .video = options->video_playback,
             .controller = controller,
             .fp = fp,
             .kp = kp,
             .mp = mp,
-            .forward_all_clicks = options->forward_all_clicks,
+            .mouse_bindings = options->mouse_bindings,
             .legacy_paste = options->legacy_paste,
             .clipboard_autosync = options->clipboard_autosync,
-            .shortcut_mods = &options->shortcut_mods,
+            .shortcut_mods = options->shortcut_mods,
             .window_title = window_title,
             .always_on_top = options->always_on_top,
             .window_x = options->window_x,
@@ -707,12 +730,15 @@ aoa_hid_end:
             .start_fps_counter = options->start_fps_counter,
         };
 
-        struct sc_frame_source *src = &s->video_decoder.frame_source;
-        if (options->display_buffer) {
-            sc_delay_buffer_init(&s->display_buffer, options->display_buffer,
-                                 true);
-            sc_frame_source_add_sink(src, &s->display_buffer.frame_sink);
-            src = &s->display_buffer.frame_source;
+        struct sc_frame_source *src;
+        if (options->video_playback) {
+            src = &s->video_decoder.frame_source;
+            if (options->display_buffer) {
+                sc_delay_buffer_init(&s->display_buffer,
+                                     options->display_buffer, true);
+                sc_frame_source_add_sink(src, &s->display_buffer.frame_sink);
+                src = &s->display_buffer.frame_source;
+            }
         }
 
         if (!sc_screen_init(&s->screen, &screen_params)) {
@@ -720,7 +746,9 @@ aoa_hid_end:
         }
         screen_initialized = true;
 
-        sc_frame_source_add_sink(src, &s->screen.frame_sink);
+        if (options->video_playback) {
+            sc_frame_source_add_sink(src, &s->screen.frame_sink);
+        }
     }
 
     if (options->audio_playback) {
@@ -802,9 +830,12 @@ aoa_hid_end:
     ret = event_loop(s);
     LOGD("quit...");
 
-    // Close the window immediately on closing, because screen_destroy() may
-    // only be called once the video demuxer thread is joined (it may take time)
-    sc_screen_hide_window(&s->screen);
+    if (options->video_playback) {
+        // Close the window immediately on closing, because screen_destroy()
+        // may only be called once the video demuxer thread is joined (it may
+        // take time)
+        sc_screen_hide_window(&s->screen);
+    }
 
 end:
     if (timeout_started) {
@@ -815,11 +846,11 @@ end:
     // end-of-stream
 #ifdef HAVE_USB
     if (aoa_hid_initialized) {
-        if (hid_keyboard_initialized) {
-            sc_hid_keyboard_destroy(&s->keyboard_hid);
+        if (keyboard_aoa_initialized) {
+            sc_keyboard_aoa_destroy(&s->keyboard_aoa);
         }
-        if (hid_mouse_initialized) {
-            sc_hid_mouse_destroy(&s->mouse_hid);
+        if (mouse_aoa_initialized) {
+            sc_mouse_aoa_destroy(&s->mouse_aoa);
         }
         sc_aoa_stop(&s->aoa);
         sc_usb_stop(&s->usb);

app/src/screen.c

@@ -205,6 +205,8 @@ sc_screen_toggle_mouse_capture(struct sc_screen *screen) {
 
 static void
 sc_screen_update_content_rect(struct sc_screen *screen) {
+    assert(screen->video);
+
     int dw;
     int dh;
     SDL_GL_GetDrawableSize(screen->window, &dw, &dh);
@@ -246,6 +248,8 @@ sc_screen_update_content_rect(struct sc_screen *screen) {
 // changed, so that the content rectangle is recomputed
 static void
 sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
+    assert(screen->video);
+
     if (update_content_rect) {
         sc_screen_update_content_rect(screen);
     }
@@ -255,6 +259,13 @@ sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
     (void) res; // any error already logged
 }
 
+static void
+sc_screen_render_novideo(struct sc_screen *screen) {
+    enum sc_display_result res =
+        sc_display_render(&screen->display, NULL, SC_ORIENTATION_0);
+    (void) res; // any error already logged
+}
+
 #if defined(__APPLE__) || defined(__WINDOWS__)
 # define CONTINUOUS_RESIZING_WORKAROUND
 #endif
@@ -268,6 +279,8 @@ sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
 static int
 event_watcher(void *data, SDL_Event *event) {
     struct sc_screen *screen = data;
+    assert(screen->video);
+
     if (event->type == SDL_WINDOWEVENT
             && event->window.event == SDL_WINDOWEVENT_RESIZED) {
         // In practice, it seems to always be called from the same thread in
@@ -326,6 +339,7 @@ sc_screen_frame_sink_close(struct sc_frame_sink *sink) {
 static bool
 sc_screen_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
     struct sc_screen *screen = DOWNCAST(sink);
+    assert(screen->video);
 
     bool previous_skipped;
     bool ok = sc_frame_buffer_push(&screen->fb, frame, &previous_skipped);
@@ -362,6 +376,11 @@ sc_screen_init(struct sc_screen *screen,
     screen->maximized = false;
     screen->minimized = false;
     screen->mouse_capture_key_pressed = 0;
+    screen->paused = false;
+    screen->resume_frame = NULL;
+    screen->orientation = SC_ORIENTATION_0;
+
+    screen->video = params->video;
 
     screen->req.x = params->window_x;
     screen->req.y = params->window_y;
@@ -379,41 +398,75 @@ sc_screen_init(struct sc_screen *screen,
         goto error_destroy_frame_buffer;
     }
 
-    screen->orientation = params->orientation;
-    if (screen->orientation != SC_ORIENTATION_0) {
-        LOGI("Initial display orientation set to %s",
-             sc_orientation_get_name(screen->orientation));
+    if (screen->video) {
+        screen->orientation = params->orientation;
+        if (screen->orientation != SC_ORIENTATION_0) {
+            LOGI("Initial display orientation set to %s",
+                 sc_orientation_get_name(screen->orientation));
+        }
     }
 
-    uint32_t window_flags = SDL_WINDOW_HIDDEN
-                          | SDL_WINDOW_RESIZABLE
-                          | SDL_WINDOW_ALLOW_HIGHDPI;
+    uint32_t window_flags = SDL_WINDOW_ALLOW_HIGHDPI;
     if (params->always_on_top) {
         window_flags |= SDL_WINDOW_ALWAYS_ON_TOP;
     }
     if (params->window_borderless) {
         window_flags |= SDL_WINDOW_BORDERLESS;
     }
+    if (params->video) {
+        // The window will be shown on first frame
+        window_flags |= SDL_WINDOW_HIDDEN
+                      | SDL_WINDOW_RESIZABLE;
+    }
+
+    const char *title = params->window_title;
+    assert(title);
+
+    int x = SDL_WINDOWPOS_UNDEFINED;
+    int y = SDL_WINDOWPOS_UNDEFINED;
+    int width = 256;
+    int height = 256;
+    if (params->window_x != SC_WINDOW_POSITION_UNDEFINED) {
+        x = params->window_x;
+    }
+    if (params->window_y != SC_WINDOW_POSITION_UNDEFINED) {
+        y = params->window_y;
+    }
+    if (params->window_width) {
+        width = params->window_width;
+    }
+    if (params->window_height) {
+        height = params->window_height;
+    }
 
     // The window will be positioned and sized on first video frame
-    screen->window =
-        SDL_CreateWindow(params->window_title, 0, 0, 0, 0, window_flags);
+    screen->window = SDL_CreateWindow(title, x, y, width, height, window_flags);
     if (!screen->window) {
         LOGE("Could not create window: %s", SDL_GetError());
         goto error_destroy_fps_counter;
     }
 
-    ok = sc_display_init(&screen->display, screen->window, params->mipmaps);
-    if (!ok) {
-        goto error_destroy_window;
-    }
-
     SDL_Surface *icon = scrcpy_icon_load();
     if (icon) {
         SDL_SetWindowIcon(screen->window, icon);
-        scrcpy_icon_destroy(icon);
-    } else {
+    } else if (params->video) {
+        // just a warning
         LOGW("Could not load icon");
+    } else {
+        // without video, the icon is used as window content, it must be present
+        LOGE("Could not load icon");
+        goto error_destroy_fps_counter;
+    }
+
+    SDL_Surface *icon_novideo = params->video ? NULL : icon;
+    bool mipmaps = params->video && params->mipmaps;
+    ok = sc_display_init(&screen->display, screen->window, icon_novideo,
+                         mipmaps);
+    if (icon) {
+        scrcpy_icon_destroy(icon);
+    }
+    if (!ok) {
+        goto error_destroy_window;
     }
 
     screen->frame = av_frame_alloc();
@@ -428,7 +481,7 @@ sc_screen_init(struct sc_screen *screen,
         .screen = screen,
         .kp = params->kp,
         .mp = params->mp,
-        .forward_all_clicks = params->forward_all_clicks,
+        .mouse_bindings = params->mouse_bindings,
         .legacy_paste = params->legacy_paste,
         .clipboard_autosync = params->clipboard_autosync,
         .shortcut_mods = params->shortcut_mods,
@@ -437,7 +490,9 @@ sc_screen_init(struct sc_screen *screen,
     sc_input_manager_init(&screen->im, &im_params);
 
 #ifdef CONTINUOUS_RESIZING_WORKAROUND
-    SDL_AddEventWatch(event_watcher, screen);
+    if (screen->video) {
+        SDL_AddEventWatch(event_watcher, screen);
+    }
 #endif
 
     static const struct sc_frame_sink_ops ops = {
@@ -452,6 +507,11 @@ sc_screen_init(struct sc_screen *screen,
     screen->open = false;
 #endif
 
+    if (!screen->video && sc_screen_is_relative_mode(screen)) {
+        // Capture mouse immediately if video mirroring is disabled
+        sc_screen_set_mouse_capture(screen, true);
+    }
+
     return true;
 
 error_destroy_display:
@@ -522,6 +582,8 @@ sc_screen_destroy(struct sc_screen *screen) {
 static void
 resize_for_content(struct sc_screen *screen, struct sc_size old_content_size,
                    struct sc_size new_content_size) {
+    assert(screen->video);
+
     struct sc_size window_size = get_window_size(screen);
     struct sc_size target_size = {
         .width = (uint32_t) window_size.width * new_content_size.width
@@ -535,6 +597,8 @@ resize_for_content(struct sc_screen *screen, struct sc_size old_content_size,
 
 static void
 set_content_size(struct sc_screen *screen, struct sc_size new_content_size) {
+    assert(screen->video);
+
     if (!screen->fullscreen && !screen->maximized && !screen->minimized) {
         resize_for_content(screen, screen->content_size, new_content_size);
     } else if (!screen->resize_pending) {
@@ -549,6 +613,8 @@ set_content_size(struct sc_screen *screen, struct sc_size new_content_size) {
 
 static void
 apply_pending_resize(struct sc_screen *screen) {
+    assert(screen->video);
+
     assert(!screen->fullscreen);
     assert(!screen->maximized);
     assert(!screen->minimized);
@@ -562,6 +628,8 @@ apply_pending_resize(struct sc_screen *screen) {
 void
 sc_screen_set_orientation(struct sc_screen *screen,
                           enum sc_orientation orientation) {
+    assert(screen->video);
+
     if (orientation == screen->orientation) {
         return;
     }
@@ -596,6 +664,8 @@ sc_screen_init_size(struct sc_screen *screen) {
 // recreate the texture and resize the window if the frame size has changed
 static enum sc_display_result
 prepare_for_frame(struct sc_screen *screen, struct sc_size new_frame_size) {
+    assert(screen->video);
+
     if (screen->frame_size.width == new_frame_size.width
             && screen->frame_size.height == new_frame_size.height) {
         return SC_DISPLAY_RESULT_OK;
@@ -614,13 +684,12 @@ prepare_for_frame(struct sc_screen *screen, struct sc_size new_frame_size) {
 }
 
 static bool
-sc_screen_update_frame(struct sc_screen *screen) {
-    av_frame_unref(screen->frame);
-    sc_frame_buffer_consume(&screen->fb, screen->frame);
-    AVFrame *frame = screen->frame;
+sc_screen_apply_frame(struct sc_screen *screen) {
+    assert(screen->video);
 
     sc_fps_counter_add_rendered_frame(&screen->fps_counter);
 
+    AVFrame *frame = screen->frame;
     struct sc_size new_frame_size = {frame->width, frame->height};
     enum sc_display_result res = prepare_for_frame(screen, new_frame_size);
     if (res == SC_DISPLAY_RESULT_ERROR) {
@@ -655,8 +724,62 @@ sc_screen_update_frame(struct sc_screen *screen) {
     return true;
 }
 
+static bool
+sc_screen_update_frame(struct sc_screen *screen) {
+    assert(screen->video);
+
+    if (screen->paused) {
+        if (!screen->resume_frame) {
+            screen->resume_frame = av_frame_alloc();
+            if (!screen->resume_frame) {
+                LOG_OOM();
+                return false;
+            }
+        } else {
+            av_frame_unref(screen->resume_frame);
+        }
+        sc_frame_buffer_consume(&screen->fb, screen->resume_frame);
+        return true;
+    }
+
+    av_frame_unref(screen->frame);
+    sc_frame_buffer_consume(&screen->fb, screen->frame);
+    return sc_screen_apply_frame(screen);
+}
+
+void
+sc_screen_set_paused(struct sc_screen *screen, bool paused) {
+    assert(screen->video);
+
+    if (!paused && !screen->paused) {
+        // nothing to do
+        return;
+    }
+
+    if (screen->paused && screen->resume_frame) {
+        // If display screen was paused, refresh the frame immediately, even if
+        // the new state is also paused.
+        av_frame_free(&screen->frame);
+        screen->frame = screen->resume_frame;
+        screen->resume_frame = NULL;
+        sc_screen_apply_frame(screen);
+    }
+
+    if (!paused) {
+        LOGI("Display screen unpaused");
+    } else if (!screen->paused) {
+        LOGI("Display screen paused");
+    } else {
+        LOGI("Display screen re-paused");
+    }
+
+    screen->paused = paused;
+}
+
 void
 sc_screen_switch_fullscreen(struct sc_screen *screen) {
+    assert(screen->video);
+
     uint32_t new_mode = screen->fullscreen ? 0 : SDL_WINDOW_FULLSCREEN_DESKTOP;
     if (SDL_SetWindowFullscreen(screen->window, new_mode)) {
         LOGW("Could not switch fullscreen mode: %s", SDL_GetError());
@@ -674,6 +797,8 @@ sc_screen_switch_fullscreen(struct sc_screen *screen) {
 
 void
 sc_screen_resize_to_fit(struct sc_screen *screen) {
+    assert(screen->video);
+
     if (screen->fullscreen || screen->maximized || screen->minimized) {
         return;
     }
@@ -698,6 +823,8 @@ sc_screen_resize_to_fit(struct sc_screen *screen) {
 
 void
 sc_screen_resize_to_pixel_perfect(struct sc_screen *screen) {
+    assert(screen->video);
+
     if (screen->fullscreen || screen->minimized) {
         return;
     }
@@ -741,6 +868,13 @@ sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event) {
             return true;
         }
         case SDL_WINDOWEVENT:
+            if (!screen->video
+                    && event->window.event == SDL_WINDOWEVENT_EXPOSED) {
+                sc_screen_render_novideo(screen);
+            }
+
+            // !video implies !has_frame
+            assert(screen->video || !screen->has_frame);
             if (!screen->has_frame) {
                 // Do nothing
                 return true;
@@ -844,6 +978,8 @@ sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event) {
 struct sc_point
 sc_screen_convert_drawable_to_frame_coords(struct sc_screen *screen,
                                            int32_t x, int32_t y) {
+    assert(screen->video);
+
     enum sc_orientation orientation = screen->orientation;
 
     int32_t w = screen->content_size.width;

app/src/screen.h

@@ -26,6 +26,8 @@ struct sc_screen {
     bool open; // track the open/close state to assert correct behavior
 #endif
 
+    bool video;
+
     struct sc_display display;
     struct sc_input_manager im;
     struct sc_frame_buffer fb;
@@ -64,18 +66,23 @@ struct sc_screen {
     SDL_Keycode mouse_capture_key_pressed;
 
     AVFrame *frame;
+
+    bool paused;
+    AVFrame *resume_frame;
 };
 
 struct sc_screen_params {
+    bool video;
+
     struct sc_controller *controller;
     struct sc_file_pusher *fp;
     struct sc_key_processor *kp;
     struct sc_mouse_processor *mp;
 
-    bool forward_all_clicks;
+    struct sc_mouse_bindings mouse_bindings;
     bool legacy_paste;
     bool clipboard_autosync;
-    const struct sc_shortcut_mods *shortcut_mods;
+    uint8_t shortcut_mods; // OR of enum sc_shortcut_mod values
 
     const char *window_title;
     bool always_on_top;
@@ -135,6 +142,10 @@ void
 sc_screen_set_orientation(struct sc_screen *screen,
                           enum sc_orientation orientation);
 
+// set the display pause state
+void
+sc_screen_set_paused(struct sc_screen *screen, bool paused);
+
 // react to SDL events
 // If this function returns false, scrcpy must exit with an error.
 bool

app/src/sys/win/process.c

@@ -176,6 +176,8 @@ sc_process_execute_p(const char *const argv[], HANDLE *handle, unsigned flags,
         free(lpAttributeList);
     }
 
+    CloseHandle(pi.hThread);
+
     // These handles are used by the child process, close them for this process
     if (pin) {
         CloseHandle(stdin_read_handle);

app/src/trait/key_processor.h

@@ -23,6 +23,13 @@ struct sc_key_processor {
      */
     bool async_paste;
 
+    /**
+     * Set by the implementation to indicate that the keyboard is HID. In
+     * practice, it is used to react on a shortcut to open the hard keyboard
+     * settings only if the keyboard is HID.
+     */
+    bool hid;
+
     const struct sc_key_processor_ops *ops;
 };
 

app/src/uhid/keyboard_uhid.c

@@ -0,0 +1,162 @@
+#include "keyboard_uhid.h"
+
+#include "util/log.h"
+
+/** Downcast key processor to keyboard_uhid */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_uhid, key_processor)
+
+/** Downcast uhid_receiver to keyboard_uhid */
+#define DOWNCAST_RECEIVER(UR) \
+    container_of(UR, struct sc_keyboard_uhid, uhid_receiver)
+
+#define UHID_KEYBOARD_ID 1
+
+static void
+sc_keyboard_uhid_send_input(struct sc_keyboard_uhid *kb,
+                            const struct sc_hid_event *event) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = UHID_KEYBOARD_ID;
+
+    assert(event->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, event->data, event->size);
+    msg.uhid_input.size = event->size;
+
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
+        LOGE("Could not send UHID_INPUT message (key)");
+    }
+}
+
+static void
+sc_keyboard_uhid_synchronize_mod(struct sc_keyboard_uhid *kb) {
+    SDL_Keymod sdl_mod = SDL_GetModState();
+    uint16_t mod = sc_mods_state_from_sdl(sdl_mod) & (SC_MOD_CAPS | SC_MOD_NUM);
+
+    uint16_t device_mod =
+        atomic_load_explicit(&kb->device_mod, memory_order_relaxed);
+    uint16_t diff = mod ^ device_mod;
+
+    if (diff) {
+        // Inherently racy (the HID output reports arrive asynchronously in
+        // response to key presses), but will re-synchronize on next key press
+        // or HID output anyway
+        atomic_store_explicit(&kb->device_mod, mod, memory_order_relaxed);
+
+        struct sc_hid_event hid_event;
+        sc_hid_keyboard_event_from_mods(&hid_event, diff);
+
+        LOGV("HID keyboard state synchronized");
+
+        sc_keyboard_uhid_send_input(kb, &hid_event);
+    }
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const struct sc_key_event *event,
+                             uint64_t ack_to_wait) {
+    (void) ack_to_wait;
+
+    if (event->repeat) {
+        // In USB HID protocol, key repeat is handled by the host (Android), so
+        // just ignore key repeat here.
+        return;
+    }
+
+    struct sc_keyboard_uhid *kb = DOWNCAST(kp);
+
+    struct sc_hid_event hid_event;
+
+    // Not all keys are supported, just ignore unsupported keys
+    if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
+        if (event->scancode == SC_SCANCODE_CAPSLOCK) {
+            atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_CAPS,
+                                      memory_order_relaxed);
+        } else if (event->scancode == SC_SCANCODE_NUMLOCK) {
+            atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_NUM,
+                                      memory_order_relaxed);
+        } else {
+            // Synchronize modifiers (only if the scancode itself does not
+            // change the modifiers)
+            sc_keyboard_uhid_synchronize_mod(kb);
+        }
+        sc_keyboard_uhid_send_input(kb, &hid_event);
+    }
+}
+
+static unsigned
+sc_keyboard_uhid_to_sc_mod(uint8_t hid_led) {
+    // <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
+    // (chapter 11: LED page)
+    unsigned mod = 0;
+    if (hid_led & 0x01) {
+        mod |= SC_MOD_NUM;
+    }
+    if (hid_led & 0x02) {
+        mod |= SC_MOD_CAPS;
+    }
+    return mod;
+}
+
+static void
+sc_uhid_receiver_process_output(struct sc_uhid_receiver *receiver,
+                                const uint8_t *data, size_t len) {
+    // Called from the thread receiving device messages
+
+    assert(len);
+
+    // Also check at runtime (do not trust the server)
+    if (!len) {
+        LOGE("Unexpected empty HID output message");
+        return;
+    }
+
+    struct sc_keyboard_uhid *kb = DOWNCAST_RECEIVER(receiver);
+
+    uint8_t hid_led = data[0];
+    uint16_t device_mod = sc_keyboard_uhid_to_sc_mod(hid_led);
+    atomic_store_explicit(&kb->device_mod, device_mod, memory_order_relaxed);
+}
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller,
+                      struct sc_uhid_devices *uhid_devices) {
+    sc_hid_keyboard_init(&kb->hid);
+
+    kb->controller = controller;
+    atomic_init(&kb->device_mod, 0);
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        // Never forward text input via HID (all the keys are injected
+        // separately)
+        .process_text = NULL,
+    };
+
+    // Clipboard synchronization is requested over the same control socket, so
+    // there is no need for a specific synchronization mechanism
+    kb->key_processor.async_paste = false;
+    kb->key_processor.hid = true;
+    kb->key_processor.ops = &ops;
+
+    static const struct sc_uhid_receiver_ops uhid_receiver_ops = {
+        .process_output = sc_uhid_receiver_process_output,
+    };
+
+    kb->uhid_receiver.id = UHID_KEYBOARD_ID;
+    kb->uhid_receiver.ops = &uhid_receiver_ops;
+    sc_uhid_devices_add_receiver(uhid_devices, &kb->uhid_receiver);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = UHID_KEYBOARD_ID;
+    msg.uhid_create.report_desc = SC_HID_KEYBOARD_REPORT_DESC;
+    msg.uhid_create.report_desc_size = SC_HID_KEYBOARD_REPORT_DESC_LEN;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not send UHID_CREATE message (keyboard)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/keyboard_uhid.h

@@ -0,0 +1,27 @@
+#ifndef SC_KEYBOARD_UHID_H
+#define SC_KEYBOARD_UHID_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "hid/hid_keyboard.h"
+#include "uhid/uhid_output.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_uhid {
+    struct sc_key_processor key_processor; // key processor trait
+    struct sc_uhid_receiver uhid_receiver;
+
+    struct sc_hid_keyboard hid;
+    struct sc_controller *controller;
+    atomic_uint_least16_t device_mod;
+};
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller,
+                      struct sc_uhid_devices *uhid_devices);
+
+#endif

app/src/uhid/mouse_uhid.c

@@ -0,0 +1,89 @@
+#include "mouse_uhid.h"
+
+#include "hid/hid_mouse.h"
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to mouse_uhid */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_uhid, mouse_processor)
+
+#define UHID_MOUSE_ID 2
+
+static void
+sc_mouse_uhid_send_input(struct sc_mouse_uhid *mouse,
+                         const struct sc_hid_event *event, const char *name) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = UHID_MOUSE_ID;
+
+    assert(event->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, event->data, event->size);
+    msg.uhid_input.size = event->size;
+
+    if (!sc_controller_push_msg(mouse->controller, &msg)) {
+        LOGE("Could not send UHID_INPUT message (%s)", name);
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_motion_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_motion(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse motion");
+}
+
+static void
+sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
+                                   const struct sc_mouse_click_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_click(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse click");
+}
+
+static void
+sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_scroll_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_scroll(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse scroll");
+}
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller) {
+    mouse->controller = controller;
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_mouse_click = sc_mouse_processor_process_mouse_click,
+        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
+        // Touch events not supported (coordinates are not relative)
+        .process_touch = NULL,
+    };
+
+    mouse->mouse_processor.ops = &ops;
+
+    mouse->mouse_processor.relative_mode = true;
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = UHID_MOUSE_ID;
+    msg.uhid_create.report_desc = SC_HID_MOUSE_REPORT_DESC;
+    msg.uhid_create.report_desc_size = SC_HID_MOUSE_REPORT_DESC_LEN;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not send UHID_CREATE message (mouse)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/mouse_uhid.h

@@ -0,0 +1,19 @@
+#ifndef SC_MOUSE_UHID_H
+#define SC_MOUSE_UHID_H
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "trait/mouse_processor.h"
+
+struct sc_mouse_uhid {
+    struct sc_mouse_processor mouse_processor; // mouse processor trait
+
+    struct sc_controller *controller;
+};
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller);
+
+#endif

app/src/uhid/uhid_output.c

@@ -0,0 +1,25 @@
+#include "uhid_output.h"
+
+#include <assert.h>
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices) {
+    devices->count = 0;
+}
+
+void
+sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
+                             struct sc_uhid_receiver *receiver) {
+    assert(devices->count < SC_UHID_MAX_RECEIVERS);
+    devices->receivers[devices->count++] = receiver;
+}
+
+struct sc_uhid_receiver *
+sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id) {
+    for (size_t i = 0; i < devices->count; ++i) {
+        if (devices->receivers[i]->id == id) {
+            return devices->receivers[i];
+        }
+    }
+    return NULL;
+}

app/src/uhid/uhid_output.h

@@ -0,0 +1,45 @@
+#ifndef SC_UHID_OUTPUT_H
+#define SC_UHID_OUTPUT_H
+
+#include "common.h"
+
+#include <stdbool.h>
+#include <stdint.h>
+
+/**
+ * The communication with UHID devices is bidirectional.
+ *
+ * This component manages the registration of receivers to handle UHID output
+ * messages (sent from the device to the computer).
+ */
+
+struct sc_uhid_receiver {
+    uint16_t id;
+
+    const struct sc_uhid_receiver_ops *ops;
+};
+
+struct sc_uhid_receiver_ops {
+    void
+    (*process_output)(struct sc_uhid_receiver *receiver,
+                      const uint8_t *data, size_t len);
+};
+
+#define SC_UHID_MAX_RECEIVERS 1
+
+struct sc_uhid_devices {
+    struct sc_uhid_receiver *receivers[SC_UHID_MAX_RECEIVERS];
+    unsigned count;
+};
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices);
+
+void
+sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
+                             struct sc_uhid_receiver *receiver);
+
+struct sc_uhid_receiver *
+sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id);
+
+#endif

app/src/usb/aoa_hid.c

@@ -5,6 +5,7 @@
 
 #include "aoa_hid.h"
 #include "util/log.h"
+#include "util/str.h"
 
 // See <https://source.android.com/devices/accessories/aoa2#hid-support>.
 #define ACCESSORY_REGISTER_HID 54
@@ -14,37 +15,18 @@
 
 #define DEFAULT_TIMEOUT 1000
 
-#define SC_HID_EVENT_QUEUE_MAX 64
+#define SC_AOA_EVENT_QUEUE_MAX 64
 
 static void
-sc_hid_event_log(const struct sc_hid_event *event) {
+sc_hid_event_log(uint16_t accessory_id, const struct sc_hid_event *event) {
     // HID Event: [00] FF FF FF FF...
     assert(event->size);
-    unsigned buffer_size = event->size * 3 + 1;
-    char *buffer = malloc(buffer_size);
-    if (!buffer) {
-        LOG_OOM();
+    char *hex = sc_str_to_hex_string(event->data, event->size);
+    if (!hex) {
         return;
     }
-    for (unsigned i = 0; i < event->size; ++i) {
-        snprintf(buffer + i * 3, 4, " %02x", event->buffer[i]);
-    }
-    LOGV("HID Event: [%d]%s", event->accessory_id, buffer);
-    free(buffer);
-}
-
-void
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
-                  unsigned char *buffer, uint16_t buffer_size) {
-    hid_event->accessory_id = accessory_id;
-    hid_event->buffer = buffer;
-    hid_event->size = buffer_size;
-    hid_event->ack_to_wait = SC_SEQUENCE_INVALID;
-}
-
-void
-sc_hid_event_destroy(struct sc_hid_event *hid_event) {
-    free(hid_event->buffer);
+    LOGV("HID Event: [%d] %s", accessory_id, hex);
+    free(hex);
 }
 
 bool
@@ -52,7 +34,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
             struct sc_acksync *acksync) {
     sc_vecdeque_init(&aoa->queue);
 
-    if (!sc_vecdeque_reserve(&aoa->queue, SC_HID_EVENT_QUEUE_MAX)) {
+    if (!sc_vecdeque_reserve(&aoa->queue, SC_AOA_EVENT_QUEUE_MAX)) {
         return false;
     }
 
@@ -76,12 +58,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
 
 void
 sc_aoa_destroy(struct sc_aoa *aoa) {
-    // Destroy remaining events
-    while (!sc_vecdeque_is_empty(&aoa->queue)) {
-        struct sc_hid_event *event = sc_vecdeque_popref(&aoa->queue);
-        assert(event);
-        sc_hid_event_destroy(event);
-    }
+    sc_vecdeque_destroy(&aoa->queue);
 
     sc_cond_destroy(&aoa->event_cond);
     sc_mutex_destroy(&aoa->mutex);
@@ -97,10 +74,10 @@ sc_aoa_register_hid(struct sc_aoa *aoa, uint16_t accessory_id,
     // index (arg1): total length of the HID report descriptor
     uint16_t value = accessory_id;
     uint16_t index = report_desc_size;
-    unsigned char *buffer = NULL;
+    unsigned char *data = NULL;
     uint16_t length = 0;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("REGISTER_HID: libusb error: %s", libusb_strerror(result));
@@ -130,14 +107,14 @@ sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
      * See <https://libusb.sourceforge.io/api-1.0/libusb_packetoverflow.html>
      */
     // value (arg0): accessory assigned ID for the HID device
-    // index (arg1): offset of data (buffer) in descriptor
+    // index (arg1): offset of data in descriptor
     uint16_t value = accessory_id;
     uint16_t index = 0;
     // libusb_control_transfer expects a pointer to non-const
-    unsigned char *buffer = (unsigned char *) report_desc;
+    unsigned char *data = (unsigned char *) report_desc;
     uint16_t length = report_desc_size;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("SET_HID_REPORT_DESC: libusb error: %s", libusb_strerror(result));
@@ -169,18 +146,19 @@ sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
 }
 
 static bool
-sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
+sc_aoa_send_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
+                      const struct sc_hid_event *event) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_SEND_HID_EVENT;
     // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
     // value (arg0): accessory assigned ID for the HID device
     // index (arg1): 0 (unused)
-    uint16_t value = event->accessory_id;
+    uint16_t value = accessory_id;
     uint16_t index = 0;
-    unsigned char *buffer = event->buffer;
+    unsigned char *data = (uint8_t *) event->data; // discard const
     uint16_t length = event->size;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("SEND_HID_EVENT: libusb error: %s", libusb_strerror(result));
@@ -192,7 +170,7 @@ sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
 }
 
 bool
-sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
+sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_UNREGISTER_HID;
     // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
@@ -200,10 +178,10 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
     // index (arg1): 0
     uint16_t value = accessory_id;
     uint16_t index = 0;
-    unsigned char *buffer = NULL;
+    unsigned char *data = NULL;
     uint16_t length = 0;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("UNREGISTER_HID: libusb error: %s", libusb_strerror(result));
@@ -215,16 +193,25 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
 }
 
 bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
+sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
+                                       uint16_t accessory_id,
+                                       const struct sc_hid_event *event,
+                                       uint64_t ack_to_wait) {
     if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
-        sc_hid_event_log(event);
+        sc_hid_event_log(accessory_id, event);
     }
 
     sc_mutex_lock(&aoa->mutex);
     bool full = sc_vecdeque_is_full(&aoa->queue);
     if (!full) {
         bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
-        sc_vecdeque_push_noresize(&aoa->queue, *event);
+
+        struct sc_aoa_event *aoa_event =
+            sc_vecdeque_push_hole_noresize(&aoa->queue);
+        aoa_event->hid = *event;
+        aoa_event->accessory_id = accessory_id;
+        aoa_event->ack_to_wait = ack_to_wait;
+
         if (was_empty) {
             sc_cond_signal(&aoa->event_cond);
         }
@@ -252,7 +239,7 @@ run_aoa_thread(void *data) {
         }
 
         assert(!sc_vecdeque_is_empty(&aoa->queue));
-        struct sc_hid_event event = sc_vecdeque_pop(&aoa->queue);
+        struct sc_aoa_event event = sc_vecdeque_pop(&aoa->queue);
         uint64_t ack_to_wait = event.ack_to_wait;
         sc_mutex_unlock(&aoa->mutex);
 
@@ -271,17 +258,14 @@ run_aoa_thread(void *data) {
 
             if (result == SC_ACKSYNC_WAIT_TIMEOUT) {
                 LOGW("Ack not received after 500ms, discarding HID event");
-                sc_hid_event_destroy(&event);
                 continue;
             } else if (result == SC_ACKSYNC_WAIT_INTR) {
                 // stopped
-                sc_hid_event_destroy(&event);
                 break;
             }
         }
 
-        bool ok = sc_aoa_send_hid_event(aoa, &event);
-        sc_hid_event_destroy(&event);
+        bool ok = sc_aoa_send_hid_event(aoa, event.accessory_id, &event.hid);
         if (!ok) {
             LOGW("Could not send HID event to USB device");
         }

app/src/usb/aoa_hid.h

@@ -6,28 +6,22 @@
 
 #include <libusb-1.0/libusb.h>
 
+#include "hid/hid_event.h"
 #include "usb.h"
 #include "util/acksync.h"
 #include "util/thread.h"
 #include "util/tick.h"
 #include "util/vecdeque.h"
 
-struct sc_hid_event {
+#define SC_HID_MAX_SIZE 8
+
+struct sc_aoa_event {
+    struct sc_hid_event hid;
     uint16_t accessory_id;
-    unsigned char *buffer;
-    uint16_t size;
     uint64_t ack_to_wait;
 };
 
-// Takes ownership of buffer
-void
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
-                  unsigned char *buffer, uint16_t buffer_size);
-
-void
-sc_hid_event_destroy(struct sc_hid_event *hid_event);
-
-struct sc_hid_event_queue SC_VECDEQUE(struct sc_hid_event);
+struct sc_aoa_event_queue SC_VECDEQUE(struct sc_aoa_event);
 
 struct sc_aoa {
     struct sc_usb *usb;
@@ -35,7 +29,7 @@ struct sc_aoa {
     sc_mutex mutex;
     sc_cond event_cond;
     bool stopped;
-    struct sc_hid_event_queue queue;
+    struct sc_aoa_event_queue queue;
 
     struct sc_acksync *acksync;
 };
@@ -63,6 +57,16 @@ bool
 sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id);
 
 bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event);
+sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
+                                       uint16_t accessory_id,
+                                       const struct sc_hid_event *event,
+                                       uint64_t ack_to_wait);
+
+static inline bool
+sc_aoa_push_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
+                      const struct sc_hid_event *event) {
+    return sc_aoa_push_hid_event_with_ack_to_wait(aoa, accessory_id, event,
+                                                  SC_SEQUENCE_INVALID);
+}
 
 #endif

app/src/usb/hid_mouse.c

@@ -1,267 +0,0 @@
-#include "hid_mouse.h"
-
-#include <assert.h>
-
-#include "input_events.h"
-#include "util/log.h"
-
-/** Downcast mouse processor to hid_mouse */
-#define DOWNCAST(MP) container_of(MP, struct sc_hid_mouse, mouse_processor)
-
-#define HID_MOUSE_ACCESSORY_ID 2
-
-// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position
-#define HID_MOUSE_EVENT_SIZE 4
-
-/**
- * Mouse descriptor from the specification:
- * <https://www.usb.org/sites/default/files/hid1_11.pdf>
- *
- * Appendix E (p71): §E.10 Report Descriptor (Mouse)
- *
- * The usage tags (like Wheel) are listed in "HID Usage Tables":
- * <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
- * §4 Generic Desktop Page (0x01) (p26)
- */
-static const unsigned char mouse_report_desc[]  = {
-    // Usage Page (Generic Desktop)
-    0x05, 0x01,
-    // Usage (Mouse)
-    0x09, 0x02,
-
-    // Collection (Application)
-    0xA1, 0x01,
-
-    // Usage (Pointer)
-    0x09, 0x01,
-
-    // Collection (Physical)
-    0xA1, 0x00,
-
-    // Usage Page (Buttons)
-    0x05, 0x09,
-
-    // Usage Minimum (1)
-    0x19, 0x01,
-     // Usage Maximum (5)
-    0x29, 0x05,
-    // Logical Minimum (0)
-    0x15, 0x00,
-    // Logical Maximum (1)
-    0x25, 0x01,
-    // Report Count (5)
-    0x95, 0x05,
-    // Report Size (1)
-    0x75, 0x01,
-    // Input (Data, Variable, Absolute): 5 buttons bits
-    0x81, 0x02,
-
-    // Report Count (1)
-    0x95, 0x01,
-    // Report Size (3)
-    0x75, 0x03,
-    // Input (Constant): 3 bits padding
-    0x81, 0x01,
-
-    // Usage Page (Generic Desktop)
-    0x05, 0x01,
-    // Usage (X)
-    0x09, 0x30,
-    // Usage (Y)
-    0x09, 0x31,
-    // Usage (Wheel)
-    0x09, 0x38,
-    // Local Minimum (-127)
-    0x15, 0x81,
-    // Local Maximum (127)
-    0x25, 0x7F,
-    // Report Size (8)
-    0x75, 0x08,
-    // Report Count (3)
-    0x95, 0x03,
-    // Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
-    0x81, 0x06,
-
-    // End Collection
-    0xC0,
-
-    // End Collection
-    0xC0,
-};
-
-/**
- * A mouse HID event is 3 bytes long:
- *
- *  - byte 0: buttons state
- *  - byte 1: relative x motion (signed byte from -127 to 127)
- *  - byte 2: relative y motion (signed byte from -127 to 127)
- *
- *                   7 6 5 4 3 2 1 0
- *                  +---------------+
- *         byte 0:  |0 0 0 . . . . .| buttons state
- *                  +---------------+
- *                         ^ ^ ^ ^ ^
- *                         | | | | `- left button
- *                         | | | `--- right button
- *                         | | `----- middle button
- *                         | `------- button 4
- *                         `--------- button 5
- *
- *                  +---------------+
- *         byte 1:  |. . . . . . . .| relative x motion
- *                  +---------------+
- *         byte 2:  |. . . . . . . .| relative y motion
- *                  +---------------+
- *         byte 3:  |. . . . . . . .| wheel motion (-1, 0 or 1)
- *                  +---------------+
- *
- * As an example, here is the report for a motion of (x=5, y=-4) with left
- * button pressed:
- *
- *                  +---------------+
- *                  |0 0 0 0 0 0 0 1| left button pressed
- *                  +---------------+
- *                  |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
- *                  +---------------+
- *                  |1 1 1 1 1 1 0 0| relative y motion (y = -4)
- *                  +---------------+
- *                  |0 0 0 0 0 0 0 0| wheel motion
- *                  +---------------+
- */
-
-static bool
-sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
-    unsigned char *buffer = calloc(1, HID_MOUSE_EVENT_SIZE);
-    if (!buffer) {
-        LOG_OOM();
-        return false;
-    }
-
-    sc_hid_event_init(hid_event, HID_MOUSE_ACCESSORY_ID, buffer,
-                      HID_MOUSE_EVENT_SIZE);
-    return true;
-}
-
-static unsigned char
-buttons_state_to_hid_buttons(uint8_t buttons_state) {
-    unsigned char c = 0;
-    if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
-        c |= 1 << 0;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
-        c |= 1 << 1;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
-        c |= 1 << 2;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_X1) {
-        c |= 1 << 3;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_X2) {
-        c |= 1 << 4;
-    }
-    return c;
-}
-
-static void
-sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
-                                    const struct sc_mouse_motion_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
-    buffer[1] = CLAMP(event->xrel, -127, 127);
-    buffer[2] = CLAMP(event->yrel, -127, 127);
-    buffer[3] = 0; // wheel coordinates only used for scrolling
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse motion)");
-    }
-}
-
-static void
-sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
-                                   const struct sc_mouse_click_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
-    buffer[1] = 0; // no x motion
-    buffer[2] = 0; // no y motion
-    buffer[3] = 0; // wheel coordinates only used for scrolling
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse click)");
-    }
-}
-
-static void
-sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
-                                    const struct sc_mouse_scroll_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = 0; // buttons state irrelevant (and unknown)
-    buffer[1] = 0; // no x motion
-    buffer[2] = 0; // no y motion
-    // In practice, vscroll is always -1, 0 or 1, but in theory other values
-    // are possible
-    buffer[3] = CLAMP(event->vscroll, -127, 127);
-    // Horizontal scrolling ignored
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse scroll)");
-    }
-}
-
-bool
-sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa) {
-    mouse->aoa = aoa;
-
-    bool ok = sc_aoa_setup_hid(aoa, HID_MOUSE_ACCESSORY_ID, mouse_report_desc,
-                               ARRAY_LEN(mouse_report_desc));
-    if (!ok) {
-        LOGW("Register HID mouse failed");
-        return false;
-    }
-
-    static const struct sc_mouse_processor_ops ops = {
-        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
-        .process_mouse_click = sc_mouse_processor_process_mouse_click,
-        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
-        // Touch events not supported (coordinates are not relative)
-        .process_touch = NULL,
-    };
-
-    mouse->mouse_processor.ops = &ops;
-
-    mouse->mouse_processor.relative_mode = true;
-
-    return true;
-}
-
-void
-sc_hid_mouse_destroy(struct sc_hid_mouse *mouse) {
-    bool ok = sc_aoa_unregister_hid(mouse->aoa, HID_MOUSE_ACCESSORY_ID);
-    if (!ok) {
-        LOGW("Could not unregister HID mouse");
-    }
-}

app/src/usb/keyboard_aoa.c

@@ -0,0 +1,110 @@
+#include "keyboard_aoa.h"
+
+#include <assert.h>
+
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast key processor to keyboard_aoa */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_aoa, key_processor)
+
+#define HID_KEYBOARD_ACCESSORY_ID 1
+
+static bool
+push_mod_lock_state(struct sc_keyboard_aoa *kb, uint16_t mods_state) {
+    struct sc_hid_event hid_event;
+    if (!sc_hid_keyboard_event_from_mods(&hid_event, mods_state)) {
+        // Nothing to do
+        return true;
+    }
+
+    if (!sc_aoa_push_hid_event(kb->aoa, HID_KEYBOARD_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mod lock state)");
+        return false;
+    }
+
+    LOGD("HID keyboard state synchronized");
+
+    return true;
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const struct sc_key_event *event,
+                             uint64_t ack_to_wait) {
+    if (event->repeat) {
+        // In USB HID protocol, key repeat is handled by the host (Android), so
+        // just ignore key repeat here.
+        return;
+    }
+
+    struct sc_keyboard_aoa *kb = DOWNCAST(kp);
+
+    struct sc_hid_event hid_event;
+
+    // Not all keys are supported, just ignore unsupported keys
+    if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
+        if (!kb->mod_lock_synchronized) {
+            // Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
+            // keyboard state
+            if (push_mod_lock_state(kb, event->mods_state)) {
+                kb->mod_lock_synchronized = true;
+            }
+        }
+
+        // If ack_to_wait is != SC_SEQUENCE_INVALID, then Ctrl+v is pressed, so
+        // clipboard synchronization has been requested. Wait until clipboard
+        // synchronization is acknowledged by the server, otherwise it could
+        // paste the old clipboard content.
+
+        if (!sc_aoa_push_hid_event_with_ack_to_wait(kb->aoa,
+                                                    HID_KEYBOARD_ACCESSORY_ID,
+                                                    &hid_event,
+                                                    ack_to_wait)) {
+            LOGW("Could not request HID event (key)");
+        }
+    }
+}
+
+bool
+sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa) {
+    kb->aoa = aoa;
+
+    bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
+                               SC_HID_KEYBOARD_REPORT_DESC,
+                               SC_HID_KEYBOARD_REPORT_DESC_LEN);
+    if (!ok) {
+        LOGW("Register HID keyboard failed");
+        return false;
+    }
+
+    sc_hid_keyboard_init(&kb->hid);
+
+    kb->mod_lock_synchronized = false;
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        // Never forward text input via HID (all the keys are injected
+        // separately)
+        .process_text = NULL,
+    };
+
+    // Clipboard synchronization is requested over the control socket, while HID
+    // events are sent over AOA, so it must wait for clipboard synchronization
+    // to be acknowledged by the device before injecting Ctrl+v.
+    kb->key_processor.async_paste = true;
+    kb->key_processor.hid = true;
+    kb->key_processor.ops = &ops;
+
+    return true;
+}
+
+void
+sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb) {
+    // Unregister HID keyboard so the soft keyboard shows again on Android
+    bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
+    if (!ok) {
+        LOGW("Could not unregister HID keyboard");
+    }
+}

app/src/usb/keyboard_aoa.h

@@ -0,0 +1,27 @@
+#ifndef SC_KEYBOARD_AOA_H
+#define SC_KEYBOARD_AOA_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "aoa_hid.h"
+#include "hid/hid_keyboard.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_aoa {
+    struct sc_key_processor key_processor; // key processor trait
+
+    struct sc_hid_keyboard hid;
+    struct sc_aoa *aoa;
+
+    bool mod_lock_synchronized;
+};
+
+bool
+sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa);
+
+void
+sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb);
+
+#endif

app/src/usb/mouse_aoa.c

@@ -0,0 +1,89 @@
+#include "mouse_aoa.h"
+
+#include <assert.h>
+
+#include "hid/hid_mouse.h"
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to mouse_aoa */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_aoa, mouse_processor)
+
+#define HID_MOUSE_ACCESSORY_ID 2
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_motion_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_motion(&hid_event, event);
+
+    if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mouse motion)");
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
+                                   const struct sc_mouse_click_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_click(&hid_event, event);
+
+    if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mouse click)");
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_scroll_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_scroll(&hid_event, event);
+
+    if (!sc_aoa_push_hid_event(mouse->aoa, HID_MOUSE_ACCESSORY_ID,
+                               &hid_event)) {
+        LOGW("Could not request HID event (mouse scroll)");
+    }
+}
+
+bool
+sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa) {
+    mouse->aoa = aoa;
+
+    bool ok = sc_aoa_setup_hid(aoa, HID_MOUSE_ACCESSORY_ID,
+                               SC_HID_MOUSE_REPORT_DESC,
+                               SC_HID_MOUSE_REPORT_DESC_LEN);
+    if (!ok) {
+        LOGW("Register HID mouse failed");
+        return false;
+    }
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_mouse_click = sc_mouse_processor_process_mouse_click,
+        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
+        // Touch events not supported (coordinates are not relative)
+        .process_touch = NULL,
+    };
+
+    mouse->mouse_processor.ops = &ops;
+
+    mouse->mouse_processor.relative_mode = true;
+
+    return true;
+}
+
+void
+sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse) {
+    bool ok = sc_aoa_unregister_hid(mouse->aoa, HID_MOUSE_ACCESSORY_ID);
+    if (!ok) {
+        LOGW("Could not unregister HID mouse");
+    }
+}

app/src/usb/mouse_aoa.h

@@ -1,5 +1,5 @@
-#ifndef SC_HID_MOUSE_H
-#define SC_HID_MOUSE_H
+#ifndef SC_MOUSE_AOA_H
+#define SC_MOUSE_AOA_H
 
 #include "common.h"
 
@@ -8,16 +8,16 @@
 #include "aoa_hid.h"
 #include "trait/mouse_processor.h"
 
-struct sc_hid_mouse {
+struct sc_mouse_aoa {
     struct sc_mouse_processor mouse_processor; // mouse processor trait
 
     struct sc_aoa *aoa;
 };
 
 bool
-sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa);
+sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa);
 
 void
-sc_hid_mouse_destroy(struct sc_hid_mouse *mouse);
+sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse);
 
 #endif

app/src/usb/scrcpy_otg.c

@@ -10,8 +10,8 @@
 struct scrcpy_otg {
     struct sc_usb usb;
     struct sc_aoa aoa;
-    struct sc_hid_keyboard keyboard;
-    struct sc_hid_mouse mouse;
+    struct sc_keyboard_aoa keyboard;
+    struct sc_mouse_aoa mouse;
 
     struct sc_screen_otg screen_otg;
 };
@@ -73,8 +73,8 @@ scrcpy_otg(struct scrcpy_options *options) {
 
     enum scrcpy_exit_code ret = SCRCPY_EXIT_FAILURE;
 
-    struct sc_hid_keyboard *keyboard = NULL;
-    struct sc_hid_mouse *mouse = NULL;
+    struct sc_keyboard_aoa *keyboard = NULL;
+    struct sc_mouse_aoa *mouse = NULL;
     bool usb_device_initialized = false;
     bool usb_connected = false;
     bool aoa_started = false;
@@ -117,19 +117,18 @@ scrcpy_otg(struct scrcpy_options *options) {
     }
     aoa_initialized = true;
 
-    bool enable_keyboard =
-        options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
-    bool enable_mouse =
-        options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
+    assert(options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA
+        || options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_DISABLED);
+    assert(options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA
+        || options->mouse_input_mode == SC_MOUSE_INPUT_MODE_DISABLED);
 
-    // If neither --hid-keyboard or --hid-mouse is passed, enable both
-    if (!enable_keyboard && !enable_mouse) {
-        enable_keyboard = true;
-        enable_mouse = true;
-    }
+    bool enable_keyboard =
+        options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
+    bool enable_mouse =
+        options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
 
     if (enable_keyboard) {
-        ok = sc_hid_keyboard_init(&s->keyboard, &s->aoa);
+        ok = sc_keyboard_aoa_init(&s->keyboard, &s->aoa);
         if (!ok) {
             goto end;
         }
@@ -137,7 +136,7 @@ scrcpy_otg(struct scrcpy_options *options) {
     }
 
     if (enable_mouse) {
-        ok = sc_hid_mouse_init(&s->mouse, &s->aoa);
+        ok = sc_mouse_aoa_init(&s->mouse, &s->aoa);
         if (!ok) {
             goto end;
         }
@@ -186,10 +185,10 @@ end:
     sc_usb_stop(&s->usb);
 
     if (mouse) {
-        sc_hid_mouse_destroy(&s->mouse);
+        sc_mouse_aoa_destroy(&s->mouse);
     }
     if (keyboard) {
-        sc_hid_keyboard_destroy(&s->keyboard);
+        sc_keyboard_aoa_destroy(&s->keyboard);
     }
 
     if (aoa_initialized) {

app/src/usb/screen_otg.c

@@ -169,7 +169,7 @@ sc_screen_otg_process_mouse_motion(struct sc_screen_otg *screen,
         // .position not used for HID events
         .xrel = event->xrel,
         .yrel = event->yrel,
-        .buttons_state = sc_mouse_buttons_state_from_sdl(event->state, true),
+        .buttons_state = sc_mouse_buttons_state_from_sdl(event->state, NULL),
     };
 
     assert(mp->ops->process_mouse_motion);
@@ -189,7 +189,7 @@ sc_screen_otg_process_mouse_button(struct sc_screen_otg *screen,
         .action = sc_action_from_sdl_mousebutton_type(event->type),
         .button = sc_mouse_button_from_sdl(event->button),
         .buttons_state =
-            sc_mouse_buttons_state_from_sdl(sdl_buttons_state, true),
+            sc_mouse_buttons_state_from_sdl(sdl_buttons_state, NULL),
     };
 
     assert(mp->ops->process_mouse_click);
@@ -209,7 +209,7 @@ sc_screen_otg_process_mouse_wheel(struct sc_screen_otg *screen,
         .hscroll = event->x,
         .vscroll = event->y,
         .buttons_state =
-            sc_mouse_buttons_state_from_sdl(sdl_buttons_state, true),
+            sc_mouse_buttons_state_from_sdl(sdl_buttons_state, NULL),
     };
 
     assert(mp->ops->process_mouse_scroll);

app/src/usb/screen_otg.h

@@ -6,12 +6,12 @@
 #include <stdbool.h>
 #include <SDL2/SDL.h>
 
-#include "hid_keyboard.h"
-#include "hid_mouse.h"
+#include "keyboard_aoa.h"
+#include "mouse_aoa.h"
 
 struct sc_screen_otg {
-    struct sc_hid_keyboard *keyboard;
-    struct sc_hid_mouse *mouse;
+    struct sc_keyboard_aoa *keyboard;
+    struct sc_mouse_aoa *mouse;
 
     SDL_Window *window;
     SDL_Renderer *renderer;
@@ -22,8 +22,8 @@ struct sc_screen_otg {
 };
 
 struct sc_screen_otg_params {
-    struct sc_hid_keyboard *keyboard;
-    struct sc_hid_mouse *mouse;
+    struct sc_keyboard_aoa *keyboard;
+    struct sc_mouse_aoa *mouse;
 
     const char *window_title;
     bool always_on_top;

app/src/util/audiobuf.c

@@ -46,6 +46,9 @@ sc_audiobuf_read(struct sc_audiobuf *buf, void *to_, uint32_t samples_count) {
     uint32_t head = atomic_load_explicit(&buf->head, memory_order_acquire);
 
     uint32_t can_read = (buf->alloc_size + head - tail) % buf->alloc_size;
+    if (!can_read) {
+        return 0;
+    }
     if (samples_count > can_read) {
         samples_count = can_read;
     }
@@ -86,6 +89,9 @@ sc_audiobuf_write(struct sc_audiobuf *buf, const void *from_,
     uint32_t tail = atomic_load_explicit(&buf->tail, memory_order_acquire);
 
     uint32_t can_write = (buf->alloc_size + tail - head - 1) % buf->alloc_size;
+    if (!can_write) {
+        return 0;
+    }
     if (samples_count > can_write) {
         samples_count = can_write;
     }

app/src/util/str.c

@@ -3,6 +3,7 @@
 #include <assert.h>
 #include <errno.h>
 #include <limits.h>
+#include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 
@@ -333,3 +334,22 @@ sc_str_remove_trailing_cr(char *s, size_t len) {
     }
     return len;
 }
+
+char *
+sc_str_to_hex_string(const uint8_t *data, size_t size) {
+    size_t buffer_size = size * 3 + 1;
+    char *buffer = malloc(buffer_size);
+    if (!buffer) {
+        LOG_OOM();
+        return NULL;
+    }
+
+    for (size_t i = 0; i < size; ++i) {
+        snprintf(buffer + i * 3, 4, "%02X ", data[i]);
+    }
+
+    // Remove the final space
+    buffer[size * 3] = '\0';
+
+    return buffer;
+}

app/src/util/str.h

@@ -138,4 +138,10 @@ sc_str_index_of_column(const char *s, unsigned col, const char *seps);
 size_t
 sc_str_remove_trailing_cr(char *s, size_t len);
 
+/**
+ * Convert binary data to hexadecimal string
+ */
+char *
+sc_str_to_hex_string(const uint8_t *data, size_t len);
+
 #endif

app/tests/test_cli.c

@@ -124,32 +124,22 @@ static void test_options2(void) {
 }
 
 static void test_parse_shortcut_mods(void) {
-    struct sc_shortcut_mods mods;
+    uint8_t mods;
     bool ok;
 
     ok = sc_parse_shortcut_mods("lctrl", &mods);
     assert(ok);
-    assert(mods.count == 1);
-    assert(mods.data[0] == SC_SHORTCUT_MOD_LCTRL);
-
-    ok = sc_parse_shortcut_mods("lctrl+lalt", &mods);
-    assert(ok);
-    assert(mods.count == 1);
-    assert(mods.data[0] == (SC_SHORTCUT_MOD_LCTRL | SC_SHORTCUT_MOD_LALT));
+    assert(mods == SC_SHORTCUT_MOD_LCTRL);
 
     ok = sc_parse_shortcut_mods("rctrl,lalt", &mods);
     assert(ok);
-    assert(mods.count == 2);
-    assert(mods.data[0] == SC_SHORTCUT_MOD_RCTRL);
-    assert(mods.data[1] == SC_SHORTCUT_MOD_LALT);
+    assert(mods == (SC_SHORTCUT_MOD_RCTRL | SC_SHORTCUT_MOD_LALT));
 
-    ok = sc_parse_shortcut_mods("lsuper,rsuper+lalt,lctrl+rctrl+ralt", &mods);
+    ok = sc_parse_shortcut_mods("lsuper,rsuper,lctrl", &mods);
     assert(ok);
-    assert(mods.count == 3);
-    assert(mods.data[0] == SC_SHORTCUT_MOD_LSUPER);
-    assert(mods.data[1] == (SC_SHORTCUT_MOD_RSUPER | SC_SHORTCUT_MOD_LALT));
-    assert(mods.data[2] == (SC_SHORTCUT_MOD_LCTRL | SC_SHORTCUT_MOD_RCTRL |
-                            SC_SHORTCUT_MOD_RALT));
+    assert(mods == (SC_SHORTCUT_MOD_LSUPER
+                  | SC_SHORTCUT_MOD_RSUPER
+                  | SC_SHORTCUT_MOD_LCTRL));
 
     ok = sc_parse_shortcut_mods("", &mods);
     assert(!ok);

app/tests/test_control_msg_serialize.c

@@ -323,6 +323,68 @@ static void test_serialize_rotate_device(void) {
     assert(!memcmp(buf, expected, sizeof(expected)));
 }
 
+static void test_serialize_uhid_create(void) {
+    const uint8_t report_desc[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11};
+    struct sc_control_msg msg = {
+        .type = SC_CONTROL_MSG_TYPE_UHID_CREATE,
+        .uhid_create = {
+            .id = 42,
+            .report_desc_size = sizeof(report_desc),
+            .report_desc = report_desc,
+        },
+    };
+
+    uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
+    size_t size = sc_control_msg_serialize(&msg, buf);
+    assert(size == 16);
+
+    const uint8_t expected[] = {
+        SC_CONTROL_MSG_TYPE_UHID_CREATE,
+        0, 42, // id
+        0, 11, // size
+        1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
+    };
+    assert(!memcmp(buf, expected, sizeof(expected)));
+}
+
+static void test_serialize_uhid_input(void) {
+    struct sc_control_msg msg = {
+        .type = SC_CONTROL_MSG_TYPE_UHID_INPUT,
+        .uhid_input = {
+            .id = 42,
+            .size = 5,
+            .data = {1, 2, 3, 4, 5},
+        },
+    };
+
+    uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
+    size_t size = sc_control_msg_serialize(&msg, buf);
+    assert(size == 10);
+
+    const uint8_t expected[] = {
+        SC_CONTROL_MSG_TYPE_UHID_INPUT,
+        0, 42, // id
+        0, 5, // size
+        1, 2, 3, 4, 5,
+    };
+    assert(!memcmp(buf, expected, sizeof(expected)));
+}
+
+static void test_serialize_open_hard_keyboard(void) {
+    struct sc_control_msg msg = {
+        .type = SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
+    };
+
+    uint8_t buf[SC_CONTROL_MSG_MAX_SIZE];
+    size_t size = sc_control_msg_serialize(&msg, buf);
+    assert(size == 1);
+
+    const uint8_t expected[] = {
+        SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
+    };
+    assert(!memcmp(buf, expected, sizeof(expected)));
+}
+
 int main(int argc, char *argv[]) {
     (void) argc;
     (void) argv;
@@ -341,5 +403,8 @@ int main(int argc, char *argv[]) {
     test_serialize_set_clipboard_long();
     test_serialize_set_screen_power_mode();
     test_serialize_rotate_device();
+    test_serialize_uhid_create();
+    test_serialize_uhid_input();
+    test_serialize_open_hard_keyboard();
     return 0;
 }

app/tests/test_device_msg_deserialize.c

@@ -61,6 +61,28 @@ static void test_deserialize_ack_set_clipboard(void) {
     assert(msg.ack_clipboard.sequence == UINT64_C(0x0102030405060708));
 }
 
+static void test_deserialize_uhid_output(void) {
+    const uint8_t input[] = {
+        DEVICE_MSG_TYPE_UHID_OUTPUT,
+        0, 42, // id
+        0, 5, // size
+        0x01, 0x02, 0x03, 0x04, 0x05, // data
+    };
+
+    struct sc_device_msg msg;
+    ssize_t r = sc_device_msg_deserialize(input, sizeof(input), &msg);
+    assert(r == 10);
+
+    assert(msg.type == DEVICE_MSG_TYPE_UHID_OUTPUT);
+    assert(msg.uhid_output.id == 42);
+    assert(msg.uhid_output.size == 5);
+
+    uint8_t expected[] = {1, 2, 3, 4, 5};
+    assert(!memcmp(msg.uhid_output.data, expected, sizeof(expected)));
+
+    sc_device_msg_destroy(&msg);
+}
+
 int main(int argc, char *argv[]) {
     (void) argc;
     (void) argv;
@@ -68,5 +90,6 @@ int main(int argc, char *argv[]) {
     test_deserialize_clipboard();
     test_deserialize_clipboard_big();
     test_deserialize_ack_set_clipboard();
+    test_deserialize_uhid_output();
     return 0;
 }

build.gradle

@@ -7,7 +7,7 @@ buildscript {
         mavenCentral()
     }
     dependencies {
-        classpath 'com.android.tools.build:gradle:8.1.3'
+        classpath 'com.android.tools.build:gradle:8.3.0'
 
         // NOTE: Do not place your application dependencies here; they belong
         // in the individual module build.gradle files

doc/audio.md

@@ -28,10 +28,17 @@ To disable only the audio playback, see [no playback](video.md#no-playback).
 
 ## Audio only
 
-To play audio only, disable the video:
+To play audio only, disable video and control:
 
 ```bash
-scrcpy --no-video
+scrcpy --no-video --no-control
+```
+
+To play audio without a window:
+
+```bash
+# --no-video and --no-control are implied by --no-window
+scrcpy --no-window
 # interrupt with Ctrl+C
 ```
 

doc/build.md

@@ -233,10 +233,10 @@ install` must be run as root)._
 
 #### Option 2: Use prebuilt server
 
- - [`scrcpy-server-v2.3.1`][direct-scrcpy-server]  
-   <sub>SHA-256: `f6814822fc308a7a532f253485c9038183c6296a6c5df470a9e383b4f8e7605b`</sub>
+ - [`scrcpy-server-v2.4`][direct-scrcpy-server]  
+   <sub>SHA-256: `93c272b7438605c055e127f7444064ed78fa9ca49f81156777fd201e79ce7ba3`</sub>
 
-[direct-scrcpy-server]: https://github.com/Genymobile/scrcpy/releases/download/v2.3.1/scrcpy-server-v2.3.1
+[direct-scrcpy-server]: https://github.com/Genymobile/scrcpy/releases/download/v2.4/scrcpy-server-v2.4
 
 Download the prebuilt server somewhere, and specify its path during the Meson
 configuration:

doc/connection.md

@@ -67,14 +67,6 @@ computer.
 An option `--tcpip` allows to configure the connection automatically. There are
 two variants.
 
-If the device (accessible at 192.168.1.1 in this example) already listens on a
-port (typically 5555) for incoming _adb_ connections, then run:
-
-```bash
-scrcpy --tcpip=192.168.1.1       # default port is 5555
-scrcpy --tcpip=192.168.1.1:5555
-```
-
 If _adb_ TCP/IP mode is disabled on the device (or if you don't know the IP
 address), connect the device over USB, then run:
 
@@ -85,6 +77,14 @@ scrcpy --tcpip   # without arguments
 It will automatically find the device IP address and adb port, enable TCP/IP
 mode if necessary, then connect to the device before starting.
 
+If the device (accessible at 192.168.1.1 in this example) already listens on a
+port (typically 5555) for incoming _adb_ connections, then run:
+
+```bash
+scrcpy --tcpip=192.168.1.1       # default port is 5555
+scrcpy --tcpip=192.168.1.1:5555
+```
+
 
 ### Manual
 

doc/control.md

@@ -10,36 +10,34 @@ scrcpy --no-control
 scrcpy -n   # short version
 ```
 
+## Keyboard and mouse
 
-## Text injection preference
+Read [keyboard](keyboard.md) and [mouse](mouse.md).
 
-Two kinds of [events][textevents] are generated when typing text:
- - _key events_, signaling that a key is pressed or released;
- - _text events_, signaling that a text has been entered.
 
-By default, letters are injected using key events, so that the keyboard behaves
-as expected in games (typically for WASD keys).
+## Control only
 
-But this may [cause issues][prefertext]. If you encounter such a problem, you
-can avoid it by:
+To control the device without mirroring:
 
 ```bash
-scrcpy --prefer-text
+scrcpy --no-video --no-audio
 ```
 
-(but this will break keyboard behavior in games)
+By default, mouse mode is switched to UHID if video mirroring is disabled (a
+relative mouse mode is required).
 
-On the contrary, you could force to always inject raw key events:
+To also use a UHID keyboard, set it explicitly:
 
 ```bash
-scrcpy --raw-key-events
+scrcpy --no-video --no-audio --keyboard=uhid
+scrcpy --no-video --no-audio -K  # short version
 ```
 
-These options have no effect on HID keyboard (all key events are sent as
-scancodes in this mode).
+To use AOA instead (over USB only):
 
-[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
-[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
+```bash
+scrcpy --no-video --no-audio --keyboard=aoa --mouse=aoa
+```
 
 
 ## Copy-paste
@@ -85,6 +83,7 @@ way as <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>v</kbd>).
 To disable automatic clipboard synchronization, use
 `--no-clipboard-autosync`.
 
+
 ## Pinch-to-zoom, rotate and tilt simulation
 
 To simulate "pinch-to-zoom": <kbd>Ctrl</kbd>+_click-and-move_.
@@ -93,37 +92,19 @@ More precisely, hold down <kbd>Ctrl</kbd> while pressing the left-click button.
 Until the left-click button is released, all mouse movements scale and rotate
 the content (if supported by the app) relative to the center of the screen.
 
+https://github.com/Genymobile/scrcpy/assets/543275/26c4a920-9805-43f1-8d4c-608752d04767
+
 To simulate a tilt gesture: <kbd>Shift</kbd>+_click-and-move-up-or-down_.
 
+https://github.com/Genymobile/scrcpy/assets/543275/1e252341-4a90-4b29-9d11-9153b324669f
+
 Technically, _scrcpy_ generates additional touch events from a "virtual finger"
 at a location inverted through the center of the screen. When pressing
-<kbd>Ctrl</kbd> the x and y coordinates are inverted. Using <kbd>Shift</kbd>
-only inverts x.
+<kbd>Ctrl</kbd> the _x_ and _y_ coordinates are inverted. Using <kbd>Shift</kbd>
+only inverts _x_.
 
+This only works for the default mouse mode (`--mouse=sdk`).
 
-## Key repeat
-
-By default, holding a key down generates repeated key events. This can cause
-performance problems in some games, where these events are useless anyway.
-
-To avoid forwarding repeated key events:
-
-```bash
-scrcpy --no-key-repeat
-```
-
-This option has no effect on HID keyboard (key repeat is handled by Android
-directly in this mode).
-
-
-## Right-click and middle-click
-
-By default, right-click triggers BACK (or POWER on) and middle-click triggers
-HOME. To disable these shortcuts and forward the clicks to the device instead:
-
-```bash
-scrcpy --forward-all-clicks
-```
 
 ## File drop
 
@@ -147,7 +128,3 @@ The target directory can be changed on start:
 ```bash
 scrcpy --push-target=/sdcard/Movies/
 ```
-
-## Physical keyboard and mouse simulation
-
-See the dedicated [HID/OTG](hid-otg.md) page.

doc/develop.md

@@ -234,7 +234,7 @@ The video and audio streams are decoded by [FFmpeg].
 The client parses the command line arguments, then [runs one of two code
 paths][run]:
  - scrcpy in "normal" mode ([`scrcpy.c`])
- - scrcpy in [OTG mode](hid-otg.md) ([`scrcpy_otg.c`])
+ - scrcpy in [OTG mode](otg.md) ([`scrcpy_otg.c`])
 
 [run]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/main.c#L81-L82
 [`scrcpy.c`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/scrcpy.c#L292-L293

doc/hid-otg.md

@@ -1,112 +0,0 @@
-# HID/OTG
-
-By default, _scrcpy_ injects input events at the Android API level. As an
-alternative, when connected over USB, it is possible to send HID events, so that
-scrcpy behaves as if it was a physical keyboard and/or mouse connected to the
-Android device.
-
-A special [OTG](#otg) mode allows to control the device without mirroring (and
-without USB debugging).
-
-
-## Physical keyboard simulation
-
-By default, _scrcpy_ uses Android key or text injection. It works everywhere,
-but is limited to ASCII.
-
-Instead, it can simulate a physical USB keyboard on Android to provide a better
-input experience (using [USB HID over AOAv2][hid-aoav2]): the virtual keyboard
-is disabled and it works for all characters and IME.
-
-[hid-aoav2]: https://source.android.com/devices/accessories/aoa2#hid-support
-
-However, it only works if the device is connected via USB.
-
-Note: On Windows, it may only work in [OTG mode](#otg), not while mirroring (it
-is not possible to open a USB device if it is already open by another process
-like the _adb daemon_).
-
-To enable this mode:
-
-```bash
-scrcpy --hid-keyboard
-scrcpy -K   # short version
-```
-
-If it fails for some reason (for example because the device is not connected via
-USB), it automatically fallbacks to the default mode (with a log in the
-console). This allows using the same command line options when connected over
-USB and TCP/IP.
-
-In this mode, raw key events (scancodes) are sent to the device, independently
-of the host key mapping. Therefore, if your keyboard layout does not match, it
-must be configured on the Android device, in Settings → System → Languages and
-input → [Physical keyboard].
-
-This settings page can be started directly:
-
-```bash
-adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
-```
-
-However, the option is only available when the HID keyboard is enabled (or when
-a physical keyboard is connected).
-
-[Physical keyboard]: https://github.com/Genymobile/scrcpy/pull/2632#issuecomment-923756915
-
-
-## Physical mouse simulation
-
-By default, _scrcpy_ uses Android mouse events injection with absolute
-coordinates. By simulating a physical mouse, a mouse pointer appears on the
-Android device, and relative mouse motion, clicks and scrolls are injected.
-
-To enable this mode:
-
-```bash
-scrcpy --hid-mouse
-scrcpy -M   # short version
-```
-
-When this mode is enabled, the computer mouse is "captured" (the mouse pointer
-disappears from the computer and appears on the Android device instead).
-
-Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
-(disable or enable) the mouse capture. Use one of them to give the control of
-the mouse back to the computer.
-
-
-## OTG
-
-It is possible to run _scrcpy_ with only physical keyboard and mouse simulation
-(HID), as if the computer keyboard and mouse were plugged directly to the device
-via an OTG cable.
-
-In this mode, `adb` (USB debugging) is not necessary, and mirroring is disabled.
-
-This is similar to `--hid-keyboard --hid-mouse`, but without mirroring.
-
-To enable OTG mode:
-
-```bash
-scrcpy --otg
-# Pass the serial if several USB devices are available
-scrcpy --otg -s 0123456789abcdef
-```
-
-It is possible to enable only HID keyboard or HID mouse:
-
-```bash
-scrcpy --otg --hid-keyboard              # keyboard only
-scrcpy --otg --hid-mouse                 # mouse only
-scrcpy --otg --hid-keyboard --hid-mouse  # keyboard and mouse
-# for convenience, enable both by default
-scrcpy --otg                             # keyboard and mouse
-```
-
-Like `--hid-keyboard` and `--hid-mouse`, it only works if the device is
-connected over USB.
-
-## HID/OTG issues on Windows
-
-See [FAQ](/FAQ.md#hidotg-issues-on-windows).

doc/keyboard.md

@@ -0,0 +1,136 @@
+# Keyboard
+
+Several keyboard input modes are available:
+
+ - `--keyboard=sdk` (default)
+ - `--keyboard=uhid` (or `-K`): simulates a physical HID keyboard using the UHID
+   kernel module on the device
+ - `--keyboard=aoa`: simulates a physical HID keyboard using the AOAv2 protocol
+ - `--keyboard=disabled`
+
+By default, `sdk` is used, but if you use scrcpy regularly, it is recommended to
+use [`uhid`](#uhid) and configure the keyboard layout once and for all.
+
+
+## SDK keyboard
+
+In this mode (`--keyboard=sdk`, or if the parameter is omitted), keyboard input
+events are injected at the Android API level. It works everywhere, but it is
+limited to ASCII and some other characters.
+
+Note that on some devices, an additional option must be enabled in developer
+options for this keyboard mode to work. See
+[prerequisites](/README.md#prerequisites).
+
+Additional parameters (specific to `--keyboard=sdk`) described below allow to
+customize the behavior.
+
+
+### Text injection preference
+
+Two kinds of [events][textevents] are generated when typing text:
+ - _key events_, signaling that a key is pressed or released;
+ - _text events_, signaling that a text has been entered.
+
+By default, numbers and "special characters" are inserted using text events, but
+letters are injected using key events, so that the keyboard behaves as expected
+in games (typically for WASD keys).
+
+But this may [cause issues][prefertext]. If you encounter such a problem, you
+can inject letters as text (or just switch to [UHID](#uhid)):
+
+```bash
+scrcpy --prefer-text
+```
+
+(but this will break keyboard behavior in games)
+
+On the contrary, you could force to always inject raw key events:
+
+```bash
+scrcpy --raw-key-events
+```
+
+[textevents]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-text-input
+[prefertext]: https://github.com/Genymobile/scrcpy/issues/650#issuecomment-512945343
+
+
+### Key repeat
+
+By default, holding a key down generates repeated key events. Ths can cause
+performance problems in some games, where these events are useless anyway.
+
+To avoid forwarding repeated key events:
+
+```bash
+scrcpy --no-key-repeat
+```
+
+
+## Physical keyboard simulation
+
+Two modes allow to simulate a physical HID keyboard on the device.
+
+To work properly, it is necessary to configure (once and for all) the keyboard
+layout on the device to match that of the computer.
+
+The configuration page can be opened in one of the following ways:
+ - from the scrcpy window (when `uhid` or `aoa` is used), by pressing
+   <kbd>MOD</kbd>+<kbd>k</kbd> (see [shortcuts](shortcuts.md))
+ - from the device, in Settings → System → Languages and input → Physical
+   devices
+ - from a terminal on the computer, by executing `adb shell am start -a
+   android.settings.HARD_KEYBOARD_SETTINGS`
+
+From this configuration page, it is also possible to enable or disable on-screen
+keyboard.
+
+
+### UHID
+
+This mode simulates a physical HID keyboard using the [UHID] kernel module on the
+device.
+
+[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
+
+To enable UHID keyboard, use:
+
+```bash
+scrcpy --keyboard=uhid
+scrcpy -K  # short version
+```
+
+Once the keyboard layout is configured (see above), it is the best mode for
+using the keyboard while mirroring:
+
+ - it works for all characters and IME (contrary to `--keyboard=sdk`)
+ - the on-screen keyboard can be disabled (contrary to `--keyboard=sdk`)
+ - it works over TCP/IP (wirelessly) (contrary to `--keyboard=aoa`)
+ - there are no issues on Windows (contrary to `--keyboard=aoa`)
+
+One drawback is that it may not work on old Android versions due to permission
+errors.
+
+
+### AOA
+
+This mode simulates a physical HID keyboard using the [AOAv2] protocol.
+
+[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
+
+To enable AOA keyboard, use:
+
+```bash
+scrcpy --keyboard=aoa
+```
+
+Contrary to the other modes, it works at the USB level directly (so it only
+works over USB).
+
+It does not use the scrcpy server, and does not require `adb` (USB debugging).
+Therefore, it is possible to control the device (but not mirror) even with USB
+debugging disabled (see [OTG](otg.md)).
+
+Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
+(it is not possible to open a USB device if it is already open by another
+process like the _adb daemon_).

doc/mouse.md

@@ -0,0 +1,118 @@
+# Mouse
+
+Several mouse input modes are available:
+
+ - `--mouse=sdk` (default)
+ - `--mouse=uhid` (or `-M`): simulates a physical HID mouse using the UHID
+   kernel module on the device
+ - `--mouse=aoa`: simulates a physical HID mouse using the AOAv2 protocol
+ - `--mouse=disabled`
+
+
+## SDK mouse
+
+In this mode (`--mouse=sdk`, or if the parameter is omitted), mouse input events
+are injected at the Android API level with absolute coordinates.
+
+Note that on some devices, an additional option must be enabled in developer
+options for this mouse mode to work. See
+[prerequisites](/README.md#prerequisites).
+
+### Mouse hover
+
+By default, mouse hover (mouse motion without any clicks) events are forwarded
+to the device. This can be disabled with:
+
+```
+scrcpy --no-mouse-hover
+```
+
+## Physical mouse simulation
+
+Two modes allow to simulate a physical HID mouse on the device.
+
+In these modes, the computer mouse is "captured": the mouse pointer disappears
+from the computer and appears on the Android device instead.
+
+Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
+(disable or enable) the mouse capture. Use one of them to give the control of
+the mouse back to the computer.
+
+
+### UHID
+
+This mode simulates a physical HID mouse using the [UHID] kernel module on the
+device.
+
+[UHID]: https://kernel.org/doc/Documentation/hid/uhid.txt
+
+To enable UHID mouse, use:
+
+```bash
+scrcpy --mouse=uhid
+scrcpy -M  # short version
+```
+
+
+### AOA
+
+This mode simulates a physical HID mouse using the [AOAv2] protocol.
+
+[AOAv2]: https://source.android.com/devices/accessories/aoa2#hid-support
+
+To enable AOA mouse, use:
+
+```bash
+scrcpy --mouse=aoa
+```
+
+Contrary to the other modes, it works at the USB level directly (so it only
+works over USB).
+
+It does not use the scrcpy server, and does not require `adb` (USB debugging).
+Therefore, it is possible to control the device (but not mirror) even with USB
+debugging disabled (see [OTG](otg.md)).
+
+Note: On Windows, it may only work in [OTG mode](otg.md), not while mirroring
+(it is not possible to open a USB device if it is already open by another
+process like the _adb daemon_).
+
+
+## Mouse bindings
+
+By default, with SDK mouse, right-click triggers BACK (or POWER on) and
+middle-click triggers HOME. In addition, the 4th click triggers APP_SWITCH and
+the 5th click expands the notification panel.
+
+In AOA and UHID mouse modes, all clicks are forwarded by default.
+
+The shortcuts can be configured using `--mouse-bind=xxxx` for any mouse mode.
+The argument must be exactly 4 characters, one for each secondary click:
+
+```
+--mouse-bind=xxxx
+             ^^^^
+             ||||
+             ||| `- 5th click
+             || `-- 4th click
+             | `--- middle click
+              `---- right click
+```
+
+Each character must be one of the following:
+
+ - `+`: forward the click to the device
+ - `-`: ignore the click
+ - `b`: trigger shortcut BACK (or turn screen on if off)
+ - `h`: trigger shortcut HOME
+ - `s`: trigger shortcut APP_SWITCH
+ - `n`: trigger shortcut "expand notification panel"
+
+For example:
+
+```bash
+scrcpy --mouse-bind=bhsn  # the default mode with SDK mouse
+scrcpy --mouse-bind=++++  # forward all clicks (default for AOA/UHID)
+scrcpy --mouse-bind=++bh  # forward right and middle clicks,
+                          # use 4th and 5th for BACK and HOME
+```

doc/otg.md

@@ -0,0 +1,58 @@
+# OTG
+
+By default, _scrcpy_ injects input events at the Android API level. As an
+alternative, it is possible to send HID events, so that scrcpy behaves as if it
+was a [physical keyboard] and/or a [physical mouse] connected to the Android
+device (see [keyboard](keyboard.md) and [mouse](mouse.md)).
+
+[physical keyboard]: keyboard.md#physical-keyboard-simulation
+[physical mouse]: physical-keyboard-simulation
+
+A special mode (OTG) allows to control the device using AOA
+[keyboard](keyboard.md#aoa) and [mouse](mouse.md#aoa), without using _adb_ at
+all (so USB debugging is not necessary). In this mode, video and audio are
+disabled, and `--keyboard=aoa and `--mouse=aoa` are implicitly set.
+
+Therefore, it is possible to run _scrcpy_ with only physical keyboard and mouse
+simulation, as if the computer keyboard and mouse were plugged directly to the
+device via an OTG cable.
+
+To enable OTG mode:
+
+```bash
+scrcpy --otg
+# Pass the serial if several USB devices are available
+scrcpy --otg -s 0123456789abcdef
+```
+
+It is possible to disable keyboard or mouse:
+
+```bash
+scrcpy --otg --keyboard=disabled
+scrcpy --otg --mouse=disabled
+```
+
+It only works if the device is connected over USB.
+
+## OTG issues on Windows
+
+See [FAQ](/FAQ.md#otg-issues-on-windows).
+
+
+## Control only
+
+Note that the purpose of OTG is to control the device without USB debugging
+(adb).
+
+If you want to solely control the device without mirroring while USB debugging
+is enabled, then OTG mode is not necessary.
+
+Instead, disable video and audio, and select UHID (or AOA):
+
+```bash
+scrcpy --no-video --no-audio --keyboard=uhid --mouse=uhid
+scrcpy --no-video --no-audio -KM  # short version
+scrcpy --no-video --no-audio --keyboard=aoa --mouse=aoa
+```
+
+One benefit of UHID is that it also works wirelessly.

doc/recording.md

@@ -58,12 +58,10 @@ orientation](video.md#orientation).
 
 ## No playback
 
-To disable playback while recording:
+To disable playback and control while recording:
 
 ```bash
-scrcpy --no-playback --record=file.mp4
-scrcpy -Nr file.mkv
-# interrupt recording with Ctrl+C
+scrcpy --no-playback --no-control --record=file.mp4
 ```
 
 It is also possible to disable video and audio playback separately:
@@ -73,6 +71,13 @@ It is also possible to disable video and audio playback separately:
 scrcpy --record=file.mkv --no-audio-playback
 ```
 
+To also disable the window:
+
+```bash
+scrcpy --no-playback --no-window --record=file.mp4
+# interrupt recording with Ctrl+C
+```
+
 ## Time limit
 
 To limit the recording time:

doc/shortcuts.md

@@ -13,8 +13,8 @@ It can be changed using `--shortcut-mod`. Possible keys are `lctrl`, `rctrl`,
 # use RCtrl for shortcuts
 scrcpy --shortcut-mod=rctrl
 
-# use either LCtrl+LAlt or LSuper for shortcuts
-scrcpy --shortcut-mod=lctrl+lalt,lsuper
+# use either LCtrl or LSuper for shortcuts
+scrcpy --shortcut-mod=lctrl,lsuper
 ```
 
 _<kbd>[Super]</kbd> is typically the <kbd>Windows</kbd> or <kbd>Cmd</kbd> key._
@@ -28,6 +28,8 @@ _<kbd>[Super]</kbd> is typically the <kbd>Windows</kbd> or <kbd>Cmd</kbd> key._
  | Rotate display right                        | <kbd>MOD</kbd>+<kbd>→</kbd> _(right)_
  | Flip display horizontally                   | <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>←</kbd> _(left)_ \| <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>→</kbd> _(right)_
  | Flip display vertically                     | <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>↑</kbd> _(up)_ \| <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>↓</kbd> _(down)_
+ | Pause or re-pause display                   | <kbd>MOD</kbd>+<kbd>z</kbd>
+ | Unpause display                             | <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>z</kbd>
  | Resize window to 1:1 (pixel-perfect)        | <kbd>MOD</kbd>+<kbd>g</kbd>
  | Resize window to remove black borders       | <kbd>MOD</kbd>+<kbd>w</kbd> \| _Double-left-click¹_
  | Click on `HOME`                             | <kbd>MOD</kbd>+<kbd>h</kbd> \| _Middle-click_
@@ -48,6 +50,7 @@ _<kbd>[Super]</kbd> is typically the <kbd>Windows</kbd> or <kbd>Cmd</kbd> key._
  | Cut to clipboard⁵                           | <kbd>MOD</kbd>+<kbd>x</kbd>
  | Synchronize clipboards and paste⁵           | <kbd>MOD</kbd>+<kbd>v</kbd>
  | Inject computer clipboard text              | <kbd>MOD</kbd>+<kbd>Shift</kbd>+<kbd>v</kbd>
+ | Open keyboard settings (HID keyboard only)  | <kbd>MOD</kbd>+<kbd>k</kbd>
  | Enable/disable FPS counter (on stdout)      | <kbd>MOD</kbd>+<kbd>i</kbd>
  | Pinch-to-zoom/rotate                        | <kbd>Ctrl</kbd>+_click-and-move_
  | Tilt (slide vertically with 2 fingers)      | <kbd>Shift</kbd>+_click-and-move_

doc/window.md

@@ -1,5 +1,14 @@
 # Window
 
+## Disable window
+
+To disable window (may be useful for recording or for playing audio only):
+
+```bash
+scrcpy --no-window --record=file.mp4
+# Ctrl+C to interrupt
+```
+
 ## Title
 
 By default, the window title is the device model. It can be changed:

doc/windows.md

@@ -4,14 +4,14 @@
 
 Download the [latest release]:
 
- - [`scrcpy-win64-v2.3.1.zip`][direct-win64] (64-bit)  
-   <sub>SHA-256: `f1f78ac98214078425804e524a1bed515b9d4b8a05b78d210a4ced2b910b262d`</sub>
- - [`scrcpy-win32-v2.3.1.zip`][direct-win32] (32-bit)  
-   <sub>SHA-256: `5dffc2d432e9b8b5b0e16f12e71428c37c70d9124cfbe7620df0b41b7efe91ff`</sub>
+ - [`scrcpy-win64-v2.4.zip`][direct-win64] (64-bit)  
+   <sub>SHA-256: `9dc56f21bfa455352ec0c58b40feaf2fb02d67372910a4235e298ece286ff3a9`</sub>
+ - [`scrcpy-win32-v2.4.zip`][direct-win32] (32-bit)  
+   <sub>SHA-256: `cf92acc45eef37c6ee2db819f92e420ced3bc50f1348dd57f7d6ca1fc80f6116`</sub>
 
 [latest release]: https://github.com/Genymobile/scrcpy/releases/latest
-[direct-win64]: https://github.com/Genymobile/scrcpy/releases/download/v2.3.1/scrcpy-win64-v2.3.1.zip
-[direct-win32]: https://github.com/Genymobile/scrcpy/releases/download/v2.3.1/scrcpy-win32-v2.3.1.zip
+[direct-win64]: https://github.com/Genymobile/scrcpy/releases/download/v2.4/scrcpy-win64-v2.4.zip
+[direct-win32]: https://github.com/Genymobile/scrcpy/releases/download/v2.4/scrcpy-win32-v2.4.zip
 
 and extract it.
 

install_release.sh

@@ -2,8 +2,8 @@
 set -e
 
 BUILDDIR=build-auto
-PREBUILT_SERVER_URL=https://github.com/Genymobile/scrcpy/releases/download/v2.3.1/scrcpy-server-v2.3.1
-PREBUILT_SERVER_SHA256=f6814822fc308a7a532f253485c9038183c6296a6c5df470a9e383b4f8e7605b
+PREBUILT_SERVER_URL=https://github.com/Genymobile/scrcpy/releases/download/v2.4/scrcpy-server-v2.4
+PREBUILT_SERVER_SHA256=93c272b7438605c055e127f7444064ed78fa9ca49f81156777fd201e79ce7ba3
 
 echo "[scrcpy] Downloading prebuilt server..."
 wget "$PREBUILT_SERVER_URL" -O scrcpy-server

meson.build

@@ -1,5 +1,5 @@
 project('scrcpy', 'c',
-        version: '2.3.1',
+        version: '2.4',
         meson_version: '>= 0.48',
         default_options: [
             'c_std=c11',

release.mk

@@ -62,38 +62,38 @@ build-server:
 		meson setup "$(SERVER_BUILD_DIR)" --buildtype release -Dcompile_app=false )
 	ninja -C "$(SERVER_BUILD_DIR)"
 
-prepare-deps:
-	@app/prebuilt-deps/prepare-adb.sh
-	@app/prebuilt-deps/prepare-sdl.sh
-	@app/prebuilt-deps/prepare-ffmpeg.sh
-	@app/prebuilt-deps/prepare-libusb.sh
+prepare-deps-win32:
+	@app/deps/adb.sh win32
+	@app/deps/sdl.sh win32
+	@app/deps/ffmpeg.sh win32
+	@app/deps/libusb.sh win32
 
-build-win32: prepare-deps
+prepare-deps-win64:
+	@app/deps/adb.sh win64
+	@app/deps/sdl.sh win64
+	@app/deps/ffmpeg.sh win64
+	@app/deps/libusb.sh win64
+
+build-win32: prepare-deps-win32
 	rm -rf "$(WIN32_BUILD_DIR)"
 	mkdir -p "$(WIN32_BUILD_DIR)/local"
-	cp -r app/prebuilt-deps/data/ffmpeg-6.1-scrcpy-3/win32/. "$(WIN32_BUILD_DIR)/local/"
-	cp -r app/prebuilt-deps/data/SDL2-2.28.5/i686-w64-mingw32/. "$(WIN32_BUILD_DIR)/local/"
-	cp -r app/prebuilt-deps/data/libusb-1.0.26/libusb-MinGW-Win32/. "$(WIN32_BUILD_DIR)/local/"
 	meson setup "$(WIN32_BUILD_DIR)" \
-		--pkg-config-path="$(WIN32_BUILD_DIR)/local/lib/pkgconfig" \
-		-Dc_args="-I$(PWD)/$(WIN32_BUILD_DIR)/local/include" \
-		-Dc_link_args="-L$(PWD)/$(WIN32_BUILD_DIR)/local/lib" \
+		--pkg-config-path="app/deps/work/install/win32/lib/pkgconfig" \
+		-Dc_args="-I$(PWD)/app/deps/work/install/win32/include" \
+		-Dc_link_args="-L$(PWD)/app/deps/work/install/win32/lib" \
 		--cross-file=cross_win32.txt \
 		--buildtype=release --strip -Db_lto=true \
 		-Dcompile_server=false \
 		-Dportable=true
 	ninja -C "$(WIN32_BUILD_DIR)"
 
-build-win64: prepare-deps
+build-win64: prepare-deps-win64
 	rm -rf "$(WIN64_BUILD_DIR)"
 	mkdir -p "$(WIN64_BUILD_DIR)/local"
-	cp -r app/prebuilt-deps/data/ffmpeg-6.1-scrcpy-3/win64/. "$(WIN64_BUILD_DIR)/local/"
-	cp -r app/prebuilt-deps/data/SDL2-2.28.5/x86_64-w64-mingw32/. "$(WIN64_BUILD_DIR)/local/"
-	cp -r app/prebuilt-deps/data/libusb-1.0.26/libusb-MinGW-x64/. "$(WIN64_BUILD_DIR)/local/"
 	meson setup "$(WIN64_BUILD_DIR)" \
-		--pkg-config-path="$(WIN64_BUILD_DIR)/local/lib/pkgconfig" \
-		-Dc_args="-I$(PWD)/$(WIN64_BUILD_DIR)/local/include" \
-		-Dc_link_args="-L$(PWD)/$(WIN64_BUILD_DIR)/local/lib" \
+		--pkg-config-path="app/deps/work/install/win64/lib/pkgconfig" \
+		-Dc_args="-I$(PWD)/app/deps/work/install/win64/include" \
+		-Dc_link_args="-L$(PWD)/app/deps/work/install/win64/lib" \
 		--cross-file=cross_win64.txt \
 		--buildtype=release --strip -Db_lto=true \
 		-Dcompile_server=false \
@@ -108,10 +108,8 @@ dist-win32: build-server build-win32
 	cp app/data/scrcpy-noconsole.vbs "$(DIST)/$(WIN32_TARGET_DIR)/"
 	cp app/data/icon.png "$(DIST)/$(WIN32_TARGET_DIR)/"
 	cp app/data/open_a_terminal_here.bat "$(DIST)/$(WIN32_TARGET_DIR)/"
-	cp app/prebuilt-deps/data/platform-tools-34.0.5/adb.exe "$(DIST)/$(WIN32_TARGET_DIR)/"
-	cp app/prebuilt-deps/data/platform-tools-34.0.5/AdbWinApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
-	cp app/prebuilt-deps/data/platform-tools-34.0.5/AdbWinUsbApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
-	cp "$(WIN32_BUILD_DIR)"/local/bin/*.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
+	cp app/deps/work/install/win32/bin/*.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
+	cp app/deps/work/install/win32/bin/adb.exe "$(DIST)/$(WIN32_TARGET_DIR)/"
 
 dist-win64: build-server build-win64
 	mkdir -p "$(DIST)/$(WIN64_TARGET_DIR)"
@@ -121,10 +119,8 @@ dist-win64: build-server build-win64
 	cp app/data/scrcpy-noconsole.vbs "$(DIST)/$(WIN64_TARGET_DIR)/"
 	cp app/data/icon.png "$(DIST)/$(WIN64_TARGET_DIR)/"
 	cp app/data/open_a_terminal_here.bat "$(DIST)/$(WIN64_TARGET_DIR)/"
-	cp app/prebuilt-deps/data/platform-tools-34.0.5/adb.exe "$(DIST)/$(WIN64_TARGET_DIR)/"
-	cp app/prebuilt-deps/data/platform-tools-34.0.5/AdbWinApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
-	cp app/prebuilt-deps/data/platform-tools-34.0.5/AdbWinUsbApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
-	cp "$(WIN64_BUILD_DIR)"/local/bin/*.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
+	cp app/deps/work/install/win64/bin/*.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
+	cp app/deps/work/install/win64/bin/adb.exe "$(DIST)/$(WIN64_TARGET_DIR)/"
 
 zip-win32: dist-win32
 	cd "$(DIST)"; \

server/build.gradle

@@ -7,8 +7,8 @@ android {
         applicationId "com.genymobile.scrcpy"
         minSdkVersion 21
         targetSdkVersion 34
-        versionCode 20301
-        versionName "2.3.1"
+        versionCode 20400
+        versionName "2.4"
         testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
     }
     buildTypes {

server/build_without_gradle.sh

@@ -12,7 +12,7 @@
 set -e
 
 SCRCPY_DEBUG=false
-SCRCPY_VERSION_NAME=2.3.1
+SCRCPY_VERSION_NAME=2.4
 
 PLATFORM=${ANDROID_PLATFORM:-34}
 BUILD_TOOLS=${ANDROID_BUILD_TOOLS:-34.0.0}

server/src/main/java/com/genymobile/scrcpy/AudioCapture.java

@@ -79,7 +79,7 @@ public final class AudioCapture {
         intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
         intent.addCategory(Intent.CATEGORY_LAUNCHER);
         intent.setComponent(new ComponentName(FakeContext.PACKAGE_NAME, "com.android.shell.HeapDumpActivity"));
-        ServiceManager.getActivityManager().startActivityAsUserWithFeature(intent);
+        ServiceManager.getActivityManager().startActivity(intent);
     }
 
     private static void stopWorkaroundAndroid11() {

server/src/main/java/com/genymobile/scrcpy/CameraCapture.java

@@ -127,6 +127,10 @@ public class CameraCapture extends SurfaceCapture {
 
         StreamConfigurationMap configs = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
         android.util.Size[] sizes = highSpeed ? configs.getHighSpeedVideoSizes() : configs.getOutputSizes(MediaCodec.class);
+        if (sizes == null) {
+            return null;
+        }
+
         Stream<android.util.Size> stream = Arrays.stream(sizes);
         if (maxSize > 0) {
             stream = stream.filter(it -> it.getWidth() <= maxSize && it.getHeight() <= maxSize);

server/src/main/java/com/genymobile/scrcpy/ControlMessage.java

@@ -17,6 +17,9 @@ public final class ControlMessage {
     public static final int TYPE_SET_CLIPBOARD = 9;
     public static final int TYPE_SET_SCREEN_POWER_MODE = 10;
     public static final int TYPE_ROTATE_DEVICE = 11;
+    public static final int TYPE_UHID_CREATE = 12;
+    public static final int TYPE_UHID_INPUT = 13;
+    public static final int TYPE_OPEN_HARD_KEYBOARD_SETTINGS = 14;
 
     public static final long SEQUENCE_INVALID = 0;
 
@@ -40,6 +43,8 @@ public final class ControlMessage {
     private boolean paste;
     private int repeat;
     private long sequence;
+    private int id;
+    private byte[] data;
 
     private ControlMessage() {
     }
@@ -123,6 +128,22 @@ public final class ControlMessage {
         return msg;
     }
 
+    public static ControlMessage createUhidCreate(int id, byte[] reportDesc) {
+        ControlMessage msg = new ControlMessage();
+        msg.type = TYPE_UHID_CREATE;
+        msg.id = id;
+        msg.data = reportDesc;
+        return msg;
+    }
+
+    public static ControlMessage createUhidInput(int id, byte[] data) {
+        ControlMessage msg = new ControlMessage();
+        msg.type = TYPE_UHID_INPUT;
+        msg.id = id;
+        msg.data = data;
+        return msg;
+    }
+
     public int getType() {
         return type;
     }
@@ -186,4 +207,12 @@ public final class ControlMessage {
     public long getSequence() {
         return sequence;
     }
+
+    public int getId() {
+        return id;
+    }
+
+    public byte[] getData() {
+        return data;
+    }
 }