Add UHID gamepad rumble WIP

There was a registration mechanism to listen to HID outputs with a
specific HID id.

However, the UHID gamepad processor handles several ids, so it will not
work. We could complexify the registration mechanism, but instead,
directly dispatch to the expected processor based on the UHID id.

Concretely, instead of passing a sc_uhid_devices instance to construct a
sc_keyboard_uhid, so that it can register itself, construct the
sc_uhid_devices with all the UHID instances (currently only
sc_keyboard_uhid) so that it can dispatch HID outputs directly.

.github/FUNDING.yml

@@ -0,0 +1,3 @@
+github: [rom1v]
+liberapay: rom1v
+custom: ["https://paypal.me/rom2v"]

.github/ISSUE_TEMPLATE/bug_report.md

@@ -7,17 +7,25 @@ assignees: ''
 
 ---
 
- - [ ] I have read the [FAQ](https://github.com/Genymobile/scrcpy/blob/master/FAQ.md).
+_Please read the [prerequisites] to run scrcpy._
- - [ ] I have searched in existing [issues](https://github.com/Genymobile/scrcpy/issues).
 
-**Environment**
+[prerequisites]: https://github.com/Genymobile/scrcpy#prerequisites
- - OS: [e.g. Debian, Windows, macOS...]
+
- - scrcpy version: [e.g. 1.12.1]
+_Also read the [FAQ] and check if your [issue][issues] already exists._
- - installation method: [e.g. manual build, apt, snap, brew, Windows release...]
+
- - device model:
+[FAQ]: https://github.com/Genymobile/scrcpy/blob/master/FAQ.md
- - Android version: [e.g. 10]
+[issues]: https://github.com/Genymobile/scrcpy/issues
+
+## Environment
+
+ - **OS:** [e.g. Debian, Windows, macOS...]
+ - **Scrcpy version:** [e.g. 2.5]
+ - **Installation method:** [e.g. manual build, apt, snap, brew, Windows release...]
+ - **Device model:**
+ - **Android version:** [e.g. 14]
+
+## Describe the bug
 
-**Describe the bug**
 A clear and concise description of what the bug is.
 
 On errors, please provide the output of the console (and `adb logcat` if relevant).

.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,8 @@
+---
+name: Question
+about: Ask a question about scrcpy
+title: ''
+labels: ''
+assignees: ''
+
+---

FAQ.md

@@ -133,9 +133,9 @@ Try with another USB cable or plug it into another USB port. See [#281] and
 [#283]: https://github.com/Genymobile/scrcpy/issues/283
 
 
-## HID/OTG issues on Windows
+## OTG issues on Windows
 
-On Windows, if `scrcpy --otg` (or `--hid-keyboard`/`--hid-mouse`) results in:
+On Windows, if `scrcpy --otg` (or `--keyboard=aoa`/`--mouse=aoa`) results in:
 
 >     ERROR: Could not find any USB device
 
@@ -170,12 +170,13 @@ The default text injection method is [limited to ASCII characters][text-input].
 A trick allows to also inject some [accented characters][accented-characters],
 but that's all. See [#37].
 
-It is also possible to simulate a [physical keyboard][hid] (HID).
+To avoid the problem, [change the keyboard mode to simulate a physical
+keyboard][hid].
 
 [text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
 [accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
 [#37]: https://github.com/Genymobile/scrcpy/issues/37
-[hid]: doc/hid-otg.md
+[hid]: doc/keyboard.md#physical-keyboard-simulation
 
 
 ## Client issues
@@ -222,7 +223,7 @@ java.lang.IllegalStateException
         at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
 ```
 
-then try with another [encoder](doc/video.md#codec).
+then try with another [encoder](doc/video.md#encoder).
 
 
 ## Translations

LICENSE

@@ -188,7 +188,7 @@
       identification within third-party archives.
 
    Copyright (C) 2018 Genymobile
-   Copyright (C) 2018-2023 Romain Vimont
+   Copyright (C) 2018-2024 Romain Vimont
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -2,7 +2,7 @@
 source for the project. Do not download releases from random websites, even if
 their name contains `scrcpy`.**
 
-# scrcpy (v2.3.1)
+# scrcpy (v2.6.1)
 
 <img src="app/data/icon.svg" width="128" height="128" alt="scrcpy" align="right" />
 
@@ -36,10 +36,13 @@ Its features include:
  - [configurable quality](doc/video.md)
  - [camera mirroring](doc/camera.md) (Android 12+)
  - [mirroring as a webcam (V4L2)](doc/v4l2.md) (Linux-only)
- - [physical keyboard/mouse simulation (HID)](doc/hid-otg.md)
+ - physical [keyboard][hid-keyboard] and [mouse][hid-mouse] simulation (HID)
- - [OTG mode](doc/hid-otg.md#otg)
+ - [OTG mode](doc/otg.md)
  - and more…
 
+[hid-keyboard]: doc/keyboard.md#physical-keyboard-simulation
+[hid-mouse]: doc/mouse.md#physical-mouse-simulation
+
 ## Prerequisites
 
 The Android device requires at least API 21 (Android 5.0).
@@ -50,15 +53,20 @@ Make sure you [enabled USB debugging][enable-adb] on your device(s).
 
 [enable-adb]: https://developer.android.com/studio/debug/dev-options#enable
 
-On some devices, you also need to enable [an additional option][control] `USB
+On some devices (especially Xiaomi), you might get the following error:
-debugging (Security Settings)` (this is an item different from `USB debugging`)
+
-to control it using a keyboard and mouse. Rebooting the device is necessary once
+```
-this option is set.
+java.lang.SecurityException: Injecting input events requires the caller (or the source of the instrumentation, if any) to have the INJECT_EVENTS permission.
+```
+
+In that case, you need to enable [an additional option][control] `USB debugging
+(Security Settings)` (this is an item different from `USB debugging`) to control
+it using a keyboard and mouse. Rebooting the device is necessary once this
+option is set.
 
 [control]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
 
-Note that USB debugging is not required to run scrcpy in [OTG
+Note that USB debugging is not required to run scrcpy in [OTG mode](doc/otg.md).
-mode](doc/hid-otg.md#otg).
 
 
 ## Get the app
@@ -68,6 +76,41 @@ mode](doc/hid-otg.md#otg).
  - [macOS](doc/macos.md)
 
 
+## Usage examples
+
+There are a lot of options, [documented](#user-documentation) in separate pages.
+Here are just some common examples.
+
+ - Capture the screen in H.265 (better quality), limit the size to 1920, limit
+   the frame rate to 60fps, disable audio, and control the device by simulating
+   a physical keyboard:
+
+    ```bash
+    scrcpy --video-codec=h265 --max-size=1920 --max-fps=60 --no-audio --keyboard=uhid
+    scrcpy --video-codec=h265 -m1920 --max-fps=60 --no-audio -K  # short version
+    ```
+
+ - Record the device camera in H.265 at 1920x1080 (and microphone) to an MP4
+   file:
+
+    ```bash
+    scrcpy --video-source=camera --video-codec=h265 --camera-size=1920x1080 --record=file.mp4
+    ```
+
+ - Capture the device front camera and expose it as a webcam on the computer (on
+   Linux):
+
+    ```bash
+    scrcpy --video-source=camera --camera-size=1920x1080 --camera-facing=front --v4l2-sink=/dev/video2 --no-playback
+    ```
+
+ - Control the device without mirroring by simulating a physical keyboard and
+   mouse (USB debugging not required):
+
+    ```bash
+    scrcpy --otg
+    ```
+
 ## User documentation
 
 The application provides a lot of features and configuration options. They are
@@ -77,11 +120,13 @@ documented in the following pages:
  - [Video](doc/video.md)
  - [Audio](doc/audio.md)
  - [Control](doc/control.md)
+ - [Keyboard](doc/keyboard.md)
+ - [Mouse](doc/mouse.md)
  - [Device](doc/device.md)
  - [Window](doc/window.md)
  - [Recording](doc/recording.md)
  - [Tunnels](doc/tunnels.md)
- - [HID/OTG](doc/hid-otg.md)
+ - [OTG](doc/otg.md)
  - [Camera](doc/camera.md)
  - [Video4Linux](doc/v4l2.md)
  - [Shortcuts](doc/shortcuts.md)
@@ -109,11 +154,14 @@ documented in the following pages:
 
 ## Contact
 
-If you encounter a bug, please read the [FAQ](FAQ.md) first, then open an [issue].
+You can open an [issue] for bug reports, feature requests or general questions.
+
+For bug reports, please read the [FAQ](FAQ.md) first, you might find a solution
+to your problem immediately.
 
 [issue]: https://github.com/Genymobile/scrcpy/issues
 
-For general questions or discussions, you can also use:
+You can also use:
 
  - Reddit: [`r/scrcpy`](https://www.reddit.com/r/scrcpy)
  - Twitter: [`@scrcpy_app`](https://twitter.com/scrcpy_app)
@@ -134,7 +182,7 @@ work][donate]:
 ## Licence
 
     Copyright (C) 2018 Genymobile
-    Copyright (C) 2018-2023 Romain Vimont
+    Copyright (C) 2018-2024 Romain Vimont
 
     Licensed under the Apache License, Version 2.0 (the "License");
     you may not use this file except in compliance with the License.

app/data/bash-completion/scrcpy

@@ -6,6 +6,7 @@ _scrcpy() {
         --audio-buffer=
         --audio-codec=
         --audio-codec-options=
+        --audio-dup
         --audio-encoder=
         --audio-source=
         --audio-output-buffer=
@@ -25,10 +26,12 @@ _scrcpy() {
         -e --select-tcpip
         -f --fullscreen
         --force-adb-forward
-        --forward-all-clicks
+        -G
+        --gamepad=
         -h --help
+        -K
+        --keyboard=
         --kill-adb-on-close
-        -K --hid-keyboard
         --legacy-paste
         --list-camera-sizes
         --list-cameras
@@ -37,8 +40,10 @@ _scrcpy() {
         --lock-video-orientation
         --lock-video-orientation=
         -m --max-size=
-        -M --hid-mouse
+        -M
         --max-fps=
+        --mouse=
+        --mouse-bind=
         -n --no-control
         -N --no-playback
         --no-audio
@@ -48,6 +53,7 @@ _scrcpy() {
         --no-downsize-on-error
         --no-key-repeat
         --no-mipmaps
+        --no-mouse-hover
         --no-power-on
         --no-video
         --no-video-playback
@@ -108,20 +114,31 @@ _scrcpy() {
             return
             ;;
         --audio-source)
-            COMPREPLY=($(compgen -W 'output mic' -- "$cur"))
+            COMPREPLY=($(compgen -W 'output mic playback' -- "$cur"))
             return
             ;;
         --camera-facing)
             COMPREPLY=($(compgen -W 'front back external' -- "$cur"))
             return
             ;;
-        --orientation
+        --keyboard)
-        --display-orientation)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
-            COMPREPLY=($(compgen -> '0 90 180 270 flip0 flip90 flip180 flip270' -- "$cur"))
+            return
+            ;;
+        --mouse)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
+            return
+            ;;
+        --gamepad)
+            COMPREPLY=($(compgen -W 'disabled uhid aoa' -- "$cur"))
+            return
+            ;;
+        --orientation|--display-orientation)
+            COMPREPLY=($(compgen -W '0 90 180 270 flip0 flip90 flip180 flip270' -- "$cur"))
             return
             ;;
         --record-orientation)
-            COMPREPLY=($(compgen -> '0 90 180 270' -- "$cur"))
+            COMPREPLY=($(compgen -W '0 90 180 270' -- "$cur"))
             return
             ;;
         --lock-video-orientation)

app/data/zsh-completion/_scrcpy

@@ -13,8 +13,9 @@ arguments=(
     '--audio-buffer=[Configure the audio buffering delay (in milliseconds)]'
     '--audio-codec=[Select the audio codec]:codec:(opus aac flac raw)'
     '--audio-codec-options=[Set a list of comma-separated key\:type=value options for the device audio encoder]'
+    '--audio-dup=[Duplicate audio]'
     '--audio-encoder=[Use a specific MediaCodec audio encoder]'
-    '--audio-source=[Select the audio source]:source:(output mic)'
+    '--audio-source=[Select the audio source]:source:(output mic playback)'
     '--audio-output-buffer=[Configure the size of the SDL audio output buffer (in milliseconds)]'
     {-b,--video-bit-rate=}'[Encode the video at the given bit-rate]'
     '--camera-ar=[Select the camera size by its aspect ratio]'
@@ -32,10 +33,12 @@ arguments=(
     {-e,--select-tcpip}'[Use TCP/IP device]'
     {-f,--fullscreen}'[Start in fullscreen]'
     '--force-adb-forward[Do not attempt to use \"adb reverse\" to connect to the device]'
-    '--forward-all-clicks[Forward clicks to device]'
+    '-G[Use UHID/AOA gamepad (same as --gamepad=uhid or --gamepad=aoa, depending on OTG mode)]'
+    '--gamepad=[Set the gamepad input mode]:mode:(disabled uhid aoa)'
     {-h,--help}'[Print the help]'
+    '-K[Use UHID/AOA keyboard (same as --keyboard=uhid or --keyboard=aoa, depending on OTG mode)]'
+    '--keyboard=[Set the keyboard input mode]:mode:(disabled sdk uhid aoa)'
     '--kill-adb-on-close[Kill adb when scrcpy terminates]'
-    {-K,--hid-keyboard}'[Simulate a physical keyboard by using HID over AOAv2]'
     '--legacy-paste[Inject computer clipboard text as a sequence of key events on Ctrl+v]'
     '--list-camera-sizes[List the valid camera capture sizes]'
     '--list-cameras[List cameras available on the device]'
@@ -43,8 +46,10 @@ arguments=(
     '--list-encoders[List video and audio encoders available on the device]'
     '--lock-video-orientation=[Lock video orientation]:orientation:(unlocked initial 0 90 180 270)'
     {-m,--max-size=}'[Limit both the width and height of the video to value]'
-    {-M,--hid-mouse}'[Simulate a physical mouse by using HID over AOAv2]'
+    '-M[Use UHID/AOA mouse (same as --mouse=uhid or --mouse=aoa, depending on OTG mode)]'
     '--max-fps=[Limit the frame rate of screen capture]'
+    '--mouse=[Set the mouse input mode]:mode:(disabled sdk uhid aoa)'
+    '--mouse-bind=[Configure bindings of secondary clicks]'
     {-n,--no-control}'[Disable device control \(mirror the device in read only\)]'
     {-N,--no-playback}'[Disable video and audio playback]'
     '--no-audio[Disable audio forwarding]'
@@ -54,6 +59,7 @@ arguments=(
     '--no-downsize-on-error[Disable lowering definition on MediaCodec error]'
     '--no-key-repeat[Do not forward repeated key events when a key is held down]'
     '--no-mipmaps[Disable the generation of mipmaps]'
+    '--no-mouse-hover[Do not forward mouse hover events]'
     '--no-power-on[Do not power on the device on start]'
     '--no-video[Disable video forwarding]'
     '--no-video-playback[Disable video playback]'

app/deps/.gitignore

@@ -0,0 +1 @@
+/work

app/deps/README

@@ -0,0 +1,27 @@
+This directory (app/deps/) contains:
+
+*.sh          : shell scripts to download and build dependencies
+
+patches/      : patches to fix dependencies (used by scripts)
+
+work/sources/ : downloaded tarballs and extracted folders
+                    ffmpeg-6.1.1.tar.xz
+                    ffmpeg-6.1.1/
+                    libusb-1.0.27.tar.gz
+                    libusb-1.0.27/
+                    ...
+work/build/   : build dirs for each dependency/version/architecture
+                    ffmpeg-6.1.1/win32/
+                    ffmpeg-6.1.1/win64/
+                    libusb-1.0.27/win32/
+                    libusb-1.0.27/win64/
+                    ...
+work/install/ : install dirs for each architexture
+                    win32/bin/
+                    win32/include/
+                    win32/lib/
+                    win32/share/
+                    win64/bin/
+                    win64/include/
+                    win64/lib/
+                    win64/share/

app/deps/adb.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=35.0.0
+FILENAME=platform-tools_r$VERSION-windows.zip
+PROJECT_DIR=platform-tools-$VERSION
+SHA256SUM=7ab78a8f8b305ae4d0de647d99c43599744de61a0838d3a47bda0cdffefee87e
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://dl.google.com/android/repository/$FILENAME" "$FILENAME" "$SHA256SUM"
+    mkdir -p "$PROJECT_DIR"
+    cd "$PROJECT_DIR"
+    ZIP_PREFIX=platform-tools
+    unzip "../$FILENAME" \
+        "$ZIP_PREFIX"/AdbWinApi.dll \
+        "$ZIP_PREFIX"/AdbWinUsbApi.dll \
+        "$ZIP_PREFIX"/adb.exe
+    mv "$ZIP_PREFIX"/* .
+    rmdir "$ZIP_PREFIX"
+fi
+
+mkdir -p "$INSTALL_DIR/$HOST/bin"
+cd "$INSTALL_DIR/$HOST/bin"
+cp -r "$SOURCES_DIR/$PROJECT_DIR"/. "$INSTALL_DIR/$HOST/bin/"

app/deps/common

@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# This file is intended to be sourced by other scripts, not executed
+
+if [[ $# != 1 ]]
+then
+    # <host>: win32 or win64
+    echo "Syntax: $0 <host>" >&2
+    exit 1
+fi
+
+HOST="$1"
+
+if [[ "$HOST" = win32 ]]
+then
+    HOST_TRIPLET=i686-w64-mingw32
+elif [[ "$HOST" = win64 ]]
+then
+    HOST_TRIPLET=x86_64-w64-mingw32
+else
+    echo "Unsupported host: $HOST" >&2
+    exit 1
+fi
+
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+
+PATCHES_DIR="$PWD/patches"
+
+WORK_DIR="$PWD/work"
+SOURCES_DIR="$WORK_DIR/sources"
+BUILD_DIR="$WORK_DIR/build"
+INSTALL_DIR="$WORK_DIR/install"
+
+mkdir -p "$INSTALL_DIR" "$SOURCES_DIR" "$WORK_DIR"
+
+checksum() {
+    local file="$1"
+    local sum="$2"
+    echo "$file: verifying checksum..."
+    echo "$sum  $file" | sha256sum -c
+}
+
+get_file() {
+    local url="$1"
+    local file="$2"
+    local sum="$3"
+    if [[ -f "$file" ]]
+    then
+        echo "$file: found"
+    else
+        echo "$file: not found, downloading..."
+        wget "$url" -O "$file"
+    fi
+    checksum "$file" "$sum"
+}

app/deps/ffmpeg.sh

@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=7.0.1
+FILENAME=ffmpeg-$VERSION.tar.xz
+PROJECT_DIR=ffmpeg-$VERSION
+SHA256SUM=bce9eeb0f17ef8982390b1f37711a61b4290dc8c2a0c1a37b5857e85bfb0e4ff
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://ffmpeg.org/releases/$FILENAME" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+if [[ "$HOST" = win32 ]]
+then
+    ARCH=x86
+elif [[ "$HOST" = win64 ]]
+then
+    ARCH=x86_64
+else
+    echo "Unsupported host: $HOST" >&2
+    exit 1
+fi
+
+# -static-libgcc to avoid missing libgcc_s_dw2-1.dll
+# -static to avoid dynamic dependency to zlib
+export CFLAGS='-static-libgcc -static'
+export CXXFLAGS="$CFLAGS"
+export LDFLAGS='-static-libgcc -static'
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --enable-cross-compile \
+        --target-os=mingw32 \
+        --arch="$ARCH" \
+        --cross-prefix="${HOST_TRIPLET}-" \
+        --cc="${HOST_TRIPLET}-gcc" \
+        --extra-cflags="-O2 -fPIC" \
+        --enable-shared \
+        --disable-static \
+        --disable-programs \
+        --disable-doc \
+        --disable-swscale \
+        --disable-postproc \
+        --disable-avfilter \
+        --disable-avdevice \
+        --disable-network \
+        --disable-everything \
+        --enable-swresample \
+        --enable-decoder=h264 \
+        --enable-decoder=hevc \
+        --enable-decoder=av1 \
+        --enable-decoder=pcm_s16le \
+        --enable-decoder=opus \
+        --enable-decoder=aac \
+        --enable-decoder=flac \
+        --enable-decoder=png \
+        --enable-protocol=file \
+        --enable-demuxer=image2 \
+        --enable-parser=png \
+        --enable-zlib \
+        --enable-muxer=matroska \
+        --enable-muxer=mp4 \
+        --enable-muxer=opus \
+        --enable-muxer=flac \
+        --enable-muxer=wav \
+        --disable-vulkan
+fi
+
+make -j
+make install

app/deps/libusb.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=1.0.27
+FILENAME=libusb-$VERSION.tar.gz
+PROJECT_DIR=libusb-$VERSION
+SHA256SUM=e8f18a7a36ecbb11fb820bd71540350d8f61bcd9db0d2e8c18a6fb80b214a3de
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://github.com/libusb/libusb/archive/refs/tags/v$VERSION.tar.gz" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+export CFLAGS='-O2'
+export CXXFLAGS="$CFLAGS"
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/bootstrap.sh
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --host="$HOST_TRIPLET" \
+        --enable-shared \
+        --disable-static
+fi
+
+make -j
+make install-strip

app/deps/sdl.sh

@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=2.30.5
+FILENAME=SDL-$VERSION.tar.gz
+PROJECT_DIR=SDL-release-$VERSION
+SHA256SUM=be3ca88f8c362704627a0bc5406edb2cd6cc6ba463596d81ebb7c2f18763d3bf
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://github.com/libsdl-org/SDL/archive/refs/tags/release-$VERSION.tar.gz" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+export CFLAGS='-O2'
+export CXXFLAGS="$CFLAGS"
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --host="$HOST_TRIPLET" \
+        --enable-shared \
+        --disable-static
+fi
+
+make -j
+# There is no "make install-strip"
+make install
+# Strip manually
+${HOST_TRIPLET}-strip "$INSTALL_DIR/$HOST/bin/SDL2.dll"

app/meson.build

@@ -15,13 +15,14 @@ src = [
     'src/demuxer.c',
     'src/device_msg.c',
     'src/display.c',
+    'src/events.c',
     'src/icon.c',
     'src/file_pusher.c',
     'src/fps_counter.c',
     'src/frame_buffer.c',
     'src/input_manager.c',
-    'src/keyboard_inject.c',
+    'src/keyboard_sdk.c',
-    'src/mouse_inject.c',
+    'src/mouse_sdk.c',
     'src/opengl.c',
     'src/options.c',
     'src/packet_merger.c',
@@ -31,11 +32,18 @@ src = [
     'src/screen.c',
     'src/server.c',
     'src/version.c',
+    'src/hid/hid_gamepad.c',
+    'src/hid/hid_keyboard.c',
+    'src/hid/hid_mouse.c',
     'src/trait/frame_source.c',
     'src/trait/packet_source.c',
+    'src/uhid/gamepad_uhid.c',
+    'src/uhid/keyboard_uhid.c',
+    'src/uhid/mouse_uhid.c',
+    'src/uhid/uhid_output.c',
     'src/util/acksync.c',
+    'src/util/audiobuf.c',
     'src/util/average.c',
-    'src/util/bytebuf.c',
     'src/util/file.c',
     'src/util/intmap.c',
     'src/util/intr.c',
@@ -88,8 +96,9 @@ usb_support = get_option('usb')
 if usb_support
     src += [
         'src/usb/aoa_hid.c',
-        'src/usb/hid_keyboard.c',
+        'src/usb/gamepad_aoa.c',
-        'src/usb/hid_mouse.c',
+        'src/usb/keyboard_aoa.c',
+        'src/usb/mouse_aoa.c',
         'src/usb/scrcpy_otg.c',
         'src/usb/screen_otg.c',
         'src/usb/usb.c',
@@ -212,9 +221,10 @@ if get_option('buildtype') == 'debug'
         ['test_binary', [
             'tests/test_binary.c',
         ]],
-        ['test_bytebuf', [
+        ['test_audiobuf', [
-            'tests/test_bytebuf.c',
+            'tests/test_audiobuf.c',
-            'src/util/bytebuf.c',
+            'src/util/audiobuf.c',
+            'src/util/memory.c',
         ]],
         ['test_cli', [
             'tests/test_cli.c',

app/prebuilt-deps/.gitignore

@@ -1 +0,0 @@
-/data

app/prebuilt-deps/common

@@ -1,22 +0,0 @@
-PREBUILT_DATA_DIR=data
-
-checksum() {
-    local file="$1"
-    local sum="$2"
-    echo "$file: verifying checksum..."
-    echo "$sum  $file" | sha256sum -c
-}
-
-get_file() {
-    local url="$1"
-    local file="$2"
-    local sum="$3"
-    if [[ -f "$file" ]]
-    then
-        echo "$file: found"
-    else
-        echo "$file: not found, downloading..."
-        wget "$url" -O "$file"
-    fi
-    checksum "$file" "$sum"
-}

app/prebuilt-deps/prepare-adb.sh

@@ -1,32 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-DEP_DIR=platform-tools-34.0.5
-
-FILENAME=platform-tools_r34.0.5-windows.zip
-SHA256SUM=3f8320152704377de150418a3c4c9d07d16d80a6c0d0d8f7289c22c499e33571
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://dl.google.com/android/repository/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-ZIP_PREFIX=platform-tools
-unzip "../$FILENAME" \
-    "$ZIP_PREFIX"/AdbWinApi.dll \
-    "$ZIP_PREFIX"/AdbWinUsbApi.dll \
-    "$ZIP_PREFIX"/adb.exe
-mv "$ZIP_PREFIX"/* .
-rmdir "$ZIP_PREFIX"

app/prebuilt-deps/prepare-ffmpeg.sh

@@ -1,30 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-VERSION=6.1-scrcpy-3
-DEP_DIR="ffmpeg-$VERSION"
-
-FILENAME="$DEP_DIR".7z
-SHA256SUM=b646d18a3d543a4e4c46881568213499f22e4454a464e1552f03f2ac9cc3a05a
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/rom1v/scrcpy-deps/releases/download/$VERSION/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-ZIP_PREFIX=ffmpeg
-7z x "../$FILENAME"
-mv "$ZIP_PREFIX"/* .
-rmdir "$ZIP_PREFIX"

app/prebuilt-deps/prepare-libusb.sh

@@ -1,39 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-VERSION=1.0.26
-DEP_DIR="libusb-$VERSION"
-
-FILENAME="libusb-$VERSION-binaries.7z"
-SHA256SUM=9c242696342dbde9cdc47239391f71833939bf9f7aa2bbb28cdaabe890465ec5
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/libusb/libusb/releases/download/v$VERSION/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-7z x "../$FILENAME" \
-    "libusb-$VERSION-binaries/libusb-MinGW-Win32/" \
-    "libusb-$VERSION-binaries/libusb-MinGW-Win32/" \
-    "libusb-$VERSION-binaries/libusb-MinGW-x64/" \
-    "libusb-$VERSION-binaries/libusb-MinGW-x64/"
-
-mv "libusb-$VERSION-binaries/libusb-MinGW-Win32" .
-mv "libusb-$VERSION-binaries/libusb-MinGW-x64" .
-rm -rf "libusb-$VERSION-binaries"
-
-# Rename the dll to get the same library name on all platforms
-mv libusb-MinGW-Win32/bin/msys-usb-1.0.dll libusb-MinGW-Win32/bin/libusb-1.0.dll
-mv libusb-MinGW-x64/bin/msys-usb-1.0.dll libusb-MinGW-x64/bin/libusb-1.0.dll

app/prebuilt-deps/prepare-sdl.sh

@@ -1,34 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-VERSION=2.28.5
-DEP_DIR="SDL2-$VERSION"
-
-FILENAME="SDL2-devel-$VERSION-mingw.tar.gz"
-SHA256SUM=3c0c655c2ebf67cad48fead72761d1601740ded30808952c3274ba223d226c21
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/libsdl-org/SDL/releases/download/release-$VERSION/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-TAR_PREFIX="$DEP_DIR" # root directory inside the tar has the same name
-tar xf "../$FILENAME" --strip-components=1 \
-    "$TAR_PREFIX"/i686-w64-mingw32/bin/SDL2.dll \
-    "$TAR_PREFIX"/i686-w64-mingw32/include/ \
-    "$TAR_PREFIX"/i686-w64-mingw32/lib/ \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/bin/SDL2.dll \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/include/ \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/lib/ \

app/scrcpy-windows.rc

@@ -13,7 +13,7 @@ BEGIN
       VALUE "LegalCopyright", "Romain Vimont, Genymobile"
       VALUE "OriginalFilename", "scrcpy.exe"
       VALUE "ProductName", "scrcpy"
-      VALUE "ProductVersion", "2.3.1"
+      VALUE "ProductVersion", "2.6.1"
     END
   END
   BLOCK "VarFileInfo"

app/scrcpy.1

@@ -29,7 +29,7 @@ Default is 128K (128000).
 .BI "\-\-audio\-buffer " ms
 Configure the audio buffering delay (in milliseconds).
 
-Lower values decrease the latency, but increase the likelyhood of buffer underrun (causing audio glitches).
+Lower values decrease the latency, but increase the likelihood of buffer underrun (causing audio glitches).
 
 Default is 50.
 
@@ -49,6 +49,12 @@ The list of possible codec options is available in the Android documentation:
 
 <https://d.android.com/reference/android/media/MediaFormat>
 
+.TP
+.B \-\-audio\-dup
+Duplicate audio (capture and keep playing on the device).
+
+This feature is only available with --audio-source=playback.
+
 .TP
 .BI "\-\-audio\-encoder " name
 Use a specific MediaCodec audio encoder (depending on the codec provided by \fB\-\-audio\-codec\fR).
@@ -57,7 +63,13 @@ The available encoders can be listed by \fB\-\-list\-encoders\fR.
 
 .TP
 .BI "\-\-audio\-source " source
-Select the audio source (output or mic).
+Select the audio source (output, mic or playback).
+
+The "output" source forwards the whole audio output, and disables playback on the device.
+
+The "playback" source captures the audio playback (Android apps can opt-out, so the whole output is not necessarily captured).
+
+The "mic" source captures the microphone.
 
 Default is output.
 
@@ -124,7 +136,7 @@ Use USB device (if there is exactly one, like adb -d).
 Also see \fB\-e\fR (\fB\-\-select\-tcpip\fR).
 
 .TP
-.BI "\-\-disable-screensaver"
+.BI "\-\-disable\-screensaver"
 Disable screensaver while scrcpy is running.
 
 .TP
@@ -164,32 +176,50 @@ Start in fullscreen.
 Do not attempt to use "adb reverse" to connect to the device.
 
 .TP
-.B \-\-forward\-all\-clicks
+.B \-G
-By default, right-click triggers BACK (or POWER on) and middle-click triggers HOME. This option disables these shortcuts and forward the clicks to the device instead.
+Same as \fB\-\-gamepad=uhid\fR, or \fB\-\-keyboard=aoa\fR if \fB\-\-otg\fR is set.
 
+.TP
+.BI "\-\-gamepad " mode
+Select how to send gamepad inputs to the device.
+
+Possible values are "disabled", "uhid" and "aoa":
+
+ - "disabled" does not send keyboard inputs to the device.
+ - "uhid" simulates a physical HID gamepad using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical HID gamepad using the AOAv2 protocol. It may only work over USB.
+
+Also see \fB\-\-keyboard\f and R\fB\-\-mouse\fR.
 .TP
 .B \-h, \-\-help
 Print this help.
 
 .TP
-.B \-\-kill\-adb\-on\-close
+.B \-K
-Kill adb when scrcpy terminates.
+Same as \fB\-\-keyboard=uhid\fR, or \fB\-\-keyboard=aoa\fR if \fB\-\-otg\fR is set.
 
 .TP
-.B \-K, \-\-hid\-keyboard
+.BI "\-\-keyboard " mode
-Simulate a physical keyboard by using HID over AOAv2.
+Select how to send keyboard inputs to the device.
 
-This provides a better experience for IME users, and allows to generate non-ASCII characters, contrary to the default injection method.
+Possible values are "disabled", "sdk", "uhid" and "aoa":
 
-It may only work over USB.
+ - "disabled" does not send keyboard inputs to the device.
+ - "sdk" uses the Android system API to deliver keyboard events to applications.
+ - "uhid" simulates a physical HID keyboard using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical HID keyboard using the AOAv2 protocol. It may only work over USB.
 
-The keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly:
+For "uhid" and "aoa", the keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly using the shortcut MOD+k (except in OTG mode), or by executing:
 
     adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
 
-However, the option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
+This option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
 
-Also see \fB\-\-hid\-mouse\fR.
+Also see \fB\-\-mouse\fR and \fB\-\-gamepad\fR.
+
+.TP
+.B \-\-kill\-adb\-on\-close
+Kill adb when scrcpy terminates.
 
 .TP
 .B \-\-legacy\-paste
@@ -230,21 +260,52 @@ Limit both the width and height of the video to \fIvalue\fR. The other dimension
 Default is 0 (unlimited).
 
 .TP
-.B \-M, \-\-hid\-mouse
+.B \-M
-Simulate a physical mouse by using HID over AOAv2.
+Same as \fB\-\-mouse=uhid\fR, or \fB\-\-mouse=aoa\fR if \fB\-\-otg\fR is set.
-
-In this mode, the computer mouse is captured to control the device directly (relative mouse mode).
-
-LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
-
-It may only work over USB.
-
-Also see \fB\-\-hid\-keyboard\fR.
 
 .TP
 .BI "\-\-max\-fps " value
 Limit the framerate of screen capture (officially supported since Android 10, but may work on earlier versions).
 
+.TP
+.BI "\-\-mouse " mode
+Select how to send mouse inputs to the device.
+
+Possible values are "disabled", "sdk", "uhid" and "aoa":
+
+ - "disabled" does not send mouse inputs to the device.
+ - "sdk" uses the Android system API to deliver mouse events to applications.
+ - "uhid" simulates a physical HID mouse using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical mouse using the AOAv2 protocol. It may only work over USB.
+
+In "uhid" and "aoa" modes, the computer mouse is captured to control the device directly (relative mouse mode).
+
+LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
+
+Also see \fB\-\-keyboard\fR and \fB\-\-gamepad\fR.
+
+.TP
+.BI "\-\-mouse\-bind " xxxx[:xxxx]
+Configure bindings of secondary clicks.
+
+The argument must be one or two sequences (separated by ':') of exactly 4 characters, one for each secondary click (in order: right click, middle click, 4th click, 5th click).
+
+The first sequence defines the primary bindings, used when a mouse button is pressed alone. The second sequence defines the secondary bindings, used when a mouse button is pressed while the Shift key is held.
+
+If the second sequence of bindings is omitted, then it is the same as the first one.
+
+Each character must be one of the following:
+
+ - '+': forward the click to the device
+ - '-': ignore the click
+ - 'b': trigger shortcut BACK (or turn screen on if off)
+ - 'h': trigger shortcut HOME
+ - 's': trigger shortcut APP_SWITCH
+ - 'n': trigger shortcut "expand notification panel"
+
+Default is 'bhsn:++++' for SDK mouse, and '++++:bhsn' for AOA and UHID.
+
+
 .TP
 .B \-n, \-\-no\-control
 Disable device control (mirror the device in read\-only).
@@ -287,6 +348,10 @@ Do not forward repeated key events when a key is held down.
 .B \-\-no\-mipmaps
 If the renderer is OpenGL 3.0+ or OpenGL ES 2.0+, then mipmaps are automatically generated to improve downscaling quality. This option disables the generation of mipmaps.
 
+.TP
+.B \-\-no\-mouse\-hover
+Do not forward mouse hover (mouse motion without any clicks) events.
+
 .TP
 .B \-\-no\-power\-on
 Do not power on the device on start.
@@ -299,6 +364,10 @@ Disable video forwarding.
 .B \-\-no\-video\-playback
 Disable video playback on the computer.
 
+.TP
+.B \-\-no\-window
+Disable scrcpy window. Implies --no-video-playback and --no-control.
+
 .TP
 .BI "\-\-orientation " value
 Same as --display-orientation=value --record-orientation=value.
@@ -315,7 +384,7 @@ If any of \fB\-\-hid\-keyboard\fR or \fB\-\-hid\-mouse\fR is set, only enable ke
 
 It may only work over USB.
 
-See \fB\-\-hid\-keyboard\fR and \fB\-\-hid\-mouse\fR.
+See \fB\-\-keyboard\fR, \fB\-\-mouse\fR and \fB\-\-gamepad\fR.
 
 .TP
 .BI "\-p, \-\-port " port\fR[:\fIport\fR]
@@ -325,7 +394,7 @@ Default is 27183:27199.
 
 .TP
 \fB\-\-pause\-on\-exit\fR[=\fImode\fR]
-Configure pause on exit. Possible values are "true" (always pause on exit), "false" (never pause on exit) and "if-error" (pause only if an error occured).
+Configure pause on exit. Possible values are "true" (always pause on exit), "false" (never pause on exit) and "if-error" (pause only if an error occurred).
 
 This is useful to prevent the terminal window from automatically closing, so that error messages can be read.
 
@@ -403,9 +472,9 @@ Turn the device screen off immediately.
 .BI "\-\-shortcut\-mod " key\fR[+...]][,...]
 Specify the modifiers to use for scrcpy shortcuts. Possible keys are "lctrl", "rctrl", "lalt", "ralt", "lsuper" and "rsuper".
 
-A shortcut can consist in several keys, separated by '+'. Several shortcuts can be specified, separated by ','.
+Several shortcut modifiers can be specified, separated by ','.
 
-For example, to use either LCtrl+LAlt or LSuper for scrcpy shortcuts, pass "lctrl+lalt,lsuper".
+For example, to use either LCtrl or LSuper for scrcpy shortcuts, pass "lctrl,lsuper".
 
 Default is "lalt,lsuper" (left-Alt or left-Super).
 
@@ -560,6 +629,14 @@ Flip display horizontally
 .B MOD+Shift+Up, MOD+Shift+Down
 Flip display vertically
 
+.TP
+.B MOD+z
+Pause or re-pause display
+
+.TP
+.B MOD+Shift+z
+Unpause display
+
 .TP
 .B MOD+g
 Resize window to 1:1 (pixel\-perfect)
@@ -636,13 +713,21 @@ Copy computer clipboard to device, then paste (inject PASTE keycode, Android >=
 .B MOD+Shift+v
 Inject computer clipboard text as a sequence of key events
 
+.TP
+.B MOD+k
+Open keyboard settings on the device (for HID keyboard only)
+
 .TP
 .B MOD+i
 Enable/disable FPS counter (print frames/second in logs)
 
 .TP
 .B Ctrl+click-and-move
-Pinch-to-zoom from the center of the screen
+Pinch-to-zoom and rotate from the center of the screen
+
+.TP
+.B Shift+click-and-move
+Tilt (slide vertically with two fingers)
 
 .TP
 .B Drag & drop APK file
@@ -689,7 +774,7 @@ Report bugs to <https://github.com/Genymobile/scrcpy/issues>.
 .SH COPYRIGHT
 Copyright \(co 2018 Genymobile <https://www.genymobile.com>
 
-Copyright \(co 2018\-2023 Romain Vimont <rom@rom1v.com>
+Copyright \(co 2018\-2024 Romain Vimont <rom@rom1v.com>
 
 Licensed under the Apache License, Version 2.0.
 

app/src/adb/adb.c

@@ -458,6 +458,7 @@ sc_adb_list_devices(struct sc_intr *intr, unsigned flags,
         // in the buffer in a single pass
         LOGW("Result of \"adb devices -l\" does not fit in 64Kb. "
              "Please report an issue.");
+        free(buf);
         return false;
     }
 

app/src/android/keycodes.h

@@ -633,7 +633,7 @@ enum android_keycode {
      * Toggles between BS and CS digital satellite services. */
     AKEYCODE_TV_SATELLITE_SERVICE = 240,
     /** Toggle Network key.
-     * Toggles selecting broacast services. */
+     * Toggles selecting broadcast services. */
     AKEYCODE_TV_NETWORK      = 241,
     /** Antenna/Cable key.
      * Toggles broadcast input source between antenna and cable. */

app/src/audio_player.c

@@ -66,8 +66,7 @@ static void SDLCALL
 sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
     struct sc_audio_player *ap = userdata;
 
-    // This callback is called with the lock used by SDL_AudioDeviceLock(), so
+    // This callback is called with the lock used by SDL_LockAudioDevice()
-    // the audiobuf is protected
 
     assert(len_int > 0);
     size_t len = len_int;
@@ -77,12 +76,12 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
     LOGD("[Audio] SDL callback requests %" PRIu32 " samples", count);
 #endif
 
-    uint32_t buffered_samples = sc_audiobuf_can_read(&ap->buf);
+    bool played = atomic_load_explicit(&ap->played, memory_order_relaxed);
-    if (!ap->played) {
+    if (!played) {
-        // Part of the buffering is handled by inserting initial silence. The
+        uint32_t buffered_samples = sc_audiobuf_can_read(&ap->buf);
-        // remaining (margin) last samples will be handled by compensation.
+        // Wait until the buffer is filled up to at least target_buffering
-        uint32_t margin = 30 * ap->sample_rate / 1000; // 30ms
+        // before playing
-        if (buffered_samples + margin < ap->target_buffering) {
+        if (buffered_samples < ap->target_buffering) {
             LOGV("[Audio] Inserting initial buffering silence: %" PRIu32
                  " samples", count);
             // Delay playback starting to reach the target buffering. Fill the
@@ -93,10 +92,7 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
         }
     }
 
-    uint32_t read = MIN(buffered_samples, count);
+    uint32_t read = sc_audiobuf_read(&ap->buf, stream, count);
-    if (read) {
-        sc_audiobuf_read(&ap->buf, stream, read);
-    }
 
     if (read < count) {
         uint32_t silence = count - read;
@@ -109,13 +105,16 @@ sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
              silence);
         memset(stream + TO_BYTES(read), 0, TO_BYTES(silence));
 
-        if (ap->received) {
+        bool received = atomic_load_explicit(&ap->received,
+                                             memory_order_relaxed);
+        if (received) {
             // Inserting additional samples immediately increases buffering
-            ap->underflow += silence;
+            atomic_fetch_add_explicit(&ap->underflow, silence,
+                                      memory_order_relaxed);
         }
     }
 
-    ap->played = true;
+    atomic_store_explicit(&ap->played, true, memory_order_relaxed);
 }
 
 static uint8_t *
@@ -162,155 +161,170 @@ sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
 
     // swr_convert() returns the number of samples which would have been
     // written if the buffer was big enough.
-    uint32_t samples_written = MIN(ret, dst_nb_samples);
+    uint32_t samples = MIN(ret, dst_nb_samples);
 #ifndef SC_AUDIO_PLAYER_NDEBUG
-    LOGD("[Audio] %" PRIu32 " samples written to buffer", samples_written);
+    LOGD("[Audio] %" PRIu32 " samples written to buffer", samples);
 #endif
 
-    // Since this function is the only writer, the current available space is
+    uint32_t cap = sc_audiobuf_capacity(&ap->buf);
-    // at least the previous available space. In practice, it should almost
+    if (samples > cap) {
-    // always be possible to write without lock.
+        // Very very unlikely: a single resampled frame should never
-    bool lockless_write = samples_written <= ap->previous_can_write;
+        // exceed the audio buffer size (or something is very wrong).
-    if (lockless_write) {
+        // Ignore the first bytes in swr_buf to avoid memory corruption anyway.
-        sc_audiobuf_prepare_write(&ap->buf, swr_buf, samples_written);
+        swr_buf += TO_BYTES(samples - cap);
+        samples = cap;
     }
 
-    SDL_LockAudioDevice(ap->device);
+    uint32_t skipped_samples = 0;
 
-    uint32_t buffered_samples = sc_audiobuf_can_read(&ap->buf);
+    uint32_t written = sc_audiobuf_write(&ap->buf, swr_buf, samples);
+    if (written < samples) {
+        uint32_t remaining = samples - written;
 
-    if (lockless_write) {
+        // All samples that could be written without locking have been written,
-        sc_audiobuf_commit_write(&ap->buf, samples_written);
+        // now we need to lock to drop/consume old samples
-    } else {
+        SDL_LockAudioDevice(ap->device);
-        uint32_t can_write = sc_audiobuf_can_write(&ap->buf);
-        if (samples_written > can_write) {
-            // Entering this branch is very unlikely, the audio buffer is
-            // allocated with a size sufficient to store 1 second more than the
-            // target buffering. If this happens, though, we have to skip old
-            // samples.
-            uint32_t cap = sc_audiobuf_capacity(&ap->buf);
-            if (samples_written > cap) {
-                // Very very unlikely: a single resampled frame should never
-                // exceed the audio buffer size (or something is very wrong).
-                // Ignore the first bytes in swr_buf
-                swr_buf += TO_BYTES(samples_written - cap);
-                // This change in samples_written will impact the
-                // instant_compensation below
-                samples_written = cap;
-            }
 
-            assert(samples_written >= can_write);
+        // Retry with the lock
-            if (samples_written > can_write) {
+        written += sc_audiobuf_write(&ap->buf,
-                uint32_t skip_samples = samples_written - can_write;
+                                     swr_buf + TO_BYTES(written),
-                assert(buffered_samples >= skip_samples);
+                                     remaining);
-                sc_audiobuf_skip(&ap->buf, skip_samples);
+        if (written < samples) {
-                buffered_samples -= skip_samples;
+            remaining = samples - written;
-                if (ap->played) {
+            // Still insufficient, drop old samples to make space
-                    // Dropping input samples instantly decreases buffering
+            skipped_samples = sc_audiobuf_read(&ap->buf, NULL, remaining);
-                    ap->avg_buffering.avg -= skip_samples;
+            assert(skipped_samples == remaining);
-                }
-            }
-
-            // It should remain exactly the expected size to write the new
-            // samples.
-            assert(sc_audiobuf_can_write(&ap->buf) == samples_written);
         }
 
-        sc_audiobuf_write(&ap->buf, swr_buf, samples_written);
+        SDL_UnlockAudioDevice(ap->device);
+
+        if (written < samples) {
+            // Now there is enough space
+            uint32_t w = sc_audiobuf_write(&ap->buf,
+                                           swr_buf + TO_BYTES(written),
+                                           remaining);
+            assert(w == remaining);
+            (void) w;
+        }
     }
 
-    buffered_samples += samples_written;
+    uint32_t underflow = 0;
-    assert(buffered_samples == sc_audiobuf_can_read(&ap->buf));
+    uint32_t max_buffered_samples;
-
+    bool played = atomic_load_explicit(&ap->played, memory_order_relaxed);
-    // Read with lock held, to be used after unlocking
-    bool played = ap->played;
-    uint32_t underflow = ap->underflow;
-
     if (played) {
-        uint32_t max_buffered_samples = ap->target_buffering
+        underflow = atomic_exchange_explicit(&ap->underflow, 0,
-                                      + 12 * ap->output_buffer
+                                             memory_order_relaxed);
-                                      + ap->target_buffering / 10;
-        if (buffered_samples > max_buffered_samples) {
-            uint32_t skip_samples = buffered_samples - max_buffered_samples;
-            sc_audiobuf_skip(&ap->buf, skip_samples);
-            LOGD("[Audio] Buffering threshold exceeded, skipping %" PRIu32
-                 " samples", skip_samples);
-        }
 
-        // reset (the current value was copied to a local variable)
+        max_buffered_samples = ap->target_buffering
-        ap->underflow = 0;
+                               + 12 * ap->output_buffer
+                               + ap->target_buffering / 10;
     } else {
         // SDL playback not started yet, do not accumulate more than
         // max_initial_buffering samples, this would cause unnecessary delay
         // (and glitches to compensate) on start.
-        uint32_t max_initial_buffering = ap->target_buffering
+        max_buffered_samples = ap->target_buffering + 2 * ap->output_buffer;
-                                       + 2 * ap->output_buffer;
+    }
-        if (buffered_samples > max_initial_buffering) {
+
-            uint32_t skip_samples = buffered_samples - max_initial_buffering;
+    uint32_t can_read = sc_audiobuf_can_read(&ap->buf);
-            sc_audiobuf_skip(&ap->buf, skip_samples);
+    if (can_read > max_buffered_samples) {
+        uint32_t skip_samples = 0;
+
+        SDL_LockAudioDevice(ap->device);
+        can_read = sc_audiobuf_can_read(&ap->buf);
+        if (can_read > max_buffered_samples) {
+            skip_samples = can_read - max_buffered_samples;
+            uint32_t r = sc_audiobuf_read(&ap->buf, NULL, skip_samples);
+            assert(r == skip_samples);
+            (void) r;
+            skipped_samples += skip_samples;
+        }
+        SDL_UnlockAudioDevice(ap->device);
+
+        if (skip_samples) {
+            if (played) {
+                LOGD("[Audio] Buffering threshold exceeded, skipping %" PRIu32
+                     " samples", skip_samples);
 #ifndef SC_AUDIO_PLAYER_NDEBUG
-            LOGD("[Audio] Playback not started, skipping %" PRIu32 " samples",
+            } else {
-                 skip_samples);
+                LOGD("[Audio] Playback not started, skipping %" PRIu32
+                     " samples", skip_samples);
 #endif
+            }
         }
     }
 
-    ap->previous_can_write = sc_audiobuf_can_write(&ap->buf);
+    atomic_store_explicit(&ap->received, true, memory_order_relaxed);
-    ap->received = true;
+    if (!played) {
+        // Nothing more to do
+        return true;
+    }
 
-    SDL_UnlockAudioDevice(ap->device);
+    // Number of samples added (or removed, if negative) for compensation
+    int32_t instant_compensation = (int32_t) written - frame->nb_samples;
+    // Inserting silence instantly increases buffering
+    int32_t inserted_silence = (int32_t) underflow;
+    // Dropping input samples instantly decreases buffering
+    int32_t dropped = (int32_t) skipped_samples;
 
-    if (played) {
+    // The compensation must apply instantly, it must not be smoothed
-        // Number of samples added (or removed, if negative) for compensation
+    ap->avg_buffering.avg += instant_compensation + inserted_silence - dropped;
-        int32_t instant_compensation =
+    if (ap->avg_buffering.avg < 0) {
-            (int32_t) samples_written - frame->nb_samples;
+        // Since dropping samples instantly reduces buffering, the difference
-        int32_t inserted_silence = (int32_t) underflow;
+        // is applied immediately to the average value, assuming that the delay
+        // between the producer and the consumer will be caught up.
+        //
+        // However, when this assumption is not valid, the average buffering
+        // may decrease indefinitely. Prevent it to become negative to limit
+        // the consequences.
+        ap->avg_buffering.avg = 0;
+    }
 
-        // The compensation must apply instantly, it must not be smoothed
+    // However, the buffering level must be smoothed
-        ap->avg_buffering.avg += instant_compensation + inserted_silence;
+    sc_average_push(&ap->avg_buffering, can_read);
-
-
-        // However, the buffering level must be smoothed
-        sc_average_push(&ap->avg_buffering, buffered_samples);
 
 #ifndef SC_AUDIO_PLAYER_NDEBUG
-        LOGD("[Audio] buffered_samples=%" PRIu32 " avg_buffering=%f",
+    LOGD("[Audio] can_read=%" PRIu32 " avg_buffering=%f",
-             buffered_samples, sc_average_get(&ap->avg_buffering));
+         can_read, sc_average_get(&ap->avg_buffering));
 #endif
 
-        ap->samples_since_resync += samples_written;
+    ap->samples_since_resync += written;
-        if (ap->samples_since_resync >= ap->sample_rate) {
+    if (ap->samples_since_resync >= ap->sample_rate) {
-            // Recompute compensation every second
+        // Recompute compensation every second
-            ap->samples_since_resync = 0;
+        ap->samples_since_resync = 0;
 
-            float avg = sc_average_get(&ap->avg_buffering);
+        float avg = sc_average_get(&ap->avg_buffering);
-            int diff = ap->target_buffering - avg;
+        int diff = ap->target_buffering - avg;
-            if (abs(diff) < (int) ap->sample_rate / 1000) {
-                // Do not compensate for less than 1ms, the error is just noise
-                diff = 0;
-            } else if (diff < 0 && buffered_samples < ap->target_buffering) {
-                // Do not accelerate if the instant buffering level is below
-                // the average, this would increase underflow
-                diff = 0;
-            }
-            // Compensate the diff over 4 seconds (but will be recomputed after
-            // 1 second)
-            int distance = 4 * ap->sample_rate;
-            // Limit compensation rate to 2%
-            int abs_max_diff = distance / 50;
-            diff = CLAMP(diff, -abs_max_diff, abs_max_diff);
-            LOGV("[Audio] Buffering: target=%" PRIu32 " avg=%f cur=%" PRIu32
-                 " compensation=%d", ap->target_buffering, avg,
-                 buffered_samples, diff);
 
-            if (diff != ap->compensation) {
+        // Enable compensation when the difference exceeds +/- 4ms.
-                int ret = swr_set_compensation(swr_ctx, diff, distance);
+        // Disable compensation when the difference is lower than +/- 1ms.
-                if (ret < 0) {
+        int threshold = ap->compensation != 0
-                    LOGW("Resampling compensation failed: %d", ret);
+                      ? ap->sample_rate     / 1000  /* 1ms */
-                    // not fatal
+                      : ap->sample_rate * 4 / 1000; /* 4ms */
-                } else {
+
-                    ap->compensation = diff;
+        if (abs(diff) < threshold) {
-                }
+            // Do not compensate for small values, the error is just noise
+            diff = 0;
+        } else if (diff < 0 && can_read < ap->target_buffering) {
+            // Do not accelerate if the instant buffering level is below the
+            // target, this would increase underflow
+            diff = 0;
+        }
+        // Compensate the diff over 4 seconds (but will be recomputed after 1
+        // second)
+        int distance = 4 * ap->sample_rate;
+        // Limit compensation rate to 2%
+        int abs_max_diff = distance / 50;
+        diff = CLAMP(diff, -abs_max_diff, abs_max_diff);
+        LOGV("[Audio] Buffering: target=%" PRIu32 " avg=%f cur=%" PRIu32
+             " compensation=%d", ap->target_buffering, avg, can_read, diff);
+
+        if (diff != ap->compensation) {
+            int ret = swr_set_compensation(swr_ctx, diff, distance);
+            if (ret < 0) {
+                LOGW("Resampling compensation failed: %d", ret);
+                // not fatal
+            } else {
+                ap->compensation = diff;
             }
         }
     }
@@ -397,7 +411,7 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
     // producer and the consumer. It's too big on purpose, to guarantee that
     // the producer and the consumer will be able to access it in parallel
     // without locking.
-    size_t audiobuf_samples = ap->target_buffering + ap->sample_rate;
+    uint32_t audiobuf_samples = ap->target_buffering + ap->sample_rate;
 
     size_t sample_size = ap->nb_channels * ap->out_bytes_per_sample;
     bool ok = sc_audiobuf_init(&ap->buf, sample_size, audiobuf_samples);
@@ -413,16 +427,15 @@ sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
     }
     ap->swr_buf_alloc_size = initial_swr_buf_size;
 
-    ap->previous_can_write = sc_audiobuf_can_write(&ap->buf);
-
     // Samples are produced and consumed by blocks, so the buffering must be
     // smoothed to get a relatively stable value.
-    sc_average_init(&ap->avg_buffering, 32);
+    sc_average_init(&ap->avg_buffering, 128);
     ap->samples_since_resync = 0;
 
     ap->received = false;
-    ap->played = false;
+    atomic_init(&ap->played, false);
-    ap->underflow = 0;
+    atomic_init(&ap->received, false);
+    atomic_init(&ap->underflow, 0);
     ap->compensation = 0;
 
     // The thread calling open() is the thread calling push(), which fills the

app/src/audio_player.h

@@ -3,17 +3,18 @@
 
 #include "common.h"
 
+#include <stdatomic.h>
 #include <stdbool.h>
-#include "trait/frame_sink.h"
-#include <util/audiobuf.h>
-#include <util/average.h>
-#include <util/thread.h>
-#include <util/tick.h>
-
 #include <libavformat/avformat.h>
 #include <libswresample/swresample.h>
 #include <SDL2/SDL.h>
 
+#include "trait/frame_sink.h"
+#include "util/audiobuf.h"
+#include "util/average.h"
+#include "util/thread.h"
+#include "util/tick.h"
+
 struct sc_audio_player {
     struct sc_frame_sink frame_sink;
 
@@ -32,13 +33,9 @@ struct sc_audio_player {
     uint16_t output_buffer;
 
     // Audio buffer to communicate between the receiver and the SDL audio
-    // callback (protected by SDL_AudioDeviceLock())
+    // callback
     struct sc_audiobuf buf;
 
-    // The previous empty space in the buffer (only used by the receiver
-    // thread)
-    uint32_t previous_can_write;
-
     // Resampler (only used from the receiver thread)
     struct SwrContext *swr_ctx;
 
@@ -47,7 +44,7 @@ struct sc_audio_player {
     // The number of channels is the same for input and output
     unsigned nb_channels;
     // The number of bytes per sample for a single channel
-    unsigned out_bytes_per_sample;
+    size_t out_bytes_per_sample;
 
     // Target buffer for resampling (only used by the receiver thread)
     uint8_t *swr_buf;
@@ -61,19 +58,16 @@ struct sc_audio_player {
     uint32_t samples_since_resync;
 
     // Number of silence samples inserted since the last received packet
-    // (protected by SDL_AudioDeviceLock())
+    atomic_uint_least32_t underflow;
-    uint32_t underflow;
 
     // Current applied compensation value (only used by the receiver thread)
     int compensation;
 
-    // Set to true the first time a sample is received (protected by
+    // Set to true the first time a sample is received
-    // SDL_AudioDeviceLock())
+    atomic_bool received;
-    bool received;
 
-    // Set to true the first time the SDL callback is called (protected by
+    // Set to true the first time the SDL callback is called
-    // SDL_AudioDeviceLock())
+    atomic_bool played;
-    bool played;
 
     const struct sc_audio_player_callbacks *cbs;
     void *cbs_userdata;

app/src/cli.h

@@ -28,7 +28,7 @@ scrcpy_parse_args(struct scrcpy_cli_args *args, int argc, char *argv[]);
 
 #ifdef SC_TEST
 bool
-sc_parse_shortcut_mods(const char *s, struct sc_shortcut_mods *mods);
+sc_parse_shortcut_mods(const char *s, uint8_t *shortcut_mods);
 #endif
 
 #endif

app/src/control_msg.c

@@ -64,13 +64,11 @@ static const char *const copy_key_labels[] = {
 static inline const char *
 get_well_known_pointer_id_name(uint64_t pointer_id) {
     switch (pointer_id) {
-        case POINTER_ID_MOUSE:
+        case SC_POINTER_ID_MOUSE:
             return "mouse";
-        case POINTER_ID_GENERIC_FINGER:
+        case SC_POINTER_ID_GENERIC_FINGER:
             return "finger";
-        case POINTER_ID_VIRTUAL_MOUSE:
+        case SC_POINTER_ID_VIRTUAL_FINGER:
-            return "vmouse";
-        case POINTER_ID_VIRTUAL_FINGER:
             return "vfinger";
         default:
             return NULL;
@@ -85,17 +83,36 @@ write_position(uint8_t *buf, const struct sc_position *position) {
     sc_write16be(&buf[10], position->screen_size.height);
 }
 
-// write length (4 bytes) + string (non null-terminated)
+// Write truncated string, and return the size
 static size_t
-write_string(const char *utf8, size_t max_len, unsigned char *buf) {
+write_string_payload(uint8_t *payload, const char *utf8, size_t max_len) {
+    if (!utf8) {
+        return 0;
+    }
     size_t len = sc_str_utf8_truncation_index(utf8, max_len);
+    memcpy(payload, utf8, len);
+    return len;
+}
+
+// Write length (4 bytes) + string (non null-terminated)
+static size_t
+write_string(uint8_t *buf, const char *utf8, size_t max_len) {
+    size_t len = write_string_payload(buf + 4, utf8, max_len);
     sc_write32be(buf, len);
-    memcpy(&buf[4], utf8, len);
     return 4 + len;
 }
 
+// Write length (1 byte) + string (non null-terminated)
+static size_t
+write_string_tiny(uint8_t *buf, const char *utf8, size_t max_len) {
+    assert(max_len <= 0xFF);
+    size_t len = write_string_payload(buf + 1, utf8, max_len);
+    buf[0] = len;
+    return 1 + len;
+}
+
 size_t
-sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf) {
+sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf) {
     buf[0] = msg->type;
     switch (msg->type) {
         case SC_CONTROL_MSG_TYPE_INJECT_KEYCODE:
@@ -105,9 +122,8 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf) {
             sc_write32be(&buf[10], msg->inject_keycode.metastate);
             return 14;
         case SC_CONTROL_MSG_TYPE_INJECT_TEXT: {
-            size_t len =
+            size_t len = write_string(&buf[1], msg->inject_text.text,
-                write_string(msg->inject_text.text,
+                                      SC_CONTROL_MSG_INJECT_TEXT_MAX_LENGTH);
-                             SC_CONTROL_MSG_INJECT_TEXT_MAX_LENGTH, &buf[1]);
             return 1 + len;
         }
         case SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT:
@@ -139,17 +155,39 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf) {
         case SC_CONTROL_MSG_TYPE_SET_CLIPBOARD:
             sc_write64be(&buf[1], msg->set_clipboard.sequence);
             buf[9] = !!msg->set_clipboard.paste;
-            size_t len = write_string(msg->set_clipboard.text,
+            size_t len = write_string(&buf[10], msg->set_clipboard.text,
-                                      SC_CONTROL_MSG_CLIPBOARD_TEXT_MAX_LENGTH,
+                                      SC_CONTROL_MSG_CLIPBOARD_TEXT_MAX_LENGTH);
-                                      &buf[10]);
             return 10 + len;
         case SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE:
             buf[1] = msg->set_screen_power_mode.mode;
             return 2;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE:
+            sc_write16be(&buf[1], msg->uhid_create.id);
+
+            size_t index = 3;
+            index += write_string_tiny(&buf[index], msg->uhid_create.name, 127);
+
+            sc_write16be(&buf[index], msg->uhid_create.report_desc_size);
+            index += 2;
+
+            memcpy(&buf[index], msg->uhid_create.report_desc,
+                                msg->uhid_create.report_desc_size);
+            index += msg->uhid_create.report_desc_size;
+
+            return index;
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT:
+            sc_write16be(&buf[1], msg->uhid_input.id);
+            sc_write16be(&buf[3], msg->uhid_input.size);
+            memcpy(&buf[5], msg->uhid_input.data, msg->uhid_input.size);
+            return 5 + msg->uhid_input.size;
+        case SC_CONTROL_MSG_TYPE_UHID_DESTROY:
+            sc_write16be(&buf[1], msg->uhid_destroy.id);
+            return 3;
         case SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL:
         case SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL:
         case SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS:
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
             // no additional data
             return 1;
         default:
@@ -242,12 +280,50 @@ sc_control_msg_log(const struct sc_control_msg *msg) {
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
             LOG_CMSG("rotate device");
             break;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE: {
+            // Quote only if name is not null
+            const char *name = msg->uhid_create.name;
+            const char *quote = name ? "\"" : "";
+            LOG_CMSG("UHID create [%" PRIu16 "] name=%s%s%s "
+                     "report_desc_size=%" PRIu16, msg->uhid_create.id,
+                     quote, name, quote, msg->uhid_create.report_desc_size);
+            break;
+        }
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT: {
+            char *hex = sc_str_to_hex_string(msg->uhid_input.data,
+                                             msg->uhid_input.size);
+            if (hex) {
+                LOG_CMSG("UHID input [%" PRIu16 "] %s",
+                         msg->uhid_input.id, hex);
+                free(hex);
+            } else {
+                LOG_CMSG("UHID input [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_input.id, msg->uhid_input.size);
+            }
+            break;
+        }
+        case SC_CONTROL_MSG_TYPE_UHID_DESTROY:
+            LOG_CMSG("UHID destroy [%" PRIu16 "]", msg->uhid_destroy.id);
+            break;
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
+            LOG_CMSG("open hard keyboard settings");
+            break;
         default:
             LOG_CMSG("unknown type: %u", (unsigned) msg->type);
             break;
     }
 }
 
+bool
+sc_control_msg_is_droppable(const struct sc_control_msg *msg) {
+    // Cannot drop UHID_CREATE messages, because it would cause all further
+    // UHID_INPUT messages for this device to be invalid.
+    // Cannot drop UHID_DESTROY messages either, because a further UHID_CREATE
+    // with the same id may fail.
+    return msg->type != SC_CONTROL_MSG_TYPE_UHID_CREATE
+        && msg->type != SC_CONTROL_MSG_TYPE_UHID_DESTROY;
+}
+
 void
 sc_control_msg_destroy(struct sc_control_msg *msg) {
     switch (msg->type) {

app/src/control_msg.h

@@ -10,6 +10,7 @@
 #include "android/input.h"
 #include "android/keycodes.h"
 #include "coords.h"
+#include "hid/hid_event.h"
 
 #define SC_CONTROL_MSG_MAX_SIZE (1 << 18) // 256k
 
@@ -17,12 +18,11 @@
 // type: 1 byte; sequence: 8 bytes; paste flag: 1 byte; length: 4 bytes
 #define SC_CONTROL_MSG_CLIPBOARD_TEXT_MAX_LENGTH (SC_CONTROL_MSG_MAX_SIZE - 14)
 
-#define POINTER_ID_MOUSE UINT64_C(-1)
+#define SC_POINTER_ID_MOUSE UINT64_C(-1)
-#define POINTER_ID_GENERIC_FINGER UINT64_C(-2)
+#define SC_POINTER_ID_GENERIC_FINGER UINT64_C(-2)
 
 // Used for injecting an additional virtual pointer for pinch-to-zoom
-#define POINTER_ID_VIRTUAL_MOUSE UINT64_C(-3)
+#define SC_POINTER_ID_VIRTUAL_FINGER UINT64_C(-3)
-#define POINTER_ID_VIRTUAL_FINGER UINT64_C(-4)
 
 enum sc_control_msg_type {
     SC_CONTROL_MSG_TYPE_INJECT_KEYCODE,
@@ -37,6 +37,10 @@ enum sc_control_msg_type {
     SC_CONTROL_MSG_TYPE_SET_CLIPBOARD,
     SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE,
     SC_CONTROL_MSG_TYPE_ROTATE_DEVICE,
+    SC_CONTROL_MSG_TYPE_UHID_CREATE,
+    SC_CONTROL_MSG_TYPE_UHID_INPUT,
+    SC_CONTROL_MSG_TYPE_UHID_DESTROY,
+    SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
 };
 
 enum sc_screen_power_mode {
@@ -92,17 +96,36 @@ struct sc_control_msg {
         struct {
             enum sc_screen_power_mode mode;
         } set_screen_power_mode;
+        struct {
+            uint16_t id;
+            const char *name; // pointer to static data
+            uint16_t report_desc_size;
+            const uint8_t *report_desc; // pointer to static data
+        } uhid_create;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t data[SC_HID_MAX_SIZE];
+        } uhid_input;
+        struct {
+            uint16_t id;
+        } uhid_destroy;
     };
 };
 
 // buf size must be at least CONTROL_MSG_MAX_SIZE
 // return the number of bytes written
 size_t
-sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf);
+sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf);
 
 void
 sc_control_msg_log(const struct sc_control_msg *msg);
 
+// Even when the buffer is "full", some messages must absolutely not be dropped
+// to avoid inconsistencies.
+bool
+sc_control_msg_is_droppable(const struct sc_control_msg *msg);
+
 void
 sc_control_msg_destroy(struct sc_control_msg *msg);
 

app/src/controller.c

@@ -4,19 +4,38 @@
 
 #include "util/log.h"
 
-#define SC_CONTROL_MSG_QUEUE_MAX 64
+// Drop droppable events above this limit
+#define SC_CONTROL_MSG_QUEUE_LIMIT 60
+
+static void
+sc_controller_receiver_on_ended(struct sc_receiver *receiver, bool error,
+                                void *userdata) {
+    (void) receiver;
+
+    struct sc_controller *controller = userdata;
+    // Forward the event to the controller listener
+    controller->cbs->on_ended(controller, error, controller->cbs_userdata);
+}
 
 bool
 sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync) {
+                   const struct sc_controller_callbacks *cbs,
+                   void *cbs_userdata) {
     sc_vecdeque_init(&controller->queue);
 
-    bool ok = sc_vecdeque_reserve(&controller->queue, SC_CONTROL_MSG_QUEUE_MAX);
+    // Add 4 to support 4 non-droppable events without re-allocation
+    bool ok = sc_vecdeque_reserve(&controller->queue,
+                                  SC_CONTROL_MSG_QUEUE_LIMIT + 4);
     if (!ok) {
         return false;
     }
 
-    ok = sc_receiver_init(&controller->receiver, control_socket, acksync);
+    static const struct sc_receiver_callbacks receiver_cbs = {
+        .on_ended = sc_controller_receiver_on_ended,
+    };
+
+    ok = sc_receiver_init(&controller->receiver, control_socket, &receiver_cbs,
+                          controller);
     if (!ok) {
         sc_vecdeque_destroy(&controller->queue);
         return false;
@@ -40,9 +59,21 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
     controller->control_socket = control_socket;
     controller->stopped = false;
 
+    assert(cbs && cbs->on_ended);
+    controller->cbs = cbs;
+    controller->cbs_userdata = cbs_userdata;
+
     return true;
 }
 
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices) {
+    controller->receiver.acksync = acksync;
+    controller->receiver.uhid_devices = uhid_devices;
+}
+
 void
 sc_controller_destroy(struct sc_controller *controller) {
     sc_cond_destroy(&controller->msg_cond);
@@ -65,39 +96,59 @@ sc_controller_push_msg(struct sc_controller *controller,
         sc_control_msg_log(msg);
     }
 
+    bool pushed = false;
+
     sc_mutex_lock(&controller->mutex);
-    bool full = sc_vecdeque_is_full(&controller->queue);
+    size_t size = sc_vecdeque_size(&controller->queue);
-    if (!full) {
+    if (size < SC_CONTROL_MSG_QUEUE_LIMIT) {
         bool was_empty = sc_vecdeque_is_empty(&controller->queue);
         sc_vecdeque_push_noresize(&controller->queue, *msg);
+        pushed = true;
         if (was_empty) {
             sc_cond_signal(&controller->msg_cond);
         }
+    } else if (!sc_control_msg_is_droppable(msg)) {
+        bool ok = sc_vecdeque_push(&controller->queue, *msg);
+        if (ok) {
+            pushed = true;
+        } else {
+            // A non-droppable event must be dropped anyway
+            LOG_OOM();
+        }
     }
     // Otherwise (if the queue is full), the msg is discarded
 
     sc_mutex_unlock(&controller->mutex);
 
-    return !full;
+    return pushed;
 }
 
 static bool
 process_msg(struct sc_controller *controller,
-            const struct sc_control_msg *msg) {
+            const struct sc_control_msg *msg, bool *eos) {
-    static unsigned char serialized_msg[SC_CONTROL_MSG_MAX_SIZE];
+    static uint8_t serialized_msg[SC_CONTROL_MSG_MAX_SIZE];
     size_t length = sc_control_msg_serialize(msg, serialized_msg);
     if (!length) {
+        *eos = false;
         return false;
     }
+
     ssize_t w =
         net_send_all(controller->control_socket, serialized_msg, length);
-    return (size_t) w == length;
+    if ((size_t) w != length) {
+        *eos = true;
+        return false;
+    }
+
+    return true;
 }
 
 static int
 run_controller(void *data) {
     struct sc_controller *controller = data;
 
+    bool error = false;
+
     for (;;) {
         sc_mutex_lock(&controller->mutex);
         while (!controller->stopped
@@ -107,6 +158,7 @@ run_controller(void *data) {
         if (controller->stopped) {
             // stop immediately, do not process further msgs
             sc_mutex_unlock(&controller->mutex);
+            LOGD("Controller stopped");
             break;
         }
 
@@ -114,13 +166,20 @@ run_controller(void *data) {
         struct sc_control_msg msg = sc_vecdeque_pop(&controller->queue);
         sc_mutex_unlock(&controller->mutex);
 
-        bool ok = process_msg(controller, &msg);
+        bool eos;
+        bool ok = process_msg(controller, &msg, &eos);
         sc_control_msg_destroy(&msg);
         if (!ok) {
-            LOGD("Could not write msg to socket");
+            if (eos) {
+                LOGD("Controller stopped (socket closed)");
+            } // else error already logged
+            error = !eos;
             break;
         }
     }
+
+    controller->cbs->on_ended(controller, error, controller->cbs_userdata);
+
     return 0;
 }
 

app/src/controller.h

@@ -22,11 +22,25 @@ struct sc_controller {
     bool stopped;
     struct sc_control_msg_queue queue;
     struct sc_receiver receiver;
+
+    const struct sc_controller_callbacks *cbs;
+    void *cbs_userdata;
+};
+
+struct sc_controller_callbacks {
+    void (*on_ended)(struct sc_controller *controller, bool error,
+                     void *userdata);
 };
 
 bool
 sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync);
+                   const struct sc_controller_callbacks *cbs,
+                   void *cbs_userdata);
+
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices);
 
 void
 sc_controller_destroy(struct sc_controller *controller);

app/src/demuxer.c

@@ -278,7 +278,6 @@ run_demuxer(void *data) {
 finally_close_sinks:
     sc_packet_source_sinks_close(&demuxer->packet_source);
 finally_free_context:
-    // This also calls avcodec_close() internally
     avcodec_free_context(&codec_ctx);
 end:
     demuxer->cbs->on_ended(demuxer, status, demuxer->cbs_userdata);

app/src/device_msg.c

@@ -8,19 +8,22 @@
 #include "util/log.h"
 
 ssize_t
-device_msg_deserialize(const unsigned char *buf, size_t len,
+sc_device_msg_deserialize(const uint8_t *buf, size_t len,
-                       struct device_msg *msg) {
+                          struct sc_device_msg *msg) {
-    if (len < 5) {
+    if (!len) {
-        // at least type + empty string length
+        return 0; // no message
-        return 0; // not available
     }
 
     msg->type = buf[0];
     switch (msg->type) {
         case DEVICE_MSG_TYPE_CLIPBOARD: {
+            if (len < 5) {
+                // at least type + empty string length
+                return 0; // no complete message
+            }
             size_t clipboard_len = sc_read32be(&buf[1]);
             if (clipboard_len > len - 5) {
-                return 0; // not available
+                return 0; // no complete message
             }
             char *text = malloc(clipboard_len + 1);
             if (!text) {
@@ -36,10 +39,38 @@ device_msg_deserialize(const unsigned char *buf, size_t len,
             return 5 + clipboard_len;
         }
         case DEVICE_MSG_TYPE_ACK_CLIPBOARD: {
+            if (len < 9) {
+                return 0; // no complete message
+            }
             uint64_t sequence = sc_read64be(&buf[1]);
             msg->ack_clipboard.sequence = sequence;
             return 9;
         }
+        case DEVICE_MSG_TYPE_UHID_OUTPUT: {
+            if (len < 5) {
+                // at least id + size
+                return 0; // not available
+            }
+            uint16_t id = sc_read16be(&buf[1]);
+            size_t size = sc_read16be(&buf[3]);
+            if (size < len - 5) {
+                return 0; // not available
+            }
+            uint8_t *data = malloc(size);
+            if (!data) {
+                LOG_OOM();
+                return -1;
+            }
+            if (size) {
+                memcpy(data, &buf[5], size);
+            }
+
+            msg->uhid_output.id = id;
+            msg->uhid_output.size = size;
+            msg->uhid_output.data = data;
+
+            return 5 + size;
+        }
         default:
             LOGW("Unknown device message type: %d", (int) msg->type);
             return -1; // error, we cannot recover
@@ -47,8 +78,16 @@ device_msg_deserialize(const unsigned char *buf, size_t len,
 }
 
 void
-device_msg_destroy(struct device_msg *msg) {
+sc_device_msg_destroy(struct sc_device_msg *msg) {
-    if (msg->type == DEVICE_MSG_TYPE_CLIPBOARD) {
+    switch (msg->type) {
-        free(msg->clipboard.text);
+        case DEVICE_MSG_TYPE_CLIPBOARD:
+            free(msg->clipboard.text);
+            break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            free(msg->uhid_output.data);
+            break;
+        default:
+            // nothing to do
+            break;
     }
 }

app/src/device_msg.h

@@ -11,13 +11,14 @@
 // type: 1 byte; length: 4 bytes
 #define DEVICE_MSG_TEXT_MAX_LENGTH (DEVICE_MSG_MAX_SIZE - 5)
 
-enum device_msg_type {
+enum sc_device_msg_type {
     DEVICE_MSG_TYPE_CLIPBOARD,
     DEVICE_MSG_TYPE_ACK_CLIPBOARD,
+    DEVICE_MSG_TYPE_UHID_OUTPUT,
 };
 
-struct device_msg {
+struct sc_device_msg {
-    enum device_msg_type type;
+    enum sc_device_msg_type type;
     union {
         struct {
             char *text; // owned, to be freed by free()
@@ -25,15 +26,20 @@ struct device_msg {
         struct {
             uint64_t sequence;
         } ack_clipboard;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t *data; // owned, to be freed by free()
+        } uhid_output;
     };
 };
 
 // return the number of bytes consumed (0 for no msg available, -1 on error)
 ssize_t
-device_msg_deserialize(const unsigned char *buf, size_t len,
+sc_device_msg_deserialize(const uint8_t *buf, size_t len,
-                       struct device_msg *msg);
+                          struct sc_device_msg *msg);
 
 void
-device_msg_destroy(struct device_msg *msg);
+sc_device_msg_destroy(struct sc_device_msg *msg);
 
 #endif

app/src/display.c

@@ -1,11 +1,34 @@
 #include "display.h"
 
 #include <assert.h>
+#include <libavutil/pixfmt.h>
 
 #include "util/log.h"
 
+static bool
+sc_display_init_novideo_icon(struct sc_display *display,
+                             SDL_Surface *icon_novideo) {
+    assert(icon_novideo);
+
+    if (SDL_RenderSetLogicalSize(display->renderer,
+                                 icon_novideo->w, icon_novideo->h)) {
+        LOGW("Could not set renderer logical size: %s", SDL_GetError());
+        // don't fail
+    }
+
+    display->texture = SDL_CreateTextureFromSurface(display->renderer,
+                                                    icon_novideo);
+    if (!display->texture) {
+        LOGE("Could not create texture: %s", SDL_GetError());
+        return false;
+    }
+
+    return true;
+}
+
 bool
-sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
+sc_display_init(struct sc_display *display, SDL_Window *window,
+                SDL_Surface *icon_novideo, bool mipmaps) {
     display->renderer =
         SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
     if (!display->renderer) {
@@ -20,6 +43,10 @@ sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
 
     display->mipmaps = false;
 
+#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
+    display->gl_context = NULL;
+#endif
+
     // starts with "opengl"
     bool use_opengl = renderer_name && !strncmp(renderer_name, "opengl", 6);
     if (use_opengl) {
@@ -59,11 +86,25 @@ sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
             LOGI("Trilinear filtering disabled");
         }
     } else if (mipmaps) {
-        LOGD("Trilinear filtering disabled (not an OpenGL renderer");
+        LOGD("Trilinear filtering disabled (not an OpenGL renderer)");
     }
 
+    display->texture = NULL;
     display->pending.flags = 0;
     display->pending.frame = NULL;
+    display->has_frame = false;
+
+    if (icon_novideo) {
+        // Without video, set a static scrcpy icon as window content
+        bool ok = sc_display_init_novideo_icon(display, icon_novideo);
+        if (!ok) {
+#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
+            SDL_GL_DeleteContext(display->gl_context);
+#endif
+            SDL_DestroyRenderer(display->renderer);
+            return false;
+        }
+    }
 
     return true;
 }
@@ -195,9 +236,25 @@ sc_display_set_texture_size(struct sc_display *display, struct sc_size size) {
     return SC_DISPLAY_RESULT_OK;
 }
 
+static SDL_YUV_CONVERSION_MODE
+sc_display_to_sdl_color_range(enum AVColorRange color_range) {
+    return color_range == AVCOL_RANGE_JPEG ? SDL_YUV_CONVERSION_JPEG
+                                           : SDL_YUV_CONVERSION_AUTOMATIC;
+}
+
 static bool
 sc_display_update_texture_internal(struct sc_display *display,
                                    const AVFrame *frame) {
+    if (!display->has_frame) {
+        // First frame
+        display->has_frame = true;
+
+        // Configure YUV color range conversion
+        SDL_YUV_CONVERSION_MODE sdl_color_range =
+            sc_display_to_sdl_color_range(frame->color_range);
+        SDL_SetYUVConversionMode(sdl_color_range);
+    }
+
     int ret = SDL_UpdateYUVTexture(display->texture, NULL,
                                    frame->data[0], frame->linesize[0],
                                    frame->data[1], frame->linesize[1],

app/src/display.h

@@ -33,6 +33,8 @@ struct sc_display {
         struct sc_size size;
         AVFrame *frame;
     } pending;
+
+    bool has_frame;
 };
 
 enum sc_display_result {
@@ -42,7 +44,8 @@ enum sc_display_result {
 };
 
 bool
-sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps);
+sc_display_init(struct sc_display *display, SDL_Window *window,
+                SDL_Surface *icon_novideo, bool mipmaps);
 
 void
 sc_display_destroy(struct sc_display *display);

app/src/events.c

@@ -0,0 +1,66 @@
+#include "events.h"
+
+#include "util/log.h"
+#include "util/thread.h"
+
+bool
+sc_push_event_impl(uint32_t type, const char *name) {
+    SDL_Event event;
+    event.type = type;
+    int ret = SDL_PushEvent(&event);
+    // ret < 0: error (queue full)
+    // ret == 0: event was filtered
+    // ret == 1: success
+    if (ret != 1) {
+        LOGE("Could not post %s event: %s", name, SDL_GetError());
+        return false;
+    }
+
+    return true;
+}
+
+bool
+sc_post_to_main_thread(sc_runnable_fn run, void *userdata) {
+    SDL_Event event = {
+        .user = {
+            .type = SC_EVENT_RUN_ON_MAIN_THREAD,
+            .data1 = run,
+            .data2 = userdata,
+        },
+    };
+    int ret = SDL_PushEvent(&event);
+    // ret < 0: error (queue full)
+    // ret == 0: event was filtered
+    // ret == 1: success
+    if (ret != 1) {
+        if (ret == 0) {
+            // if ret == 0, this is expected on exit, log in debug mode
+            LOGD("Could not post runnable to main thread (filtered)");
+        } else {
+            assert(ret < 0);
+            LOGW("Coud not post to main thread: %s", SDL_GetError());
+        }
+        return false;
+    }
+
+    return true;
+}
+
+static int SDLCALL
+task_event_filter(void *userdata, SDL_Event *event) {
+    (void) userdata;
+
+    if (event->type == SC_EVENT_RUN_ON_MAIN_THREAD) {
+        // Reject this event type from now on
+        return 0;
+    }
+
+    return 1;
+}
+
+void
+sc_reject_new_runnables(void) {
+    assert(sc_thread_get_id() == SC_MAIN_THREAD_ID);
+
+    SDL_SetEventFilter(task_event_filter, NULL);
+}

app/src/events.h

@@ -1,9 +1,38 @@
-#define SC_EVENT_NEW_FRAME                 SDL_USEREVENT
+#ifndef SC_EVENTS_H
-#define SC_EVENT_DEVICE_DISCONNECTED      (SDL_USEREVENT + 1)
+#define SC_EVENTS_H
-#define SC_EVENT_SERVER_CONNECTION_FAILED (SDL_USEREVENT + 2)
+
-#define SC_EVENT_SERVER_CONNECTED         (SDL_USEREVENT + 3)
+#include "common.h"
-#define SC_EVENT_USB_DEVICE_DISCONNECTED  (SDL_USEREVENT + 4)
+
-#define SC_EVENT_DEMUXER_ERROR            (SDL_USEREVENT + 5)
+#include <stdbool.h>
-#define SC_EVENT_RECORDER_ERROR           (SDL_USEREVENT + 6)
+#include <stdint.h>
-#define SC_EVENT_SCREEN_INIT_SIZE         (SDL_USEREVENT + 7)
+#include <SDL_events.h>
-#define SC_EVENT_TIME_LIMIT_REACHED       (SDL_USEREVENT + 8)
+
+enum {
+    SC_EVENT_NEW_FRAME = SDL_USEREVENT,
+    SC_EVENT_RUN_ON_MAIN_THREAD,
+    SC_EVENT_DEVICE_DISCONNECTED,
+    SC_EVENT_SERVER_CONNECTION_FAILED,
+    SC_EVENT_SERVER_CONNECTED,
+    SC_EVENT_USB_DEVICE_DISCONNECTED,
+    SC_EVENT_DEMUXER_ERROR,
+    SC_EVENT_RECORDER_ERROR,
+    SC_EVENT_SCREEN_INIT_SIZE,
+    SC_EVENT_TIME_LIMIT_REACHED,
+    SC_EVENT_CONTROLLER_ERROR,
+    SC_EVENT_AOA_OPEN_ERROR,
+};
+
+bool
+sc_push_event_impl(uint32_t type, const char *name);
+
+#define sc_push_event(TYPE) sc_push_event_impl(TYPE, # TYPE)
+
+typedef void (*sc_runnable_fn)(void *userdata);
+
+bool
+sc_post_to_main_thread(sc_runnable_fn run, void *userdata);
+
+void
+sc_reject_new_runnables(void);
+
+#endif

app/src/hid/hid_event.h

@@ -0,0 +1,27 @@
+#ifndef SC_HID_EVENT_H
+#define SC_HID_EVENT_H
+
+#include "common.h"
+
+#include <stdint.h>
+
+#define SC_HID_MAX_SIZE 15
+
+struct sc_hid_input {
+    uint16_t hid_id;
+    uint8_t data[SC_HID_MAX_SIZE];
+    uint8_t size;
+};
+
+struct sc_hid_open {
+    uint16_t hid_id;
+    const char *name; // pointer to static memory
+    const uint8_t *report_desc; // pointer to static memory
+    size_t report_desc_size;
+};
+
+struct sc_hid_close {
+    uint16_t hid_id;
+};
+
+#endif

app/src/hid/hid_gamepad.c

@@ -0,0 +1,458 @@
+#include "hid_gamepad.h"
+
+#include <assert.h>
+#include <inttypes.h>
+
+#include "util/binary.h"
+#include "util/log.h"
+
+// 2 bytes for left stick (X, Y)
+// 2 bytes for right stick (Z, Rz)
+// 2 bytes for L2/R2 triggers
+// 2 bytes for buttons + padding,
+// 1 byte for hat switch (dpad) + padding
+#define SC_HID_GAMEPAD_EVENT_SIZE 15
+
+// The ->buttons field stores the state for all buttons, but only some of them
+// (the 16 LSB) must be transmitted "as is". The DPAD (hat switch) are stored
+// locally in the MSB, but not transmitted as is: they are transformed to
+// generate another specific byte.
+#define SC_HID_BUTTONS_MASK 0xFFFF
+
+// outside SC_HID_BUTTONS_MASK
+#define SC_GAMEPAD_BUTTONS_BIT_DPAD_UP    UINT32_C(0x10000)
+#define SC_GAMEPAD_BUTTONS_BIT_DPAD_DOWN  UINT32_C(0x20000)
+#define SC_GAMEPAD_BUTTONS_BIT_DPAD_LEFT  UINT32_C(0x40000)
+#define SC_GAMEPAD_BUTTONS_BIT_DPAD_RIGHT UINT32_C(0x80000)
+
+/**
+ * Gamepad descriptor manually crafted to transmit the events.
+ *
+ * The HID specification is available here:
+ * <https://www.usb.org/document-library/device-class-definition-hid-111>
+ *
+ * The HID Usage Tables is also useful:
+ * <https://www.usb.org/document-library/hid-usage-tables-15>
+ */
+static const uint8_t SC_HID_GAMEPAD_REPORT_DESC[] = {
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (Gamepad)
+    0x09, 0x05,
+
+    // Collection (Application)
+    0xA1, 0x01,
+
+    // Collection (Physical)
+    0xA1, 0x00,
+
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (X)   Left stick x
+    0x09, 0x30,
+    // Usage (Y)   Left stick y
+    0x09, 0x31,
+    // Usage (Z)   Right stick x
+    0x09, 0x32,
+    // Usage (Rz)  Right stick y
+    0x09, 0x35,
+    // Logical Minimum (0)
+    0x15, 0x00,
+    // Logical Maximum (65535)
+    // Cannot use 26 FF FF because 0xFFFF is interpreted as signed 16-bit
+    0x27, 0xFF, 0xFF, 0x00, 0x00, // little-endian
+    // Report Size (16)
+    0x75, 0x10,
+    // Report Count (4)
+    0x95, 0x04,
+    // Input (Data, Variable, Absolute): 4 bytes (X, Y, Z, Rz)
+    0x81, 0x02,
+
+    // Usage Page (Simulation Controls)
+    0x05, 0x02,
+    // Usage (Brake)
+    0x09, 0xC5,
+    // Usage (Accelerator)
+    0x09, 0xC4,
+    // Logical Minimum (0)
+    0x15, 0x00,
+    // Logical Maximum (32767)
+    0x26, 0xFF, 0x7F,
+    // Report Size (16)
+    0x75, 0x10,
+    // Report Count (2)
+    0x95, 0x02,
+    // Input (Data, Variable, Absolute): 2 bytes (L2, R2)
+    0x81, 0x02,
+
+    // Usage Page (Buttons)
+    0x05, 0x09,
+    // Usage Minimum (1)
+    0x19, 0x01,
+    // Usage Maximum (16)
+    0x29, 0x10,
+    // Logical Minimum (0)
+    0x15, 0x00,
+    // Logical Maximum (1)
+    0x25, 0x01,
+    // Report Count (16)
+    0x95, 0x10,
+    // Report Size (1)
+    0x75, 0x01,
+    // Input (Data, Variable, Absolute): 16 buttons bits
+    0x81, 0x02,
+
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (Hat switch)
+    0x09, 0x39,
+    // Logical Minimum (1)
+    0x15, 0x01,
+    // Logical Maximum (8)
+    0x25, 0x08,
+    // Report Size (4)
+    0x75, 0x04,
+    // Report Count (1)
+    0x95, 0x01,
+    // Input (Data, Variable, Null State): 4-bit value
+    0x81, 0x42,
+
+
+    // End Collection
+    0xC0,
+
+    // End Collection
+    0xC0,
+};
+
+/**
+ * A gamepad HID event is 15 bytes long:
+ *  - bytes 0-3:   left stick state
+ *  - bytes 4-7:   right stick state
+ *  - bytes 8-11:  L2/R2 triggers state
+ *  - bytes 12-13: buttons state
+ *  - bytes 14:    hat switch position (dpad)
+ *
+ *                  +---------------+
+ *         byte 0:  |. . . . . . . .|
+ *                  |               | left stick x state (0-65535)
+ *         byte 1:  |. . . . . . . .|
+ *                  +---------------+
+ *         byte 2:  |. . . . . . . .|
+ *                  |               | left stick y state (0-65535)
+ *         byte 3:  |. . . . . . . .|
+ *                  +---------------+
+ *         byte 4:  |. . . . . . . .|
+ *                  |               | right stick x state (0-65535)
+ *         byte 5:  |. . . . . . . .|
+ *                  +---------------+
+ *         byte 6:  |. . . . . . . .|
+ *                  |               | right stick y state (0-65535)
+ *         byte 7:  |. . . . . . . .|
+ *                  +---------------+
+ *         byte 8:  |. . . . . . . .|
+ *                  |               | L2 trigger state (0-65535)
+ *         byte 9:  |. . . . . . . .|
+ *                  +---------------+
+ *        byte 10:  |. . . . . . . .|
+ *                  |               | R2 trigger state (0-65535)
+ *        byte 11:  |. . . . . . . .|
+ *                  +---------------+
+ *
+ *                   ,--------------- SC_GAMEPAD_BUTTON_RIGHT_SHOULDER
+ *                   | ,------------- SC_GAMEPAD_BUTTON_LEFT_SHOULDER
+ *                   | |
+ *                   | |   ,--------- SC_GAMEPAD_BUTTON_NORTH
+ *                   | |   | ,------- SC_GAMEPAD_BUTTON_WEST
+ *                   | |   | |
+ *                   | |   | |   ,--- SC_GAMEPAD_BUTTON_EAST
+ *                   | |   | |   | ,- SC_GAMEPAD_BUTTON_SOUTH
+ *                   v v   v v   v v
+ *                  +---------------+
+ *        byte 12:  |. . 0 . . 0 . .|
+ *                  |               | Buttons (16-bit little-endian)
+ *        byte 13:  |0 . . . . . 0 0|
+ *                  +---------------+
+ *                     ^ ^ ^ ^ ^
+ *                     | | | | |
+ *                     | | | | |
+ *                     | | | | `----- SC_GAMEPAD_BUTTON_BACK
+ *                     | | | `------- SC_GAMEPAD_BUTTON_START
+ *                     | | `--------- SC_GAMEPAD_BUTTON_GUIDE
+ *                     | `----------- SC_GAMEPAD_BUTTON_LEFT_STICK
+ *                     `------------- SC_GAMEPAD_BUTTON_RIGHT_STICK
+ *
+ *                  +---------------+
+ *        byte 14:  |0 0 0 . . . . .| hat switch (dpad) position (0-8)
+ *                  +---------------+
+ *                     9 possible positions and their values:
+ *                             8 1 2
+ *                             7 0 3
+ *                             6 5 4
+ *                     (8 is top-left, 1 is top, 2 is top-right, etc.)
+ */
+
+static void
+sc_hid_gamepad_slot_init(struct sc_hid_gamepad_slot *slot,
+                         uint32_t gamepad_id) {
+    assert(gamepad_id != SC_GAMEPAD_ID_INVALID);
+    slot->gamepad_id = gamepad_id;
+    slot->buttons = 0;
+    slot->axis_left_x = 0;
+    slot->axis_left_y = 0;
+    slot->axis_right_x = 0;
+    slot->axis_right_y = 0;
+    slot->axis_left_trigger = 0;
+    slot->axis_right_trigger = 0;
+}
+
+static ssize_t
+sc_hid_gamepad_slot_find(struct sc_hid_gamepad *hid, uint32_t gamepad_id) {
+    for (size_t i = 0; i < SC_MAX_GAMEPADS; ++i) {
+        if (gamepad_id == hid->slots[i].gamepad_id) {
+            // found
+            return i;
+        }
+    }
+
+    return -1;
+}
+
+void
+sc_hid_gamepad_init(struct sc_hid_gamepad *hid) {
+    for (size_t i = 0; i < SC_MAX_GAMEPADS; ++i) {
+        hid->slots[i].gamepad_id = SC_GAMEPAD_ID_INVALID;
+    }
+}
+
+static inline uint16_t
+sc_hid_gamepad_slot_get_id(size_t slot_idx) {
+    assert(slot_idx < SC_MAX_GAMEPADS);
+    return SC_HID_ID_GAMEPAD_FIRST + slot_idx;
+}
+
+bool
+sc_hid_gamepad_generate_open(struct sc_hid_gamepad *hid,
+                             struct sc_hid_open *hid_open,
+                             uint32_t gamepad_id) {
+    assert(gamepad_id != SC_GAMEPAD_ID_INVALID);
+    ssize_t slot_idx = sc_hid_gamepad_slot_find(hid, SC_GAMEPAD_ID_INVALID);
+    if (slot_idx == -1) {
+        LOGW("No gamepad slot available for new gamepad %" PRIu32, gamepad_id);
+        return false;
+    }
+
+    sc_hid_gamepad_slot_init(&hid->slots[slot_idx], gamepad_id);
+
+    SDL_GameController* game_controller =
+        SDL_GameControllerFromInstanceID(gamepad_id);
+    assert(game_controller);
+    const char *name = SDL_GameControllerName(game_controller);
+
+    uint16_t hid_id = sc_hid_gamepad_slot_get_id(slot_idx);
+    hid_open->hid_id = hid_id;
+    hid_open->name = name;
+    hid_open->report_desc = SC_HID_GAMEPAD_REPORT_DESC;
+    hid_open->report_desc_size = sizeof(SC_HID_GAMEPAD_REPORT_DESC);
+
+    return true;
+}
+
+bool
+sc_hid_gamepad_generate_close(struct sc_hid_gamepad *hid,
+                              struct sc_hid_close *hid_close,
+                              uint32_t gamepad_id) {
+    assert(gamepad_id != SC_GAMEPAD_ID_INVALID);
+    ssize_t slot_idx = sc_hid_gamepad_slot_find(hid, gamepad_id);
+    if (slot_idx == -1) {
+        LOGW("Unknown gamepad removed %" PRIu32, gamepad_id);
+        return false;
+    }
+
+    hid->slots[slot_idx].gamepad_id = SC_GAMEPAD_ID_INVALID;
+
+    uint16_t hid_id = sc_hid_gamepad_slot_get_id(slot_idx);
+    hid_close->hid_id = hid_id;
+
+    return true;
+}
+
+static uint8_t
+sc_hid_gamepad_get_dpad_value(uint32_t buttons) {
+    // Value depending on direction:
+    //     8 1 2
+    //     7 0 3
+    //     6 5 4
+    if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_UP) {
+        if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_LEFT) {
+            return 8;
+        }
+        if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_RIGHT) {
+            return 2;
+        }
+        return 1;
+    }
+    if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_DOWN) {
+        if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_LEFT) {
+            return 6;
+        }
+        if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_RIGHT) {
+            return 4;
+        }
+        return 5;
+    }
+    if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_LEFT) {
+        return 7;
+    }
+    if (buttons & SC_GAMEPAD_BUTTONS_BIT_DPAD_RIGHT) {
+        return 3;
+    }
+
+    return 0;
+}
+
+static void
+sc_hid_gamepad_event_from_slot(uint16_t hid_id,
+                               const struct sc_hid_gamepad_slot *slot,
+                               struct sc_hid_input *hid_input) {
+    hid_input->hid_id = hid_id;
+    hid_input->size = SC_HID_GAMEPAD_EVENT_SIZE;
+
+    uint8_t *data = hid_input->data;
+    // Values must be written in little-endian
+    sc_write16le(data, slot->axis_left_x);
+    sc_write16le(data + 2, slot->axis_left_y);
+    sc_write16le(data + 4, slot->axis_right_x);
+    sc_write16le(data + 6, slot->axis_right_y);
+    sc_write16le(data + 8, slot->axis_left_trigger);
+    sc_write16le(data + 10, slot->axis_right_trigger);
+    sc_write16le(data + 12, slot->buttons & SC_HID_BUTTONS_MASK);
+    data[14] = sc_hid_gamepad_get_dpad_value(slot->buttons);
+}
+
+static uint32_t
+sc_hid_gamepad_get_button_id(enum sc_gamepad_button button) {
+    switch (button) {
+        case SC_GAMEPAD_BUTTON_SOUTH:
+            return 0x0001;
+        case SC_GAMEPAD_BUTTON_EAST:
+            return 0x0002;
+        case SC_GAMEPAD_BUTTON_WEST:
+            return 0x0008;
+        case SC_GAMEPAD_BUTTON_NORTH:
+            return 0x0010;
+        case SC_GAMEPAD_BUTTON_BACK:
+            return 0x0400;
+        case SC_GAMEPAD_BUTTON_GUIDE:
+            return 0x1000;
+        case SC_GAMEPAD_BUTTON_START:
+            return 0x0800;
+        case SC_GAMEPAD_BUTTON_LEFT_STICK:
+            return 0x2000;
+        case SC_GAMEPAD_BUTTON_RIGHT_STICK:
+            return 0x4000;
+        case SC_GAMEPAD_BUTTON_LEFT_SHOULDER:
+            return 0x0040;
+        case SC_GAMEPAD_BUTTON_RIGHT_SHOULDER:
+            return 0x0080;
+        case SC_GAMEPAD_BUTTON_DPAD_UP:
+            return SC_GAMEPAD_BUTTONS_BIT_DPAD_UP;
+        case SC_GAMEPAD_BUTTON_DPAD_DOWN:
+            return SC_GAMEPAD_BUTTONS_BIT_DPAD_DOWN;
+        case SC_GAMEPAD_BUTTON_DPAD_LEFT:
+            return SC_GAMEPAD_BUTTONS_BIT_DPAD_LEFT;
+        case SC_GAMEPAD_BUTTON_DPAD_RIGHT:
+            return SC_GAMEPAD_BUTTONS_BIT_DPAD_RIGHT;
+        default:
+            // unknown button, ignore
+            return 0;
+    }
+}
+
+bool
+sc_hid_gamepad_generate_input_from_button(struct sc_hid_gamepad *hid,
+                                          struct sc_hid_input *hid_input,
+                                const struct sc_gamepad_button_event *event) {
+    if ((event->button < 0) || (event->button > 15)) {
+        return false;
+    }
+
+    uint32_t gamepad_id = event->gamepad_id;
+
+    ssize_t slot_idx = sc_hid_gamepad_slot_find(hid, gamepad_id);
+    if (slot_idx == -1) {
+        LOGW("Axis event for unknown gamepad %" PRIu32, gamepad_id);
+        return false;
+    }
+
+    assert(slot_idx < SC_MAX_GAMEPADS);
+
+    struct sc_hid_gamepad_slot *slot = &hid->slots[slot_idx];
+
+    uint32_t button = sc_hid_gamepad_get_button_id(event->button);
+    if (!button) {
+        // unknown button, ignore
+        return false;
+    }
+
+    if (event->action == SC_ACTION_DOWN) {
+        slot->buttons |= button;
+    } else {
+        assert(event->action == SC_ACTION_UP);
+        slot->buttons &= ~button;
+    }
+
+    uint16_t hid_id = sc_hid_gamepad_slot_get_id(slot_idx);
+    sc_hid_gamepad_event_from_slot(hid_id, slot, hid_input);
+
+    return true;
+}
+
+bool
+sc_hid_gamepad_generate_input_from_axis(struct sc_hid_gamepad *hid,
+                                        struct sc_hid_input *hid_input,
+                                const struct sc_gamepad_axis_event *event) {
+    uint32_t gamepad_id = event->gamepad_id;
+
+    ssize_t slot_idx = sc_hid_gamepad_slot_find(hid, gamepad_id);
+    if (slot_idx == -1) {
+        LOGW("Button event for unknown gamepad %" PRIu32, gamepad_id);
+        return false;
+    }
+
+    assert(slot_idx < SC_MAX_GAMEPADS);
+
+    struct sc_hid_gamepad_slot *slot = &hid->slots[slot_idx];
+
+// [-32768 to 32767] -> [0 to 65535]
+#define AXIS_RESCALE(V) (uint16_t) (((int32_t) V) + 0x8000)
+    switch (event->axis) {
+        case SC_GAMEPAD_AXIS_LEFTX:
+            slot->axis_left_x = AXIS_RESCALE(event->value);
+            break;
+        case SC_GAMEPAD_AXIS_LEFTY:
+            slot->axis_left_y = AXIS_RESCALE(event->value);
+            break;
+        case SC_GAMEPAD_AXIS_RIGHTX:
+            slot->axis_right_x = AXIS_RESCALE(event->value);
+            break;
+        case SC_GAMEPAD_AXIS_RIGHTY:
+            slot->axis_right_y = AXIS_RESCALE(event->value);
+            break;
+        case SC_GAMEPAD_AXIS_LEFT_TRIGGER:
+            // Trigger is always positive between 0 and 32767
+            slot->axis_left_trigger = MAX(0, event->value);
+            break;
+        case SC_GAMEPAD_AXIS_RIGHT_TRIGGER:
+            // Trigger is always positive between 0 and 32767
+            slot->axis_right_trigger = MAX(0, event->value);
+            break;
+        default:
+            return false;
+    }
+
+    uint16_t hid_id = sc_hid_gamepad_slot_get_id(slot_idx);
+    sc_hid_gamepad_event_from_slot(hid_id, slot, hid_input);
+
+    return true;
+}

app/src/hid/hid_gamepad.h

@@ -0,0 +1,53 @@
+#ifndef SC_HID_GAMEPAD_H
+#define SC_HID_GAMEPAD_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "hid/hid_event.h"
+#include "input_events.h"
+
+#define SC_MAX_GAMEPADS 8
+#define SC_HID_ID_GAMEPAD_FIRST 3
+#define SC_HID_ID_GAMEPAD_LAST (SC_HID_ID_GAMEPAD_FIRST + SC_MAX_GAMEPADS - 1)
+
+struct sc_hid_gamepad_slot {
+    uint32_t gamepad_id;
+    uint32_t buttons;
+    uint16_t axis_left_x;
+    uint16_t axis_left_y;
+    uint16_t axis_right_x;
+    uint16_t axis_right_y;
+    uint16_t axis_left_trigger;
+    uint16_t axis_right_trigger;
+};
+
+struct sc_hid_gamepad {
+    struct sc_hid_gamepad_slot slots[SC_MAX_GAMEPADS];
+};
+
+void
+sc_hid_gamepad_init(struct sc_hid_gamepad *hid);
+
+bool
+sc_hid_gamepad_generate_open(struct sc_hid_gamepad *hid,
+                             struct sc_hid_open *hid_open,
+                             uint32_t gamepad_id);
+
+bool
+sc_hid_gamepad_generate_close(struct sc_hid_gamepad *hid,
+                              struct sc_hid_close *hid_close,
+                              uint32_t gamepad_id);
+
+bool
+sc_hid_gamepad_generate_input_from_button(struct sc_hid_gamepad *hid,
+                                          struct sc_hid_input *hid_input,
+                                const struct sc_gamepad_button_event *event);
+
+bool
+sc_hid_gamepad_generate_input_from_axis(struct sc_hid_gamepad *hid,
+                                        struct sc_hid_input *hid_input,
+                                const struct sc_gamepad_axis_event *event);
+
+#endif

app/src/hid/hid_keyboard.c

@@ -1,49 +1,46 @@
 #include "hid_keyboard.h"
 
-#include <assert.h>
+#include <string.h>
 
-#include "input_events.h"
 #include "util/log.h"
 
-/** Downcast key processor to hid_keyboard */
+#define SC_HID_MOD_NONE 0x00
-#define DOWNCAST(KP) container_of(KP, struct sc_hid_keyboard, key_processor)
+#define SC_HID_MOD_LEFT_CONTROL (1 << 0)
+#define SC_HID_MOD_LEFT_SHIFT (1 << 1)
+#define SC_HID_MOD_LEFT_ALT (1 << 2)
+#define SC_HID_MOD_LEFT_GUI (1 << 3)
+#define SC_HID_MOD_RIGHT_CONTROL (1 << 4)
+#define SC_HID_MOD_RIGHT_SHIFT (1 << 5)
+#define SC_HID_MOD_RIGHT_ALT (1 << 6)
+#define SC_HID_MOD_RIGHT_GUI (1 << 7)
 
-#define HID_KEYBOARD_ACCESSORY_ID 1
+#define SC_HID_KEYBOARD_INDEX_MODS 0
-
+#define SC_HID_KEYBOARD_INDEX_KEYS 2
-#define HID_MODIFIER_NONE 0x00
-#define HID_MODIFIER_LEFT_CONTROL (1 << 0)
-#define HID_MODIFIER_LEFT_SHIFT (1 << 1)
-#define HID_MODIFIER_LEFT_ALT (1 << 2)
-#define HID_MODIFIER_LEFT_GUI (1 << 3)
-#define HID_MODIFIER_RIGHT_CONTROL (1 << 4)
-#define HID_MODIFIER_RIGHT_SHIFT (1 << 5)
-#define HID_MODIFIER_RIGHT_ALT (1 << 6)
-#define HID_MODIFIER_RIGHT_GUI (1 << 7)
-
-#define HID_KEYBOARD_INDEX_MODIFIER 0
-#define HID_KEYBOARD_INDEX_KEYS 2
 
 // USB HID protocol says 6 keys in an event is the requirement for BIOS
 // keyboard support, though OS could support more keys via modifying the report
 // desc. 6 should be enough for scrcpy.
-#define HID_KEYBOARD_MAX_KEYS 6
+#define SC_HID_KEYBOARD_MAX_KEYS 6
-#define HID_KEYBOARD_EVENT_SIZE \
+#define SC_HID_KEYBOARD_INPUT_SIZE \
-    (HID_KEYBOARD_INDEX_KEYS + HID_KEYBOARD_MAX_KEYS)
+    (SC_HID_KEYBOARD_INDEX_KEYS + SC_HID_KEYBOARD_MAX_KEYS)
 
-#define HID_RESERVED 0x00
+#define SC_HID_RESERVED 0x00
-#define HID_ERROR_ROLL_OVER 0x01
+#define SC_HID_ERROR_ROLL_OVER 0x01
 
 /**
- * For HID over AOAv2, only report descriptor is needed.
+ * For HID, only report descriptor is needed.
  *
  * The specification is available here:
- * <https://www.usb.org/sites/default/files/hid1_11.pdf>
+ * <https://www.usb.org/document-library/device-class-definition-hid-111>
  *
  * In particular, read:
- *  - 6.2.2 Report Descriptor
+ *  - §6.2.2 Report Descriptor
  *  - Appendix B.1 Protocol 1 (Keyboard)
  *  - Appendix C: Keyboard Implementation
  *
+ * The HID Usage Tables is also useful:
+ * <https://www.usb.org/document-library/hid-usage-tables-15>
+ *
  * Normally a basic HID keyboard uses 8 bytes:
  *     Modifier Reserved Key Key Key Key Key Key
  *
@@ -53,7 +50,7 @@
  *
  * (change vid:pid' to your device's vendor ID and product ID).
  */
-static const unsigned char keyboard_report_desc[]  = {
+static const uint8_t SC_HID_KEYBOARD_REPORT_DESC[] = {
     // Usage Page (Generic Desktop)
     0x05, 0x01,
     // Usage (Keyboard)
@@ -66,7 +63,7 @@ static const unsigned char keyboard_report_desc[]  = {
     0x05, 0x07,
     // Usage Minimum (224)
     0x19, 0xE0,
-     // Usage Maximum (231)
+    // Usage Maximum (231)
     0x29, 0xE7,
     // Logical Minimum (0)
     0x15, 0x00,
@@ -119,7 +116,7 @@ static const unsigned char keyboard_report_desc[]  = {
     // Report Size (8)
     0x75, 0x08,
     // Report Count (6)
-    0x95, HID_KEYBOARD_MAX_KEYS,
+    0x95, SC_HID_KEYBOARD_MAX_KEYS,
     // Input (Data, Array): Keys
     0x81, 0x00,
 
@@ -201,51 +198,51 @@ static const unsigned char keyboard_report_desc[]  = {
  *                  +---------------+
  */
 
-static unsigned char
+static void
-sdl_keymod_to_hid_modifiers(uint16_t mod) {
+sc_hid_keyboard_input_init(struct sc_hid_input *hid_input) {
-    unsigned char modifiers = HID_MODIFIER_NONE;
+    hid_input->hid_id = SC_HID_ID_KEYBOARD;
-    if (mod & SC_MOD_LCTRL) {
+    hid_input->size = SC_HID_KEYBOARD_INPUT_SIZE;
-        modifiers |= HID_MODIFIER_LEFT_CONTROL;
+
-    }
+    uint8_t *data = hid_input->data;
-    if (mod & SC_MOD_LSHIFT) {
+
-        modifiers |= HID_MODIFIER_LEFT_SHIFT;
+    data[SC_HID_KEYBOARD_INDEX_MODS] = SC_HID_MOD_NONE;
-    }
+    data[1] = SC_HID_RESERVED;
-    if (mod & SC_MOD_LALT) {
+    memset(&data[SC_HID_KEYBOARD_INDEX_KEYS], 0, SC_HID_KEYBOARD_MAX_KEYS);
-        modifiers |= HID_MODIFIER_LEFT_ALT;
-    }
-    if (mod & SC_MOD_LGUI) {
-        modifiers |= HID_MODIFIER_LEFT_GUI;
-    }
-    if (mod & SC_MOD_RCTRL) {
-        modifiers |= HID_MODIFIER_RIGHT_CONTROL;
-    }
-    if (mod & SC_MOD_RSHIFT) {
-        modifiers |= HID_MODIFIER_RIGHT_SHIFT;
-    }
-    if (mod & SC_MOD_RALT) {
-        modifiers |= HID_MODIFIER_RIGHT_ALT;
-    }
-    if (mod & SC_MOD_RGUI) {
-        modifiers |= HID_MODIFIER_RIGHT_GUI;
-    }
-    return modifiers;
 }
 
-static bool
+static uint16_t
-sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
+sc_hid_mod_from_sdl_keymod(uint16_t mod) {
-    unsigned char *buffer = malloc(HID_KEYBOARD_EVENT_SIZE);
+    uint16_t mods = SC_HID_MOD_NONE;
-    if (!buffer) {
+    if (mod & SC_MOD_LCTRL) {
-        LOG_OOM();
+        mods |= SC_HID_MOD_LEFT_CONTROL;
-        return false;
     }
+    if (mod & SC_MOD_LSHIFT) {
+        mods |= SC_HID_MOD_LEFT_SHIFT;
+    }
+    if (mod & SC_MOD_LALT) {
+        mods |= SC_HID_MOD_LEFT_ALT;
+    }
+    if (mod & SC_MOD_LGUI) {
+        mods |= SC_HID_MOD_LEFT_GUI;
+    }
+    if (mod & SC_MOD_RCTRL) {
+        mods |= SC_HID_MOD_RIGHT_CONTROL;
+    }
+    if (mod & SC_MOD_RSHIFT) {
+        mods |= SC_HID_MOD_RIGHT_SHIFT;
+    }
+    if (mod & SC_MOD_RALT) {
+        mods |= SC_HID_MOD_RIGHT_ALT;
+    }
+    if (mod & SC_MOD_RGUI) {
+        mods |= SC_HID_MOD_RIGHT_GUI;
+    }
+    return mods;
+}
 
-    buffer[HID_KEYBOARD_INDEX_MODIFIER] = HID_MODIFIER_NONE;
+void
-    buffer[1] = HID_RESERVED;
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid) {
-    memset(&buffer[HID_KEYBOARD_INDEX_KEYS], 0, HID_KEYBOARD_MAX_KEYS);
+    memset(hid->keys, false, SC_HID_KEYBOARD_KEYS);
-
-    sc_hid_event_init(hid_event, HID_KEYBOARD_ACCESSORY_ID, buffer,
-                      HID_KEYBOARD_EVENT_SIZE);
-    return true;
 }
 
 static inline bool
@@ -253,10 +250,10 @@ scancode_is_modifier(enum sc_scancode scancode) {
     return scancode >= SC_SCANCODE_LCTRL && scancode <= SC_SCANCODE_RGUI;
 }
 
-static bool
+bool
-convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
+sc_hid_keyboard_generate_input_from_key(struct sc_hid_keyboard *hid,
-                           struct sc_hid_event *hid_event,
+                                        struct sc_hid_input *hid_input,
-                           const struct sc_key_event *event) {
+                                        const struct sc_key_event *event) {
     enum sc_scancode scancode = event->scancode;
     assert(scancode >= 0);
 
@@ -268,39 +265,37 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
         return false;
     }
 
-    if (!sc_hid_keyboard_event_init(hid_event)) {
+    sc_hid_keyboard_input_init(hid_input);
-        LOGW("Could not initialize HID keyboard event");
-        return false;
-    }
 
-    unsigned char modifiers = sdl_keymod_to_hid_modifiers(event->mods_state);
+    uint16_t mods = sc_hid_mod_from_sdl_keymod(event->mods_state);
 
     if (scancode < SC_HID_KEYBOARD_KEYS) {
         // Pressed is true and released is false
-        kb->keys[scancode] = (event->action == SC_ACTION_DOWN);
+        hid->keys[scancode] = (event->action == SC_ACTION_DOWN);
         LOGV("keys[%02x] = %s", scancode,
-             kb->keys[scancode] ? "true" : "false");
+             hid->keys[scancode] ? "true" : "false");
     }
 
-    hid_event->buffer[HID_KEYBOARD_INDEX_MODIFIER] = modifiers;
+    hid_input->data[SC_HID_KEYBOARD_INDEX_MODS] = mods;
 
-    unsigned char *keys_buffer = &hid_event->buffer[HID_KEYBOARD_INDEX_KEYS];
+    uint8_t *keys_data = &hid_input->data[SC_HID_KEYBOARD_INDEX_KEYS];
     // Re-calculate pressed keys every time
     int keys_pressed_count = 0;
     for (int i = 0; i < SC_HID_KEYBOARD_KEYS; ++i) {
-        if (kb->keys[i]) {
+        if (hid->keys[i]) {
             // USB HID protocol says that if keys exceeds report count, a
             // phantom state should be reported
-            if (keys_pressed_count >= HID_KEYBOARD_MAX_KEYS) {
+            if (keys_pressed_count >= SC_HID_KEYBOARD_MAX_KEYS) {
                 // Phantom state:
                 //  - Modifiers
                 //  - Reserved
                 //  - ErrorRollOver * HID_MAX_KEYS
-                memset(keys_buffer, HID_ERROR_ROLL_OVER, HID_KEYBOARD_MAX_KEYS);
+                memset(keys_data, SC_HID_ERROR_ROLL_OVER,
+                       SC_HID_KEYBOARD_MAX_KEYS);
                 goto end;
             }
 
-            keys_buffer[keys_pressed_count] = i;
+            keys_data[keys_pressed_count] = i;
             ++keys_pressed_count;
         }
     }
@@ -308,124 +303,43 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
 end:
     LOGV("hid keyboard: key %-4s scancode=%02x (%u) mod=%02x",
          event->action == SC_ACTION_DOWN ? "down" : "up", event->scancode,
-         event->scancode, modifiers);
+         event->scancode, mods);
 
     return true;
 }
 
-
+bool
-static bool
+sc_hid_keyboard_generate_input_from_mods(struct sc_hid_input *hid_input,
-push_mod_lock_state(struct sc_hid_keyboard *kb, uint16_t mods_state) {
+                                         uint16_t mods_state) {
     bool capslock = mods_state & SC_MOD_CAPS;
     bool numlock = mods_state & SC_MOD_NUM;
     if (!capslock && !numlock) {
         // Nothing to do
-        return true;
-    }
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_keyboard_event_init(&hid_event)) {
-        LOGW("Could not initialize HID keyboard event");
         return false;
     }
 
+    sc_hid_keyboard_input_init(hid_input);
+
     unsigned i = 0;
     if (capslock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
+        hid_input->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
         ++i;
     }
     if (numlock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
+        hid_input->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
         ++i;
     }
 
-    if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mod lock state)");
-        return false;
-    }
-
-    LOGD("HID keyboard state synchronized");
-
     return true;
 }
 
-static void
+void sc_hid_keyboard_generate_open(struct sc_hid_open *hid_open) {
-sc_key_processor_process_key(struct sc_key_processor *kp,
+    hid_open->hid_id = SC_HID_ID_KEYBOARD;
-                             const struct sc_key_event *event,
+    hid_open->name = "Keyboard";
-                             uint64_t ack_to_wait) {
+    hid_open->report_desc = SC_HID_KEYBOARD_REPORT_DESC;
-    if (event->repeat) {
+    hid_open->report_desc_size = sizeof(SC_HID_KEYBOARD_REPORT_DESC);
-        // In USB HID protocol, key repeat is handled by the host (Android), so
-        // just ignore key repeat here.
-        return;
-    }
-
-    struct sc_hid_keyboard *kb = DOWNCAST(kp);
-
-    struct sc_hid_event hid_event;
-    // Not all keys are supported, just ignore unsupported keys
-    if (convert_hid_keyboard_event(kb, &hid_event, event)) {
-        if (!kb->mod_lock_synchronized) {
-            // Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
-            // keyboard state
-            if (push_mod_lock_state(kb, event->mods_state)) {
-                kb->mod_lock_synchronized = true;
-            }
-        }
-
-        if (ack_to_wait) {
-            // Ctrl+v is pressed, so clipboard synchronization has been
-            // requested. Wait until clipboard synchronization is acknowledged
-            // by the server, otherwise it could paste the old clipboard
-            // content.
-            hid_event.ack_to_wait = ack_to_wait;
-        }
-
-        if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-            sc_hid_event_destroy(&hid_event);
-            LOGW("Could not request HID event (key)");
-        }
-    }
 }
 
-bool
+void sc_hid_keyboard_generate_close(struct sc_hid_close *hid_close) {
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa) {
+    hid_close->hid_id = SC_HID_ID_KEYBOARD;
-    kb->aoa = aoa;
-
-    bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
-                               keyboard_report_desc,
-                               ARRAY_LEN(keyboard_report_desc));
-    if (!ok) {
-        LOGW("Register HID keyboard failed");
-        return false;
-    }
-
-    // Reset all states
-    memset(kb->keys, false, SC_HID_KEYBOARD_KEYS);
-
-    kb->mod_lock_synchronized = false;
-
-    static const struct sc_key_processor_ops ops = {
-        .process_key = sc_key_processor_process_key,
-        // Never forward text input via HID (all the keys are injected
-        // separately)
-        .process_text = NULL,
-    };
-
-    // Clipboard synchronization is requested over the control socket, while HID
-    // events are sent over AOA, so it must wait for clipboard synchronization
-    // to be acknowledged by the device before injecting Ctrl+v.
-    kb->key_processor.async_paste = true;
-    kb->key_processor.ops = &ops;
-
-    return true;
-}
-
-void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb) {
-    // Unregister HID keyboard so the soft keyboard shows again on Android
-    bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
-    if (!ok) {
-        LOGW("Could not unregister HID keyboard");
-    }
 }

app/src/hid/hid_keyboard.h

@@ -5,8 +5,8 @@
 
 #include <stdbool.h>
 
-#include "aoa_hid.h"
+#include "hid/hid_event.h"
-#include "trait/key_processor.h"
+#include "input_events.h"
 
 // See "SDL2/SDL_scancode.h".
 // Maybe SDL_Keycode is used by most people, but SDL_Scancode is taken from USB
@@ -14,6 +14,8 @@
 // 0x65 is Application, typically AT-101 Keyboard ends here.
 #define SC_HID_KEYBOARD_KEYS 0x66
 
+#define SC_HID_ID_KEYBOARD 1
+
 /**
  * HID keyboard events are sequence-based, every time keyboard state changes
  * it sends an array of currently pressed keys, the host is responsible for
@@ -27,18 +29,25 @@
  * phantom state.
  */
 struct sc_hid_keyboard {
-    struct sc_key_processor key_processor; // key processor trait
-
-    struct sc_aoa *aoa;
     bool keys[SC_HID_KEYBOARD_KEYS];
-
-    bool mod_lock_synchronized;
 };
 
-bool
+void
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa);
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid);
 
 void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb);
+sc_hid_keyboard_generate_open(struct sc_hid_open *hid_open);
+
+void
+sc_hid_keyboard_generate_close(struct sc_hid_close *hid_close);
+
+bool
+sc_hid_keyboard_generate_input_from_key(struct sc_hid_keyboard *hid,
+                                        struct sc_hid_input *hid_input,
+                                        const struct sc_key_event *event);
+
+bool
+sc_hid_keyboard_generate_input_from_mods(struct sc_hid_input *hid_input,
+                                         uint16_t mods_state);
 
 #endif

app/src/hid/hid_mouse.c

@@ -0,0 +1,200 @@
+#include "hid_mouse.h"
+
+// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position,
+// 1 byte for wheel motion
+#define SC_HID_MOUSE_INPUT_SIZE 4
+
+/**
+ * Mouse descriptor from the specification:
+ * <https://www.usb.org/document-library/device-class-definition-hid-111>
+ *
+ * Appendix E (p71): §E.10 Report Descriptor (Mouse)
+ *
+ * The usage tags (like Wheel) are listed in "HID Usage Tables":
+ * <https://www.usb.org/document-library/hid-usage-tables-15>
+ * §4 Generic Desktop Page (0x01) (p32)
+ */
+const uint8_t SC_HID_MOUSE_REPORT_DESC[] = {
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (Mouse)
+    0x09, 0x02,
+
+    // Collection (Application)
+    0xA1, 0x01,
+
+    // Usage (Pointer)
+    0x09, 0x01,
+
+    // Collection (Physical)
+    0xA1, 0x00,
+
+    // Usage Page (Buttons)
+    0x05, 0x09,
+
+    // Usage Minimum (1)
+    0x19, 0x01,
+    // Usage Maximum (5)
+    0x29, 0x05,
+    // Logical Minimum (0)
+    0x15, 0x00,
+    // Logical Maximum (1)
+    0x25, 0x01,
+    // Report Count (5)
+    0x95, 0x05,
+    // Report Size (1)
+    0x75, 0x01,
+    // Input (Data, Variable, Absolute): 5 buttons bits
+    0x81, 0x02,
+
+    // Report Count (1)
+    0x95, 0x01,
+    // Report Size (3)
+    0x75, 0x03,
+    // Input (Constant): 3 bits padding
+    0x81, 0x01,
+
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (X)
+    0x09, 0x30,
+    // Usage (Y)
+    0x09, 0x31,
+    // Usage (Wheel)
+    0x09, 0x38,
+    // Logical Minimum (-127)
+    0x15, 0x81,
+    // Logical Maximum (127)
+    0x25, 0x7F,
+    // Report Size (8)
+    0x75, 0x08,
+    // Report Count (3)
+    0x95, 0x03,
+    // Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
+    0x81, 0x06,
+
+    // End Collection
+    0xC0,
+
+    // End Collection
+    0xC0,
+};
+
+/**
+ * A mouse HID event is 4 bytes long:
+ *
+ *  - byte 0: buttons state
+ *  - byte 1: relative x motion (signed byte from -127 to 127)
+ *  - byte 2: relative y motion (signed byte from -127 to 127)
+ *  - byte 3: wheel motion (-1, 0 or 1)
+ *
+ *                   7 6 5 4 3 2 1 0
+ *                  +---------------+
+ *         byte 0:  |0 0 0 . . . . .| buttons state
+ *                  +---------------+
+ *                         ^ ^ ^ ^ ^
+ *                         | | | | `- left button
+ *                         | | | `--- right button
+ *                         | | `----- middle button
+ *                         | `------- button 4
+ *                         `--------- button 5
+ *
+ *                  +---------------+
+ *         byte 1:  |. . . . . . . .| relative x motion
+ *                  +---------------+
+ *         byte 2:  |. . . . . . . .| relative y motion
+ *                  +---------------+
+ *         byte 3:  |. . . . . . . .| wheel motion
+ *                  +---------------+
+ *
+ * As an example, here is the report for a motion of (x=5, y=-4) with left
+ * button pressed:
+ *
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 1| left button pressed
+ *                  +---------------+
+ *                  |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
+ *                  +---------------+
+ *                  |1 1 1 1 1 1 0 0| relative y motion (y = -4)
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 0| wheel motion
+ *                  +---------------+
+ */
+
+static void
+sc_hid_mouse_input_init(struct sc_hid_input *hid_input) {
+    hid_input->hid_id = SC_HID_ID_MOUSE;
+    hid_input->size = SC_HID_MOUSE_INPUT_SIZE;
+    // Leave ->data uninitialized, it will be fully initialized by callers
+}
+
+static uint8_t
+sc_hid_buttons_from_buttons_state(uint8_t buttons_state) {
+    uint8_t c = 0;
+    if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
+        c |= 1 << 0;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
+        c |= 1 << 1;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
+        c |= 1 << 2;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X1) {
+        c |= 1 << 3;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X2) {
+        c |= 1 << 4;
+    }
+    return c;
+}
+
+void
+sc_hid_mouse_generate_input_from_motion(struct sc_hid_input *hid_input,
+                                    const struct sc_mouse_motion_event *event) {
+    sc_hid_mouse_input_init(hid_input);
+
+    uint8_t *data = hid_input->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = CLAMP(event->xrel, -127, 127);
+    data[2] = CLAMP(event->yrel, -127, 127);
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_generate_input_from_click(struct sc_hid_input *hid_input,
+                                     const struct sc_mouse_click_event *event) {
+    sc_hid_mouse_input_init(hid_input);
+
+    uint8_t *data = hid_input->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_generate_input_from_scroll(struct sc_hid_input *hid_input,
+                                    const struct sc_mouse_scroll_event *event) {
+    sc_hid_mouse_input_init(hid_input);
+
+    uint8_t *data = hid_input->data;
+    data[0] = 0; // buttons state irrelevant (and unknown)
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    // In practice, vscroll is always -1, 0 or 1, but in theory other values
+    // are possible
+    data[3] = CLAMP(event->vscroll, -127, 127);
+    // Horizontal scrolling ignored
+}
+
+void sc_hid_mouse_generate_open(struct sc_hid_open *hid_open) {
+    hid_open->hid_id = SC_HID_ID_MOUSE;
+    hid_open->name = "Mouse";
+    hid_open->report_desc = SC_HID_MOUSE_REPORT_DESC;
+    hid_open->report_desc_size = sizeof(SC_HID_MOUSE_REPORT_DESC);
+}
+
+void sc_hid_mouse_generate_close(struct sc_hid_close *hid_close) {
+    hid_close->hid_id = SC_HID_ID_MOUSE;
+}

app/src/hid/hid_mouse.h

@@ -0,0 +1,31 @@
+#ifndef SC_HID_MOUSE_H
+#define SC_HID_MOUSE_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "hid/hid_event.h"
+#include "input_events.h"
+
+#define SC_HID_ID_MOUSE 2
+
+void
+sc_hid_mouse_generate_open(struct sc_hid_open *hid_open);
+
+void
+sc_hid_mouse_generate_close(struct sc_hid_close *hid_close);
+
+void
+sc_hid_mouse_generate_input_from_motion(struct sc_hid_input *hid_input,
+                                    const struct sc_mouse_motion_event *event);
+
+void
+sc_hid_mouse_generate_input_from_click(struct sc_hid_input *hid_input,
+                                    const struct sc_mouse_click_event *event);
+
+void
+sc_hid_mouse_generate_input_from_scroll(struct sc_hid_input *hid_input,
+                                    const struct sc_mouse_scroll_event *event);
+
+#endif

app/src/icon.c

@@ -78,7 +78,19 @@ decode_image(const char *path) {
         goto close_input;
     }
 
-    int stream = av_find_best_stream(ctx, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
+
+// In ffmpeg/doc/APIchanges:
+// 2021-04-27 - 46dac8cf3d - lavf 59.0.100 - avformat.h
+//   av_find_best_stream now uses a const AVCodec ** parameter
+//   for the returned decoder.
+#if LIBAVFORMAT_VERSION_INT >= AV_VERSION_INT(59, 0, 100)
+    const AVCodec *codec;
+#else
+    AVCodec *codec;
+#endif
+
+    int stream =
+        av_find_best_stream(ctx, AVMEDIA_TYPE_VIDEO, -1, -1, &codec, 0);
     if (stream < 0 ) {
         LOGE("Could not find best image stream");
         goto close_input;
@@ -86,12 +98,6 @@ decode_image(const char *path) {
 
     AVCodecParameters *params = ctx->streams[stream]->codecpar;
 
-    const AVCodec *codec = avcodec_find_decoder(params->codec_id);
-    if (!codec) {
-        LOGE("Could not find image decoder");
-        goto close_input;
-    }
-
     AVCodecContext *codec_ctx = avcodec_alloc_context3(codec);
     if (!codec_ctx) {
         LOG_OOM();
@@ -111,21 +117,21 @@ decode_image(const char *path) {
     AVFrame *frame = av_frame_alloc();
     if (!frame) {
         LOG_OOM();
-        goto close_codec;
+        goto free_codec_ctx;
     }
 
     AVPacket *packet = av_packet_alloc();
     if (!packet) {
         LOG_OOM();
         av_frame_free(&frame);
-        goto close_codec;
+        goto free_codec_ctx;
     }
 
     if (av_read_frame(ctx, packet) < 0) {
         LOGE("Could not read frame");
         av_packet_free(&packet);
         av_frame_free(&frame);
-        goto close_codec;
+        goto free_codec_ctx;
     }
 
     int ret;
@@ -133,22 +139,20 @@ decode_image(const char *path) {
         LOGE("Could not send icon packet: %d", ret);
         av_packet_free(&packet);
         av_frame_free(&frame);
-        goto close_codec;
+        goto free_codec_ctx;
     }
 
     if ((ret = avcodec_receive_frame(codec_ctx, frame)) != 0) {
         LOGE("Could not receive icon frame: %d", ret);
         av_packet_free(&packet);
         av_frame_free(&frame);
-        goto close_codec;
+        goto free_codec_ctx;
     }
 
     av_packet_free(&packet);
 
     result = frame;
 
-close_codec:
-    avcodec_close(codec_ctx);
 free_codec_ctx:
     avcodec_free_context(&codec_ctx);
 close_input:

app/src/input_events.h

@@ -9,6 +9,7 @@
 #include <SDL2/SDL_events.h>
 
 #include "coords.h"
+#include "options.h"
 
 /* The representation of input events in scrcpy is very close to the SDL API,
  * for simplicity.
@@ -322,6 +323,38 @@ enum sc_mouse_button {
     SC_MOUSE_BUTTON_X2 = SDL_BUTTON(SDL_BUTTON_X2),
 };
 
+// Use the naming from SDL3 for gamepad axis and buttons:
+// <https://wiki.libsdl.org/SDL3/README/migration>
+
+enum sc_gamepad_axis {
+    SC_GAMEPAD_AXIS_UNKNOWN = -1,
+    SC_GAMEPAD_AXIS_LEFTX = SDL_CONTROLLER_AXIS_LEFTX,
+    SC_GAMEPAD_AXIS_LEFTY = SDL_CONTROLLER_AXIS_LEFTY,
+    SC_GAMEPAD_AXIS_RIGHTX = SDL_CONTROLLER_AXIS_RIGHTX,
+    SC_GAMEPAD_AXIS_RIGHTY = SDL_CONTROLLER_AXIS_RIGHTY,
+    SC_GAMEPAD_AXIS_LEFT_TRIGGER = SDL_CONTROLLER_AXIS_TRIGGERLEFT,
+    SC_GAMEPAD_AXIS_RIGHT_TRIGGER = SDL_CONTROLLER_AXIS_TRIGGERRIGHT,
+};
+
+enum sc_gamepad_button {
+    SC_GAMEPAD_BUTTON_UNKNOWN = -1,
+    SC_GAMEPAD_BUTTON_SOUTH = SDL_CONTROLLER_BUTTON_A,
+    SC_GAMEPAD_BUTTON_EAST = SDL_CONTROLLER_BUTTON_B,
+    SC_GAMEPAD_BUTTON_WEST = SDL_CONTROLLER_BUTTON_X,
+    SC_GAMEPAD_BUTTON_NORTH = SDL_CONTROLLER_BUTTON_Y,
+    SC_GAMEPAD_BUTTON_BACK = SDL_CONTROLLER_BUTTON_BACK,
+    SC_GAMEPAD_BUTTON_GUIDE = SDL_CONTROLLER_BUTTON_GUIDE,
+    SC_GAMEPAD_BUTTON_START = SDL_CONTROLLER_BUTTON_START,
+    SC_GAMEPAD_BUTTON_LEFT_STICK = SDL_CONTROLLER_BUTTON_LEFTSTICK,
+    SC_GAMEPAD_BUTTON_RIGHT_STICK = SDL_CONTROLLER_BUTTON_RIGHTSTICK,
+    SC_GAMEPAD_BUTTON_LEFT_SHOULDER = SDL_CONTROLLER_BUTTON_LEFTSHOULDER,
+    SC_GAMEPAD_BUTTON_RIGHT_SHOULDER = SDL_CONTROLLER_BUTTON_RIGHTSHOULDER,
+    SC_GAMEPAD_BUTTON_DPAD_UP = SDL_CONTROLLER_BUTTON_DPAD_UP,
+    SC_GAMEPAD_BUTTON_DPAD_DOWN = SDL_CONTROLLER_BUTTON_DPAD_DOWN,
+    SC_GAMEPAD_BUTTON_DPAD_LEFT = SDL_CONTROLLER_BUTTON_DPAD_LEFT,
+    SC_GAMEPAD_BUTTON_DPAD_RIGHT = SDL_CONTROLLER_BUTTON_DPAD_RIGHT,
+};
+
 static_assert(sizeof(enum sc_mod) >= sizeof(SDL_Keymod),
               "SDL_Keymod must be convertible to sc_mod");
 
@@ -379,6 +412,33 @@ struct sc_touch_event {
     float pressure;
 };
 
+enum sc_gamepad_device_event_type {
+    SC_GAMEPAD_DEVICE_ADDED,
+    SC_GAMEPAD_DEVICE_REMOVED,
+};
+
+// As documented in <https://wiki.libsdl.org/SDL2/SDL_JoystickID>:
+// The ID value starts at 0 and increments from there. The value -1 is an
+// invalid ID.
+#define SC_GAMEPAD_ID_INVALID UINT32_C(-1)
+
+struct sc_gamepad_device_event {
+    enum sc_gamepad_device_event_type type;
+    uint32_t gamepad_id;
+};
+
+struct sc_gamepad_button_event {
+    uint32_t gamepad_id;
+    enum sc_action action;
+    enum sc_gamepad_button button;
+};
+
+struct sc_gamepad_axis_event {
+    uint32_t gamepad_id;
+    enum sc_gamepad_axis axis;
+    int16_t value;
+};
+
 static inline uint16_t
 sc_mods_state_from_sdl(uint16_t mods_state) {
     return mods_state;
@@ -436,19 +496,50 @@ sc_mouse_button_from_sdl(uint8_t button) {
 }
 
 static inline uint8_t
-sc_mouse_buttons_state_from_sdl(uint32_t buttons_state,
+sc_mouse_buttons_state_from_sdl(uint32_t buttons_state) {
-                                bool forward_all_clicks) {
     assert(buttons_state < 0x100); // fits in uint8_t
 
-    uint8_t mask = SC_MOUSE_BUTTON_LEFT;
+    // SC_MOUSE_BUTTON_* constants are initialized from SDL_BUTTON(index)
-    if (forward_all_clicks) {
+    return buttons_state;
-        mask |= SC_MOUSE_BUTTON_RIGHT
+}
-              | SC_MOUSE_BUTTON_MIDDLE
-              | SC_MOUSE_BUTTON_X1
-              | SC_MOUSE_BUTTON_X2;
-    }
 
-    return buttons_state & mask;
+static inline enum sc_gamepad_device_event_type
+sc_gamepad_device_event_type_from_sdl_type(uint32_t type) {
+    assert(type == SDL_CONTROLLERDEVICEADDED
+        || type == SDL_CONTROLLERDEVICEREMOVED);
+    if (type == SDL_CONTROLLERDEVICEADDED) {
+        return SC_GAMEPAD_DEVICE_ADDED;
+    }
+    return SC_GAMEPAD_DEVICE_REMOVED;
+}
+
+static inline enum sc_gamepad_axis
+sc_gamepad_axis_from_sdl(uint8_t axis) {
+    if (axis <= SDL_CONTROLLER_AXIS_TRIGGERRIGHT) {
+        // SDL_GAMEPAD_AXIS_* constants are initialized from
+        // SDL_CONTROLLER_AXIS_*
+        return axis;
+    }
+    return SC_GAMEPAD_AXIS_UNKNOWN;
+}
+
+static inline enum sc_gamepad_button
+sc_gamepad_button_from_sdl(uint8_t button) {
+    if (button <= SDL_CONTROLLER_BUTTON_DPAD_RIGHT) {
+        // SC_GAMEPAD_BUTTON_* constants are initialized from
+        // SDL_CONTROLLER_BUTTON_*
+        return button;
+    }
+    return SC_GAMEPAD_BUTTON_UNKNOWN;
+}
+
+static inline enum sc_action
+sc_action_from_sdl_controllerbutton_type(uint32_t type) {
+    assert(type == SDL_CONTROLLERBUTTONDOWN || type == SDL_CONTROLLERBUTTONUP);
+    if (type == SDL_CONTROLLERBUTTONDOWN) {
+        return SC_ACTION_DOWN;
+    }
+    return SC_ACTION_UP;
 }
 
 #endif

app/src/input_manager.h

@@ -11,6 +11,7 @@
 #include "file_pusher.h"
 #include "fps_counter.h"
 #include "options.h"
+#include "trait/gamepad_processor.h"
 #include "trait/key_processor.h"
 #include "trait/mouse_processor.h"
 
@@ -21,17 +22,19 @@ struct sc_input_manager {
 
     struct sc_key_processor *kp;
     struct sc_mouse_processor *mp;
+    struct sc_gamepad_processor *gp;
 
-    bool forward_all_clicks;
+    struct sc_mouse_bindings mouse_bindings;
     bool legacy_paste;
     bool clipboard_autosync;
 
-    struct {
+    uint16_t sdl_shortcut_mods;
-        unsigned data[SC_MAX_SHORTCUT_MODS];
-        unsigned count;
-    } sdl_shortcut_mods;
 
     bool vfinger_down;
+    bool vfinger_invert_x;
+    bool vfinger_invert_y;
+
+    uint8_t mouse_buttons_state; // OR of enum sc_mouse_button values
 
     // Tracks the number of identical consecutive shortcut key down events.
     // Not to be confused with event->repeat, which counts the number of
@@ -49,11 +52,12 @@ struct sc_input_manager_params {
     struct sc_screen *screen;
     struct sc_key_processor *kp;
     struct sc_mouse_processor *mp;
+    struct sc_gamepad_processor *gp;
 
-    bool forward_all_clicks;
+    struct sc_mouse_bindings mouse_bindings;
     bool legacy_paste;
     bool clipboard_autosync;
-    const struct sc_shortcut_mods *shortcut_mods;
+    uint8_t shortcut_mods; // OR of enum sc_shortcut_mod values
 };
 
 void

app/src/keyboard_sdk.c

@@ -1,4 +1,4 @@
-#include "keyboard_inject.h"
+#include "keyboard_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast key processor to sc_keyboard_inject */
+/** Downcast key processor to sc_keyboard_sdk */
-#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_inject, key_processor)
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_sdk, key_processor)
 
 static enum android_keyevent_action
 convert_keycode_action(enum sc_action action) {
@@ -271,20 +271,20 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
     // is set before injecting Ctrl+v.
     (void) ack_to_wait;
 
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
     if (event->repeat) {
-        if (!ki->forward_key_repeat) {
+        if (!kb->forward_key_repeat) {
             return;
         }
-        ++ki->repeat;
+        ++kb->repeat;
     } else {
-        ki->repeat = 0;
+        kb->repeat = 0;
     }
 
     struct sc_control_msg msg;
-    if (convert_input_key(event, &msg, ki->key_inject_mode, ki->repeat)) {
+    if (convert_input_key(event, &msg, kb->key_inject_mode, kb->repeat)) {
-        if (!sc_controller_push_msg(ki->controller, &msg)) {
+        if (!sc_controller_push_msg(kb->controller, &msg)) {
             LOGW("Could not request 'inject keycode'");
         }
     }
@@ -293,14 +293,14 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
 static void
 sc_key_processor_process_text(struct sc_key_processor *kp,
                               const struct sc_text_event *event) {
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
         // Never inject text events
         return;
     }
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
         char c = event->text[0];
         if (isalpha(c) || c == ' ') {
             assert(event->text[1] == '\0');
@@ -316,22 +316,22 @@ sc_key_processor_process_text(struct sc_key_processor *kp,
         LOGW("Could not strdup input text");
         return;
     }
-    if (!sc_controller_push_msg(ki->controller, &msg)) {
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
         free(msg.inject_text.text);
         LOGW("Could not request 'inject text'");
     }
 }
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
-                        struct sc_controller *controller,
+                     struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
+                     enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat) {
+                     bool forward_key_repeat) {
-    ki->controller = controller;
+    kb->controller = controller;
-    ki->key_inject_mode = key_inject_mode;
+    kb->key_inject_mode = key_inject_mode;
-    ki->forward_key_repeat = forward_key_repeat;
+    kb->forward_key_repeat = forward_key_repeat;
 
-    ki->repeat = 0;
+    kb->repeat = 0;
 
     static const struct sc_key_processor_ops ops = {
         .process_key = sc_key_processor_process_key,
@@ -339,6 +339,7 @@ sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
     };
 
     // Key injection and clipboard synchronization are serialized
-    ki->key_processor.async_paste = false;
+    kb->key_processor.async_paste = false;
-    ki->key_processor.ops = &ops;
+    kb->key_processor.hid = false;
+    kb->key_processor.ops = &ops;
 }

app/src/keyboard_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_KEYBOARD_INJECT_H
+#ifndef SC_KEYBOARD_SDK_H
-#define SC_KEYBOARD_INJECT_H
+#define SC_KEYBOARD_SDK_H
 
 #include "common.h"
 
@@ -9,7 +9,7 @@
 #include "options.h"
 #include "trait/key_processor.h"
 
-struct sc_keyboard_inject {
+struct sc_keyboard_sdk {
     struct sc_key_processor key_processor; // key processor trait
 
     struct sc_controller *controller;
@@ -23,9 +23,9 @@ struct sc_keyboard_inject {
 };
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
-                        struct sc_controller *controller,
+                     struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
+                     enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat);
+                     bool forward_key_repeat);
 
 #endif

app/src/main.c

@@ -16,6 +16,7 @@
 #include "usb/scrcpy_otg.h"
 #include "util/log.h"
 #include "util/net.h"
+#include "util/thread.h"
 #include "version.h"
 
 #ifdef _WIN32
@@ -67,6 +68,9 @@ main_scrcpy(int argc, char *argv[]) {
         goto end;
     }
 
+    // The current thread is the main thread
+    SC_MAIN_THREAD_ID = sc_thread_get_id();
+
 #ifdef SCRCPY_LAVF_REQUIRES_REGISTER_ALL
     av_register_all();
 #endif

app/src/mouse_sdk.c

@@ -1,4 +1,4 @@
-#include "mouse_inject.h"
+#include "mouse_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast mouse processor to sc_mouse_inject */
+/** Downcast mouse processor to sc_mouse_sdk */
-#define DOWNCAST(MP) container_of(MP, struct sc_mouse_inject, mouse_processor)
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_sdk, mouse_processor)
 
 static enum android_motionevent_buttons
 convert_mouse_buttons(uint32_t state) {
@@ -58,17 +58,18 @@ convert_touch_action(enum sc_touch_action action) {
 static void
 sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
                                     const struct sc_mouse_motion_event *event) {
-    if (!event->buttons_state) {
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
+
+    if (!m->mouse_hover && !event->buttons_state) {
         // Do not send motion events when no click is pressed
         return;
     }
 
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
-
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
         .inject_touch_event = {
-            .action = AMOTION_EVENT_ACTION_MOVE,
+            .action = event->buttons_state ? AMOTION_EVENT_ACTION_MOVE
+                                           : AMOTION_EVENT_ACTION_HOVER_MOVE,
             .pointer_id = event->pointer_id,
             .position = event->position,
             .pressure = 1.f,
@@ -76,7 +77,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse motion event'");
     }
 }
@@ -84,7 +85,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
                                     const struct sc_mouse_click_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -98,7 +99,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse click event'");
     }
 }
@@ -106,7 +107,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
                                    const struct sc_mouse_scroll_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT,
@@ -118,7 +119,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse scroll event'");
     }
 }
@@ -126,7 +127,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
                                  const struct sc_touch_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -139,15 +140,16 @@ sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject touch event'");
     }
 }
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller,
-                     struct sc_controller *controller) {
+                  bool mouse_hover) {
-    mi->controller = controller;
+    m->controller = controller;
+    m->mouse_hover = mouse_hover;
 
     static const struct sc_mouse_processor_ops ops = {
         .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
@@ -156,7 +158,7 @@ sc_mouse_inject_init(struct sc_mouse_inject *mi,
         .process_touch = sc_mouse_processor_process_touch,
     };
 
-    mi->mouse_processor.ops = &ops;
+    m->mouse_processor.ops = &ops;
 
-    mi->mouse_processor.relative_mode = false;
+    m->mouse_processor.relative_mode = false;
 }

app/src/mouse_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_MOUSE_INJECT_H
+#ifndef SC_MOUSE_SDK_H
-#define SC_MOUSE_INJECT_H
+#define SC_MOUSE_SDK_H
 
 #include "common.h"
 
@@ -9,14 +9,15 @@
 #include "screen.h"
 #include "trait/mouse_processor.h"
 
-struct sc_mouse_inject {
+struct sc_mouse_sdk {
     struct sc_mouse_processor mouse_processor; // mouse processor trait
 
     struct sc_controller *controller;
+    bool mouse_hover;
 };
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller,
-                     struct sc_controller *controller);
+                  bool mouse_hover);
 
 #endif

app/src/options.c

@@ -21,8 +21,23 @@ const struct scrcpy_options scrcpy_options_default = {
     .video_source = SC_VIDEO_SOURCE_DISPLAY,
     .audio_source = SC_AUDIO_SOURCE_AUTO,
     .record_format = SC_RECORD_FORMAT_AUTO,
-    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT,
+    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_AUTO,
-    .mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT,
+    .mouse_input_mode = SC_MOUSE_INPUT_MODE_AUTO,
+    .gamepad_input_mode = SC_GAMEPAD_INPUT_MODE_AUTO,
+    .mouse_bindings = {
+        .pri = {
+            .right_click = SC_MOUSE_BINDING_AUTO,
+            .middle_click = SC_MOUSE_BINDING_AUTO,
+            .click4 = SC_MOUSE_BINDING_AUTO,
+            .click5 = SC_MOUSE_BINDING_AUTO,
+        },
+        .sec = {
+            .right_click = SC_MOUSE_BINDING_AUTO,
+            .middle_click = SC_MOUSE_BINDING_AUTO,
+            .click4 = SC_MOUSE_BINDING_AUTO,
+            .click5 = SC_MOUSE_BINDING_AUTO,
+        },
+    },
     .camera_facing = SC_CAMERA_FACING_ANY,
     .port_range = {
         .first = DEFAULT_LOCAL_PORT_RANGE_FIRST,
@@ -30,10 +45,7 @@ const struct scrcpy_options scrcpy_options_default = {
     },
     .tunnel_host = 0,
     .tunnel_port = 0,
-    .shortcut_mods = {
+    .shortcut_mods = SC_SHORTCUT_MOD_LALT | SC_SHORTCUT_MOD_LSUPER,
-        .data = {SC_SHORTCUT_MOD_LALT, SC_SHORTCUT_MOD_LSUPER},
-        .count = 2,
-    },
     .max_size = 0,
     .video_bit_rate = 0,
     .audio_bit_rate = 0,
@@ -71,7 +83,6 @@ const struct scrcpy_options scrcpy_options_default = {
     .force_adb_forward = false,
     .disable_screensaver = false,
     .forward_key_repeat = true,
-    .forward_all_clicks = false,
     .legacy_paste = false,
     .power_off_on_close = false,
     .clipboard_autosync = true,
@@ -89,6 +100,9 @@ const struct scrcpy_options scrcpy_options_default = {
     .kill_adb_on_close = false,
     .camera_high_speed = false,
     .list = 0,
+    .window = true,
+    .mouse_hover = true,
+    .audio_dup = false,
 };
 
 enum sc_orientation

app/src/options.h

@@ -59,6 +59,7 @@ enum sc_audio_source {
     SC_AUDIO_SOURCE_AUTO, // OUTPUT for video DISPLAY, MIC for video CAMERA
     SC_AUDIO_SOURCE_OUTPUT,
     SC_AUDIO_SOURCE_MIC,
+    SC_AUDIO_SOURCE_PLAYBACK,
 };
 
 enum sc_camera_facing {
@@ -140,13 +141,51 @@ enum sc_lock_video_orientation {
 };
 
 enum sc_keyboard_input_mode {
-    SC_KEYBOARD_INPUT_MODE_INJECT,
+    SC_KEYBOARD_INPUT_MODE_AUTO,
-    SC_KEYBOARD_INPUT_MODE_HID,
+    SC_KEYBOARD_INPUT_MODE_UHID_OR_AOA, // normal vs otg mode
+    SC_KEYBOARD_INPUT_MODE_DISABLED,
+    SC_KEYBOARD_INPUT_MODE_SDK,
+    SC_KEYBOARD_INPUT_MODE_UHID,
+    SC_KEYBOARD_INPUT_MODE_AOA,
 };
 
 enum sc_mouse_input_mode {
-    SC_MOUSE_INPUT_MODE_INJECT,
+    SC_MOUSE_INPUT_MODE_AUTO,
-    SC_MOUSE_INPUT_MODE_HID,
+    SC_MOUSE_INPUT_MODE_UHID_OR_AOA, // normal vs otg mode
+    SC_MOUSE_INPUT_MODE_DISABLED,
+    SC_MOUSE_INPUT_MODE_SDK,
+    SC_MOUSE_INPUT_MODE_UHID,
+    SC_MOUSE_INPUT_MODE_AOA,
+};
+
+enum sc_gamepad_input_mode {
+    SC_GAMEPAD_INPUT_MODE_AUTO,
+    SC_GAMEPAD_INPUT_MODE_UHID_OR_AOA, // normal vs otg mode
+    SC_GAMEPAD_INPUT_MODE_DISABLED,
+    SC_GAMEPAD_INPUT_MODE_UHID,
+    SC_GAMEPAD_INPUT_MODE_AOA,
+};
+
+enum sc_mouse_binding {
+    SC_MOUSE_BINDING_AUTO,
+    SC_MOUSE_BINDING_DISABLED,
+    SC_MOUSE_BINDING_CLICK,
+    SC_MOUSE_BINDING_BACK,
+    SC_MOUSE_BINDING_HOME,
+    SC_MOUSE_BINDING_APP_SWITCH,
+    SC_MOUSE_BINDING_EXPAND_NOTIFICATION_PANEL,
+};
+
+struct sc_mouse_binding_set {
+    enum sc_mouse_binding right_click;
+    enum sc_mouse_binding middle_click;
+    enum sc_mouse_binding click4;
+    enum sc_mouse_binding click5;
+};
+
+struct sc_mouse_bindings {
+    struct sc_mouse_binding_set pri;
+    struct sc_mouse_binding_set sec; // When Shift is pressed
 };
 
 enum sc_key_inject_mode {
@@ -163,8 +202,6 @@ enum sc_key_inject_mode {
     SC_KEY_INJECT_MODE_RAW,
 };
 
-#define SC_MAX_SHORTCUT_MODS 8
-
 enum sc_shortcut_mod {
     SC_SHORTCUT_MOD_LCTRL = 1 << 0,
     SC_SHORTCUT_MOD_RCTRL = 1 << 1,
@@ -174,11 +211,6 @@ enum sc_shortcut_mod {
     SC_SHORTCUT_MOD_RSUPER = 1 << 5,
 };
 
-struct sc_shortcut_mods {
-    unsigned data[SC_MAX_SHORTCUT_MODS];
-    unsigned count;
-};
-
 struct sc_port_range {
     uint16_t first;
     uint16_t last;
@@ -209,11 +241,13 @@ struct scrcpy_options {
     enum sc_record_format record_format;
     enum sc_keyboard_input_mode keyboard_input_mode;
     enum sc_mouse_input_mode mouse_input_mode;
+    enum sc_gamepad_input_mode gamepad_input_mode;
+    struct sc_mouse_bindings mouse_bindings;
     enum sc_camera_facing camera_facing;
     struct sc_port_range port_range;
     uint32_t tunnel_host;
     uint16_t tunnel_port;
-    struct sc_shortcut_mods shortcut_mods;
+    uint8_t shortcut_mods; // OR of enum sc_shortcut_mod values
     uint16_t max_size;
     uint32_t video_bit_rate;
     uint32_t audio_bit_rate;
@@ -251,7 +285,6 @@ struct scrcpy_options {
     bool force_adb_forward;
     bool disable_screensaver;
     bool forward_key_repeat;
-    bool forward_all_clicks;
     bool legacy_paste;
     bool power_off_on_close;
     bool clipboard_autosync;
@@ -273,6 +306,9 @@ struct scrcpy_options {
 #define SC_OPTION_LIST_CAMERAS 0x4
 #define SC_OPTION_LIST_CAMERA_SIZES 0x8
     uint8_t list;
+    bool window;
+    bool mouse_hover;
+    bool audio_dup;
 };
 
 extern const struct scrcpy_options scrcpy_options_default;

app/src/receiver.c

@@ -1,21 +1,37 @@
 #include "receiver.h"
 
 #include <assert.h>
+#include <inttypes.h>
+#include <stdint.h>
 #include <SDL2/SDL_clipboard.h>
 
 #include "device_msg.h"
+#include "events.h"
 #include "util/log.h"
+#include "util/str.h"
+
+struct sc_uhid_output_task_data {
+    struct sc_uhid_devices *uhid_devices;
+    uint16_t id;
+    uint16_t size;
+    uint8_t *data;
+};
 
 bool
 sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
-              struct sc_acksync *acksync) {
+                 const struct sc_receiver_callbacks *cbs, void *cbs_userdata) {
     bool ok = sc_mutex_init(&receiver->mutex);
     if (!ok) {
         return false;
     }
 
     receiver->control_socket = control_socket;
-    receiver->acksync = acksync;
+    receiver->acksync = NULL;
+    receiver->uhid_devices = NULL;
+
+    assert(cbs && cbs->on_ended);
+    receiver->cbs = cbs;
+    receiver->cbs_userdata = cbs_userdata;
 
     return true;
 }
@@ -26,36 +42,125 @@ sc_receiver_destroy(struct sc_receiver *receiver) {
 }
 
 static void
-process_msg(struct sc_receiver *receiver, struct device_msg *msg) {
+task_set_clipboard(void *userdata) {
+    char *text = userdata;
+
+    char *current = SDL_GetClipboardText();
+    bool same = current && !strcmp(current, text);
+    SDL_free(current);
+    if (same) {
+        LOGD("Computer clipboard unchanged");
+        free(text);
+        return;
+    }
+
+    LOGI("Device clipboard copied");
+    SDL_SetClipboardText(text);
+    free(text);
+}
+
+static void
+task_uhid_output(void *userdata) {
+    struct sc_uhid_output_task_data *data = userdata;
+
+    sc_uhid_devices_process_hid_output(data->uhid_devices, data->id, data->data,
+                                       data->size);
+
+    free(data->data);
+    free(data);
+}
+
+static void
+process_msg(struct sc_receiver *receiver, struct sc_device_msg *msg) {
     switch (msg->type) {
         case DEVICE_MSG_TYPE_CLIPBOARD: {
-            char *current = SDL_GetClipboardText();
+            // Take ownership of the text (do not destroy the msg)
-            bool same = current && !strcmp(current, msg->clipboard.text);
+            char *text = msg->clipboard.text;
-            SDL_free(current);
+
-            if (same) {
+            bool ok = sc_post_to_main_thread(task_set_clipboard, text);
-                LOGD("Computer clipboard unchanged");
+            if (!ok) {
+                LOGW("Could not post clipboard to main thread");
+                free(text);
                 return;
             }
 
-            LOGI("Device clipboard copied");
-            SDL_SetClipboardText(msg->clipboard.text);
             break;
         }
         case DEVICE_MSG_TYPE_ACK_CLIPBOARD:
-            assert(receiver->acksync);
             LOGD("Ack device clipboard sequence=%" PRIu64_,
                  msg->ack_clipboard.sequence);
+
+            // This is a programming error to receive this message if there is
+            // no ACK synchronization mechanism
+            assert(receiver->acksync);
+
+            // Also check at runtime (do not trust the server)
+            if (!receiver->acksync) {
+                LOGE("Received unexpected ack");
+                return;
+            }
+
             sc_acksync_ack(receiver->acksync, msg->ack_clipboard.sequence);
+            // No allocation to free in the msg
+            break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
+                char *hex = sc_str_to_hex_string(msg->uhid_output.data,
+                                                 msg->uhid_output.size);
+                if (hex) {
+                    LOGV("UHID output [%" PRIu16 "] %s",
+                         msg->uhid_output.id, hex);
+                    free(hex);
+                } else {
+                    LOGV("UHID output [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_output.id, msg->uhid_output.size);
+                }
+            }
+
+            // This is a programming error to receive this message if there is
+            // no uhid_devices instance
+            assert(receiver->uhid_devices);
+
+            // Also check at runtime (do not trust the server)
+            if (!receiver->uhid_devices) {
+                LOGE("Received unexpected HID output message");
+                sc_device_msg_destroy(msg);
+                return;
+            }
+
+            struct sc_uhid_output_task_data *data = malloc(sizeof(*data));
+            if (!data) {
+                LOG_OOM();
+                return;
+            }
+
+            // It is guaranteed that these pointers will still be valid when
+            // the main thread will process them (the main thread will stop
+            // processing SC_EVENT_RUN_ON_MAIN_THREAD on exit, when everything
+            // gets deinitialized)
+            data->uhid_devices = receiver->uhid_devices;
+            data->id = msg->uhid_output.id;
+            data->data = msg->uhid_output.data; // take ownership
+            data->size = msg->uhid_output.size;
+
+            bool ok = sc_post_to_main_thread(task_uhid_output, data);
+            if (!ok) {
+                LOGW("Could not post UHID output to main thread");
+                free(data->data);
+                free(data);
+                return;
+            }
+
             break;
     }
 }
 
 static ssize_t
-process_msgs(struct sc_receiver *receiver, const unsigned char *buf, size_t len) {
+process_msgs(struct sc_receiver *receiver, const uint8_t *buf, size_t len) {
     size_t head = 0;
     for (;;) {
-        struct device_msg msg;
+        struct sc_device_msg msg;
-        ssize_t r = device_msg_deserialize(&buf[head], len - head, &msg);
+        ssize_t r = sc_device_msg_deserialize(&buf[head], len - head, &msg);
         if (r == -1) {
             return -1;
         }
@@ -64,7 +169,7 @@ process_msgs(struct sc_receiver *receiver, const unsigned char *buf, size_t len)
         }
 
         process_msg(receiver, &msg);
-        device_msg_destroy(&msg);
+        // the device msg must be destroyed by process_msg()
 
         head += r;
         assert(head <= len);
@@ -78,15 +183,18 @@ static int
 run_receiver(void *data) {
     struct sc_receiver *receiver = data;
 
-    static unsigned char buf[DEVICE_MSG_MAX_SIZE];
+    static uint8_t buf[DEVICE_MSG_MAX_SIZE];
     size_t head = 0;
 
+    bool error = false;
+
     for (;;) {
         assert(head < DEVICE_MSG_MAX_SIZE);
         ssize_t r = net_recv(receiver->control_socket, buf + head,
                              DEVICE_MSG_MAX_SIZE - head);
         if (r <= 0) {
             LOGD("Receiver stopped");
+            // device disconnected: keep error=false
             break;
         }
 
@@ -94,6 +202,7 @@ run_receiver(void *data) {
         ssize_t consumed = process_msgs(receiver, buf, head);
         if (consumed == -1) {
             // an error occurred
+            error = true;
             break;
         }
 
@@ -104,6 +213,8 @@ run_receiver(void *data) {
         }
     }
 
+    receiver->cbs->on_ended(receiver, error, receiver->cbs_userdata);
+
     return 0;
 }
 

app/src/receiver.h

@@ -5,6 +5,7 @@
 
 #include <stdbool.h>
 
+#include "uhid/uhid_output.h"
 #include "util/acksync.h"
 #include "util/net.h"
 #include "util/thread.h"
@@ -17,11 +18,19 @@ struct sc_receiver {
     sc_mutex mutex;
 
     struct sc_acksync *acksync;
+    struct sc_uhid_devices *uhid_devices;
+
+    const struct sc_receiver_callbacks *cbs;
+    void *cbs_userdata;
+};
+
+struct sc_receiver_callbacks {
+    void (*on_ended)(struct sc_receiver *receiver, bool error, void *userdata);
 };
 
 bool
 sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket,
-                 struct sc_acksync *acksync);
+                 const struct sc_receiver_callbacks *cbs, void *cbs_userdata);
 
 void
 sc_receiver_destroy(struct sc_receiver *receiver);

app/src/scrcpy.c

@@ -20,15 +20,19 @@
 #include "demuxer.h"
 #include "events.h"
 #include "file_pusher.h"
-#include "keyboard_inject.h"
+#include "keyboard_sdk.h"
-#include "mouse_inject.h"
+#include "mouse_sdk.h"
 #include "recorder.h"
 #include "screen.h"
 #include "server.h"
+#include "uhid/gamepad_uhid.h"
+#include "uhid/keyboard_uhid.h"
+#include "uhid/mouse_uhid.h"
 #ifdef HAVE_USB
 # include "usb/aoa_hid.h"
-# include "usb/hid_keyboard.h"
+# include "usb/gamepad_aoa.h"
-# include "usb/hid_mouse.h"
+# include "usb/keyboard_aoa.h"
+# include "usb/mouse_aoa.h"
 # include "usb/usb.h"
 #endif
 #include "util/acksync.h"
@@ -61,38 +65,35 @@ struct scrcpy {
     struct sc_aoa aoa;
     // sequence/ack helper to synchronize clipboard and Ctrl+v via HID
     struct sc_acksync acksync;
+    struct sc_uhid_devices uhid_devices;
 #endif
     union {
-        struct sc_keyboard_inject keyboard_inject;
+        struct sc_keyboard_sdk keyboard_sdk;
+        struct sc_keyboard_uhid keyboard_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_keyboard keyboard_hid;
+        struct sc_keyboard_aoa keyboard_aoa;
 #endif
     };
     union {
-        struct sc_mouse_inject mouse_inject;
+        struct sc_mouse_sdk mouse_sdk;
+        struct sc_mouse_uhid mouse_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_mouse mouse_hid;
+        struct sc_mouse_aoa mouse_aoa;
+#endif
+    };
+    union {
+        struct sc_gamepad_uhid gamepad_uhid;
+#ifdef HAVE_USB
+        struct sc_gamepad_aoa gamepad_aoa;
 #endif
     };
     struct sc_timeout timeout;
 };
 
-static inline void
-push_event(uint32_t type, const char *name) {
-    SDL_Event event;
-    event.type = type;
-    int ret = SDL_PushEvent(&event);
-    if (ret < 0) {
-        LOGE("Could not post %s event: %s", name, SDL_GetError());
-        // What could we do?
-    }
-}
-#define PUSH_EVENT(TYPE) push_event(TYPE, # TYPE)
-
 #ifdef _WIN32
 static BOOL WINAPI windows_ctrl_handler(DWORD ctrl_type) {
     if (ctrl_type == CTRL_C_EVENT) {
-        PUSH_EVENT(SDL_QUIT);
+        sc_push_event(SDL_QUIT);
         return TRUE;
     }
     return FALSE;
@@ -101,7 +102,6 @@ static BOOL WINAPI windows_ctrl_handler(DWORD ctrl_type) {
 
 static void
 sdl_set_hints(const char *render_driver) {
-
     if (render_driver && !SDL_SetHint(SDL_HINT_RENDER_DRIVER, render_driver)) {
         LOGW("Could not set render driver");
     }
@@ -170,15 +170,26 @@ event_loop(struct scrcpy *s) {
             case SC_EVENT_DEMUXER_ERROR:
                 LOGE("Demuxer error");
                 return SCRCPY_EXIT_FAILURE;
+            case SC_EVENT_CONTROLLER_ERROR:
+                LOGE("Controller error");
+                return SCRCPY_EXIT_FAILURE;
             case SC_EVENT_RECORDER_ERROR:
                 LOGE("Recorder error");
                 return SCRCPY_EXIT_FAILURE;
+            case SC_EVENT_AOA_OPEN_ERROR:
+                LOGE("AOA open error");
+                return SCRCPY_EXIT_FAILURE;
             case SC_EVENT_TIME_LIMIT_REACHED:
                 LOGI("Time limit reached");
                 return SCRCPY_EXIT_SUCCESS;
             case SDL_QUIT:
                 LOGD("User requested to quit");
                 return SCRCPY_EXIT_SUCCESS;
+            case SC_EVENT_RUN_ON_MAIN_THREAD:
+                sc_runnable_fn run = event.user.data1;
+                void *userdata = event.user.data2;
+                run(userdata);
+                break;
             default:
                 if (!sc_screen_handle_event(&s->screen, &event)) {
                     return SCRCPY_EXIT_FAILURE;
@@ -189,6 +200,21 @@ event_loop(struct scrcpy *s) {
     return SCRCPY_EXIT_FAILURE;
 }
 
+static void
+terminate_event_loop(void) {
+    sc_reject_new_runnables();
+
+    SDL_Event event;
+    while (SDL_PollEvent(&event)) {
+        if (event.type == SC_EVENT_RUN_ON_MAIN_THREAD) {
+            // Make sure all posted runnables are run, to avoid memory leaks
+            sc_runnable_fn run = event.user.data1;
+            void *userdata = event.user.data2;
+            run(userdata);
+        }
+    }
+}
+
 // Return true on success, false on error
 static bool
 await_for_server(bool *connected) {
@@ -223,7 +249,7 @@ sc_recorder_on_ended(struct sc_recorder *recorder, bool success,
     (void) userdata;
 
     if (!success) {
-        PUSH_EVENT(SC_EVENT_RECORDER_ERROR);
+        sc_push_event(SC_EVENT_RECORDER_ERROR);
     }
 }
 
@@ -237,9 +263,9 @@ sc_video_demuxer_on_ended(struct sc_demuxer *demuxer,
     assert(status != SC_DEMUXER_STATUS_DISABLED);
 
     if (status == SC_DEMUXER_STATUS_EOS) {
-        PUSH_EVENT(SC_EVENT_DEVICE_DISCONNECTED);
+        sc_push_event(SC_EVENT_DEVICE_DISCONNECTED);
     } else {
-        PUSH_EVENT(SC_EVENT_DEMUXER_ERROR);
+        sc_push_event(SC_EVENT_DEMUXER_ERROR);
     }
 }
 
@@ -253,11 +279,26 @@ sc_audio_demuxer_on_ended(struct sc_demuxer *demuxer,
     // Contrary to the video demuxer, keep mirroring if only the audio fails
     // (unless --require-audio is set).
     if (status == SC_DEMUXER_STATUS_EOS) {
-        PUSH_EVENT(SC_EVENT_DEVICE_DISCONNECTED);
+        sc_push_event(SC_EVENT_DEVICE_DISCONNECTED);
     } else if (status == SC_DEMUXER_STATUS_ERROR
             || (status == SC_DEMUXER_STATUS_DISABLED
                 && options->require_audio)) {
-        PUSH_EVENT(SC_EVENT_DEMUXER_ERROR);
+        sc_push_event(SC_EVENT_DEMUXER_ERROR);
+    }
+}
+
+static void
+sc_controller_on_ended(struct sc_controller *controller, bool error,
+                       void *userdata) {
+    // Note: this function may be called twice, once from the controller thread
+    // and once from the receiver thread
+    (void) controller;
+    (void) userdata;
+
+    if (error) {
+        sc_push_event(SC_EVENT_CONTROLLER_ERROR);
+    } else {
+        sc_push_event(SC_EVENT_DEVICE_DISCONNECTED);
     }
 }
 
@@ -266,7 +307,7 @@ sc_server_on_connection_failed(struct sc_server *server, void *userdata) {
     (void) server;
     (void) userdata;
 
-    PUSH_EVENT(SC_EVENT_SERVER_CONNECTION_FAILED);
+    sc_push_event(SC_EVENT_SERVER_CONNECTION_FAILED);
 }
 
 static void
@@ -274,7 +315,7 @@ sc_server_on_connected(struct sc_server *server, void *userdata) {
     (void) server;
     (void) userdata;
 
-    PUSH_EVENT(SC_EVENT_SERVER_CONNECTED);
+    sc_push_event(SC_EVENT_SERVER_CONNECTED);
 }
 
 static void
@@ -292,7 +333,7 @@ sc_timeout_on_timeout(struct sc_timeout *timeout, void *userdata) {
     (void) timeout;
     (void) userdata;
 
-    PUSH_EVENT(SC_EVENT_TIME_LIMIT_REACHED);
+    sc_push_event(SC_EVENT_TIME_LIMIT_REACHED);
 }
 
 // Generate a scrcpy id to differentiate multiple running scrcpy instances
@@ -304,9 +345,28 @@ scrcpy_generate_scid(void) {
     return sc_rand_u32(&rand) & 0x7FFFFFFF;
 }
 
+static void
+init_sdl_gamepads(void) {
+    // Trigger a SDL_CONTROLLERDEVICEADDED event for all gamepads already
+    // connected
+    int num_joysticks = SDL_NumJoysticks();
+    for (int i = 0; i < num_joysticks; ++i) {
+        if (SDL_IsGameController(i)) {
+            SDL_Event event;
+            event.cdevice.type = SDL_CONTROLLERDEVICEADDED;
+            event.cdevice.which = i;
+            SDL_PushEvent(&event);
+        }
+    }
+}
+
 enum scrcpy_exit_code
 scrcpy(struct scrcpy_options *options) {
     static struct scrcpy scrcpy;
+#ifndef NDEBUG
+    // Detect missing initializations
+    memset(&scrcpy, 42, sizeof(scrcpy));
+#endif
     struct scrcpy *s = &scrcpy;
 
     // Minimal SDL initialization
@@ -330,8 +390,9 @@ scrcpy(struct scrcpy_options *options) {
     bool audio_demuxer_started = false;
 #ifdef HAVE_USB
     bool aoa_hid_initialized = false;
-    bool hid_keyboard_initialized = false;
+    bool keyboard_aoa_initialized = false;
-    bool hid_mouse_initialized = false;
+    bool mouse_aoa_initialized = false;
+    bool gamepad_aoa_initialized = false;
 #endif
     bool controller_initialized = false;
     bool controller_started = false;
@@ -367,6 +428,7 @@ scrcpy(struct scrcpy_options *options) {
         .display_id = options->display_id,
         .video = options->video,
         .audio = options->audio,
+        .audio_dup = options->audio_dup,
         .show_touches = options->show_touches,
         .stay_awake = options->stay_awake,
         .video_codec_options = options->video_codec_options,
@@ -399,6 +461,12 @@ scrcpy(struct scrcpy_options *options) {
         return SCRCPY_EXIT_FAILURE;
     }
 
+    if (options->window) {
+        // Set hints before starting the server thread to avoid race conditions
+        // in SDL
+        sdl_set_hints(options->render_driver);
+    }
+
     if (!sc_server_start(&s->server)) {
         goto end;
     }
@@ -415,11 +483,7 @@ scrcpy(struct scrcpy_options *options) {
     assert(!options->video_playback || options->video);
     assert(!options->audio_playback || options->audio);
 
-    if (options->video_playback) {
+    if (options->window ||
-        sdl_set_hints(options->render_driver);
-    }
-
-    if (options->video_playback ||
             (options->control && options->clipboard_autosync)) {
         // Initialize the video subsystem even if --no-video or
         // --no-video-playback is passed so that clipboard synchronization
@@ -443,6 +507,13 @@ scrcpy(struct scrcpy_options *options) {
         }
     }
 
+    if (options->gamepad_input_mode != SC_GAMEPAD_INPUT_MODE_DISABLED) {
+        if (SDL_Init(SDL_INIT_GAMECONTROLLER)) {
+            LOGE("Could not initialize SDL gamepad: %s", SDL_GetError());
+            goto end;
+        }
+    }
+
     sdl_configure(options->video_playback, options->disable_screensaver);
 
     // Await for server without blocking Ctrl+C handling
@@ -540,14 +611,29 @@ scrcpy(struct scrcpy_options *options) {
     struct sc_controller *controller = NULL;
     struct sc_key_processor *kp = NULL;
     struct sc_mouse_processor *mp = NULL;
+    struct sc_gamepad_processor *gp = NULL;
 
     if (options->control) {
+        static const struct sc_controller_callbacks controller_cbs = {
+            .on_ended = sc_controller_on_ended,
+        };
+
+        if (!sc_controller_init(&s->controller, s->server.control_socket,
+            &controller_cbs, NULL)) {
+            goto end;
+        }
+        controller_initialized = true;
+
+        controller = &s->controller;
+
 #ifdef HAVE_USB
-        bool use_hid_keyboard =
+        bool use_keyboard_aoa =
-            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
+            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
-        bool use_hid_mouse =
+        bool use_mouse_aoa =
-            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
+            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
-        if (use_hid_keyboard || use_hid_mouse) {
+        bool use_gamepad_aoa =
+            options->gamepad_input_mode == SC_GAMEPAD_INPUT_MODE_AOA;
+        if (use_keyboard_aoa || use_mouse_aoa || use_gamepad_aoa) {
             bool ok = sc_acksync_init(&s->acksync);
             if (!ok) {
                 goto end;
@@ -557,7 +643,7 @@ scrcpy(struct scrcpy_options *options) {
             if (!ok) {
                 LOGE("Failed to initialize USB");
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             assert(serial);
@@ -565,7 +651,7 @@ scrcpy(struct scrcpy_options *options) {
             ok = sc_usb_select_device(&s->usb, serial, &usb_device);
             if (!ok) {
                 sc_usb_destroy(&s->usb);
-                goto aoa_hid_end;
+                goto end;
             }
 
             LOGI("USB device: %s (%04" PRIx16 ":%04" PRIx16 ") %s %s",
@@ -578,7 +664,7 @@ scrcpy(struct scrcpy_options *options) {
                 LOGE("Failed to connect to USB device %s", serial);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             ok = sc_aoa_init(&s->aoa, &s->usb, &s->acksync);
@@ -587,113 +673,124 @@ scrcpy(struct scrcpy_options *options) {
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
-            if (use_hid_keyboard) {
+            bool aoa_fail = false;
-                if (sc_hid_keyboard_init(&s->keyboard_hid, &s->aoa)) {
+            if (use_keyboard_aoa) {
-                    hid_keyboard_initialized = true;
+                if (sc_keyboard_aoa_init(&s->keyboard_aoa, &s->aoa)) {
-                    kp = &s->keyboard_hid.key_processor;
+                    keyboard_aoa_initialized = true;
+                    kp = &s->keyboard_aoa.key_processor;
                 } else {
                     LOGE("Could not initialize HID keyboard");
+                    aoa_fail = true;
+                    goto aoa_complete;
                 }
             }
 
-            if (use_hid_mouse) {
+            if (use_mouse_aoa) {
-                if (sc_hid_mouse_init(&s->mouse_hid, &s->aoa)) {
+                if (sc_mouse_aoa_init(&s->mouse_aoa, &s->aoa)) {
-                    hid_mouse_initialized = true;
+                    mouse_aoa_initialized = true;
-                    mp = &s->mouse_hid.mouse_processor;
+                    mp = &s->mouse_aoa.mouse_processor;
                 } else {
                     LOGE("Could not initialized HID mouse");
+                    aoa_fail = true;
+                    goto aoa_complete;
                 }
             }
 
-            bool need_aoa = hid_keyboard_initialized || hid_mouse_initialized;
+            if (use_gamepad_aoa) {
+                sc_gamepad_aoa_init(&s->gamepad_aoa, &s->aoa);
+                gp = &s->gamepad_aoa.gamepad_processor;
+                gamepad_aoa_initialized = true;
+            }
 
-            if (!need_aoa || !sc_aoa_start(&s->aoa)) {
+aoa_complete:
+            if (aoa_fail || !sc_aoa_start(&s->aoa)) {
                 sc_acksync_destroy(&s->acksync);
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_aoa_destroy(&s->aoa);
-                goto aoa_hid_end;
+                goto end;
             }
 
             acksync = &s->acksync;
 
             aoa_hid_initialized = true;
-
-aoa_hid_end:
-            if (!aoa_hid_initialized) {
-                if (hid_keyboard_initialized) {
-                    sc_hid_keyboard_destroy(&s->keyboard_hid);
-                    hid_keyboard_initialized = false;
-                }
-                if (hid_mouse_initialized) {
-                    sc_hid_mouse_destroy(&s->mouse_hid);
-                    hid_mouse_initialized = false;
-                }
-            }
-
-            if (use_hid_keyboard && !hid_keyboard_initialized) {
-                LOGE("Fallback to default keyboard injection method "
-                     "(-K/--hid-keyboard ignored)");
-                options->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT;
-            }
-
-            if (use_hid_mouse && !hid_mouse_initialized) {
-                LOGE("Fallback to default mouse injection method "
-                     "(-M/--hid-mouse ignored)");
-                options->mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT;
-            }
         }
 #else
-        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_HID);
+        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_AOA);
-        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_HID);
+        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_AOA);
 #endif
 
-        // keyboard_input_mode may have been reset if HID mode failed
+        struct sc_keyboard_uhid *uhid_keyboard = NULL;
-        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_INJECT) {
+        struct sc_gamepad_uhid *uhid_gamepad = NULL;
-            sc_keyboard_inject_init(&s->keyboard_inject, &s->controller,
+
-                                    options->key_inject_mode,
+        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_SDK) {
-                                    options->forward_key_repeat);
+            sc_keyboard_sdk_init(&s->keyboard_sdk, &s->controller,
-            kp = &s->keyboard_inject.key_processor;
+                                 options->key_inject_mode,
+                                 options->forward_key_repeat);
+            kp = &s->keyboard_sdk.key_processor;
+        } else if (options->keyboard_input_mode
+                == SC_KEYBOARD_INPUT_MODE_UHID) {
+            bool ok = sc_keyboard_uhid_init(&s->keyboard_uhid, &s->controller);
+            if (!ok) {
+                goto end;
+            }
+            kp = &s->keyboard_uhid.key_processor;
+            uhid_keyboard = &s->keyboard_uhid;
         }
 
-        // mouse_input_mode may have been reset if HID mode failed
+        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_SDK) {
-        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_INJECT) {
+            sc_mouse_sdk_init(&s->mouse_sdk, &s->controller,
-            sc_mouse_inject_init(&s->mouse_inject, &s->controller);
+                              options->mouse_hover);
-            mp = &s->mouse_inject.mouse_processor;
+            mp = &s->mouse_sdk.mouse_processor;
+        } else if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_UHID) {
+            bool ok = sc_mouse_uhid_init(&s->mouse_uhid, &s->controller);
+            if (!ok) {
+                goto end;
+            }
+            mp = &s->mouse_uhid.mouse_processor;
         }
 
-        if (!sc_controller_init(&s->controller, s->server.control_socket,
+        if (options->gamepad_input_mode == SC_GAMEPAD_INPUT_MODE_UHID) {
-                                acksync)) {
+            sc_gamepad_uhid_init(&s->gamepad_uhid, &s->controller);
-            goto end;
+            gp = &s->gamepad_uhid.gamepad_processor;
+            uhid_gamepad = &s->gamepad_uhid;
         }
-        controller_initialized = true;
+
+        struct sc_uhid_devices *uhid_devices = NULL;
+        if (uhid_keyboard || uhid_gamepad) {
+            sc_uhid_devices_init(&s->uhid_devices, uhid_keyboard, uhid_gamepad);
+            uhid_devices = &s->uhid_devices;
+        }
+
+        sc_controller_configure(&s->controller, acksync, uhid_devices);
 
         if (!sc_controller_start(&s->controller)) {
             goto end;
         }
         controller_started = true;
-        controller = &s->controller;
     }
 
     // There is a controller if and only if control is enabled
     assert(options->control == !!controller);
 
-    if (options->video_playback) {
+    if (options->window) {
         const char *window_title =
             options->window_title ? options->window_title : info->device_name;
 
         struct sc_screen_params screen_params = {
+            .video = options->video_playback,
             .controller = controller,
             .fp = fp,
             .kp = kp,
             .mp = mp,
-            .forward_all_clicks = options->forward_all_clicks,
+            .gp = gp,
+            .mouse_bindings = options->mouse_bindings,
             .legacy_paste = options->legacy_paste,
             .clipboard_autosync = options->clipboard_autosync,
-            .shortcut_mods = &options->shortcut_mods,
+            .shortcut_mods = options->shortcut_mods,
             .window_title = window_title,
             .always_on_top = options->always_on_top,
             .window_x = options->window_x,
@@ -707,20 +804,22 @@ aoa_hid_end:
             .start_fps_counter = options->start_fps_counter,
         };
 
-        struct sc_frame_source *src = &s->video_decoder.frame_source;
-        if (options->display_buffer) {
-            sc_delay_buffer_init(&s->display_buffer, options->display_buffer,
-                                 true);
-            sc_frame_source_add_sink(src, &s->display_buffer.frame_sink);
-            src = &s->display_buffer.frame_source;
-        }
-
         if (!sc_screen_init(&s->screen, &screen_params)) {
             goto end;
         }
         screen_initialized = true;
 
-        sc_frame_source_add_sink(src, &s->screen.frame_sink);
+        if (options->video_playback) {
+            struct sc_frame_source *src = &s->video_decoder.frame_source;
+            if (options->display_buffer) {
+                sc_delay_buffer_init(&s->display_buffer,
+                                     options->display_buffer, true);
+                sc_frame_source_add_sink(src, &s->display_buffer.frame_sink);
+                src = &s->display_buffer.frame_source;
+            }
+
+            sc_frame_source_add_sink(src, &s->screen.frame_sink);
+        }
     }
 
     if (options->audio_playback) {
@@ -799,12 +898,21 @@ aoa_hid_end:
         timeout_started = true;
     }
 
+    bool use_gamepads = true;
+    if (use_gamepads) {
+        init_sdl_gamepads();
+    }
+
     ret = event_loop(s);
+    terminate_event_loop();
     LOGD("quit...");
 
-    // Close the window immediately on closing, because screen_destroy() may
+    if (options->video_playback) {
-    // only be called once the video demuxer thread is joined (it may take time)
+        // Close the window immediately on closing, because screen_destroy()
-    sc_screen_hide_window(&s->screen);
+        // may only be called once the video demuxer thread is joined (it may
+        // take time)
+        sc_screen_hide_window(&s->screen);
+    }
 
 end:
     if (timeout_started) {
@@ -815,11 +923,14 @@ end:
     // end-of-stream
 #ifdef HAVE_USB
     if (aoa_hid_initialized) {
-        if (hid_keyboard_initialized) {
+        if (keyboard_aoa_initialized) {
-            sc_hid_keyboard_destroy(&s->keyboard_hid);
+            sc_keyboard_aoa_destroy(&s->keyboard_aoa);
         }
-        if (hid_mouse_initialized) {
+        if (mouse_aoa_initialized) {
-            sc_hid_mouse_destroy(&s->mouse_hid);
+            sc_mouse_aoa_destroy(&s->mouse_aoa);
+        }
+        if (gamepad_aoa_initialized) {
+            sc_gamepad_aoa_destroy(&s->gamepad_aoa);
         }
         sc_aoa_stop(&s->aoa);
         sc_usb_stop(&s->usb);

app/src/screen.c

@@ -205,6 +205,8 @@ sc_screen_toggle_mouse_capture(struct sc_screen *screen) {
 
 static void
 sc_screen_update_content_rect(struct sc_screen *screen) {
+    assert(screen->video);
+
     int dw;
     int dh;
     SDL_GL_GetDrawableSize(screen->window, &dw, &dh);
@@ -246,6 +248,8 @@ sc_screen_update_content_rect(struct sc_screen *screen) {
 // changed, so that the content rectangle is recomputed
 static void
 sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
+    assert(screen->video);
+
     if (update_content_rect) {
         sc_screen_update_content_rect(screen);
     }
@@ -255,6 +259,13 @@ sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
     (void) res; // any error already logged
 }
 
+static void
+sc_screen_render_novideo(struct sc_screen *screen) {
+    enum sc_display_result res =
+        sc_display_render(&screen->display, NULL, SC_ORIENTATION_0);
+    (void) res; // any error already logged
+}
+
 #if defined(__APPLE__) || defined(__WINDOWS__)
 # define CONTINUOUS_RESIZING_WORKAROUND
 #endif
@@ -268,6 +279,8 @@ sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
 static int
 event_watcher(void *data, SDL_Event *event) {
     struct sc_screen *screen = data;
+    assert(screen->video);
+
     if (event->type == SDL_WINDOWEVENT
             && event->window.event == SDL_WINDOWEVENT_RESIZED) {
         // In practice, it seems to always be called from the same thread in
@@ -293,13 +306,9 @@ sc_screen_frame_sink_open(struct sc_frame_sink *sink,
     screen->frame_size.width = ctx->width;
     screen->frame_size.height = ctx->height;
 
-    static SDL_Event event = {
-        .type = SC_EVENT_SCREEN_INIT_SIZE,
-    };
-
     // Post the event on the UI thread (the texture must be created from there)
-    int ret = SDL_PushEvent(&event);
+    bool ok = sc_push_event(SC_EVENT_SCREEN_INIT_SIZE);
-    if (ret < 0) {
+    if (!ok) {
         LOGW("Could not post init size event: %s", SDL_GetError());
         return false;
     }
@@ -326,6 +335,7 @@ sc_screen_frame_sink_close(struct sc_frame_sink *sink) {
 static bool
 sc_screen_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
     struct sc_screen *screen = DOWNCAST(sink);
+    assert(screen->video);
 
     bool previous_skipped;
     bool ok = sc_frame_buffer_push(&screen->fb, frame, &previous_skipped);
@@ -338,13 +348,9 @@ sc_screen_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
         // The SC_EVENT_NEW_FRAME triggered for the previous frame will consume
         // this new frame instead
     } else {
-        static SDL_Event new_frame_event = {
-            .type = SC_EVENT_NEW_FRAME,
-        };
-
         // Post the event on the UI thread
-        int ret = SDL_PushEvent(&new_frame_event);
+        bool ok = sc_push_event(SC_EVENT_NEW_FRAME);
-        if (ret < 0) {
+        if (!ok) {
             LOGW("Could not post new frame event: %s", SDL_GetError());
             return false;
         }
@@ -362,6 +368,11 @@ sc_screen_init(struct sc_screen *screen,
     screen->maximized = false;
     screen->minimized = false;
     screen->mouse_capture_key_pressed = 0;
+    screen->paused = false;
+    screen->resume_frame = NULL;
+    screen->orientation = SC_ORIENTATION_0;
+
+    screen->video = params->video;
 
     screen->req.x = params->window_x;
     screen->req.y = params->window_y;
@@ -379,41 +390,75 @@ sc_screen_init(struct sc_screen *screen,
         goto error_destroy_frame_buffer;
     }
 
-    screen->orientation = params->orientation;
+    if (screen->video) {
-    if (screen->orientation != SC_ORIENTATION_0) {
+        screen->orientation = params->orientation;
-        LOGI("Initial display orientation set to %s",
+        if (screen->orientation != SC_ORIENTATION_0) {
-             sc_orientation_get_name(screen->orientation));
+            LOGI("Initial display orientation set to %s",
+                 sc_orientation_get_name(screen->orientation));
+        }
     }
 
-    uint32_t window_flags = SDL_WINDOW_HIDDEN
+    uint32_t window_flags = SDL_WINDOW_ALLOW_HIGHDPI;
-                          | SDL_WINDOW_RESIZABLE
-                          | SDL_WINDOW_ALLOW_HIGHDPI;
     if (params->always_on_top) {
         window_flags |= SDL_WINDOW_ALWAYS_ON_TOP;
     }
     if (params->window_borderless) {
         window_flags |= SDL_WINDOW_BORDERLESS;
     }
+    if (params->video) {
+        // The window will be shown on first frame
+        window_flags |= SDL_WINDOW_HIDDEN
+                      | SDL_WINDOW_RESIZABLE;
+    }
+
+    const char *title = params->window_title;
+    assert(title);
+
+    int x = SDL_WINDOWPOS_UNDEFINED;
+    int y = SDL_WINDOWPOS_UNDEFINED;
+    int width = 256;
+    int height = 256;
+    if (params->window_x != SC_WINDOW_POSITION_UNDEFINED) {
+        x = params->window_x;
+    }
+    if (params->window_y != SC_WINDOW_POSITION_UNDEFINED) {
+        y = params->window_y;
+    }
+    if (params->window_width) {
+        width = params->window_width;
+    }
+    if (params->window_height) {
+        height = params->window_height;
+    }
 
     // The window will be positioned and sized on first video frame
-    screen->window =
+    screen->window = SDL_CreateWindow(title, x, y, width, height, window_flags);
-        SDL_CreateWindow(params->window_title, 0, 0, 0, 0, window_flags);
     if (!screen->window) {
         LOGE("Could not create window: %s", SDL_GetError());
         goto error_destroy_fps_counter;
     }
 
-    ok = sc_display_init(&screen->display, screen->window, params->mipmaps);
-    if (!ok) {
-        goto error_destroy_window;
-    }
-
     SDL_Surface *icon = scrcpy_icon_load();
     if (icon) {
         SDL_SetWindowIcon(screen->window, icon);
-        scrcpy_icon_destroy(icon);
+    } else if (params->video) {
-    } else {
+        // just a warning
         LOGW("Could not load icon");
+    } else {
+        // without video, the icon is used as window content, it must be present
+        LOGE("Could not load icon");
+        goto error_destroy_fps_counter;
+    }
+
+    SDL_Surface *icon_novideo = params->video ? NULL : icon;
+    bool mipmaps = params->video && params->mipmaps;
+    ok = sc_display_init(&screen->display, screen->window, icon_novideo,
+                         mipmaps);
+    if (icon) {
+        scrcpy_icon_destroy(icon);
+    }
+    if (!ok) {
+        goto error_destroy_window;
     }
 
     screen->frame = av_frame_alloc();
@@ -428,7 +473,8 @@ sc_screen_init(struct sc_screen *screen,
         .screen = screen,
         .kp = params->kp,
         .mp = params->mp,
-        .forward_all_clicks = params->forward_all_clicks,
+        .gp = params->gp,
+        .mouse_bindings = params->mouse_bindings,
         .legacy_paste = params->legacy_paste,
         .clipboard_autosync = params->clipboard_autosync,
         .shortcut_mods = params->shortcut_mods,
@@ -437,7 +483,9 @@ sc_screen_init(struct sc_screen *screen,
     sc_input_manager_init(&screen->im, &im_params);
 
 #ifdef CONTINUOUS_RESIZING_WORKAROUND
-    SDL_AddEventWatch(event_watcher, screen);
+    if (screen->video) {
+        SDL_AddEventWatch(event_watcher, screen);
+    }
 #endif
 
     static const struct sc_frame_sink_ops ops = {
@@ -452,6 +500,11 @@ sc_screen_init(struct sc_screen *screen,
     screen->open = false;
 #endif
 
+    if (!screen->video && sc_screen_is_relative_mode(screen)) {
+        // Capture mouse immediately if video mirroring is disabled
+        sc_screen_set_mouse_capture(screen, true);
+    }
+
     return true;
 
 error_destroy_display:
@@ -522,6 +575,8 @@ sc_screen_destroy(struct sc_screen *screen) {
 static void
 resize_for_content(struct sc_screen *screen, struct sc_size old_content_size,
                    struct sc_size new_content_size) {
+    assert(screen->video);
+
     struct sc_size window_size = get_window_size(screen);
     struct sc_size target_size = {
         .width = (uint32_t) window_size.width * new_content_size.width
@@ -535,6 +590,8 @@ resize_for_content(struct sc_screen *screen, struct sc_size old_content_size,
 
 static void
 set_content_size(struct sc_screen *screen, struct sc_size new_content_size) {
+    assert(screen->video);
+
     if (!screen->fullscreen && !screen->maximized && !screen->minimized) {
         resize_for_content(screen, screen->content_size, new_content_size);
     } else if (!screen->resize_pending) {
@@ -549,6 +606,8 @@ set_content_size(struct sc_screen *screen, struct sc_size new_content_size) {
 
 static void
 apply_pending_resize(struct sc_screen *screen) {
+    assert(screen->video);
+
     assert(!screen->fullscreen);
     assert(!screen->maximized);
     assert(!screen->minimized);
@@ -562,6 +621,8 @@ apply_pending_resize(struct sc_screen *screen) {
 void
 sc_screen_set_orientation(struct sc_screen *screen,
                           enum sc_orientation orientation) {
+    assert(screen->video);
+
     if (orientation == screen->orientation) {
         return;
     }
@@ -596,6 +657,8 @@ sc_screen_init_size(struct sc_screen *screen) {
 // recreate the texture and resize the window if the frame size has changed
 static enum sc_display_result
 prepare_for_frame(struct sc_screen *screen, struct sc_size new_frame_size) {
+    assert(screen->video);
+
     if (screen->frame_size.width == new_frame_size.width
             && screen->frame_size.height == new_frame_size.height) {
         return SC_DISPLAY_RESULT_OK;
@@ -614,13 +677,12 @@ prepare_for_frame(struct sc_screen *screen, struct sc_size new_frame_size) {
 }
 
 static bool
-sc_screen_update_frame(struct sc_screen *screen) {
+sc_screen_apply_frame(struct sc_screen *screen) {
-    av_frame_unref(screen->frame);
+    assert(screen->video);
-    sc_frame_buffer_consume(&screen->fb, screen->frame);
-    AVFrame *frame = screen->frame;
 
     sc_fps_counter_add_rendered_frame(&screen->fps_counter);
 
+    AVFrame *frame = screen->frame;
     struct sc_size new_frame_size = {frame->width, frame->height};
     enum sc_display_result res = prepare_for_frame(screen, new_frame_size);
     if (res == SC_DISPLAY_RESULT_ERROR) {
@@ -655,8 +717,62 @@ sc_screen_update_frame(struct sc_screen *screen) {
     return true;
 }
 
+static bool
+sc_screen_update_frame(struct sc_screen *screen) {
+    assert(screen->video);
+
+    if (screen->paused) {
+        if (!screen->resume_frame) {
+            screen->resume_frame = av_frame_alloc();
+            if (!screen->resume_frame) {
+                LOG_OOM();
+                return false;
+            }
+        } else {
+            av_frame_unref(screen->resume_frame);
+        }
+        sc_frame_buffer_consume(&screen->fb, screen->resume_frame);
+        return true;
+    }
+
+    av_frame_unref(screen->frame);
+    sc_frame_buffer_consume(&screen->fb, screen->frame);
+    return sc_screen_apply_frame(screen);
+}
+
+void
+sc_screen_set_paused(struct sc_screen *screen, bool paused) {
+    assert(screen->video);
+
+    if (!paused && !screen->paused) {
+        // nothing to do
+        return;
+    }
+
+    if (screen->paused && screen->resume_frame) {
+        // If display screen was paused, refresh the frame immediately, even if
+        // the new state is also paused.
+        av_frame_free(&screen->frame);
+        screen->frame = screen->resume_frame;
+        screen->resume_frame = NULL;
+        sc_screen_apply_frame(screen);
+    }
+
+    if (!paused) {
+        LOGI("Display screen unpaused");
+    } else if (!screen->paused) {
+        LOGI("Display screen paused");
+    } else {
+        LOGI("Display screen re-paused");
+    }
+
+    screen->paused = paused;
+}
+
 void
 sc_screen_switch_fullscreen(struct sc_screen *screen) {
+    assert(screen->video);
+
     uint32_t new_mode = screen->fullscreen ? 0 : SDL_WINDOW_FULLSCREEN_DESKTOP;
     if (SDL_SetWindowFullscreen(screen->window, new_mode)) {
         LOGW("Could not switch fullscreen mode: %s", SDL_GetError());
@@ -674,6 +790,8 @@ sc_screen_switch_fullscreen(struct sc_screen *screen) {
 
 void
 sc_screen_resize_to_fit(struct sc_screen *screen) {
+    assert(screen->video);
+
     if (screen->fullscreen || screen->maximized || screen->minimized) {
         return;
     }
@@ -698,6 +816,8 @@ sc_screen_resize_to_fit(struct sc_screen *screen) {
 
 void
 sc_screen_resize_to_pixel_perfect(struct sc_screen *screen) {
+    assert(screen->video);
+
     if (screen->fullscreen || screen->minimized) {
         return;
     }
@@ -741,6 +861,13 @@ sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event) {
             return true;
         }
         case SDL_WINDOWEVENT:
+            if (!screen->video
+                    && event->window.event == SDL_WINDOWEVENT_EXPOSED) {
+                sc_screen_render_novideo(screen);
+            }
+
+            // !video implies !has_frame
+            assert(screen->video || !screen->has_frame);
             if (!screen->has_frame) {
                 // Do nothing
                 return true;
@@ -844,6 +971,8 @@ sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event) {
 struct sc_point
 sc_screen_convert_drawable_to_frame_coords(struct sc_screen *screen,
                                            int32_t x, int32_t y) {
+    assert(screen->video);
+
     enum sc_orientation orientation = screen->orientation;
 
     int32_t w = screen->content_size.width;

app/src/screen.h

@@ -26,6 +26,8 @@ struct sc_screen {
     bool open; // track the open/close state to assert correct behavior
 #endif
 
+    bool video;
+
     struct sc_display display;
     struct sc_input_manager im;
     struct sc_frame_buffer fb;
@@ -64,18 +66,24 @@ struct sc_screen {
     SDL_Keycode mouse_capture_key_pressed;
 
     AVFrame *frame;
+
+    bool paused;
+    AVFrame *resume_frame;
 };
 
 struct sc_screen_params {
+    bool video;
+
     struct sc_controller *controller;
     struct sc_file_pusher *fp;
     struct sc_key_processor *kp;
     struct sc_mouse_processor *mp;
+    struct sc_gamepad_processor *gp;
 
-    bool forward_all_clicks;
+    struct sc_mouse_bindings mouse_bindings;
     bool legacy_paste;
     bool clipboard_autosync;
-    const struct sc_shortcut_mods *shortcut_mods;
+    uint8_t shortcut_mods; // OR of enum sc_shortcut_mod values
 
     const char *window_title;
     bool always_on_top;
@@ -135,6 +143,10 @@ void
 sc_screen_set_orientation(struct sc_screen *screen,
                           enum sc_orientation orientation);
 
+// set the display pause state
+void
+sc_screen_set_paused(struct sc_screen *screen, bool paused);
+
 // react to SDL events
 // If this function returns false, scrcpy must exit with an error.
 bool

app/src/server.c

@@ -147,7 +147,7 @@ log_level_to_server_string(enum sc_log_level level) {
             return "error";
         default:
             assert(!"unexpected log level");
-            return "(unknown)";
+            return NULL;
     }
 }
 
@@ -183,6 +183,7 @@ sc_server_get_codec_name(enum sc_codec codec) {
         case SC_CODEC_RAW:
             return "raw";
         default:
+            assert(!"unexpected codec");
             return NULL;
     }
 }
@@ -197,6 +198,22 @@ sc_server_get_camera_facing_name(enum sc_camera_facing camera_facing) {
         case SC_CAMERA_FACING_EXTERNAL:
             return "external";
         default:
+            assert(!"unexpected camera facing");
+            return NULL;
+    }
+}
+
+static const char *
+sc_server_get_audio_source_name(enum sc_audio_source audio_source) {
+    switch (audio_source) {
+        case SC_AUDIO_SOURCE_OUTPUT:
+            return "output";
+        case SC_AUDIO_SOURCE_MIC:
+            return "mic";
+        case SC_AUDIO_SOURCE_PLAYBACK:
+            return "playback";
+        default:
+            assert(!"unexpected audio source");
             return NULL;
     }
 }
@@ -271,8 +288,14 @@ execute_server(struct sc_server *server,
         assert(params->video_source == SC_VIDEO_SOURCE_CAMERA);
         ADD_PARAM("video_source=camera");
     }
-    if (params->audio_source == SC_AUDIO_SOURCE_MIC) {
+    // If audio is enabled, an "auto" audio source must have been resolved
-        ADD_PARAM("audio_source=mic");
+    assert(params->audio_source != SC_AUDIO_SOURCE_AUTO || !params->audio);
+    if (params->audio_source != SC_AUDIO_SOURCE_OUTPUT && params->audio) {
+        ADD_PARAM("audio_source=%s",
+                  sc_server_get_audio_source_name(params->audio_source));
+    }
+    if (params->audio_dup) {
+        ADD_PARAM("audio_dup=true");
     }
     if (params->max_size) {
         ADD_PARAM("max_size=%" PRIu16, params->max_size);
@@ -498,7 +521,7 @@ sc_server_init(struct sc_server *server, const struct sc_server_params *params,
 static bool
 device_read_info(struct sc_intr *intr, sc_socket device_socket,
                  struct sc_server_info *info) {
-    unsigned char buf[SC_DEVICE_NAME_FIELD_LENGTH];
+    uint8_t buf[SC_DEVICE_NAME_FIELD_LENGTH];
     ssize_t r = net_recv_all_intr(intr, device_socket, buf, sizeof(buf));
     if (r < SC_DEVICE_NAME_FIELD_LENGTH) {
         LOGE("Could not retrieve device information");

app/src/server.h

@@ -50,6 +50,7 @@ struct sc_server_params {
     uint32_t display_id;
     bool video;
     bool audio;
+    bool audio_dup;
     bool show_touches;
     bool stay_awake;
     bool force_adb_forward;

app/src/sys/win/process.c

@@ -176,6 +176,8 @@ sc_process_execute_p(const char *const argv[], HANDLE *handle, unsigned flags,
         free(lpAttributeList);
     }
 
+    CloseHandle(pi.hThread);
+
     // These handles are used by the child process, close them for this process
     if (pin) {
         CloseHandle(stdin_read_handle);

app/src/trait/gamepad_processor.h

@@ -0,0 +1,50 @@
+#ifndef SC_GAMEPAD_PROCESSOR_H
+#define SC_GAMEPAD_PROCESSOR_H
+
+#include "common.h"
+
+#include <assert.h>
+#include <stdbool.h>
+
+#include "input_events.h"
+
+/**
+ * Gamepad processor trait.
+ *
+ * Component able to handle gamepads devices and inject buttons and axis events.
+ */
+struct sc_gamepad_processor {
+    const struct sc_gamepad_processor_ops *ops;
+};
+
+struct sc_gamepad_processor_ops {
+
+    /**
+     * Process a gamepad device added or removed
+     *
+     * This function is mandatory.
+     */
+    void
+    (*process_gamepad_device)(struct sc_gamepad_processor *gp,
+                              const struct sc_gamepad_device_event *event);
+
+    /**
+     * Process a gamepad axis event
+     *
+     * This function is mandatory.
+     */
+    void
+    (*process_gamepad_axis)(struct sc_gamepad_processor *gp,
+                            const struct sc_gamepad_axis_event *event);
+
+    /**
+     * Process a gamepad button event
+     *
+     * This function is mandatory.
+     */
+    void
+    (*process_gamepad_button)(struct sc_gamepad_processor *gp,
+                              const struct sc_gamepad_button_event *event);
+};
+
+#endif

app/src/trait/key_processor.h

@@ -23,6 +23,13 @@ struct sc_key_processor {
      */
     bool async_paste;
 
+    /**
+     * Set by the implementation to indicate that the keyboard is HID. In
+     * practice, it is used to react on a shortcut to open the hard keyboard
+     * settings only if the keyboard is HID.
+     */
+    bool hid;
+
     const struct sc_key_processor_ops *ops;
 };
 

app/src/uhid/gamepad_uhid.c

@@ -0,0 +1,140 @@
+#include "gamepad_uhid.h"
+
+#include "hid/hid_gamepad.h"
+#include "input_events.h"
+#include "util/log.h"
+#include "util/str.h"
+
+/** Downcast gamepad processor to sc_gamepad_uhid */
+#define DOWNCAST(GP) container_of(GP, struct sc_gamepad_uhid, gamepad_processor)
+
+static void
+sc_gamepad_uhid_send_input(struct sc_gamepad_uhid *gamepad,
+                           const struct sc_hid_input *hid_input,
+                           const char *name) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = hid_input->hid_id;
+
+    assert(hid_input->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, hid_input->data, hid_input->size);
+    msg.uhid_input.size = hid_input->size;
+
+    if (!sc_controller_push_msg(gamepad->controller, &msg)) {
+        LOGE("Could not push UHID_INPUT message (%s)", name);
+    }
+}
+
+static void
+sc_gamepad_uhid_send_open(struct sc_gamepad_uhid *gamepad,
+                          const struct sc_hid_open *hid_open) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = hid_open->hid_id;
+    msg.uhid_create.name = hid_open->name;
+    msg.uhid_create.report_desc = hid_open->report_desc;
+    msg.uhid_create.report_desc_size = hid_open->report_desc_size;
+
+    if (!sc_controller_push_msg(gamepad->controller, &msg)) {
+        LOGE("Could not push UHID_CREATE message (gamepad)");
+    }
+}
+
+static void
+sc_gamepad_uhid_send_close(struct sc_gamepad_uhid *gamepad,
+                           const struct sc_hid_close *hid_close) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_DESTROY;
+    msg.uhid_create.id = hid_close->hid_id;
+
+    if (!sc_controller_push_msg(gamepad->controller, &msg)) {
+        LOGE("Could not push UHID_DESTROY message (gamepad)");
+    }
+}
+
+static void
+sc_gamepad_processor_process_gamepad_device(struct sc_gamepad_processor *gp,
+                                const struct sc_gamepad_device_event *event) {
+    struct sc_gamepad_uhid *gamepad = DOWNCAST(gp);
+
+    if (event->type == SC_GAMEPAD_DEVICE_ADDED) {
+        struct sc_hid_open hid_open;
+        if (!sc_hid_gamepad_generate_open(&gamepad->hid, &hid_open,
+                                          event->gamepad_id)) {
+            return;
+        }
+
+        sc_gamepad_uhid_send_open(gamepad, &hid_open);
+    } else {
+        assert(event->type == SC_GAMEPAD_DEVICE_REMOVED);
+
+        struct sc_hid_close hid_close;
+        if (!sc_hid_gamepad_generate_close(&gamepad->hid, &hid_close,
+                                           event->gamepad_id)) {
+            return;
+        }
+
+        sc_gamepad_uhid_send_close(gamepad, &hid_close);
+    }
+}
+
+static void
+sc_gamepad_processor_process_gamepad_axis(struct sc_gamepad_processor *gp,
+                                const struct sc_gamepad_axis_event *event) {
+    struct sc_gamepad_uhid *gamepad = DOWNCAST(gp);
+
+    struct sc_hid_input hid_input;
+    if (!sc_hid_gamepad_generate_input_from_axis(&gamepad->hid, &hid_input,
+                                                 event)) {
+        return;
+    }
+
+    sc_gamepad_uhid_send_input(gamepad, &hid_input, "gamepad axis");
+}
+
+static void
+sc_gamepad_processor_process_gamepad_button(struct sc_gamepad_processor *gp,
+                                const struct sc_gamepad_button_event *event) {
+    struct sc_gamepad_uhid *gamepad = DOWNCAST(gp);
+
+    struct sc_hid_input hid_input;
+    if (!sc_hid_gamepad_generate_input_from_button(&gamepad->hid, &hid_input,
+                                                   event)) {
+        return;
+    }
+
+    sc_gamepad_uhid_send_input(gamepad, &hid_input, "gamepad button");
+
+}
+
+void
+sc_gamepad_uhid_process_hid_output(struct sc_gamepad_uhid *gamepad,
+                                   uint16_t hid_id, const uint8_t *data,
+                                   size_t size) {
+    (void) gamepad;
+    char *hex = sc_str_to_hex_string(data, size);
+    if (hex) {
+        LOGI("==== HID output [%" PRIu16 "] %s", hid_id, hex);
+        free(hex);
+    } else {
+        LOGI("==== HID output [%" PRIu16 "]", hid_id);
+    }
+
+    // TODO
+}
+
+void
+sc_gamepad_uhid_init(struct sc_gamepad_uhid *gamepad,
+                     struct sc_controller *controller) {
+    sc_hid_gamepad_init(&gamepad->hid);
+
+    gamepad->controller = controller;
+
+    static const struct sc_gamepad_processor_ops ops = {
+        .process_gamepad_device = sc_gamepad_processor_process_gamepad_device,
+        .process_gamepad_axis = sc_gamepad_processor_process_gamepad_axis,
+        .process_gamepad_button = sc_gamepad_processor_process_gamepad_button,
+    };
+
+    gamepad->gamepad_processor.ops = &ops;
+}

app/src/uhid/gamepad_uhid.h

@@ -0,0 +1,28 @@
+#ifndef SC_GAMEPAD_UHID_H
+#define SC_GAMEPAD_UHID_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "hid/hid_gamepad.h"
+#include "trait/gamepad_processor.h"
+
+struct sc_gamepad_uhid {
+    struct sc_gamepad_processor gamepad_processor; // gamepad processor trait
+
+    struct sc_hid_gamepad hid;
+    struct sc_controller *controller;
+};
+
+void
+sc_gamepad_uhid_init(struct sc_gamepad_uhid *mouse,
+                     struct sc_controller *controller);
+
+void
+sc_gamepad_uhid_process_hid_output(struct sc_gamepad_uhid *gamepad,
+                                   uint16_t hid_id, const uint8_t *data,
+                                   size_t size);
+
+#endif

app/src/uhid/keyboard_uhid.c

@@ -0,0 +1,153 @@
+#include "keyboard_uhid.h"
+
+#include "util/log.h"
+
+/** Downcast key processor to keyboard_uhid */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_uhid, key_processor)
+
+/** Downcast uhid_receiver to keyboard_uhid */
+#define DOWNCAST_RECEIVER(UR) \
+    container_of(UR, struct sc_keyboard_uhid, uhid_receiver)
+
+static void
+sc_keyboard_uhid_send_input(struct sc_keyboard_uhid *kb,
+                            const struct sc_hid_input *hid_input) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = hid_input->hid_id;
+
+    assert(hid_input->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, hid_input->data, hid_input->size);
+    msg.uhid_input.size = hid_input->size;
+
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
+        LOGE("Could not push UHID_INPUT message (key)");
+    }
+}
+
+static void
+sc_keyboard_uhid_synchronize_mod(struct sc_keyboard_uhid *kb) {
+    SDL_Keymod sdl_mod = SDL_GetModState();
+    uint16_t mod = sc_mods_state_from_sdl(sdl_mod) & (SC_MOD_CAPS | SC_MOD_NUM);
+    uint16_t diff = mod ^ kb->device_mod;
+
+    if (diff) {
+        // Inherently racy (the HID output reports arrive asynchronously in
+        // response to key presses), but will re-synchronize on next key press
+        // or HID output anyway
+        kb->device_mod = mod;
+
+        struct sc_hid_input hid_input;
+        if (!sc_hid_keyboard_generate_input_from_mods(&hid_input, diff)) {
+            return;
+        }
+
+        LOGV("HID keyboard state synchronized");
+
+        sc_keyboard_uhid_send_input(kb, &hid_input);
+    }
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const struct sc_key_event *event,
+                             uint64_t ack_to_wait) {
+    (void) ack_to_wait;
+
+    assert(sc_thread_get_id() == SC_MAIN_THREAD_ID);
+
+    if (event->repeat) {
+        // In USB HID protocol, key repeat is handled by the host (Android), so
+        // just ignore key repeat here.
+        return;
+    }
+
+    struct sc_keyboard_uhid *kb = DOWNCAST(kp);
+
+    struct sc_hid_input hid_input;
+
+    // Not all keys are supported, just ignore unsupported keys
+    if (sc_hid_keyboard_generate_input_from_key(&kb->hid, &hid_input, event)) {
+        if (event->scancode == SC_SCANCODE_CAPSLOCK) {
+            kb->device_mod ^= SC_MOD_CAPS;
+        } else if (event->scancode == SC_SCANCODE_NUMLOCK) {
+            kb->device_mod ^= SC_MOD_NUM;
+        } else {
+            // Synchronize modifiers (only if the scancode itself does not
+            // change the modifiers)
+            sc_keyboard_uhid_synchronize_mod(kb);
+        }
+        sc_keyboard_uhid_send_input(kb, &hid_input);
+    }
+}
+
+static unsigned
+sc_keyboard_uhid_to_sc_mod(uint8_t hid_led) {
+    // <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
+    // (chapter 11: LED page)
+    unsigned mod = 0;
+    if (hid_led & 0x01) {
+        mod |= SC_MOD_NUM;
+    }
+    if (hid_led & 0x02) {
+        mod |= SC_MOD_CAPS;
+    }
+    return mod;
+}
+
+void
+sc_keyboard_uhid_process_hid_output(struct sc_keyboard_uhid *kb,
+                                    const uint8_t *data, size_t size) {
+    assert(sc_thread_get_id() == SC_MAIN_THREAD_ID);
+
+    assert(size);
+
+    // Also check at runtime (do not trust the server)
+    if (!size) {
+        LOGE("Unexpected empty HID output message");
+        return;
+    }
+
+    uint8_t hid_led = data[0];
+    uint16_t device_mod = sc_keyboard_uhid_to_sc_mod(hid_led);
+    kb->device_mod = device_mod;
+}
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller) {
+    sc_hid_keyboard_init(&kb->hid);
+
+    kb->controller = controller;
+    kb->device_mod = 0;
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        // Never forward text input via HID (all the keys are injected
+        // separately)
+        .process_text = NULL,
+    };
+
+    // Clipboard synchronization is requested over the same control socket, so
+    // there is no need for a specific synchronization mechanism
+    kb->key_processor.async_paste = false;
+    kb->key_processor.hid = true;
+    kb->key_processor.ops = &ops;
+
+    struct sc_hid_open hid_open;
+    sc_hid_keyboard_generate_open(&hid_open);
+    assert(hid_open.hid_id == SC_HID_ID_KEYBOARD);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = SC_HID_ID_KEYBOARD;
+    msg.uhid_create.name = hid_open.name;
+    msg.uhid_create.report_desc = hid_open.report_desc;
+    msg.uhid_create.report_desc_size = hid_open.report_desc_size;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not send UHID_CREATE message (keyboard)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/keyboard_uhid.h

@@ -0,0 +1,28 @@
+#ifndef SC_KEYBOARD_UHID_H
+#define SC_KEYBOARD_UHID_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "hid/hid_keyboard.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_uhid {
+    struct sc_key_processor key_processor; // key processor trait
+
+    struct sc_hid_keyboard hid;
+    struct sc_controller *controller;
+    uint16_t device_mod;
+};
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller);
+
+void
+sc_keyboard_uhid_process_hid_output(struct sc_keyboard_uhid *kb,
+                                    const uint8_t *data, size_t size);
+
+#endif

app/src/uhid/mouse_uhid.c

@@ -0,0 +1,93 @@
+#include "mouse_uhid.h"
+
+#include "hid/hid_mouse.h"
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to mouse_uhid */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_uhid, mouse_processor)
+
+static void
+sc_mouse_uhid_send_input(struct sc_mouse_uhid *mouse,
+                         const struct sc_hid_input *hid_input,
+                         const char *name) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = hid_input->hid_id;
+
+    assert(hid_input->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, hid_input->data, hid_input->size);
+    msg.uhid_input.size = hid_input->size;
+
+    if (!sc_controller_push_msg(mouse->controller, &msg)) {
+        LOGE("Could not push UHID_INPUT message (%s)", name);
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_motion_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_input hid_input;
+    sc_hid_mouse_generate_input_from_motion(&hid_input, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_input, "mouse motion");
+}
+
+static void
+sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
+                                   const struct sc_mouse_click_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_input hid_input;
+    sc_hid_mouse_generate_input_from_click(&hid_input, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_input, "mouse click");
+}
+
+static void
+sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_scroll_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_input hid_input;
+    sc_hid_mouse_generate_input_from_scroll(&hid_input, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_input, "mouse scroll");
+}
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller) {
+    mouse->controller = controller;
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_mouse_click = sc_mouse_processor_process_mouse_click,
+        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
+        // Touch events not supported (coordinates are not relative)
+        .process_touch = NULL,
+    };
+
+    mouse->mouse_processor.ops = &ops;
+
+    mouse->mouse_processor.relative_mode = true;
+
+    struct sc_hid_open hid_open;
+    sc_hid_mouse_generate_open(&hid_open);
+    assert(hid_open.hid_id == SC_HID_ID_MOUSE);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = SC_HID_ID_MOUSE;
+    msg.uhid_create.name = hid_open.name;
+    msg.uhid_create.report_desc = hid_open.report_desc;
+    msg.uhid_create.report_desc_size = hid_open.report_desc_size;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not push UHID_CREATE message (mouse)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/mouse_uhid.h

@@ -0,0 +1,19 @@
+#ifndef SC_MOUSE_UHID_H
+#define SC_MOUSE_UHID_H
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "trait/mouse_processor.h"
+
+struct sc_mouse_uhid {
+    struct sc_mouse_processor mouse_processor; // mouse processor trait
+
+    struct sc_controller *controller;
+};
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller);
+
+#endif

app/src/uhid/uhid_output.c

@@ -0,0 +1,37 @@
+#include "uhid_output.h"
+
+#include <assert.h>
+#include <inttypes.h>
+
+#include "uhid/keyboard_uhid.h"
+#include "uhid/gamepad_uhid.h"
+#include "util/log.h"
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices,
+                     struct sc_keyboard_uhid *keyboard,
+                     struct sc_gamepad_uhid *gamepad) {
+    devices->keyboard = keyboard;
+    devices->gamepad = gamepad;
+}
+
+void
+sc_uhid_devices_process_hid_output(struct sc_uhid_devices *devices, uint16_t id,
+                                   const uint8_t *data, size_t size) {
+    if (id == SC_HID_ID_KEYBOARD) {
+        if (devices->keyboard) {
+            sc_keyboard_uhid_process_hid_output(devices->keyboard, data, size);
+        } else {
+            LOGW("Unexpected keyboard HID output without UHID keyboard");
+        }
+    } else if (id >= SC_HID_ID_GAMEPAD_FIRST && id <= SC_HID_ID_GAMEPAD_LAST) {
+        if (devices->gamepad) {
+            sc_gamepad_uhid_process_hid_output(devices->gamepad, id, data,
+                                               size);
+        } else {
+            LOGW("Unexpected gamepad HID output without UHID gamepad");
+        }
+    } else {
+        LOGW("HID output ignored for id %" PRIu16, id);
+    }
+}

app/src/uhid/uhid_output.h

@@ -0,0 +1,29 @@
+#ifndef SC_UHID_OUTPUT_H
+#define SC_UHID_OUTPUT_H
+
+#include "common.h"
+
+#include <stdbool.h>
+#include <stdint.h>
+
+/**
+ * The communication with UHID devices is bidirectional.
+ *
+ * This component dispatches HID outputs to the expected processor.
+ */
+
+struct sc_uhid_devices {
+    struct sc_keyboard_uhid *keyboard;
+    struct sc_gamepad_uhid *gamepad;
+};
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices,
+                     struct sc_keyboard_uhid *keyboard,
+                     struct sc_gamepad_uhid *gamepad);
+
+void
+sc_uhid_devices_process_hid_output(struct sc_uhid_devices *devices, uint16_t id,
+                                   const uint8_t *data, size_t size);
+
+#endif

app/src/usb/aoa_hid.c

@@ -1,10 +1,14 @@
 #include "util/log.h"
 
 #include <assert.h>
+#include <inttypes.h>
 #include <stdio.h>
 
 #include "aoa_hid.h"
+#include "events.h"
 #include "util/log.h"
+#include "util/str.h"
+#include "util/vector.h"
 
 // See <https://source.android.com/devices/accessories/aoa2#hid-support>.
 #define ACCESSORY_REGISTER_HID 54
@@ -14,37 +18,40 @@
 
 #define DEFAULT_TIMEOUT 1000
 
-#define SC_HID_EVENT_QUEUE_MAX 64
+// Drop droppable events above this limit
+#define SC_AOA_EVENT_QUEUE_LIMIT 60
+
+struct sc_vec_hid_ids SC_VECTOR(uint16_t);
 
 static void
-sc_hid_event_log(const struct sc_hid_event *event) {
+sc_hid_input_log(const struct sc_hid_input *hid_input) {
-    // HID Event: [00] FF FF FF FF...
+    // HID input: [00] FF FF FF FF...
-    assert(event->size);
+    assert(hid_input->size);
-    unsigned buffer_size = event->size * 3 + 1;
+    char *hex = sc_str_to_hex_string(hid_input->data, hid_input->size);
-    char *buffer = malloc(buffer_size);
+    if (!hex) {
-    if (!buffer) {
-        LOG_OOM();
         return;
     }
-    for (unsigned i = 0; i < event->size; ++i) {
+    LOGV("HID input: [%" PRIu16 "] %s", hid_input->hid_id, hex);
-        snprintf(buffer + i * 3, 4, " %02x", event->buffer[i]);
+    free(hex);
+}
+
+static void
+sc_hid_open_log(const struct sc_hid_open *hid_open) {
+    // HID open: [00] FF FF FF FF...
+    assert(hid_open->report_desc_size);
+    char *hex = sc_str_to_hex_string(hid_open->report_desc,
+                                     hid_open->report_desc_size);
+    if (!hex) {
+        return;
     }
-    LOGV("HID Event: [%d]%s", event->accessory_id, buffer);
+    LOGV("HID open: [%" PRIu16 "] %s", hid_open->hid_id, hex);
-    free(buffer);
+    free(hex);
 }
 
-void
+static void
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
+sc_hid_close_log(const struct sc_hid_close *hid_close) {
-                  unsigned char *buffer, uint16_t buffer_size) {
+    // HID close: [00]
-    hid_event->accessory_id = accessory_id;
+    LOGV("HD close: [%" PRIu16 "]", hid_close->hid_id);
-    hid_event->buffer = buffer;
-    hid_event->size = buffer_size;
-    hid_event->ack_to_wait = SC_SEQUENCE_INVALID;
-}
-
-void
-sc_hid_event_destroy(struct sc_hid_event *hid_event) {
-    free(hid_event->buffer);
 }
 
 bool
@@ -52,7 +59,8 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
             struct sc_acksync *acksync) {
     sc_vecdeque_init(&aoa->queue);
 
-    if (!sc_vecdeque_reserve(&aoa->queue, SC_HID_EVENT_QUEUE_MAX)) {
+    // Add 4 to support 4 non-droppable events without re-allocation
+    if (!sc_vecdeque_reserve(&aoa->queue, SC_AOA_EVENT_QUEUE_LIMIT + 4)) {
         return false;
     }
 
@@ -76,12 +84,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
 
 void
 sc_aoa_destroy(struct sc_aoa *aoa) {
-    // Destroy remaining events
+    sc_vecdeque_destroy(&aoa->queue);
-    while (!sc_vecdeque_is_empty(&aoa->queue)) {
-        struct sc_hid_event *event = sc_vecdeque_popref(&aoa->queue);
-        assert(event);
-        sc_hid_event_destroy(event);
-    }
 
     sc_cond_destroy(&aoa->event_cond);
     sc_mutex_destroy(&aoa->mutex);
@@ -97,10 +100,10 @@ sc_aoa_register_hid(struct sc_aoa *aoa, uint16_t accessory_id,
     // index (arg1): total length of the HID report descriptor
     uint16_t value = accessory_id;
     uint16_t index = report_desc_size;
-    unsigned char *buffer = NULL;
+    unsigned char *data = NULL;
     uint16_t length = 0;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("REGISTER_HID: libusb error: %s", libusb_strerror(result));
@@ -113,7 +116,7 @@ sc_aoa_register_hid(struct sc_aoa *aoa, uint16_t accessory_id,
 
 static bool
 sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
-                           const unsigned char *report_desc,
+                           const uint8_t *report_desc,
                            uint16_t report_desc_size) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_SET_HID_REPORT_DESC;
@@ -130,14 +133,14 @@ sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
      * See <https://libusb.sourceforge.io/api-1.0/libusb_packetoverflow.html>
      */
     // value (arg0): accessory assigned ID for the HID device
-    // index (arg1): offset of data (buffer) in descriptor
+    // index (arg1): offset of data in descriptor
     uint16_t value = accessory_id;
     uint16_t index = 0;
     // libusb_control_transfer expects a pointer to non-const
-    unsigned char *buffer = (unsigned char *) report_desc;
+    unsigned char *data = (unsigned char *) report_desc;
     uint16_t length = report_desc_size;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("SET_HID_REPORT_DESC: libusb error: %s", libusb_strerror(result));
@@ -148,9 +151,56 @@ sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
     return true;
 }
 
-bool
+static bool
+sc_aoa_send_hid_event(struct sc_aoa *aoa,
+                      const struct sc_hid_input *hid_input) {
+    uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
+    uint8_t request = ACCESSORY_SEND_HID_EVENT;
+    // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
+    // value (arg0): accessory assigned ID for the HID device
+    // index (arg1): 0 (unused)
+    uint16_t value = hid_input->hid_id;
+    uint16_t index = 0;
+    unsigned char *data = (uint8_t *) hid_input->data; // discard const
+    uint16_t length = hid_input->size;
+    int result = libusb_control_transfer(aoa->usb->handle, request_type,
+                                         request, value, index, data, length,
+                                         DEFAULT_TIMEOUT);
+    if (result < 0) {
+        LOGE("SEND_HID_EVENT: libusb error: %s", libusb_strerror(result));
+        sc_usb_check_disconnected(aoa->usb, result);
+        return false;
+    }
+
+    return true;
+}
+
+static bool
+sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id) {
+    uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
+    uint8_t request = ACCESSORY_UNREGISTER_HID;
+    // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
+    // value (arg0): accessory assigned ID for the HID device
+    // index (arg1): 0
+    uint16_t value = accessory_id;
+    uint16_t index = 0;
+    unsigned char *data = NULL;
+    uint16_t length = 0;
+    int result = libusb_control_transfer(aoa->usb->handle, request_type,
+                                         request, value, index, data, length,
+                                         DEFAULT_TIMEOUT);
+    if (result < 0) {
+        LOGE("UNREGISTER_HID: libusb error: %s", libusb_strerror(result));
+        sc_usb_check_disconnected(aoa->usb, result);
+        return false;
+    }
+
+    return true;
+}
+
+static bool
 sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
-                 const unsigned char *report_desc, uint16_t report_desc_size) {
+                 const uint8_t *report_desc, uint16_t report_desc_size) {
     bool ok = sc_aoa_register_hid(aoa, accessory_id, report_desc_size);
     if (!ok) {
         return false;
@@ -168,63 +218,25 @@ sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
     return true;
 }
 
-static bool
-sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
-    uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
-    uint8_t request = ACCESSORY_SEND_HID_EVENT;
-    // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
-    // value (arg0): accessory assigned ID for the HID device
-    // index (arg1): 0 (unused)
-    uint16_t value = event->accessory_id;
-    uint16_t index = 0;
-    unsigned char *buffer = event->buffer;
-    uint16_t length = event->size;
-    int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
-                                         DEFAULT_TIMEOUT);
-    if (result < 0) {
-        LOGE("SEND_HID_EVENT: libusb error: %s", libusb_strerror(result));
-        sc_usb_check_disconnected(aoa->usb, result);
-        return false;
-    }
-
-    return true;
-}
-
 bool
-sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
+sc_aoa_push_input_with_ack_to_wait(struct sc_aoa *aoa,
-    uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
+                                   const struct sc_hid_input *hid_input,
-    uint8_t request = ACCESSORY_UNREGISTER_HID;
+                                   uint64_t ack_to_wait) {
-    // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
-    // value (arg0): accessory assigned ID for the HID device
-    // index (arg1): 0
-    uint16_t value = accessory_id;
-    uint16_t index = 0;
-    unsigned char *buffer = NULL;
-    uint16_t length = 0;
-    int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
-                                         DEFAULT_TIMEOUT);
-    if (result < 0) {
-        LOGE("UNREGISTER_HID: libusb error: %s", libusb_strerror(result));
-        sc_usb_check_disconnected(aoa->usb, result);
-        return false;
-    }
-
-    return true;
-}
-
-bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
     if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
-        sc_hid_event_log(event);
+        sc_hid_input_log(hid_input);
     }
 
     sc_mutex_lock(&aoa->mutex);
     bool full = sc_vecdeque_is_full(&aoa->queue);
     if (!full) {
         bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
-        sc_vecdeque_push_noresize(&aoa->queue, *event);
+
+        struct sc_aoa_event *aoa_event =
+            sc_vecdeque_push_hole_noresize(&aoa->queue);
+        aoa_event->type = SC_AOA_EVENT_TYPE_INPUT;
+        aoa_event->input.hid = *hid_input;
+        aoa_event->input.ack_to_wait = ack_to_wait;
+
         if (was_empty) {
             sc_cond_signal(&aoa->event_cond);
         }
@@ -236,10 +248,159 @@ sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
     return !full;
 }
 
+bool
+sc_aoa_push_open(struct sc_aoa *aoa, const struct sc_hid_open *hid_open,
+                 bool exit_on_open_error) {
+    if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
+        sc_hid_open_log(hid_open);
+    }
+
+    sc_mutex_lock(&aoa->mutex);
+    bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
+
+    // an OPEN event is non-droppable, so push it to the queue even above the
+    // SC_AOA_EVENT_QUEUE_LIMIT
+    struct sc_aoa_event *aoa_event = sc_vecdeque_push_hole(&aoa->queue);
+    if (!aoa_event) {
+        LOG_OOM();
+        sc_mutex_unlock(&aoa->mutex);
+        return false;
+    }
+
+    aoa_event->type = SC_AOA_EVENT_TYPE_OPEN;
+    aoa_event->open.hid = *hid_open;
+    aoa_event->open.exit_on_error = exit_on_open_error;
+
+    if (was_empty) {
+        sc_cond_signal(&aoa->event_cond);
+    }
+
+    sc_mutex_unlock(&aoa->mutex);
+
+    return true;
+}
+
+bool
+sc_aoa_push_close(struct sc_aoa *aoa, const struct sc_hid_close *hid_close) {
+    if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
+        sc_hid_close_log(hid_close);
+    }
+
+    sc_mutex_lock(&aoa->mutex);
+    bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
+
+    // an OPEN event is non-droppable, so push it to the queue even above the
+    // SC_AOA_EVENT_QUEUE_LIMIT
+    struct sc_aoa_event *aoa_event = sc_vecdeque_push_hole(&aoa->queue);
+    if (!aoa_event) {
+        LOG_OOM();
+        sc_mutex_unlock(&aoa->mutex);
+        return false;
+    }
+
+    aoa_event->type = SC_AOA_EVENT_TYPE_CLOSE;
+    aoa_event->close.hid = *hid_close;
+
+    if (was_empty) {
+        sc_cond_signal(&aoa->event_cond);
+    }
+
+    sc_mutex_unlock(&aoa->mutex);
+
+    return true;
+}
+
+static bool
+sc_aoa_process_event(struct sc_aoa *aoa, struct sc_aoa_event *event,
+                     struct sc_vec_hid_ids *vec_open) {
+    switch (event->type) {
+        case SC_AOA_EVENT_TYPE_INPUT: {
+            uint64_t ack_to_wait = event->input.ack_to_wait;
+            if (ack_to_wait != SC_SEQUENCE_INVALID) {
+                LOGD("Waiting ack from server sequence=%" PRIu64_, ack_to_wait);
+
+                // If some events have ack_to_wait set, then sc_aoa must have
+                // been initialized with a non NULL acksync
+                assert(aoa->acksync);
+
+                // Do not block the loop indefinitely if the ack never comes (it
+                // should never happen)
+                sc_tick deadline = sc_tick_now() + SC_TICK_FROM_MS(500);
+                enum sc_acksync_wait_result result =
+                    sc_acksync_wait(aoa->acksync, ack_to_wait, deadline);
+
+                if (result == SC_ACKSYNC_WAIT_TIMEOUT) {
+                    LOGW("Ack not received after 500ms, discarding HID event");
+                    // continue to process events
+                    return true;
+                } else if (result == SC_ACKSYNC_WAIT_INTR) {
+                    // stopped
+                    return false;
+                }
+            }
+
+            struct sc_hid_input *hid_input = &event->input.hid;
+            bool ok = sc_aoa_send_hid_event(aoa, hid_input);
+            if (!ok) {
+                LOGW("Could not send HID event to USB device: %" PRIu16,
+                     hid_input->hid_id);
+            }
+
+            break;
+        }
+        case SC_AOA_EVENT_TYPE_OPEN: {
+            struct sc_hid_open *hid_open = &event->open.hid;
+            bool ok = sc_aoa_setup_hid(aoa, hid_open->hid_id,
+                                       hid_open->report_desc,
+                                       hid_open->report_desc_size);
+            if (ok) {
+                // The device is now open, add it to the list of devices to
+                // close automatically on exit
+                bool pushed = sc_vector_push(vec_open, hid_open->hid_id);
+                if (!pushed) {
+                    LOG_OOM();
+                    // this is not fatal, the HID device will just not be
+                    // explicitly unregistered
+                }
+            } else {
+                LOGW("Could not open AOA device: %" PRIu16, hid_open->hid_id);
+                if (event->open.exit_on_error) {
+                    // Notify the error to the main thread, which will exit
+                    sc_push_event(SC_EVENT_AOA_OPEN_ERROR);
+                }
+            }
+
+            break;
+        }
+        case SC_AOA_EVENT_TYPE_CLOSE: {
+            struct sc_hid_close *hid_close = &event->close.hid;
+            bool ok = sc_aoa_unregister_hid(aoa, hid_close->hid_id);
+            if (ok) {
+                // The device is not open anymore, remove it from the list of
+                // devices to close automatically on exit
+                ssize_t idx = sc_vector_index_of(vec_open, hid_close->hid_id);
+                if (idx >= 0) {
+                    sc_vector_remove(vec_open, idx);
+                }
+            } else {
+                LOGW("Could not close AOA device: %" PRIu16, hid_close->hid_id);
+            }
+
+            break;
+        }
+    }
+
+    // continue to process events
+    return true;
+}
+
 static int
 run_aoa_thread(void *data) {
     struct sc_aoa *aoa = data;
 
+    // Store the HID ids of opened devices to unregister them all before exiting
+    struct sc_vec_hid_ids vec_open = SC_VECTOR_INITIALIZER;
+
     for (;;) {
         sc_mutex_lock(&aoa->mutex);
         while (!aoa->stopped && sc_vecdeque_is_empty(&aoa->queue)) {
@@ -252,40 +413,27 @@ run_aoa_thread(void *data) {
         }
 
         assert(!sc_vecdeque_is_empty(&aoa->queue));
-        struct sc_hid_event event = sc_vecdeque_pop(&aoa->queue);
+        struct sc_aoa_event event = sc_vecdeque_pop(&aoa->queue);
-        uint64_t ack_to_wait = event.ack_to_wait;
         sc_mutex_unlock(&aoa->mutex);
 
-        if (ack_to_wait != SC_SEQUENCE_INVALID) {
+        bool cont = sc_aoa_process_event(aoa, &event, &vec_open);
-            LOGD("Waiting ack from server sequence=%" PRIu64_, ack_to_wait);
+        if (!cont) {
-
+            // stopped
-            // If some events have ack_to_wait set, then sc_aoa must have been
+            break;
-            // initialized with a non NULL acksync
-            assert(aoa->acksync);
-
-            // Do not block the loop indefinitely if the ack never comes (it
-            // should never happen)
-            sc_tick deadline = sc_tick_now() + SC_TICK_FROM_MS(500);
-            enum sc_acksync_wait_result result =
-                sc_acksync_wait(aoa->acksync, ack_to_wait, deadline);
-
-            if (result == SC_ACKSYNC_WAIT_TIMEOUT) {
-                LOGW("Ack not received after 500ms, discarding HID event");
-                sc_hid_event_destroy(&event);
-                continue;
-            } else if (result == SC_ACKSYNC_WAIT_INTR) {
-                // stopped
-                sc_hid_event_destroy(&event);
-                break;
-            }
-        }
-
-        bool ok = sc_aoa_send_hid_event(aoa, &event);
-        sc_hid_event_destroy(&event);
-        if (!ok) {
-            LOGW("Could not send HID event to USB device");
         }
     }
+
+    // Explicitly unregister all registered HID ids before exiting
+    for (size_t i = 0; i < vec_open.size; ++i) {
+        uint16_t hid_id = vec_open.data[i];
+        LOGD("Unregistering AOA device %" PRIu16 "...", hid_id);
+        bool ok = sc_aoa_unregister_hid(aoa, hid_id);
+        if (!ok) {
+            LOGW("Could not close AOA device: %" PRIu16, hid_id);
+        }
+    }
+    sc_vector_destroy(&vec_open);
+
     return 0;
 }
 

app/src/usb/aoa_hid.h

@@ -6,28 +6,37 @@
 
 #include <libusb-1.0/libusb.h>
 
+#include "hid/hid_event.h"
 #include "usb.h"
 #include "util/acksync.h"
 #include "util/thread.h"
 #include "util/tick.h"
 #include "util/vecdeque.h"
 
-struct sc_hid_event {
+enum sc_aoa_event_type {
-    uint16_t accessory_id;
+    SC_AOA_EVENT_TYPE_OPEN,
-    unsigned char *buffer;
+    SC_AOA_EVENT_TYPE_INPUT,
-    uint16_t size;
+    SC_AOA_EVENT_TYPE_CLOSE,
-    uint64_t ack_to_wait;
 };
 
-// Takes ownership of buffer
+struct sc_aoa_event {
-void
+    enum sc_aoa_event_type type;
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
+    union {
-                  unsigned char *buffer, uint16_t buffer_size);
+        struct {
+            struct sc_hid_open hid;
+            bool exit_on_error;
+        } open;
+        struct {
+            struct sc_hid_close hid;
+        } close;
+        struct {
+            struct sc_hid_input hid;
+            uint64_t ack_to_wait;
+        } input;
+    };
+};
 
-void
+struct sc_aoa_event_queue SC_VECDEQUE(struct sc_aoa_event);
-sc_hid_event_destroy(struct sc_hid_event *hid_event);
-
-struct sc_hid_event_queue SC_VECDEQUE(struct sc_hid_event);
 
 struct sc_aoa {
     struct sc_usb *usb;
@@ -35,7 +44,7 @@ struct sc_aoa {
     sc_mutex mutex;
     sc_cond event_cond;
     bool stopped;
-    struct sc_hid_event_queue queue;
+    struct sc_aoa_event_queue queue;
 
     struct sc_acksync *acksync;
 };
@@ -55,14 +64,31 @@ sc_aoa_stop(struct sc_aoa *aoa);
 void
 sc_aoa_join(struct sc_aoa *aoa);
 
+//bool
+//sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
+//              const uint8_t *report_desc, uint16_t report_desc_size);
+//
+//bool
+//sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id);
+
+// report_desc must be a pointer to static memory, accessed at any time from
+// another thread
 bool
-sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
+sc_aoa_push_open(struct sc_aoa *aoa, const struct sc_hid_open *hid_open,
-              const unsigned char *report_desc, uint16_t report_desc_size);
+                 bool exit_on_open_error);
 
 bool
-sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id);
+sc_aoa_push_close(struct sc_aoa *aoa, const struct sc_hid_close *hid_close);
 
 bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event);
+sc_aoa_push_input_with_ack_to_wait(struct sc_aoa *aoa,
+                                   const struct sc_hid_input *hid_input,
+                                   uint64_t ack_to_wait);
+
+static inline bool
+sc_aoa_push_input(struct sc_aoa *aoa, const struct sc_hid_input *hid_input) {
+    return sc_aoa_push_input_with_ack_to_wait(aoa, hid_input,
+                                              SC_SEQUENCE_INVALID);
+}
 
 #endif

app/src/usb/gamepad_aoa.c

@@ -0,0 +1,91 @@
+#include "gamepad_aoa.h"
+
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast gamepad processor to gamepad_aoa */
+#define DOWNCAST(GP) container_of(GP, struct sc_gamepad_aoa, gamepad_processor)
+
+static void
+sc_gamepad_processor_process_gamepad_device(struct sc_gamepad_processor *gp,
+                                const struct sc_gamepad_device_event *event) {
+    struct sc_gamepad_aoa *gamepad = DOWNCAST(gp);
+
+    if (event->type == SC_GAMEPAD_DEVICE_ADDED) {
+        struct sc_hid_open hid_open;
+        if (!sc_hid_gamepad_generate_open(&gamepad->hid, &hid_open,
+                                          event->gamepad_id)) {
+            return;
+        }
+
+        // exit_on_error: false (a gamepad open failure should not exit scrcpy)
+        if (!sc_aoa_push_open(gamepad->aoa, &hid_open, false)) {
+            LOGW("Could not push AOA HID open (gamepad)");
+        }
+    } else {
+        assert(event->type == SC_GAMEPAD_DEVICE_REMOVED);
+
+        struct sc_hid_close hid_close;
+        if (!sc_hid_gamepad_generate_close(&gamepad->hid, &hid_close,
+                                           event->gamepad_id)) {
+            return;
+        }
+
+        if (!sc_aoa_push_close(gamepad->aoa, &hid_close)) {
+            LOGW("Could not push AOA HID close (gamepad)");
+        }
+    }
+}
+
+static void
+sc_gamepad_processor_process_gamepad_axis(struct sc_gamepad_processor *gp,
+                                const struct sc_gamepad_axis_event *event) {
+    struct sc_gamepad_aoa *gamepad = DOWNCAST(gp);
+
+    struct sc_hid_input hid_input;
+    if (!sc_hid_gamepad_generate_input_from_axis(&gamepad->hid, &hid_input,
+                                                 event)) {
+        return;
+    }
+
+    if (!sc_aoa_push_input(gamepad->aoa, &hid_input)) {
+        LOGW("Could not push AOA HID input (gamepad axis)");
+    }
+}
+
+static void
+sc_gamepad_processor_process_gamepad_button(struct sc_gamepad_processor *gp,
+                                const struct sc_gamepad_button_event *event) {
+    struct sc_gamepad_aoa *gamepad = DOWNCAST(gp);
+
+    struct sc_hid_input hid_input;
+    if (!sc_hid_gamepad_generate_input_from_button(&gamepad->hid, &hid_input,
+                                                   event)) {
+        return;
+    }
+
+    if (!sc_aoa_push_input(gamepad->aoa, &hid_input)) {
+        LOGW("Could not push AOA HID input (gamepad button)");
+    }
+}
+
+void
+sc_gamepad_aoa_init(struct sc_gamepad_aoa *gamepad, struct sc_aoa *aoa) {
+    gamepad->aoa = aoa;
+
+    sc_hid_gamepad_init(&gamepad->hid);
+
+    static const struct sc_gamepad_processor_ops ops = {
+        .process_gamepad_device = sc_gamepad_processor_process_gamepad_device,
+        .process_gamepad_axis = sc_gamepad_processor_process_gamepad_axis,
+        .process_gamepad_button = sc_gamepad_processor_process_gamepad_button,
+    };
+
+    gamepad->gamepad_processor.ops = &ops;
+}
+
+void
+sc_gamepad_aoa_destroy(struct sc_gamepad_aoa *gamepad) {
+    (void) gamepad;
+    // Do nothing, gamepad->aoa will automatically unregister all devices
+}

app/src/usb/gamepad_aoa.h

@@ -0,0 +1,25 @@
+#ifndef SC_GAMEPAD_AOA_H
+#define SC_GAMEPAD_AOA_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "aoa_hid.h"
+#include "hid/hid_gamepad.h"
+#include "trait/gamepad_processor.h"
+
+struct sc_gamepad_aoa {
+    struct sc_gamepad_processor gamepad_processor; // gamepad processor trait
+
+    struct sc_hid_gamepad hid;
+    struct sc_aoa *aoa;
+};
+
+void
+sc_gamepad_aoa_init(struct sc_gamepad_aoa *gamepad, struct sc_aoa *aoa);
+
+void
+sc_gamepad_aoa_destroy(struct sc_gamepad_aoa *gamepad);
+
+#endif

app/src/usb/hid_mouse.c

@@ -1,267 +0,0 @@
-#include "hid_mouse.h"
-
-#include <assert.h>
-
-#include "input_events.h"
-#include "util/log.h"
-
-/** Downcast mouse processor to hid_mouse */
-#define DOWNCAST(MP) container_of(MP, struct sc_hid_mouse, mouse_processor)
-
-#define HID_MOUSE_ACCESSORY_ID 2
-
-// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position
-#define HID_MOUSE_EVENT_SIZE 4
-
-/**
- * Mouse descriptor from the specification:
- * <https://www.usb.org/sites/default/files/hid1_11.pdf>
- *
- * Appendix E (p71): §E.10 Report Descriptor (Mouse)
- *
- * The usage tags (like Wheel) are listed in "HID Usage Tables":
- * <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
- * §4 Generic Desktop Page (0x01) (p26)
- */
-static const unsigned char mouse_report_desc[]  = {
-    // Usage Page (Generic Desktop)
-    0x05, 0x01,
-    // Usage (Mouse)
-    0x09, 0x02,
-
-    // Collection (Application)
-    0xA1, 0x01,
-
-    // Usage (Pointer)
-    0x09, 0x01,
-
-    // Collection (Physical)
-    0xA1, 0x00,
-
-    // Usage Page (Buttons)
-    0x05, 0x09,
-
-    // Usage Minimum (1)
-    0x19, 0x01,
-     // Usage Maximum (5)
-    0x29, 0x05,
-    // Logical Minimum (0)
-    0x15, 0x00,
-    // Logical Maximum (1)
-    0x25, 0x01,
-    // Report Count (5)
-    0x95, 0x05,
-    // Report Size (1)
-    0x75, 0x01,
-    // Input (Data, Variable, Absolute): 5 buttons bits
-    0x81, 0x02,
-
-    // Report Count (1)
-    0x95, 0x01,
-    // Report Size (3)
-    0x75, 0x03,
-    // Input (Constant): 3 bits padding
-    0x81, 0x01,
-
-    // Usage Page (Generic Desktop)
-    0x05, 0x01,
-    // Usage (X)
-    0x09, 0x30,
-    // Usage (Y)
-    0x09, 0x31,
-    // Usage (Wheel)
-    0x09, 0x38,
-    // Local Minimum (-127)
-    0x15, 0x81,
-    // Local Maximum (127)
-    0x25, 0x7F,
-    // Report Size (8)
-    0x75, 0x08,
-    // Report Count (3)
-    0x95, 0x03,
-    // Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
-    0x81, 0x06,
-
-    // End Collection
-    0xC0,
-
-    // End Collection
-    0xC0,
-};
-
-/**
- * A mouse HID event is 3 bytes long:
- *
- *  - byte 0: buttons state
- *  - byte 1: relative x motion (signed byte from -127 to 127)
- *  - byte 2: relative y motion (signed byte from -127 to 127)
- *
- *                   7 6 5 4 3 2 1 0
- *                  +---------------+
- *         byte 0:  |0 0 0 . . . . .| buttons state
- *                  +---------------+
- *                         ^ ^ ^ ^ ^
- *                         | | | | `- left button
- *                         | | | `--- right button
- *                         | | `----- middle button
- *                         | `------- button 4
- *                         `--------- button 5
- *
- *                  +---------------+
- *         byte 1:  |. . . . . . . .| relative x motion
- *                  +---------------+
- *         byte 2:  |. . . . . . . .| relative y motion
- *                  +---------------+
- *         byte 3:  |. . . . . . . .| wheel motion (-1, 0 or 1)
- *                  +---------------+
- *
- * As an example, here is the report for a motion of (x=5, y=-4) with left
- * button pressed:
- *
- *                  +---------------+
- *                  |0 0 0 0 0 0 0 1| left button pressed
- *                  +---------------+
- *                  |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
- *                  +---------------+
- *                  |1 1 1 1 1 1 0 0| relative y motion (y = -4)
- *                  +---------------+
- *                  |0 0 0 0 0 0 0 0| wheel motion
- *                  +---------------+
- */
-
-static bool
-sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
-    unsigned char *buffer = calloc(1, HID_MOUSE_EVENT_SIZE);
-    if (!buffer) {
-        LOG_OOM();
-        return false;
-    }
-
-    sc_hid_event_init(hid_event, HID_MOUSE_ACCESSORY_ID, buffer,
-                      HID_MOUSE_EVENT_SIZE);
-    return true;
-}
-
-static unsigned char
-buttons_state_to_hid_buttons(uint8_t buttons_state) {
-    unsigned char c = 0;
-    if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
-        c |= 1 << 0;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
-        c |= 1 << 1;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
-        c |= 1 << 2;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_X1) {
-        c |= 1 << 3;
-    }
-    if (buttons_state & SC_MOUSE_BUTTON_X2) {
-        c |= 1 << 4;
-    }
-    return c;
-}
-
-static void
-sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
-                                    const struct sc_mouse_motion_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
-    buffer[1] = CLAMP(event->xrel, -127, 127);
-    buffer[2] = CLAMP(event->yrel, -127, 127);
-    buffer[3] = 0; // wheel coordinates only used for scrolling
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse motion)");
-    }
-}
-
-static void
-sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
-                                   const struct sc_mouse_click_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = buttons_state_to_hid_buttons(event->buttons_state);
-    buffer[1] = 0; // no x motion
-    buffer[2] = 0; // no y motion
-    buffer[3] = 0; // wheel coordinates only used for scrolling
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse click)");
-    }
-}
-
-static void
-sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
-                                    const struct sc_mouse_scroll_event *event) {
-    struct sc_hid_mouse *mouse = DOWNCAST(mp);
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_mouse_event_init(&hid_event)) {
-        return;
-    }
-
-    unsigned char *buffer = hid_event.buffer;
-    buffer[0] = 0; // buttons state irrelevant (and unknown)
-    buffer[1] = 0; // no x motion
-    buffer[2] = 0; // no y motion
-    // In practice, vscroll is always -1, 0 or 1, but in theory other values
-    // are possible
-    buffer[3] = CLAMP(event->vscroll, -127, 127);
-    // Horizontal scrolling ignored
-
-    if (!sc_aoa_push_hid_event(mouse->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mouse scroll)");
-    }
-}
-
-bool
-sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa) {
-    mouse->aoa = aoa;
-
-    bool ok = sc_aoa_setup_hid(aoa, HID_MOUSE_ACCESSORY_ID, mouse_report_desc,
-                               ARRAY_LEN(mouse_report_desc));
-    if (!ok) {
-        LOGW("Register HID mouse failed");
-        return false;
-    }
-
-    static const struct sc_mouse_processor_ops ops = {
-        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
-        .process_mouse_click = sc_mouse_processor_process_mouse_click,
-        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
-        // Touch events not supported (coordinates are not relative)
-        .process_touch = NULL,
-    };
-
-    mouse->mouse_processor.ops = &ops;
-
-    mouse->mouse_processor.relative_mode = true;
-
-    return true;
-}
-
-void
-sc_hid_mouse_destroy(struct sc_hid_mouse *mouse) {
-    bool ok = sc_aoa_unregister_hid(mouse->aoa, HID_MOUSE_ACCESSORY_ID);
-    if (!ok) {
-        LOGW("Could not unregister HID mouse");
-    }
-}

app/src/usb/keyboard_aoa.c

@@ -0,0 +1,103 @@
+#include "keyboard_aoa.h"
+
+#include <assert.h>
+
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast key processor to keyboard_aoa */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_aoa, key_processor)
+
+static bool
+push_mod_lock_state(struct sc_keyboard_aoa *kb, uint16_t mods_state) {
+    struct sc_hid_input hid_input;
+    if (!sc_hid_keyboard_generate_input_from_mods(&hid_input, mods_state)) {
+        // Nothing to do
+        return true;
+    }
+
+    if (!sc_aoa_push_input(kb->aoa, &hid_input)) {
+        LOGW("Could not push AOA HID input (mod lock state)");
+        return false;
+    }
+
+    LOGD("HID keyboard state synchronized");
+
+    return true;
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const struct sc_key_event *event,
+                             uint64_t ack_to_wait) {
+    if (event->repeat) {
+        // In USB HID protocol, key repeat is handled by the host (Android), so
+        // just ignore key repeat here.
+        return;
+    }
+
+    struct sc_keyboard_aoa *kb = DOWNCAST(kp);
+
+    struct sc_hid_input hid_input;
+
+    // Not all keys are supported, just ignore unsupported keys
+    if (sc_hid_keyboard_generate_input_from_key(&kb->hid, &hid_input, event)) {
+        if (!kb->mod_lock_synchronized) {
+            // Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
+            // keyboard state
+            if (push_mod_lock_state(kb, event->mods_state)) {
+                kb->mod_lock_synchronized = true;
+            }
+        }
+
+        // If ack_to_wait is != SC_SEQUENCE_INVALID, then Ctrl+v is pressed, so
+        // clipboard synchronization has been requested. Wait until clipboard
+        // synchronization is acknowledged by the server, otherwise it could
+        // paste the old clipboard content.
+
+        if (!sc_aoa_push_input_with_ack_to_wait(kb->aoa, &hid_input,
+                                                ack_to_wait)) {
+            LOGW("Could not push AOA HID input (key)");
+        }
+    }
+}
+
+bool
+sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa) {
+    kb->aoa = aoa;
+
+    struct sc_hid_open hid_open;
+    sc_hid_keyboard_generate_open(&hid_open);
+
+    bool ok = sc_aoa_push_open(aoa, &hid_open, true);
+    if (!ok) {
+        LOGW("Could not push AOA HID open (keyboard)");
+        return false;
+    }
+
+    sc_hid_keyboard_init(&kb->hid);
+
+    kb->mod_lock_synchronized = false;
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        // Never forward text input via HID (all the keys are injected
+        // separately)
+        .process_text = NULL,
+    };
+
+    // Clipboard synchronization is requested over the control socket, while HID
+    // events are sent over AOA, so it must wait for clipboard synchronization
+    // to be acknowledged by the device before injecting Ctrl+v.
+    kb->key_processor.async_paste = true;
+    kb->key_processor.hid = true;
+    kb->key_processor.ops = &ops;
+
+    return true;
+}
+
+void
+sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb) {
+    (void) kb;
+    // Do nothing, kb->aoa will automatically unregister all devices
+}

app/src/usb/keyboard_aoa.h

@@ -0,0 +1,27 @@
+#ifndef SC_KEYBOARD_AOA_H
+#define SC_KEYBOARD_AOA_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "aoa_hid.h"
+#include "hid/hid_keyboard.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_aoa {
+    struct sc_key_processor key_processor; // key processor trait
+
+    struct sc_hid_keyboard hid;
+    struct sc_aoa *aoa;
+
+    bool mod_lock_synchronized;
+};
+
+bool
+sc_keyboard_aoa_init(struct sc_keyboard_aoa *kb, struct sc_aoa *aoa);
+
+void
+sc_keyboard_aoa_destroy(struct sc_keyboard_aoa *kb);
+
+#endif

app/src/usb/mouse_aoa.c

@@ -0,0 +1,83 @@
+#include "mouse_aoa.h"
+
+#include <assert.h>
+
+#include "hid/hid_mouse.h"
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to mouse_aoa */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_aoa, mouse_processor)
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_motion_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_input hid_input;
+    sc_hid_mouse_generate_input_from_motion(&hid_input, event);
+
+    if (!sc_aoa_push_input(mouse->aoa, &hid_input)) {
+        LOGW("Could not push AOA HID input (mouse motion)");
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
+                                   const struct sc_mouse_click_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_input hid_input;
+    sc_hid_mouse_generate_input_from_click(&hid_input, event);
+
+    if (!sc_aoa_push_input(mouse->aoa, &hid_input)) {
+        LOGW("Could not push AOA HID input (mouse click)");
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_scroll_event *event) {
+    struct sc_mouse_aoa *mouse = DOWNCAST(mp);
+
+    struct sc_hid_input hid_input;
+    sc_hid_mouse_generate_input_from_scroll(&hid_input, event);
+
+    if (!sc_aoa_push_input(mouse->aoa, &hid_input)) {
+        LOGW("Could not push AOA HID input (mouse scroll)");
+    }
+}
+
+bool
+sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa) {
+    mouse->aoa = aoa;
+
+    struct sc_hid_open hid_open;
+    sc_hid_mouse_generate_open(&hid_open);
+
+    bool ok = sc_aoa_push_open(aoa, &hid_open, true);
+    if (!ok) {
+        LOGW("Could not push AOA HID open (mouse)");
+        return false;
+    }
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_mouse_click = sc_mouse_processor_process_mouse_click,
+        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
+        // Touch events not supported (coordinates are not relative)
+        .process_touch = NULL,
+    };
+
+    mouse->mouse_processor.ops = &ops;
+
+    mouse->mouse_processor.relative_mode = true;
+
+    return true;
+}
+
+void
+sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse) {
+    (void) mouse;
+    // Do nothing, mouse->aoa will automatically unregister all devices
+}

app/src/usb/mouse_aoa.h

@@ -1,5 +1,5 @@
-#ifndef SC_HID_MOUSE_H
+#ifndef SC_MOUSE_AOA_H
-#define SC_HID_MOUSE_H
+#define SC_MOUSE_AOA_H
 
 #include "common.h"
 
@@ -8,16 +8,16 @@
 #include "aoa_hid.h"
 #include "trait/mouse_processor.h"
 
-struct sc_hid_mouse {
+struct sc_mouse_aoa {
     struct sc_mouse_processor mouse_processor; // mouse processor trait
 
     struct sc_aoa *aoa;
 };
 
 bool
-sc_hid_mouse_init(struct sc_hid_mouse *mouse, struct sc_aoa *aoa);
+sc_mouse_aoa_init(struct sc_mouse_aoa *mouse, struct sc_aoa *aoa);
 
 void
-sc_hid_mouse_destroy(struct sc_hid_mouse *mouse);
+sc_mouse_aoa_destroy(struct sc_mouse_aoa *mouse);
 
 #endif

app/src/usb/scrcpy_otg.c

@@ -10,8 +10,9 @@
 struct scrcpy_otg {
     struct sc_usb usb;
     struct sc_aoa aoa;
-    struct sc_hid_keyboard keyboard;
+    struct sc_keyboard_aoa keyboard;
-    struct sc_hid_mouse mouse;
+    struct sc_mouse_aoa mouse;
+    struct sc_gamepad_aoa gamepad;
 
     struct sc_screen_otg screen_otg;
 };
@@ -21,10 +22,8 @@ sc_usb_on_disconnected(struct sc_usb *usb, void *userdata) {
     (void) usb;
     (void) userdata;
 
-    SDL_Event event;
+    bool ok = sc_push_event(SC_EVENT_USB_DEVICE_DISCONNECTED);
-    event.type = SC_EVENT_USB_DEVICE_DISCONNECTED;
+    if (!ok) {
-    int ret = SDL_PushEvent(&event);
-    if (ret < 0) {
         LOGE("Could not post USB disconnection event: %s", SDL_GetError());
     }
 }
@@ -37,6 +36,9 @@ event_loop(struct scrcpy_otg *s) {
             case SC_EVENT_USB_DEVICE_DISCONNECTED:
                 LOGW("Device disconnected");
                 return SCRCPY_EXIT_DISCONNECTED;
+            case SC_EVENT_AOA_OPEN_ERROR:
+                LOGE("AOA open error");
+                return SCRCPY_EXIT_FAILURE;
             case SDL_QUIT:
                 LOGD("User requested to quit");
                 return SCRCPY_EXIT_SUCCESS;
@@ -62,7 +64,14 @@ scrcpy_otg(struct scrcpy_options *options) {
     // Minimal SDL initialization
     if (SDL_Init(SDL_INIT_EVENTS)) {
         LOGE("Could not initialize SDL: %s", SDL_GetError());
-        return false;
+        return SCRCPY_EXIT_FAILURE;
+    }
+
+    if (options->gamepad_input_mode != SC_GAMEPAD_INPUT_MODE_DISABLED) {
+        if (SDL_Init(SDL_INIT_GAMECONTROLLER)) {
+            LOGE("Could not initialize SDL controller: %s", SDL_GetError());
+            // Not fatal, keyboard/mouse should still work
+        }
     }
 
     atexit(SDL_Quit);
@@ -73,8 +82,9 @@ scrcpy_otg(struct scrcpy_options *options) {
 
     enum scrcpy_exit_code ret = SCRCPY_EXIT_FAILURE;
 
-    struct sc_hid_keyboard *keyboard = NULL;
+    struct sc_keyboard_aoa *keyboard = NULL;
-    struct sc_hid_mouse *mouse = NULL;
+    struct sc_mouse_aoa *mouse = NULL;
+    struct sc_gamepad_aoa *gamepad = NULL;
     bool usb_device_initialized = false;
     bool usb_connected = false;
     bool aoa_started = false;
@@ -117,19 +127,22 @@ scrcpy_otg(struct scrcpy_options *options) {
     }
     aoa_initialized = true;
 
-    bool enable_keyboard =
+    assert(options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA
-        options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
+        || options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_DISABLED);
-    bool enable_mouse =
+    assert(options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA
-        options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
+        || options->mouse_input_mode == SC_MOUSE_INPUT_MODE_DISABLED);
+    assert(options->gamepad_input_mode == SC_GAMEPAD_INPUT_MODE_AOA
+        || options->gamepad_input_mode == SC_GAMEPAD_INPUT_MODE_DISABLED);
 
-    // If neither --hid-keyboard or --hid-mouse is passed, enable both
+    bool enable_keyboard =
-    if (!enable_keyboard && !enable_mouse) {
+        options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
-        enable_keyboard = true;
+    bool enable_mouse =
-        enable_mouse = true;
+        options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
-    }
+    bool enable_gamepad =
+        options->gamepad_input_mode == SC_GAMEPAD_INPUT_MODE_AOA;
 
     if (enable_keyboard) {
-        ok = sc_hid_keyboard_init(&s->keyboard, &s->aoa);
+        ok = sc_keyboard_aoa_init(&s->keyboard, &s->aoa);
         if (!ok) {
             goto end;
         }
@@ -137,13 +150,18 @@ scrcpy_otg(struct scrcpy_options *options) {
     }
 
     if (enable_mouse) {
-        ok = sc_hid_mouse_init(&s->mouse, &s->aoa);
+        ok = sc_mouse_aoa_init(&s->mouse, &s->aoa);
         if (!ok) {
             goto end;
         }
         mouse = &s->mouse;
     }
 
+    if (enable_gamepad) {
+        sc_gamepad_aoa_init(&s->gamepad, &s->aoa);
+        gamepad = &s->gamepad;
+    }
+
     ok = sc_aoa_start(&s->aoa);
     if (!ok) {
         goto end;
@@ -158,6 +176,7 @@ scrcpy_otg(struct scrcpy_options *options) {
     struct sc_screen_otg_params params = {
         .keyboard = keyboard,
         .mouse = mouse,
+        .gamepad = gamepad,
         .window_title = window_title,
         .always_on_top = options->always_on_top,
         .window_x = options->window_x,
@@ -186,10 +205,13 @@ end:
     sc_usb_stop(&s->usb);
 
     if (mouse) {
-        sc_hid_mouse_destroy(&s->mouse);
+        sc_mouse_aoa_destroy(&s->mouse);
     }
     if (keyboard) {
-        sc_hid_keyboard_destroy(&s->keyboard);
+        sc_keyboard_aoa_destroy(&s->keyboard);
+    }
+    if (gamepad) {
+        sc_gamepad_aoa_destroy(&s->gamepad);
     }
 
     if (aoa_initialized) {

app/src/usb/screen_otg.c

@@ -59,6 +59,7 @@ sc_screen_otg_init(struct sc_screen_otg *screen,
                    const struct sc_screen_otg_params *params) {
     screen->keyboard = params->keyboard;
     screen->mouse = params->mouse;
+    screen->gamepad = params->gamepad;
 
     screen->mouse_capture_key_pressed = 0;
 
@@ -169,7 +170,7 @@ sc_screen_otg_process_mouse_motion(struct sc_screen_otg *screen,
         // .position not used for HID events
         .xrel = event->xrel,
         .yrel = event->yrel,
-        .buttons_state = sc_mouse_buttons_state_from_sdl(event->state, true),
+        .buttons_state = sc_mouse_buttons_state_from_sdl(event->state),
     };
 
     assert(mp->ops->process_mouse_motion);
@@ -188,8 +189,7 @@ sc_screen_otg_process_mouse_button(struct sc_screen_otg *screen,
         // .position not used for HID events
         .action = sc_action_from_sdl_mousebutton_type(event->type),
         .button = sc_mouse_button_from_sdl(event->button),
-        .buttons_state =
+        .buttons_state = sc_mouse_buttons_state_from_sdl(sdl_buttons_state),
-            sc_mouse_buttons_state_from_sdl(sdl_buttons_state, true),
     };
 
     assert(mp->ops->process_mouse_click);
@@ -208,14 +208,84 @@ sc_screen_otg_process_mouse_wheel(struct sc_screen_otg *screen,
         // .position not used for HID events
         .hscroll = event->x,
         .vscroll = event->y,
-        .buttons_state =
+        .buttons_state = sc_mouse_buttons_state_from_sdl(sdl_buttons_state),
-            sc_mouse_buttons_state_from_sdl(sdl_buttons_state, true),
     };
 
     assert(mp->ops->process_mouse_scroll);
     mp->ops->process_mouse_scroll(mp, &evt);
 }
 
+static void
+sc_screen_otg_process_gamepad_device(struct sc_screen_otg *screen,
+                                     const SDL_ControllerDeviceEvent *event) {
+    assert(screen->gamepad);
+    struct sc_gamepad_processor *gp = &screen->gamepad->gamepad_processor;
+
+    SDL_JoystickID id;
+    if (event->type == SDL_CONTROLLERDEVICEADDED) {
+        SDL_GameController *gc = SDL_GameControllerOpen(event->which);
+        if (!gc) {
+            LOGW("Could not open game controller");
+            return;
+        }
+
+        SDL_Joystick *joystick = SDL_GameControllerGetJoystick(gc);
+        if (!joystick) {
+            LOGW("Could not get controller joystick");
+            SDL_GameControllerClose(gc);
+            return;
+        }
+
+        id = SDL_JoystickInstanceID(joystick);
+    } else if (event->type == SDL_CONTROLLERDEVICEREMOVED) {
+        id = event->which;
+
+        SDL_GameController *gc = SDL_GameControllerFromInstanceID(id);
+        if (gc) {
+            SDL_GameControllerClose(gc);
+        } else {
+            LOGW("Unknown gamepad device removed");
+        }
+    } else {
+        // Nothing to do
+        return;
+    }
+
+    struct sc_gamepad_device_event evt = {
+        .type = sc_gamepad_device_event_type_from_sdl_type(event->type),
+        .gamepad_id = id,
+    };
+    gp->ops->process_gamepad_device(gp, &evt);
+}
+
+static void
+sc_screen_otg_process_gamepad_axis(struct sc_screen_otg *screen,
+                                   const SDL_ControllerAxisEvent *event) {
+    assert(screen->gamepad);
+    struct sc_gamepad_processor *gp = &screen->gamepad->gamepad_processor;
+
+    struct sc_gamepad_axis_event evt = {
+        .gamepad_id = event->which,
+        .axis = sc_gamepad_axis_from_sdl(event->axis),
+        .value = event->value,
+    };
+    gp->ops->process_gamepad_axis(gp, &evt);
+}
+
+static void
+sc_screen_otg_process_gamepad_button(struct sc_screen_otg *screen,
+                                     const SDL_ControllerButtonEvent *event) {
+    assert(screen->gamepad);
+    struct sc_gamepad_processor *gp = &screen->gamepad->gamepad_processor;
+
+    struct sc_gamepad_button_event evt = {
+        .gamepad_id = event->which,
+        .action = sc_action_from_sdl_controllerbutton_type(event->type),
+        .button = sc_gamepad_button_from_sdl(event->button),
+    };
+    gp->ops->process_gamepad_button(gp, &evt);
+}
+
 void
 sc_screen_otg_handle_event(struct sc_screen_otg *screen, SDL_Event *event) {
     switch (event->type) {
@@ -295,5 +365,23 @@ sc_screen_otg_handle_event(struct sc_screen_otg *screen, SDL_Event *event) {
                 sc_screen_otg_process_mouse_wheel(screen, &event->wheel);
             }
             break;
+        case SDL_CONTROLLERDEVICEADDED:
+        case SDL_CONTROLLERDEVICEREMOVED:
+            // Handle device added or removed even if paused
+            if (screen->gamepad) {
+                sc_screen_otg_process_gamepad_device(screen, &event->cdevice);
+            }
+            break;
+        case SDL_CONTROLLERAXISMOTION:
+            if (screen->gamepad) {
+                sc_screen_otg_process_gamepad_axis(screen, &event->caxis);
+            }
+            break;
+        case SDL_CONTROLLERBUTTONDOWN:
+        case SDL_CONTROLLERBUTTONUP:
+            if (screen->gamepad) {
+                sc_screen_otg_process_gamepad_button(screen, &event->cbutton);
+            }
+            break;
     }
 }

app/src/usb/screen_otg.h

@@ -6,12 +6,14 @@
 #include <stdbool.h>
 #include <SDL2/SDL.h>
 
-#include "hid_keyboard.h"
+#include "keyboard_aoa.h"
-#include "hid_mouse.h"
+#include "mouse_aoa.h"
+#include "gamepad_aoa.h"
 
 struct sc_screen_otg {
-    struct sc_hid_keyboard *keyboard;
+    struct sc_keyboard_aoa *keyboard;
-    struct sc_hid_mouse *mouse;
+    struct sc_mouse_aoa *mouse;
+    struct sc_gamepad_aoa *gamepad;
 
     SDL_Window *window;
     SDL_Renderer *renderer;
@@ -22,8 +24,9 @@ struct sc_screen_otg {
 };
 
 struct sc_screen_otg_params {
-    struct sc_hid_keyboard *keyboard;
+    struct sc_keyboard_aoa *keyboard;
-    struct sc_hid_mouse *mouse;
+    struct sc_mouse_aoa *mouse;
+    struct sc_gamepad_aoa *gamepad;
 
     const char *window_title;
     bool always_on_top;

app/src/util/audiobuf.c

@@ -0,0 +1,118 @@
+#include "audiobuf.h"
+
+#include <stdlib.h>
+#include <string.h>
+#include <util/log.h>
+#include <util/memory.h>
+
+bool
+sc_audiobuf_init(struct sc_audiobuf *buf, size_t sample_size,
+                 uint32_t capacity) {
+    assert(sample_size);
+    assert(capacity);
+
+    // The actual capacity is (alloc_size - 1) so that head == tail is
+    // non-ambiguous
+    buf->alloc_size = capacity + 1;
+    buf->data = sc_allocarray(buf->alloc_size, sample_size);
+    if (!buf->data) {
+        LOG_OOM();
+        return false;
+    }
+
+    buf->sample_size = sample_size;
+    atomic_init(&buf->head, 0);
+    atomic_init(&buf->tail, 0);
+
+    return true;
+}
+
+void
+sc_audiobuf_destroy(struct sc_audiobuf *buf) {
+    free(buf->data);
+}
+
+uint32_t
+sc_audiobuf_read(struct sc_audiobuf *buf, void *to_, uint32_t samples_count) {
+    assert(samples_count);
+
+    uint8_t *to = to_;
+
+    // Only the reader thread can write tail without synchronization, so
+    // memory_order_relaxed is sufficient
+    uint32_t tail = atomic_load_explicit(&buf->tail, memory_order_relaxed);
+
+    // The head cursor is updated after the data is written to the array
+    uint32_t head = atomic_load_explicit(&buf->head, memory_order_acquire);
+
+    uint32_t can_read = (buf->alloc_size + head - tail) % buf->alloc_size;
+    if (!can_read) {
+        return 0;
+    }
+    if (samples_count > can_read) {
+        samples_count = can_read;
+    }
+
+    if (to) {
+        uint32_t right_count = buf->alloc_size - tail;
+        if (right_count > samples_count) {
+            right_count = samples_count;
+        }
+        memcpy(to,
+               buf->data + (tail * buf->sample_size),
+               right_count * buf->sample_size);
+
+        if (samples_count > right_count) {
+            uint32_t left_count = samples_count - right_count;
+            memcpy(to + (right_count * buf->sample_size),
+                   buf->data,
+                   left_count * buf->sample_size);
+        }
+    }
+
+    uint32_t new_tail = (tail + samples_count) % buf->alloc_size;
+    atomic_store_explicit(&buf->tail, new_tail, memory_order_release);
+
+    return samples_count;
+}
+
+uint32_t
+sc_audiobuf_write(struct sc_audiobuf *buf, const void *from_,
+                  uint32_t samples_count) {
+    const uint8_t *from = from_;
+
+    // Only the writer thread can write head, so memory_order_relaxed is
+    // sufficient
+    uint32_t head = atomic_load_explicit(&buf->head, memory_order_relaxed);
+
+    // The tail cursor is updated after the data is consumed by the reader
+    uint32_t tail = atomic_load_explicit(&buf->tail, memory_order_acquire);
+
+    uint32_t can_write = (buf->alloc_size + tail - head - 1) % buf->alloc_size;
+    if (!can_write) {
+        return 0;
+    }
+    if (samples_count > can_write) {
+        samples_count = can_write;
+    }
+
+    uint32_t right_count = buf->alloc_size - head;
+    if (right_count > samples_count) {
+        right_count = samples_count;
+    }
+    memcpy(buf->data + (head * buf->sample_size),
+           from,
+           right_count * buf->sample_size);
+
+    if (samples_count > right_count) {
+        uint32_t left_count = samples_count - right_count;
+        memcpy(buf->data,
+               from + (right_count * buf->sample_size),
+               left_count * buf->sample_size);
+    }
+
+    uint32_t new_head = (head + samples_count) % buf->alloc_size;
+    atomic_store_explicit(&buf->head, new_head, memory_order_release);
+
+    return samples_count;
+}

app/src/util/audiobuf.h

@@ -3,19 +3,25 @@
 
 #include "common.h"
 
+#include <assert.h>
+#include <stdatomic.h>
 #include <stdbool.h>
 #include <stdint.h>
 
-#include "util/bytebuf.h"
-
 /**
  * Wrapper around bytebuf to read and write samples
  *
  * Each sample takes sample_size bytes.
  */
 struct sc_audiobuf {
-    struct sc_bytebuf buf;
+    uint8_t *data;
+    uint32_t alloc_size; // in samples
     size_t sample_size;
+
+    atomic_uint_least32_t head; // writer cursor, in samples
+    atomic_uint_least32_t tail; // reader cursor, in samples
+    // empty: tail == head
+    // full: ((tail + 1) % alloc_size) == head
 };
 
 static inline uint32_t
@@ -29,66 +35,31 @@ sc_audiobuf_to_bytes(struct sc_audiobuf *buf, uint32_t samples) {
     return samples * buf->sample_size;
 }
 
-static inline bool
+bool
 sc_audiobuf_init(struct sc_audiobuf *buf, size_t sample_size,
-                 uint32_t capacity) {
+                 uint32_t capacity);
-    buf->sample_size = sample_size;
-    return sc_bytebuf_init(&buf->buf, capacity * sample_size + 1);
-}
 
-static inline void
+void
-sc_audiobuf_read(struct sc_audiobuf *buf, uint8_t *to, uint32_t samples) {
+sc_audiobuf_destroy(struct sc_audiobuf *buf);
-    size_t bytes = sc_audiobuf_to_bytes(buf, samples);
-    sc_bytebuf_read(&buf->buf, to, bytes);
-}
 
-static inline void
+uint32_t
-sc_audiobuf_skip(struct sc_audiobuf *buf, uint32_t samples) {
+sc_audiobuf_read(struct sc_audiobuf *buf, void *to, uint32_t samples_count);
-    size_t bytes = sc_audiobuf_to_bytes(buf, samples);
-    sc_bytebuf_skip(&buf->buf, bytes);
-}
 
-static inline void
+uint32_t
-sc_audiobuf_write(struct sc_audiobuf *buf, const uint8_t *from,
+sc_audiobuf_write(struct sc_audiobuf *buf, const void *from,
-                  uint32_t samples) {
+                  uint32_t samples_count);
-    size_t bytes = sc_audiobuf_to_bytes(buf, samples);
-    sc_bytebuf_write(&buf->buf, from, bytes);
-}
 
-static inline void
+static inline uint32_t
-sc_audiobuf_prepare_write(struct sc_audiobuf *buf, const uint8_t *from,
+sc_audiobuf_capacity(struct sc_audiobuf *buf) {
-                          uint32_t samples) {
+    assert(buf->alloc_size);
-    size_t bytes = sc_audiobuf_to_bytes(buf, samples);
+    return buf->alloc_size - 1;
-    sc_bytebuf_prepare_write(&buf->buf, from, bytes);
-}
-
-static inline void
-sc_audiobuf_commit_write(struct sc_audiobuf *buf, uint32_t samples) {
-    size_t bytes = sc_audiobuf_to_bytes(buf, samples);
-    sc_bytebuf_commit_write(&buf->buf, bytes);
 }
 
 static inline uint32_t
 sc_audiobuf_can_read(struct sc_audiobuf *buf) {
-    size_t bytes = sc_bytebuf_can_read(&buf->buf);
+    uint32_t head = atomic_load_explicit(&buf->head, memory_order_acquire);
-    return sc_audiobuf_to_samples(buf, bytes);
+    uint32_t tail = atomic_load_explicit(&buf->tail, memory_order_acquire);
-}
+    return (buf->alloc_size + head - tail) % buf->alloc_size;
-
-static inline uint32_t
-sc_audiobuf_can_write(struct sc_audiobuf *buf) {
-    size_t bytes = sc_bytebuf_can_write(&buf->buf);
-    return sc_audiobuf_to_samples(buf, bytes);
-}
-
-static inline uint32_t
-sc_audiobuf_capacity(struct sc_audiobuf *buf) {
-    size_t bytes = sc_bytebuf_capacity(&buf->buf);
-    return sc_audiobuf_to_samples(buf, bytes);
-}
-
-static inline void
-sc_audiobuf_destroy(struct sc_audiobuf *buf) {
-    sc_bytebuf_destroy(&buf->buf);
 }
 
 #endif

app/src/util/binary.h

@@ -13,6 +13,12 @@ sc_write16be(uint8_t *buf, uint16_t value) {
     buf[1] = value;
 }
 
+static inline void
+sc_write16le(uint8_t *buf, uint16_t value) {
+    buf[0] = value;
+    buf[1] = value >> 8;
+}
+
 static inline void
 sc_write32be(uint8_t *buf, uint32_t value) {
     buf[0] = value >> 24;
@@ -21,12 +27,26 @@ sc_write32be(uint8_t *buf, uint32_t value) {
     buf[3] = value;
 }
 
+static inline void
+sc_write32le(uint8_t *buf, uint32_t value) {
+    buf[0] = value;
+    buf[1] = value >> 8;
+    buf[2] = value >> 16;
+    buf[3] = value >> 24;
+}
+
 static inline void
 sc_write64be(uint8_t *buf, uint64_t value) {
     sc_write32be(buf, value >> 32);
     sc_write32be(&buf[4], (uint32_t) value);
 }
 
+static inline void
+sc_write64le(uint8_t *buf, uint64_t value) {
+    sc_write32le(buf, (uint32_t) value);
+    sc_write32le(buf, value >> 32);
+}
+
 static inline uint16_t
 sc_read16be(const uint8_t *buf) {
     return (buf[0] << 8) | buf[1];

app/src/util/bytebuf.c

@@ -1,104 +0,0 @@
-#include "bytebuf.h"
-
-#include <assert.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include "util/log.h"
-
-bool
-sc_bytebuf_init(struct sc_bytebuf *buf, size_t alloc_size) {
-    assert(alloc_size);
-    buf->data = malloc(alloc_size);
-    if (!buf->data) {
-        LOG_OOM();
-        return false;
-    }
-
-    buf->alloc_size = alloc_size;
-    buf->head = 0;
-    buf->tail = 0;
-
-    return true;
-}
-
-void
-sc_bytebuf_destroy(struct sc_bytebuf *buf) {
-    free(buf->data);
-}
-
-void
-sc_bytebuf_read(struct sc_bytebuf *buf, uint8_t *to, size_t len) {
-    assert(len);
-    assert(len <= sc_bytebuf_can_read(buf));
-    assert(buf->tail != buf->head); // the buffer could not be empty
-
-    size_t right_limit = buf->tail < buf->head ? buf->head : buf->alloc_size;
-    size_t right_len = right_limit - buf->tail;
-    if (len < right_len) {
-        right_len = len;
-    }
-    memcpy(to, buf->data + buf->tail, right_len);
-
-    if (len > right_len) {
-        memcpy(to + right_len, buf->data, len - right_len);
-    }
-
-    buf->tail = (buf->tail + len) % buf->alloc_size;
-}
-
-void
-sc_bytebuf_skip(struct sc_bytebuf *buf, size_t len) {
-    assert(len);
-    assert(len <= sc_bytebuf_can_read(buf));
-    assert(buf->tail != buf->head); // the buffer could not be empty
-
-    buf->tail = (buf->tail + len) % buf->alloc_size;
-}
-
-static inline void
-sc_bytebuf_write_step0(struct sc_bytebuf *buf, const uint8_t *from,
-                       size_t len) {
-    size_t right_len = buf->alloc_size - buf->head;
-    if (len < right_len) {
-        right_len = len;
-    }
-    memcpy(buf->data + buf->head, from, right_len);
-
-    if (len > right_len) {
-        memcpy(buf->data, from + right_len, len - right_len);
-    }
-}
-
-static inline void
-sc_bytebuf_write_step1(struct sc_bytebuf *buf, size_t len) {
-    buf->head = (buf->head + len) % buf->alloc_size;
-}
-
-void
-sc_bytebuf_write(struct sc_bytebuf *buf, const uint8_t *from, size_t len) {
-    assert(len);
-    assert(len <= sc_bytebuf_can_write(buf));
-
-    sc_bytebuf_write_step0(buf, from, len);
-    sc_bytebuf_write_step1(buf, len);
-}
-
-void
-sc_bytebuf_prepare_write(struct sc_bytebuf *buf, const uint8_t *from,
-                         size_t len) {
-    // *This function MUST NOT access buf->tail (even in assert()).*
-    // The purpose of this function is to allow a reader and a writer to access
-    // different parts of the buffer in parallel simultaneously. It is intended
-    // to be called without lock (only sc_bytebuf_commit_write() is intended to
-    // be called with lock held).
-
-    assert(len < buf->alloc_size - 1);
-    sc_bytebuf_write_step0(buf, from, len);
-}
-
-void
-sc_bytebuf_commit_write(struct sc_bytebuf *buf, size_t len) {
-    assert(len <= sc_bytebuf_can_write(buf));
-    sc_bytebuf_write_step1(buf, len);
-}

app/src/util/bytebuf.h

@@ -1,114 +0,0 @@
-#ifndef SC_BYTEBUF_H
-#define SC_BYTEBUF_H
-
-#include "common.h"
-
-#include <stdbool.h>
-#include <stdint.h>
-
-struct sc_bytebuf {
-    uint8_t *data;
-    // The actual capacity is (allocated - 1) so that head == tail is
-    // non-ambiguous
-    size_t alloc_size;
-    size_t head; // writter cursor
-    size_t tail; // reader cursor
-    // empty: tail == head
-    // full: ((tail + 1) % alloc_size) == head
-};
-
-bool
-sc_bytebuf_init(struct sc_bytebuf *buf, size_t alloc_size);
-
-/**
- * Copy from the bytebuf to a user-provided array
- *
- * The caller must check that len <= sc_bytebuf_read_available() (it is an
- * error to attempt to read more bytes than available).
- *
- * This function is guaranteed not to write to buf->head.
- */
-void
-sc_bytebuf_read(struct sc_bytebuf *buf, uint8_t *to, size_t len);
-
-/**
- * Drop len bytes from the buffer
- *
- * The caller must check that len <= sc_bytebuf_read_available() (it is an
- * error to attempt to skip more bytes than available).
- *
- * This function is guaranteed not to write to buf->head.
- *
- * It is equivalent to call sc_bytebuf_read() to some array and discard the
- * array (but this function is more efficient since there is no copy).
- */
-void
-sc_bytebuf_skip(struct sc_bytebuf *buf, size_t len);
-
-/**
- * Copy the user-provided array to the bytebuf
- *
- * The caller must check that len <= sc_bytebuf_write_available() (it is an
- * error to write more bytes than the remaining available space).
- *
- * This function is guaranteed not to write to buf->tail.
- */
-void
-sc_bytebuf_write(struct sc_bytebuf *buf, const uint8_t *from, size_t len);
-
-/**
- * Copy the user-provided array to the bytebuf, but do not advance the cursor
- *
- * The caller must check that len <= sc_bytebuf_write_available() (it is an
- * error to write more bytes than the remaining available space).
- *
- * After this function is called, the write must be committed with
- * sc_bytebuf_commit_write().
- *
- * The purpose of this mechanism is to acquire a lock only to commit the write,
- * but not to perform the actual copy.
- *
- * This function is guaranteed not to access buf->tail.
- */
-void
-sc_bytebuf_prepare_write(struct sc_bytebuf *buf, const uint8_t *from,
-                         size_t len);
-
-/**
- * Commit a prepared write
- */
-void
-sc_bytebuf_commit_write(struct sc_bytebuf *buf, size_t len);
-
-/**
- * Return the number of bytes which can be read
- *
- * It is an error to read more bytes than available.
- */
-static inline size_t
-sc_bytebuf_can_read(struct sc_bytebuf *buf) {
-    return (buf->alloc_size + buf->head - buf->tail) % buf->alloc_size;
-}
-
-/**
- * Return the number of bytes which can be written
- *
- * It is an error to write more bytes than available.
- */
-static inline size_t
-sc_bytebuf_can_write(struct sc_bytebuf *buf) {
-    return (buf->alloc_size + buf->tail - buf->head - 1) % buf->alloc_size;
-}
-
-/**
- * Return the actual capacity of the buffer (can_read() + can_write())
- */
-static inline size_t
-sc_bytebuf_capacity(struct sc_bytebuf *buf) {
-    return buf->alloc_size - 1;
-}
-
-void
-sc_bytebuf_destroy(struct sc_bytebuf *buf);
-
-#endif

app/src/util/str.c

@@ -3,6 +3,7 @@
 #include <assert.h>
 #include <errno.h>
 #include <limits.h>
+#include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 
@@ -333,3 +334,22 @@ sc_str_remove_trailing_cr(char *s, size_t len) {
     }
     return len;
 }
+
+char *
+sc_str_to_hex_string(const uint8_t *data, size_t size) {
+    size_t buffer_size = size * 3 + 1;
+    char *buffer = malloc(buffer_size);
+    if (!buffer) {
+        LOG_OOM();
+        return NULL;
+    }
+
+    for (size_t i = 0; i < size; ++i) {
+        snprintf(buffer + i * 3, 4, "%02X ", data[i]);
+    }
+
+    // Remove the final space
+    buffer[size * 3] = '\0';
+
+    return buffer;
+}

app/src/util/str.h

@@ -138,4 +138,10 @@ sc_str_index_of_column(const char *s, unsigned col, const char *seps);
 size_t
 sc_str_remove_trailing_cr(char *s, size_t len);
 
+/**
+ * Convert binary data to hexadecimal string
+ */
+char *
+sc_str_to_hex_string(const uint8_t *data, size_t len);
+
 #endif

app/src/util/thread.c

@@ -6,6 +6,8 @@
 
 #include "log.h"
 
+sc_thread_id SC_MAIN_THREAD_ID;
+
 bool
 sc_thread_create(sc_thread *thread, sc_thread_fn fn, const char *name,
                  void *userdata) {

app/src/util/thread.h

@@ -39,6 +39,8 @@ typedef struct sc_cond {
     SDL_cond *cond;
 } sc_cond;
 
+extern sc_thread_id SC_MAIN_THREAD_ID;
+
 bool
 sc_thread_create(sc_thread *thread, sc_thread_fn fn, const char *name,
                  void *userdata);

app/src/v4l2_sink.c

@@ -240,7 +240,7 @@ sc_v4l2_sink_open(struct sc_v4l2_sink *vs, const AVCodecContext *ctx) {
     vs->frame = av_frame_alloc();
     if (!vs->frame) {
         LOG_OOM();
-        goto error_avcodec_close;
+        goto error_avcodec_free_context;
     }
 
     vs->packet = av_packet_alloc();
@@ -268,8 +268,6 @@ error_av_packet_free:
     av_packet_free(&vs->packet);
 error_av_frame_free:
     av_frame_free(&vs->frame);
-error_avcodec_close:
-    avcodec_close(vs->encoder_ctx);
 error_avcodec_free_context:
     avcodec_free_context(&vs->encoder_ctx);
 error_avio_close:
@@ -297,7 +295,6 @@ sc_v4l2_sink_close(struct sc_v4l2_sink *vs) {
 
     av_packet_free(&vs->packet);
     av_frame_free(&vs->frame);
-    avcodec_close(vs->encoder_ctx);
     avcodec_free_context(&vs->encoder_ctx);
     avio_close(vs->format_ctx->pb);
     avformat_free_context(vs->format_ctx);

app/tests/test_audiobuf.c

@@ -0,0 +1,128 @@
+#include "common.h"
+
+#include <assert.h>
+#include <string.h>
+
+#include "util/audiobuf.h"
+
+static void test_audiobuf_simple(void) {
+    struct sc_audiobuf buf;
+    uint32_t data[20];
+
+    bool ok = sc_audiobuf_init(&buf, 4, 20);
+    assert(ok);
+
+    uint32_t samples[] = {1, 2, 3, 4, 5};
+    uint32_t w = sc_audiobuf_write(&buf, samples, 5);
+    assert(w == 5);
+
+    uint32_t r = sc_audiobuf_read(&buf, data, 4);
+    assert(r == 4);
+    assert(!memcmp(data, samples, 16));
+
+    uint32_t samples2[] = {6, 7, 8};
+    w = sc_audiobuf_write(&buf, samples2, 3);
+    assert(w == 3);
+
+    uint32_t single = 9;
+    w = sc_audiobuf_write(&buf, &single, 1);
+    assert(w == 1);
+
+    r = sc_audiobuf_read(&buf, &data[4], 8);
+    assert(r == 5);
+
+    uint32_t expected[] = {1, 2, 3, 4, 5, 6, 7, 8, 9};
+    assert(!memcmp(data, expected, 36));
+
+    sc_audiobuf_destroy(&buf);
+}
+
+static void test_audiobuf_boundaries(void) {
+    struct sc_audiobuf buf;
+    uint32_t data[20];
+
+    bool ok = sc_audiobuf_init(&buf, 4, 20);
+    assert(ok);
+
+    uint32_t samples[] = {1, 2, 3, 4, 5, 6};
+    uint32_t w = sc_audiobuf_write(&buf, samples, 6);
+    assert(w == 6);
+
+    w = sc_audiobuf_write(&buf, samples, 6);
+    assert(w == 6);
+
+    w = sc_audiobuf_write(&buf, samples, 6);
+    assert(w == 6);
+
+    uint32_t r = sc_audiobuf_read(&buf, data, 9);
+    assert(r == 9);
+
+    uint32_t expected[] = {1, 2, 3, 4, 5, 6, 1, 2, 3};
+    assert(!memcmp(data, expected, 36));
+
+    uint32_t samples2[] = {7, 8, 9, 10, 11};
+    w = sc_audiobuf_write(&buf, samples2, 5);
+    assert(w == 5);
+
+    uint32_t single = 12;
+    w = sc_audiobuf_write(&buf, &single, 1);
+    assert(w == 1);
+
+    w = sc_audiobuf_read(&buf, NULL, 3);
+    assert(w == 3);
+
+    assert(sc_audiobuf_can_read(&buf) == 12);
+
+    r = sc_audiobuf_read(&buf, data, 12);
+    assert(r == 12);
+
+    uint32_t expected2[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};
+    assert(!memcmp(data, expected2, 48));
+
+    sc_audiobuf_destroy(&buf);
+}
+
+static void test_audiobuf_partial_read_write(void) {
+    struct sc_audiobuf buf;
+    uint32_t data[15];
+
+    bool ok = sc_audiobuf_init(&buf, 4, 10);
+    assert(ok);
+
+    uint32_t samples[] = {1, 2, 3, 4, 5, 6};
+    uint32_t w = sc_audiobuf_write(&buf, samples, 6);
+    assert(w == 6);
+
+    w = sc_audiobuf_write(&buf, samples, 6);
+    assert(w == 4);
+
+    w = sc_audiobuf_write(&buf, samples, 6);
+    assert(w == 0);
+
+    uint32_t r = sc_audiobuf_read(&buf, data, 3);
+    assert(r == 3);
+
+    uint32_t expected[] = {1, 2, 3};
+    assert(!memcmp(data, expected, 12));
+
+    w = sc_audiobuf_write(&buf, samples, 6);
+    assert(w == 3);
+
+    r = sc_audiobuf_read(&buf, data, 15);
+    assert(r == 10);
+    uint32_t expected2[] = {4, 5, 6, 1, 2, 3, 4, 1, 2, 3};
+    assert(!memcmp(data, expected2, 12));
+
+    sc_audiobuf_destroy(&buf);
+}
+
+int main(int argc, char *argv[]) {
+    (void) argc;
+    (void) argv;
+
+    test_audiobuf_simple();
+    test_audiobuf_boundaries();
+    test_audiobuf_partial_read_write();
+
+    return 0;
+}