Extract mouse processor trait

Mainly for consistency with the keyboard processor trait.

This could allow to provide alternative mouse processors.

BUILD.md

@@ -268,10 +268,10 @@ install` must be run as root)._
 
 #### Option 2: Use prebuilt server
 
- - [`scrcpy-server-v1.18`][direct-scrcpy-server]  
-   _(SHA-256: 641c5c6beda9399dfae72d116f5ff43b5ed1059d871c9ebc3f47610fd33c51a3)_
+ - [`scrcpy-server-v1.19`][direct-scrcpy-server]  
+   _(SHA-256: 876f9322182e6aac6a58db1334f4225855ef3a17eaebc80aab6601d9d1ecb867)_
 
-[direct-scrcpy-server]: https://github.com/Genymobile/scrcpy/releases/download/v1.18/scrcpy-server-v1.18
+[direct-scrcpy-server]: https://github.com/Genymobile/scrcpy/releases/download/v1.19/scrcpy-server-v1.19
 
 Download the prebuilt server somewhere, and specify its path during the Meson
 configuration:

FAQ.md

@@ -153,6 +153,26 @@ You may also need to configure the [scaling behavior]:
 [scaling behavior]: https://github.com/Genymobile/scrcpy/issues/40#issuecomment-424466723
 
 
+### Issue with Wayland
+
+By default, SDL uses x11 on Linux. The [video driver] can be changed via the
+`SDL_VIDEODRIVER` environment variable:
+
+[video driver]: https://wiki.libsdl.org/FAQUsingSDL#how_do_i_choose_a_specific_video_driver
+
+```bash
+export SDL_VIDEODRIVER=wayland
+scrcpy
+```
+
+On some distributions (at least Fedora), the package `libdecor` must be
+installed manually.
+
+See issues [#2554] and [#2559].
+
+[#2554]: https://github.com/Genymobile/scrcpy/issues/2554
+[#2559]: https://github.com/Genymobile/scrcpy/issues/2559
+
 
 ### KWin compositor crashes
 
@@ -240,3 +260,4 @@ This FAQ is available in other languages:
 
  - [Italiano (Italiano, `it`) - v1.17](FAQ.it.md)
  - [한국어 (Korean, `ko`) - v1.11](FAQ.ko.md)
+ - [简体中文 (Simplified Chinese, `zh-Hans`) - v1.18](FAQ.zh-Hans.md)

FAQ.zh-Hans.md

@@ -0,0 +1,240 @@
+只有原版的[FAQ](FAQ.md)会保持更新。
+本文根据[d6aaa5]翻译。
+
+[d6aaa5]:https://github.com/Genymobile/scrcpy/blob/d6aaa5bf9aa3710660c683b6e3e0ed971ee44af5/FAQ.md
+
+# 常见问题
+
+这里是一些常见的问题以及他们的状态。
+
+## `adb` 相关问题
+
+`scrcpy` 执行 `adb` 命令来初始化和设备之间的连接。如果`adb` 执行失败了， scrcpy 就无法工作。
+
+在这种情况中，将会输出这个错误:
+
+>     ERROR: "adb push" returned with value 1
+
+这通常不是 _scrcpy_ 的bug，而是你的环境的问题。
+
+要找出原因，请执行以下操作：
+
+```bash
+adb devices
+```
+
+### 找不到`adb`
+
+
+你的`PATH`中需要能访问到`adb`。
+
+在Windows上，当前目录会包含在`PATH`中，并且`adb.exe`也包含在发行版中，因此它应该是开箱即用（直接解压就可以）的。
+
+
+### 设备未授权
+
+参见这里 [stackoverflow][device-unauthorized].
+
+[device-unauthorized]: https://stackoverflow.com/questions/23081263/adb-android-device-unauthorized
+
+
+### 未检测到设备
+
+>     adb: error: failed to get feature set: no devices/emulators found
+
+确认已经正确启用 [adb debugging][enable-adb].
+
+如果你的设备没有被检测到，你可能需要一些[驱动][drivers] (在 Windows上).
+
+[enable-adb]: https://developer.android.com/studio/command-line/adb.html#Enabling
+[drivers]: https://developer.android.com/studio/run/oem-usb.html
+
+
+### 已连接多个设备
+
+如果连接了多个设备，您将遇到以下错误：
+
+>     adb: error: failed to get feature set: more than one device/emulator
+
+必须提供要镜像的设备的标识符：
+
+```bash
+scrcpy -s 01234567890abcdef
+```
+
+注意，如果你的设备是通过 TCP/IP 连接的， 你将会收到以下消息：
+
+>     adb: error: more than one device/emulator
+>     ERROR: "adb reverse" returned with value 1
+>     WARN: 'adb reverse' failed, fallback to 'adb forward'
+
+这是意料之中的 (由于旧版安卓的一个bug, 请参见 [#5])，但是在这种情况下，scrcpy会退回到另一种方法，这种方法应该可以起作用。
+
+[#5]: https://github.com/Genymobile/scrcpy/issues/5
+
+
+### adb版本之间冲突
+
+>     adb server version (41) doesn't match this client (39); killing...
+
+同时使用多个版本的`adb`时会发生此错误。你必须查找使用不同`adb`版本的程序，并在所有地方使用相同版本的`adb`。
+
+你可以覆盖另一个程序中的`adb`二进制文件，或者通过设置`ADB`环境变量来让 _scrcpy_ 使用特定的`adb`二进制文件。
+
+```bash
+set ADB=/path/to/your/adb
+scrcpy
+```
+
+
+### 设备断开连接
+
+如果 _scrcpy_ 在警告“设备连接断开”的情况下自动中止，那就意味着`adb`连接已经断开了。
+请尝试使用另一条USB线或者电脑上的另一个USB接口。
+请参看 [#281] 和 [#283]。
+
+[#281]: https://github.com/Genymobile/scrcpy/issues/281
+[#283]: https://github.com/Genymobile/scrcpy/issues/283
+
+## 控制相关问题
+
+### 鼠标和键盘不起作用
+
+
+在某些设备上，您可能需要启用一个选项以允许 [模拟输入][simulating input]。
+
+在开发者选项中，打开：
+
+> **USB调试 (安全设置)**  
+> _允许通过USB调试修改权限或模拟点击_
+
+[simulating input]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
+
+
+### 特殊字符不起作用
+
+可输入的文本[被限制为ASCII字符][text-input]。也可以用一些小技巧输入一些[带重音符号的字符][accented-characters]，但是仅此而已。参见[#37]。
+
+
+[text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
+[accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
+[#37]: https://github.com/Genymobile/scrcpy/issues/37
+
+
+## 客户端相关问题
+
+### 效果很差
+
+如果你的客户端窗口分辨率比你的设备屏幕小，则可能出现效果差的问题，尤其是在文本上（参见 [#40]）。
+
+[#40]: https://github.com/Genymobile/scrcpy/issues/40
+
+
+为了提升降尺度的质量，如果渲染器是OpenGL并且支持mip映射，就会自动开启三线性过滤。
+
+在Windows上，你可能希望强制使用OpenGL：
+
+```
+scrcpy --render-driver=opengl
+```
+
+你可能还需要配置[缩放行为][scaling behavior]：
+
+> `scrcpy.exe` > Properties > Compatibility > Change high DPI settings >
+> Override high DPI scaling behavior > Scaling performed by: _Application_.
+
+[scaling behavior]: https://github.com/Genymobile/scrcpy/issues/40#issuecomment-424466723
+
+
+### Wayland相关的问题
+
+在Linux上，SDL默认使用x11。可以通过`SDL_VIDEODRIVER`环境变量来更改[视频驱动][video driver]：
+
+[video driver]: https://wiki.libsdl.org/FAQUsingSDL#how_do_i_choose_a_specific_video_driver
+
+```bash
+export SDL_VIDEODRIVER=wayland
+scrcpy
+```
+
+在一些发行版上 (至少包括 Fedora)， `libdecor` 包必须手动安装。
+
+参见 [#2554] 和 [#2559]。
+
+[#2554]: https://github.com/Genymobile/scrcpy/issues/2554
+[#2559]: https://github.com/Genymobile/scrcpy/issues/2559
+
+
+### KWin compositor 崩溃
+
+在Plasma桌面中，当 _scrcpy_ 运行时，会禁用compositor。
+
+一种解决方法是， [禁用 "Block compositing"][kwin].
+
+[kwin]: https://github.com/Genymobile/scrcpy/issues/114#issuecomment-378778613
+
+
+## 崩溃
+
+### 异常
+可能有很多原因。一个常见的原因是您的设备无法按给定清晰度进行编码：
+
+> ```
+> ERROR: Exception on thread Thread[main,5,main]
+> android.media.MediaCodec$CodecException: Error 0xfffffc0e
+> ...
+> Exit due to uncaughtException in main thread:
+> ERROR: Could not open video stream
+> INFO: Initial texture: 1080x2336
+> ```
+
+或者
+
+> ```
+> ERROR: Exception on thread Thread[main,5,main]
+> java.lang.IllegalStateException
+>         at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
+> ```
+
+请尝试使用更低的清晰度：
+
+```
+scrcpy -m 1920
+scrcpy -m 1024
+scrcpy -m 800
+```
+
+你也可以尝试另一种 [编码器](README.md#encoder)。
+
+
+## Windows命令行
+
+一些Windows用户不熟悉命令行。以下是如何打开终端并带参数执行`scrcpy`：
+
+ 1. 按下 <kbd>Windows</kbd>+<kbd>r</kbd>，打开一个对话框。
+ 2. 输入 `cmd` 并按 <kbd>Enter</kbd>，这样就打开了一个终端。
+ 3. 通过输入以下命令，切换到你的 _scrcpy_ 所在的目录 （根据你的实际位置修改路径）:
+
+    ```bat
+    cd C:\Users\user\Downloads\scrcpy-win64-xxx
+    ```
+
+    然后按 <kbd>Enter</kbd>
+ 4. 输入你的命令。比如:
+
+    ```bat
+    scrcpy --record file.mkv
+    ```
+
+如果你打算总是使用相同的参数，在`scrcpy`目录创建一个文件 `myscrcpy.bat`
+（启用 [显示文件拓展名][show file extensions] 避免混淆），文件中包含你的命令。例如：
+
+```bat
+scrcpy --prefer-text --turn-screen-off --stay-awake
+```
+
+然后双击刚刚创建的文件。
+
+你也可以编辑 `scrcpy-console.bat` 或者 `scrcpy-noconsole.vbs`（的副本）来添加参数。
+
+[show file extensions]: https://www.howtogeek.com/205086/beginner-how-to-make-windows-show-file-extensions/

README.md

@@ -1,4 +1,4 @@
-# scrcpy (v1.18)
+# scrcpy (v1.19)
 
 [Read in another language](#translations)
 
@@ -88,10 +88,10 @@ process][BUILD_simple]).
 For Windows, for simplicity, a prebuilt archive with all the dependencies
 (including `adb`) is available:
 
- - [`scrcpy-win64-v1.18.zip`][direct-win64]  
-   _(SHA-256: 37212f5087fe6f3e258f1d44fa5c02207496b30e1d7ec442cbcf8358910a5c63)_
+ - [`scrcpy-win64-v1.19.zip`][direct-win64]  
+   _(SHA-256: 383d6483f25ac0092d4bb9fef6c967351ecd50fc248e0c82932db97d6d32f11b)_
 
-[direct-win64]: https://github.com/Genymobile/scrcpy/releases/download/v1.18/scrcpy-win64-v1.18.zip
+[direct-win64]: https://github.com/Genymobile/scrcpy/releases/download/v1.19/scrcpy-win64-v1.19.zip
 
 It is also available in [Chocolatey]:
 
@@ -321,6 +321,24 @@ For example, you could capture the video within [OBS].
 [OBS]: https://obsproject.com/
 
 
+#### Buffering
+
+It is possible to add buffering. This increases latency but reduces jitter (see
+#2464).
+
+The option is available for display buffering:
+
+```bash
+scrcpy --display-buffer=50  # add 50 ms buffering for display
+```
+
+and V4L2 sink:
+
+```bash
+scrcpy --v4l2-buffer=500    # add 500 ms buffering for v4l2 sink
+```
+
+
 ### Connection
 
 #### Wireless

app/meson.build

@@ -2,15 +2,18 @@ src = [
     'src/main.c',
     'src/adb.c',
     'src/cli.c',
+    'src/clock.c',
     'src/compat.c',
     'src/control_msg.c',
     'src/controller.c',
     'src/decoder.c',
     'src/device_msg.c',
-    'src/event_converter.c',
     'src/file_handler.c',
     'src/fps_counter.c',
+    'src/frame_buffer.c',
     'src/input_manager.c',
+    'src/keyboard_inject.c',
+    'src/mouse_inject.c',
     'src/opengl.c',
     'src/receiver.c',
     'src/recorder.c',
@@ -25,6 +28,7 @@ src = [
     'src/util/process.c',
     'src/util/str_util.c',
     'src/util/thread.c',
+    'src/util/tick.c',
 ]
 
 if host_machine.system() == 'windows'
@@ -165,6 +169,10 @@ if get_option('buildtype') == 'debug'
             'src/cli.c',
             'src/util/str_util.c',
         ]],
+        ['test_clock', [
+            'tests/test_clock.c',
+            'src/clock.c',
+        ]],
         ['test_control_msg_serialize', [
             'tests/test_control_msg_serialize.c',
             'src/control_msg.c',

app/scrcpy.1

@@ -56,6 +56,12 @@ The list of possible display ids can be listed by "adb shell dumpsys display"
 
 Default is 0.
 
+.TP
+.BI "\-\-display\-buffer ms
+Add a buffering delay (in milliseconds) before displaying. This increases latency to compensate for jitter.
+
+Default is 0 (no buffering).
+
 .TP
 .BI "\-\-encoder " name
 Use a specific MediaCodec encoder (must be a H.264 encoder).
@@ -83,8 +89,8 @@ Inject computer clipboard text as a sequence of key events on Ctrl+v (like MOD+S
 This is a workaround for some devices not behaving as expected when setting the device clipboard programmatically.
 
 .TP
-.BI "\-\-lock\-video\-orientation " [value]
-Lock video orientation to \fIvalue\fR. Possible values are "unlocked", "initial" (locked to the initial orientation), 0, 1, 2 and 3. Natural device orientation is 0, and each increment adds a 90 degrees otation counterclockwise.
+.BI "\-\-lock\-video\-orientation[=value]
+Lock video orientation to \fIvalue\fR. Possible values are "unlocked", "initial" (locked to the initial orientation), 0, 1, 2 and 3. Natural device orientation is 0, and each increment adds a 90 degrees rotation counterclockwise.
 
 Default is "unlocked".
 
@@ -189,7 +195,15 @@ It only shows physical touches (not clicks from scrcpy).
 .BI "\-\-v4l2-sink " /dev/videoN
 Output to v4l2loopback device.
 
-It requires to lock the video orientation (see --lock-video-orientation).
+It requires to lock the video orientation (see \fB\-\-lock\-video\-orientation\fR).
+
+.TP
+.BI "\-\-v4l2-buffer " ms
+Add a buffering delay (in milliseconds) before pushing frames. This increases latency to compensate for jitter.
+
+This option is similar to \fB\-\-display\-buffer\fR, but specific to V4L2 sink.
+
+Default is 0 (no buffering).
 
 .TP
 .BI "\-V, \-\-verbosity " value
@@ -240,7 +254,7 @@ Default is 0 (automatic).
 .SH SHORTCUTS
 
 In the following list, MOD is the shortcut modifier. By default, it's (left)
-Alt or (left) Super, but it can be configured by \-\-shortcut-mod (see above).
+Alt or (left) Super, but it can be configured by \fB\-\-shortcut\-mod\fR (see above).
 
 .TP
 .B MOD+f

app/src/cli.c

@@ -55,6 +55,12 @@ scrcpy_print_usage(const char *arg0) {
         "\n"
         "        Default is 0.\n"
         "\n"
+        "    --display-buffer ms\n"
+        "        Add a buffering delay (in milliseconds) before displaying.\n"
+        "        This increases latency to compensate for jitter.\n"
+        "\n"
+        "        Default is 0 (no buffering).\n"
+        "\n"
         "    --encoder name\n"
         "        Use a specific MediaCodec encoder (must be a H.264 encoder).\n"
         "\n"
@@ -79,7 +85,7 @@ scrcpy_print_usage(const char *arg0) {
         "        This is a workaround for some devices not behaving as\n"
         "        expected when setting the device clipboard programmatically.\n"
         "\n"
-        "    --lock-video-orientation [value]\n"
+        "    --lock-video-orientation[=value]\n"
         "        Lock video orientation to value.\n"
         "        Possible values are \"unlocked\", \"initial\" (locked to the\n"
         "        initial orientation), 0, 1, 2 and 3.\n"
@@ -182,6 +188,15 @@ scrcpy_print_usage(const char *arg0) {
         "        It requires to lock the video orientation (see\n"
         "        --lock-video-orientation).\n"
         "\n"
+        "    --v4l2-buffer ms\n"
+        "        Add a buffering delay (in milliseconds) before pushing\n"
+        "        frames. This increases latency to compensate for jitter.\n"
+        "\n"
+        "        This option is similar to --display-buffer, but specific to\n"
+        "        V4L2 sink.\n"
+        "\n"
+        "        Default is 0 (no buffering).\n"
+        "\n"
 #endif
         "    -V, --verbosity value\n"
         "        Set the log level (verbose, debug, info, warn or error).\n"
@@ -392,6 +407,19 @@ parse_max_fps(const char *s, uint16_t *max_fps) {
     return true;
 }
 
+static bool
+parse_buffering_time(const char *s, sc_tick *tick) {
+    long value;
+    bool ok = parse_integer_arg(s, &value, false, 0, 0x7FFFFFFF,
+                                "buffering time");
+    if (!ok) {
+        return false;
+    }
+
+    *tick = SC_TICK_FROM_MS(value);
+    return true;
+}
+
 static bool
 parse_lock_video_orientation(const char *s,
                              enum sc_lock_video_orientation *lock_mode) {
@@ -689,6 +717,8 @@ guess_record_format(const char *filename) {
 #define OPT_ENCODER_NAME           1025
 #define OPT_POWER_OFF_ON_CLOSE     1026
 #define OPT_V4L2_SINK              1027
+#define OPT_DISPLAY_BUFFER         1028
+#define OPT_V4L2_BUFFER            1029
 
 bool
 scrcpy_parse_args(struct scrcpy_cli_args *args, int argc, char *argv[]) {
@@ -700,6 +730,7 @@ scrcpy_parse_args(struct scrcpy_cli_args *args, int argc, char *argv[]) {
         {"disable-screensaver",    no_argument,       NULL,
                                                   OPT_DISABLE_SCREENSAVER},
         {"display",                required_argument, NULL, OPT_DISPLAY_ID},
+        {"display-buffer",         required_argument, NULL, OPT_DISPLAY_BUFFER},
         {"encoder",                required_argument, NULL, OPT_ENCODER_NAME},
         {"force-adb-forward",      no_argument,       NULL,
                                                   OPT_FORCE_ADB_FORWARD},
@@ -732,6 +763,7 @@ scrcpy_parse_args(struct scrcpy_cli_args *args, int argc, char *argv[]) {
         {"turn-screen-off",        no_argument,       NULL, 'S'},
 #ifdef HAVE_V4L2
         {"v4l2-sink",              required_argument, NULL, OPT_V4L2_SINK},
+        {"v4l2-buffer",            required_argument, NULL, OPT_V4L2_BUFFER},
 #endif
         {"verbosity",              required_argument, NULL, 'V'},
         {"version",                no_argument,       NULL, 'v'},
@@ -917,10 +949,20 @@ scrcpy_parse_args(struct scrcpy_cli_args *args, int argc, char *argv[]) {
             case OPT_POWER_OFF_ON_CLOSE:
                 opts->power_off_on_close = true;
                 break;
+            case OPT_DISPLAY_BUFFER:
+                if (!parse_buffering_time(optarg, &opts->display_buffer)) {
+                    return false;
+                }
+                break;
 #ifdef HAVE_V4L2
             case OPT_V4L2_SINK:
                 opts->v4l2_device = optarg;
                 break;
+            case OPT_V4L2_BUFFER:
+                if (!parse_buffering_time(optarg, &opts->v4l2_buffer)) {
+                    return false;
+                }
+                break;
 #endif
             default:
                 // getopt prints the error message on stderr
@@ -941,6 +983,11 @@ scrcpy_parse_args(struct scrcpy_cli_args *args, int argc, char *argv[]) {
              "See --lock-video-orientation.");
         opts->lock_video_orientation = SC_LOCK_VIDEO_ORIENTATION_INITIAL;
     }
+
+    if (opts->v4l2_buffer && !opts->v4l2_device) {
+        LOGE("V4L2 buffer value without V4L2 sink\n");
+        return false;
+    }
 #else
     if (!opts->display && !opts->record_filename) {
         LOGE("-N/--no-display requires screen recording (-r/--record)");

app/src/clock.c

@@ -0,0 +1,111 @@
+#include "clock.h"
+
+#include "util/log.h"
+
+#define SC_CLOCK_NDEBUG // comment to debug
+
+void
+sc_clock_init(struct sc_clock *clock) {
+    clock->count = 0;
+    clock->head = 0;
+    clock->left_sum.system = 0;
+    clock->left_sum.stream = 0;
+    clock->right_sum.system = 0;
+    clock->right_sum.stream = 0;
+}
+
+// Estimate the affine function f(stream) = slope * stream + offset
+static void
+sc_clock_estimate(struct sc_clock *clock,
+                  double *out_slope, sc_tick *out_offset) {
+    assert(clock->count > 1); // two points are necessary
+
+    struct sc_clock_point left_avg = {
+        .system = clock->left_sum.system / (clock->count / 2),
+        .stream = clock->left_sum.stream / (clock->count / 2),
+    };
+    struct sc_clock_point right_avg = {
+        .system = clock->right_sum.system / ((clock->count + 1) / 2),
+        .stream = clock->right_sum.stream / ((clock->count + 1) / 2),
+    };
+
+    double slope = (double) (right_avg.system - left_avg.system)
+                 / (right_avg.stream - left_avg.stream);
+
+    if (clock->count < SC_CLOCK_RANGE) {
+        /* The first frames are typically received and decoded with more delay
+         * than the others, causing a wrong slope estimation on start. To
+         * compensate, assume an initial slope of 1, then progressively use the
+         * estimated slope. */
+        slope = (clock->count * slope + (SC_CLOCK_RANGE - clock->count))
+              / SC_CLOCK_RANGE;
+    }
+
+    struct sc_clock_point global_avg = {
+        .system = (clock->left_sum.system + clock->right_sum.system)
+                 / clock->count,
+        .stream = (clock->left_sum.stream + clock->right_sum.stream)
+                 / clock->count,
+    };
+
+    sc_tick offset = global_avg.system - (sc_tick) (global_avg.stream * slope);
+
+    *out_slope = slope;
+    *out_offset = offset;
+}
+
+void
+sc_clock_update(struct sc_clock *clock, sc_tick system, sc_tick stream) {
+    struct sc_clock_point *point = &clock->points[clock->head];
+
+    if (clock->count == SC_CLOCK_RANGE || clock->count & 1) {
+        // One point passes from the right sum to the left sum
+
+        unsigned mid;
+        if (clock->count == SC_CLOCK_RANGE) {
+            mid = (clock->head + SC_CLOCK_RANGE / 2) % SC_CLOCK_RANGE;
+        } else {
+            // Only for the first frames
+            mid = clock->count / 2;
+        }
+
+        struct sc_clock_point *mid_point = &clock->points[mid];
+        clock->left_sum.system += mid_point->system;
+        clock->left_sum.stream += mid_point->stream;
+        clock->right_sum.system -= mid_point->system;
+        clock->right_sum.stream -= mid_point->stream;
+    }
+
+    if (clock->count == SC_CLOCK_RANGE) {
+        // The current point overwrites the previous value in the circular
+        // array, update the left sum accordingly
+        clock->left_sum.system -= point->system;
+        clock->left_sum.stream -= point->stream;
+    } else {
+        ++clock->count;
+    }
+
+    point->system = system;
+    point->stream = stream;
+
+    clock->right_sum.system += system;
+    clock->right_sum.stream += stream;
+
+    clock->head = (clock->head + 1) % SC_CLOCK_RANGE;
+
+    if (clock->count > 1) {
+        // Update estimation
+        sc_clock_estimate(clock, &clock->slope, &clock->offset);
+
+#ifndef SC_CLOCK_NDEBUG
+        LOGD("Clock estimation: %g * pts + %" PRItick,
+             clock->slope, clock->offset);
+#endif
+    }
+}
+
+sc_tick
+sc_clock_to_system_time(struct sc_clock *clock, sc_tick stream) {
+    assert(clock->count > 1); // sc_clock_update() must have been called
+    return (sc_tick) (stream * clock->slope) + clock->offset;
+}

app/src/clock.h

@@ -0,0 +1,70 @@
+#ifndef SC_CLOCK_H
+#define SC_CLOCK_H
+
+#include "common.h"
+
+#include <assert.h>
+
+#include "util/tick.h"
+
+#define SC_CLOCK_RANGE 32
+static_assert(!(SC_CLOCK_RANGE & 1), "SC_CLOCK_RANGE must be even");
+
+struct sc_clock_point {
+    sc_tick system;
+    sc_tick stream;
+};
+
+/**
+ * The clock aims to estimate the affine relation between the stream (device)
+ * time and the system time:
+ *
+ *     f(stream) = slope * stream + offset
+ *
+ * To that end, it stores the SC_CLOCK_RANGE last clock points (the timestamps
+ * of a frame expressed both in stream time and system time) in a circular
+ * array.
+ *
+ * To estimate the slope, it splits the last SC_CLOCK_RANGE points into two
+ * sets of SC_CLOCK_RANGE/2 points, and compute their centroid ("average
+ * point"). The slope of the estimated affine function is that of the line
+ * passing through these two points.
+ *
+ * To estimate the offset, it computes the centroid of all the SC_CLOCK_RANGE
+ * points. The resulting affine function passes by this centroid.
+ *
+ * With a circular array, the rolling sums (and average) are quick to compute.
+ * In practice, the estimation is stable and the evolution is smooth.
+ */
+struct sc_clock {
+    // Circular array
+    struct sc_clock_point points[SC_CLOCK_RANGE];
+
+    // Number of points in the array (count <= SC_CLOCK_RANGE)
+    unsigned count;
+
+    // Index of the next point to write
+    unsigned head;
+
+    // Sum of the first count/2 points
+    struct sc_clock_point left_sum;
+
+    // Sum of the last (count+1)/2 points
+    struct sc_clock_point right_sum;
+
+    // Estimated slope and offset
+    // (computed on sc_clock_update(), used by sc_clock_to_system_time())
+    double slope;
+    sc_tick offset;
+};
+
+void
+sc_clock_init(struct sc_clock *clock);
+
+void
+sc_clock_update(struct sc_clock *clock, sc_tick system, sc_tick stream);
+
+sc_tick
+sc_clock_to_system_time(struct sc_clock *clock, sc_tick stream);
+
+#endif

app/src/controller.c

@@ -70,7 +70,7 @@ process_msg(struct controller *controller,
     if (!length) {
         return false;
     }
-    int w = net_send_all(controller->control_socket, serialized_msg, length);
+    ssize_t w = net_send_all(controller->control_socket, serialized_msg, length);
     return (size_t) w == length;
 }
 

app/src/decoder.c

@@ -1,5 +1,6 @@
 #include "decoder.h"
 
+#include <libavcodec/avcodec.h>
 #include <libavformat/avformat.h>
 
 #include "events.h"

app/src/event_converter.h

@@ -1,30 +0,0 @@
-#ifndef CONVERT_H
-#define CONVERT_H
-
-#include "common.h"
-
-#include <stdbool.h>
-#include <SDL2/SDL_events.h>
-
-#include "control_msg.h"
-
-bool
-convert_keycode_action(SDL_EventType from, enum android_keyevent_action *to);
-
-enum android_metastate
-convert_meta_state(SDL_Keymod mod);
-
-bool
-convert_keycode(SDL_Keycode from, enum android_keycode *to, uint16_t mod,
-                bool prefer_text);
-
-enum android_motionevent_buttons
-convert_mouse_buttons(uint32_t state);
-
-bool
-convert_mouse_action(SDL_EventType from, enum android_motionevent_action *to);
-
-bool
-convert_touch_action(SDL_EventType from, enum android_motionevent_action *to);
-
-#endif

app/src/fps_counter.c

@@ -1,11 +1,10 @@
 #include "fps_counter.h"
 
 #include <assert.h>
-#include <SDL2/SDL_timer.h>
 
 #include "util/log.h"
 
-#define FPS_COUNTER_INTERVAL_MS 1000
+#define FPS_COUNTER_INTERVAL SC_TICK_FROM_SEC(1)
 
 bool
 fps_counter_init(struct fps_counter *counter) {
@@ -47,7 +46,7 @@ set_started(struct fps_counter *counter, bool started) {
 static void
 display_fps(struct fps_counter *counter) {
     unsigned rendered_per_second =
-        counter->nr_rendered * 1000 / FPS_COUNTER_INTERVAL_MS;
+        counter->nr_rendered * SC_TICK_FREQ / FPS_COUNTER_INTERVAL;
     if (counter->nr_skipped) {
         LOGI("%u fps (+%u frames skipped)", rendered_per_second,
                                             counter->nr_skipped);
@@ -68,8 +67,8 @@ check_interval_expired(struct fps_counter *counter, uint32_t now) {
     counter->nr_skipped = 0;
     // add a multiple of the interval
     uint32_t elapsed_slices =
-        (now - counter->next_timestamp) / FPS_COUNTER_INTERVAL_MS + 1;
-    counter->next_timestamp += FPS_COUNTER_INTERVAL_MS * elapsed_slices;
+        (now - counter->next_timestamp) / FPS_COUNTER_INTERVAL + 1;
+    counter->next_timestamp += FPS_COUNTER_INTERVAL * elapsed_slices;
 }
 
 static int
@@ -82,14 +81,12 @@ run_fps_counter(void *data) {
             sc_cond_wait(&counter->state_cond, &counter->mutex);
         }
         while (!counter->interrupted && is_started(counter)) {
-            uint32_t now = SDL_GetTicks();
+            sc_tick now = sc_tick_now();
             check_interval_expired(counter, now);
 
-            assert(counter->next_timestamp > now);
-            uint32_t remaining = counter->next_timestamp - now;
-
             // ignore the reason (timeout or signaled), we just loop anyway
-            sc_cond_timedwait(&counter->state_cond, &counter->mutex, remaining);
+            sc_cond_timedwait(&counter->state_cond, &counter->mutex,
+                              counter->next_timestamp);
         }
     }
     sc_mutex_unlock(&counter->mutex);
@@ -99,7 +96,7 @@ run_fps_counter(void *data) {
 bool
 fps_counter_start(struct fps_counter *counter) {
     sc_mutex_lock(&counter->mutex);
-    counter->next_timestamp = SDL_GetTicks() + FPS_COUNTER_INTERVAL_MS;
+    counter->next_timestamp = sc_tick_now() + FPS_COUNTER_INTERVAL;
     counter->nr_rendered = 0;
     counter->nr_skipped = 0;
     sc_mutex_unlock(&counter->mutex);
@@ -165,7 +162,7 @@ fps_counter_add_rendered_frame(struct fps_counter *counter) {
     }
 
     sc_mutex_lock(&counter->mutex);
-    uint32_t now = SDL_GetTicks();
+    sc_tick now = sc_tick_now();
     check_interval_expired(counter, now);
     ++counter->nr_rendered;
     sc_mutex_unlock(&counter->mutex);
@@ -178,7 +175,7 @@ fps_counter_add_skipped_frame(struct fps_counter *counter) {
     }
 
     sc_mutex_lock(&counter->mutex);
-    uint32_t now = SDL_GetTicks();
+    sc_tick now = sc_tick_now();
     check_interval_expired(counter, now);
     ++counter->nr_skipped;
     sc_mutex_unlock(&counter->mutex);

app/src/fps_counter.h

@@ -24,7 +24,7 @@ struct fps_counter {
     bool interrupted;
     unsigned nr_rendered;
     unsigned nr_skipped;
-    uint32_t next_timestamp;
+    sc_tick next_timestamp;
 };
 
 bool

app/src/frame_buffer.c

@@ -0,0 +1,88 @@
+#include "frame_buffer.h"
+
+#include <assert.h>
+#include <libavutil/avutil.h>
+#include <libavformat/avformat.h>
+
+#include "util/log.h"
+
+bool
+sc_frame_buffer_init(struct sc_frame_buffer *fb) {
+    fb->pending_frame = av_frame_alloc();
+    if (!fb->pending_frame) {
+        return false;
+    }
+
+    fb->tmp_frame = av_frame_alloc();
+    if (!fb->tmp_frame) {
+        av_frame_free(&fb->pending_frame);
+        return false;
+    }
+
+    bool ok = sc_mutex_init(&fb->mutex);
+    if (!ok) {
+        av_frame_free(&fb->pending_frame);
+        av_frame_free(&fb->tmp_frame);
+        return false;
+    }
+
+    // there is initially no frame, so consider it has already been consumed
+    fb->pending_frame_consumed = true;
+
+    return true;
+}
+
+void
+sc_frame_buffer_destroy(struct sc_frame_buffer *fb) {
+    sc_mutex_destroy(&fb->mutex);
+    av_frame_free(&fb->pending_frame);
+    av_frame_free(&fb->tmp_frame);
+}
+
+static inline void
+swap_frames(AVFrame **lhs, AVFrame **rhs) {
+    AVFrame *tmp = *lhs;
+    *lhs = *rhs;
+    *rhs = tmp;
+}
+
+bool
+sc_frame_buffer_push(struct sc_frame_buffer *fb, const AVFrame *frame,
+                  bool *previous_frame_skipped) {
+    sc_mutex_lock(&fb->mutex);
+
+    // Use a temporary frame to preserve pending_frame in case of error.
+    // tmp_frame is an empty frame, no need to call av_frame_unref() beforehand.
+    int r = av_frame_ref(fb->tmp_frame, frame);
+    if (r) {
+        LOGE("Could not ref frame: %d", r);
+        return false;
+    }
+
+    // Now that av_frame_ref() succeeded, we can replace the previous
+    // pending_frame
+    swap_frames(&fb->pending_frame, &fb->tmp_frame);
+    av_frame_unref(fb->tmp_frame);
+
+    if (previous_frame_skipped) {
+        *previous_frame_skipped = !fb->pending_frame_consumed;
+    }
+    fb->pending_frame_consumed = false;
+
+    sc_mutex_unlock(&fb->mutex);
+
+    return true;
+}
+
+void
+sc_frame_buffer_consume(struct sc_frame_buffer *fb, AVFrame *dst) {
+    sc_mutex_lock(&fb->mutex);
+    assert(!fb->pending_frame_consumed);
+    fb->pending_frame_consumed = true;
+
+    av_frame_move_ref(dst, fb->pending_frame);
+    // av_frame_move_ref() resets its source frame, so no need to call
+    // av_frame_unref()
+
+    sc_mutex_unlock(&fb->mutex);
+}

app/src/frame_buffer.h

@@ -0,0 +1,44 @@
+#ifndef SC_FRAME_BUFFER_H
+#define SC_FRAME_BUFFER_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "util/thread.h"
+
+// forward declarations
+typedef struct AVFrame AVFrame;
+
+/**
+ * A frame buffer holds 1 pending frame, which is the last frame received from
+ * the producer (typically, the decoder).
+ *
+ * If a pending frame has not been consumed when the producer pushes a new
+ * frame, then it is lost. The intent is to always provide access to the very
+ * last frame to minimize latency.
+ */
+
+struct sc_frame_buffer {
+    AVFrame *pending_frame;
+    AVFrame *tmp_frame; // To preserve the pending frame on error
+
+    sc_mutex mutex;
+
+    bool pending_frame_consumed;
+};
+
+bool
+sc_frame_buffer_init(struct sc_frame_buffer *fb);
+
+void
+sc_frame_buffer_destroy(struct sc_frame_buffer *fb);
+
+bool
+sc_frame_buffer_push(struct sc_frame_buffer *fb, const AVFrame *frame,
+                     bool *skipped);
+
+void
+sc_frame_buffer_consume(struct sc_frame_buffer *fb, AVFrame *dst);
+
+#endif

app/src/input_manager.c

@@ -3,7 +3,6 @@
 #include <assert.h>
 #include <SDL2/SDL_keycode.h>
 
-#include "event_converter.h"
 #include "util/log.h"
 
 static const int ACTION_DOWN = 1;
@@ -53,15 +52,18 @@ is_shortcut_mod(struct input_manager *im, uint16_t sdl_mod) {
 
 void
 input_manager_init(struct input_manager *im, struct controller *controller,
-                   struct screen *screen,
+                   struct screen *screen, struct sc_key_processor *kp,
+                   struct sc_mouse_processor *mp,
                    const struct scrcpy_options *options) {
+    assert(!options->control || (kp && kp->ops));
+    assert(!options->control || (mp && mp->ops));
+
     im->controller = controller;
     im->screen = screen;
-    im->repeat = 0;
+    im->kp = kp;
+    im->mp = mp;
 
     im->control = options->control;
-    im->forward_key_repeat = options->forward_key_repeat;
-    im->prefer_text = options->prefer_text;
     im->forward_all_clicks = options->forward_all_clicks;
     im->legacy_paste = options->legacy_paste;
 
@@ -323,26 +325,8 @@ input_manager_process_text_input(struct input_manager *im,
         // A shortcut must never generate text events
         return;
     }
-    if (!im->prefer_text) {
-        char c = event->text[0];
-        if (isalpha(c) || c == ' ') {
-            assert(event->text[1] == '\0');
-            // letters and space are handled as raw key event
-            return;
-        }
-    }
 
-    struct control_msg msg;
-    msg.type = CONTROL_MSG_TYPE_INJECT_TEXT;
-    msg.inject_text.text = strdup(event->text);
-    if (!msg.inject_text.text) {
-        LOGW("Could not strdup input text");
-        return;
-    }
-    if (!controller_push_msg(im->controller, &msg)) {
-        free(msg.inject_text.text);
-        LOGW("Could not request 'inject text'");
-    }
+    im->kp->ops->process_text(im->kp, event);
 }
 
 static bool
@@ -375,27 +359,6 @@ inverse_point(struct point point, struct size size) {
     return point;
 }
 
-static bool
-convert_input_key(const SDL_KeyboardEvent *from, struct control_msg *to,
-                  bool prefer_text, uint32_t repeat) {
-    to->type = CONTROL_MSG_TYPE_INJECT_KEYCODE;
-
-    if (!convert_keycode_action(from->type, &to->inject_keycode.action)) {
-        return false;
-    }
-
-    uint16_t mod = from->keysym.mod;
-    if (!convert_keycode(from->keysym.sym, &to->inject_keycode.keycode, mod,
-                         prefer_text)) {
-        return false;
-    }
-
-    to->inject_keycode.repeat = repeat;
-    to->inject_keycode.metastate = convert_meta_state(mod);
-
-    return true;
-}
-
 static void
 input_manager_process_key(struct input_manager *im,
                           const SDL_KeyboardEvent *event) {
@@ -549,15 +512,6 @@ input_manager_process_key(struct input_manager *im,
         return;
     }
 
-    if (event->repeat) {
-        if (!im->forward_key_repeat) {
-            return;
-        }
-        ++im->repeat;
-    } else {
-        im->repeat = 0;
-    }
-
     if (ctrl && !shift && keycode == SDLK_v && down && !repeat) {
         if (im->legacy_paste) {
             // inject the text as input events
@@ -569,27 +523,7 @@ input_manager_process_key(struct input_manager *im,
         set_device_clipboard(controller, false);
     }
 
-    struct control_msg msg;
-    if (convert_input_key(event, &msg, im->prefer_text, im->repeat)) {
-        if (!controller_push_msg(controller, &msg)) {
-            LOGW("Could not request 'inject keycode'");
-        }
-    }
-}
-
-static bool
-convert_mouse_motion(const SDL_MouseMotionEvent *from, struct screen *screen,
-                     struct control_msg *to) {
-    to->type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT;
-    to->inject_touch_event.action = AMOTION_EVENT_ACTION_MOVE;
-    to->inject_touch_event.pointer_id = POINTER_ID_MOUSE;
-    to->inject_touch_event.position.screen_size = screen->frame_size;
-    to->inject_touch_event.position.point =
-        screen_convert_window_to_frame_coords(screen, from->x, from->y);
-    to->inject_touch_event.pressure = 1.f;
-    to->inject_touch_event.buttons = convert_mouse_buttons(from->state);
-
-    return true;
+    im->kp->ops->process_key(im->kp, event);
 }
 
 static void
@@ -607,79 +541,22 @@ input_manager_process_mouse_motion(struct input_manager *im,
         // simulated from touch events, so it's a duplicate
         return;
     }
-    struct control_msg msg;
-    if (!convert_mouse_motion(event, im->screen, &msg)) {
-        return;
-    }
 
-    if (!controller_push_msg(im->controller, &msg)) {
-        LOGW("Could not request 'inject mouse motion event'");
-    }
+    im->mp->ops->process_mouse_motion(im->mp, event);
 
     if (im->vfinger_down) {
-        struct point mouse = msg.inject_touch_event.position.point;
+        struct point mouse =
+            screen_convert_window_to_frame_coords(im->screen, event->x,
+                                                  event->y);
         struct point vfinger = inverse_point(mouse, im->screen->frame_size);
         simulate_virtual_finger(im, AMOTION_EVENT_ACTION_MOVE, vfinger);
     }
 }
 
-static bool
-convert_touch(const SDL_TouchFingerEvent *from, struct screen *screen,
-              struct control_msg *to) {
-    to->type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT;
-
-    if (!convert_touch_action(from->type, &to->inject_touch_event.action)) {
-        return false;
-    }
-
-    to->inject_touch_event.pointer_id = from->fingerId;
-    to->inject_touch_event.position.screen_size = screen->frame_size;
-
-    int dw;
-    int dh;
-    SDL_GL_GetDrawableSize(screen->window, &dw, &dh);
-
-    // SDL touch event coordinates are normalized in the range [0; 1]
-    int32_t x = from->x * dw;
-    int32_t y = from->y * dh;
-    to->inject_touch_event.position.point =
-        screen_convert_drawable_to_frame_coords(screen, x, y);
-
-    to->inject_touch_event.pressure = from->pressure;
-    to->inject_touch_event.buttons = 0;
-    return true;
-}
-
 static void
 input_manager_process_touch(struct input_manager *im,
                             const SDL_TouchFingerEvent *event) {
-    struct control_msg msg;
-    if (convert_touch(event, im->screen, &msg)) {
-        if (!controller_push_msg(im->controller, &msg)) {
-            LOGW("Could not request 'inject touch event'");
-        }
-    }
-}
-
-static bool
-convert_mouse_button(const SDL_MouseButtonEvent *from, struct screen *screen,
-                     struct control_msg *to) {
-    to->type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT;
-
-    if (!convert_mouse_action(from->type, &to->inject_touch_event.action)) {
-        return false;
-    }
-
-    to->inject_touch_event.pointer_id = POINTER_ID_MOUSE;
-    to->inject_touch_event.position.screen_size = screen->frame_size;
-    to->inject_touch_event.position.point =
-        screen_convert_window_to_frame_coords(screen, from->x, from->y);
-    to->inject_touch_event.pressure =
-        from->type == SDL_MOUSEBUTTONDOWN ? 1.f : 0.f;
-    to->inject_touch_event.buttons =
-        convert_mouse_buttons(SDL_BUTTON(from->button));
-
-    return true;
+    im->mp->ops->process_touch(im->mp, event);
 }
 
 static void
@@ -739,15 +616,7 @@ input_manager_process_mouse_button(struct input_manager *im,
         return;
     }
 
-    struct control_msg msg;
-    if (!convert_mouse_button(event, im->screen, &msg)) {
-        return;
-    }
-
-    if (!controller_push_msg(im->controller, &msg)) {
-        LOGW("Could not request 'inject mouse button event'");
-        return;
-    }
+    im->mp->ops->process_mouse_button(im->mp, event);
 
     // Pinch-to-zoom simulation.
     //
@@ -761,7 +630,9 @@ input_manager_process_mouse_button(struct input_manager *im,
 #define CTRL_PRESSED (SDL_GetModState() & (KMOD_LCTRL | KMOD_RCTRL))
     if ((down && !im->vfinger_down && CTRL_PRESSED)
             || (!down && im->vfinger_down)) {
-        struct point mouse = msg.inject_touch_event.position.point;
+        struct point mouse =
+            screen_convert_window_to_frame_coords(im->screen, event->x,
+                                                              event->y);
         struct point vfinger = inverse_point(mouse, im->screen->frame_size);
         enum android_motionevent_action action = down
                                                ? AMOTION_EVENT_ACTION_DOWN
@@ -773,39 +644,10 @@ input_manager_process_mouse_button(struct input_manager *im,
     }
 }
 
-static bool
-convert_mouse_wheel(const SDL_MouseWheelEvent *from, struct screen *screen,
-                    struct control_msg *to) {
-
-    // mouse_x and mouse_y are expressed in pixels relative to the window
-    int mouse_x;
-    int mouse_y;
-    SDL_GetMouseState(&mouse_x, &mouse_y);
-
-    struct position position = {
-        .screen_size = screen->frame_size,
-        .point = screen_convert_window_to_frame_coords(screen,
-                                                       mouse_x, mouse_y),
-    };
-
-    to->type = CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT;
-
-    to->inject_scroll_event.position = position;
-    to->inject_scroll_event.hscroll = from->x;
-    to->inject_scroll_event.vscroll = from->y;
-
-    return true;
-}
-
 static void
 input_manager_process_mouse_wheel(struct input_manager *im,
                                   const SDL_MouseWheelEvent *event) {
-    struct control_msg msg;
-    if (convert_mouse_wheel(event, im->screen, &msg)) {
-        if (!controller_push_msg(im->controller, &msg)) {
-            LOGW("Could not request 'inject mouse wheel event'");
-        }
-    }
+    im->mp->ops->process_mouse_wheel(im->mp, event);
 }
 
 bool

app/src/input_manager.h

@@ -11,18 +11,17 @@
 #include "fps_counter.h"
 #include "scrcpy.h"
 #include "screen.h"
+#include "trait/key_processor.h"
+#include "trait/mouse_processor.h"
 
 struct input_manager {
     struct controller *controller;
     struct screen *screen;
 
-    // SDL reports repeated events as a boolean, but Android expects the actual
-    // number of repetitions. This variable keeps track of the count.
-    unsigned repeat;
+    struct sc_key_processor *kp;
+    struct sc_mouse_processor *mp;
 
     bool control;
-    bool forward_key_repeat;
-    bool prefer_text;
     bool forward_all_clicks;
     bool legacy_paste;
 
@@ -43,7 +42,9 @@ struct input_manager {
 
 void
 input_manager_init(struct input_manager *im, struct controller *controller,
-                   struct screen *screen, const struct scrcpy_options *options);
+                   struct screen *screen, struct sc_key_processor *kp,
+                   struct sc_mouse_processor *mp,
+                   const struct scrcpy_options *options);
 
 bool
 input_manager_handle_event(struct input_manager *im, SDL_Event *event);

app/src/keyboard_inject.c

@@ -1,9 +1,20 @@
-#include "event_converter.h"
+#include "keyboard_inject.h"
+
+#include <assert.h>
+#include <SDL2/SDL_events.h>
+
+#include "android/input.h"
+#include "control_msg.h"
+#include "controller.h"
+#include "util/log.h"
+
+/** Downcast key processor to sc_keyboard_inject */
+#define DOWNCAST(KP) \
+    container_of(KP, struct sc_keyboard_inject, key_processor)
 
 #define MAP(FROM, TO) case FROM: *to = TO; return true
 #define FAIL default: return false
-
-bool
+static bool
 convert_keycode_action(SDL_EventType from, enum android_keyevent_action *to) {
     switch (from) {
         MAP(SDL_KEYDOWN, AKEY_EVENT_ACTION_DOWN);
@@ -12,67 +23,7 @@ convert_keycode_action(SDL_EventType from, enum android_keyevent_action *to) {
     }
 }
 
-static enum android_metastate
-autocomplete_metastate(enum android_metastate metastate) {
-    // fill dependent flags
-    if (metastate & (AMETA_SHIFT_LEFT_ON | AMETA_SHIFT_RIGHT_ON)) {
-        metastate |= AMETA_SHIFT_ON;
-    }
-    if (metastate & (AMETA_CTRL_LEFT_ON | AMETA_CTRL_RIGHT_ON)) {
-        metastate |= AMETA_CTRL_ON;
-    }
-    if (metastate & (AMETA_ALT_LEFT_ON | AMETA_ALT_RIGHT_ON)) {
-        metastate |= AMETA_ALT_ON;
-    }
-    if (metastate & (AMETA_META_LEFT_ON | AMETA_META_RIGHT_ON)) {
-        metastate |= AMETA_META_ON;
-    }
-
-    return metastate;
-}
-
-enum android_metastate
-convert_meta_state(SDL_Keymod mod) {
-    enum android_metastate metastate = 0;
-    if (mod & KMOD_LSHIFT) {
-        metastate |= AMETA_SHIFT_LEFT_ON;
-    }
-    if (mod & KMOD_RSHIFT) {
-        metastate |= AMETA_SHIFT_RIGHT_ON;
-    }
-    if (mod & KMOD_LCTRL) {
-        metastate |= AMETA_CTRL_LEFT_ON;
-    }
-    if (mod & KMOD_RCTRL) {
-        metastate |= AMETA_CTRL_RIGHT_ON;
-    }
-    if (mod & KMOD_LALT) {
-        metastate |= AMETA_ALT_LEFT_ON;
-    }
-    if (mod & KMOD_RALT) {
-        metastate |= AMETA_ALT_RIGHT_ON;
-    }
-    if (mod & KMOD_LGUI) { // Windows key
-        metastate |= AMETA_META_LEFT_ON;
-    }
-    if (mod & KMOD_RGUI) { // Windows key
-        metastate |= AMETA_META_RIGHT_ON;
-    }
-    if (mod & KMOD_NUM) {
-        metastate |= AMETA_NUM_LOCK_ON;
-    }
-    if (mod & KMOD_CAPS) {
-        metastate |= AMETA_CAPS_LOCK_ON;
-    }
-    if (mod & KMOD_MODE) { // Alt Gr
-        // no mapping?
-    }
-
-    // fill the dependent fields
-    return autocomplete_metastate(metastate);
-}
-
-bool
+static bool
 convert_keycode(SDL_Keycode from, enum android_keycode *to, uint16_t mod,
                 bool prefer_text) {
     switch (from) {
@@ -154,42 +105,150 @@ convert_keycode(SDL_Keycode from, enum android_keycode *to, uint16_t mod,
     }
 }
 
-enum android_motionevent_buttons
-convert_mouse_buttons(uint32_t state) {
-    enum android_motionevent_buttons buttons = 0;
-    if (state & SDL_BUTTON_LMASK) {
-        buttons |= AMOTION_EVENT_BUTTON_PRIMARY;
+static enum android_metastate
+autocomplete_metastate(enum android_metastate metastate) {
+    // fill dependent flags
+    if (metastate & (AMETA_SHIFT_LEFT_ON | AMETA_SHIFT_RIGHT_ON)) {
+        metastate |= AMETA_SHIFT_ON;
     }
-    if (state & SDL_BUTTON_RMASK) {
-        buttons |= AMOTION_EVENT_BUTTON_SECONDARY;
+    if (metastate & (AMETA_CTRL_LEFT_ON | AMETA_CTRL_RIGHT_ON)) {
+        metastate |= AMETA_CTRL_ON;
     }
-    if (state & SDL_BUTTON_MMASK) {
-        buttons |= AMOTION_EVENT_BUTTON_TERTIARY;
+    if (metastate & (AMETA_ALT_LEFT_ON | AMETA_ALT_RIGHT_ON)) {
+        metastate |= AMETA_ALT_ON;
     }
-    if (state & SDL_BUTTON_X1MASK) {
-        buttons |= AMOTION_EVENT_BUTTON_BACK;
+    if (metastate & (AMETA_META_LEFT_ON | AMETA_META_RIGHT_ON)) {
+        metastate |= AMETA_META_ON;
     }
-    if (state & SDL_BUTTON_X2MASK) {
-        buttons |= AMOTION_EVENT_BUTTON_FORWARD;
-    }
-    return buttons;
+
+    return metastate;
 }
 
-bool
-convert_mouse_action(SDL_EventType from, enum android_motionevent_action *to) {
-    switch (from) {
-        MAP(SDL_MOUSEBUTTONDOWN, AMOTION_EVENT_ACTION_DOWN);
-        MAP(SDL_MOUSEBUTTONUP,   AMOTION_EVENT_ACTION_UP);
-        FAIL;
+static enum android_metastate
+convert_meta_state(SDL_Keymod mod) {
+    enum android_metastate metastate = 0;
+    if (mod & KMOD_LSHIFT) {
+        metastate |= AMETA_SHIFT_LEFT_ON;
+    }
+    if (mod & KMOD_RSHIFT) {
+        metastate |= AMETA_SHIFT_RIGHT_ON;
+    }
+    if (mod & KMOD_LCTRL) {
+        metastate |= AMETA_CTRL_LEFT_ON;
+    }
+    if (mod & KMOD_RCTRL) {
+        metastate |= AMETA_CTRL_RIGHT_ON;
+    }
+    if (mod & KMOD_LALT) {
+        metastate |= AMETA_ALT_LEFT_ON;
+    }
+    if (mod & KMOD_RALT) {
+        metastate |= AMETA_ALT_RIGHT_ON;
+    }
+    if (mod & KMOD_LGUI) { // Windows key
+        metastate |= AMETA_META_LEFT_ON;
+    }
+    if (mod & KMOD_RGUI) { // Windows key
+        metastate |= AMETA_META_RIGHT_ON;
+    }
+    if (mod & KMOD_NUM) {
+        metastate |= AMETA_NUM_LOCK_ON;
+    }
+    if (mod & KMOD_CAPS) {
+        metastate |= AMETA_CAPS_LOCK_ON;
+    }
+    if (mod & KMOD_MODE) { // Alt Gr
+        // no mapping?
+    }
+
+    // fill the dependent fields
+    return autocomplete_metastate(metastate);
+}
+
+static bool
+convert_input_key(const SDL_KeyboardEvent *from, struct control_msg *to,
+                  bool prefer_text, uint32_t repeat) {
+    to->type = CONTROL_MSG_TYPE_INJECT_KEYCODE;
+
+    if (!convert_keycode_action(from->type, &to->inject_keycode.action)) {
+        return false;
+    }
+
+    uint16_t mod = from->keysym.mod;
+    if (!convert_keycode(from->keysym.sym, &to->inject_keycode.keycode, mod,
+                         prefer_text)) {
+        return false;
+    }
+
+    to->inject_keycode.repeat = repeat;
+    to->inject_keycode.metastate = convert_meta_state(mod);
+
+    return true;
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const SDL_KeyboardEvent *event) {
+    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+
+    if (event->repeat) {
+        if (!ki->forward_key_repeat) {
+            return;
+        }
+        ++ki->repeat;
+    } else {
+        ki->repeat = 0;
+    }
+
+    struct control_msg msg;
+    if (convert_input_key(event, &msg, ki->prefer_text, ki->repeat)) {
+        if (!controller_push_msg(ki->controller, &msg)) {
+            LOGW("Could not request 'inject keycode'");
+        }
     }
 }
 
-bool
-convert_touch_action(SDL_EventType from, enum android_motionevent_action *to) {
-    switch (from) {
-        MAP(SDL_FINGERMOTION, AMOTION_EVENT_ACTION_MOVE);
-        MAP(SDL_FINGERDOWN,   AMOTION_EVENT_ACTION_DOWN);
-        MAP(SDL_FINGERUP,     AMOTION_EVENT_ACTION_UP);
-        FAIL;
+static void
+sc_key_processor_process_text(struct sc_key_processor *kp,
+                              const SDL_TextInputEvent *event) {
+    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+
+    if (!ki->prefer_text) {
+        char c = event->text[0];
+        if (isalpha(c) || c == ' ') {
+            assert(event->text[1] == '\0');
+            // letters and space are handled as raw key event
+            return;
+        }
+    }
+
+    struct control_msg msg;
+    msg.type = CONTROL_MSG_TYPE_INJECT_TEXT;
+    msg.inject_text.text = strdup(event->text);
+    if (!msg.inject_text.text) {
+        LOGW("Could not strdup input text");
+        return;
+    }
+    if (!controller_push_msg(ki->controller, &msg)) {
+        free(msg.inject_text.text);
+        LOGW("Could not request 'inject text'");
     }
 }
+
+void
+sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
+                        struct controller *controller,
+                        const struct scrcpy_options *options) {
+    ki->controller = controller;
+    ki->prefer_text = options->prefer_text;
+    ki->forward_key_repeat = options->forward_key_repeat;
+
+    ki->repeat = 0;
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        .process_text = sc_key_processor_process_text,
+    };
+
+    ki->key_processor.ops = &ops;
+}

app/src/keyboard_inject.h

@@ -0,0 +1,30 @@
+#ifndef SC_KEYBOARD_INJECT_H
+#define SC_KEYBOARD_INJECT_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "scrcpy.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_inject {
+    struct sc_key_processor key_processor; // key processor trait
+
+    struct controller *controller;
+
+    // SDL reports repeated events as a boolean, but Android expects the actual
+    // number of repetitions. This variable keeps track of the count.
+    unsigned repeat;
+
+    bool prefer_text;
+    bool forward_key_repeat;
+};
+
+void
+sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
+                        struct controller *controller,
+                        const struct scrcpy_options *options);
+
+#endif

app/src/mouse_inject.c

@@ -0,0 +1,211 @@
+#include "mouse_inject.h"
+
+#include <assert.h>
+#include <SDL2/SDL_events.h>
+
+#include "android/input.h"
+#include "control_msg.h"
+#include "controller.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to sc_mouse_inject */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_inject, mouse_processor)
+
+static enum android_motionevent_buttons
+convert_mouse_buttons(uint32_t state) {
+    enum android_motionevent_buttons buttons = 0;
+    if (state & SDL_BUTTON_LMASK) {
+        buttons |= AMOTION_EVENT_BUTTON_PRIMARY;
+    }
+    if (state & SDL_BUTTON_RMASK) {
+        buttons |= AMOTION_EVENT_BUTTON_SECONDARY;
+    }
+    if (state & SDL_BUTTON_MMASK) {
+        buttons |= AMOTION_EVENT_BUTTON_TERTIARY;
+    }
+    if (state & SDL_BUTTON_X1MASK) {
+        buttons |= AMOTION_EVENT_BUTTON_BACK;
+    }
+    if (state & SDL_BUTTON_X2MASK) {
+        buttons |= AMOTION_EVENT_BUTTON_FORWARD;
+    }
+    return buttons;
+}
+
+#define MAP(FROM, TO) case FROM: *to = TO; return true
+#define FAIL default: return false
+static bool
+convert_mouse_action(SDL_EventType from, enum android_motionevent_action *to) {
+    switch (from) {
+        MAP(SDL_MOUSEBUTTONDOWN, AMOTION_EVENT_ACTION_DOWN);
+        MAP(SDL_MOUSEBUTTONUP,   AMOTION_EVENT_ACTION_UP);
+        FAIL;
+    }
+}
+
+static bool
+convert_touch_action(SDL_EventType from, enum android_motionevent_action *to) {
+    switch (from) {
+        MAP(SDL_FINGERMOTION, AMOTION_EVENT_ACTION_MOVE);
+        MAP(SDL_FINGERDOWN,   AMOTION_EVENT_ACTION_DOWN);
+        MAP(SDL_FINGERUP,     AMOTION_EVENT_ACTION_UP);
+        FAIL;
+    }
+}
+
+static bool
+convert_mouse_motion(const SDL_MouseMotionEvent *from, struct screen *screen,
+                     struct control_msg *to) {
+    to->type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT;
+    to->inject_touch_event.action = AMOTION_EVENT_ACTION_MOVE;
+    to->inject_touch_event.pointer_id = POINTER_ID_MOUSE;
+    to->inject_touch_event.position.screen_size = screen->frame_size;
+    to->inject_touch_event.position.point =
+        screen_convert_window_to_frame_coords(screen, from->x, from->y);
+    to->inject_touch_event.pressure = 1.f;
+    to->inject_touch_event.buttons = convert_mouse_buttons(from->state);
+
+    return true;
+}
+
+static bool
+convert_touch(const SDL_TouchFingerEvent *from, struct screen *screen,
+              struct control_msg *to) {
+    to->type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT;
+
+    if (!convert_touch_action(from->type, &to->inject_touch_event.action)) {
+        return false;
+    }
+
+    to->inject_touch_event.pointer_id = from->fingerId;
+    to->inject_touch_event.position.screen_size = screen->frame_size;
+
+    int dw;
+    int dh;
+    SDL_GL_GetDrawableSize(screen->window, &dw, &dh);
+
+    // SDL touch event coordinates are normalized in the range [0; 1]
+    int32_t x = from->x * dw;
+    int32_t y = from->y * dh;
+    to->inject_touch_event.position.point =
+        screen_convert_drawable_to_frame_coords(screen, x, y);
+
+    to->inject_touch_event.pressure = from->pressure;
+    to->inject_touch_event.buttons = 0;
+    return true;
+}
+
+static bool
+convert_mouse_button(const SDL_MouseButtonEvent *from, struct screen *screen,
+                     struct control_msg *to) {
+    to->type = CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT;
+
+    if (!convert_mouse_action(from->type, &to->inject_touch_event.action)) {
+        return false;
+    }
+
+    to->inject_touch_event.pointer_id = POINTER_ID_MOUSE;
+    to->inject_touch_event.position.screen_size = screen->frame_size;
+    to->inject_touch_event.position.point =
+        screen_convert_window_to_frame_coords(screen, from->x, from->y);
+    to->inject_touch_event.pressure =
+        from->type == SDL_MOUSEBUTTONDOWN ? 1.f : 0.f;
+    to->inject_touch_event.buttons =
+        convert_mouse_buttons(SDL_BUTTON(from->button));
+
+    return true;
+}
+
+static bool
+convert_mouse_wheel(const SDL_MouseWheelEvent *from, struct screen *screen,
+                    struct control_msg *to) {
+
+    // mouse_x and mouse_y are expressed in pixels relative to the window
+    int mouse_x;
+    int mouse_y;
+    SDL_GetMouseState(&mouse_x, &mouse_y);
+
+    struct position position = {
+        .screen_size = screen->frame_size,
+        .point = screen_convert_window_to_frame_coords(screen,
+                                                       mouse_x, mouse_y),
+    };
+
+    to->type = CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT;
+
+    to->inject_scroll_event.position = position;
+    to->inject_scroll_event.hscroll = from->x;
+    to->inject_scroll_event.vscroll = from->y;
+
+    return true;
+}
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                        const SDL_MouseMotionEvent *event) {
+    struct sc_mouse_inject *mi = DOWNCAST(mp);
+
+    struct control_msg msg;
+    if (!convert_mouse_motion(event, mi->screen, &msg)) {
+        return;
+    }
+
+    if (!controller_push_msg(mi->controller, &msg)) {
+        LOGW("Could not request 'inject mouse motion event'");
+    }
+}
+
+static void
+sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
+                                 const SDL_TouchFingerEvent *event) {
+    struct sc_mouse_inject *mi = DOWNCAST(mp);
+
+    struct control_msg msg;
+    if (convert_touch(event, mi->screen, &msg)) {
+        if (!controller_push_msg(mi->controller, &msg)) {
+            LOGW("Could not request 'inject touch event'");
+        }
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_button(struct sc_mouse_processor *mp,
+                                        const SDL_MouseButtonEvent *event) {
+    struct sc_mouse_inject *mi = DOWNCAST(mp);
+
+    struct control_msg msg;
+    if (convert_mouse_button(event, mi->screen, &msg)) {
+        if (!controller_push_msg(mi->controller, &msg)) {
+            LOGW("Could not request 'inject mouse button event'");
+        }
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_wheel(struct sc_mouse_processor *mp,
+                                       const SDL_MouseWheelEvent *event) {
+    struct sc_mouse_inject *mi = DOWNCAST(mp);
+
+    struct control_msg msg;
+    if (convert_mouse_wheel(event, mi->screen, &msg)) {
+        if (!controller_push_msg(mi->controller, &msg)) {
+            LOGW("Could not request 'inject mouse wheel event'");
+        }
+    }
+}
+
+void
+sc_mouse_inject_init(struct sc_mouse_inject *mi, struct controller *controller,
+                     struct screen *screen) {
+    mi->controller = controller;
+    mi->screen = screen;
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_touch = sc_mouse_processor_process_touch,
+        .process_mouse_button = sc_mouse_processor_process_mouse_button,
+        .process_mouse_wheel = sc_mouse_processor_process_mouse_wheel,
+    };
+
+    mi->mouse_processor.ops = &ops;
+}

app/src/mouse_inject.h

@@ -0,0 +1,24 @@
+#ifndef SC_MOUSE_INJECT_H
+#define SC_MOUSE_INJECT_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "scrcpy.h"
+#include "screen.h"
+#include "trait/mouse_processor.h"
+
+struct sc_mouse_inject {
+    struct sc_mouse_processor mouse_processor; // mouse processor trait
+
+    struct controller *controller;
+    struct screen *screen;
+};
+
+void
+sc_mouse_inject_init(struct sc_mouse_inject *mi, struct controller *controller,
+                     struct screen *screen);
+
+#endif

app/src/recorder.c

@@ -1,6 +1,8 @@
 #include "recorder.h"
 
 #include <assert.h>
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
 #include <libavutil/time.h>
 
 #include "util/log.h"
@@ -51,16 +53,15 @@ record_packet_new(const AVPacket *packet) {
 
 static void
 record_packet_delete(struct record_packet *rec) {
-    av_packet_unref(rec->packet);
     av_packet_free(&rec->packet);
     free(rec);
 }
 
 static void
 recorder_queue_clear(struct recorder_queue *queue) {
-    while (!queue_is_empty(queue)) {
+    while (!sc_queue_is_empty(queue)) {
         struct record_packet *rec;
-        queue_take(queue, next, &rec);
+        sc_queue_take(queue, next, &rec);
         record_packet_delete(rec);
     }
 }
@@ -136,14 +137,14 @@ run_recorder(void *data) {
     for (;;) {
         sc_mutex_lock(&recorder->mutex);
 
-        while (!recorder->stopped && queue_is_empty(&recorder->queue)) {
+        while (!recorder->stopped && sc_queue_is_empty(&recorder->queue)) {
             sc_cond_wait(&recorder->queue_cond, &recorder->mutex);
         }
 
         // if stopped is set, continue to process the remaining events (to
         // finish the recording) before actually stopping
 
-        if (recorder->stopped && queue_is_empty(&recorder->queue)) {
+        if (recorder->stopped && sc_queue_is_empty(&recorder->queue)) {
             sc_mutex_unlock(&recorder->mutex);
             struct record_packet *last = recorder->previous;
             if (last) {
@@ -162,7 +163,7 @@ run_recorder(void *data) {
         }
 
         struct record_packet *rec;
-        queue_take(&recorder->queue, next, &rec);
+        sc_queue_take(&recorder->queue, next, &rec);
 
         sc_mutex_unlock(&recorder->mutex);
 
@@ -214,7 +215,8 @@ run_recorder(void *data) {
         LOGE("Recording failed to %s", recorder->filename);
     } else {
         const char *format_name = recorder_get_format_name(recorder->format);
-        LOGI("Recording complete to %s file: %s", format_name, recorder->filename);
+        LOGI("Recording complete to %s file: %s", format_name,
+                                                  recorder->filename);
     }
 
     LOGD("Recorder thread ended");
@@ -236,7 +238,7 @@ recorder_open(struct recorder *recorder, const AVCodec *input_codec) {
         goto error_mutex_destroy;
     }
 
-    queue_init(&recorder->queue);
+    sc_queue_init(&recorder->queue);
     recorder->stopped = false;
     recorder->failed = false;
     recorder->header_written = false;
@@ -341,7 +343,7 @@ recorder_push(struct recorder *recorder, const AVPacket *packet) {
         return false;
     }
 
-    queue_push(&recorder->queue, next, rec);
+    sc_queue_push(&recorder->queue, next, rec);
     sc_cond_signal(&recorder->queue_cond);
 
     sc_mutex_unlock(&recorder->mutex);

app/src/recorder.h

@@ -17,7 +17,7 @@ struct record_packet {
     struct record_packet *next;
 };
 
-struct recorder_queue QUEUE(struct record_packet);
+struct recorder_queue SC_QUEUE(struct record_packet);
 
 struct recorder {
     struct sc_packet_sink packet_sink; // packet sink trait

app/src/scrcpy.c

@@ -18,6 +18,8 @@
 #include "events.h"
 #include "file_handler.h"
 #include "input_manager.h"
+#include "keyboard_inject.h"
+#include "mouse_inject.h"
 #include "recorder.h"
 #include "screen.h"
 #include "server.h"
@@ -40,6 +42,8 @@ struct scrcpy {
 #endif
     struct controller controller;
     struct file_handler file_handler;
+    struct sc_keyboard_inject keyboard_inject;
+    struct sc_mouse_inject mouse_inject;
     struct input_manager input_manager;
 };
 
@@ -330,7 +334,7 @@ scrcpy(const struct scrcpy_options *options) {
 
     av_log_set_callback(av_log_callback);
 
-    const struct stream_callbacks stream_cbs = {
+    static const struct stream_callbacks stream_cbs = {
         .on_eos = stream_on_eos,
     };
     stream_init(&s->stream, s->server.video_socket, &stream_cbs, NULL);
@@ -343,19 +347,29 @@ scrcpy(const struct scrcpy_options *options) {
         stream_add_sink(&s->stream, &rec->packet_sink);
     }
 
-    if (options->display) {
-        if (options->control) {
-            if (!controller_init(&s->controller, s->server.control_socket)) {
-                goto end;
-            }
-            controller_initialized = true;
-
-            if (!controller_start(&s->controller)) {
-                goto end;
-            }
-            controller_started = true;
+    if (options->control) {
+        if (!controller_init(&s->controller, s->server.control_socket)) {
+            goto end;
         }
+        controller_initialized = true;
 
+        if (!controller_start(&s->controller)) {
+            goto end;
+        }
+        controller_started = true;
+
+        if (options->turn_screen_off) {
+            struct control_msg msg;
+            msg.type = CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
+            msg.set_screen_power_mode.mode = SCREEN_POWER_MODE_OFF;
+
+            if (!controller_push_msg(&s->controller, &msg)) {
+                LOGW("Could not request 'set screen power mode'");
+            }
+        }
+    }
+
+    if (options->display) {
         const char *window_title =
             options->window_title ? options->window_title : device_name;
 
@@ -371,6 +385,7 @@ scrcpy(const struct scrcpy_options *options) {
             .rotation = options->rotation,
             .mipmaps = options->mipmaps,
             .fullscreen = options->fullscreen,
+            .buffering_time = options->display_buffer,
         };
 
         if (!screen_init(&s->screen, &screen_params)) {
@@ -379,21 +394,12 @@ scrcpy(const struct scrcpy_options *options) {
         screen_initialized = true;
 
         decoder_add_sink(&s->decoder, &s->screen.frame_sink);
-
-        if (options->turn_screen_off) {
-            struct control_msg msg;
-            msg.type = CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
-            msg.set_screen_power_mode.mode = SCREEN_POWER_MODE_OFF;
-
-            if (!controller_push_msg(&s->controller, &msg)) {
-                LOGW("Could not request 'set screen power mode'");
-            }
-        }
     }
 
 #ifdef HAVE_V4L2
     if (options->v4l2_device) {
-        if (!sc_v4l2_sink_init(&s->v4l2_sink, options->v4l2_device, frame_size)) {
+        if (!sc_v4l2_sink_init(&s->v4l2_sink, options->v4l2_device, frame_size,
+                               options->v4l2_buffer)) {
             goto end;
         }
 
@@ -410,7 +416,19 @@ scrcpy(const struct scrcpy_options *options) {
     }
     stream_started = true;
 
-    input_manager_init(&s->input_manager, &s->controller, &s->screen, options);
+    struct sc_key_processor *kp = NULL;
+    struct sc_mouse_processor *mp = NULL;
+
+    if (options->control) {
+        sc_keyboard_inject_init(&s->keyboard_inject, &s->controller, options);
+        kp = &s->keyboard_inject.key_processor;
+
+        sc_mouse_inject_init(&s->mouse_inject, &s->controller, &s->screen);
+        mp = &s->mouse_inject.mouse_processor;
+    }
+
+    input_manager_init(&s->input_manager, &s->controller, &s->screen, kp, mp,
+                       options);
 
     ret = event_loop(s, options);
     LOGD("quit...");

app/src/scrcpy.h

@@ -7,6 +7,8 @@
 #include <stddef.h>
 #include <stdint.h>
 
+#include "util/tick.h"
+
 enum sc_log_level {
     SC_LOG_LEVEL_VERBOSE,
     SC_LOG_LEVEL_DEBUG,
@@ -78,6 +80,8 @@ struct scrcpy_options {
     uint16_t window_width;
     uint16_t window_height;
     uint32_t display_id;
+    sc_tick display_buffer;
+    sc_tick v4l2_buffer;
     bool show_touches;
     bool fullscreen;
     bool always_on_top;
@@ -126,6 +130,8 @@ struct scrcpy_options {
     .window_width = 0, \
     .window_height = 0, \
     .display_id = 0, \
+    .display_buffer = 0, \
+    .v4l2_buffer = 0, \
     .show_touches = false, \
     .fullscreen = false, \
     .always_on_top = false, \

app/src/screen.c

@@ -274,14 +274,16 @@ screen_frame_sink_close(struct sc_frame_sink *sink) {
 static bool
 screen_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
     struct screen *screen = DOWNCAST(sink);
+    return sc_video_buffer_push(&screen->vb, frame);
+}
 
-    bool previous_frame_skipped;
-    bool ok = video_buffer_push(&screen->vb, frame, &previous_frame_skipped);
-    if (!ok) {
-        return false;
-    }
+static void
+sc_video_buffer_on_new_frame(struct sc_video_buffer *vb, bool previous_skipped,
+                             void *userdata) {
+    (void) vb;
+    struct screen *screen = userdata;
 
-    if (previous_frame_skipped) {
+    if (previous_skipped) {
         fps_counter_add_skipped_frame(&screen->fps_counter);
         // The EVENT_NEW_FRAME triggered for the previous frame will consume
         // this new frame instead
@@ -293,8 +295,6 @@ screen_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
         // Post the event on the UI thread
         SDL_PushEvent(&new_frame_event);
     }
-
-    return true;
 }
 
 bool
@@ -304,15 +304,26 @@ screen_init(struct screen *screen, const struct screen_params *params) {
     screen->fullscreen = false;
     screen->maximized = false;
 
-    bool ok = video_buffer_init(&screen->vb);
+    static const struct sc_video_buffer_callbacks cbs = {
+        .on_new_frame = sc_video_buffer_on_new_frame,
+    };
+
+    bool ok = sc_video_buffer_init(&screen->vb, params->buffering_time, &cbs,
+                                   screen);
     if (!ok) {
         LOGE("Could not initialize video buffer");
         return false;
     }
 
+    ok = sc_video_buffer_start(&screen->vb);
+    if (!ok) {
+        LOGE("Could not start video_buffer");
+        goto error_destroy_video_buffer;
+    }
+
     if (!fps_counter_init(&screen->fps_counter)) {
         LOGE("Could not initialize FPS counter");
-        goto error_destroy_video_buffer;
+        goto error_stop_and_join_video_buffer;
     }
 
     screen->frame_size = params->frame_size;
@@ -453,8 +464,11 @@ error_destroy_window:
     SDL_DestroyWindow(screen->window);
 error_destroy_fps_counter:
     fps_counter_destroy(&screen->fps_counter);
+error_stop_and_join_video_buffer:
+    sc_video_buffer_stop(&screen->vb);
+    sc_video_buffer_join(&screen->vb);
 error_destroy_video_buffer:
-    video_buffer_destroy(&screen->vb);
+    sc_video_buffer_destroy(&screen->vb);
 
     return false;
 }
@@ -471,11 +485,13 @@ screen_hide_window(struct screen *screen) {
 
 void
 screen_interrupt(struct screen *screen) {
+    sc_video_buffer_stop(&screen->vb);
     fps_counter_interrupt(&screen->fps_counter);
 }
 
 void
 screen_join(struct screen *screen) {
+    sc_video_buffer_join(&screen->vb);
     fps_counter_join(&screen->fps_counter);
 }
 
@@ -489,7 +505,7 @@ screen_destroy(struct screen *screen) {
     SDL_DestroyRenderer(screen->renderer);
     SDL_DestroyWindow(screen->window);
     fps_counter_destroy(&screen->fps_counter);
-    video_buffer_destroy(&screen->vb);
+    sc_video_buffer_destroy(&screen->vb);
 }
 
 static void
@@ -595,7 +611,7 @@ update_texture(struct screen *screen, const AVFrame *frame) {
 static bool
 screen_update_frame(struct screen *screen) {
     av_frame_unref(screen->frame);
-    video_buffer_consume(&screen->vb, screen->frame);
+    sc_video_buffer_consume(&screen->vb, screen->frame);
     AVFrame *frame = screen->frame;
 
     fps_counter_add_rendered_frame(&screen->fps_counter);

app/src/screen.h

@@ -8,6 +8,7 @@
 #include <libavformat/avformat.h>
 
 #include "coords.h"
+#include "fps_counter.h"
 #include "opengl.h"
 #include "trait/frame_sink.h"
 #include "video_buffer.h"
@@ -19,7 +20,7 @@ struct screen {
     bool open; // track the open/close state to assert correct behavior
 #endif
 
-    struct video_buffer vb;
+    struct sc_video_buffer vb;
     struct fps_counter fps_counter;
 
     SDL_Window *window;
@@ -62,6 +63,8 @@ struct screen_params {
     bool mipmaps;
 
     bool fullscreen;
+
+    sc_tick buffering_time;
 };
 
 // initialize screen, create window, renderer and texture (window is hidden)

app/src/server.c

@@ -60,7 +60,20 @@ get_server_path(void) {
         // not found, use current directory
         return strdup(SERVER_FILENAME);
     }
-    char *dir = dirname(executable_path);
+
+    // dirname() does not work correctly everywhere, so get the parent
+    // directory manually.
+    // See <https://github.com/Genymobile/scrcpy/issues/2619>
+    char *p = strrchr(executable_path, PATH_SEPARATOR);
+    if (!p) {
+        LOGE("Unexpected executable path: \"%s\" (it should contain a '%c')",
+             executable_path, PATH_SEPARATOR);
+        free(executable_path);
+        return strdup(SERVER_FILENAME);
+    }
+
+    *p = '\0'; // modify executable_path in place
+    char *dir = executable_path;
     size_t dirlen = strlen(dir);
 
     // sizeof(SERVER_FILENAME) gives statically the size including the null byte
@@ -261,7 +274,8 @@ execute_server(struct server *server, const struct server_params *params) {
     sprintf(max_size_string, "%"PRIu16, params->max_size);
     sprintf(bit_rate_string, "%"PRIu32, params->bit_rate);
     sprintf(max_fps_string, "%"PRIu16, params->max_fps);
-    sprintf(lock_video_orientation_string, "%"PRIi8, params->lock_video_orientation);
+    sprintf(lock_video_orientation_string, "%"PRIi8,
+            params->lock_video_orientation);
     sprintf(display_id_string, "%"PRIu32, params->display_id);
     const char *const cmd[] = {
         "shell",
@@ -271,7 +285,8 @@ execute_server(struct server *server, const struct server_params *params) {
 # define SERVER_DEBUGGER_PORT "5005"
 # ifdef SERVER_DEBUGGER_METHOD_NEW
         /* Android 9 and above */
-        "-XjdwpProvider:internal -XjdwpOptions:transport=dt_socket,suspend=y,server=y,address="
+        "-XjdwpProvider:internal -XjdwpOptions:transport=dt_socket,suspend=y,"
+        "server=y,address="
 # else
         /* Android 8 and below */
         "-agentlib:jdwp=transport=dt_socket,suspend=y,server=y,address="
@@ -468,7 +483,7 @@ error:
 static bool
 device_read_info(socket_t device_socket, char *device_name, struct size *size) {
     unsigned char buf[DEVICE_NAME_FIELD_LENGTH + 4];
-    int r = net_recv_all(device_socket, buf, sizeof(buf));
+    ssize_t r = net_recv_all(device_socket, buf, sizeof(buf));
     if (r < DEVICE_NAME_FIELD_LENGTH + 4) {
         LOGE("Could not retrieve device information");
         return false;
@@ -554,10 +569,10 @@ server_stop(struct server *server) {
     sc_mutex_lock(&server->mutex);
     bool signaled = false;
     if (!server->process_terminated) {
-#define WATCHDOG_DELAY_MS 1000
+#define WATCHDOG_DELAY SC_TICK_FROM_SEC(1)
         signaled = sc_cond_timedwait(&server->process_terminated_cond,
                                      &server->mutex,
-                                     WATCHDOG_DELAY_MS);
+                                     sc_tick_now() + WATCHDOG_DELAY);
     }
     sc_mutex_unlock(&server->mutex);
 

app/src/stream.c

@@ -151,7 +151,6 @@ stream_push_packet(struct stream *stream, AVPacket *packet) {
 
         if (stream->pending) {
             // the pending packet must be discarded (consumed or error)
-            av_packet_unref(stream->pending);
             av_packet_free(&stream->pending);
         }
 
@@ -244,7 +243,6 @@ run_stream(void *data) {
     LOGD("End of frames");
 
     if (stream->pending) {
-        av_packet_unref(stream->pending);
         av_packet_free(&stream->pending);
     }
 

app/src/trait/frame_sink.h

@@ -1,5 +1,5 @@
-#ifndef SC_FRAME_SINK
-#define SC_FRAME_SINK
+#ifndef SC_FRAME_SINK_H
+#define SC_FRAME_SINK_H
 
 #include "common.h"
 

app/src/trait/key_processor.h

@@ -0,0 +1,29 @@
+#ifndef SC_KEY_PROCESSOR_H
+#define SC_KEY_PROCESSOR_H
+
+#include "common.h"
+
+#include <assert.h>
+#include <stdbool.h>
+
+#include <SDL2/SDL_events.h>
+
+/**
+ * Key processor trait.
+ *
+ * Component able to process and inject keys should implement this trait.
+ */
+struct sc_key_processor {
+    const struct sc_key_processor_ops *ops;
+};
+
+struct sc_key_processor_ops {
+    void
+    (*process_key)(struct sc_key_processor *kp, const SDL_KeyboardEvent *event);
+
+    void
+    (*process_text)(struct sc_key_processor *kp,
+                    const SDL_TextInputEvent *event);
+};
+
+#endif

app/src/trait/mouse_processor.h

@@ -0,0 +1,39 @@
+#ifndef SC_MOUSE_PROCESSOR_H
+#define SC_MOUSE_PROCESSOR_H
+
+#include "common.h"
+
+#include <assert.h>
+#include <stdbool.h>
+
+#include <SDL2/SDL_events.h>
+
+/**
+ * Mouse processor trait.
+ *
+ * Component able to process and inject mouse events should implement this
+ * trait.
+ */
+struct sc_mouse_processor {
+    const struct sc_mouse_processor_ops *ops;
+};
+
+struct sc_mouse_processor_ops {
+    void
+    (*process_mouse_motion)(struct sc_mouse_processor *mp,
+                            const SDL_MouseMotionEvent *event);
+
+    void
+    (*process_touch)(struct sc_mouse_processor *mp,
+                     const SDL_TouchFingerEvent *event);
+
+    void
+    (*process_mouse_button)(struct sc_mouse_processor *mp,
+                            const SDL_MouseButtonEvent *event);
+
+    void
+    (*process_mouse_wheel)(struct sc_mouse_processor *mp,
+                           const SDL_MouseWheelEvent *event);
+};
+
+#endif

app/src/trait/packet_sink.h

@@ -1,5 +1,5 @@
-#ifndef SC_PACKET_SINK
-#define SC_PACKET_SINK
+#ifndef SC_PACKET_SINK_H
+#define SC_PACKET_SINK_H
 
 #include "common.h"
 

app/src/util/net.c

@@ -19,11 +19,27 @@
   typedef struct in_addr IN_ADDR;
 #endif
 
+static void
+net_perror(const char *s) {
+#ifdef _WIN32
+    int error = WSAGetLastError();
+    char *wsa_message;
+    FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM,
+            NULL, error, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
+            (char *) &wsa_message, 0, NULL);
+    // no explicit '\n', wsa_message already contains a trailing '\n'
+    fprintf(stderr, "%s: [%d] %s", s, error, wsa_message);
+    LocalFree(wsa_message);
+#else
+    perror(s);
+#endif
+}
+
 socket_t
 net_connect(uint32_t addr, uint16_t port) {
     socket_t sock = socket(AF_INET, SOCK_STREAM, 0);
     if (sock == INVALID_SOCKET) {
-        perror("socket");
+        net_perror("socket");
         return INVALID_SOCKET;
     }
 
@@ -33,7 +49,7 @@ net_connect(uint32_t addr, uint16_t port) {
     sin.sin_port = htons(port);
 
     if (connect(sock, (SOCKADDR *) &sin, sizeof(sin)) == SOCKET_ERROR) {
-        perror("connect");
+        net_perror("connect");
         net_close(sock);
         return INVALID_SOCKET;
     }
@@ -45,14 +61,14 @@ socket_t
 net_listen(uint32_t addr, uint16_t port, int backlog) {
     socket_t sock = socket(AF_INET, SOCK_STREAM, 0);
     if (sock == INVALID_SOCKET) {
-        perror("socket");
+        net_perror("socket");
         return INVALID_SOCKET;
     }
 
     int reuse = 1;
     if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, (const void *) &reuse,
                    sizeof(reuse)) == -1) {
-        perror("setsockopt(SO_REUSEADDR)");
+        net_perror("setsockopt(SO_REUSEADDR)");
     }
 
     SOCKADDR_IN sin;
@@ -61,13 +77,13 @@ net_listen(uint32_t addr, uint16_t port, int backlog) {
     sin.sin_port = htons(port);
 
     if (bind(sock, (SOCKADDR *) &sin, sizeof(sin)) == SOCKET_ERROR) {
-        perror("bind");
+        net_perror("bind");
         net_close(sock);
         return INVALID_SOCKET;
     }
 
     if (listen(sock, backlog) == SOCKET_ERROR) {
-        perror("listen");
+        net_perror("listen");
         net_close(sock);
         return INVALID_SOCKET;
     }
@@ -99,16 +115,18 @@ net_send(socket_t socket, const void *buf, size_t len) {
 
 ssize_t
 net_send_all(socket_t socket, const void *buf, size_t len) {
+    size_t copied = 0;
     ssize_t w = 0;
     while (len > 0) {
         w = send(socket, buf, len, 0);
         if (w == -1) {
-            return -1;
+            return copied ? (ssize_t) copied : -1;
         }
         len -= w;
         buf = (char *) buf + w;
+        copied += w;
     }
-    return w;
+    return copied;
 }
 
 bool

app/src/util/queue.h

@@ -1,6 +1,6 @@
 // generic intrusive FIFO queue
-#ifndef QUEUE_H
-#define QUEUE_H
+#ifndef SC_QUEUE_H
+#define SC_QUEUE_H
 
 #include "common.h"
 
@@ -10,15 +10,15 @@
 
 // To define a queue type of "struct foo":
 //    struct queue_foo QUEUE(struct foo);
-#define QUEUE(TYPE) { \
+#define SC_QUEUE(TYPE) { \
     TYPE *first; \
     TYPE *last; \
 }
 
-#define queue_init(PQ) \
+#define sc_queue_init(PQ) \
     (void) ((PQ)->first = (PQ)->last = NULL)
 
-#define queue_is_empty(PQ) \
+#define sc_queue_is_empty(PQ) \
     !(PQ)->first
 
 // NEXTFIELD is the field in the ITEM type used for intrusive linked-list
@@ -30,30 +30,30 @@
 //    };
 //
 //    // define the type "struct my_queue"
-//    struct my_queue QUEUE(struct foo);
+//    struct my_queue SC_QUEUE(struct foo);
 //
 //    struct my_queue queue;
-//    queue_init(&queue);
+//    sc_queue_init(&queue);
 //
 //    struct foo v1 = { .value = 42 };
 //    struct foo v2 = { .value = 27 };
 //
-//    queue_push(&queue, next, v1);
-//    queue_push(&queue, next, v2);
+//    sc_queue_push(&queue, next, v1);
+//    sc_queue_push(&queue, next, v2);
 //
 //    struct foo *foo;
-//    queue_take(&queue, next, &foo);
+//    sc_queue_take(&queue, next, &foo);
 //    assert(foo->value == 42);
-//    queue_take(&queue, next, &foo);
+//    sc_queue_take(&queue, next, &foo);
 //    assert(foo->value == 27);
-//    assert(queue_is_empty(&queue));
+//    assert(sc_queue_is_empty(&queue));
 //
 
 // push a new item into the queue
-#define queue_push(PQ, NEXTFIELD, ITEM) \
+#define sc_queue_push(PQ, NEXTFIELD, ITEM) \
     (void) ({ \
         (ITEM)->NEXTFIELD = NULL; \
-        if (queue_is_empty(PQ)) { \
+        if (sc_queue_is_empty(PQ)) { \
             (PQ)->first = (PQ)->last = (ITEM); \
         } else { \
             (PQ)->last->NEXTFIELD = (ITEM); \
@@ -65,9 +65,9 @@
 // the result is stored in *(PITEM)
 // (without typeof(), we could not store a local variable having the correct
 // type so that we can "return" it)
-#define queue_take(PQ, NEXTFIELD, PITEM) \
+#define sc_queue_take(PQ, NEXTFIELD, PITEM) \
     (void) ({ \
-        assert(!queue_is_empty(PQ)); \
+        assert(!sc_queue_is_empty(PQ)); \
         *(PITEM) = (PQ)->first; \
         (PQ)->first = (PQ)->first->NEXTFIELD; \
     })

app/src/util/thread.c

@@ -31,7 +31,7 @@ sc_mutex_init(sc_mutex *mutex) {
 
     mutex->mutex = sdl_mutex;
 #ifndef NDEBUG
-    mutex->locker = 0;
+    atomic_init(&mutex->locker, 0);
 #endif
     return true;
 }
@@ -52,7 +52,8 @@ sc_mutex_lock(sc_mutex *mutex) {
         abort();
     }
 
-    mutex->locker = sc_thread_get_id();
+    atomic_store_explicit(&mutex->locker, sc_thread_get_id(),
+                          memory_order_relaxed);
 #else
     (void) r;
 #endif
@@ -62,7 +63,7 @@ void
 sc_mutex_unlock(sc_mutex *mutex) {
 #ifndef NDEBUG
     assert(sc_mutex_held(mutex));
-    mutex->locker = 0;
+    atomic_store_explicit(&mutex->locker, 0, memory_order_relaxed);
 #endif
     int r = SDL_UnlockMutex(mutex->mutex);
 #ifndef NDEBUG
@@ -83,7 +84,9 @@ sc_thread_get_id(void) {
 #ifndef NDEBUG
 bool
 sc_mutex_held(struct sc_mutex *mutex) {
-    return mutex->locker == sc_thread_get_id();
+    sc_thread_id locker_id =
+        atomic_load_explicit(&mutex->locker, memory_order_relaxed);
+    return locker_id == sc_thread_get_id();
 }
 #endif
 
@@ -112,14 +115,21 @@ sc_cond_wait(sc_cond *cond, sc_mutex *mutex) {
         abort();
     }
 
-    mutex->locker = sc_thread_get_id();
+    atomic_store_explicit(&mutex->locker, sc_thread_get_id(),
+                          memory_order_relaxed);
 #else
     (void) r;
 #endif
 }
 
 bool
-sc_cond_timedwait(sc_cond *cond, sc_mutex *mutex, uint32_t ms) {
+sc_cond_timedwait(sc_cond *cond, sc_mutex *mutex, sc_tick deadline) {
+    sc_tick now = sc_tick_now();
+    if (deadline <= now) {
+        return false; // timeout
+    }
+
+    uint32_t ms = SC_TICK_TO_MS(deadline - now);
     int r = SDL_CondWaitTimeout(cond->cond, mutex->mutex, ms);
 #ifndef NDEBUG
     if (r < 0) {
@@ -127,7 +137,8 @@ sc_cond_timedwait(sc_cond *cond, sc_mutex *mutex, uint32_t ms) {
         abort();
     }
 
-    mutex->locker = sc_thread_get_id();
+    atomic_store_explicit(&mutex->locker, sc_thread_get_id(),
+                          memory_order_relaxed);
 #endif
     assert(r == 0 || r == SDL_MUTEX_TIMEDOUT);
     return r == 0;

app/src/util/thread.h

@@ -3,8 +3,10 @@
 
 #include "common.h"
 
+#include <stdatomic.h>
 #include <stdbool.h>
-#include <stdint.h>
+
+#include "tick.h"
 
 /* Forward declarations */
 typedef struct SDL_Thread SDL_Thread;
@@ -12,7 +14,8 @@ typedef struct SDL_mutex SDL_mutex;
 typedef struct SDL_cond SDL_cond;
 
 typedef int sc_thread_fn(void *);
-typedef unsigned int sc_thread_id;
+typedef unsigned sc_thread_id;
+typedef atomic_uint sc_atomic_thread_id;
 
 typedef struct sc_thread {
     SDL_Thread *thread;
@@ -21,7 +24,7 @@ typedef struct sc_thread {
 typedef struct sc_mutex {
     SDL_mutex *mutex;
 #ifndef NDEBUG
-    sc_thread_id locker;
+    sc_atomic_thread_id locker;
 #endif
 } sc_mutex;
 
@@ -70,7 +73,7 @@ sc_cond_wait(sc_cond *cond, sc_mutex *mutex);
 
 // return true on signaled, false on timeout
 bool
-sc_cond_timedwait(sc_cond *cond, sc_mutex *mutex, uint32_t ms);
+sc_cond_timedwait(sc_cond *cond, sc_mutex *mutex, sc_tick deadline);
 
 void
 sc_cond_signal(sc_cond *cond);

app/src/util/tick.c

@@ -0,0 +1,16 @@
+#include "tick.h"
+
+#include <SDL2/SDL_timer.h>
+
+sc_tick
+sc_tick_now(void) {
+    // SDL_GetTicks() resolution is in milliseconds, but sc_tick are expressed
+    // in microseconds to store PTS without precision loss.
+    //
+    // As an alternative, SDL_GetPerformanceCounter() and
+    // SDL_GetPerformanceFrequency() could be used, but:
+    //  - the conversions (avoiding overflow) are expansive, since the
+    //    frequency is not known at compile time;
+    //  - in practice, we don't need more precision for now.
+    return (sc_tick) SDL_GetTicks() * 1000;
+}

app/src/util/tick.h

@@ -0,0 +1,21 @@
+#ifndef SC_TICK_H
+#define SC_TICK_H
+
+#include <stdint.h>
+
+typedef int64_t sc_tick;
+#define PRItick PRIi64
+#define SC_TICK_FREQ 1000000 // microsecond
+
+// To be adapted if SC_TICK_FREQ changes
+#define SC_TICK_TO_US(tick) (tick)
+#define SC_TICK_TO_MS(tick) ((tick) / 1000)
+#define SC_TICK_TO_SEC(tick) ((tick) / 1000000)
+#define SC_TICK_FROM_US(us) (us)
+#define SC_TICK_FROM_MS(ms) ((ms) * 1000)
+#define SC_TICK_FROM_SEC(sec) ((sec) * 1000000)
+
+sc_tick
+sc_tick_now(void);
+
+#endif

app/src/v4l2_sink.c

@@ -112,7 +112,7 @@ run_v4l2_sink(void *data) {
     for (;;) {
         sc_mutex_lock(&vs->mutex);
 
-        while (!vs->stopped && vs->vb.pending_frame_consumed) {
+        while (!vs->stopped && !vs->has_frame) {
             sc_cond_wait(&vs->cond, &vs->mutex);
         }
 
@@ -121,9 +121,11 @@ run_v4l2_sink(void *data) {
             break;
         }
 
+        vs->has_frame = false;
         sc_mutex_unlock(&vs->mutex);
 
-        video_buffer_consume(&vs->vb, vs->frame);
+        sc_video_buffer_consume(&vs->vb, vs->frame);
+
         bool ok = encode_and_write_frame(vs, vs->frame);
         av_frame_unref(vs->frame);
         if (!ok) {
@@ -137,17 +139,42 @@ run_v4l2_sink(void *data) {
     return 0;
 }
 
+static void
+sc_video_buffer_on_new_frame(struct sc_video_buffer *vb, bool previous_skipped,
+                             void *userdata) {
+    (void) vb;
+    struct sc_v4l2_sink *vs = userdata;
+
+    if (!previous_skipped) {
+        sc_mutex_lock(&vs->mutex);
+        vs->has_frame = true;
+        sc_cond_signal(&vs->cond);
+        sc_mutex_unlock(&vs->mutex);
+    }
+}
+
 static bool
 sc_v4l2_sink_open(struct sc_v4l2_sink *vs) {
-    bool ok = video_buffer_init(&vs->vb);
+    static const struct sc_video_buffer_callbacks cbs = {
+        .on_new_frame = sc_video_buffer_on_new_frame,
+    };
+
+    bool ok = sc_video_buffer_init(&vs->vb, vs->buffering_time, &cbs, vs);
     if (!ok) {
+        LOGE("Could not initialize video buffer");
         return false;
     }
 
+    ok = sc_video_buffer_start(&vs->vb);
+    if (!ok) {
+        LOGE("Could not start video buffer");
+        goto error_video_buffer_destroy;
+    }
+
     ok = sc_mutex_init(&vs->mutex);
     if (!ok) {
         LOGC("Could not create mutex");
-        goto error_video_buffer_destroy;
+        goto error_video_buffer_stop_and_join;
     }
 
     ok = sc_cond_init(&vs->cond);
@@ -241,6 +268,10 @@ sc_v4l2_sink_open(struct sc_v4l2_sink *vs) {
         goto error_av_frame_free;
     }
 
+    vs->has_frame = false;
+    vs->header_written = false;
+    vs->stopped = false;
+
     LOGD("Starting v4l2 thread");
     ok = sc_thread_create(&vs->thread, run_v4l2_sink, "v4l2", vs);
     if (!ok) {
@@ -248,9 +279,6 @@ sc_v4l2_sink_open(struct sc_v4l2_sink *vs) {
         goto error_av_packet_free;
     }
 
-    vs->header_written = false;
-    vs->stopped = false;
-
     LOGI("v4l2 sink started to device: %s", vs->device_name);
 
     return true;
@@ -271,8 +299,11 @@ error_cond_destroy:
     sc_cond_destroy(&vs->cond);
 error_mutex_destroy:
     sc_mutex_destroy(&vs->mutex);
+error_video_buffer_stop_and_join:
+    sc_video_buffer_stop(&vs->vb);
+    sc_video_buffer_join(&vs->vb);
 error_video_buffer_destroy:
-    video_buffer_destroy(&vs->vb);
+    sc_video_buffer_destroy(&vs->vb);
 
     return false;
 }
@@ -284,7 +315,10 @@ sc_v4l2_sink_close(struct sc_v4l2_sink *vs) {
     sc_cond_signal(&vs->cond);
     sc_mutex_unlock(&vs->mutex);
 
+    sc_video_buffer_stop(&vs->vb);
+
     sc_thread_join(&vs->thread, NULL);
+    sc_video_buffer_join(&vs->vb);
 
     av_packet_free(&vs->packet);
     av_frame_free(&vs->frame);
@@ -294,20 +328,12 @@ sc_v4l2_sink_close(struct sc_v4l2_sink *vs) {
     avformat_free_context(vs->format_ctx);
     sc_cond_destroy(&vs->cond);
     sc_mutex_destroy(&vs->mutex);
-    video_buffer_destroy(&vs->vb);
+    sc_video_buffer_destroy(&vs->vb);
 }
 
 static bool
 sc_v4l2_sink_push(struct sc_v4l2_sink *vs, const AVFrame *frame) {
-    bool ok = video_buffer_push(&vs->vb, frame, NULL);
-    if (!ok) {
-        return false;
-    }
-
-    // signal possible change of vs->vb.pending_frame_consumed
-    sc_cond_signal(&vs->cond);
-
-    return true;
+    return sc_video_buffer_push(&vs->vb, frame);
 }
 
 static bool
@@ -330,7 +356,7 @@ sc_v4l2_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
 
 bool
 sc_v4l2_sink_init(struct sc_v4l2_sink *vs, const char *device_name,
-                  struct size frame_size) {
+                  struct size frame_size, sc_tick buffering_time) {
     vs->device_name = strdup(device_name);
     if (!vs->device_name) {
         LOGE("Could not strdup v4l2 device name");
@@ -338,6 +364,7 @@ sc_v4l2_sink_init(struct sc_v4l2_sink *vs, const char *device_name,
     }
 
     vs->frame_size = frame_size;
+    vs->buffering_time = buffering_time;
 
     static const struct sc_frame_sink_ops ops = {
         .open = sc_v4l2_frame_sink_open,

app/src/v4l2_sink.h

@@ -6,22 +6,25 @@
 #include "coords.h"
 #include "trait/frame_sink.h"
 #include "video_buffer.h"
+#include "util/tick.h"
 
 #include <libavformat/avformat.h>
 
 struct sc_v4l2_sink {
     struct sc_frame_sink frame_sink; // frame sink trait
 
-    struct video_buffer vb;
+    struct sc_video_buffer vb;
     AVFormatContext *format_ctx;
     AVCodecContext *encoder_ctx;
 
     char *device_name;
     struct size frame_size;
+    sc_tick buffering_time;
 
     sc_thread thread;
     sc_mutex mutex;
     sc_cond cond;
+    bool has_frame;
     bool stopped;
     bool header_written;
 
@@ -31,7 +34,7 @@ struct sc_v4l2_sink {
 
 bool
 sc_v4l2_sink_init(struct sc_v4l2_sink *vs, const char *device_name,
-                  struct size frame_size);
+                  struct size frame_size, sc_tick buffering_time);
 
 void
 sc_v4l2_sink_destroy(struct sc_v4l2_sink *vs);

app/src/video_buffer.c

@@ -1,88 +1,255 @@
 #include "video_buffer.h"
 
 #include <assert.h>
+#include <stdlib.h>
+
 #include <libavutil/avutil.h>
 #include <libavformat/avformat.h>
 
 #include "util/log.h"
 
-bool
-video_buffer_init(struct video_buffer *vb) {
-    vb->pending_frame = av_frame_alloc();
-    if (!vb->pending_frame) {
-        return false;
+#define SC_BUFFERING_NDEBUG // comment to debug
+
+static struct sc_video_buffer_frame *
+sc_video_buffer_frame_new(const AVFrame *frame) {
+    struct sc_video_buffer_frame *vb_frame = malloc(sizeof(*vb_frame));
+    if (!vb_frame) {
+        return NULL;
     }
 
-    vb->tmp_frame = av_frame_alloc();
-    if (!vb->tmp_frame) {
-        av_frame_free(&vb->pending_frame);
-        return false;
+    vb_frame->frame = av_frame_alloc();
+    if (!vb_frame->frame) {
+        free(vb_frame);
+        return NULL;
     }
 
-    bool ok = sc_mutex_init(&vb->mutex);
+    if (av_frame_ref(vb_frame->frame, frame)) {
+        av_frame_free(&vb_frame->frame);
+        free(vb_frame);
+        return NULL;
+    }
+
+    return vb_frame;
+}
+
+static void
+sc_video_buffer_frame_delete(struct sc_video_buffer_frame *vb_frame) {
+    av_frame_unref(vb_frame->frame);
+    av_frame_free(&vb_frame->frame);
+    free(vb_frame);
+}
+
+static bool
+sc_video_buffer_offer(struct sc_video_buffer *vb, const AVFrame *frame) {
+    bool previous_skipped;
+    bool ok = sc_frame_buffer_push(&vb->fb, frame, &previous_skipped);
     if (!ok) {
-        av_frame_free(&vb->pending_frame);
-        av_frame_free(&vb->tmp_frame);
         return false;
     }
 
-    // there is initially no frame, so consider it has already been consumed
-    vb->pending_frame_consumed = true;
-
+    vb->cbs->on_new_frame(vb, previous_skipped, vb->cbs_userdata);
     return true;
 }
 
-void
-video_buffer_destroy(struct video_buffer *vb) {
-    sc_mutex_destroy(&vb->mutex);
-    av_frame_free(&vb->pending_frame);
-    av_frame_free(&vb->tmp_frame);
-}
+static int
+run_buffering(void *data) {
+    struct sc_video_buffer *vb = data;
 
-static inline void
-swap_frames(AVFrame **lhs, AVFrame **rhs) {
-    AVFrame *tmp = *lhs;
-    *lhs = *rhs;
-    *rhs = tmp;
+    assert(vb->buffering_time > 0);
+
+    for (;;) {
+        sc_mutex_lock(&vb->b.mutex);
+
+        while (!vb->b.stopped && sc_queue_is_empty(&vb->b.queue)) {
+            sc_cond_wait(&vb->b.queue_cond, &vb->b.mutex);
+        }
+
+        if (vb->b.stopped) {
+            sc_mutex_unlock(&vb->b.mutex);
+            goto stopped;
+        }
+
+        struct sc_video_buffer_frame *vb_frame;
+        sc_queue_take(&vb->b.queue, next, &vb_frame);
+
+        sc_tick max_deadline = sc_tick_now() + vb->buffering_time;
+        // PTS (written by the server) are expressed in microseconds
+        sc_tick pts = SC_TICK_TO_US(vb_frame->frame->pts);
+
+        bool timed_out = false;
+        while (!vb->b.stopped && !timed_out) {
+            sc_tick deadline = sc_clock_to_system_time(&vb->b.clock, pts)
+                             + vb->buffering_time;
+            if (deadline > max_deadline) {
+                deadline = max_deadline;
+            }
+
+            timed_out =
+                !sc_cond_timedwait(&vb->b.wait_cond, &vb->b.mutex, deadline);
+        }
+
+        if (vb->b.stopped) {
+            sc_video_buffer_frame_delete(vb_frame);
+            sc_mutex_unlock(&vb->b.mutex);
+            goto stopped;
+        }
+
+        sc_mutex_unlock(&vb->b.mutex);
+
+#ifndef SC_BUFFERING_NDEBUG
+        LOGD("Buffering: %" PRItick ";%" PRItick ";%" PRItick,
+             pts, vb_frame->push_date, sc_tick_now());
+#endif
+
+        sc_video_buffer_offer(vb, vb_frame->frame);
+
+        sc_video_buffer_frame_delete(vb_frame);
+    }
+
+stopped:
+    // Flush queue
+    while (!sc_queue_is_empty(&vb->b.queue)) {
+        struct sc_video_buffer_frame *vb_frame;
+        sc_queue_take(&vb->b.queue, next, &vb_frame);
+        sc_video_buffer_frame_delete(vb_frame);
+    }
+
+    LOGD("Buffering thread ended");
+
+    return 0;
 }
 
 bool
-video_buffer_push(struct video_buffer *vb, const AVFrame *frame,
-                  bool *previous_frame_skipped) {
-    sc_mutex_lock(&vb->mutex);
-
-    // Use a temporary frame to preserve pending_frame in case of error.
-    // tmp_frame is an empty frame, no need to call av_frame_unref() beforehand.
-    int r = av_frame_ref(vb->tmp_frame, frame);
-    if (r) {
-        LOGE("Could not ref frame: %d", r);
+sc_video_buffer_init(struct sc_video_buffer *vb, sc_tick buffering_time,
+                     const struct sc_video_buffer_callbacks *cbs,
+                     void *cbs_userdata) {
+    bool ok = sc_frame_buffer_init(&vb->fb);
+    if (!ok) {
         return false;
     }
 
-    // Now that av_frame_ref() succeeded, we can replace the previous
-    // pending_frame
-    swap_frames(&vb->pending_frame, &vb->tmp_frame);
-    av_frame_unref(vb->tmp_frame);
+    assert(buffering_time >= 0);
+    if (buffering_time) {
+        ok = sc_mutex_init(&vb->b.mutex);
+        if (!ok) {
+            LOGC("Could not create mutex");
+            sc_frame_buffer_destroy(&vb->fb);
+            return false;
+        }
 
-    if (previous_frame_skipped) {
-        *previous_frame_skipped = !vb->pending_frame_consumed;
+        ok = sc_cond_init(&vb->b.queue_cond);
+        if (!ok) {
+            LOGC("Could not create cond");
+            sc_mutex_destroy(&vb->b.mutex);
+            sc_frame_buffer_destroy(&vb->fb);
+            return false;
+        }
+
+        ok = sc_cond_init(&vb->b.wait_cond);
+        if (!ok) {
+            LOGC("Could not create wait cond");
+            sc_cond_destroy(&vb->b.queue_cond);
+            sc_mutex_destroy(&vb->b.mutex);
+            sc_frame_buffer_destroy(&vb->fb);
+            return false;
+        }
+
+        sc_clock_init(&vb->b.clock);
+        sc_queue_init(&vb->b.queue);
     }
-    vb->pending_frame_consumed = false;
 
-    sc_mutex_unlock(&vb->mutex);
+    assert(cbs);
+    assert(cbs->on_new_frame);
+
+    vb->buffering_time = buffering_time;
+    vb->cbs = cbs;
+    vb->cbs_userdata = cbs_userdata;
+    return true;
+}
+
+bool
+sc_video_buffer_start(struct sc_video_buffer *vb) {
+    if (vb->buffering_time) {
+        bool ok =
+            sc_thread_create(&vb->b.thread, run_buffering, "buffering", vb);
+        if (!ok) {
+            LOGE("Could not start buffering thread");
+            return false;
+        }
+    }
 
     return true;
 }
 
 void
-video_buffer_consume(struct video_buffer *vb, AVFrame *dst) {
-    sc_mutex_lock(&vb->mutex);
-    assert(!vb->pending_frame_consumed);
-    vb->pending_frame_consumed = true;
-
-    av_frame_move_ref(dst, vb->pending_frame);
-    // av_frame_move_ref() resets its source frame, so no need to call
-    // av_frame_unref()
-
-    sc_mutex_unlock(&vb->mutex);
+sc_video_buffer_stop(struct sc_video_buffer *vb) {
+    if (vb->buffering_time) {
+        sc_mutex_lock(&vb->b.mutex);
+        vb->b.stopped = true;
+        sc_cond_signal(&vb->b.queue_cond);
+        sc_cond_signal(&vb->b.wait_cond);
+        sc_mutex_unlock(&vb->b.mutex);
+    }
+}
+
+void
+sc_video_buffer_join(struct sc_video_buffer *vb) {
+    if (vb->buffering_time) {
+        sc_thread_join(&vb->b.thread, NULL);
+    }
+}
+
+void
+sc_video_buffer_destroy(struct sc_video_buffer *vb) {
+    sc_frame_buffer_destroy(&vb->fb);
+    if (vb->buffering_time) {
+        sc_cond_destroy(&vb->b.wait_cond);
+        sc_cond_destroy(&vb->b.queue_cond);
+        sc_mutex_destroy(&vb->b.mutex);
+    }
+}
+
+bool
+sc_video_buffer_push(struct sc_video_buffer *vb, const AVFrame *frame) {
+    if (!vb->buffering_time) {
+        // No buffering
+        return sc_video_buffer_offer(vb, frame);
+    }
+
+    sc_mutex_lock(&vb->b.mutex);
+
+    sc_tick pts = SC_TICK_FROM_US(frame->pts);
+    sc_clock_update(&vb->b.clock, sc_tick_now(), pts);
+    sc_cond_signal(&vb->b.wait_cond);
+
+    if (vb->b.clock.count == 1) {
+        sc_mutex_unlock(&vb->b.mutex);
+        // First frame, offer it immediately, for two reasons:
+        //  - not to delay the opening of the scrcpy window
+        //  - the buffering estimation needs at least two clock points, so it
+        //  could not handle the first frame
+        return sc_video_buffer_offer(vb, frame);
+    }
+
+    struct sc_video_buffer_frame *vb_frame = sc_video_buffer_frame_new(frame);
+    if (!vb_frame) {
+        sc_mutex_unlock(&vb->b.mutex);
+        LOGE("Could not allocate frame");
+        return false;
+    }
+
+#ifndef SC_BUFFERING_NDEBUG
+    vb_frame->push_date = sc_tick_now();
+#endif
+    sc_queue_push(&vb->b.queue, next, vb_frame);
+    sc_cond_signal(&vb->b.queue_cond);
+
+    sc_mutex_unlock(&vb->b.mutex);
+
+    return true;
+}
+
+void
+sc_video_buffer_consume(struct sc_video_buffer *vb, AVFrame *dst) {
+    sc_frame_buffer_consume(&vb->fb, dst);
 }

app/src/video_buffer.h

@@ -1,50 +1,76 @@
-#ifndef VIDEO_BUFFER_H
-#define VIDEO_BUFFER_H
+#ifndef SC_VIDEO_BUFFER_H
+#define SC_VIDEO_BUFFER_H
 
 #include "common.h"
 
 #include <stdbool.h>
 
-#include "fps_counter.h"
+#include "clock.h"
+#include "frame_buffer.h"
+#include "util/queue.h"
 #include "util/thread.h"
+#include "util/tick.h"
 
 // forward declarations
 typedef struct AVFrame AVFrame;
 
-/**
- * A video buffer holds 1 pending frame, which is the last frame received from
- * the producer (typically, the decoder).
- *
- * If a pending frame has not been consumed when the producer pushes a new
- * frame, then it is lost. The intent is to always provide access to the very
- * last frame to minimize latency.
- *
- * The producer and the consumer typically do not live in the same thread.
- * That's the reason why the callback on_frame_available() does not provide the
- * frame as parameter: the consumer might post an event to its own thread to
- * retrieve the pending frame from there, and that frame may have changed since
- * the callback if producer pushed a new one in between.
- */
+struct sc_video_buffer_frame {
+    AVFrame *frame;
+    struct sc_video_buffer_frame *next;
+#ifndef NDEBUG
+    sc_tick push_date;
+#endif
+};
 
-struct video_buffer {
-    AVFrame *pending_frame;
-    AVFrame *tmp_frame; // To preserve the pending frame on error
+struct sc_video_buffer_frame_queue SC_QUEUE(struct sc_video_buffer_frame);
 
-    sc_mutex mutex;
+struct sc_video_buffer {
+    struct sc_frame_buffer fb;
 
-    bool pending_frame_consumed;
+    sc_tick buffering_time;
+
+    // only if buffering_time > 0
+    struct {
+        sc_thread thread;
+        sc_mutex mutex;
+        sc_cond queue_cond;
+        sc_cond wait_cond;
+
+        struct sc_clock clock;
+        struct sc_video_buffer_frame_queue queue;
+        bool stopped;
+    } b; // buffering
+
+    const struct sc_video_buffer_callbacks *cbs;
+    void *cbs_userdata;
+};
+
+struct sc_video_buffer_callbacks {
+    void (*on_new_frame)(struct sc_video_buffer *vb, bool previous_skipped,
+                         void *userdata);
 };
 
 bool
-video_buffer_init(struct video_buffer *vb);
-
-void
-video_buffer_destroy(struct video_buffer *vb);
+sc_video_buffer_init(struct sc_video_buffer *vb, sc_tick buffering_time,
+                     const struct sc_video_buffer_callbacks *cbs,
+                     void *cbs_userdata);
 
 bool
-video_buffer_push(struct video_buffer *vb, const AVFrame *frame, bool *skipped);
+sc_video_buffer_start(struct sc_video_buffer *vb);
 
 void
-video_buffer_consume(struct video_buffer *vb, AVFrame *dst);
+sc_video_buffer_stop(struct sc_video_buffer *vb);
+
+void
+sc_video_buffer_join(struct sc_video_buffer *vb);
+
+void
+sc_video_buffer_destroy(struct sc_video_buffer *vb);
+
+bool
+sc_video_buffer_push(struct sc_video_buffer *vb, const AVFrame *frame);
+
+void
+sc_video_buffer_consume(struct sc_video_buffer *vb, AVFrame *dst);
 
 #endif

app/tests/test_clock.c

@@ -0,0 +1,79 @@
+#include "common.h"
+
+#include <assert.h>
+
+#include "clock.h"
+
+void test_small_rolling_sum(void) {
+    struct sc_clock clock;
+    sc_clock_init(&clock);
+
+    assert(clock.count == 0);
+    assert(clock.left_sum.system == 0);
+    assert(clock.left_sum.stream == 0);
+    assert(clock.right_sum.system == 0);
+    assert(clock.right_sum.stream == 0);
+
+    sc_clock_update(&clock, 2, 3);
+    assert(clock.count == 1);
+    assert(clock.left_sum.system == 0);
+    assert(clock.left_sum.stream == 0);
+    assert(clock.right_sum.system == 2);
+    assert(clock.right_sum.stream == 3);
+
+    sc_clock_update(&clock, 10, 20);
+    assert(clock.count == 2);
+    assert(clock.left_sum.system == 2);
+    assert(clock.left_sum.stream == 3);
+    assert(clock.right_sum.system == 10);
+    assert(clock.right_sum.stream == 20);
+
+    sc_clock_update(&clock, 40, 80);
+    assert(clock.count == 3);
+    assert(clock.left_sum.system == 2);
+    assert(clock.left_sum.stream == 3);
+    assert(clock.right_sum.system == 50);
+    assert(clock.right_sum.stream == 100);
+
+    sc_clock_update(&clock, 400, 800);
+    assert(clock.count == 4);
+    assert(clock.left_sum.system == 12);
+    assert(clock.left_sum.stream == 23);
+    assert(clock.right_sum.system == 440);
+    assert(clock.right_sum.stream == 880);
+}
+
+void test_large_rolling_sum(void) {
+    const unsigned half_range = SC_CLOCK_RANGE / 2;
+
+    struct sc_clock clock1;
+    sc_clock_init(&clock1);
+    for (unsigned i = 0; i < 5 * half_range; ++i) {
+        sc_clock_update(&clock1, i, 2 * i + 1);
+    }
+
+    struct sc_clock clock2;
+    sc_clock_init(&clock2);
+    for (unsigned i = 3 * half_range; i < 5 * half_range; ++i) {
+        sc_clock_update(&clock2, i, 2 * i + 1);
+    }
+
+    assert(clock1.count == SC_CLOCK_RANGE);
+    assert(clock2.count == SC_CLOCK_RANGE);
+
+    // The values before the last SC_CLOCK_RANGE points in clock1 should have
+    // no impact
+    assert(clock1.left_sum.system == clock2.left_sum.system);
+    assert(clock1.left_sum.stream == clock2.left_sum.stream);
+    assert(clock1.right_sum.system == clock2.right_sum.system);
+    assert(clock1.right_sum.stream == clock2.right_sum.stream);
+}
+
+int main(int argc, char *argv[]) {
+    (void) argc;
+    (void) argv;
+
+    test_small_rolling_sum();
+    test_large_rolling_sum();
+    return 0;
+};

app/tests/test_queue.c

@@ -10,28 +10,28 @@ struct foo {
 };
 
 static void test_queue(void) {
-    struct my_queue QUEUE(struct foo) queue;
-    queue_init(&queue);
+    struct my_queue SC_QUEUE(struct foo) queue;
+    sc_queue_init(&queue);
 
-    assert(queue_is_empty(&queue));
+    assert(sc_queue_is_empty(&queue));
 
     struct foo v1 = { .value = 42 };
     struct foo v2 = { .value = 27 };
 
-    queue_push(&queue, next, &v1);
-    queue_push(&queue, next, &v2);
+    sc_queue_push(&queue, next, &v1);
+    sc_queue_push(&queue, next, &v2);
 
     struct foo *foo;
 
-    assert(!queue_is_empty(&queue));
-    queue_take(&queue, next, &foo);
+    assert(!sc_queue_is_empty(&queue));
+    sc_queue_take(&queue, next, &foo);
     assert(foo->value == 42);
 
-    assert(!queue_is_empty(&queue));
-    queue_take(&queue, next, &foo);
+    assert(!sc_queue_is_empty(&queue));
+    sc_queue_take(&queue, next, &foo);
     assert(foo->value == 27);
 
-    assert(queue_is_empty(&queue));
+    assert(sc_queue_is_empty(&queue));
 }
 
 int main(int argc, char *argv[]) {

cross_win32.txt

@@ -17,4 +17,4 @@ endian = 'little'
 [properties]
 prebuilt_ffmpeg_shared = 'ffmpeg-4.3.1-win32-shared'
 prebuilt_ffmpeg_dev = 'ffmpeg-4.3.1-win32-dev'
-prebuilt_sdl2 = 'SDL2-2.0.14/i686-w64-mingw32'
+prebuilt_sdl2 = 'SDL2-2.0.16/i686-w64-mingw32'

cross_win64.txt

@@ -17,4 +17,4 @@ endian = 'little'
 [properties]
 prebuilt_ffmpeg_shared = 'ffmpeg-4.3.1-win64-shared'
 prebuilt_ffmpeg_dev = 'ffmpeg-4.3.1-win64-dev'
-prebuilt_sdl2 = 'SDL2-2.0.14/x86_64-w64-mingw32'
+prebuilt_sdl2 = 'SDL2-2.0.16/x86_64-w64-mingw32'

install_release.sh

@@ -2,8 +2,8 @@
 set -e
 
 BUILDDIR=build-auto
-PREBUILT_SERVER_URL=https://github.com/Genymobile/scrcpy/releases/download/v1.18/scrcpy-server-v1.18
-PREBUILT_SERVER_SHA256=641c5c6beda9399dfae72d116f5ff43b5ed1059d871c9ebc3f47610fd33c51a3
+PREBUILT_SERVER_URL=https://github.com/Genymobile/scrcpy/releases/download/v1.19/scrcpy-server-v1.19
+PREBUILT_SERVER_SHA256=876f9322182e6aac6a58db1334f4225855ef3a17eaebc80aab6601d9d1ecb867
 
 echo "[scrcpy] Downloading prebuilt server..."
 wget "$PREBUILT_SERVER_URL" -O scrcpy-server

meson.build

@@ -1,5 +1,5 @@
 project('scrcpy', 'c',
-        version: '1.18',
+        version: '1.19',
         meson_version: '>= 0.48',
         default_options: [
             'c_std=c11',

prebuilt-deps/Makefile

@@ -30,11 +30,11 @@ prepare-ffmpeg-dev-win64:
 		ffmpeg-4.3.1-win64-dev
 
 prepare-sdl2:
-	@./prepare-dep https://libsdl.org/release/SDL2-devel-2.0.14-mingw.tar.gz \
-		405eaff3eb18f2e08fe669ef9e63bc9a8710b7d343756f238619761e9b60407d \
-		SDL2-2.0.14
+	@./prepare-dep https://libsdl.org/release/SDL2-devel-2.0.16-mingw.tar.gz \
+		2bfe48628aa9635c12eac7d421907e291525de1d0b04b3bca4a5bd6e6c881a6f \
+		SDL2-2.0.16
 
 prepare-adb:
-	@./prepare-dep https://dl.google.com/android/repository/platform-tools_r31.0.2-windows.zip \
-		d560cb8ded83ae04763b94632673481f14843a5969256569623cfeac82db4ba5 \
+	@./prepare-dep https://dl.google.com/android/repository/platform-tools_r31.0.3-windows.zip \
+		0f4b8fdd26af2c3733539d6eebb3c2ed499ea1d4bb1f4e0ecc2d6016961a6e24 \
 		platform-tools

release.mk

@@ -102,7 +102,7 @@ dist-win32: build-server build-win32
 	cp prebuilt-deps/platform-tools/adb.exe "$(DIST)/$(WIN32_TARGET_DIR)/"
 	cp prebuilt-deps/platform-tools/AdbWinApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
 	cp prebuilt-deps/platform-tools/AdbWinUsbApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
-	cp prebuilt-deps/SDL2-2.0.14/i686-w64-mingw32/bin/SDL2.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
+	cp prebuilt-deps/SDL2-2.0.16/i686-w64-mingw32/bin/SDL2.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
 
 dist-win64: build-server build-win64
 	mkdir -p "$(DIST)/$(WIN64_TARGET_DIR)"
@@ -118,7 +118,7 @@ dist-win64: build-server build-win64
 	cp prebuilt-deps/platform-tools/adb.exe "$(DIST)/$(WIN64_TARGET_DIR)/"
 	cp prebuilt-deps/platform-tools/AdbWinApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
 	cp prebuilt-deps/platform-tools/AdbWinUsbApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
-	cp prebuilt-deps/SDL2-2.0.14/x86_64-w64-mingw32/bin/SDL2.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
+	cp prebuilt-deps/SDL2-2.0.16/x86_64-w64-mingw32/bin/SDL2.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
 
 zip-win32: dist-win32
 	cd "$(DIST)/$(WIN32_TARGET_DIR)"; \

server/build.gradle

@@ -6,8 +6,8 @@ android {
         applicationId "com.genymobile.scrcpy"
         minSdkVersion 21
         targetSdkVersion 30
-        versionCode 11800
-        versionName "1.18"
+        versionCode 11900
+        versionName "1.19"
         testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
     }
     buildTypes {

server/build_without_gradle.sh

@@ -12,7 +12,7 @@
 set -e
 
 SCRCPY_DEBUG=false
-SCRCPY_VERSION_NAME=1.18
+SCRCPY_VERSION_NAME=1.19
 
 PLATFORM=${ANDROID_PLATFORM:-30}
 BUILD_TOOLS=${ANDROID_BUILD_TOOLS:-30.0.0}

server/src/main/java/com/genymobile/scrcpy/Controller.java

@@ -241,7 +241,7 @@ public class Controller {
 
         MotionEvent event = MotionEvent
                 .obtain(lastTouchDown, now, MotionEvent.ACTION_SCROLL, 1, pointerProperties, pointerCoords, 0, 0, 1f, 1f, DEFAULT_DEVICE_ID, 0,
-                        InputDevice.SOURCE_TOUCHSCREEN, 0);
+                        InputDevice.SOURCE_MOUSE, 0);
         return device.injectEvent(event);
     }
 

server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java

@@ -56,17 +56,13 @@ public class ScreenEncoder implements Device.RotationListener {
 
     public void streamScreen(Device device, FileDescriptor fd) throws IOException {
         Workarounds.prepareMainLooper();
-
-        try {
-            internalStreamScreen(device, fd);
-        } catch (NullPointerException e) {
-            // Retry with workarounds enabled:
-            // <https://github.com/Genymobile/scrcpy/issues/365>
-            // <https://github.com/Genymobile/scrcpy/issues/940>
-            Ln.d("Applying workarounds to avoid NullPointerException");
+        if (Build.BRAND.equalsIgnoreCase("meizu")) {
+            // <https://github.com/Genymobile/scrcpy/issues/240>
+            // <https://github.com/Genymobile/scrcpy/issues/2656>
             Workarounds.fillAppInfo();
-            internalStreamScreen(device, fd);
         }
+
+        internalStreamScreen(device, fd);
     }
 
     private void internalStreamScreen(Device device, FileDescriptor fd) throws IOException {

server/src/main/java/com/genymobile/scrcpy/wrappers/StatusBarManager.java

@@ -11,6 +11,7 @@ public class StatusBarManager {
 
     private final IInterface manager;
     private Method expandNotificationsPanelMethod;
+    private boolean expandNotificationPanelMethodCustomVersion;
     private Method expandSettingsPanelMethod;
     private boolean expandSettingsPanelMethodNewVersion = true;
     private Method collapsePanelsMethod;
@@ -21,7 +22,13 @@ public class StatusBarManager {
 
     private Method getExpandNotificationsPanelMethod() throws NoSuchMethodException {
         if (expandNotificationsPanelMethod == null) {
-            expandNotificationsPanelMethod = manager.getClass().getMethod("expandNotificationsPanel");
+            try {
+                expandNotificationsPanelMethod = manager.getClass().getMethod("expandNotificationsPanel");
+            } catch (NoSuchMethodException e) {
+                // Custom version for custom vendor ROM: <https://github.com/Genymobile/scrcpy/issues/2551>
+                expandNotificationsPanelMethod = manager.getClass().getMethod("expandNotificationsPanel", int.class);
+                expandNotificationPanelMethodCustomVersion = true;
+            }
         }
         return expandNotificationsPanelMethod;
     }
@@ -50,7 +57,11 @@ public class StatusBarManager {
     public void expandNotificationsPanel() {
         try {
             Method method = getExpandNotificationsPanelMethod();
-            method.invoke(manager);
+            if (expandNotificationPanelMethodCustomVersion) {
+                method.invoke(manager, 0);
+            } else {
+                method.invoke(manager);
+            }
         } catch (InvocationTargetException | IllegalAccessException | NoSuchMethodException e) {
             Ln.e("Could not invoke method", e);
         }