Compare commits

...

22 Commits

Author SHA1 Message Date
2dab1f7024 Bump version to 2.1 2023-06-22 01:15:44 +02:00
744312ec64 Merge branch 'master' into release 2023-06-22 01:15:39 +02:00
b9315620e2 Fix adb forward initialization
In forward mode, the dummy byte must be written immediately after the
first accept(), otherwise the client will wait indefinitely, causing a
deadlock (or a timeout).

Regression introduced by 8c650e53cd.
2023-06-22 01:13:53 +02:00
ea59d525bd Fix code style
The code should fit in 80 columns.
2023-06-22 01:07:09 +02:00
0ffcfa0f5c Accept failure in rotation or fold registration
Do not make scrcpy fail if rotation or display fold listeners could not
be registered.
2023-06-22 00:52:54 +02:00
c0f3c080b6 Register DisplayFoldListener only for Android 10+
This listener does not exist on Android < 10, and it makes scrcpy fail.
2023-06-22 00:49:11 +02:00
d046678f85 Upgrade platform-tools (34.0.3) for Windows
Include the latest version of adb in Windows releases.
2023-06-22 00:10:37 +02:00
fae3fbc934 Update developer documentation 2023-06-22 00:03:26 +02:00
5061b7e02c Fix build without gradle
Add missing class generation from IDisplayFoldListener.aidl.

Refs 24999d0d32
2023-06-22 00:03:26 +02:00
4ad7479425 Add missing shortcut in documentation
MOD+Backspace also triggers BACK.
2023-06-08 08:52:21 +02:00
958f22490b Document installation via winget on Windows
PR #4005 <https://github.com/Genymobile/scrcpy/pull/4005>
Refs #1444 <https://github.com/Genymobile/scrcpy/issues/1444>
Refs #3932 <https://github.com/Genymobile/scrcpy/issues/3932>

Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-05-17 19:36:39 +02:00
cb20bcb16f Clarify API versions that support Audio Forwarding
Reword the supported API versions for audio forwarding sentence to
clarify that it supports API >= 30

PR #3949 <https://github.com/Genymobile/scrcpy/pull/3949>

Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-04-26 19:33:28 +02:00
8f0b38cc4f Specify in README that OTG does not require adb 2023-03-31 07:55:13 +02:00
a1e8a34001 Fix documentation link in FAQ 2023-03-28 08:32:07 +02:00
00534b0b2d Fix typo in FAQ 2023-03-28 08:31:01 +02:00
21df2c240e Mention necessary reboot
After setting "USB debugging (security settings)", a reboot is
necessary.
2023-03-23 19:06:44 +01:00
2d3059e1ab Reference FAQ from HID/OTG documentation
Reference the FAQ section about "HID/OTG issues on Windows" from the
HID/OTG documentation.
2023-03-20 19:42:23 +01:00
478aece68f Replace "bit-rate" with "bit rate" 2023-03-20 08:35:13 +01:00
55899c091e Fix typo in doc/audio.md
The documentation is about audio bit rate, not video bit rate.

PR #3839 <https://github.com/Genymobile/scrcpy/pull/3839>

Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-20 08:32:58 +01:00
d9a644df9c Clarify V4L2 feature in README
The previous formulation could suggest that the device camera could be
used as a webcam. This is not the case (yet?).
2023-03-15 10:46:36 +01:00
45717733a1 Document missing Opus encoder error
And how to solve it.
2023-03-15 00:41:07 +01:00
6ad037ea04 Update Gentoo instructions
scrcpy is available directly in the distro, drop link to the overlay
(which only contains older versions).

PR #3816 <https://github.com/Genymobile/scrcpy/pull/3816>

Signed-off-by: Romain Vimont <rom@rom1v.com>
2023-03-14 22:03:21 +01:00
21 changed files with 403 additions and 195 deletions

8
FAQ.md
View File

@ -159,6 +159,8 @@ In developer options, enable:
> **USB debugging (Security settings)**
> _Allow granting permissions and simulating input via USB debugging_
Rebooting the device is necessary once this option is set.
[simulating input]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
@ -168,12 +170,12 @@ The default text injection method is [limited to ASCII characters][text-input].
A trick allows to also inject some [accented characters][accented-characters],
but that's all. See [#37].
Since scrcpy v1.20, it is possible to simulate a [physical keyboard][hid] (HID).
It is also possible to simulate a [physical keyboard][hid] (HID).
[text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
[accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
[#37]: https://github.com/Genymobile/scrcpy/issues/37
[hid]: README.md#physical-keyboard-simulation-hid
[hid]: doc/hid-otg.md
## Client issues
@ -229,4 +231,4 @@ Translations of this FAQ in other languages are available in the [wiki].
[wiki]: https://github.com/Genymobile/scrcpy/wiki
Only this README file is guaranteed to be up-to-date.
Only this FAQ file is guaranteed to be up-to-date.

View File

@ -30,7 +30,7 @@ Its features include:
- mirroring with [Android device screen off](doc/device.md#turn-screen-off)
- [copy-paste](doc/control.md#copy-paste) in both directions
- [configurable quality](doc/video.md)
- Android device [as a webcam (V4L2)](doc/v4l2.md) (Linux-only)
- Android device screen [as a webcam (V4L2)](doc/v4l2.md) (Linux-only)
- [physical keyboard/mouse simulation (HID)](doc/hid-otg.md)
- [OTG mode](doc/hid-otg.md#otg)
- and more…
@ -39,7 +39,7 @@ Its features include:
The Android device requires at least API 21 (Android 5.0).
[Audio forwarding](doc/audio.md) is supported from API 30 (Android 11).
[Audio forwarding](doc/audio.md) is supported for API >= 30 (Android 11+).
Make sure you [enabled USB debugging][enable-adb] on your device(s).
@ -47,10 +47,14 @@ Make sure you [enabled USB debugging][enable-adb] on your device(s).
On some devices, you also need to enable [an additional option][control] `USB
debugging (Security Settings)` (this is an item different from `USB debugging`)
to control it using a keyboard and mouse.
to control it using a keyboard and mouse. Rebooting the device is necessary once
this option is set.
[control]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
Note that USB debugging is not required to run scrcpy in [OTG
mode](doc/hid-otg.md#otg).
## Get the app

View File

@ -6,10 +6,10 @@ cd "$DIR"
mkdir -p "$PREBUILT_DATA_DIR"
cd "$PREBUILT_DATA_DIR"
DEP_DIR=platform-tools-34.0.1
DEP_DIR=platform-tools-34.0.3
FILENAME=platform-tools_r34.0.1-windows.zip
SHA256SUM=5dd9c2be744c224fa3a7cbe30ba02d2cb378c763bd0f797a7e47e9f3156a5daa
FILENAME=platform-tools_r34.0.3-windows.zip
SHA256SUM=fce992e93eb786fc9f47df93d83a7b912c46742d45c39d712c02e06d05b72e2b
if [[ -d "$DEP_DIR" ]]
then

View File

@ -13,7 +13,7 @@ BEGIN
VALUE "LegalCopyright", "Romain Vimont, Genymobile"
VALUE "OriginalFilename", "scrcpy.exe"
VALUE "ProductName", "scrcpy"
VALUE "ProductVersion", "2.0"
VALUE "ProductVersion", "2.1"
END
END
BLOCK "VarFileInfo"

View File

@ -21,7 +21,7 @@ Make scrcpy window always on top (above other windows).
.TP
.BI "\-\-audio\-bit\-rate " value
Encode the audio at the given bit\-rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
Encode the audio at the given bit rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
Default is 128K (128000).
@ -71,7 +71,7 @@ Default is 5.
.TP
.BI "\-b, \-\-video\-bit\-rate " value
Encode the video at the given bit\-rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
Encode the video at the given bit rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
Default is 8M (8000000).

View File

@ -124,7 +124,7 @@ static const struct sc_option options[] = {
.longopt_id = OPT_AUDIO_BIT_RATE,
.longopt = "audio-bit-rate",
.argdesc = "value",
.text = "Encode the audio at the given bit-rate, expressed in bits/s. "
.text = "Encode the audio at the given bit rate, expressed in bits/s. "
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is 128K (128000).",
},
@ -185,7 +185,7 @@ static const struct sc_option options[] = {
.shortopt = 'b',
.longopt = "video-bit-rate",
.argdesc = "value",
.text = "Encode the video at the given bit-rate, expressed in bits/s. "
.text = "Encode the video at the given bit rate, expressed in bits/s. "
"Unit suffixes are supported: 'K' (x1000) and 'M' (x1000000).\n"
"Default is 8M (8000000).",
},

View File

@ -533,8 +533,8 @@ sc_server_connect_to(struct sc_server *server, struct sc_server_info *info) {
if (audio_socket == SC_SOCKET_NONE) {
goto fail;
}
bool ok = net_connect_intr(&server->intr, audio_socket, tunnel_host,
tunnel_port);
bool ok = net_connect_intr(&server->intr, audio_socket,
tunnel_host, tunnel_port);
if (!ok) {
goto fail;
}

View File

@ -71,6 +71,13 @@ scrcpy --audio-codec=aac
scrcpy --audio-codec=raw
```
In particular, if you get the following error:
> Failed to initialize audio/opus, error 0xfffffffe
then your device has no Opus encoder: try `scrcpy --audio-codec=aac`.
Several encoders may be available on the device. They can be listed by:
```bash
@ -91,7 +98,7 @@ check `--audio-codec-options` in the manpage or in `scrcpy --help`.
## Bit rate
The default video bit-rate is 128Kbps. To change it:
The default audio bit rate is 128Kbps. To change it:
```bash
scrcpy --audio-bit-rate=64K

View File

@ -9,16 +9,52 @@ This application is composed of two parts:
The client is responsible to push the server to the device and start its
execution.
Once the client and the server are connected to each other, the server initially
sends device information (name and initial screen dimensions), then starts to
send a raw H.264 video stream of the device screen. The client decodes the video
frames, and display them as soon as possible, without buffering, to minimize
latency. The client is not aware of the device rotation (which is handled by the
server), it just knows the dimensions of the video frames.
The client and the server establish communication using separate sockets for
video, audio and controls. Any of them may be disabled (but not all), so
there are 1, 2 or 3 socket(s).
The client captures relevant keyboard and mouse events, that it transmits to the
server, which injects them to the device.
The server initially sends the device name on the first socket (it is used for
the scrcpy window title), then each socket is used for its own purpose. All
reads and writes are performed from a dedicated thread for each socket, both on
the client and on the server.
If video is enabled, then the server sends a raw video stream (H.264 by default)
of the device screen, with some additional headers for each packet. The client
decodes the video frames, and displays them as soon as possible, without
buffering (unless `--display-buffer=delay` is specified) to minimize latency.
The client is not aware of the device rotation (which is handled by the server),
it just knows the dimensions of the video frames it receives.
Similarly, if audio is enabled, then the server sends a raw audio stream (OPUS
by default) of the device audio output (or the microphone if
`--audio-source=mic` is specified), with some additional headers for each
packet. The client decodes the stream, attempts to keep a minimal latency by
maintaining an average buffering. The [blog post][scrcpy2] of the scrcpy v2.0
release gives more details about the audio feature.
If control is enabled, then the client captures relevant keyboard and mouse
events, that it transmits to the server, which injects them to the device. This
is the only socket which is used in both direction: input events are sent from
the client to the device, and when the device clipboard changes, the new content
is sent from the device to the client to support seamless copy-paste.
[scrcpy2]: https://blog.rom1v.com/2023/03/scrcpy-2-0-with-audio/
Note that the client-server roles are expressed at the application level:
- the server _serves_ video and audio streams, and handle requests from the
client,
- the client _controls_ the device through the server.
However, by default (when `--force-adb-forward` is not set), the roles are
reversed at the network level:
- the client opens a server socket and listen on a port before starting the
server,
- the server connects to the client.
This role inversion guarantees that the connection will not fail due to race
conditions without polling.
## Server
@ -32,15 +68,14 @@ The server is a Java application (with a [`public static void main(String...
args)`][main] method), compiled against the Android framework, and executed as
`shell` on the Android device.
[main]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Server.java#L123
[main]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/Server.java#L193
To run such a Java application, the classes must be [_dexed_][dex] (typically,
to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
`classes.dex`, pushed to the device in `/data/local/tmp`, then it can be run
with:
adb shell CLASSPATH=/data/local/tmp/classes.dex \
app_process / my.package.MainClass
adb shell CLASSPATH=/data/local/tmp/classes.dex app_process / my.package.MainClass
_The path `/data/local/tmp` is a good candidate to push the server, since it's
readable and writable by `shell`, but not world-writable, so a malicious
@ -49,7 +84,7 @@ application may not replace the server just before the client executes it._
Instead of a raw _dex_ file, `app_process` accepts a _jar_ containing
`classes.dex` (e.g. an [APK]). For simplicity, and to benefit from the gradle
build system, the server is built to an (unsigned) APK (renamed to
`scrcpy-server`).
`scrcpy-server.jar`).
[dex]: https://en.wikipedia.org/wiki/Dalvik_(software)
[apk]: https://en.wikipedia.org/wiki/Android_application_package
@ -65,42 +100,77 @@ They can be called using reflection though. The communication with hidden
components is provided by [_wrappers_ classes][wrappers] and [aidl].
[hidden]: https://stackoverflow.com/a/31908373/1987178
[wrappers]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers
[aidl]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/aidl/android/view
[wrappers]: https://github.com/Genymobile/scrcpy/tree/master/server/src/main/java/com/genymobile/scrcpy/wrappers
[aidl]: https://github.com/Genymobile/scrcpy/tree/master/server/src/main/aidl
### Threading
The server uses 3 threads:
### Execution
- the **main** thread, encoding and streaming the video to the client;
- the **controller** thread, listening for _control messages_ (typically,
keyboard and mouse events) from the client;
- the **receiver** thread (managed by the controller), sending _device messages_
to the clients (currently, it is only used to send the device clipboard
content).
The server is started by the client basically by executing the following
commands:
Since the video encoding is typically hardware, there would be no benefit in
encoding and streaming in two different threads.
```bash
adb push scrcpy-server /data/local/tmp/scrcpy-server.jar
adb forward tcp:27183 localabstract:scrcpy
adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar app_process / com.genymobile.scrcpy.Server 2.1
```
The first argument (`2.1` in the example) is the client scrcpy version. The
server fails if the client and the server do not have the exact same version.
The protocol between the client and the server may change from version to
version (see [protocol](#protocol) below), and there is no backward or forward
compatibility (there is no point to use different client and server versions).
This check allows to detect misconfiguration (running an older or newer server
by mistake).
It is followed by any number of arguments, in the form of `key=value` pairs.
Their order is irrelevant. The possible keys and associated value types can be
found in the [server][server-options] and [client][client-options] code.
[server-options]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/Options.java#L181
[client-options]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/server.c#L226
For example, if we execute `scrcpy -m1920 --no-audio`, then the server
execution will look like this:
```bash
# scid is a random number to identify different clients running on the same device
adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar app_process / com.genymobile.scrcpy.Server 2.1 scid=12345678 log_level=info audio=false max_size=1920
```
### Components
When executed, its [`main()`][main] method is executed (on the "main" thread).
It parses the arguments, establishes the connection with the client and starts
the other "components":
- the **video** streamer: it captures the video screen and send encoded video
packets on the _video_ socket (from the _video_ thread).
- the **audio** streamer: it uses several threads to capture raw packets,
submits them to encoding and retrieve encoded packets, which it sends on the
_audio_ socket.
- the **controller**: it receives _control messages_ (typically input events)
on the _control_ socket from one thread, and sends _device messages_ (e.g. to
transmit the device clipboard content to the client) on the same _control
socket_ from another thread. Thus, the _control_ socket is used in both
directions (contrary to the _video_ and _audio_ sockets).
### Screen video encoding
The encoding is managed by [`ScreenEncoder`].
The video is encoded using the [`MediaCodec`] API. The codec takes its input
from a [surface] associated to the display, and writes the resulting H.264
stream to the provided output stream (the socket connected to the client).
The video is encoded using the [`MediaCodec`] API. The codec encodes the content
of a `Surface` associated to the display, and writes the encoding packets to the
client (on the _video_ socket).
[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
[surface]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L68-L69
On device [rotation], the codec, surface and display are reinitialized, and a
new video stream is produced.
On device rotation (or folding), the encoding session is [reset] and restarted.
New frames are produced only when changes occur on the surface. This is good
because it avoids to send unnecessary frames, but there are drawbacks:
New frames are produced only when changes occur on the surface. This avoids to
send unnecessary frames, but by default there might be drawbacks:
- it does not send any frame on start if the device screen does not change,
- after fast motion changes, the last frame may have poor quality.
@ -108,11 +178,24 @@ because it avoids to send unnecessary frames, but there are drawbacks:
Both problems are [solved][repeat] by the flag
[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
[reset]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L179
[rotation]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90
[repeat]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
[repeat]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L246-L247
[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
### Audio encoding
Similarly, the audio is [captured] using an [`AudioRecord`], and [encoded] using
the [`MediaCodec`] asynchronous API.
More details are available on the [blog post][scrcpy2] introducing the audio feature.
[captured]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/AudioCapture.java
[encoded]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/AudioEncoder.java
[`AudioRecord`]: https://developer.android.com/reference/android/media/AudioRecord
### Input events injection
_Control messages_ are received from the client by the [`Controller`] (run in a
@ -124,13 +207,13 @@ separate thread). There are several types of input events:
- other commands (e.g. to switch the screen on or to copy the clipboard).
Some of them need to inject input events to the system. To do so, they use the
_hidden_ method [`InputManager.injectInputEvent`] (exposed by our
_hidden_ method [`InputManager.injectInputEvent()`] (exposed by the
[`InputManager` wrapper][inject-wrapper]).
[`Controller`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Controller.java#L81
[`Controller`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/Controller.java
[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
[`InputManager.injectInputEvent()`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L34
[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
@ -140,126 +223,222 @@ _hidden_ method [`InputManager.injectInputEvent`] (exposed by our
The client relies on [SDL], which provides cross-platform API for UI, input
events, threading, etc.
The video stream is decoded by [libav] (FFmpeg).
The video and audio streams are decoded by [FFmpeg].
[SDL]: https://www.libsdl.org
[libav]: https://www.libav.org/
[ffmpeg]: https://ffmpeg.org/
### Initialization
On startup, in addition to _libav_ and _SDL_ initialization, the client must
push and start the server on the device, and open two sockets (one for the video
stream, one for control) so that they may communicate.
The client parses the command line arguments, then [runs one of two code
paths][run]:
- scrcpy in "normal" mode ([`scrcpy.c`])
- scrcpy in [OTG mode](hid-otg.md) ([`scrcpy_otg.c`])
Note that the client-server roles are expressed at the application level:
[run]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/main.c#L81-L82
[`scrcpy.c`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/scrcpy.c#L292-L293
[`scrcpy_otg.c`]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/usb/scrcpy_otg.c#L51-L52
- the server _serves_ video stream and handle requests from the client,
- the client _controls_ the device through the server.
In the remaining of this document, we assume that the "normal" mode is used
(read the code for the OTG mode).
However, the roles are reversed at the network level:
- the client opens a server socket and listen on a port before starting the
server,
- the server connects to the client.
This role inversion guarantees that the connection will not fail due to race
conditions, and avoids polling.
_(Note that over TCP/IP, the roles are not reversed, due to a bug in `adb
reverse`. See commit [1038bad] and [issue #5].)_
Once the server is connected, it sends the device information (name and initial
screen dimensions). Thus, the client may init the window and renderer, before
the first frame is available.
To minimize startup time, SDL initialization is performed while listening for
the connection from the server (see commit [90a46b4]).
[1038bad]: https://github.com/Genymobile/scrcpy/commit/1038bad3850f18717a048a4d5c0f8110e54ee172
[issue #5]: https://github.com/Genymobile/scrcpy/issues/5
[90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e
On startup, the client:
- opens the _video_, _audio_ and _control_ sockets;
- pushes and starts the server on the device;
- initializes its components (demuxers, decoders, recorder…).
### Threading
### Video and audio streams
The client uses 4 threads:
- the **main** thread, executing the SDL event loop,
- the **stream** thread, receiving the video and used for decoding and
recording,
- the **controller** thread, sending _control messages_ to the server,
- the **receiver** thread (managed by the controller), receiving _device
messages_ from the server.
In addition, another thread can be started if necessary to handle APK
installation or file push requests (via drag&drop on the main window) or to
print the framerate regularly in the console.
### Stream
The video [stream] is received from the socket (connected to the server on the
device) in a separate thread.
If a [decoder] is present (i.e. `--no-display` is not set), then it uses _libav_
to decode the H.264 stream from the socket, and notifies the main thread when a
new frame is available.
There are two [frames][video_buffer] simultaneously in memory:
- the **decoding** frame, written by the decoder from the decoder thread,
- the **rendering** frame, rendered in a texture from the main thread.
When a new decoded frame is available, the decoder _swaps_ the decoding and
rendering frame (with proper synchronization). Thus, it immediately starts
to decode a new frame while the main thread renders the last one.
If a [recorder] is present (i.e. `--record` is enabled), then it muxes the raw
H.264 packet to the output video file.
[stream]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/stream.h
[decoder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/decoder.h
[video_buffer]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/video_buffer.h
[recorder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/recorder.h
Depending on the arguments passed to `scrcpy`, several components may be used.
Here is an overview of the video and audio components:
```
+----------+ +----------+
---> | decoder | ---> | screen |
+---------+ / +----------+ +----------+
socket ---> | stream | ----
+---------+ \ +----------+
---> | recorder |
+----------+
V4L2 sink
/
decoder
/ \
VIDEO -------------> demuxer display
\
recorder
/
AUDIO -------------> demuxer
\
decoder --- audio player
```
The _demuxer_ is responsible to extract video and audio packets (read some
header, split the video stream into packets at correct boundaries, etc.).
The demuxed packets may be sent to a _decoder_ (one per stream, to produce
frames) and to a recorder (receiving both video and audio stream to record a
single file). The packets are encoded on the device (by `MediaCodec`), but when
recording, they are _muxed_ (asynchronously) into a container (MKV or MP4) on
the client side.
Video frames are sent to the screen/display to be rendered in the scrcpy window.
They may also be sent to a [V4L2 sink](v4l2.md).
Audio "frames" (an array of decoded samples) are sent to the audio player.
### Controller
The [controller] is responsible to send _control messages_ to the device. It
The _controller_ is responsible to send _control messages_ to the device. It
runs in a separate thread, to avoid I/O on the main thread.
On SDL event, received on the main thread, the [input manager][inputmanager]
creates appropriate [_control messages_][controlmsg]. It is responsible to
convert SDL events to Android events (using [convert]). It pushes the _control
messages_ to a queue hold by the controller. On its own thread, the controller
takes messages from the queue, that it serializes and sends to the client.
[controller]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/controller.h
[controlmsg]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/control_msg.h
[inputmanager]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/input_manager.h
[convert]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/convert.h
On SDL event, received on the main thread, the _input manager_ creates
appropriate _control messages_. It is responsible to convert SDL events to
Android events. It then pushes the _control messages_ to a queue hold by the
controller. On its own thread, the controller takes messages from the queue,
that it serializes and sends to the client.
### UI and event loop
## Protocol
Initialization, input events and rendering are all [managed][scrcpy] in the main
thread.
The protocol between the client and the server must be considered _internal_: it
may (and will) change at any time for any reason. Everything may change (the
number of sockets, the order in which the sockets must be opened, the data
format on the wire…) from version to version. A client must always be run with a
matching server version.
Events are handled in the [event loop], which either updates the [screen] or
delegates to the [input manager][inputmanager].
This section documents the current protocol in scrcpy v2.1.
[scrcpy]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c
[event loop]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c#L201
[screen]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/screen.h
### Connection
Firstly, the client sets up an adb tunnel:
```bash
# By default, a reverse redirection: the computer listens, the device connects
adb reverse localabstract:scrcpy_<SCID> tcp:27183
# As a fallback (or if --force-adb forward is set), a forward redirection:
# the device listens, the computer connects
adb forward tcp:27183 localabstract:scrcpy_<SCID>
```
(`<SCID>` is a 31-bit random number, so that it does not fail when several
scrcpy instances start "at the same time" for the same device.)
Then, up to 3 sockets are opened, in that order:
- a _video_ socket
- an _audio_ socket
- a _control_ socket
Each one may be disabled (respectively by `--no-video`, `--no-audio` and
`--no-control`, directly or indirectly). For example, if `--no-audio` is set,
then the _video_ socket is opened first, then the _control_ socket.
On the _first_ socket opened (whichever it is), if the tunnel is _forward_, then
a [dummy byte] is sent from the device to the client. This allows to detect a
connection error (the client connection does not fail as long as there is an adb
forward redirection, even if nothing is listening on the device side).
Still on this _first_ socket, the device sends some [metadata][device meta] to
the client (currently only the device name, used as the window title, but there
might be other fields in the future).
[dummy byte]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/DesktopConnection.java#L93
[device meta]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/DesktopConnection.java#L151
You can read the [client][client-connection] and [server][server-connection]
code for more details.
[client-connection]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/app/src/server.c#L465-L466
[server-connection]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/DesktopConnection.java#L63
Then each socket is used for its intended purpose.
### Video and audio
On the _video_ and _audio_ sockets, the device first sends some [codec
metadata]:
- On the _video_ socket, 12 bytes:
- the codec id (`u32`) (H264, H265 or AV1)
- the initial video width (`u32`)
- the initial video height (`u32`)
- On the _audio_ socket, 4 bytes:
- the codec id (`u32`) (OPUS, AAC or RAW)
[codec metadata]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/Streamer.java#L33-L51
Then each packet produced by `MediaCodec` is sent, prefixed by a 12-byte [frame
header]:
- config packet flag (`u1`)
- key frame flag (`u1`)
- PTS (`u62`)
- packet size (`u32`)
Here is a schema describing the frame header:
```
[. . . . . . . .|. . . .]. . . . . . . . . . . . . . . ...
<-------------> <-----> <-----------------------------...
PTS packet raw packet
size
<--------------------->
frame header
The most significant bits of the PTS are used for packet flags:
byte 7 byte 6 byte 5 byte 4 byte 3 byte 2 byte 1 byte 0
CK...... ........ ........ ........ ........ ........ ........ ........
^^<------------------------------------------------------------------->
|| PTS
| `- key frame
`-- config packet
```
[frame header]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/Streamer.java#L83
### Controls
Controls messages are sent via a custom binary protocol.
The only documentation for this protocol is the set of unit tests on both sides:
- `ControlMessage` (from client to device): [serialization](https://github.com/Genymobile/scrcpy/blob/master/app/tests/test_control_msg_serialize.c) | [deserialization](https://github.com/Genymobile/scrcpy/blob/master/server/src/test/java/com/genymobile/scrcpy/ControlMessageReaderTest.java)
- `DeviceMessage` (from device to client) [serialization](https://github.com/Genymobile/scrcpy/blob/master/server/src/test/java/com/genymobile/scrcpy/DeviceMessageWriterTest.java) | [deserialization](https://github.com/Genymobile/scrcpy/blob/master/app/tests/test_device_msg_deserialize.c)
## Standalone server
Although the server is designed to work for the scrcpy client, it can be used
with any client which uses the same protocol.
For simplicity, some [server-specific options] have been added to produce raw
streams easily:
- `send_device_meta=false`: disable the device metata (in practice, the device
name) sent on the _first_ socket
- `send_frame_meta=false`: disable the 12-byte header for each packet
- `send_dummy_byte`: disable the dummy byte sent on forward connections
- `send_codec_meta`: disable the codec information (and initial device size for
video)
- `raw_stream`: disable all the above
[server-specific options]: https://github.com/Genymobile/scrcpy/blob/a3cdf1a6b86ea22786e1f7d09b9c202feabc6949/server/src/main/java/com/genymobile/scrcpy/Options.java#L309-L329
Concretely, here is how to expose a raw H.264 stream on a TCP socket:
```bash
adb push scrcpy-server-v2.1 /data/local/tmp/scrcpy-server-manual.jar
adb forward tcp:1234 localabstract:scrcpy
adb shell CLASSPATH=/data/local/tmp/scrcpy-server-manual.jar \
app_process / com.genymobile.scrcpy.Server 2.1 \
tunnel_forward=true audio=false control=false cleanup=false \
raw_stream=true max_size=1920
```
As soon as a client connects over TCP on port 1234, the device will start
streaming the video. For example, VLC can play the video (although you will
experience a very high latency, more details [here][vlc-0latency]):
```
vlc -Idummy --demux=h264 --network-caching=0 tcp://localhost:1234
```
[vlc-0latency]: https://code.videolan.org/rom1v/vlc/-/merge_requests/20
## Hack

View File

@ -106,3 +106,7 @@ scrcpy --otg # keyboard and mouse
Like `--hid-keyboard` and `--hid-mouse`, it only works if the device is
connected over USB.
## HID/OTG issues on Windows
See [FAQ](/FAQ.md#hidotg-issues-on-windows).

View File

@ -9,12 +9,10 @@ Scrcpy is packaged in several distributions and package managers:
- Debian/Ubuntu: `apt install scrcpy`
- Arch Linux: `pacman -S scrcpy`
- Fedora: `dnf copr enable zeno/scrcpy && dnf install scrcpy`
- Gentoo: [ebuild][ebuild-link] file
- Gentoo: `emerge scrcpy`
- Snap: `snap install scrcpy`
- … (see [repology](https://repology.org/project/scrcpy/versions))
[ebuild-link]: https://github.com/maggu2810/maggu2810-overlay/tree/master/app-mobilephone/scrcpy
### Latest version
However, the packaged version is not always the latest release. To install the

View File

@ -29,7 +29,7 @@ _<kbd>[Super]</kbd> is typically the <kbd>Windows</kbd> or <kbd>Cmd</kbd> key._
| Resize window to 1:1 (pixel-perfect) | <kbd>MOD</kbd>+<kbd>g</kbd>
| Resize window to remove black borders | <kbd>MOD</kbd>+<kbd>w</kbd> \| _Double-left-click¹_
| Click on `HOME` | <kbd>MOD</kbd>+<kbd>h</kbd> \| _Middle-click_
| Click on `BACK` | <kbd>MOD</kbd>+<kbd>b</kbd> \| _Right-click²_
| Click on `BACK` | <kbd>MOD</kbd>+<kbd>b</kbd> \| <kbd>MOD</kbd>+<kbd>Backspace</kbd> \| _Right-click²_
| Click on `APP_SWITCH` | <kbd>MOD</kbd>+<kbd>s</kbd> \| _4th-click³_
| Click on `MENU` (unlock screen)⁴ | <kbd>MOD</kbd>+<kbd>m</kbd>
| Click on `VOLUME_UP` | <kbd>MOD</kbd>+<kbd></kbd> _(up)_

View File

@ -21,7 +21,7 @@ If encoding fails, scrcpy automatically tries again with a lower definition
## Bit rate
The default video bit-rate is 8 Mbps. To change it:
The default video bit rate is 8 Mbps. To change it:
```bash
scrcpy --video-bit-rate=2M

View File

@ -15,7 +15,13 @@ Download the [latest release]:
and extract it.
Alternatively, you could install it from packages manager, like [Chocolatey]:
Alternatively, you could install it from packages manager, like [Winget]:
```bash
winget install scrcpy
```
or [Chocolatey]:
```bash
choco install scrcpy
@ -30,6 +36,7 @@ scoop install scrcpy
scoop install adb # if you don't have it yet
```
[Winget]: https://github.com/microsoft/winget-cli
[Chocolatey]: https://chocolatey.org/
[Scoop]: https://scoop.sh

View File

@ -1,5 +1,5 @@
project('scrcpy', 'c',
version: '2.0',
version: '2.1',
meson_version: '>= 0.48',
default_options: [
'c_std=c11',

View File

@ -98,9 +98,9 @@ dist-win32: build-server build-win32
cp app/prebuilt-deps/data/ffmpeg-6.0-scrcpy-4/win32/bin/avcodec-60.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/ffmpeg-6.0-scrcpy-4/win32/bin/avformat-60.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/ffmpeg-6.0-scrcpy-4/win32/bin/swresample-4.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.1/adb.exe "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.1/AdbWinApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.1/AdbWinUsbApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.3/adb.exe "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.3/AdbWinApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.3/AdbWinUsbApi.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/SDL2-2.28.0/i686-w64-mingw32/bin/SDL2.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
cp app/prebuilt-deps/data/libusb-1.0.26/libusb-MinGW-Win32/bin/msys-usb-1.0.dll "$(DIST)/$(WIN32_TARGET_DIR)/"
@ -116,9 +116,9 @@ dist-win64: build-server build-win64
cp app/prebuilt-deps/data/ffmpeg-6.0-scrcpy-4/win64/bin/avcodec-60.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/ffmpeg-6.0-scrcpy-4/win64/bin/avformat-60.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/ffmpeg-6.0-scrcpy-4/win64/bin/swresample-4.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.1/adb.exe "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.1/AdbWinApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.1/AdbWinUsbApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.3/adb.exe "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.3/AdbWinApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/platform-tools-34.0.3/AdbWinUsbApi.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/SDL2-2.28.0/x86_64-w64-mingw32/bin/SDL2.dll "$(DIST)/$(WIN64_TARGET_DIR)/"
cp app/prebuilt-deps/data/libusb-1.0.26/libusb-MinGW-x64/bin/msys-usb-1.0.dll "$(DIST)/$(WIN64_TARGET_DIR)/"

View File

@ -7,8 +7,8 @@ android {
applicationId "com.genymobile.scrcpy"
minSdkVersion 21
targetSdkVersion 33
versionCode 20000
versionName "2.0"
versionCode 20100
versionName "2.1"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
buildTypes {

View File

@ -12,7 +12,7 @@
set -e
SCRCPY_DEBUG=false
SCRCPY_VERSION_NAME=2.0
SCRCPY_VERSION_NAME=2.1
PLATFORM=${ANDROID_PLATFORM:-33}
BUILD_TOOLS=${ANDROID_BUILD_TOOLS:-33.0.0}
@ -48,6 +48,7 @@ cd "$SERVER_DIR/src/main/aidl"
"$BUILD_TOOLS_DIR/aidl" -o"$GEN_DIR" android/view/IRotationWatcher.aidl
"$BUILD_TOOLS_DIR/aidl" -o"$GEN_DIR" \
android/content/IOnPrimaryClipChangedListener.aidl
"$BUILD_TOOLS_DIR/aidl" -o"$GEN_DIR" android/view/IDisplayFoldListener.aidl
echo "Compiling java sources..."
cd ../java

View File

@ -64,8 +64,6 @@ public final class DesktopConnection implements Closeable {
throws IOException {
String socketName = getSocketName(scid);
LocalSocket firstSocket = null;
LocalSocket videoSocket = null;
LocalSocket audioSocket = null;
LocalSocket controlSocket = null;
@ -74,24 +72,28 @@ public final class DesktopConnection implements Closeable {
try (LocalServerSocket localServerSocket = new LocalServerSocket(socketName)) {
if (video) {
videoSocket = localServerSocket.accept();
firstSocket = videoSocket;
if (sendDummyByte) {
// send one byte so the client may read() to detect a connection error
videoSocket.getOutputStream().write(0);
sendDummyByte = false;
}
}
if (audio) {
audioSocket = localServerSocket.accept();
if (firstSocket == null) {
firstSocket = audioSocket;
if (sendDummyByte) {
// send one byte so the client may read() to detect a connection error
audioSocket.getOutputStream().write(0);
sendDummyByte = false;
}
}
if (control) {
controlSocket = localServerSocket.accept();
if (firstSocket == null) {
firstSocket = controlSocket;
if (sendDummyByte) {
// send one byte so the client may read() to detect a connection error
controlSocket.getOutputStream().write(0);
sendDummyByte = false;
}
}
if (sendDummyByte) {
// send one byte so the client may read() to detect a connection error
firstSocket.getOutputStream().write(0);
}
}
} else {
if (video) {

View File

@ -99,25 +99,27 @@ public final class Device {
}
}, displayId);
ServiceManager.getWindowManager().registerDisplayFoldListener(new IDisplayFoldListener.Stub() {
@Override
public void onDisplayFoldChanged(int displayId, boolean folded) {
synchronized (Device.this) {
DisplayInfo displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(displayId);
if (displayInfo == null) {
Ln.e("Display " + displayId + " not found\n" + LogUtils.buildDisplayListMessage());
return;
}
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
ServiceManager.getWindowManager().registerDisplayFoldListener(new IDisplayFoldListener.Stub() {
@Override
public void onDisplayFoldChanged(int displayId, boolean folded) {
synchronized (Device.this) {
DisplayInfo displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(displayId);
if (displayInfo == null) {
Ln.e("Display " + displayId + " not found\n" + LogUtils.buildDisplayListMessage());
return;
}
screenInfo = ScreenInfo.computeScreenInfo(displayInfo.getRotation(), displayInfo.getSize(), options.getCrop(),
options.getMaxSize(), options.getLockVideoOrientation());
// notify
if (foldListener != null) {
foldListener.onFoldChanged(displayId, folded);
screenInfo = ScreenInfo.computeScreenInfo(displayInfo.getRotation(), displayInfo.getSize(), options.getCrop(),
options.getMaxSize(), options.getLockVideoOrientation());
// notify
if (foldListener != null) {
foldListener.onFoldChanged(displayId, folded);
}
}
}
}
});
});
}
if (options.getControl() && options.getClipboardAutosync()) {
// If control and autosync are enabled, synchronize Android clipboard to the computer automatically

View File

@ -2,6 +2,7 @@ package com.genymobile.scrcpy.wrappers;
import com.genymobile.scrcpy.Ln;
import android.annotation.TargetApi;
import android.os.IInterface;
import android.view.IRotationWatcher;
import android.view.IDisplayFoldListener;
@ -106,16 +107,17 @@ public final class WindowManager {
cls.getMethod("watchRotation", IRotationWatcher.class).invoke(manager, rotationWatcher);
}
} catch (Exception e) {
throw new AssertionError(e);
Ln.e("Could not register rotation watcher", e);
}
}
@TargetApi(29)
public void registerDisplayFoldListener(IDisplayFoldListener foldListener) {
try {
Class<?> cls = manager.getClass();
cls.getMethod("registerDisplayFoldListener", IDisplayFoldListener.class).invoke(manager, foldListener);
} catch (Exception e) {
throw new AssertionError(e);
Ln.e("Could not register display fold listener", e);
}
}
}