Compare commits

...

16 Commits

Author SHA1 Message Date
3174ce2d5a Listen to display changed events
Replace RotationWatcher and DisplayFoldListener by a single
DisplayListener, which is notified whenever the display size or dpi
changes.

Still use the old version specifically for Android 14, where
DisplayListener may be broken (it is fixed in recent Android 14
upgrades), until we receive the first DisplayListener event (which
proves that it works).

Refs #4469 <https://github.com/Genymobile/scrcpy/pull/4469>

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2024-10-22 09:51:38 +02:00
05c80a036f Use explicit constants in @TargetApi 2024-10-21 18:58:49 +02:00
db241065d1 Add option to start an app by its name
By adding the '?' prefix, the app is searched by its name instead of its
package name (it may take some time to retrieve the app names on the
device):

    scrcpy --start-app=?firefox

An app matches if its label starts with the given name,
case-insensitive.

If '+' is also passed to force-stop the app before starting, then the
prefixes must be in that order:

    scrcpy --start-app=+?firefox
2024-10-21 18:52:05 +02:00
85302acf5e Add option to force-stop app before starting
The previous commit introduced:

    scrcpy --start-app=name

By adding a '+' prefix, the app is stopped beforehand:

    scrcpy --start-app=+name

This may be useful to start a fresh app on a new virtual display:

    scrcpy --new-display --start-app=+org.mozilla.firefox
2024-10-21 18:52:05 +02:00
d2989920c1 Add --start-app
Add a command line option --start-app=name to start an Android app by
its package name. For example:

    scrcpy --start-app=org.mozilla.firefox

The app will be started on the correct target display:

    scrcpy --new-display --start-app=org.mozilla.firefox

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
2024-10-21 18:52:05 +02:00
6ff3d9b571 Add --list-apps
Add an option to list all apps installed on the device:

    scrcpy --list-apps
2024-10-21 18:52:05 +02:00
1ea229fc2c Add virtual display feature
Add a feature to create a new (separate) virtual display instead of
mirroring the device screen:

    scrcpy --new-display=1920x1080
    scrcpy --new-display=1920x1080/420  # force 420 dpi
    scrcpy --new-display       # use the default screen size and density
    scrcpy --new-display=/240  # use the default screen size and 240 dpi

Co-authored-by: Simon Chan <1330321+yume-chan@users.noreply.github.com>
Co-authored-by: anirudhb <anirudhb@users.noreply.github.com>
2024-10-21 18:52:05 +02:00
49c7d1037f Add DPI to DisplayInfo
It will be useful to automatically set an appropriate DPI for new
virtual displays.
2024-10-21 18:52:05 +02:00
57b91b1fee Inject display-related events to virtual display
Mouse and touch events must be sent to the virtual display id (used for
mirroring), other events (like key events) must be sent to the original
display id.

Co-authored-by: nightmare <mengyanshou@gmail.com>
2024-10-21 18:52:05 +02:00
49cf48614e Send PositionMapper to Controller directly
When a new capture starts, send a new PositionMapper to the Controller
without using the global Device as an intermediate.

Now all Device methods are static.
2024-10-21 18:52:05 +02:00
699c5c7efb Introduce PositionMapper
Extract the function to convert coordinates from video space to display
space to a separate component.

It only requires the data it uses, it does not need a full ScreenInfo
(although it can be created from a ScreenInfo instance).
2024-10-21 18:52:05 +02:00
c20aae5331 Move clipboard management to Controller
Continue to declutter the global Device.
2024-10-21 18:52:05 +02:00
7dc21e87f9 Move implicit displayId to Controller
Remove from Device the functions using an implicit displayId.

Move them to Controller, which knows best which displayId it must use.

This will allow to properly dispatch events either to the origin display
or to the virtual display created for mirroring.
2024-10-21 18:52:05 +02:00
cf3ae53e97 Add capture prepare() step
Add a function called before each capture starts (before getSize() is
called).

This allows to compute the ScreenInfo instance once exactly when needed.
2024-10-21 18:52:05 +02:00
5c1482559a Refactor to call getSize() only once
There is no need to call capture.getSize() (provided by the
SurfaceCapture implementation) twice.
2024-10-21 18:52:05 +02:00
68476b6d28 Move screen-related features out of Device.java
Move the code related to screen size and rotation/fold to ScreenCapture.

For now, keep the ScreenInfo instance in the Device class to communicate
with the Controller, but it will be removed by further commits.
2024-10-21 18:52:05 +02:00
34 changed files with 1218 additions and 310 deletions

View File

@ -33,6 +33,7 @@ _scrcpy() {
--keyboard=
--kill-adb-on-close
--legacy-paste
--list-apps
--list-camera-sizes
--list-cameras
--list-displays
@ -46,6 +47,8 @@ _scrcpy() {
--mouse-bind=
-n --no-control
-N --no-playback
--new-display
--new-display=
--no-audio
--no-audio-playback
--no-cleanup
@ -76,6 +79,7 @@ _scrcpy() {
-s --serial=
-S --turn-screen-off
--shortcut-mod=
--start-app=
-t --show-touches
--tcpip
--tcpip=

View File

@ -40,6 +40,7 @@ arguments=(
'--keyboard=[Set the keyboard input mode]:mode:(disabled sdk uhid aoa)'
'--kill-adb-on-close[Kill adb when scrcpy terminates]'
'--legacy-paste[Inject computer clipboard text as a sequence of key events on Ctrl+v]'
'--list-apps[List Android apps installed on the device]'
'--list-camera-sizes[List the valid camera capture sizes]'
'--list-cameras[List cameras available on the device]'
'--list-displays[List displays available on the device]'
@ -52,6 +53,7 @@ arguments=(
'--mouse-bind=[Configure bindings of secondary clicks]'
{-n,--no-control}'[Disable device control \(mirror the device in read only\)]'
{-N,--no-playback}'[Disable video and audio playback]'
'--new-display=[Create a new display]'
'--no-audio[Disable audio forwarding]'
'--no-audio-playback[Disable audio playback]'
'--no-cleanup[Disable device cleanup actions on exit]'
@ -80,6 +82,7 @@ arguments=(
{-s,--serial=}'[The device serial number \(mandatory for multiple devices only\)]:serial:($("${ADB-adb}" devices | awk '\''$2 == "device" {print $1}'\''))'
{-S,--turn-screen-off}'[Turn the device screen off immediately]'
'--shortcut-mod=[\[key1,key2+key3,...\] Specify the modifiers to use for scrcpy shortcuts]:shortcut mod:(lctrl rctrl lalt ralt lsuper rsuper)'
'--start-app=[Start an Android app]'
{-t,--show-touches}'[Show physical touches]'
'--tcpip[\(optional \[ip\:port\]\) Configure and connect the device over TCP/IP]'
'--time-limit=[Set the maximum mirroring time, in seconds]'

View File

@ -227,6 +227,10 @@ Inject computer clipboard text as a sequence of key events on Ctrl+v (like MOD+S
This is a workaround for some devices not behaving as expected when setting the device clipboard programmatically.
.TP
.B \-\-list\-apps
List Android apps installed on the device.
.TP
.B \-\-list\-camera\-sizes
List the valid camera capture sizes.
@ -314,6 +318,17 @@ Disable device control (mirror the device in read\-only).
.B \-N, \-\-no\-playback
Disable video and audio playback on the computer (equivalent to \fB\-\-no\-video\-playback \-\-no\-audio\-playback\fR).
.TP
\fB\-\-new\-display\fR[=[\fIwidth\fRx\fIheight\fR][/\fIdpi\fR]]
Create a new display with the specified resolution and density. If not provided, they default to the main display dimensions and DPI, and \fB\-\-max\-size\fR is considered.
Examples:
\-\-new\-display=1920x1080
\-\-new\-display=1920x1080/420
\-\-new\-display # default screen size and density
\-\-new\-display=240 # default screen size and 240 dpi
.TP
.B \-\-no\-audio
Disable audio forwarding.
@ -478,6 +493,22 @@ For example, to use either LCtrl or LSuper for scrcpy shortcuts, pass "lctrl,lsu
Default is "lalt,lsuper" (left-Alt or left-Super).
.TP
.BI "\-\-start\-app " name
Start an Android app, by its exact package name.
Add a '?' prefix to select an app whose name starts with the given name, case-insensitive (it may take some time to retrieve the app names on the device):
scrcpy --start-app=?firefox
Add a '+' prefix to force-stop before starting the app:
scrcpy --new-display --start-app=+org.mozilla.firefox
Both prefixes can be used, in that order:
scrcpy --start-app=+?firefox
.TP
.B \-t, \-\-show\-touches
Enable "show touches" on start, restore the initial value on exit.

View File

@ -102,6 +102,9 @@ enum {
OPT_NO_MOUSE_HOVER,
OPT_AUDIO_DUP,
OPT_GAMEPAD,
OPT_NEW_DISPLAY,
OPT_LIST_APPS,
OPT_START_APP,
};
struct sc_option {
@ -442,6 +445,11 @@ static const struct sc_option options[] = {
"This is a workaround for some devices not behaving as "
"expected when setting the device clipboard programmatically.",
},
{
.longopt_id = OPT_LIST_APPS,
.longopt = "list-apps",
.text = "List Android apps installed on the device.",
},
{
.longopt_id = OPT_LIST_CAMERAS,
.longopt = "list-cameras",
@ -557,6 +565,20 @@ static const struct sc_option options[] = {
.text = "Disable video and audio playback on the computer (equivalent "
"to --no-video-playback --no-audio-playback).",
},
{
.longopt_id = OPT_NEW_DISPLAY,
.longopt = "new-display",
.argdesc = "[<width>x<height>][/<dpi>]",
.optional_arg = true,
.text = "Create a new display with the specified resolution and "
"density. If not provided, they default to the main display "
"dimensions and DPI, and --max-size is considered.\n"
"Examples:\n"
" --new-display=1920x1080\n"
" --new-display=1920x1080/420 # force 420 dpi\n"
" --new-display # default screen size and density\n"
" --new-display=240 # default screen size and 240 dpi",
},
{
.longopt_id = OPT_NO_AUDIO,
.longopt = "no-audio",
@ -784,6 +806,20 @@ static const struct sc_option options[] = {
"shortcuts, pass \"lctrl,lsuper\".\n"
"Default is \"lalt,lsuper\" (left-Alt or left-Super).",
},
{
.longopt_id = OPT_START_APP,
.longopt = "start-app",
.argdesc = "name",
.text = "Start an Android app, by its exact package name.\n"
"Add a '?' prefix to select an app whose name starts with the "
"given name, case-insensitive (it may take some time to "
"retrieve the app names on the device):\n"
" scrcpy --start-app=?firefox\n"
"Add a '+' prefix to force-stop before starting the app:\n"
" scrcpy --new-display --start-app=+org.mozilla.firefox\n"
"Both prefixes can be used, in that order:\n"
" scrcpy --start-app=+?firefox",
},
{
.shortopt = 't',
.longopt = "show-touches",
@ -2595,6 +2631,9 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
case OPT_LIST_CAMERA_SIZES:
opts->list |= SC_OPTION_LIST_CAMERA_SIZES;
break;
case OPT_LIST_APPS:
opts->list |= SC_OPTION_LIST_APPS;
break;
case OPT_REQUIRE_AUDIO:
opts->require_audio = true;
break;
@ -2668,6 +2707,12 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
return false;
}
break;
case OPT_NEW_DISPLAY:
opts->new_display = optarg ? optarg : "auto";
break;
case OPT_START_APP:
opts->start_app = optarg;
break;
default:
// getopt prints the error message on stderr
return false;
@ -2918,6 +2963,11 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
return false;
}
if (opts->new_display) {
LOGE("--new-display is only available with --video-source=display");
return false;
}
if (opts->camera_id && opts->camera_facing != SC_CAMERA_FACING_ANY) {
LOGE("Cannot specify both --camera-id and --camera-facing");
return false;
@ -2954,6 +3004,11 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
return false;
}
if (opts->display_id != 0 && opts->new_display) {
LOGE("Cannot specify both --display-id and --new-display");
return false;
}
if (opts->audio && opts->audio_source == SC_AUDIO_SOURCE_AUTO) {
// Select the audio source according to the video source
if (opts->video_source == SC_VIDEO_SOURCE_DISPLAY) {
@ -3086,6 +3141,10 @@ parse_args_with_getopt(struct scrcpy_cli_args *args, int argc, char *argv[],
LOGE("Cannot request power off on close if control is disabled");
return false;
}
if (opts->start_app) {
LOGE("Cannot start an Android app if control is disabled");
return false;
}
}
# ifdef _WIN32

View File

@ -183,6 +183,10 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf) {
case SC_CONTROL_MSG_TYPE_UHID_DESTROY:
sc_write16be(&buf[1], msg->uhid_destroy.id);
return 3;
case SC_CONTROL_MSG_TYPE_START_APP: {
size_t len = write_string_tiny(&buf[1], msg->start_app.name, 255);
return 1 + len;
}
case SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL:
case SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL:
case SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS:
@ -308,6 +312,9 @@ sc_control_msg_log(const struct sc_control_msg *msg) {
case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
LOG_CMSG("open hard keyboard settings");
break;
case SC_CONTROL_MSG_TYPE_START_APP:
LOG_CMSG("start app \"%s\"", msg->start_app.name);
break;
default:
LOG_CMSG("unknown type: %u", (unsigned) msg->type);
break;
@ -333,6 +340,9 @@ sc_control_msg_destroy(struct sc_control_msg *msg) {
case SC_CONTROL_MSG_TYPE_SET_CLIPBOARD:
free(msg->set_clipboard.text);
break;
case SC_CONTROL_MSG_TYPE_START_APP:
free(msg->start_app.name);
break;
default:
// do nothing
break;

View File

@ -41,6 +41,7 @@ enum sc_control_msg_type {
SC_CONTROL_MSG_TYPE_UHID_INPUT,
SC_CONTROL_MSG_TYPE_UHID_DESTROY,
SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
SC_CONTROL_MSG_TYPE_START_APP,
};
enum sc_screen_power_mode {
@ -110,6 +111,9 @@ struct sc_control_msg {
struct {
uint16_t id;
} uhid_destroy;
struct {
char *name;
} start_app;
};
};

View File

@ -103,6 +103,8 @@ const struct scrcpy_options scrcpy_options_default = {
.window = true,
.mouse_hover = true,
.audio_dup = false,
.new_display = NULL,
.start_app = NULL,
};
enum sc_orientation

View File

@ -304,10 +304,13 @@ struct scrcpy_options {
#define SC_OPTION_LIST_DISPLAYS 0x2
#define SC_OPTION_LIST_CAMERAS 0x4
#define SC_OPTION_LIST_CAMERA_SIZES 0x8
#define SC_OPTION_LIST_APPS 0x10
uint8_t list;
bool window;
bool mouse_hover;
bool audio_dup;
const char *new_display; // [<width>x<height>][/<dpi>] parsed by the server
const char *start_app;
};
extern const struct scrcpy_options scrcpy_options_default;

View File

@ -431,6 +431,7 @@ scrcpy(struct scrcpy_options *options) {
.lock_video_orientation = options->lock_video_orientation,
.control = options->control,
.display_id = options->display_id,
.new_display = options->new_display,
.video = options->video,
.audio = options->audio,
.audio_dup = options->audio_dup,
@ -906,6 +907,25 @@ aoa_complete:
init_sdl_gamepads();
}
if (options->control && options->start_app) {
assert(controller);
char *name = strdup(options->start_app);
if (!name) {
LOG_OOM();
goto end;
}
struct sc_control_msg msg;
msg.type = SC_CONTROL_MSG_TYPE_START_APP;
msg.start_app.name = name;
if (!sc_controller_push_msg(controller, &msg)) {
LOGW("Could not request start app '%s'", name);
free(name);
}
}
ret = event_loop(s);
terminate_event_loop();
LOGD("quit...");

View File

@ -355,6 +355,10 @@ execute_server(struct sc_server *server,
// By default, power_on is true
ADD_PARAM("power_on=false");
}
if (params->new_display) {
VALIDATE_STRING(params->new_display);
ADD_PARAM("new_display=%s", params->new_display);
}
if (params->list & SC_OPTION_LIST_ENCODERS) {
ADD_PARAM("list_encoders=true");
}
@ -367,6 +371,9 @@ execute_server(struct sc_server *server,
if (params->list & SC_OPTION_LIST_CAMERA_SIZES) {
ADD_PARAM("list_camera_sizes=true");
}
if (params->list & SC_OPTION_LIST_APPS) {
ADD_PARAM("list_apps=true");
}
#undef ADD_PARAM

View File

@ -48,6 +48,7 @@ struct sc_server_params {
int8_t lock_video_orientation;
bool control;
uint32_t display_id;
const char *new_display;
bool video;
bool audio;
bool audio_dup;

View File

@ -139,8 +139,10 @@ public final class CleanUp {
if (Device.isScreenOn()) {
if (powerOffScreen) {
Ln.i("Power off screen");
Device.powerOffScreen(displayId);
if (displayId != Device.DISPLAY_ID_NONE) {
Ln.i("Power off screen");
Device.powerOffScreen(displayId);
}
} else if (restoreNormalPowerMode) {
Ln.i("Restoring normal power mode");
Device.setScreenPowerMode(Device.POWER_MODE_NORMAL);

View File

@ -2,6 +2,7 @@ package com.genymobile.scrcpy;
import com.genymobile.scrcpy.audio.AudioCodec;
import com.genymobile.scrcpy.audio.AudioSource;
import com.genymobile.scrcpy.device.NewDisplay;
import com.genymobile.scrcpy.device.Size;
import com.genymobile.scrcpy.util.CodecOption;
import com.genymobile.scrcpy.util.Ln;
@ -54,10 +55,13 @@ public class Options {
private boolean cleanup = true;
private boolean powerOn = true;
private NewDisplay newDisplay;
private boolean listEncoders;
private boolean listDisplays;
private boolean listCameras;
private boolean listCameraSizes;
private boolean listApps;
// Options not used by the scrcpy client, but useful to use scrcpy-server directly
private boolean sendDeviceMeta = true; // send device name and size
@ -205,8 +209,12 @@ public class Options {
return powerOn;
}
public NewDisplay getNewDisplay() {
return newDisplay;
}
public boolean getList() {
return listEncoders || listDisplays || listCameras || listCameraSizes;
return listEncoders || listDisplays || listCameras || listCameraSizes || listApps;
}
public boolean getListEncoders() {
@ -225,6 +233,10 @@ public class Options {
return listCameraSizes;
}
public boolean getListApps() {
return listApps;
}
public boolean getSendDeviceMeta() {
return sendDeviceMeta;
}
@ -388,6 +400,9 @@ public class Options {
case "list_camera_sizes":
options.listCameraSizes = Boolean.parseBoolean(value);
break;
case "list_apps":
options.listApps = Boolean.parseBoolean(value);
break;
case "camera_id":
if (!value.isEmpty()) {
options.cameraId = value;
@ -418,6 +433,9 @@ public class Options {
case "camera_high_speed":
options.cameraHighSpeed = Boolean.parseBoolean(value);
break;
case "new_display":
options.newDisplay = parseNewDisplay(value);
break;
case "send_device_meta":
options.sendDeviceMeta = Boolean.parseBoolean(value);
break;
@ -504,4 +522,32 @@ public class Options {
throw new IllegalArgumentException("Invalid float value for " + key + ": \"" + value + "\"");
}
}
private static NewDisplay parseNewDisplay(String newDisplay) {
// input format: "auto", or "<width>x<height>", or "<width>x<height>:<dpi>"
if ("auto".equals(newDisplay)) {
return new NewDisplay();
}
String[] tokens = newDisplay.split("/");
Size size;
if (!tokens[0].isEmpty()) {
size = parseSize(tokens[0]);
} else {
size = null;
}
int dpi;
if (tokens.length >= 2) {
dpi = Integer.parseInt(tokens[1]);
if (dpi <= 0) {
throw new IllegalArgumentException("Invalid non-positive dpi: " + tokens[1]);
}
} else {
dpi = 0;
}
return new NewDisplay(size, dpi);
}
}

View File

@ -9,16 +9,17 @@ import com.genymobile.scrcpy.audio.AudioRawRecorder;
import com.genymobile.scrcpy.audio.AudioSource;
import com.genymobile.scrcpy.control.ControlChannel;
import com.genymobile.scrcpy.control.Controller;
import com.genymobile.scrcpy.control.DeviceMessage;
import com.genymobile.scrcpy.device.ConfigurationException;
import com.genymobile.scrcpy.device.DesktopConnection;
import com.genymobile.scrcpy.device.Device;
import com.genymobile.scrcpy.device.NewDisplay;
import com.genymobile.scrcpy.device.Streamer;
import com.genymobile.scrcpy.util.Ln;
import com.genymobile.scrcpy.util.LogUtils;
import com.genymobile.scrcpy.util.Settings;
import com.genymobile.scrcpy.util.SettingsException;
import com.genymobile.scrcpy.video.CameraCapture;
import com.genymobile.scrcpy.video.NewDisplayCapture;
import com.genymobile.scrcpy.video.ScreenCapture;
import com.genymobile.scrcpy.video.SurfaceCapture;
import com.genymobile.scrcpy.video.SurfaceEncoder;
@ -129,8 +130,11 @@ public final class Server {
CleanUp cleanUp = null;
Thread initThread = null;
NewDisplay newDisplay = options.getNewDisplay();
int displayId = newDisplay == null ? options.getDisplayId() : Device.DISPLAY_ID_NONE;
if (options.getCleanup()) {
cleanUp = CleanUp.configure(options.getDisplayId());
cleanUp = CleanUp.configure(displayId);
initThread = startInitThread(options, cleanUp);
}
@ -140,9 +144,6 @@ public final class Server {
boolean video = options.getVideo();
boolean audio = options.getAudio();
boolean sendDummyByte = options.getSendDummyByte();
boolean camera = video && options.getVideoSource() == VideoSource.CAMERA;
final Device device = camera ? null : new Device(options);
Workarounds.apply();
@ -154,13 +155,11 @@ public final class Server {
connection.sendDeviceMeta(Device.getDeviceName());
}
Controller controller = null;
if (control) {
ControlChannel controlChannel = connection.getControlChannel();
Controller controller = new Controller(device, controlChannel, cleanUp, options.getClipboardAutosync(), options.getPowerOn());
device.setClipboardListener(text -> {
DeviceMessage msg = DeviceMessage.createClipboard(text);
controller.getSender().send(msg);
});
controller = new Controller(displayId, controlChannel, cleanUp, options.getClipboardAutosync(), options.getPowerOn());
asyncProcessors.add(controller);
}
@ -190,7 +189,13 @@ public final class Server {
options.getSendFrameMeta());
SurfaceCapture surfaceCapture;
if (options.getVideoSource() == VideoSource.DISPLAY) {
surfaceCapture = new ScreenCapture(device);
if (newDisplay != null) {
surfaceCapture = new NewDisplayCapture(controller, newDisplay, options.getMaxSize());
} else {
assert displayId != Device.DISPLAY_ID_NONE;
surfaceCapture = new ScreenCapture(controller, displayId, options.getMaxSize(), options.getCrop(),
options.getLockVideoOrientation());
}
} else {
surfaceCapture = new CameraCapture(options.getCameraId(), options.getCameraFacing(), options.getCameraSize(),
options.getMaxSize(), options.getCameraAspectRatio(), options.getCameraFps(), options.getCameraHighSpeed());
@ -282,6 +287,11 @@ public final class Server {
Workarounds.apply();
Ln.i(LogUtils.buildCameraListMessage(options.getListCameraSizes()));
}
if (options.getListApps()) {
Workarounds.apply();
Ln.i("Processing Android apps... (this may take some time)");
Ln.i(LogUtils.buildAppListMessage());
}
// Just print the requested data, do not mirror
return;
}

View File

@ -23,6 +23,7 @@ public final class ControlMessage {
public static final int TYPE_UHID_INPUT = 13;
public static final int TYPE_UHID_DESTROY = 14;
public static final int TYPE_OPEN_HARD_KEYBOARD_SETTINGS = 15;
public static final int TYPE_START_APP = 16;
public static final long SEQUENCE_INVALID = 0;
@ -155,6 +156,13 @@ public final class ControlMessage {
return msg;
}
public static ControlMessage createStartApp(String name) {
ControlMessage msg = new ControlMessage();
msg.type = TYPE_START_APP;
msg.text = name;
return msg;
}
public int getType() {
return type;
}

View File

@ -53,6 +53,8 @@ public class ControlMessageReader {
return parseUhidInput();
case ControlMessage.TYPE_UHID_DESTROY:
return parseUhidDestroy();
case ControlMessage.TYPE_START_APP:
return parseStartApp();
default:
throw new ControlProtocolException("Unknown event type: " + type);
}
@ -155,6 +157,11 @@ public class ControlMessageReader {
return ControlMessage.createUhidDestroy(id);
}
private ControlMessage parseStartApp() throws IOException {
String name = parseString(1);
return ControlMessage.createStartApp(name);
}
private Position parsePosition() throws IOException {
int x = dis.readInt();
int y = dis.readInt();

View File

@ -4,12 +4,17 @@ import com.genymobile.scrcpy.AndroidVersions;
import com.genymobile.scrcpy.AsyncProcessor;
import com.genymobile.scrcpy.CleanUp;
import com.genymobile.scrcpy.device.Device;
import com.genymobile.scrcpy.device.DeviceApp;
import com.genymobile.scrcpy.device.Point;
import com.genymobile.scrcpy.device.Position;
import com.genymobile.scrcpy.util.Ln;
import com.genymobile.scrcpy.util.LogUtils;
import com.genymobile.scrcpy.video.VirtualDisplayListener;
import com.genymobile.scrcpy.wrappers.ClipboardManager;
import com.genymobile.scrcpy.wrappers.InputManager;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import android.content.IOnPrimaryClipChangedListener;
import android.content.Intent;
import android.os.Build;
import android.os.SystemClock;
@ -19,11 +24,40 @@ import android.view.KeyEvent;
import android.view.MotionEvent;
import java.io.IOException;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
public class Controller implements AsyncProcessor {
public class Controller implements AsyncProcessor, VirtualDisplayListener {
/*
* For event injection, there are two display ids:
* - the displayId passed to the constructor (which comes from --display-id passed by the client, 0 for the main display);
* - the virtualDisplayId used for mirroring, notified by the capture instance via the VirtualDisplayListener interface.
*
* (In case the ScreenCapture uses the "SurfaceControl API", then both ids are equals, but this is an implementation detail.)
*
* In order to make events work correctly in all cases:
* - virtualDisplayId must be used for events relative to the display (mouse and touch events with coordinates);
* - displayId must be used for other events (like key events).
*
* If a new separate virtual display is created (using --new-display), then displayId == Device.DISPLAY_ID_NONE. In that case, all events are
* sent to the virtual display id.
*/
private static final class DisplayData {
private final int virtualDisplayId;
private final PositionMapper positionMapper;
private DisplayData(int virtualDisplayId, PositionMapper positionMapper) {
this.virtualDisplayId = virtualDisplayId;
this.positionMapper = positionMapper;
}
}
private static final int DEFAULT_DEVICE_ID = 0;
@ -31,12 +65,14 @@ public class Controller implements AsyncProcessor {
private static final int POINTER_ID_MOUSE = -1;
private static final ScheduledExecutorService EXECUTOR = Executors.newSingleThreadScheduledExecutor();
private ExecutorService startAppExecutor;
private Thread thread;
private UhidManager uhidManager;
private final Device device;
private final int displayId;
private final boolean supportsInputEvents;
private final ControlChannel controlChannel;
private final CleanUp cleanUp;
private final DeviceMessageSender sender;
@ -45,6 +81,11 @@ public class Controller implements AsyncProcessor {
private final KeyCharacterMap charMap = KeyCharacterMap.load(KeyCharacterMap.VIRTUAL_KEYBOARD);
private final AtomicBoolean isSettingClipboard = new AtomicBoolean();
private final AtomicReference<DisplayData> displayData = new AtomicReference<>();
private final Object displayDataAvailable = new Object(); // condition variable
private long lastTouchDown;
private final PointersState pointersState = new PointersState();
private final MotionEvent.PointerProperties[] pointerProperties = new MotionEvent.PointerProperties[PointersState.MAX_POINTERS];
@ -52,14 +93,54 @@ public class Controller implements AsyncProcessor {
private boolean keepPowerModeOff;
public Controller(Device device, ControlChannel controlChannel, CleanUp cleanUp, boolean clipboardAutosync, boolean powerOn) {
this.device = device;
public Controller(int displayId, ControlChannel controlChannel, CleanUp cleanUp, boolean clipboardAutosync, boolean powerOn) {
this.displayId = displayId;
this.controlChannel = controlChannel;
this.cleanUp = cleanUp;
this.clipboardAutosync = clipboardAutosync;
this.powerOn = powerOn;
initPointers();
sender = new DeviceMessageSender(controlChannel);
// main display or any display on Android >= Q
supportsInputEvents = Device.supportsInputEvents(displayId);
if (!supportsInputEvents) {
Ln.w("Input events are not supported for secondary displays before Android 10");
}
if (clipboardAutosync) {
// If control and autosync are enabled, synchronize Android clipboard to the computer automatically
ClipboardManager clipboardManager = ServiceManager.getClipboardManager();
if (clipboardManager != null) {
clipboardManager.addPrimaryClipChangedListener(new IOnPrimaryClipChangedListener.Stub() {
@Override
public void dispatchPrimaryClipChanged() {
if (isSettingClipboard.get()) {
// This is a notification for the change we are currently applying, ignore it
return;
}
String text = Device.getClipboardText();
if (text != null) {
DeviceMessage msg = DeviceMessage.createClipboard(text);
sender.send(msg);
}
}
});
} else {
Ln.w("No clipboard manager, copy-paste between device and computer will not work");
}
}
}
@Override
public void onNewVirtualDisplay(int virtualDisplayId, PositionMapper positionMapper) {
DisplayData data = new DisplayData(virtualDisplayId, positionMapper);
DisplayData old = this.displayData.getAndSet(data);
if (old == null) {
synchronized (displayDataAvailable) {
displayDataAvailable.notify();
}
}
}
private UhidManager getUhidManager() {
@ -85,8 +166,8 @@ public class Controller implements AsyncProcessor {
private void control() throws IOException {
// on start, power on the device
if (powerOn && !Device.isScreenOn()) {
device.pressReleaseKeycode(KeyEvent.KEYCODE_POWER, Device.INJECT_MODE_ASYNC);
if (powerOn && displayId != Device.DISPLAY_ID_NONE && !Device.isScreenOn()) {
Device.pressReleaseKeycode(KeyEvent.KEYCODE_POWER, displayId, Device.INJECT_MODE_ASYNC);
// dirty hack
// After POWER is injected, the device is powered on asynchronously.
@ -139,10 +220,6 @@ public class Controller implements AsyncProcessor {
sender.join();
}
public DeviceMessageSender getSender() {
return sender;
}
private boolean handleEvent() throws IOException {
ControlMessage msg;
try {
@ -154,27 +231,27 @@ public class Controller implements AsyncProcessor {
switch (msg.getType()) {
case ControlMessage.TYPE_INJECT_KEYCODE:
if (device.supportsInputEvents()) {
if (supportsInputEvents) {
injectKeycode(msg.getAction(), msg.getKeycode(), msg.getRepeat(), msg.getMetaState());
}
break;
case ControlMessage.TYPE_INJECT_TEXT:
if (device.supportsInputEvents()) {
if (supportsInputEvents) {
injectText(msg.getText());
}
break;
case ControlMessage.TYPE_INJECT_TOUCH_EVENT:
if (device.supportsInputEvents()) {
if (supportsInputEvents) {
injectTouch(msg.getAction(), msg.getPointerId(), msg.getPosition(), msg.getPressure(), msg.getActionButton(), msg.getButtons());
}
break;
case ControlMessage.TYPE_INJECT_SCROLL_EVENT:
if (device.supportsInputEvents()) {
if (supportsInputEvents) {
injectScroll(msg.getPosition(), msg.getHScroll(), msg.getVScroll(), msg.getButtons());
}
break;
case ControlMessage.TYPE_BACK_OR_SCREEN_ON:
if (device.supportsInputEvents()) {
if (supportsInputEvents) {
pressBackOrTurnScreenOn(msg.getAction());
}
break;
@ -194,7 +271,7 @@ public class Controller implements AsyncProcessor {
setClipboard(msg.getText(), msg.getPaste(), msg.getSequence());
break;
case ControlMessage.TYPE_SET_SCREEN_POWER_MODE:
if (device.supportsInputEvents()) {
if (supportsInputEvents) {
int mode = msg.getAction();
boolean setPowerModeOk = Device.setScreenPowerMode(mode);
if (setPowerModeOk) {
@ -208,7 +285,7 @@ public class Controller implements AsyncProcessor {
}
break;
case ControlMessage.TYPE_ROTATE_DEVICE:
device.rotateDevice();
Device.rotateDevice(getActionDisplayId());
break;
case ControlMessage.TYPE_UHID_CREATE:
getUhidManager().open(msg.getId(), msg.getText(), msg.getData());
@ -222,6 +299,9 @@ public class Controller implements AsyncProcessor {
case ControlMessage.TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
openHardKeyboardSettings();
break;
case ControlMessage.TYPE_START_APP:
startAppAsync(msg.getText());
break;
default:
// do nothing
}
@ -233,7 +313,7 @@ public class Controller implements AsyncProcessor {
if (keepPowerModeOff && action == KeyEvent.ACTION_UP && (keycode == KeyEvent.KEYCODE_POWER || keycode == KeyEvent.KEYCODE_WAKEUP)) {
schedulePowerModeOff();
}
return device.injectKeyEvent(action, keycode, repeat, metaState, Device.INJECT_MODE_ASYNC);
return injectKeyEvent(action, keycode, repeat, metaState, Device.INJECT_MODE_ASYNC);
}
private boolean injectChar(char c) {
@ -243,8 +323,10 @@ public class Controller implements AsyncProcessor {
if (events == null) {
return false;
}
int actionDisplayId = getActionDisplayId();
for (KeyEvent event : events) {
if (!device.injectEvent(event, Device.INJECT_MODE_ASYNC)) {
if (!Device.injectEvent(event, actionDisplayId, Device.INJECT_MODE_ASYNC)) {
return false;
}
}
@ -266,7 +348,12 @@ public class Controller implements AsyncProcessor {
private boolean injectTouch(int action, long pointerId, Position position, float pressure, int actionButton, int buttons) {
long now = SystemClock.uptimeMillis();
Point point = device.getPhysicalPoint(position);
// it hides the field on purpose, to read it from the atomic once
@SuppressWarnings("checkstyle:HiddenField")
DisplayData displayData = this.displayData.get();
assert displayData != null : "Cannot receive a touch event without a display";
Point point = displayData.positionMapper.map(position);
if (point == null) {
Ln.w("Ignore touch event, it was generated for a different device size");
return false;
@ -325,7 +412,7 @@ public class Controller implements AsyncProcessor {
// First button pressed: ACTION_DOWN
MotionEvent downEvent = MotionEvent.obtain(lastTouchDown, now, MotionEvent.ACTION_DOWN, pointerCount, pointerProperties,
pointerCoords, 0, buttons, 1f, 1f, DEFAULT_DEVICE_ID, 0, source, 0);
if (!device.injectEvent(downEvent, Device.INJECT_MODE_ASYNC)) {
if (!Device.injectEvent(downEvent, displayData.virtualDisplayId, Device.INJECT_MODE_ASYNC)) {
return false;
}
}
@ -336,7 +423,7 @@ public class Controller implements AsyncProcessor {
if (!InputManager.setActionButton(pressEvent, actionButton)) {
return false;
}
if (!device.injectEvent(pressEvent, Device.INJECT_MODE_ASYNC)) {
if (!Device.injectEvent(pressEvent, displayData.virtualDisplayId, Device.INJECT_MODE_ASYNC)) {
return false;
}
@ -350,7 +437,7 @@ public class Controller implements AsyncProcessor {
if (!InputManager.setActionButton(releaseEvent, actionButton)) {
return false;
}
if (!device.injectEvent(releaseEvent, Device.INJECT_MODE_ASYNC)) {
if (!Device.injectEvent(releaseEvent, displayData.virtualDisplayId, Device.INJECT_MODE_ASYNC)) {
return false;
}
@ -358,7 +445,7 @@ public class Controller implements AsyncProcessor {
// Last button released: ACTION_UP
MotionEvent upEvent = MotionEvent.obtain(lastTouchDown, now, MotionEvent.ACTION_UP, pointerCount, pointerProperties,
pointerCoords, 0, buttons, 1f, 1f, DEFAULT_DEVICE_ID, 0, source, 0);
if (!device.injectEvent(upEvent, Device.INJECT_MODE_ASYNC)) {
if (!Device.injectEvent(upEvent, displayData.virtualDisplayId, Device.INJECT_MODE_ASYNC)) {
return false;
}
}
@ -369,14 +456,20 @@ public class Controller implements AsyncProcessor {
MotionEvent event = MotionEvent.obtain(lastTouchDown, now, action, pointerCount, pointerProperties, pointerCoords, 0, buttons, 1f, 1f,
DEFAULT_DEVICE_ID, 0, source, 0);
return device.injectEvent(event, Device.INJECT_MODE_ASYNC);
return Device.injectEvent(event, displayData.virtualDisplayId, Device.INJECT_MODE_ASYNC);
}
private boolean injectScroll(Position position, float hScroll, float vScroll, int buttons) {
long now = SystemClock.uptimeMillis();
Point point = device.getPhysicalPoint(position);
// it hides the field on purpose, to read it from the atomic once
@SuppressWarnings("checkstyle:HiddenField")
DisplayData displayData = this.displayData.get();
assert displayData != null : "Cannot receive a scroll event without a display";
Point point = displayData.positionMapper.map(position);
if (point == null) {
// ignore event
Ln.w("Ignore scroll event, it was generated for a different device size");
return false;
}
@ -391,7 +484,7 @@ public class Controller implements AsyncProcessor {
MotionEvent event = MotionEvent.obtain(lastTouchDown, now, MotionEvent.ACTION_SCROLL, 1, pointerProperties, pointerCoords, 0, buttons, 1f, 1f,
DEFAULT_DEVICE_ID, 0, InputDevice.SOURCE_MOUSE, 0);
return device.injectEvent(event, Device.INJECT_MODE_ASYNC);
return Device.injectEvent(event, displayData.virtualDisplayId, Device.INJECT_MODE_ASYNC);
}
/**
@ -406,7 +499,7 @@ public class Controller implements AsyncProcessor {
private boolean pressBackOrTurnScreenOn(int action) {
if (Device.isScreenOn()) {
return device.injectKeyEvent(action, KeyEvent.KEYCODE_BACK, 0, 0, Device.INJECT_MODE_ASYNC);
return injectKeyEvent(action, KeyEvent.KEYCODE_BACK, 0, 0, Device.INJECT_MODE_ASYNC);
}
// Screen is off
@ -419,15 +512,15 @@ public class Controller implements AsyncProcessor {
if (keepPowerModeOff) {
schedulePowerModeOff();
}
return device.pressReleaseKeycode(KeyEvent.KEYCODE_POWER, Device.INJECT_MODE_ASYNC);
return pressReleaseKeycode(KeyEvent.KEYCODE_POWER, Device.INJECT_MODE_ASYNC);
}
private void getClipboard(int copyKey) {
// On Android >= 7, press the COPY or CUT key if requested
if (copyKey != ControlMessage.COPY_KEY_NONE && Build.VERSION.SDK_INT >= AndroidVersions.API_24_ANDROID_7_0 && device.supportsInputEvents()) {
if (copyKey != ControlMessage.COPY_KEY_NONE && Build.VERSION.SDK_INT >= AndroidVersions.API_24_ANDROID_7_0 && supportsInputEvents) {
int key = copyKey == ControlMessage.COPY_KEY_COPY ? KeyEvent.KEYCODE_COPY : KeyEvent.KEYCODE_CUT;
// Wait until the event is finished, to ensure that the clipboard text we read just after is the correct one
device.pressReleaseKeycode(key, Device.INJECT_MODE_WAIT_FOR_FINISH);
pressReleaseKeycode(key, Device.INJECT_MODE_WAIT_FOR_FINISH);
}
// If clipboard autosync is enabled, then the device clipboard is synchronized to the computer clipboard whenever it changes, in
@ -443,14 +536,16 @@ public class Controller implements AsyncProcessor {
}
private boolean setClipboard(String text, boolean paste, long sequence) {
boolean ok = device.setClipboardText(text);
isSettingClipboard.set(true);
boolean ok = Device.setClipboardText(text);
isSettingClipboard.set(false);
if (ok) {
Ln.i("Device clipboard set");
}
// On Android >= 7, also press the PASTE key if requested
if (paste && Build.VERSION.SDK_INT >= AndroidVersions.API_24_ANDROID_7_0 && device.supportsInputEvents()) {
device.pressReleaseKeycode(KeyEvent.KEYCODE_PASTE, Device.INJECT_MODE_ASYNC);
if (paste && Build.VERSION.SDK_INT >= AndroidVersions.API_24_ANDROID_7_0 && supportsInputEvents) {
pressReleaseKeycode(KeyEvent.KEYCODE_PASTE, Device.INJECT_MODE_ASYNC);
}
if (sequence != ControlMessage.SEQUENCE_INVALID) {
@ -466,4 +561,118 @@ public class Controller implements AsyncProcessor {
Intent intent = new Intent("android.settings.HARD_KEYBOARD_SETTINGS");
ServiceManager.getActivityManager().startActivity(intent);
}
private boolean injectKeyEvent(int action, int keyCode, int repeat, int metaState, int injectMode) {
return Device.injectKeyEvent(action, keyCode, repeat, metaState, getActionDisplayId(), injectMode);
}
private boolean pressReleaseKeycode(int keyCode, int injectMode) {
return Device.pressReleaseKeycode(keyCode, getActionDisplayId(), injectMode);
}
private int getActionDisplayId() {
if (displayId != Device.DISPLAY_ID_NONE) {
// Real screen mirrored, use the source display id
return displayId;
}
// Virtual display created by --new-display, use the virtualDisplayId
DisplayData data = displayData.get();
if (data == null) {
// If no virtual display id is initialized yet, use the main display id
return 0;
}
return data.virtualDisplayId;
}
private void startAppAsync(String name) {
if (startAppExecutor == null) {
startAppExecutor = Executors.newSingleThreadExecutor();
}
// Listing and selecting the app may take a lot of time
startAppExecutor.submit(() -> startApp(name));
}
private void startApp(String name) {
boolean forceStopBeforeStart = name.startsWith("+");
if (forceStopBeforeStart) {
name = name.substring(1);
}
DeviceApp app;
boolean searchByName = name.startsWith("?");
if (searchByName) {
name = name.substring(1);
Ln.i("Processing Android apps... (this may take some time)");
List<DeviceApp> apps = Device.findByName(name);
if (apps.isEmpty()) {
Ln.w("No app found for name \"" + name + "\"");
return;
}
if (apps.size() > 1) {
String title = "No unique app found for name \"" + name + "\":";
Ln.w(LogUtils.buildAppListMessage(title, apps));
return;
}
app = apps.get(0);
} else {
app = Device.findByPackageName(name);
if (app == null) {
Ln.w("No app found for package \"" + name + "\"");
return;
}
}
int startAppDisplayId = getStartAppDisplayId();
if (startAppDisplayId == Device.DISPLAY_ID_NONE) {
Ln.e("No known display id to start app \"" + name + "\"");
return;
}
Ln.i("Starting app \"" + app.getName() + "\" [" + app.getPackageName() + "] on display " + startAppDisplayId + "...");
Device.startApp(app.getPackageName(), startAppDisplayId, forceStopBeforeStart);
}
private int getStartAppDisplayId() {
if (displayId != Device.DISPLAY_ID_NONE) {
return displayId;
}
// Mirroring a new virtual display id (using --new-display-id feature)
try {
// Wait for at most 1 second until a virtual display id is known
DisplayData data = waitDisplayData(1000);
if (data != null) {
return data.virtualDisplayId;
}
} catch (InterruptedException e) {
// do nothing
}
// No display id available
return Device.DISPLAY_ID_NONE;
}
private DisplayData waitDisplayData(long timeoutMillis) throws InterruptedException {
long deadline = System.currentTimeMillis() + timeoutMillis;
synchronized (displayDataAvailable) {
DisplayData data = displayData.get();
while (data == null) {
long timeout = deadline - System.currentTimeMillis();
if (timeout < 0) {
return null;
}
displayDataAvailable.wait(timeout);
data = displayData.get();
}
return data;
}
}
}

View File

@ -0,0 +1,46 @@
package com.genymobile.scrcpy.control;
import com.genymobile.scrcpy.device.Point;
import com.genymobile.scrcpy.device.Position;
import com.genymobile.scrcpy.device.Size;
import com.genymobile.scrcpy.video.ScreenInfo;
import android.graphics.Rect;
public final class PositionMapper {
private final Size videoSize;
private final Rect contentRect;
private final int coordsRotation;
public PositionMapper(Size videoSize, Rect contentRect, int videoRotation) {
this.videoSize = videoSize;
this.contentRect = contentRect;
this.coordsRotation = reverseRotation(videoRotation);
}
public static PositionMapper from(ScreenInfo screenInfo) {
return new PositionMapper(screenInfo.getUnlockedVideoSize(), screenInfo.getContentRect(), screenInfo.getVideoRotation());
}
private static int reverseRotation(int rotation) {
return (4 - rotation) % 4;
}
public Point map(Position position) {
// reverse the video rotation to apply the events
Position devicePosition = position.rotate(coordsRotation);
Size clientVideoSize = devicePosition.getScreenSize();
if (!videoSize.equals(clientVideoSize)) {
// The client sends a click relative to a video with wrong dimensions,
// the device may have been rotated since the event was generated, so ignore the event
return null;
}
Point point = devicePosition.getPoint();
int convertedX = contentRect.left + point.getX() * contentRect.width() / videoSize.getWidth();
int convertedY = contentRect.top + point.getY() * contentRect.height() / videoSize.getHeight();
return new Point(convertedX, convertedY);
}
}

View File

@ -1,10 +1,9 @@
package com.genymobile.scrcpy.device;
import com.genymobile.scrcpy.AndroidVersions;
import com.genymobile.scrcpy.Options;
import com.genymobile.scrcpy.FakeContext;
import com.genymobile.scrcpy.util.Ln;
import com.genymobile.scrcpy.util.LogUtils;
import com.genymobile.scrcpy.video.ScreenInfo;
import com.genymobile.scrcpy.wrappers.ActivityManager;
import com.genymobile.scrcpy.wrappers.ClipboardManager;
import com.genymobile.scrcpy.wrappers.DisplayControl;
import com.genymobile.scrcpy.wrappers.InputManager;
@ -12,22 +11,28 @@ import com.genymobile.scrcpy.wrappers.ServiceManager;
import com.genymobile.scrcpy.wrappers.SurfaceControl;
import com.genymobile.scrcpy.wrappers.WindowManager;
import android.content.IOnPrimaryClipChangedListener;
import android.graphics.Rect;
import android.annotation.SuppressLint;
import android.content.Intent;
import android.app.ActivityOptions;
import android.content.pm.ApplicationInfo;
import android.content.pm.PackageManager;
import android.os.Build;
import android.os.Bundle;
import android.os.IBinder;
import android.os.SystemClock;
import android.view.IDisplayFoldListener;
import android.view.IRotationWatcher;
import android.view.InputDevice;
import android.view.InputEvent;
import android.view.KeyCharacterMap;
import android.view.KeyEvent;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
public final class Device {
public static final int DISPLAY_ID_NONE = -1;
public static final int POWER_MODE_OFF = SurfaceControl.POWER_MODE_OFF;
public static final int POWER_MODE_NORMAL = SurfaceControl.POWER_MODE_NORMAL;
@ -38,177 +43,8 @@ public final class Device {
public static final int LOCK_VIDEO_ORIENTATION_UNLOCKED = -1;
public static final int LOCK_VIDEO_ORIENTATION_INITIAL = -2;
public interface RotationListener {
void onRotationChanged(int rotation);
}
public interface FoldListener {
void onFoldChanged(int displayId, boolean folded);
}
public interface ClipboardListener {
void onClipboardTextChanged(String text);
}
private final Rect crop;
private int maxSize;
private final int lockVideoOrientation;
private Size deviceSize;
private ScreenInfo screenInfo;
private RotationListener rotationListener;
private FoldListener foldListener;
private ClipboardListener clipboardListener;
private final AtomicBoolean isSettingClipboard = new AtomicBoolean();
/**
* Logical display identifier
*/
private final int displayId;
/**
* The surface flinger layer stack associated with this logical display
*/
private final int layerStack;
private final boolean supportsInputEvents;
public Device(Options options) throws ConfigurationException {
displayId = options.getDisplayId();
DisplayInfo displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(displayId);
if (displayInfo == null) {
Ln.e("Display " + displayId + " not found\n" + LogUtils.buildDisplayListMessage());
throw new ConfigurationException("Unknown display id: " + displayId);
}
int displayInfoFlags = displayInfo.getFlags();
deviceSize = displayInfo.getSize();
crop = options.getCrop();
maxSize = options.getMaxSize();
lockVideoOrientation = options.getLockVideoOrientation();
screenInfo = ScreenInfo.computeScreenInfo(displayInfo.getRotation(), deviceSize, crop, maxSize, lockVideoOrientation);
layerStack = displayInfo.getLayerStack();
ServiceManager.getWindowManager().registerRotationWatcher(new IRotationWatcher.Stub() {
@Override
public void onRotationChanged(int rotation) {
synchronized (Device.this) {
screenInfo = screenInfo.withDeviceRotation(rotation);
// notify
if (rotationListener != null) {
rotationListener.onRotationChanged(rotation);
}
}
}
}, displayId);
if (Build.VERSION.SDK_INT >= AndroidVersions.API_29_ANDROID_10) {
ServiceManager.getWindowManager().registerDisplayFoldListener(new IDisplayFoldListener.Stub() {
@Override
public void onDisplayFoldChanged(int displayId, boolean folded) {
if (Device.this.displayId != displayId) {
// Ignore events related to other display ids
return;
}
synchronized (Device.this) {
DisplayInfo displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(displayId);
if (displayInfo == null) {
Ln.e("Display " + displayId + " not found\n" + LogUtils.buildDisplayListMessage());
return;
}
deviceSize = displayInfo.getSize();
screenInfo = ScreenInfo.computeScreenInfo(displayInfo.getRotation(), deviceSize, crop, maxSize, lockVideoOrientation);
// notify
if (foldListener != null) {
foldListener.onFoldChanged(displayId, folded);
}
}
}
});
}
if (options.getControl() && options.getClipboardAutosync()) {
// If control and autosync are enabled, synchronize Android clipboard to the computer automatically
ClipboardManager clipboardManager = ServiceManager.getClipboardManager();
if (clipboardManager != null) {
clipboardManager.addPrimaryClipChangedListener(new IOnPrimaryClipChangedListener.Stub() {
@Override
public void dispatchPrimaryClipChanged() {
if (isSettingClipboard.get()) {
// This is a notification for the change we are currently applying, ignore it
return;
}
synchronized (Device.this) {
if (clipboardListener != null) {
String text = getClipboardText();
if (text != null) {
clipboardListener.onClipboardTextChanged(text);
}
}
}
}
});
} else {
Ln.w("No clipboard manager, copy-paste between device and computer will not work");
}
}
if ((displayInfoFlags & DisplayInfo.FLAG_SUPPORTS_PROTECTED_BUFFERS) == 0) {
Ln.w("Display doesn't have FLAG_SUPPORTS_PROTECTED_BUFFERS flag, mirroring can be restricted");
}
// main display or any display on Android >= 10
supportsInputEvents = displayId == 0 || Build.VERSION.SDK_INT >= AndroidVersions.API_29_ANDROID_10;
if (!supportsInputEvents) {
Ln.w("Input events are not supported for secondary displays before Android 10");
}
}
public int getDisplayId() {
return displayId;
}
public synchronized void setMaxSize(int newMaxSize) {
maxSize = newMaxSize;
screenInfo = ScreenInfo.computeScreenInfo(screenInfo.getReverseVideoRotation(), deviceSize, crop, newMaxSize, lockVideoOrientation);
}
public synchronized ScreenInfo getScreenInfo() {
return screenInfo;
}
public int getLayerStack() {
return layerStack;
}
public Point getPhysicalPoint(Position position) {
// it hides the field on purpose, to read it with a lock
@SuppressWarnings("checkstyle:HiddenField")
ScreenInfo screenInfo = getScreenInfo(); // read with synchronization
// ignore the locked video orientation, the events will apply in coordinates considered in the physical device orientation
Size unlockedVideoSize = screenInfo.getUnlockedVideoSize();
int reverseVideoRotation = screenInfo.getReverseVideoRotation();
// reverse the video rotation to apply the events
Position devicePosition = position.rotate(reverseVideoRotation);
Size clientVideoSize = devicePosition.getScreenSize();
if (!unlockedVideoSize.equals(clientVideoSize)) {
// The client sends a click relative to a video with wrong dimensions,
// the device may have been rotated since the event was generated, so ignore the event
return null;
}
Rect contentRect = screenInfo.getContentRect();
Point point = devicePosition.getPoint();
int convertedX = contentRect.left + point.getX() * contentRect.width() / unlockedVideoSize.getWidth();
int convertedY = contentRect.top + point.getY() * contentRect.height() / unlockedVideoSize.getHeight();
return new Point(convertedX, convertedY);
private Device() {
// not instantiable
}
public static String getDeviceName() {
@ -219,10 +55,6 @@ public final class Device {
return displayId == 0 || Build.VERSION.SDK_INT >= AndroidVersions.API_29_ANDROID_10;
}
public boolean supportsInputEvents() {
return supportsInputEvents;
}
public static boolean injectEvent(InputEvent inputEvent, int displayId, int injectMode) {
if (!supportsInputEvents(displayId)) {
throw new AssertionError("Could not inject input event if !supportsInputEvents()");
@ -235,10 +67,6 @@ public final class Device {
return ServiceManager.getInputManager().injectInputEvent(inputEvent, injectMode);
}
public boolean injectEvent(InputEvent event, int injectMode) {
return injectEvent(event, displayId, injectMode);
}
public static boolean injectKeyEvent(int action, int keyCode, int repeat, int metaState, int displayId, int injectMode) {
long now = SystemClock.uptimeMillis();
KeyEvent event = new KeyEvent(now, now, action, keyCode, repeat, metaState, KeyCharacterMap.VIRTUAL_KEYBOARD, 0, 0,
@ -246,35 +74,15 @@ public final class Device {
return injectEvent(event, displayId, injectMode);
}
public boolean injectKeyEvent(int action, int keyCode, int repeat, int metaState, int injectMode) {
return injectKeyEvent(action, keyCode, repeat, metaState, displayId, injectMode);
}
public static boolean pressReleaseKeycode(int keyCode, int displayId, int injectMode) {
return injectKeyEvent(KeyEvent.ACTION_DOWN, keyCode, 0, 0, displayId, injectMode)
&& injectKeyEvent(KeyEvent.ACTION_UP, keyCode, 0, 0, displayId, injectMode);
}
public boolean pressReleaseKeycode(int keyCode, int injectMode) {
return pressReleaseKeycode(keyCode, displayId, injectMode);
}
public static boolean isScreenOn() {
return ServiceManager.getPowerManager().isScreenOn();
}
public synchronized void setRotationListener(RotationListener rotationListener) {
this.rotationListener = rotationListener;
}
public synchronized void setFoldListener(FoldListener foldlistener) {
this.foldListener = foldlistener;
}
public synchronized void setClipboardListener(ClipboardListener clipboardListener) {
this.clipboardListener = clipboardListener;
}
public static void expandNotificationPanel() {
ServiceManager.getStatusBarManager().expandNotificationsPanel();
}
@ -299,7 +107,7 @@ public final class Device {
return s.toString();
}
public boolean setClipboardText(String text) {
public static boolean setClipboardText(String text) {
ClipboardManager clipboardManager = ServiceManager.getClipboardManager();
if (clipboardManager == null) {
return false;
@ -314,10 +122,7 @@ public final class Device {
return false;
}
isSettingClipboard.set(true);
boolean ok = clipboardManager.setText(text);
isSettingClipboard.set(false);
return ok;
return clipboardManager.setText(text);
}
/**
@ -367,6 +172,8 @@ public final class Device {
}
public static boolean powerOffScreen(int displayId) {
assert displayId != DISPLAY_ID_NONE;
if (!isScreenOn()) {
return true;
}
@ -376,7 +183,9 @@ public final class Device {
/**
* Disable auto-rotation (if enabled), set the screen rotation and re-enable auto-rotation (if it was enabled).
*/
public void rotateDevice() {
public static void rotateDevice(int displayId) {
assert displayId != DISPLAY_ID_NONE;
WindowManager wm = ServiceManager.getWindowManager();
boolean accelerometerRotation = !wm.isRotationFrozen(displayId);
@ -395,6 +204,8 @@ public final class Device {
}
private static int getCurrentRotation(int displayId) {
assert displayId != DISPLAY_ID_NONE;
if (displayId == 0) {
return ServiceManager.getWindowManager().getRotation();
}
@ -402,4 +213,98 @@ public final class Device {
DisplayInfo displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(displayId);
return displayInfo.getRotation();
}
public static List<DeviceApp> listApps() {
List<DeviceApp> apps = new ArrayList<>();
PackageManager pm = FakeContext.get().getPackageManager();
for (ApplicationInfo appInfo : getLaunchableApps(pm)) {
apps.add(toApp(pm, appInfo));
}
return apps;
}
@SuppressLint("QueryPermissionsNeeded")
private static List<ApplicationInfo> getLaunchableApps(PackageManager pm) {
List<ApplicationInfo> result = new ArrayList<>();
for (ApplicationInfo appInfo : pm.getInstalledApplications(PackageManager.GET_META_DATA)) {
if (getLaunchIntent(pm, appInfo.packageName) != null) {
result.add(appInfo);
}
}
return result;
}
public static Intent getLaunchIntent(PackageManager pm, String packageName) {
Intent launchIntent = pm.getLaunchIntentForPackage(packageName);
if (launchIntent != null) {
return launchIntent;
}
return pm.getLeanbackLaunchIntentForPackage(packageName);
}
private static DeviceApp toApp(PackageManager pm, ApplicationInfo appInfo) {
String name = pm.getApplicationLabel(appInfo).toString();
boolean system = (appInfo.flags & ApplicationInfo.FLAG_SYSTEM) != 0;
return new DeviceApp(appInfo.packageName, name, system, appInfo.enabled);
}
@SuppressLint("QueryPermissionsNeeded")
public static DeviceApp findByPackageName(String packageName) {
PackageManager pm = FakeContext.get().getPackageManager();
// No need to filter by "launchable" apps, an error will be reported on start if the app is not launchable
for (ApplicationInfo appInfo : pm.getInstalledApplications(PackageManager.GET_META_DATA)) {
if (packageName.equals(appInfo.packageName)) {
return toApp(pm, appInfo);
}
}
return null;
}
@SuppressLint("QueryPermissionsNeeded")
public static List<DeviceApp> findByName(String searchName) {
List<DeviceApp> result = new ArrayList<>();
searchName = searchName.toLowerCase(Locale.getDefault());
PackageManager pm = FakeContext.get().getPackageManager();
for (ApplicationInfo appInfo : getLaunchableApps(pm)) {
if (appInfo.enabled) {
String name = pm.getApplicationLabel(appInfo).toString();
if (name.toLowerCase(Locale.getDefault()).startsWith(searchName)) {
boolean system = (appInfo.flags & ApplicationInfo.FLAG_SYSTEM) != 0;
result.add(new DeviceApp(appInfo.packageName, name, system, appInfo.enabled));
}
}
}
return result;
}
public static void startApp(String packageName, int displayId, boolean forceStop) {
PackageManager pm = FakeContext.get().getPackageManager();
Intent launchIntent = getLaunchIntent(pm, packageName);
if (launchIntent == null) {
Ln.w("Cannot create launch intent for app " + packageName);
return;
}
launchIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
Bundle options = null;
if (Build.VERSION.SDK_INT >= AndroidVersions.API_26_ANDROID_8_0) {
ActivityOptions launchOptions = ActivityOptions.makeBasic();
launchOptions.setLaunchDisplayId(displayId);
options = launchOptions.toBundle();
}
ActivityManager am = ServiceManager.getActivityManager();
if (forceStop) {
am.forceStopPackage(packageName);
}
am.startActivity(launchIntent, options);
}
}

View File

@ -0,0 +1,32 @@
package com.genymobile.scrcpy.device;
public final class DeviceApp {
private final String packageName;
private final String name;
private final boolean system;
private final boolean enabled;
public DeviceApp(String packageName, String name, boolean system, boolean enabled) {
this.packageName = packageName;
this.name = name;
this.system = system;
this.enabled = enabled;
}
public String getPackageName() {
return packageName;
}
public String getName() {
return name;
}
public boolean isSystem() {
return system;
}
public boolean isEnabled() {
return enabled;
}
}

View File

@ -6,15 +6,17 @@ public final class DisplayInfo {
private final int rotation;
private final int layerStack;
private final int flags;
private final int dpi;
public static final int FLAG_SUPPORTS_PROTECTED_BUFFERS = 0x00000001;
public DisplayInfo(int displayId, Size size, int rotation, int layerStack, int flags) {
public DisplayInfo(int displayId, Size size, int rotation, int layerStack, int flags, int dpi) {
this.displayId = displayId;
this.size = size;
this.rotation = rotation;
this.layerStack = layerStack;
this.flags = flags;
this.dpi = dpi;
}
public int getDisplayId() {
@ -36,5 +38,9 @@ public final class DisplayInfo {
public int getFlags() {
return flags;
}
public int getDpi() {
return dpi;
}
}

View File

@ -0,0 +1,31 @@
package com.genymobile.scrcpy.device;
public final class NewDisplay {
private Size size;
private int dpi;
public NewDisplay() {
// Auto size and dpi
}
public NewDisplay(Size size, int dpi) {
this.size = size;
this.dpi = dpi;
}
public Size getSize() {
return size;
}
public int getDpi() {
return dpi;
}
public boolean hasExplicitSize() {
return size != null;
}
public boolean hasExplicitDpi() {
return dpi != 0;
}
}

View File

@ -21,6 +21,10 @@ public final class Size {
return height;
}
public int getMax() {
return Math.max(width, height);
}
public Size rotate() {
return new Size(height, width);
}

View File

@ -1,10 +1,13 @@
package com.genymobile.scrcpy.util;
import com.genymobile.scrcpy.device.Device;
import com.genymobile.scrcpy.device.DeviceApp;
import com.genymobile.scrcpy.device.DisplayInfo;
import com.genymobile.scrcpy.device.Size;
import com.genymobile.scrcpy.wrappers.DisplayManager;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import android.annotation.SuppressLint;
import android.graphics.Rect;
import android.hardware.camera2.CameraAccessException;
import android.hardware.camera2.CameraCharacteristics;
@ -13,7 +16,9 @@ import android.hardware.camera2.params.StreamConfigurationMap;
import android.media.MediaCodec;
import android.util.Range;
import java.util.Collections;
import java.util.List;
import java.util.Objects;
import java.util.SortedSet;
import java.util.TreeSet;
@ -154,4 +159,59 @@ public final class LogUtils {
}
return set;
}
public static String buildAppListMessage() {
List<DeviceApp> apps = Device.listApps();
return buildAppListMessage("List of apps:", apps);
}
@SuppressLint("QueryPermissionsNeeded")
public static String buildAppListMessage(String title, List<DeviceApp> apps) {
StringBuilder builder = new StringBuilder(title);
// Sort by:
// 1. system flag (system apps are before non-system apps)
// 2. name
// 3. package name
Collections.sort(apps, (thisApp, otherApp) -> {
// System apps first
int cmp = -Boolean.compare(thisApp.isSystem(), otherApp.isSystem());
if (cmp != 0) {
return cmp;
}
cmp = Objects.compare(thisApp.getName(), otherApp.getName(), String::compareTo);
if (cmp != 0) {
return cmp;
}
return Objects.compare(thisApp.getPackageName(), otherApp.getPackageName(), String::compareTo);
});
final int column = 30;
for (DeviceApp app : apps) {
String name = app.getName();
int padding = column - name.length();
builder.append("\n ");
if (app.isSystem()) {
builder.append("* ");
} else {
builder.append("- ");
}
builder.append(name);
if (padding > 0) {
builder.append(String.format("%" + padding + "s", " "));
} else {
builder.append("\n ").append(String.format("%" + column + "s", " "));
}
builder.append(" [").append(app.getPackageName()).append(']');
if (!app.isEnabled()) {
builder.append(" (disabled)");
}
}
return builder.toString();
}
}

View File

@ -0,0 +1,146 @@
package com.genymobile.scrcpy.video;
import com.genymobile.scrcpy.AndroidVersions;
import com.genymobile.scrcpy.control.PositionMapper;
import com.genymobile.scrcpy.device.DisplayInfo;
import com.genymobile.scrcpy.device.NewDisplay;
import com.genymobile.scrcpy.device.Size;
import com.genymobile.scrcpy.util.Ln;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import android.graphics.Rect;
import android.hardware.display.DisplayManager;
import android.hardware.display.VirtualDisplay;
import android.os.Build;
import android.view.Surface;
public class NewDisplayCapture extends SurfaceCapture {
// Internal fields copied from android.hardware.display.DisplayManager
private static final int VIRTUAL_DISPLAY_FLAG_SUPPORTS_TOUCH = 1 << 6;
private static final int VIRTUAL_DISPLAY_FLAG_ROTATES_WITH_CONTENT = 1 << 7;
private static final int VIRTUAL_DISPLAY_FLAG_DESTROY_CONTENT_ON_REMOVAL = 1 << 8;
private static final int VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS = 1 << 9;
private static final int VIRTUAL_DISPLAY_FLAG_TRUSTED = 1 << 10;
private static final int VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP = 1 << 11;
private static final int VIRTUAL_DISPLAY_FLAG_ALWAYS_UNLOCKED = 1 << 12;
private static final int VIRTUAL_DISPLAY_FLAG_TOUCH_FEEDBACK_DISABLED = 1 << 13;
private static final int VIRTUAL_DISPLAY_FLAG_OWN_FOCUS = 1 << 14;
private static final int VIRTUAL_DISPLAY_FLAG_DEVICE_DISPLAY_GROUP = 1 << 15;
private final VirtualDisplayListener vdListener;
private final NewDisplay newDisplay;
private Size mainDisplaySize;
private int mainDisplayDpi;
private int maxSize; // only used if newDisplay.getSize() != null
private VirtualDisplay virtualDisplay;
private Size size;
private int dpi;
public NewDisplayCapture(VirtualDisplayListener vdListener, NewDisplay newDisplay, int maxSize) {
this.vdListener = vdListener;
this.newDisplay = newDisplay;
this.maxSize = maxSize;
}
@Override
public void init() {
size = newDisplay.getSize();
dpi = newDisplay.getDpi();
if (size == null || dpi == 0) {
DisplayInfo displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(0);
if (displayInfo != null) {
mainDisplaySize = displayInfo.getSize();
mainDisplayDpi = displayInfo.getDpi();
} else {
Ln.w("Main display not found, fallback to 1920x1080 240dpi");
mainDisplaySize = new Size(1920, 1080);
mainDisplayDpi = 240;
}
}
}
@Override
public void prepare() {
if (!newDisplay.hasExplicitSize()) {
size = ScreenInfo.computeVideoSize(mainDisplaySize.getWidth(), mainDisplaySize.getHeight(), maxSize);
}
if (!newDisplay.hasExplicitDpi()) {
dpi = scaleDpi(mainDisplaySize, mainDisplayDpi, size);
}
}
@Override
public void start(Surface surface) {
if (virtualDisplay != null) {
virtualDisplay.release();
virtualDisplay = null;
}
int virtualDisplayId;
try {
int flags = DisplayManager.VIRTUAL_DISPLAY_FLAG_PUBLIC
| DisplayManager.VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY
| VIRTUAL_DISPLAY_FLAG_SUPPORTS_TOUCH
| VIRTUAL_DISPLAY_FLAG_ROTATES_WITH_CONTENT
| VIRTUAL_DISPLAY_FLAG_DESTROY_CONTENT_ON_REMOVAL
| VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS;
if (Build.VERSION.SDK_INT >= AndroidVersions.API_33_ANDROID_13) {
flags |= VIRTUAL_DISPLAY_FLAG_TRUSTED
| VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP
| VIRTUAL_DISPLAY_FLAG_ALWAYS_UNLOCKED
| VIRTUAL_DISPLAY_FLAG_TOUCH_FEEDBACK_DISABLED;
if (Build.VERSION.SDK_INT >= AndroidVersions.API_34_ANDROID_14) {
flags |= VIRTUAL_DISPLAY_FLAG_OWN_FOCUS
| VIRTUAL_DISPLAY_FLAG_DEVICE_DISPLAY_GROUP;
}
}
virtualDisplay = ServiceManager.getDisplayManager()
.createNewVirtualDisplay("scrcpy", size.getWidth(), size.getHeight(), dpi, surface, flags);
virtualDisplayId = virtualDisplay.getDisplay().getDisplayId();
Ln.i("New display id: " + virtualDisplayId);
} catch (Exception e) {
Ln.e("Could not create display", e);
throw new AssertionError("Could not create display");
}
if (vdListener != null) {
virtualDisplayId = virtualDisplay.getDisplay().getDisplayId();
Rect contentRect = new Rect(0, 0, size.getWidth(), size.getHeight());
PositionMapper positionMapper = new PositionMapper(size, contentRect, 0);
vdListener.onNewVirtualDisplay(virtualDisplayId, positionMapper);
}
}
@Override
public void release() {
if (virtualDisplay != null) {
virtualDisplay.release();
virtualDisplay = null;
}
}
@Override
public synchronized Size getSize() {
return size;
}
@Override
public synchronized boolean setMaxSize(int newMaxSize) {
if (newDisplay.hasExplicitSize()) {
// Cannot retry with a different size if the display size was explicitly provided
return false;
}
maxSize = newMaxSize;
return true;
}
private static int scaleDpi(Size initialSize, int initialDpi, Size size) {
int den = initialSize.getMax();
int num = size.getMax();
return initialDpi * num / den;
}
}

View File

@ -1,44 +1,97 @@
package com.genymobile.scrcpy.video;
import com.genymobile.scrcpy.AndroidVersions;
import com.genymobile.scrcpy.device.Device;
import com.genymobile.scrcpy.control.PositionMapper;
import com.genymobile.scrcpy.device.DisplayInfo;
import com.genymobile.scrcpy.device.Size;
import com.genymobile.scrcpy.util.Ln;
import com.genymobile.scrcpy.util.LogUtils;
import com.genymobile.scrcpy.wrappers.DisplayManager;
import com.genymobile.scrcpy.wrappers.ServiceManager;
import com.genymobile.scrcpy.wrappers.SurfaceControl;
import android.graphics.Rect;
import android.hardware.display.VirtualDisplay;
import android.os.Build;
import android.os.Handler;
import android.os.HandlerThread;
import android.os.IBinder;
import android.view.IDisplayFoldListener;
import android.view.IRotationWatcher;
import android.view.Surface;
public class ScreenCapture extends SurfaceCapture implements Device.RotationListener, Device.FoldListener {
public class ScreenCapture extends SurfaceCapture {
private final VirtualDisplayListener vdListener;
private final int displayId;
private int maxSize;
private final Rect crop;
private final int lockVideoOrientation;
private DisplayInfo displayInfo;
private ScreenInfo screenInfo;
private final Device device;
private IBinder display;
private VirtualDisplay virtualDisplay;
public ScreenCapture(Device device) {
this.device = device;
private DisplayManager.DisplayListenerHandle displayListenerHandle;
private HandlerThread handlerThread;
// On Android 14, the DisplayListener may be broken (it never send events). This is fixed in recent Android 14 upgrades, but we can't really know.
// So register a RotationWatcher and a DisplayFoldListener as a fallback, until we receive the first event from DisplayListener (which proves
// that it works).
private boolean displayListenerWorks; // only accessed from the display listener thread
private IRotationWatcher rotationWatcher;
private IDisplayFoldListener displayFoldListener;
public ScreenCapture(VirtualDisplayListener vdListener, int displayId, int maxSize, Rect crop, int lockVideoOrientation) {
this.vdListener = vdListener;
this.displayId = displayId;
this.maxSize = maxSize;
this.crop = crop;
this.lockVideoOrientation = lockVideoOrientation;
}
@Override
public void init() {
device.setRotationListener(this);
device.setFoldListener(this);
if (Build.VERSION.SDK_INT == AndroidVersions.API_34_ANDROID_14) {
registerDisplayListenerFallbacks();
}
handlerThread = new HandlerThread("DisplayListener");
handlerThread.start();
Handler handler = new Handler(handlerThread.getLooper());
displayListenerHandle = ServiceManager.getDisplayManager().registerDisplayListener(displayId -> {
if (Build.VERSION.SDK_INT == AndroidVersions.API_34_ANDROID_14) {
if (!displayListenerWorks) {
// On the first display listener event, we know it works, we can unregister the fallbacks
displayListenerWorks = true;
unregisterDisplayListenerFallbacks();
}
}
if (this.displayId == displayId) {
requestReset();
}
}, handler);
}
@Override
public void prepare() {
displayInfo = ServiceManager.getDisplayManager().getDisplayInfo(displayId);
if (displayInfo == null) {
Ln.e("Display " + displayId + " not found\n" + LogUtils.buildDisplayListMessage());
throw new AssertionError("Display " + display + " not found");
}
if ((displayInfo.getFlags() & DisplayInfo.FLAG_SUPPORTS_PROTECTED_BUFFERS) == 0) {
Ln.w("Display doesn't have FLAG_SUPPORTS_PROTECTED_BUFFERS flag, mirroring can be restricted");
}
screenInfo = ScreenInfo.computeScreenInfo(displayInfo.getRotation(), displayInfo.getSize(), crop, maxSize, lockVideoOrientation);
}
@Override
public void start(Surface surface) {
ScreenInfo screenInfo = device.getScreenInfo();
Rect contentRect = screenInfo.getContentRect();
// does not include the locked video orientation
Rect unlockedVideoRect = screenInfo.getUnlockedVideoSize().toRect();
int videoRotation = screenInfo.getVideoRotation();
int layerStack = device.getLayerStack();
if (display != null) {
SurfaceControl.destroyDisplay(display);
display = null;
@ -48,15 +101,32 @@ public class ScreenCapture extends SurfaceCapture implements Device.RotationList
virtualDisplay = null;
}
Size displaySize = screenInfo.getVideoSize();
int virtualDisplayId;
PositionMapper positionMapper;
try {
Rect videoRect = screenInfo.getVideoSize().toRect();
virtualDisplay = ServiceManager.getDisplayManager()
.createVirtualDisplay("scrcpy", videoRect.width(), videoRect.height(), device.getDisplayId(), surface);
.createVirtualDisplay("scrcpy", displaySize.getWidth(), displaySize.getHeight(), displayId, surface);
virtualDisplayId = virtualDisplay.getDisplay().getDisplayId();
Rect contentRect = new Rect(0, 0, displaySize.getWidth(), displaySize.getHeight());
// The position are relative to the virtual display, not the original display
positionMapper = new PositionMapper(displaySize, contentRect, 0);
Ln.d("Display: using DisplayManager API");
} catch (Exception displayManagerException) {
try {
display = createDisplay();
Rect contentRect = screenInfo.getContentRect();
// does not include the locked video orientation
Rect unlockedVideoRect = screenInfo.getUnlockedVideoSize().toRect();
int videoRotation = screenInfo.getVideoRotation();
int layerStack = displayInfo.getLayerStack();
setDisplaySurface(display, surface, videoRotation, contentRect, unlockedVideoRect, layerStack);
virtualDisplayId = displayId;
positionMapper = PositionMapper.from(screenInfo);
Ln.d("Display: using SurfaceControl API");
} catch (Exception surfaceControlException) {
Ln.e("Could not create display using DisplayManager", displayManagerException);
@ -64,12 +134,19 @@ public class ScreenCapture extends SurfaceCapture implements Device.RotationList
throw new AssertionError("Could not create display");
}
}
if (vdListener != null) {
vdListener.onNewVirtualDisplay(virtualDisplayId, positionMapper);
}
}
@Override
public void release() {
device.setRotationListener(null);
device.setFoldListener(null);
if (Build.VERSION.SDK_INT == AndroidVersions.API_34_ANDROID_14) {
unregisterDisplayListenerFallbacks();
}
handlerThread.quitSafely();
ServiceManager.getDisplayManager().unregisterDisplayListener(displayListenerHandle);
if (display != null) {
SurfaceControl.destroyDisplay(display);
display = null;
@ -82,25 +159,15 @@ public class ScreenCapture extends SurfaceCapture implements Device.RotationList
@Override
public Size getSize() {
return device.getScreenInfo().getVideoSize();
return screenInfo.getVideoSize();
}
@Override
public boolean setMaxSize(int maxSize) {
device.setMaxSize(maxSize);
public boolean setMaxSize(int newMaxSize) {
maxSize = newMaxSize;
return true;
}
@Override
public void onFoldChanged(int displayId, boolean folded) {
requestReset();
}
@Override
public void onRotationChanged(int rotation) {
requestReset();
}
private static IBinder createDisplay() throws Exception {
// Since Android 12 (preview), secure displays could not be created with shell permissions anymore.
// On Android 12 preview, SDK_INT is still R (not S), but CODENAME is "S".
@ -119,4 +186,45 @@ public class ScreenCapture extends SurfaceCapture implements Device.RotationList
SurfaceControl.closeTransaction();
}
}
private void registerDisplayListenerFallbacks() {
if (displayId == 0) {
rotationWatcher = new IRotationWatcher.Stub() {
@Override
public void onRotationChanged(int rotation) {
Ln.i("=== rotation");
requestReset();
}
};
ServiceManager.getWindowManager().registerRotationWatcher(rotationWatcher, displayId);
}
// Build.VERSION.SDK_INT >= AndroidVersions.API_29_ANDROID_10 (but implied by == API_34_ANDROID 14)
displayFoldListener = new IDisplayFoldListener.Stub() {
@Override
public void onDisplayFoldChanged(int displayId, boolean folded) {
if (ScreenCapture.this.displayId != displayId) {
// Ignore events related to other display ids
return;
}
requestReset();
}
};
ServiceManager.getWindowManager().registerDisplayFoldListener(displayFoldListener);
}
private void unregisterDisplayListenerFallbacks() {
synchronized (this) {
if (rotationWatcher != null) {
ServiceManager.getWindowManager().unregisterRotationWatcher(rotationWatcher);
rotationWatcher = null;
}
if (displayFoldListener != null) {
// Build.VERSION.SDK_INT >= AndroidVersions.API_29_ANDROID_10 (but implied by == API_34_ANDROID 14)
ServiceManager.getWindowManager().unregisterDisplayFoldListener(displayFoldListener);
displayFoldListener = null;
}
}
}
}

View File

@ -112,7 +112,7 @@ public final class ScreenInfo {
return rect.width() + ":" + rect.height() + ":" + rect.left + ":" + rect.top;
}
private static Size computeVideoSize(int w, int h, int maxSize) {
public static Size computeVideoSize(int w, int h, int maxSize) {
// Compute the video size and the padding of the content inside this video.
// Principle:
// - scale down the great side of the screen to maxSize (if necessary);

View File

@ -1,5 +1,6 @@
package com.genymobile.scrcpy.video;
import com.genymobile.scrcpy.device.ConfigurationException;
import com.genymobile.scrcpy.device.Size;
import android.view.Surface;
@ -32,15 +33,22 @@ public abstract class SurfaceCapture {
}
/**
* Called once before the capture starts.
* Called once before the first capture starts.
*/
public abstract void init() throws IOException;
public abstract void init() throws ConfigurationException, IOException;
/**
* Called after the capture ends (if and only if {@link #init()} has been called).
* Called after the last capture ends (if and only if {@link #init()} has been called).
*/
public abstract void release();
/**
* Called once before each capture starts, before {@link #getSize()}.
*/
public void prepare() {
// empty by default
}
/**
* Start the capture to the target surface.
*

View File

@ -68,12 +68,17 @@ public class SurfaceEncoder implements AsyncProcessor {
capture.init();
try {
streamer.writeVideoHeader(capture.getSize());
boolean alive;
boolean headerWritten = false;
do {
capture.prepare();
Size size = capture.getSize();
if (!headerWritten) {
streamer.writeVideoHeader(size);
headerWritten = true;
}
format.setInteger(MediaFormat.KEY_WIDTH, size.getWidth());
format.setInteger(MediaFormat.KEY_HEIGHT, size.getHeight());

View File

@ -0,0 +1,7 @@
package com.genymobile.scrcpy.video;
import com.genymobile.scrcpy.control.PositionMapper;
public interface VirtualDisplayListener {
void onNewVirtualDisplay(int displayId, PositionMapper positionMapper);
}

View File

@ -118,8 +118,12 @@ public final class ActivityManager {
return startActivityAsUserMethod;
}
@SuppressWarnings("ConstantConditions")
public int startActivity(Intent intent) {
return startActivity(intent, null);
}
@SuppressWarnings("ConstantConditions")
public int startActivity(Intent intent, Bundle options) {
try {
Method method = getStartActivityAsUserMethod();
return (int) method.invoke(
@ -133,7 +137,7 @@ public final class ActivityManager {
/* requestCode */ 0,
/* startFlags */ 0,
/* profilerInfo */ null,
/* bOptions */ null,
/* bOptions */ options,
/* userId */ /* UserHandle.USER_CURRENT */ -2);
} catch (Throwable e) {
Ln.e("Could not invoke method", e);

View File

@ -1,22 +1,48 @@
package com.genymobile.scrcpy.wrappers;
import com.genymobile.scrcpy.FakeContext;
import com.genymobile.scrcpy.device.DisplayInfo;
import com.genymobile.scrcpy.device.Size;
import com.genymobile.scrcpy.util.Command;
import com.genymobile.scrcpy.util.Ln;
import android.annotation.SuppressLint;
import android.content.Context;
import android.hardware.display.VirtualDisplay;
import android.os.Handler;
import android.view.Display;
import android.view.Surface;
import java.lang.reflect.Constructor;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
@SuppressLint("PrivateApi,DiscouragedPrivateApi")
public final class DisplayManager {
// android.hardware.display.DisplayManager.EVENT_FLAG_DISPLAY_CHANGED
public static final long EVENT_FLAG_DISPLAY_CHANGED = 1L << 2;
public interface DisplayListener {
/**
* Called whenever the properties of a logical {@link android.view.Display},
* such as size and density, have changed.
*
* @param displayId The id of the logical display that changed.
*/
void onDisplayChanged(int displayId);
}
public static final class DisplayListenerHandle {
private final Object displayListenerProxy;
private DisplayListenerHandle(Object displayListenerProxy) {
this.displayListenerProxy = displayListenerProxy;
}
}
private final Object manager; // instance of hidden class android.hardware.display.DisplayManagerGlobal
private Method createVirtualDisplayMethod;
@ -39,7 +65,7 @@ public final class DisplayManager {
public static DisplayInfo parseDisplayInfo(String dumpsysDisplayOutput, int displayId) {
Pattern regex = Pattern.compile(
"^ mOverrideDisplayInfo=DisplayInfo\\{\".*?, displayId " + displayId + ".*?(, FLAG_.*)?, real ([0-9]+) x ([0-9]+).*?, "
+ "rotation ([0-9]+).*?, layerStack ([0-9]+)",
+ "rotation ([0-9]+).*?, density ([0-9]+).*?, layerStack ([0-9]+)",
Pattern.MULTILINE);
Matcher m = regex.matcher(dumpsysDisplayOutput);
if (!m.find()) {
@ -49,9 +75,10 @@ public final class DisplayManager {
int width = Integer.parseInt(m.group(2));
int height = Integer.parseInt(m.group(3));
int rotation = Integer.parseInt(m.group(4));
int layerStack = Integer.parseInt(m.group(5));
int density = Integer.parseInt(m.group(5));
int layerStack = Integer.parseInt(m.group(6));
return new DisplayInfo(displayId, new Size(width, height), rotation, layerStack, flags);
return new DisplayInfo(displayId, new Size(width, height), rotation, layerStack, flags, density);
}
private static DisplayInfo getDisplayInfoFromDumpsysDisplay(int displayId) {
@ -98,7 +125,8 @@ public final class DisplayManager {
int rotation = cls.getDeclaredField("rotation").getInt(displayInfo);
int layerStack = cls.getDeclaredField("layerStack").getInt(displayInfo);
int flags = cls.getDeclaredField("flags").getInt(displayInfo);
return new DisplayInfo(displayId, new Size(width, height), rotation, layerStack, flags);
int dpi = cls.getDeclaredField("logicalDensityDpi").getInt(displayInfo);
return new DisplayInfo(displayId, new Size(width, height), rotation, layerStack, flags, dpi);
} catch (ReflectiveOperationException e) {
throw new AssertionError(e);
}
@ -124,4 +152,58 @@ public final class DisplayManager {
Method method = getCreateVirtualDisplayMethod();
return (VirtualDisplay) method.invoke(null, name, width, height, displayIdToMirror, surface);
}
public VirtualDisplay createNewVirtualDisplay(String name, int width, int height, int dpi, Surface surface, int flags) throws Exception {
Constructor<android.hardware.display.DisplayManager> ctor = android.hardware.display.DisplayManager.class.getDeclaredConstructor(
Context.class);
ctor.setAccessible(true);
android.hardware.display.DisplayManager dm = ctor.newInstance(FakeContext.get());
return dm.createVirtualDisplay(name, width, height, dpi, surface, flags);
}
public DisplayListenerHandle registerDisplayListener(DisplayListener listener, Handler handler) {
try {
Class<?> displayListenerClass = Class.forName("android.hardware.display.DisplayManager$DisplayListener");
Object displayListenerProxy = Proxy.newProxyInstance(
ClassLoader.getSystemClassLoader(),
new Class[] {displayListenerClass},
(proxy, method, args) -> {
if ("onDisplayChanged".equals(method.getName())) {
listener.onDisplayChanged((int) args[0]);
}
return null;
});
try {
manager.getClass()
.getMethod("registerDisplayListener", displayListenerClass, Handler.class, long.class, String.class)
.invoke(manager, displayListenerProxy, handler, EVENT_FLAG_DISPLAY_CHANGED, FakeContext.PACKAGE_NAME);
} catch (NoSuchMethodException e) {
try {
manager.getClass()
.getMethod("registerDisplayListener", displayListenerClass, Handler.class, long.class)
.invoke(manager, displayListenerProxy, handler, EVENT_FLAG_DISPLAY_CHANGED);
} catch (NoSuchMethodException e2) {
manager.getClass()
.getMethod("registerDisplayListener", displayListenerClass, Handler.class)
.invoke(manager, displayListenerProxy, handler);
}
}
return new DisplayListenerHandle(displayListenerProxy);
} catch (Exception e) {
// Rotation and screen size won't be updated, not a fatal error
Ln.e("Could not register display listener", e);
}
return null;
}
public void unregisterDisplayListener(DisplayListenerHandle listener) {
try {
Class<?> displayListenerClass = Class.forName("android.hardware.display.DisplayManager$DisplayListener");
manager.getClass().getMethod("unregisterDisplayListener", displayListenerClass).invoke(manager, listener.displayListenerProxy);
} catch (Exception e) {
Ln.e("Could not unregister display listener", e);
}
}
}

View File

@ -1,5 +1,6 @@
package com.genymobile.scrcpy.wrappers;
import com.genymobile.scrcpy.AndroidVersions;
import com.genymobile.scrcpy.util.Ln;
import android.annotation.TargetApi;
@ -200,13 +201,29 @@ public final class WindowManager {
}
}
@TargetApi(29)
public void unregisterRotationWatcher(IRotationWatcher rotationWatcher) {
try {
manager.getClass().getMethod("removeRotationWatcher", IRotationWatcher.class).invoke(manager, rotationWatcher);
} catch (Exception e) {
Ln.e("Could not unregister rotation watcher", e);
}
}
@TargetApi(AndroidVersions.API_29_ANDROID_10)
public void registerDisplayFoldListener(IDisplayFoldListener foldListener) {
try {
Class<?> cls = manager.getClass();
cls.getMethod("registerDisplayFoldListener", IDisplayFoldListener.class).invoke(manager, foldListener);
manager.getClass().getMethod("registerDisplayFoldListener", IDisplayFoldListener.class).invoke(manager, foldListener);
} catch (Exception e) {
Ln.e("Could not register display fold listener", e);
}
}
@TargetApi(AndroidVersions.API_29_ANDROID_10)
public void unregisterDisplayFoldListener(IDisplayFoldListener foldListener) {
try {
manager.getClass().getMethod("unregisterDisplayFoldListener", IDisplayFoldListener.class).invoke(manager, foldListener);
} catch (Exception e) {
Ln.e("Could not unregister display fold listener", e);
}
}
}

View File

@ -399,6 +399,27 @@ public class ControlMessageReaderTest {
Assert.assertEquals(-1, bis.read()); // EOS
}
@Test
public void testParseStartApp() throws IOException {
byte[] name = "firefox".getBytes(StandardCharsets.UTF_8);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(bos);
dos.writeByte(ControlMessage.TYPE_START_APP);
dos.writeByte(name.length);
dos.write(name);
byte[] packet = bos.toByteArray();
ByteArrayInputStream bis = new ByteArrayInputStream(packet);
ControlMessageReader reader = new ControlMessageReader(bis);
ControlMessage event = reader.read();
Assert.assertEquals(ControlMessage.TYPE_START_APP, event.getType());
Assert.assertEquals("firefox", event.getText());
Assert.assertEquals(-1, bis.read()); // EOS
}
@Test
public void testMultiEvents() throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();