Skip to content

Commit a871e67

Browse files
quanruclaude
quanru
and
claude
authored
feat(playground): implement task cancellation for Android/iOS playgrounds (#1355)
* feat(playground): implement task cancellation for Android/iOS playgrounds This PR implements task cancellation functionality for Android and iOS playgrounds using a singleton + recreation pattern. When users clicked the "Stop" button in Android/iOS playground, the task continued to execute and control the device via ADB commands. This was because: - Agent instances were global singletons created at server startup - The /cancel endpoint only deleted progress tips without stopping execution - There was no mechanism to interrupt ongoing tasks Implemented a singleton + recreation pattern: - PlaygroundServer now accepts factory functions instead of instances - Added task locking mechanism (currentTaskId) to prevent concurrent tasks - When cancel is triggered, the agent is destroyed and recreated - Device operations stop immediately as destroyed agents reject new commands 1. **PlaygroundServer** (packages/playground/src/server.ts) - Added factory function support for page and agent creation - Added `recreateAgent()` method to destroy and recreate agent - Added `currentTaskId` to track running tasks - Enhanced `/execute` endpoint with task conflict detection - Enhanced `/cancel` endpoint to recreate agent on cancellation - Backward compatible with existing instance-based usage 2. **Android Playground** (packages/android-playground/src/bin.ts) - Updated to use factory pattern for server creation - Each recreation creates fresh AndroidDevice and AndroidAgent instances 3. **iOS Playground** (packages/ios/src/bin.ts) - Updated to use factory pattern for server creation - Each recreation creates fresh IOSDevice and IOSAgent instances - Added test script `test-cancel-android.sh` for automated testing - Manual testing confirmed device operations stop when cancel is triggered ``` User clicks Stop ↓ Frontend calls /cancel/:requestId ↓ Server checks if current running task ↓ Call recreateAgent() ├─ Destroy old agent (agent.destroy()) ├─ Destroy old device (device.destroy()) ├─ Create new device (pageFactory()) └─ Create new agent (agentFactory(device)) ↓ Clear task lock and progress tips ↓ Device stops operations ✅ ``` - ✅ Simple implementation (minimal code changes) - ✅ Effective cancellation (destroy() immediately sets destroyed flag) - ✅ Backward compatible (still accepts instances) - ✅ Natural serialization (one task at a time per device) ```bash pnpm run android:playground ./test-cancel-android.sh ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(page): ensure keyboard actions return promises for better async handling * refactor(playground): update PlaygroundServer to use agent factories and simplify server creation * fix(ios): round coordinates for tap and swipe actions to improve accuracy * fix(android): round coordinates in scrolling and gesture methods for improved accuracy * refactor(playground): simplify PlaygroundServer instantiation and improve code readability --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent ba3849a commit a871e67

File tree

10 files changed

+226
-102
lines changed

10 files changed

+226
-102
lines changed

‎packages/android-playground/src/bin.ts‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -119,13 +119,17 @@ const main = async () => {
119119
const selectedDeviceId = await selectDevice();
120120
console.log(`✅ Selected device: ${selectedDeviceId}`);
121121

122-
// Create device and agent instances with selected device
123-
const device = new AndroidDevice(selectedDeviceId, {
124-
alwaysRefreshScreenInfo: true,
125-
});
126-
const agent = new AndroidAgent(device);
122+
// Create PlaygroundServer with agent factory
123+
const playgroundServer = new PlaygroundServer(
124+
// Agent factory - creates new agent with device each time
125+
async () => {
126+
const device = new AndroidDevice(selectedDeviceId);
127+
await device.connect();
128+
return new AndroidAgent(device);
129+
},
130+
staticDir,
131+
);
127132

128-
const playgroundServer = new PlaygroundServer(device, agent, staticDir);
129133
const scrcpyServer = new ScrcpyServer();
130134

131135
// Set the selected device in scrcpy server

‎packages/android/src/device.ts‎

Lines changed: 56 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -882,19 +882,21 @@ ${Object.keys(size)
882882
}
883883

884884
// Calculate final end coordinates, ensuring they stay within screen bounds
885-
const endX =
885+
const endX = Math.round(
886886
deltaX === 0
887887
? start.x // No horizontal movement
888888
: deltaX > 0
889889
? Math.min(maxWidth, start.x + actualScrollDistanceX) // Scroll right, cap at maxWidth
890-
: Math.max(0, start.x - actualScrollDistanceX); // Scroll left, cap at 0
890+
: Math.max(0, start.x - actualScrollDistanceX), // Scroll left, cap at 0
891+
);
891892

892-
const endY =
893+
const endY = Math.round(
893894
deltaY === 0
894895
? start.y // No vertical movement
895896
: deltaY > 0
896897
? Math.min(maxHeight, start.y + actualScrollDistanceY) // Scroll down, cap at maxHeight
897-
: Math.max(0, start.y - actualScrollDistanceY); // Scroll up, cap at 0
898+
: Math.max(0, start.y - actualScrollDistanceY), // Scroll up, cap at 0
899+
);
898900

899901
return { x: endX, y: endY };
900902
}
@@ -1025,8 +1027,11 @@ ${Object.keys(size)
10251027
async scrollUntilTop(startPoint?: Point): Promise<void> {
10261028
if (startPoint) {
10271029
const { height } = await this.size();
1028-
const start = { x: startPoint.left, y: startPoint.top };
1029-
const end = { x: start.x, y: height };
1030+
const start = {
1031+
x: Math.round(startPoint.left),
1032+
y: Math.round(startPoint.top),
1033+
};
1034+
const end = { x: start.x, y: Math.round(height) };
10301035

10311036
await repeat(defaultScrollUntilTimes, () =>
10321037
this.mouseDrag(start, end, defaultFastScrollDuration),
@@ -1043,7 +1048,10 @@ ${Object.keys(size)
10431048

10441049
async scrollUntilBottom(startPoint?: Point): Promise<void> {
10451050
if (startPoint) {
1046-
const start = { x: startPoint.left, y: startPoint.top };
1051+
const start = {
1052+
x: Math.round(startPoint.left),
1053+
y: Math.round(startPoint.top),
1054+
};
10471055
const end = { x: start.x, y: 0 };
10481056

10491057
await repeat(defaultScrollUntilTimes, () =>
@@ -1062,8 +1070,11 @@ ${Object.keys(size)
10621070
async scrollUntilLeft(startPoint?: Point): Promise<void> {
10631071
if (startPoint) {
10641072
const { width } = await this.size();
1065-
const start = { x: startPoint.left, y: startPoint.top };
1066-
const end = { x: width, y: start.y };
1073+
const start = {
1074+
x: Math.round(startPoint.left),
1075+
y: Math.round(startPoint.top),
1076+
};
1077+
const end = { x: Math.round(width), y: start.y };
10671078

10681079
await repeat(defaultScrollUntilTimes, () =>
10691080
this.mouseDrag(start, end, defaultFastScrollDuration),
@@ -1080,7 +1091,10 @@ ${Object.keys(size)
10801091

10811092
async scrollUntilRight(startPoint?: Point): Promise<void> {
10821093
if (startPoint) {
1083-
const start = { x: startPoint.left, y: startPoint.top };
1094+
const start = {
1095+
x: Math.round(startPoint.left),
1096+
y: Math.round(startPoint.top),
1097+
};
10841098
const end = { x: 0, y: start.y };
10851099

10861100
await repeat(defaultScrollUntilTimes, () =>
@@ -1098,10 +1112,13 @@ ${Object.keys(size)
10981112

10991113
async scrollUp(distance?: number, startPoint?: Point): Promise<void> {
11001114
const { height } = await this.size();
1101-
const scrollDistance = distance || height;
1115+
const scrollDistance = Math.round(distance || height);
11021116

11031117
if (startPoint) {
1104-
const start = { x: startPoint.left, y: startPoint.top };
1118+
const start = {
1119+
x: Math.round(startPoint.left),
1120+
y: Math.round(startPoint.top),
1121+
};
11051122
const end = this.calculateScrollEndPoint(
11061123
start,
11071124
0,
@@ -1118,10 +1135,13 @@ ${Object.keys(size)
11181135

11191136
async scrollDown(distance?: number, startPoint?: Point): Promise<void> {
11201137
const { height } = await this.size();
1121-
const scrollDistance = distance || height;
1138+
const scrollDistance = Math.round(distance || height);
11221139

11231140
if (startPoint) {
1124-
const start = { x: startPoint.left, y: startPoint.top };
1141+
const start = {
1142+
x: Math.round(startPoint.left),
1143+
y: Math.round(startPoint.top),
1144+
};
11251145
const end = this.calculateScrollEndPoint(
11261146
start,
11271147
0,
@@ -1138,10 +1158,13 @@ ${Object.keys(size)
11381158

11391159
async scrollLeft(distance?: number, startPoint?: Point): Promise<void> {
11401160
const { width } = await this.size();
1141-
const scrollDistance = distance || width;
1161+
const scrollDistance = Math.round(distance || width);
11421162

11431163
if (startPoint) {
1144-
const start = { x: startPoint.left, y: startPoint.top };
1164+
const start = {
1165+
x: Math.round(startPoint.left),
1166+
y: Math.round(startPoint.top),
1167+
};
11451168
const end = this.calculateScrollEndPoint(
11461169
start,
11471170
scrollDistance,
@@ -1158,10 +1181,13 @@ ${Object.keys(size)
11581181

11591182
async scrollRight(distance?: number, startPoint?: Point): Promise<void> {
11601183
const { width } = await this.size();
1161-
const scrollDistance = distance || width;
1184+
const scrollDistance = Math.round(distance || width);
11621185

11631186
if (startPoint) {
1164-
const start = { x: startPoint.left, y: startPoint.top };
1187+
const start = {
1188+
x: Math.round(startPoint.left),
1189+
y: Math.round(startPoint.top),
1190+
};
11651191
const end = this.calculateScrollEndPoint(
11661192
start,
11671193
-scrollDistance,
@@ -1343,14 +1369,14 @@ ${Object.keys(size)
13431369
const n = 4; // Divide the screen into n equal parts
13441370

13451371
// Set the starting point based on the swipe direction
1346-
const startX = deltaX < 0 ? (n - 1) * (width / n) : width / n;
1347-
const startY = deltaY < 0 ? (n - 1) * (height / n) : height / n;
1372+
const startX = Math.round(deltaX < 0 ? (n - 1) * (width / n) : width / n);
1373+
const startY = Math.round(deltaY < 0 ? (n - 1) * (height / n) : height / n);
13481374

13491375
// Calculate the maximum swipeable range
13501376
const maxNegativeDeltaX = startX;
1351-
const maxPositiveDeltaX = (n - 1) * (width / n);
1377+
const maxPositiveDeltaX = Math.round((n - 1) * (width / n));
13521378
const maxNegativeDeltaY = startY;
1353-
const maxPositiveDeltaY = (n - 1) * (height / n);
1379+
const maxPositiveDeltaY = Math.round((n - 1) * (height / n));
13541380

13551381
// Limit the swipe distance
13561382
deltaX = Math.max(-maxNegativeDeltaX, Math.min(deltaX, maxPositiveDeltaX));
@@ -1360,8 +1386,8 @@ ${Object.keys(size)
13601386
// Note: For swipe, we need to reverse the delta direction
13611387
// because positive deltaY should scroll up (show top content),
13621388
// which requires swiping from bottom to top (decreasing Y)
1363-
const endX = startX - deltaX;
1364-
const endY = startY - deltaY;
1389+
const endX = Math.round(startX - deltaX);
1390+
const endY = Math.round(startY - deltaY);
13651391

13661392
// Adjust coordinates to fit device ratio
13671393
const { x: adjustedStartX, y: adjustedStartY } = this.adjustCoordinates(
@@ -1435,11 +1461,11 @@ ${Object.keys(size)
14351461

14361462
// Default start point is near top of screen (but not too close to edge)
14371463
const start = startPoint
1438-
? { x: startPoint.left, y: startPoint.top }
1439-
: { x: width / 2, y: height * 0.15 };
1464+
? { x: Math.round(startPoint.left), y: Math.round(startPoint.top) }
1465+
: { x: Math.round(width / 2), y: Math.round(height * 0.15) };
14401466

14411467
// Default distance is larger to ensure refresh is triggered
1442-
const pullDistance = distance || height * 0.5;
1468+
const pullDistance = Math.round(distance || height * 0.5);
14431469
const end = { x: start.x, y: start.y + pullDistance };
14441470

14451471
// Use custom drag with specified duration for better pull-to-refresh detection
@@ -1473,11 +1499,11 @@ ${Object.keys(size)
14731499

14741500
// Default start point is bottom center of screen
14751501
const start = startPoint
1476-
? { x: startPoint.left, y: startPoint.top }
1477-
: { x: width / 2, y: height * 0.85 };
1502+
? { x: Math.round(startPoint.left), y: Math.round(startPoint.top) }
1503+
: { x: Math.round(width / 2), y: Math.round(height * 0.85) };
14781504

14791505
// Default distance is 1/3 of screen height
1480-
const pullDistance = distance || height * 0.4;
1506+
const pullDistance = Math.round(distance || height * 0.4);
14811507
const end = { x: start.x, y: start.y - pullDistance };
14821508

14831509
// Use pullDrag for consistent pull gesture handling

‎packages/ios/src/bin.ts‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -86,15 +86,13 @@ const main = async () => {
8686

8787
try {
8888
let wdaConfig = { host: 'localhost', port: DEFAULT_WDA_PORT };
89-
let device: IOSDevice;
90-
let agent: IOSAgent;
9189
let connected = false;
9290

9391
while (!connected) {
9492
try {
9593
// Create device with WebDriverAgent configuration
9694
// deviceId will be auto-detected from WebDriverAgent connection
97-
device = new IOSDevice({
95+
const device = new IOSDevice({
9896
wdaHost: wdaConfig.host,
9997
wdaPort: wdaConfig.port,
10098
});
@@ -105,7 +103,6 @@ const main = async () => {
105103
);
106104
await device.connect();
107105

108-
agent = new IOSAgent(device);
109106
connected = true;
110107

111108
// Get real device info after connection
@@ -165,7 +162,18 @@ const main = async () => {
165162
}
166163
}
167164

168-
const playgroundServer = new PlaygroundServer(device!, agent!, staticDir);
165+
// Create agent factory with explicit type
166+
const agentFactory = async (): Promise<IOSAgent> => {
167+
const newDevice = new IOSDevice({
168+
wdaHost: wdaConfig.host,
169+
wdaPort: wdaConfig.port,
170+
});
171+
await newDevice.connect();
172+
return new IOSAgent(newDevice);
173+
};
174+
175+
// Create PlaygroundServer with agent factory
176+
const playgroundServer = new PlaygroundServer(agentFactory, staticDir);
169177

170178
console.log('🚀 Starting server...');
171179

0 commit comments

Comments
 (0)