mcp-name: io.github.pedro-rivas/android-puppeteer-mcp

Android Puppeteer

Android Puppeteer is a lightweight, visual-first MCP (Model Context Protocol) server that enables AI agents to interact with Android devices through intelligent UI element detection and automated interactions. Built on uiautomator2, it provides comprehensive Android automation capabilities including visual element detection, touch interactions, text input, and video recording.

🎥 Watch the demo in action

Features

Visual Element Detection Automatically detects and annotates interactive UI elements with numbered overlays for precise targeting.
Comprehensive Touch Interactions Support for tap, long press, swipe, scroll, and drag gestures with coordinate-based precision.
Multi-Device Support Connect to multiple Android devices or emulators simultaneously with device-specific targeting.
Video Recording Integration Built-in screen recording capabilities using scrcpy for documentation and testing workflows.
Real-Time UI Analysis Live UI hierarchy parsing and element information extraction for dynamic interaction strategies.
MCP Protocol Integration Seamless integration with Claude Desktop and other MCP-compatible AI platforms.

Supported Operating Systems

Android 10+
Windows, macOS, Linux (host systems)

Installation

Prerequisites

Python 3.10+
uiautomator2
Android 10+ (Emulator or Physical Device)
ADB (Android Debug Bridge)
scrcpy (for video recording features)

Getting Started

Clone the repository

git clone https://github.com/pedro-rivas/android-puppeteer-mcp.git
cd android-puppeteer

Install dependencies

uv python install 3.10
uv sync

Setup Android device

# Enable USB debugging on your Android device
# For emulator, ensure it's running
adb devices  # Verify device connection

Connect to the MCP server
Locate your Claude Desktop configuration file:
- Windows: %APPDATA%\Claude\claude_desktop_config.json
- macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Add the following JSON to your Claude Desktop config:

{
  "mcpServers": {
    "android-puppeteer": {
      "command": "path/to/uv",
      "args": [
        "--directory",
        "path/to/android-puppeteer",
        "run",
        "puppeteer.py"
      ]
    }
  }
}

Replace:

path/to/uv with the actual path to your uv executable
path/to/android-puppeteer with the absolute path to where you have cloned this repo

Restart Claude Desktop

Restart your Claude Desktop. You should see "android-puppeteer" listed as an available integration.

Available Tools

Android Puppeteer provides the following tools for comprehensive Android device interaction:

Device Management

list_emulators: List all available Android emulators and devices with their status and dimensions
get_device_dimensions: Get the screen dimensions of a specific Android device
get_ui_elements_info: Get detailed information about all interactive UI elements on screen

Visual Interaction

take_screenshot: Capture annotated screenshots with numbered UI element overlays
press: Tap on specific coordinates with optional long press duration
long_press: Perform long press gestures on specific coordinates

Navigation & Input

press_back: Press the hardware back button
swipe: Perform directional or custom coordinate swipes
type_text: Type text into focused input fields with optional text clearing
scroll_element: Scroll specific UI elements in any direction

Recording & Documentation

record_video: Start screen recording with customizable quality settings
stop_video: Stop active screen recordings and save to local storage

Usage Examples

Basic Device Interaction

# Take an annotated screenshot
screenshot = await take_screenshot()

# Tap on a specific element (element 5 from screenshot)
await press(x=500, y=300)

# Type text into an input field
await type_text("Hello, Android!")

# Swipe to scroll down
await swipe(direction="down")

Multi-Device Automation

# List available devices
devices = await list_emulators()

# Target specific device
await take_screenshot(device_id="emulator-5554")
await press(x=200, y=400, device_id="emulator-5554")

Video Recording Workflow

# Start recording
await record_video(filename="test_session.mp4")

# Perform automation steps
await press(x=300, y=500)
await type_text("Automated test input")

# Stop recording
await stop_video()

Project Structure

android-puppeteer/
    puppeteer.py          # Main MCP server implementation
    main.py              # Entry point
    pyproject.toml       # Project configuration
    ss/                  # Screenshots directory
    videos/              # Video recordings directory
    README.md           # This file

Important Notes

Device Permissions: Ensure USB debugging is enabled on target Android devices
Network Access: Some features require network connectivity for device communication
Storage: Screenshot and video files are saved locally in ss/ and videos/ directories
Performance: Response times depend on device performance and network latency

Troubleshooting

Common Issues

Device not found: Verify ADB connection with adb devices
Permission denied: Check USB debugging and device authorization
Screenshot failures: Ensure device screen is unlocked and accessible
Video recording issues: Verify scrcpy installation and device compatibility

Debug Mode

Run the server directly for debugging:

uv run puppeteer.py

License

This project is licensed under the MIT License. See LICENSE for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

Development Setup

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes
Run tests and ensure code quality
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Related Projects

Android MCP - Alternative Android automation MCP server
uiautomator2 - Core Android automation library
MCP Protocol - Model Context Protocol specification

Star this repo if you find it useful!

android-puppeteer-mcp

Documentation