Skip to content

SKILL.md

Latest commit

 

History

History
262 lines (204 loc) · 8.11 KB

SKILL.md

File metadata and controls

262 lines (204 loc) · 8.11 KB
Raw
name constrained-browser-automation
description Run headless browsers on constrained ARM/Android devices without root, Playwright, or X11
title Constrained Browser Automation
version 1.1
trigger Running headless browser engines on resource-constrained ARM/Android devices (Termux, old phones, embedded Linux) for web scraping, JS rendering, or agent-driven navigation without root access or desktop dependencies.
prerequisites
Termux or similar constrained ARM environment
No root / no TUN / no systemd
Need JS-capable page rendering without Playwright wheels

Constrained Browser Automation

How to run real browser engines on low-resource ARM devices (Android via Termux, Raspberry Pi, old phones) where standard tooling fails: no Playwright wheels, no glibc compatibility, no X11 server, limited RAM/disk.

The Discovery: Chromium/x11 on Termux (Headless)

What looks impossible is possible. The chromium/x11 package in Termux contains a working headless Chrome binary even without a display server.

Pre-flight: Is Chromium already installed?

Before running pkg install, check if a previous session or system bundle already placed Chromium on the device:

which chromium-browser
which chromium
pkg list-installed | grep chromium 2>/dev/null

Some environments already ship with chromium/x11 available. If the binary exists, skip installation and proceed directly to launch flags.

Installation

# Termux
pkg install tur-repo   # if not already enabled (community repo)
pkg install chromium   # pulls chromium/x11

# Raspberry Pi OS / Debian ARM
sudo apt install chromium-browser

# Alpine
apk add chromium

Launch flags (tested on Android 15, no root)

chromium-browser \
  --headless \
  --headless=new \
  --no-sandbox \
  --disable-setuid-sandbox \
  --disable-dev-shm-usage \
  --disable-gpu \
  --disable-software-rasterizer \
  --disable-features=VizDisplayCompositor \
  --disable-crash-reporter \
  --disable-breakpad \
  --remote-debugging-port=9222 \
  --remote-allow-origins='*' \
  --user-data-dir=$HOME/.chromium-termux \
  --enable-logging \
  --v=0 \
  about:blank

Key flags:

  • --no-sandbox + --disable-setuid-sandbox — required in Termux/chroot and many containers
  • --remote-allow-origins='*' — critical; without this, WebSocket CDP rejects connections with 403 Forbidden
  • --disable-dev-shm-usage — saves RAM on devices without /dev/shm
  • --disable-features=VizDisplayCompositor — helps in headless mode on some Android builds
  • --user-data-dir — prevents Chrome from fighting over/corrupting the default profile
  • --disable-crash-reporter + --disable-breakpad — reduce background noise, slight RAM savings

See scripts/chromium-start.sh for a production-ready launcher script.

Verification

# HTTP CDP metadata
curl -s http://localhost:9222/json/version

# Open a page via REST
curl -s -X PUT 'http://localhost:9222/json/new?https://example.com'

# Returns JSON with id + webSocketDebuggerUrl

WebSocket CDP (no Playwright needed)

import websocket, json

ws_url = 'ws://localhost:9222/devtools/page/<PAGE_ID>'
ws = websocket.create_connection(ws_url, timeout=10)

# Enable domains
ws.send(json.dumps({'id':1,'method':'Page.enable'}))
ws.send(json.dumps({'id':2,'method':'Runtime.enable'}))

# Wait for Page.loadEventFired
while True:
    msg = json.loads(ws.recv())
    if msg.get('method') == 'Page.loadEventFired':
        break

# Evaluate JS and extract
ws.send(json.dumps({
    'id':10,
    'method':'Runtime.evaluate',
    'params':{'expression':'document.body.innerText'}
}))
resp = json.loads(ws.recv())
value = resp['result']['result']['value']
ws.close()

See references/termux-chromium-cdp.md for full reproduction recipe.

What Does NOT Work (Pitfalls)

Lightpanda (glibc binary on Bionic libc)

Lightpanda provides an aarch64-linux binary, but Termux uses Android's Bionic libc, not glibc. Result:

cannot execute: required file not found

Do not try to patch ELF or force-link glibc. Risk of system corruption on no-root devices. Workarounds (proot-distro, chroot) add 200MB+ and are slow. On glibc systems (Raspberry Pi OS, Debian VPS) Lightpanda works fine.

Playwright via pip

No ARM wheel available for many constrained platforms. pip install playwright fails:

Could not find a version that satisfies the requirement playwright

Use direct CDP instead.

Zombie Process Management

Chromium spawns multiple child processes. killall chromium often leaves orphans. Use pkill -9 -f chromium and verify with ps. The launcher script handles cleanup automatically.

Resource Auto-Throttle Rules (Low-RAM Devices)

Don't let Chrome starve the system. Maintain a resource guardian loop:

Kill Chrome when:

  • MemAvailable < 1024 MB — system under memory pressure
  • Battery < 20% and NOT charging — preserve power
  • CPU thermal throttling detected

Start/Resume Chrome when:

  • MemAvailable > 2048 MB — headroom exists
  • Charger connected
  • Agent explicitly requests JS-rendered page

Implementation via Python daemon:

import subprocess, json

def get_available_mb():
    with open('/proc/meminfo') as f:
        for line in f:
            if line.startswith('MemAvailable'):
                return int(line.split()[1]) / 1024
    return 0

def should_kill_chrome(bat_pct, bat_status, mem_mb):
    if mem_mb < 1024:
        return True
    if bat_pct < 20 and bat_status not in ('CHARGING', 'FULL'):
        return True
    return False

def chrome_rss_mb():
    r = subprocess.run(
        ["ps", "aux"], capture_output=True, text=True
    )
    rss = 0
    for line in r.stdout.splitlines():
        if 'chromium' in line and 'grep' not in line:
            parts = line.split()
            if len(parts) > 5:
                try:
                    rss += int(parts[5])
                except:
                    pass
    return rss / 1024

See scripts/chrome-resource-guard.py for full implementation.

Alternatives by Constraint

Constraint Solution
No disk space for Chrome (~400MB) requests + BeautifulSoup4 for static pages
No RAM for Chrome (~900MB runtime) VPS browser farm; send URLs, receive DOM remotely
Need JS but Chrome too heavy Lightpanda on glibc system (Debian VPS, Raspberry Pi OS)
Need stealth/rotation CDP Network.setUserAgentOverride per page

References

  • references/termux-chromium-cdp.md — Full CDP reproduction recipe
  • references/termux-chromium-headless.md — Verified session log: flags, CORS 403 fix, resource usage, extraction example
  • references/lightpanda-termux-failure.md — Why glibc binaries fail on Bionic
  • references/termux-chromium-dbus-notes.md — D-Bus errors are harmless in headless; do not chase them
  • templates/dashboard.html — Standalone monitoring UI (no build step)
  • scripts/chromium-start.sh — Production-ready launcher with auto-cleanup
  • scripts/chrome-resource-guard.py — Auto-throttle module
  • scripts/web-dashboard.py — stdlib-only Python HTTP dashboard

Self-Monitoring Dashboard (stdlib + SQLite)

On constrained devices even a lightweight pip/npm install is painful. A monitoring UI can be built with Python stdlib alone:

Stack: http.server + socketserver + sqlite3 (native FTS5 in Termux) No dependencies. No pip. No wheel hell. No node.

What the dashboard serves

  • Real-time cards: battery %, RAM available, daemon PID, Chromium alive/version
  • Live log tail from the awareness daemon
  • Episodic memory retrieval from sqlite FTS5
  • Remote command execution (sandboxed prefix whitelist)

Architecture

Constrained ARM host
  ├─ chrome-resource-guard.py   (periodic state logging)
  ├─ episodes.db                (sqlite3 + FTS5)
  ├─ web-dashboard.py           (http.server on :8765)
  └─ chromium --headless        (on :9222, when needed)

Key technique: native FTS5

import sqlite3
conn = sqlite3.connect(':memory:')
conn.execute('CREATE VIRTUAL TABLE mem USING fts5(body)')
# Works immediately. No pip install.

Full implementation: templates/dashboard.html + scripts/web-dashboard.py