iPad + Apple Pencil → a pressure- and tilt-aware pen for Windows, over Wi-Fi.
No app to install.
No native iPad app: Safari captures Pencil samples, ships them over a WebRTC
data channel (UDP-like: unordered + unreliable), and a Bun server injects them as
synthetic Windows pen input via the Pointer Injection API. Because it's real pen
input (not a mouse, not SendInput), pressure and tilt reach Windows Ink apps:
Whiteboard, OneNote, Krita, Photoshop, Clip Studio, etc.
Highlights
- Real pen input: pressure (0–1024) and tilt (−90..90°) into Windows Ink apps.
- UDP-like transport (WebRTC unreliable data channel) with sub-millisecond server-side injection on LAN.
- Loss-tolerant (datagram redundancy) and an optional adaptive playout buffer for jittery Wi-Fi.
- Multi-monitor aware, absolute or relative mapping, adjustable pressure curve.
- QR-code pairing with a per-session token. Terminal UI plus an optional native control panel (Electrobun).
- One self-contained HTML page per surface, no build step; oscilloscope-instrument look.
iPad Safari ──Pointer Events──┐
(capture + getCoalescedEvents)│ binary datagrams over a WebRTC data channel
▼ (ordered:false, maxRetransmits:0)
WebSocket signaling ──► Bun server (werift) ──► bun:ffi → user32.dll
│ CreateSyntheticPointerDevice
│ InjectSyntheticPointerInput(PT_PEN)
▼ │
Electrobun control panel ▼
(optional native window) Windows Ink (pressure + tilt)
- Windows 10 1809+ or Windows 11. Synthetic pen injection
(
CreateSyntheticPointerDevice/InjectSyntheticPointerInput) was introduced in Windows 10 1809 (build 17763). Earlier Windows cannot run this. - Bun 1.2+ on the PC.
- An iPad with Apple Pencil, on the same LAN/Wi-Fi as the PC, using Safari.
- For the optional native control panel: WebView2 (preinstalled on Windows 11 and current Windows 10). Electrobun downloads its own runtime on first build.
This tool is Windows-only by design. The injection backend is structured behind an
interface (InputBackend), so another platform could be added later, but only the
Windows synthetic-pen backend is implemented.
bun install
bun run startYou'll see a QR code in the terminal. Scan it with the iPad camera (or open the
printed http://<lan-ip>:8787/?token=… link in Safari). The page auto-connects and
you can draw immediately. Pressure drives stroke width; tilt is delivered to apps
that use it.
Open a Windows Ink app (the Whiteboard app or Krita are great for testing), put focus on it, and draw on the iPad.
The page validates a per-session token from the QR URL. Connections without the correct token are rejected.
A minimal native dashboard (QR, link state, live telemetry, mapping controls, release pen / quit) is available as an Electrobun app:
bun run ui # build + launch the native window (electrobun dev)
bun run ui:build # just build the app into ./buildThe Electrobun shell is intentionally thin: its main process (src/bun/index.ts)
launches the same standalone server as bun run start, waits for it to be
healthy, then shows its /dashboard page in a WebView2 window. Closing the window
(or the server exiting) tears everything down and releases the pen cleanly. The
dashboard is plain HTML, so it also opens in any browser at
http://<lan-ip>:8787/dashboard?token=….
| Flag | Default | Description |
|---|---|---|
--port <n> |
8787 |
HTTP/WS port. |
--host <ip> |
auto-detected LAN IP | LAN IP advertised in the QR/URL. Override if autodetect picks the wrong adapter. |
--bind <addr> |
0.0.0.0 |
Interface the server binds to. |
--display <id> |
0 |
Initial target display (see the list the server prints at startup). |
--mode <absolute|relative> |
absolute |
Initial coordinate mapping. |
--token <str> |
random | Override the session token (otherwise a random one is generated each run). |
--debug |
off | Log packet rate, drop rate, and injection latency every 2s. |
--raw-coords |
off | Disable the virtual-desktop-origin coordinate offset (see Multi-monitor). |
--jitter <off|auto|ms> |
off |
Playout buffer that paces injection to smooth network bursts. off injects immediately (lowest latency, default); auto sizes it to the link's jitter; a number sets a fixed depth in ms. Also adjustable live in the UI. See Smoothness. |
--https |
off | Serve TLS. Requires --cert <path> and --key <path> (see HTTPS note). |
Example:
bun run start -- --port 9000 --display 1 --mode absolute --debugBrowsers can't send raw UDP, so input rides a WebRTC data channel configured
ordered: false, maxRetransmits: 0 (fire-and-forget, like UDP). Signaling
(offer/answer/ICE) goes over a WebSocket. The server side uses
werift, a pure-TypeScript WebRTC
stack that runs under Bun (verified: it creates peer connections and unreliable data
channels and completes ICE on a LAN with no STUN/TURN — host candidates only, so it
works fully offline). node-datachannel was the documented fallback but was not
needed.
Out-of-order datagrams are dropped by sequence number (newest wins). Samples are
injected immediately and in order on arrival; each FFI call is microseconds and
batches are capped at 48 samples, so the event loop is never blocked. Coalesced
Pencil samples (getCoalescedEvents()) are batched into one datagram when they fit.
Loss resilience. Each datagram also re-sends the previous batch's samples. Since
the server dedups by sequence number, a single dropped datagram is transparently
recovered by the next one, so isolated Wi-Fi packet loss doesn't make the pen jump.
Only two consecutive losses produce a visible gap. In a 12%-packet-loss test this
cut the visible loss rate from ~7.8% to ~0.9%. The cost is a few extra bytes per
datagram (the deduped copies show up as dup/s in --debug, not as loss).
Smoothness (playout buffer). iOS Wi-Fi power-save delivers evenly-sent packets in
~100ms bursts (the radio parks between beacons). Injecting a burst instantly makes the
pen lurch. The server runs a small playout buffer: each sample carries its client
timestamp, and a high-frequency drain tick (with the Windows timer resolution raised
to 1ms) releases samples paced to their original cadence, delayed by the buffer depth.
A ~100ms arrival burst becomes a steady ~one-sample-per-few-ms stroke. It is off by
default (--jitter off) for the lowest possible latency; turn it on when a jittery
link makes the pen lurch. --jitter auto sizes the buffer to the link's measured
jitter (~10-16ms on a clean LAN, growing only as the link demands); --jitter <ms>
pins a fixed depth. The --jitter flag only sets the startup value: smoothing is also adjustable live
from the iPad client and the dashboard (a Smoothing control: Off / Auto / Fixed,
with a Buffer-ms slider in Fixed mode), and the change syncs to both. The current depth
is shown as buffer= in --debug and the dashboard's Buffer tile.
Note: a buffer can only smooth bursts up to its own depth, so a 100ms-bursty link needs
~100ms of buffer (and thus latency) to fully smooth; fixing the link (below) lets auto
stay small and keeps latency low.
At startup the server calls CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) once. For every sample it fills a POINTER_TYPE_INFO
struct and calls InjectSyntheticPointerInput(device, &info, 1). On shutdown it calls
DestroySyntheticPointerDevice. For a single pen, one OS call injects one moment in
time, so a stroke is replayed as a sequence of single-sample calls.
Pointer-flag state machine (POINTER_FLAGS):
| Phase | Flags |
|---|---|
| contact start (down) | DOWN | INRANGE | INCONTACT |
| move while down | UPDATE | INRANGE | INCONTACT |
| hover (no contact) | UPDATE | INRANGE |
| lift (still in range) | UP | INRANGE |
| fully away | UP |
On any disconnect (data channel close, peer failure, signaling drop, server
shutdown) the server forces a clean UP so a contact can never get stuck down.
Pen data: penMask = PEN_MASK_PRESSURE | PEN_MASK_TILT_X | PEN_MASK_TILT_Y,
pressure 0–1024, tiltX/tiltY −90..90.
This is the error-prone part. The struct is built by hand via bun:ffi, matching
winuser.h. POINTER_TYPE_INFO is { POINTER_INPUT_TYPE type; union { POINTER_TOUCH_INFO; POINTER_PEN_INFO } }. The union is 8-byte aligned, so the pen arm starts at offset 8.
The union's largest member is POINTER_TOUCH_INFO (144 bytes), making the whole
struct 152 bytes; we allocate 152 and only write the pen arm.
| Offset | Size | Field |
|---|---|---|
| 0 | 4 | POINTER_TYPE_INFO.type = PT_PEN (3) |
| 4 | 4 | (padding) |
| 8 | 4 | penInfo.pointerInfo.pointerType = PT_PEN |
| 12 | 4 | pointerId |
| 16 | 4 | frameId = 0 |
| 20 | 4 | pointerFlags (POINTER_FLAG_*) |
| 24 | 8 | sourceDevice (HANDLE) = 0 |
| 32 | 8 | hwndTarget (HWND) = 0 |
| 40 | 4 | ptPixelLocation.x (LONG) |
| 44 | 4 | ptPixelLocation.y (LONG) |
| 48 | 24 | himetric / raw locations = 0 |
| 72 | 4 | dwTime = 0 (OS timestamps) |
| 76 | 4 | historyCount = 0 |
| 80 | 4 | InputData = 0 |
| 84 | 4 | dwKeyStates = 0 |
| 88 | 8 | PerformanceCount (UINT64) = 0 |
| 96 | 4 | ButtonChangeType = 0 |
| 104 | 4 | penInfo.penFlags (PEN_FLAG_*) |
| 108 | 4 | penInfo.penMask (PEN_MASK_*) |
| 112 | 4 | penInfo.pressure (0–1024) |
| 116 | 4 | penInfo.rotation = 0 |
| 120 | 4 | penInfo.tiltX (−90..90) |
| 124 | 4 | penInfo.tiltY (−90..90) |
| 128 | 24 | (struct padded to 152) |
See src/input/backends/windowsPen.ts for the annotated implementation.
Little-endian, fixed-size records (src/protocol/binary.ts). Datagram = 1-byte
kind, then a body.
KIND_INPUT(0x01):u8 kind,u8 count,f64 tBase, thencount× 24-byte samples:u32 seq,f32 x,f32 y,f32 pressure(all normalized),i8 tiltX,i8 tiltY,u8 flags,u8 buttons,f32 tRel. Datagrams stay under 1200 bytes (≤ 48 samples each).KIND_PING/KIND_PONG(0x02 / 0x03):u8 kind,f64 clientTime. The server echoes the client's timestamp so the iPad measures round-trip latency.
Settings live on the iPad (a thumb-reachable drawer) and on the dashboard; both push changes to the server, which is authoritative and echoes the result so they stay in sync.
- Absolute: the whole capture surface maps to a chosen display's pixel bounds.
Multi-monitor aware (
EnumDisplayMonitors+ virtual-screen geometry, DPI-aware viaSetProcessDpiAwarenessContext). See Multi-monitor below for the coordinate detail. - Relative: normalized deltas move the cursor, with an adjustable pointer speed.
- Pressure curve:
linear, orgamma(client 0–1 →pow(p, γ)→ 0–1024).
Synthetic pen injection feeds the Windows Ink / Pointer input stack. Many pro art apps can read tablet input from two different APIs, and you must pick the Windows Ink one:
- Clip Studio Paint → Preferences → Tablet → set "Use tablet service" to Tablet PC (Windows Ink), not Wintab. Wacom drivers install Wintab, so if you normally draw with a Wacom your CSP is almost certainly in Wintab mode, and in that mode CSP ignores this tool's pressure (you'll get position but flat pressure). Switch to Windows Ink for draw-stream (switch back to Wintab for the Wacom).
- Photoshop → there's a
PSUserConfig.txttoggle (# use WinTabvs Windows Ink); use the Windows Ink path. - Krita → Settings → Configure Krita → Tablet → use Windows Ink (Windows 8+ Pointer Input). Krita's "WinTab" mode won't see injected pressure.
- Whiteboard / OneNote / Fresh Paint / most UWP apps → Windows Ink natively; work with no changes (good for a first test).
The pen is injected with INRANGE (hover) arriving just before contact so the ink
stack starts a proper pressured stroke rather than a flat tap.
InjectSyntheticPointerInput's ptPixelLocation is interpreted relative to the
virtual-desktop origin (the top-left of the bounding box of all monitors), not as
raw absolute screen coordinates. When your primary monitor is at the virtual origin
(the usual single-monitor / monitors-to-the-right case) the origin is (0,0) and this
makes no difference. But if you have monitors to the left of or above the primary,
the virtual origin is negative, and raw screen coordinates would land on the wrong
monitor. draw-stream therefore subtracts the virtual origin before injecting, which is
correct for both cases (verified on a 3-monitor layout with a negative origin).
If your particular machine maps to the wrong monitor because of this, run with
--raw-coords to disable the offset. The startup banner lists each detected display
with its bounds so you can confirm which --display id is which.
InjectSyntheticPointerInput works without elevation for target windows at the
same or lower integrity level as the server. To inject into elevated windows
(apps "Run as administrator"), run the server elevated too (or grant it UIAccess).
At startup the server prints whether it's running elevated. At runtime, if an
injection is denied with ERROR_ACCESS_DENIED (the target window is higher
integrity), it logs a one-time hint to run elevated. Most apps (Whiteboard, OneNote,
Krita at default integrity) need no elevation.
RTCPeerConnection and Pointer Events are not gated to secure contexts on iOS
Safari (only getUserMedia is, which this app never uses), so plain HTTP over the
LAN works for the data channel. HTTPS is therefore off by default and not required.
If you still want TLS, pass --https --cert <cert.pem> --key <key.pem>. A self-signed
cert (trust it on the iPad via Settings → General → About → Certificate Trust) can be
generated with, e.g., OpenSSL from Git for Windows:
openssl req -x509 -newkey rsa:2048 -nodes -days 825 \
-keyout key.pem -out cert.pem -subj "/CN=draw-stream" \
-addext "subjectAltName=IP:<your-lan-ip>"(Cert generation is intentionally not bundled, to keep dependencies minimal.)
- Pointer doesn't move / nothing injects. Make sure a Windows Ink-capable app has focus. If the target app is elevated, restart the server elevated.
- Pressure looks flat / binary (e.g. in Clip Studio Paint). The app is almost
certainly reading Wintab instead of Windows Ink. Switch it to Windows Ink
(see Pressure: Windows Ink vs Wintab above). Confirm injection itself works in the
Whiteboard app first. The
gammacurve can add low-end dynamic range. - iPad can't reach the server. Confirm both are on the same Wi-Fi. Allow Bun
through the Windows Firewall (private networks). If the QR uses the wrong IP (VPN /
Hyper-V / WSL adapter), override with
--host <your-lan-ip>. - Drawing lands on the wrong monitor. Pick the correct target display (the
startup banner lists each with its bounds). draw-stream offsets coordinates by the
virtual-desktop origin for multi-monitor correctness; if your machine still maps
wrong, try
--raw-coords(see Multi-monitor above). - Stuck pen after a drop. It shouldn't happen (the server forces
UPon every disconnect), but the dashboard's Release pen button forces a clean lift. - The pen "jumps" / lurches periodically (every ~100-300ms). iOS Wi-Fi power-save
delivers packets in bursts (the radio parks between beacons). It's the link, not the
server: a localhost loopback stays at ~10ms
arrivalMaxwithlost/s=0, and the client's reported send gap stays ~8ms while the server'sarrivalMaxspikes to ~100ms — that split proves the network is bunching, not either runtime. Turn on the playout buffer to absorb it: set Smoothing → Auto in the client/dashboard (or run--jitter auto) and watchbuffer=grow in--debug. The tradeoff is added latency equal to the buffer depth, so to keep latency low, also fix the link: keep the iPad on 5GHz, near the access point, plugged in, with Low Power Mode off, on an uncongested channel, and lower the router's DTIM/beacon interval if it's exposed. Smoothing is off by default (lowest latency); a clean link needs none.
Run with --debug to watch packet rate, loss/gap timing, dup/s, injection latency,
and the max inter-packet arrival stall.
src/
index.ts # `bun run start`: flags, QR, banner, lifecycle
server.ts # HTTP + WS (signaling + control) + injection orchestration
config.ts # CLI parsing, LAN-IP detection, token
stats.ts # rolling packet/drop/latency counters
protocol/binary.ts # wire format (encode/decode)
net/webrtc.ts # werift peer + unreliable data channel
input/
types.ts # InputBackend interface
pipeline.ts # drop-reorder, mapping, pressure curve, phase, inject
display.ts # monitor enumeration / virtual-screen geometry (FFI)
backends/windowsPen.ts # synthetic pen injection (FFI, struct layout)
win/ffi.ts # user32/kernel32/shell32 via bun:ffi, DPI, elevation
bun/index.ts # Electrobun main process (native control panel)
public/
client.html # iPad capture surface (self-contained, no build)
dashboard.html # server control panel (self-contained, no build)
electrobun.config.ts # Electrobun app config (entrypoint, Windows icon)
assets/ # logo.svg, logo.png, icon.ico
- ✅ Scan QR → drawing with the Pencil moves the Windows pen pointer; raw FFI
injection ~0.06 ms, WebRTC RTT ~0.7–1.2 ms on loopback. With smoothing off
(the default) the added latency stays well under the 30 ms LAN budget; the optional
--jitter autoplayout buffer trades a small, link-sized delay (~10-16 ms on a clean LAN) for a smooth, non-lurching stroke under Wi-Fi jitter. - ✅ Pressure (0–1024) and tilt (−90..90) are delivered as real pen input.
- ✅ Pen down/up reliable; a clean
UPis forced on every disconnect (no stuck contact). - ✅ Server survives client reconnects without restart (new connection replaces the old session and resets the sequence gate).
- ✅ werift verified under Bun; Electrobun verified to build for Windows (WebView2).
MIT — see LICENSE.