memory: consume data in external client

[huberp]

Both are very feasible. Here is the complete picture.

---

Standalone Node.js listener — is it feasible?

Yes, completely. Node.js has native UDP multicast support via dgram built into the standard library — no npm dependencies needed for the networking layer. The RawSamplesPacket struct is straightforward binary: fixed-size header, known field offsets, little-endian floats. Node.js Buffer can parse it directly with readFloatLE().

The challenges

1. Binary struct parsing
The packet is #pragma pack(push, 1) — no padding. You need to know the exact byte offsets. From the current RawSamplesPacket:

Offset  Size  Field
0       4     magic          (uint32 LE)
4       4     version        (uint32 LE)
8       4     instanceID     (uint32 LE)
12      4     sequenceNumber (uint32 LE)
16      8     ppqOfFirstSample (float64 LE)
24      8     bpm            (float64 LE)
32      4     displayRangeBeats (float32 LE)
36      2     numSamples     (uint16 LE)
38      1470×4 = 5880  samples (float32 LE array)
Total header: 38 bytes
Total packet: 5918 bytes

Node.js reads this trivially:

const magic = buf.readUInt32LE(0);       // 0x534D504C
const version = buf.readUInt32LE(4);     // must === 3
const instanceID = buf.readUInt32LE(8);
const seqNum = buf.readUInt32LE(12);
const ppq = buf.readDoubleLe(16);
const bpm = buf.readDoubleLE(24);
const displayRange = buf.readFloatLE(32);
const numSamples = buf.readUInt16LE(36);
const samples = [];
for (let i = 0; i < numSamples; i++)
    samples.push(buf.readFloatLE(38 + i * 4));

2. Multicast group join on loopback
Node.js dgram supports addMembership(multicastGroup, localInterface). On loopback you bind to 0.0.0.0, port 49423, and join 239.255.42.1 on the loopback interface. On macOS/Linux this works out of the box. On Windows, SO_REUSEPORT vs SO_REUSEADDR semantics differ slightly — use socket.setReuseAddress(true) before binding.

3. Waveform display in Node.js
This is the only real challenge. Node.js has no native canvas. Options:

• Terminal: use blessed + a braille-character oscilloscope renderer (lightweight, no browser needed, works in SSH). Several npm packages exist for terminal waveform display.
• Browser: run a tiny express + ws (WebSocket) bridge — listener receives UDP, forwards JSON/binary to a browser tab that draws on a <canvas>. ~50 lines of code total.
• Electron: full native window with canvas, ships Node.js inside. More weight but gives a real GUI.

The browser bridge is the pragmatic choice: it is the least code, works cross-platform, and gives a full-resolution waveform display immediately.

4. Sample rate assumption
The packet carries bpm and ppqOfFirstSample but not sampleRate. The receiver needs to know the sample rate to reconstruct per-sample PPQ positions: ppq_i = ppqOfFirstSample + i * (bpm / (60.0 * sampleRate)). The Node.js listener must either assume 44100 Hz or the protocol needs to be extended to include sampleRate in the packet header. Adding it is a one-field change.

---

Architecture of the Node.js standalone client

flowchart LR
    subgraph DAW["DAW (localhost)"]
        P[phu-beat-sync-multi-scope\nPlugin Instance]
    end

    subgraph NODE["Node.js process"]
        U[UDP dgram socket\nbind 0.0.0.0:49423\njoin 239.255.42.1]
        P2[Packet parser\nreadUInt32LE / readFloatLE]
        S[State store\nper instanceID:\n  accumBins[4096]\n  rmsAccum[128]\n  cancelAccum[256]]
        WS[WebSocket server\nbroadcasts JSON snapshots\nat 30 Hz]
    end

    subgraph BROWSER["Browser tab"]
        C[Canvas oscilloscope\nwaveform + RMS overlay]
    end

    P -->|UDP multicast\n239.255.42.1:49423| U
    U --> P2
    P2 --> S
    S -->|setInterval 30Hz| WS
    WS -->|ws://localhost:8080| C

Loading

The Node.js process is about 120 lines of TypeScript — less if plain JS. It is a passive listener; it never sends anything back to the plugin.

---

Integration with huberp/agentloop

agentloop is a TypeScript-first LangChain agent loop with dynamic tool discovery — drop a .ts file exporting toolDefinition into src/tools/ and it is auto-registered at startup. This is the perfect integration point.

What an agentloop tool would look like

A audioScope tool that lets the agent query the current audio state:

// src/tools/audioScope.ts
import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { getScopeState } from "../../scope-bridge/ScopeBridge";

export const toolDefinition = tool(
  async ({ instanceId, metric }) => {
    const state = getScopeState(instanceId);
    if (!state) return `No data for instance ${instanceId}`;

    switch (metric) {
      case "rms":
        return JSON.stringify(state.rmsSlots);
      case "cancellation":
        return JSON.stringify(state.cancelSlots);
      case "bpm":
        return String(state.bpm);
      case "ppq":
        return String(state.latestPpq);
      case "waveform":
        // Return a downsampled summary — not full 1470 raw samples
        return JSON.stringify(state.accumBins.filter((_, i) => i % 32 === 0));
    }
  },
  {
    name: "audioScope",
    description:
      "Query live audio metrics from a running phu-beat-sync-multi-scope " +
      "plugin instance. Returns RMS levels, cancellation index, BPM, PPQ " +
      "position, or a downsampled waveform snapshot.",
    schema: z.object({
      instanceId: z.number().optional().describe(
        "Instance ID (omit for first available)"
      ),
      metric: z.enum(["rms", "cancellation", "bpm", "ppq", "waveform"]),
    }),
  }
);

ScopeBridge is a small singleton that runs the UDP listener in the background and maintains the latest state per instanceID. The agent calls the tool, gets back JSON, and can reason about it — "instance 2 has a cancellation index above 0.8 in slots 40–60, which corresponds to beats 2.5–3.1 at 120 BPM".

What the agent could do with it

Given agentloop's Planner + Orchestrator architecture, practical uses are:

Agent task	What it queries	What it reasons about
"Is the mix cancelling in the mids?"	cancellation metric	CI values across slots, flag if > 0.7
"What's the energy distribution across the bar?"	rms metric	RMS envelope shape, identify sparse beats
"Are the two instances in sync?"	ppq from both	Compare PPQ timestamps, detect drift
"Monitor and alert when cancellation spikes"	Repeated cancellation polls	Trigger alert tool when threshold crossed
"Describe what's happening in the audio"	waveform + rms	Summarise dynamics in natural language

Integration architecture

flowchart TD
    subgraph PLUGIN["phu-beat-sync-multi-scope (JUCE)"]
        PL[RawSamplesPacket\nUDP multicast 239.255.42.1:49423]
    end

    subgraph BRIDGE["ScopeBridge (Node.js singleton)"]
        UDP[dgram UDP listener]
        PARSE[Packet parser]
        STATE["ScopeState per instanceID\n{ accumBins, rmsAccum,\n  cancelAccum, bpm, ppq }"]
        UDP --> PARSE --> STATE
    end

    subgraph AGENTLOOP["huberp/agentloop"]
        TOOL[audioScope tool\nsrc/tools/audioScope.ts]
        AGENT[LangChain Agent Loop\nMistral LLM]
        TOOL -->|getScopeState| STATE
        AGENT -->|tool call| TOOL
        TOOL -->|JSON metric| AGENT
    end

    PL -->|loopback UDP| UDP
    USER[User prompt:\n'Is the mix cancelling?'] --> AGENT
    AGENT --> RESPONSE[Natural language answer\n+ metric data]

Loading

The sampleRate gap

The one missing piece: RawSamplesPacket does not carry sampleRate. The bridge needs it to reconstruct per-sample PPQ. Options:

1. Add uint32_t sampleRate to RawSamplesPacket — one field, bump protocol to version 4
2. The bridge assumes 44100 and documents the assumption
3. The bridge infers it from the packet cadence (sequence number delta × BROADCAST_CHUNK_SAMPLES ÷ elapsed wall-clock ms) — approximate but usable

Option 1 is the right answer and is a trivial change to SampleBroadcaster.h.

---

Copilot's recommendation

Build the Node.js bridge as a separate npm package (@huberp/scope-bridge) that can be consumed both by the standalone display client and by agentloop as a peer dependency. The bridge is ~200 lines of TypeScript, has zero runtime npm dependencies (only Node.js built-ins), and exports a clean ScopeBridge singleton with on('packet', ...) events and a getState(instanceId) query API.

The agentloop tool is then a thin adapter over that API — 40 lines. The standalone display client is an Express + WebSocket server over the same API — another 80 lines. Both share exactly the same binary parsing and state accumulation logic, and both benefit automatically from any protocol improvements (like adding sampleRate).

The only change needed in phu-beat-sync-multi-scope is adding sampleRate to RawSamplesPacket — everything else the Node.js side can implement independently, today, without touching the C++ plugin at all if you are willing to hardcode 44100 as a temporary measure.