Skip to content

README.md

Latest commit

 

History

History
572 lines (426 loc) · 20.2 KB

README.md

File metadata and controls

572 lines (426 loc) · 20.2 KB

Secure MQTT Core

The baseline TrailMQ stack. Use this recipe when you want:

  • TLS-secured MQTT broker
  • REST API for control and audit retrieval
  • Web UI via reverse proxy
  • Cryptographic audit chain
  • Policy-based access control

Nothing more. This is the foundation every other recipe builds on.

What's in the stack

Service Image Purpose
backend rainergewalt/trailmq-backend MQTT broker + REST API + audit
frontend rainergewalt/trailmq-frontend Web UI
nginx nginx:stable-alpine Reverse proxy + TLS termination

Exposed ports

Port Purpose
80 HTTP / Web UI
8883 MQTT over TLS

The backend REST API runs inside the stack on port 8443. Public access goes through the reverse proxy on port 80.

Default local URLs:

Surface URL
Web UI http://localhost/trailmq/
REST API http://localhost/api/v1
MQTT TLS localhost:8883
MQTT WS ws://localhost/mqtt

First-run requirements

The certs/ folder must contain:

  • server_cert.pem
  • server_key.pem
  • ca_cert.pem

./trailmq launch offers to generate local demo certificates automatically, or you can run ./trailmq certs at any time. The launcher also generates a JWT secret into secrets/jwtsecret.txt if openssl is available.

Evaluation users

Two evaluation users ship with this recipe. They are for local evaluation only — change or disable them before any non-local deployment.

User Role Credential source
testadmin admin secrets/testadmin.pwd (generated by launcher)
testuser publisher secrets/testuser.pwd (generated by launcher)

On first run, ./trailmq launch generates random passwords into those files and prints them once to stdout. Re-run cat secrets/testadmin.pwd if you lose track of them. To rotate: delete the files and re-launch.

Credential examples

The active config.yaml intentionally contains only testadmin and testuser. Do not keep disabled demo users with missing password_file entries in the active config, because the backend validates password files while loading the configuration.

Examples for password_file, password_hash, and placeholder plain-password users live in:

docs/config-examples/users.yaml

Copy an example into config.yaml only after setting real credentials.

REST API Quickstart

TrailMQ provides a REST API for controlling and reviewing MQTT behavior.

The Web UI uses the same API, so the most important product functions can also be accessed from scripts, local checks, monitoring tools or external integrations.

Use the API to answer questions like:

  • Which MQTT topics are controlled?
  • Which settings apply to a topic?
  • Which policy applies to a message path?
  • What is currently queued or dead-lettered?
  • What was recorded in the audit trail?
  • Is the audit chain still valid?
  • Which evidence-oriented records can be exported for review?

Default local API endpoint:

http://localhost/api/v1

Authentication

The local evaluation setup uses configured users, not open self-registration.

Read the generated admin password:

cd recipes/secure-mqtt-core

ADMIN_USER="testadmin"
ADMIN_PASS="$(cat secrets/testadmin.pwd)"

Login and receive a JWT:

TOKEN="$(
  curl -sS -X POST "http://localhost/api/v1/auth" \
    -H "Content-Type: application/json" \
    -d "{\"username\":\"${ADMIN_USER}\",\"password\":\"${ADMIN_PASS}\"}" \
  | jq -r '.token // .accessToken // .data.token // .data.accessToken'
)"

Check the authenticated user:

curl -sS "http://localhost/api/v1/auth/me" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

Most control and review endpoints require:

Authorization: Bearer <token>

Public Runtime Checks

Use these endpoints for basic smoke tests, Docker health checks, monitoring and local readiness checks.

Method Endpoint What it tells you
GET /health REST service is reachable
GET /live Process is alive
GET /ready TrailMQ is ready to serve broker/API work
GET /api/v1/version Running version/build information
GET /api/v1/metrics JSON metrics
GET /api/v1/metrics/json JSON metrics
GET /api/v1/metrics/prometheus Prometheus-style metrics
GET /metrics Prometheus-style metrics

Example:

curl -sS http://localhost/ready | jq
curl -sS http://localhost/api/v1/metrics | jq

Product meaning:

Is TrailMQ running, reachable and ready before MQTT traffic is sent through it?

Topic Control

Topics are the controlled MQTT namespace in TrailMQ.

Use these endpoints to create, inspect and configure MQTT paths. This includes active state, QoS, queue behavior, audit settings and effective runtime configuration.

Method Endpoint Product purpose
GET /api/v1/topics List controlled MQTT topics
POST /api/v1/topics Create a controlled topic
GET /api/v1/topics/tree Inspect the topic hierarchy
POST /api/v1/topics/tree Create a topic path with parent chain
POST /api/v1/topics/autocreate Auto-create a topic for local evaluation
GET /api/v1/topics/{topic}/settings Read topic behavior
PUT /api/v1/topics/{topic}/settings Update queue, QoS, audit or active settings
GET /api/v1/topics/by-name/{path} Read topic by MQTT path
DELETE /api/v1/topics/by-name/{path} Delete topic by MQTT path
GET /api/v1/topics/by-name/{path}/effective Show effective runtime configuration

Example:

curl -sS -X POST "http://localhost/api/v1/topics" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "demo/line1/temperature",
    "qosLevel": 1,
    "isActive": true,
    "queueEnabled": true,
    "queueSize": 1000,
    "auditTrail": true
  }' | jq

Check what TrailMQ will effectively apply:

curl -sS "http://localhost/api/v1/topics/by-name/demo/line1/temperature/effective" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

Product meaning:

Which MQTT paths are controlled, and what behavior applies to messages on this path?

Topic Rules

Topic rules help apply behavior to topic patterns instead of configuring every topic individually.

Method Endpoint Product purpose
GET /api/v1/topic-rules List topic rules
POST /api/v1/topic-rules Create a topic rule
GET /api/v1/topic-rules/{id} Read one topic rule
PUT /api/v1/topic-rules/{id} Update a topic rule
DELETE /api/v1/topic-rules/{id} Delete a topic rule
GET /api/v1/topic-rules/validate Validate a topic rule pattern
GET /api/v1/topic-rules/match/{topic} Test which rules match a topic
POST /api/v1/topic-rules/apply Apply topic rules

Product meaning:

Which rule affects this topic, and why does the topic behave this way?

Policy Control

Policies describe how TrailMQ should treat messages on selected topic paths.

They can be used to inspect, resolve and validate message behavior before or during local evaluation.

Method Endpoint Product purpose
GET /api/v1/policies List policies
POST /api/v1/policies Create a policy
GET /api/v1/policies/{id} Read a policy
PUT /api/v1/policies/{id} Update a policy
DELETE /api/v1/policies/{id} Delete a policy
GET /api/v1/policies/stats Inspect policy usage
GET /api/v1/policies/violations Inspect denied or invalid behavior
POST /api/v1/policies/resolve Resolve policy for a topic
POST /api/v1/policies/validate Validate a message against policy
POST /api/v1/policies/handshake Generate client policy handshake
GET /api/v1/policies/handshake/{clientId} Read client policy handshake
GET /api/v1/policies/bindings List policy-to-topic bindings
POST /api/v1/policies/bindings Bind a policy to a topic pattern

Create a policy:

curl -sS -X POST "http://localhost/api/v1/policies" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "demo-policy",
    "description": "Demo policy",
    "maxPayloadKb": 64,
    "qosRequired": 1,
    "onViolation": "deny",
    "auditLevel": "basic",
    "enabled": true
  }' | jq

Bind it to a topic pattern:

curl -sS -X POST "http://localhost/api/v1/policies/bindings" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "pattern": "demo/line1/#",
    "policyId": "demo-policy",
    "priority": 100,
    "enabled": true
  }' | jq

Resolve the policy for a concrete topic:

curl -sS -X POST "http://localhost/api/v1/policies/resolve" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"topic":"demo/line1/temperature"}' | jq

Product meaning:

Why would this MQTT message be allowed, denied, audited or treated as a violation?

Queue and Dead-Letter Review

Queue endpoints show how TrailMQ behaves when messages cannot be delivered immediately or need to be isolated for later review.

Method Endpoint Product purpose
GET /api/v1/queue/status Inspect global queue health
GET /api/v1/queue/deadletter Review dead-lettered messages
PUT /api/v1/queue/config Update global queue behavior
POST /api/v1/queue/purge/{topic} Reset queue state for one topic
GET /api/v1/queues/{topic}/stats Inspect per-topic queue state
GET /api/v1/queues/{topic}/messages Read queued messages
POST /api/v1/queues/{topic}/enqueue Enqueue a test message
POST /api/v1/queues/{topic}/dispatch Dispatch queued message
POST /api/v1/queues/{topic}/ack Acknowledge delivery
POST /api/v1/queues/{topic}/nack Negative acknowledge delivery

Example:

curl -sS "http://localhost/api/v1/queue/status" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

Product meaning:

If delivery is delayed or fails, can I see what happened instead of losing visibility?

Audit and Evidence Review

Audit endpoints expose the reviewable record of what happened inside TrailMQ.

They are useful for evaluation, debugging, traceability checks and exported review data. TrailMQ can support evidence-oriented workflows, but using TrailMQ does not by itself make a system compliant with GMP, GxP, 21 CFR Part 11 or any other regulatory framework.

Method Endpoint Product purpose
GET /api/v1/audit List audit entries
GET /api/v1/audit/summary Read audit summary
GET /api/v1/audit/summary/extended Read extended audit summary
GET /api/v1/audit/validatechain Validate audit chain
GET /api/v1/audit/validatechain/details Inspect chain validation details
GET /api/v1/audit/contract-status Check evidence contract status
POST /api/v1/audit/contract-status/reset Reset evidence contract status
GET /api/v1/audit/search Search audit records
GET /api/v1/audit/auth List authentication audit events
GET /api/v1/audit/auth/stats Read authentication audit stats
GET /api/v1/audit/export?format=json Export audit data
GET /api/v1/audit/export/json Export audit data as JSON

Some builds also expose focused review views such as /api/v1/audit/gmp. Treat these as audit/event views for evaluation and review, not as compliance certification.

Example:

curl -sS "http://localhost/api/v1/audit/validatechain/details" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

Product meaning:

What happened, was it recorded, and is the audit chain still intact?

Product Read Models

Product read models provide higher-level views over broker, queue, audit and capability state.

They are useful for dashboards, demos, evaluation reports and external tooling that should not need to reconstruct product state from low-level records.

Method Endpoint Product view
GET /api/v1/flows Controlled message/control flows
GET /api/v1/flows/{scope} Flow detail for one topic/scope
GET /api/v1/trust/summary Trust and evidence summary
GET /api/v1/capabilities/summary Enabled capability/plugin state
GET /api/v1/events/gmp Focused event view for review
GET /api/v1/exports/trust?format=jsonl Export trust/evidence records
GET /api/v1/exports/trust?format=csv Export trust/evidence records

The /api/v1/events/gmp view is a focused event view for regulated evaluation scenarios. It is not a compliance claim.

Examples:

curl -sS "http://localhost/api/v1/flows" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq
curl -sS "http://localhost/api/v1/trust/summary" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq
curl -sS "http://localhost/api/v1/exports/trust?format=jsonl" \
  -H "Authorization: Bearer ${TOKEN}"

Product meaning:

TrailMQ should not only move MQTT messages. It should help explain what happened around those messages.

Recommended Local Evaluation Flow

A simple evaluation flow is:

# 1. Check runtime readiness
curl -sS http://localhost/ready | jq

# 2. Login
cd recipes/secure-mqtt-core

ADMIN_USER="testadmin"
ADMIN_PASS="$(cat secrets/testadmin.pwd)"

TOKEN="$(
  curl -sS -X POST "http://localhost/api/v1/auth" \
    -H "Content-Type: application/json" \
    -d "{\"username\":\"${ADMIN_USER}\",\"password\":\"${ADMIN_PASS}\"}" \
  | jq -r '.token // .accessToken // .data.token // .data.accessToken'
)"

# 3. Inspect controlled topics
curl -sS "http://localhost/api/v1/topics" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

# 4. Resolve policy for a topic
curl -sS -X POST "http://localhost/api/v1/policies/resolve" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"topic":"demo/line1/temperature"}' \
  | jq

# 5. Check queue and DLQ behavior
curl -sS "http://localhost/api/v1/queue/status" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

# 6. Validate audit chain
curl -sS "http://localhost/api/v1/audit/validatechain/details" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

# 7. Read product-level trust state
curl -sS "http://localhost/api/v1/trust/summary" \
  -H "Authorization: Bearer ${TOKEN}" \
  | jq

This demonstrates the core evaluation path:

Configure MQTT behavior, inspect the effective state, send or simulate traffic, review what happened and check whether the recorded evidence is still intact.

Internal Verification Philosophy

TrailMQ is developed with an internal scenario-based verification approach.

The internal verification suite is not part of the public evaluation package. It is used during development to exercise TrailMQ through its real product surfaces:

  • MQTT over TLS
  • REST authentication
  • topic control
  • policy resolution
  • queue and dead-letter behavior
  • audit recording
  • audit-chain validation
  • health, readiness and metrics endpoints
  • negative and security behavior

The goal is not only to check whether MQTT messages can be transported.

The goal is to check whether TrailMQ remains controllable, inspectable and reviewable under realistic conditions.

Internally verified product areas include:

Area What is checked internally Product meaning
Authentication Missing, invalid or wrong credentials are rejected Access is explicit
Authorization Restricted actions are blocked for insufficient roles Runtime control is role-aware
Topic control Topic creation, hierarchy and effective settings are exercised MQTT namespaces are manageable
Queue behavior Queue pressure, ACK/NACK behavior and DLQ paths are exercised Delivery issues remain visible
Audit trail Audit entries and chain validation are exercised Runtime behavior can be reviewed
Integrity Audit-chain behavior and tamper-oriented checks are exercised Evidence changes should become visible
Durability Topics, queues, policies and audit state are exercised across runtime changes State should survive more than one process moment
Concurrency MQTT traffic and REST changes are exercised in parallel Runtime behavior is tested under realistic interaction
API contracts Common response shapes and error behavior are exercised Automation can depend on stable API behavior
Negative behavior Bad input and forbidden actions are exercised Failures should be explicit and diagnosable

This principle is central to TrailMQ:

TrailMQ should not only transport MQTT messages. It should make MQTT behavior easier to control, inspect and review.

The public REST API is therefore the main integration surface for local evaluation, automation and external tooling.

Compliance Note

TrailMQ is built for environments where traceability, reviewability and controlled messaging matter.

It can support regulated engineering practices by exposing audit records, policy decisions, queue state and evidence-oriented exports.

However, TrailMQ itself does not certify a system as GMP, GxP, CSV, Annex 11 or 21 CFR Part 11 compliant. Compliance depends on the full validated system, configuration, operating procedures, user management, infrastructure and organizational controls.

Run directly without the CLI

cd recipes/secure-mqtt-core
docker compose up -d

You usually don't need to — use ./trailmq up from the repo root.