Skip to content

feat: dockerize goscribe for homelab deployment#3

Merged
fabienpiette merged 25 commits into
mainfrom
feat/dockerize-app
Mar 16, 2026
Merged

feat: dockerize goscribe for homelab deployment#3
fabienpiette merged 25 commits into
mainfrom
feat/dockerize-app

Conversation

@fabienpiette

@fabienpiette fabienpiette commented Mar 16, 2026

Copy link
Copy Markdown
Owner

Summary

Transform goscribe from a local CLI tool into a homelab-deployable HTTP service with Redis-backed async job queue. Adds a new server binary (cmd/server) while keeping the existing CLI untouched.

Why

Enable goscribe to be consumed by other homelab services via HTTP API. The async job queue allows long-running transcription tasks to be processed in the background with webhook notifications on completion.

How

  • HTTP API layer using chi router with endpoints: POST /jobs, GET /jobs/{id}, GET /actions, GET /health
  • Async queue using asynq + Redis for job processing
  • Worker processor handles transcription and post-processing with the existing provider package
  • Server modes: all (default), api (HTTP only), worker (queue consumer only)
  • Docker multi-stage build with non-root user

Changes

  • Add cmd/server - HTTP server with mode switching and graceful shutdown
  • Add internal/api/handler.go - HTTP handlers for all endpoints
  • Add internal/api/router.go - chi router configuration
  • Add internal/worker/processor.go - asynq task processor
  • Add internal/worker/tasks.go - task types and result structs
  • Add Dockerfile - multi-stage build (golang:1.24-alpine → alpine:3.19)
  • Add docker-compose.yml - goscribe + redis services with profiles for split-mode
  • Add .env.example - environment variable documentation
  • Modify pkg/config - add DefaultPostActions() for server use

Security Fixes (Post-Initial Review)

  • SSRF protection: Validate webhook URLs against private IP ranges using custom http.Transport with DialContext validation
  • DNS rebinding protection: Resolve DNS once and connect directly to validated IP, closing TOCTOU gap
  • IPv4-mapped IPv6: Properly handle ::ffff:x.x.x.x addresses
  • Graceful shutdown: Changed log.Fatalf to log.Printf to ensure shutdown code is reached
  • HTTP server timeouts: Added ReadHeaderTimeout, WriteTimeout, IdleTimeout for slow-loris protection
  • REDIS_URL parsing: Use url.Parse for proper host extraction

Split-Mode Deployment

  • Default: docker compose up → starts goscribe (MODE=all) + redis
  • Split: docker compose --profile split up → starts goscribe-api + goscribe-worker + redis

Testing

  • All existing tests pass
  • Docker image builds successfully (~147MB)
  • Image runs as non-root user (goscribe)
  • golangci-lint passes

Checklist

  • Code follows project style guidelines
  • Self-reviewed own code
  • No new warnings introduced
  • Tests pass locally
  • Docker build verified

Impact

  • Breaking changes: None
  • Database migrations: None

@fabienpiette

Copy link
Copy Markdown
Owner Author

Code review

Found 6 issues:

  1. SSRF via unvalidated webhook_urlwebhook_url from the form is passed directly to http.Post() in the worker with no URL validation. An attacker can submit a job with a webhook pointing to internal services (e.g., http://169.254.169.254/latest/meta-data/, Redis on localhost:6379, or internal Kubernetes endpoints) and the worker will faithfully POST job results there.

provider := r.FormValue("provider")
webhookURL := r.FormValue("webhook_url")

}
resp, err := http.Post(url, "application/json", bytes.NewReader(b))
if err != nil {
return
}
resp.Body.Close()
}

  1. log.Fatalf bypasses graceful shutdown — when either server goroutine sends to errCh, log.Fatalf calls os.Exit(1) immediately. The shutdown block below the select (which drains HTTP connections and stops the asynq worker) is never reached, abandoning in-flight jobs and open connections.

select {
case err := <-errCh:
log.Fatalf("fatal error: %v", err)
case sig := <-quit:
log.Printf("received %v, shutting down (timeout=%s)...", sig, cfg.shutdownTimeout)
}
shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.shutdownTimeout)
defer cancel()
if httpSrv != nil {
if err := httpSrv.Shutdown(shutdownCtx); err != nil {
log.Printf("HTTP shutdown error: %v", err)
}
}
if asynqSrv != nil {
asynqSrv.Shutdown()

  1. Split-mode deployment silently broken — when MODE=api and MODE=worker run in separate containers, uploaded audio files are saved to the API container's local filesystem. The worker receives only the path string and cannot access the file; no shared volume is defined in docker-compose.yml. Every transcription job will fail with a file-not-found error in split mode.

services:
goscribe:
build: .
ports:
- "8080:8080"
env_file:
- .env
environment:
- REDIS_URL=redis://redis:6379
depends_on:
redis:
condition: service_healthy
restart: unless-stopped
redis:
image: redis:7-alpine
volumes:
- redis_data:/data
command: redis-server --appendonly yes
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 5
restart: unless-stopped
volumes:

  1. REDIS_URL parsing silently breaks for non-trivial URLsstrings.TrimPrefix(redisURL, "redis://") does not match rediss:// (TLS), leaving the full URL as the address. It also passes database numbers and credentials fragments as part of the host string for URLs like redis://user:pass@host:6379/1. Both asynq and go-redis expect a bare host:port. Use url.Parse instead.

goscribe/cmd/server/main.go

Lines 124 to 127 in f8df8ba

func loadConfig() serverConfig {
redisURL := getenv("REDIS_URL", "redis://redis:6379")
redisAddr := strings.TrimPrefix(redisURL, "redis://")

  1. fireWebhook uses http.DefaultClient with no timeouthttp.Post() uses the package-level default client, which has no timeout. If the webhook endpoint is slow or unreachable, the worker goroutine blocks indefinitely. With the default concurrency of 5 workers, 5 concurrent stalled webhooks exhaust the pool.

func (p *Processor) fireWebhook(url string, result JobResult) {
b, err := json.Marshal(result)
if err != nil {
return
}
resp, err := http.Post(url, "application/json", bytes.NewReader(b))
if err != nil {
return
}
resp.Body.Close()
}

  1. http.Server has no read/write timeouts — the server is created without ReadHeaderTimeout, ReadTimeout, or WriteTimeout, making it vulnerable to slow-loris connection exhaustion. Standard practice for any publicly accessible Go HTTP server is to set at minimum ReadHeaderTimeout.

goscribe/cmd/server/main.go

Lines 174 to 181 in f8df8ba

DefaultProvider: cfg.provider,
})
return &http.Server{
Addr: ":" + cfg.port,
Handler: api.NewRouter(h),
}
}

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

- Add SSRF protection for webhook URLs (validate against private IPs)
- Fix graceful shutdown: log.Printf instead of log.Fatalf
- Add shared volume for uploads in docker-compose for split-mode deployment
- Fix REDIS_URL parsing using url.Parse for proper host extraction
- Add 10s timeout to webhook HTTP client
- Add ReadTimeout, WriteTimeout, and IdleTimeout to HTTP server
- Use custom http.Transport with DialContext that validates resolved IPs
  at connection time to prevent DNS rebinding attacks
- Fix IPv4-mapped IPv6 address handling (::ffff:10.x.x.x etc)
- Validate scheme (http/https) in isAllowedWebhookURL, actual IP validation
  happens during dial to prevent rebinding
- Add profiles: [split] to goscribe-api and goscribe-worker
- Default: docker compose up starts goscribe (MODE=all) + redis
- Split mode: docker compose --profile split up starts api + worker
@fabienpiette

Copy link
Copy Markdown
Owner Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request successfully dockerizes goscribe, introducing an HTTP REST API, a Redis-backed async job queue, and a worker processor. The Docker setup uses a multi-stage build, and docker-compose.yml is provided for easy deployment. New Makefile targets simplify Docker operations. Overall, the changes are well-structured and introduce significant new functionality. However, there are some areas for improvement regarding error handling and security hardening, particularly around configuration parsing and webhook validation.

Comment thread cmd/server/main.go Outdated
Comment thread internal/worker/processor.go Outdated
Comment thread cmd/server/main.go
Comment thread internal/api/handler.go Outdated
Comment thread internal/api/handler.go Outdated
Comment thread internal/worker/processor.go
- Add error checking for RESULT_TTL_HOURS, MAX_UPLOAD_MB, SHUTDOWN_TIMEOUT_SECONDS env vars
- Add error checking for json.Marshal in handler
- Add logging for webhook delivery attempts and non-2xx responses
- Fix parseRedisAddr to log warning on parse failure
- Add OpenAPI/Swagger endpoints to router
- Generate openapi.yaml spec
- Add swagger.go for embedded Swagger UI
- Update README with CLI and Docker/Server features
- Update ARCHITECTURE.md with server mode documentation
- Update docker-compose.portainer.yml with ghcr.io image
- Builds cross-platform binaries (linux/darwin/windows, amd64/arm64)
- Pushes multi-arch Docker images to GHCR and Docker Hub
- Creates GitHub releases with auto-generated release notes
- Supports manual trigger via workflow_dispatch
Comment thread .github/workflows/release.yml Fixed
fabienpiette and others added 3 commits March 16, 2026 16:47
…n permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
- Add TestParseRedisAddr, TestLoadConfig, TestLoadConfigDefaults, TestLoadConfigWithAllVars
- Replace real Redis with miniredis for worker tests
- Add test for SSRF webhook blocking (localhost blocked)
- All tests now run without requiring Redis
@fabienpiette fabienpiette merged commit 95660fa into main Mar 16, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants