A reproducible object-storage benchmark for S3-compatible providers, built on Files SDK. It runs the same upload, HEAD, download, LIST, integrity, and cleanup workload against local storage, Neon, Cloudflare R2, and Railway Storage Buckets.
S3 Bench records raw samples and produces both JSON and readable Markdown reports. It is designed for honest provider comparisons, so retries are disabled, downloads are SHA-256 verified, network conditions are recorded, and public campaigns rotate provider order.
Requires Bun 1.2 or newer.
bun install
cp .env.example .env
chmod 600 .env
bun run benchmark:localThe local benchmark needs no cloud credentials. Reports are written to .results/, and temporary objects are removed after each run.
The default workload uses 1 KiB, 1 MiB, and 8 MiB objects, with three objects per size, concurrency two, one warmup cycle, and three measured cycles.
Each measured cycle:
- Uploads every object.
- Runs HEAD and verifies the stored byte size.
- Downloads every object and verifies its length and SHA-256 hash.
- Lists the run prefix and verifies every expected key exists.
- Deletes every uploaded object in a
finallyblock.
Within upload, HEAD, and download, each size runs as a separate concurrent phase. Measured cycles rotate the size order so one size does not permanently benefit from running first or last.
Before each provider run, S3 Bench also records a Cloudflare HTTP preflight consisting of a small request, a 10 MB download, and a 2 MB upload. This is a connection snapshot, not a direct latency measurement to the storage provider.
Copy .env.example to .env, then fill in one provider section. Keep .env local and mode 600.
NEON_BUCKET=your-bucket
AWS_ENDPOINT_URL_S3=https://your-branch.storage.your-region.aws.neon.tech
AWS_ACCESS_KEY_ID=generated-key-id
AWS_SECRET_ACCESS_KEY=generated-secret
AWS_REGION=us-east-2The Neon adapter uses path-style addressing through Files SDK's AWS SDK v3 S3 client.
R2_BUCKET=your-bucket
R2_ACCOUNT_ID=your-account-id
R2_ACCESS_KEY_ID=generated-key-id
R2_SECRET_ACCESS_KEY=generated-secretThe benchmark explicitly selects Files SDK's AWS SDK client path for R2 instead of its optional fetch client.
RAILWAY_S3_ENDPOINT=https://storage.railway.app
RAILWAY_S3_REGION=auto
RAILWAY_S3_BUCKET=your-bucket
RAILWAY_S3_ACCESS_KEY_ID=generated-key-id
RAILWAY_S3_SECRET_ACCESS_KEY=generated-secret
RAILWAY_S3_FORCE_PATH_STYLE=falseUse the values and URL style shown in Railway's Bucket credentials tab. Older buckets can require path-style URLs. Railway documents that its Storage Buckets run on Tigris infrastructure, but S3 Bench treats Railway Bucket as the product being measured.
All remote providers use the same Files SDK AWS SDK v3 S3Client path. Files SDK retries are disabled, and AWS_MAX_ATTEMPTS=1 limits the underlying AWS SDK to one total attempt so retries cannot hide failures or inflate latency.
Run one provider:
bun run benchmark -- --provider r2
bun run benchmark -- --provider neon
bun run benchmark -- --provider railwayOverride the workload when needed:
bun run benchmark -- \
--provider r2 \
--sizes 16777216,67108864,104857600 \
--files 2 \
--concurrency 2 \
--warmups 1 \
--runs 3Sizes are bytes. Use --skip-network only for tests or offline development. --keep leaves uploaded objects behind; cleanup is enabled by default.
A single sequential pass is vulnerable to provider-order and transient network effects. The campaign command runs three Latin-square rounds, so every provider appears once in each sequence position.
bun run campaign
bun run publicationThe publication generator validates that all nine reports used an identical workload, one AWS SDK attempt, balanced positions, and zero failures before producing aggregate and per-size tables.
bun run compare remains available for a quick comparison of the latest individual reports, but the balanced campaign is the appropriate path for published results.
The shared files.upload() path is a single object upload. Provider limits can differ. Neon accepted 100 MiB in testing but rejected 128 MiB and 256 MiB single uploads; Neon recommends multipart upload through the AWS SDK for larger files. Multipart is outside this benchmark's shared Files SDK operation and should be tested as a separate workload rather than mixed into these results.
Each run writes:
- JSON containing every raw operation sample, workload configuration, runtime context, network preflight, integrity failures, cleanup errors, and run errors.
- Markdown containing p50 and p95 request latency plus aggregate wall-clock throughput.
Generated reports live in .results/ and are intentionally ignored by Git because they may contain machine, bucket, endpoint, and timing metadata. Review any report before publishing it.
S3 Bench can run as a small authenticated Bun service, including on Railway. Set provider credentials and a strong BENCH_API_TOKEN, then start it with:
bun run startcurl -X POST https://YOUR-SERVICE.example/benchmark \
-H "Authorization: Bearer $BENCH_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"provider":"neon"}'Health check:
curl https://YOUR-SERVICE.example/healthOnly one HTTP benchmark runs at a time. Attach persistent storage or retain the returned JSON if reports must survive redeploys.
Benchmark numbers are observations, not universal provider rankings. Region placement, Wi-Fi conditions, internet routing, object size, concurrency, hot versus cold reads, and provider limits all affect the outcome. Compare per-size tables, inspect round consistency, and publish the recorded limitations with the numbers.
Running this project against paid storage can incur request, storage, and egress charges. Start with the local adapter, use isolated test buckets, and keep cleanup enabled.
bun run typecheck
bun testSee CONTRIBUTING.md. Security issues should follow SECURITY.md.