[codex] Add SDK compiler perf attribution#686
Conversation
|
Additional V-373 optimizer investigation after the initial PR note: I tried several narrower root-cause fixes and removed each prototype after benchmarking when it failed the acceptance bar.
Current conclusion: the goal was pursued beyond the initial implementation failure. The variants that affect real optimized output either do not improve runtime or regress size/compile/runtime, while the safe variants are optimized away to the same final Wasm shape. No scalar optimizer infrastructure is kept without representative benchmark value. |
Summary
scripts/bench-v326.tsbenchmark coverage with VTrace, web framework, VX, and scalar aggregate representative scenarios plus WAT counters forstruct.new,array.new*,ref.cast, andcall_ref.[voyd:compiler:perf]summaries.V-373 investigation outcome
V-410 blocked useful attribution for SDK-based benchmark runs, so this PR fixes that instrumentation first.
I investigated a local-handler scalar aggregate result specialization prototype. The root cause is that scalar aggregate call/result specialization rejects effectful metadata before local handler specialization can produce residual-free handled clones, so local effectful paths currently lose scalar aggregate replacement/direct ABI opportunities.
The prototype was removed because it failed the V-373 acceptance bar: optimized VTrace
mainchanged from the expected3825271to3644126. After removing the prototype, optimized VTrace returned3825271again. No dormant optimizer infrastructure is kept.Benchmarks
Baseline:
/tmp/v373-baseline.csv.Final instrumented run:
/tmp/v373-final.csvand/tmp/v373-final-perf.log.node --conditions=development --import tsx scripts/bench-v326.ts compare /tmp/v373-baseline.csv /tmp/v373-final.csvshowed identical wasm hashes, wasm/gzip sizes, and WAT counters across all scenarios. Representative final rows:251.365 msmedian,60979wasm bytes,19825gzip bytes,struct.new=772,array.new=196,ref.cast=893,call_ref=258.0.292 msmedian,131818wasm bytes,33499gzip bytes,struct.new=1145,array.new=772,ref.cast=1631,call_ref=766.0.246 msmedian,30001wasm bytes,9954gzip bytes,struct.new=207,array.new=223,ref.cast=412,call_ref=235.0.513 msmedian,17658wasm bytes,6486gzip bytes,struct.new=119,array.new=109,ref.cast=293,call_ref=162.Perf summaries now include
sdk.finalizeCompile; concurrent perf sessions are marked withoverlapped: truebecause the underlying compiler counters/phases are global.Validation
npx vitest run --config vitest.config.ts --testTimeout 30000 packages/sdk/src/__tests__/sdk-perf.test.tsnpm test(rerun with local-listener permissions after sandbox-onlylisten EPERM)npm run typecheck