Skip to content

simplifier: Use double precision for quadric storage and evaluation#1062

Closed
snowbldr wants to merge 1 commit into
zeux:masterfrom
snowbldr:simplify-quadric-precision
Closed

simplifier: Use double precision for quadric storage and evaluation#1062
snowbldr wants to merge 1 commit into
zeux:masterfrom
snowbldr:simplify-quadric-precision

Conversation

@snowbldr

@snowbldr snowbldr commented Jul 9, 2026

Copy link
Copy Markdown

meshopt_simplify has a position-dependent floor on target_error: geometry
far from the mesh bounding-box minimum cannot be simplified below an error of
roughly 3e-4 × extent, while identical geometry near the bbox minimum
simplifies to completion.

Cause

The expanded quadric form r = vᵀAv + 2bᵀv + c cancels catastrophically in
single precision. For a plane with offset d (after internal rescale),
quadricFromPlane computes b = d·w and c = d²·w, whose independent
float32 roundings don't cancel — evaluating at a point on the plane
leaves a residual r ≈ 1e-7·w·d² instead of 0, i.e. a distance-error noise
floor of ~3e-4·extent·d. Since the internal rescale anchors at the bbox
minimum, d — and the floor — grows with distance from the min corner.
Collapse candidates on exactly flat, exactly coplanar surfaces then rank
above tight error limits and the region never simplifies.

Repro

Two identical 150×150 flat plates, one at z=0 and one at z=900, in a mesh of
extent 1000 (the offset must be along the plate normal, and the far
plate's rescaled offset must be inexact in binary — z=0.9 here; a tangential
offset leaves b = c = 0 exactly and hides the effect). Simplify with
target_index_count = index_count/100, target_error = 5e-5:

master (float) this PR (double)
plate A (z=0) 2 tris 446 tris
plate B (z=900) 38,602 tris 454 tris
total vs target 38,604 / 900 900 / 900
reported error 4.75e-05 (noise floor) 2.56e-06 (true geometric)

Master, unable to make progress on plate B, spends the entire reduction
budget over-collapsing plate A; this PR reaches the requested target and
stops. Asked for the actual minimum (target = 4 triangles), master still
returns 38,604 while this PR returns exactly 4 — 2 per plate.

Result meshes rendered with edges (master left/right = near/far plate, then
this PR):

master: near plate collapses to 2 triangles, far plate frozen at render density

this PR: both plates collapse symmetrically

Found in the wild on a 127mm CAD part where one exactly-planar box wall kept
~17M render-resolution triangles at tight target_error while the
geometrically identical opposite wall collapsed to a handful; with this
change the whole part reaches 16k triangles with measured surface deviation
of 2e-4 mm.

Change

Confined to the Quadric/QuadricGrad fields, the products in
quadricFromPlane, and the accumulation in quadricEval/quadricError
(22 insertions, 19 deletions). Storing double without computing the b/c
products in double is not sufficient — the coefficients are already rounded
at construction. Plane fitting from vertex positions stays in single
precision; its per-triangle rounding is second-order for this effect.

Includes a regression test (simplifyPrecision in demo/tests.cpp) that
builds the two-plate geometry procedurally and asserts symmetric collapse —
it fails on master and passes with this change.

make test passes; builds warning-free with -Wall -Wextra -Wshadow.

Trade-off

This doubles quadric array memory (per remap-unique vertex) and may
measurably affect simplification throughput on some targets. If that's
unacceptable for game-LOD workloads — which never request errors below the
noise floor and are unaffected by the bug — the same change works gated
behind a compile-time define, or I'm happy to rework it in whatever
direction you prefer (recentring the internal rescale only halves the worst
case, so some precision change appears necessary for a full fix).

The expanded quadric form r = vAv + 2bv + c cancels catastrophically in
single precision: quadricFromPlane computes b = d*w and c = d*d*w whose
independent roundings do not cancel, so evaluating at a point on the
plane leaves a residual of ~1e-7*w*d^2 — an error noise floor of roughly
3e-4 * extent * d in distance terms. Since d grows with distance from
the mesh bounding box minimum, the effective target_error floor is
position dependent: geometry near the bbox minimum simplifies to
completion while identical geometry on the far side stalls.

Repro: two identical 150x150 flat plates at z=0 and z=900 in a mesh of
extent 1000, simplified with target_error 5e-5 — the near plate reaches
2 triangles while the far plate keeps 38602 of 45000 (19301x asymmetry),
with reported error pinned at the noise floor. With this change both
plates simplify symmetrically at any target (899 total at target 900;
4 total at target 4) and the reported error drops to the true geometric
value (2.6e-6).

The change is confined to the Quadric/QuadricGrad fields, the products
in quadricFromPlane, and the accumulation in quadricEval/quadricError;
plane fitting from vertex positions can stay in single precision, as its
per-triangle rounding is second-order for this effect.

This doubles the memory used by quadric arrays; if that is a concern for
some workloads the same change works gated behind a define.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PkB4ZiMNML9ga58hGxvixe
@zeux zeux added the slop label Jul 9, 2026
@zeux zeux closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants