simplifier: Use double precision for quadric storage and evaluation#1062
Closed
snowbldr wants to merge 1 commit into
Closed
simplifier: Use double precision for quadric storage and evaluation#1062snowbldr wants to merge 1 commit into
snowbldr wants to merge 1 commit into
Conversation
The expanded quadric form r = vAv + 2bv + c cancels catastrophically in single precision: quadricFromPlane computes b = d*w and c = d*d*w whose independent roundings do not cancel, so evaluating at a point on the plane leaves a residual of ~1e-7*w*d^2 — an error noise floor of roughly 3e-4 * extent * d in distance terms. Since d grows with distance from the mesh bounding box minimum, the effective target_error floor is position dependent: geometry near the bbox minimum simplifies to completion while identical geometry on the far side stalls. Repro: two identical 150x150 flat plates at z=0 and z=900 in a mesh of extent 1000, simplified with target_error 5e-5 — the near plate reaches 2 triangles while the far plate keeps 38602 of 45000 (19301x asymmetry), with reported error pinned at the noise floor. With this change both plates simplify symmetrically at any target (899 total at target 900; 4 total at target 4) and the reported error drops to the true geometric value (2.6e-6). The change is confined to the Quadric/QuadricGrad fields, the products in quadricFromPlane, and the accumulation in quadricEval/quadricError; plane fitting from vertex positions can stay in single precision, as its per-triangle rounding is second-order for this effect. This doubles the memory used by quadric arrays; if that is a concern for some workloads the same change works gated behind a define. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PkB4ZiMNML9ga58hGxvixe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
meshopt_simplifyhas a position-dependent floor ontarget_error: geometryfar from the mesh bounding-box minimum cannot be simplified below an error of
roughly
3e-4 × extent, while identical geometry near the bbox minimumsimplifies to completion.
Cause
The expanded quadric form
r = vᵀAv + 2bᵀv + ccancels catastrophically insingle precision. For a plane with offset
d(after internal rescale),quadricFromPlanecomputesb = d·wandc = d²·w, whose independentfloat32 roundings don't cancel — evaluating at a point on the plane
leaves a residual
r ≈ 1e-7·w·d²instead of 0, i.e. a distance-error noisefloor of
~3e-4·extent·d. Since the internal rescale anchors at the bboxminimum,
d— and the floor — grows with distance from the min corner.Collapse candidates on exactly flat, exactly coplanar surfaces then rank
above tight error limits and the region never simplifies.
Repro
Two identical 150×150 flat plates, one at z=0 and one at z=900, in a mesh of
extent 1000 (the offset must be along the plate normal, and the far
plate's rescaled offset must be inexact in binary — z=0.9 here; a tangential
offset leaves
b = c = 0exactly and hides the effect). Simplify withtarget_index_count = index_count/100,target_error = 5e-5:Master, unable to make progress on plate B, spends the entire reduction
budget over-collapsing plate A; this PR reaches the requested target and
stops. Asked for the actual minimum (
target = 4triangles), master stillreturns 38,604 while this PR returns exactly 4 — 2 per plate.
Result meshes rendered with edges (master left/right = near/far plate, then
this PR):
Found in the wild on a 127mm CAD part where one exactly-planar box wall kept
~17M render-resolution triangles at tight
target_errorwhile thegeometrically identical opposite wall collapsed to a handful; with this
change the whole part reaches 16k triangles with measured surface deviation
of 2e-4 mm.
Change
Confined to the Quadric/QuadricGrad fields, the products in
quadricFromPlane, and the accumulation inquadricEval/quadricError(22 insertions, 19 deletions). Storing double without computing the
b/cproducts in double is not sufficient — the coefficients are already rounded
at construction. Plane fitting from vertex positions stays in single
precision; its per-triangle rounding is second-order for this effect.
Includes a regression test (
simplifyPrecisionin demo/tests.cpp) thatbuilds the two-plate geometry procedurally and asserts symmetric collapse —
it fails on master and passes with this change.
make testpasses; builds warning-free with-Wall -Wextra -Wshadow.Trade-off
This doubles quadric array memory (per remap-unique vertex) and may
measurably affect simplification throughput on some targets. If that's
unacceptable for game-LOD workloads — which never request errors below the
noise floor and are unaffected by the bug — the same change works gated
behind a compile-time define, or I'm happy to rework it in whatever
direction you prefer (recentring the internal rescale only halves the worst
case, so some precision change appears necessary for a full fix).