Conversation
This is a heavily reduced copy of simplify.html; it is a somewhat similar structure but it only supports one model and has a much reduced simplfication pipeline. We keep the debug visualization code for now although it doesn't work correctly yet, because I'd expect that it will be useful for post-simplify topology visualization.
Making progress towards the full remeshing pipeline; we now merge all input meshes (pre-skinned at their current pose) into a single position-only mesh; that is then simplified and displayed. The normals on the mesh are auto-generated using three.js logic. Also remove the debug overlay logic for now; we will need it back but it's too messy to maintain alongside the rest of the code.
We do not have the actual remesher implemented yet so we approximate it with the sloppy simplifier; after that, the regular simplification remains as an optional step. The remesher uses resolution instead of error target, and the post-simplification just targets an error bound instead of ratio now. Running two simplification steps looks a little awkward but the initial sloppy simplify will be replaced with a remesher in the future.
We now build a BVH around the merged mesh, and also track triangle -> source geometry correspondence. This allows us to query an arbitrary point against the BVH and translate this into source mesh/triangle. We use this capability to generate vertex colors for the resampled mesh. Each vertex and each triangle center is evaluated against a very basic material model (color/map/ao/metalness); result is averaged per vertex and stored in each vertex for display. To perform resampling we need access to source images from JS. This... is more difficult than makes sense, and the most straightforward route seems to involve making a canvas, drawing the texture into the canvas and getting the pixel data. This is not very fast but we do it once per image.
In addition to the vertex painting we now provide texture painting. We use watlas to UV unwrap the remeshed geometry, and then fill it based on source material. For transfer, each triangle is rasterized into UV space and each pixel is converted back into world space, followed by the same BVH query. This is fairly slow at higher texture resolutions; there might be ways to reuse this information but also maybe the lookup can be faster in the future. The texture has unfilled areas that result in bilinear artifacts; we'll fix this separately.
We now ask watlas to leave a bit of space between charts and fill the resulting space. This could be done as a post-process, but we seem to get higher quality results on seams if instead each triangle projection covers the gutter instead and resamples the original data there. We accumulate the samples in each pixel and average them at the end; also rework color average to work off the same structure for clarity.
Remeshing pipeline is too slow to re-trigger it; so we always add another group to the new mesh and dynamically toggle material visibility instead. This only works on the remeshed mesh, not on the original geometry, but that's a reasonable limitation for now.
- Use ray cast along the normal and fall back to closest point; this improves quality if the mesh has multiple layers close to each other. - Fix behavior for non-indexed source geometry and support multiple material groups. - Change some defaults to have more reasonable performance.
We'll likely have more functions and an enum with options in the future but we'll start with just a function. The output will use a flat triangle array, with an uncertain number of per-corner floats (might need to communicate extra data to establish corner ids).
For now we use resolution-2 as the target because we'll need the grid padding to be left in tact to simplify meshing, as the outer shell should be treated as empty; this might change in the future.
In this pass, we only tag the voxels that intersect any triangles. We will need more information in boundary voxels to be able to mesh the shape accurately, but that requires a lot more storage which should not scale as N^3. For the implementation, for now instead of doing precise box-triangle intersection tests, we estimate a barycentric sampling grid and splat the points into voxels instead. This should guarantee hole-free voxelization as long as the point density is high enough that the distance between the two points is never more than a voxel's edge along any axis. For thin triangles, we currently over-sample which might be unreasonably slow; this and some other inefficiencies could be fixed later.
We use a somewhat unconventional implementation of Marching Cubes, where instead of interpolating the point on the edge, we take the value from the occupied point as is. This will need to change a bit as we add actual voxel solve, but this is necessary because the voxels we use for occupancy tracking are also going to be the unit of positional accumulation, so we don't really have data in non-occupied voxels. This doesn't yet produce a coherent mesh due to some occupancy connectivity issues, but we're getting closer. Because of this construction we produce many degenerate triangles. We could use positions to filter them out, but positions may not be available if the destination buffer is NULL, so we use vertex codes instead.
Instead of using voxel grid coordinates for the output vertices, we now accumulate triangle samples into each occupied voxel and use the resulting position during meshing. To avoid storing N^3 voxels, we compact the data and replace the original binary occupancy with 1-255 row offsets; since resolution is limited, each row can have at most 255 occupied voxels. Because we do not track the triangle indices per voxel, to accumulate the data we need to run voxelization again, using the same code and the same source triangles. It's important that this results in the same samples, otherwise the addressing logic may break and we'll end up overwriting data from other rows.
Previously, we were disabling degenerate triangle filtering when destination buffer was not provided; this resulted in the initial call reporting a higher number of triangles vs the actual number that was returned after. This is perhaps not critical and we might need to go back to the approximate capacity in the future if this helps with performance, but for now this results in more confusing behavior so we now use triangle codes to reject degenerate triangles regardless of whether positions are computed.
While it's canonical and correct to classify occupied voxels as 1 during the MC lookup, this is resulting in some holes in diagonal sheet surfaces; this appears to be due to ambiguous cases in the MC meshing being resolved in a particular way. Interestingly, flipping the categorization (treating occupied as 0 and empty as 1) fixes these gaps. Not sure if this is a durable fix and if we'll need further changes to the tables, but for now flip this - and also flip the triangle winding to maintain consistent orientation.
Instead of using original Bourke tables as is, we reindex them so that vertex codes map to the more intuitive XYZ bit order (or rather ZYX). Instead of storing edge ids and remapping them back to vertex ids, we embed this directly into the table, using nibbles to signify the edge. This is valuable because it allows us to orient each edge such that its first vertex always points at the occupied voxel. Also apply another triangle winding flip; our previous corner data was actually incorrectly indexed and flipped Y & Z, these tables should more clearly reflect original data.
With resolution voxels and first & last column being reserved for empty padding, the coordinates inside range from 1..resolution-2, and because we use 0-based coordinates in the voxelization loops, the range is 0..resolution-3. We were off by 1 which could result in writing voxel data to the boundary that should remain open. Also update resolution limits accordingly; resolution up to 256 is actually safe because that guarantees up to 254 active voxels in each row (with two values remaining for storing voxel states; we only use one but that's about to change).
A single-voxel thin sheet does polygonize from both sides; due to how we compute the positions, they are identical on both sides so we get a clean double-sided infinitely thin layer. This is fine by itself; but code running downstream of this may or may not expect this. As an option, we now allow expanding this into a thin closed plate instead. This requires reading the opposite voxel to the two voxels that the edge we are meshing connects; because we now orient edges in a consistent direction, this voxel is always in the same logical direction (O -> A -> B), and is guaranteed to be in bounds. The voxels on the side of the plate do not get offset along the orthogonal direction (e.g. if plate has a constant Z, then computing edge from the boundary that also has a constant Z does not get recognized as a thin sheet because the opposite voxel is occupied. To ensure we only get *one* layer, not two, for thin sheets, we restrict this to positive directions for now: the negative-facing side is not offset, but the positive-facing side is.
In the future it might make sense to emit cube geometry if Debug is specified, but for now it's sufficient to simply output original grid position that does not depend on the accumulation code or future solve.
- Use creased normals with a customizable cutoff instead of naive average - Show transferred texture in a 2D canvas - Add a 'solve' option for post-remesh simplification - Compact the mesh after simplification to see real vertex counts - Disable polygon offset during regular rendering
Because our voxelization does not record the side, a mesh is voxelized as a shell and meshing the resulting grid produces two layers of geometry: exterior and interior. This may be desireable for some cases but is usually inefficient and redundant. We now solidify the voxel grid by default: any voxel that does not have a path (through any of the 6 neighbors) that reaches the grid boundary is classified as 'inside'. This uses the voxel state 0xff, which is free because we only need states 1-254 to represent row offsets as there can be at most 254 occupied voxels per row. Voxels tagged as 'inside' do not have a full voxel representation: we can not compute the position for them. For now this is fine because the solidification ensures an empty and inside voxel can never be neighbors; if we need this in the future it would be possible to synthesize a fake position, but that's only necessary if we close gaps with dilation.
Instead of an explicit branch we now use a ternary to compute the new cell state, and compute changed mask by accumulating changed bits. This results in cleaner codegen on Clang and fixes vectorization problems in GCC which makes solidification noticeably faster.
For now instead of adding a separate module, expose remesh() function with flags via MeshoptSimplifier module. We'll see if this holds, but the functionality is thematically similar and most uses of remeshing probably need simplification too; by itself, remesher doesn't increase the simplifier size *that* much. This change does *not* rebuild the Wasm blob as the implementation is in flux.
We still maintain the old option to use the sloppy simplifier behind a separate setting, but by default we use the new remesher, controlled via a separate set of flags.
We now accumulate a plane quadric per voxel and solve it to get a more precise estimate of surface position compared to average. The quadrics are area weighted; for simplicity, we use area weighted positions instead too when solve is disabled. To stabilize the solve, we use the same LDL solve as the simplifier uses together with regularization. Regularization is applied during the solve, using the centroid (weighted average) as the point; note that this is equivalent to adding a point quadric to every voxel sample we accumulate, just easier/faster. For now, regularization weight is a constant and has been tuned to balance instability with quality. Further tuning might be required here in the future.
- Fix sloppy remesher setting; it no longer routed to simplifySloppy - Add solve remesher setting - Change regularization to use light regularization for better balance with the same threshold - Change default simplification threshold to 10^-2.5
Instead of reading 8 voxels per iteration, we only read 4 and use the bits of the cube mask from the previous iteration. Also we simply read the correctly offset bits now instead of using loops and going through offset masks. This makes remeshing ~10% faster on Clang and ~20% faster on GCC; Clang's codegen of the original code was better so the difference is less stark there.
Instead of solving each vertex as we emit it, we now compute the final position (minus offset) per voxel in a separate pass. This is marginally more efficient when solve is disabled (~5%) and noticeably more efficient when solve is enabled (~10-15%); the function has very predictable code flow and also each voxel is solved once as opposed to multiple times (once per edge it participates in, including degenerate edges). In some cases we may be solving additional voxels that are inside a fully closed volume, but this should still be worthwhile and in the future we could filter those out by restructuring solidification/row allocation.
Quadric accumulation costs ~7% of runtime and we only need it when solve is enabled; disable it to save a bit of time and skip redundant work. Note that we keep the existing memory structure: we still allocate and clear memory for the quadrics even if they aren't used. When solve *is* enabled, separate allocation is slower as memory access is less local, and this doesn't buy us that much time when solve is disabled. Most likely in the future solve will be enabled by default, so this is a much better balance.
A separate function makes it easier to profile and tune it independently of the rest of the code. The initial implementation had a branchy structure. While voxel occupancy often has predictable content, GCC doesn't compile that super well; we can instead use the fact that voxelize only emits binary voxel state, and accumulate voxel data directly to produce count. This ends up being lowered to branchless code with cmov for data replacement and is faster on GCC and has similar performance on Clang.
We need this to fully test the new solve implementation. Also reformat simplifier source/types with Prettier; this was missed in the previous changes as auto-formatting got disabled.
Prettier auto-formatting got disabled and a lot of this code is not formatted in line with expectations. For now just reformat the entire file until we figure out a new strategy.
Any triangle that results in weight=0 samples is not useful: the samples contribute nothing to the affected voxels. What's more problematic is the fact that if a voxel *only* receives weight=0 samples, its weight stays at 0 which means we can't compute a valid position. To fix this, skip triangles like this early so that they don't tag voxels as occupied.
Previously we were solving each voxel multiple times as the solve would be done per edge in emitVertex. Typically, most voxels we end up rasterize are on the narrow band and as such they produce more than one edge, so we repeat the work. Additionally this is done in code that has unpredictable branches which reduces IPC artificially. Instead, solve each voxel once in a separate pass, which might produce a little bit of redundant work for inner voxels but this is more than paid for with the lack of redundancy and improved efficiency. Also, in addition to regularization, this adds a fail-safe where we never replace voxel centroid with a position that's more than a voxel diagonal away from it. This is similar to the neighborhood clamping we do in the simplifier; it doesn't appear to be strictly necessary with the current (high) regularization weight, but with a lower regularization weight some quadric configurations solve into an unreasonable minimizer and this fixes that.
We were placing the vertices on the grid corners and the grid was additionally offset by 1. This resulted in positions that did not correspond to visual geometry very well. We now place these in the voxel centers instead on a correctly placed grid. Also remove RemeshDebug from the public API and use a high bit, similar to simplification; this option will likely be removed in the future.
Solve is now using bit 2 and debug is now internal.
- Add correct handling for texture-less metallic - Add handling for emissive color/texture - Fix UV transformation logic; multiple textures would result in multiple transformations of the same underlying UV vector - Fix a rare exception when material group wasn't found - Fix debug texture UV staying around on revert/reload - Fix VRAM leak as we were not disposing the live geometry objects - Adjust defaults for some parameters The output colors are still clamped at 1.0, so materials that heavily rely on emiss emissive will not preserve appearance perfectly but it's better than not using emissive at all.
This exercises most of the newly added code, and does a simple two-pass remesh similar to what we do in JS: count first, then generate. In the future we could switch to an optimistic estimate but this requires more investigation.
meshopt_remesh is now correctly tagged as MESHOPTIMIZER_EXPERIMENTAL; also move it to a more appropriate place in the file. Finally also add a C++ templated wrapper for different index types; this also allows us to default options to 0.
When thickening is disabled, our mesher operates in "pure" dual mode: each voxel gets a unique position and code, and triangles with two vertices with identical codes/positions are filtered out. When looking at a cube configuration, we know exactly which edges the triangles connect, and which corner of each edge is occupied or not. Because of this, we actually can pre-filter degenerate triangles ahead of time. For now, we use this information to make counting faster when thickening is disabled; we could also use this to accelerate emission (and skip degenerate triangles), but it doesn't appear to be profitable in the current form. When thickening is enabled, whether each edge uses the voxel position is dynamic so we can't do this processing statically.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change implements the initial version of the voxel remesher. Given a triangle mesh and the target resolution, the algorithm voxelizes the mesh, post-processes the grid to close internal areas and then reconstructs a mesh from the resulting grid, trying to match the appearance of the original mesh.
The remeshing algorithm is an unconventional hybrid: it does not use traditional triangle-voxel intersection, instead splatting barycentric samples into the grid; it does not use distance fields or Hermite data, instead accumulating quadrics representing original geometry; it uses a hybrid polygonizer with MC topology and dual-like vertices (one vertex per cell). All of this is the best compromise found so far during experimentation / prototyping but is subject to change. The output has occasional redundant connecting triangles due to how MC handles non-contiguous occupancy, which I'd like to improve if I come up with good ideas :)
Voxelization is fundamentally a tradeoff: it will merge thin features and close small holes, which is good, but it can also merge disconnected features which can look good or bad depending on where it happens. There might be ways to improve this in the future by recording supersampled occupancy and doing something intelligent with the result; for now, resolution is the main quality lever. A high resolution remesh has a high number of triangles, but it can be simplified to a pretty much arbitrary number of triangles with the existing simplifier, using regularization as necessary to achieve nice triangle shapes.
The algorithm preserves thin surfaces (sheets) and by default, meshes them as dual-sided triangles, producing an infinitely thin result.
RemeshThickenoption can expand the resulting thin surface a little bit; this currently has some false positives due to how it's implemented and would probably be reworked in the future. Wires require thickening to be preserved (as they are infinitely thin in two dimensions by default).Resulting vertices can use average placement (fairly regular result) or solved placement (minimized quadrics with regularization, for better adherence to the underlying surface at the cost of less regular output and occasional intersections). This might need to be improved in the future, as there are some issues with the resulting surface smoothness, or expanded with additional options.
The output size is not known a-priori and the worst case bound (~
5*res^3triangles) is not practically useful. It is possible to run the algorithm in two passes (remesh(NULL)to count, allocate,remesh(buf)to fill), which is ~1.4x more expensive vs fill. It's also possible to estimate the output size usingK*(res-2)^2formula (K is content dependent but could be set to 10..20 to balance memory vs success rate), and optimistically remesh using this estimate as a limit; if the output doesn't fit,remeshwill return the real count and the buffer can be reallocated and remeshing - retried (~2x more expensive vs fill).Alongside the core algorithm,
demo/remesh.htmlcontains the remeshing harness that assembles a more complex algorithm: meshopt remesh -> meshopt simplify -> generate normals -> paint. Painting can either generate vertex colors, or unwrap UVs (watlas) and generate texture; in either case, we find the point on the original mesh using ray cast / closest point query and evaluate the material. For now the baking is diffuse-only and slow on high texture resolutions as it's using single threaded JS raycasts (three-mesh-bvh).All of this is not quite ready and is in high degree of flux: "experimental" in this case really means "everything might change before the actual experimental release", but it's been baking for a while and seems to be in a good enough state to merge to master at least.
An example output (second image is remeshed & retextured output; third image uses some unpublished changes to bake a normal & metalroughness maps, this is not part of this PR for now but shows the limits of a full rebake)
This contribution is sponsored by Valve.