Conversation
In addition to vK_ prefixed files we now support iK_ prefixed files that contain encoded index data. The file names encode the index count; we do not technically know the max index. To exercise both 16-bit and 32-bit decoding paths, we guess the maximum index from the index count.
fec == 0 branch is difficult to predict; the fast path (edge reuse) decoding is intentionally written in a way that promotes branchless code generation, as reducing the mispredictions saves more time than the extra executed instructions take. Unfortunately, Clang's x64-cmov-conversion pass assumes that cmov dependent on a memory read (vertexfifo here) is unprofitable and converts it back to a branch, regressing the performance. Because the issue is Clang specific, we can use a Clang specific attribute to flag that the branch is unpredictable. This fixes codegen to use cmov on Clang when targeting x86_64 and makes decoding ~10% faster. This was not a problem for other architectures like aarch64.
We rework the code to avoid materializing the triangle index; we are short on registers and this code is sensitive to the choice of induction variables. This also reduces the address generation complexity for the triangle stores. Also take this opportunity to move the data bounds check inside the two slow paths it's necessary in; two fast paths don't read data and don't need it. This makes data_safe_end less hot which frees up a register too. And replace a branchless +/-1 with simpler code. Altogether this results in ~5% faster index decoding on clang/gcc.
zeux
force-pushed
the
idxopt
branch
2 times, most recently
from
July 10, 2026 19:56
1d9c6dc to
614184c
Compare
gcc has the same problem as clang, but it can't be solved using an attribute: gcc doesn't support __builtin_unpredictable and __builtin_expect_with_probability does not affect the codegen here. Additionally, some versions of gcc (gcc-16+, possibly earlier versions) can fuse two reads from edgefifo[] for a/b into a single 64-bit load, but keep separate stores into edgefifo[]. This breaks store to load forwarding, as it usually requires that a load is covered by a single store, which results in further significant performance loss. Both of these can be fixed with an asm barrier: cf is forced to be materialized in a register which prevents folding it into the ternary, at which point the select is simple and emits cmov; the +r restriction on a also blocks fusing the loads into a single load. While clang could technically keep using the attribute workaround, the same workaround appears to work fine for it, so for simplicity we use the same asm barrier for both compilers for now. This makes index decoding ~10-12% faster for gcc14/15 and ~19% faster for gcc16.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change implements several optimizations to existing index decoder,
mainly targeting Clang/GCC on x86-64.
fec == 0branch is difficult to predict; the fast path (edge reuse)decoding is intentionally written in a way that promotes branchless code
generation, as reducing the mispredictions saves more time than the extra
executed instructions take.
Unfortunately, both clang and gcc compile it as a branch, for two different
(bad) reasons. clang uses x86-cmov-conversion pass that replaces select
with a branch because one of the sources is a memory load. This could be fixed
with
__builtin_unpredictable; however, GCC doesn't support it. GCC takesthe code that loads
cfand moves it into the ternary, which then makes theternary look like a branch where one side is expensive and one side is cheap.
Attempts to use
__builtin_expect_with_probabilitydo not help.What does help is using an
asmbarrier that usescf; this fixes both problemson both compilers. Additionally, GCC has problems with
edgefifouse on someversions: it can fuse
a/bloads into a single 64-bit load but keepedgefifostores as 32-bit, which breaks store-to-load forwarding and results in significant
performance loss too.
Once these issues are fixed, clang suffers from register pressure because of the
indexed load. So this change also reworks the loop structure so that we never
need the triangle index. Finally, the
databounds checks are only needed onthe paths that read from
data, and these paths are fairly cold so we can movethese there too.
Altogether this makes index decoder ~10-20% faster depending on the compiler
and data set. MSVC performance is not affected to a significant degree; it already
correctly emitted
cmovbut has some other weaknesses that are more difficultto fix, so Clang produces the fastest code. Clang can get up to 25% faster on
large meshes with few seams because the fast path is used for the majority of
the triangles.
This contribution is sponsored by Valve.