Skip to content

feat(move): add compiler-first graph ingestion#2524

Open
zwxxb wants to merge 4 commits into
abhigyanpatwari:main-aptosfrom
zwxxb:upstream/move-compiler-ingestion
Open

feat(move): add compiler-first graph ingestion#2524
zwxxb wants to merge 4 commits into
abhigyanpatwari:main-aptosfrom
zwxxb:upstream/move-compiler-ingestion

Conversation

@zwxxb

@zwxxb zwxxb commented Jul 16, 2026

Copy link
Copy Markdown

Summary

  • add optional compiler-first Move/Aptos indexing through pinned move-flow 2.0.0, including package-aware symbols, calls, resources, friends, attributes, and source locations
  • persist Move graph data through the shared schema/CSV/LadybugDB path and expose focused MCP tools for entries, resources, and impact
  • keep shared ingestion language-agnostic through a generic standaloneIngest phase, with safe full rebuilds when compiler availability or package-wide Move inputs change
  • keep Move discovery, CLI availability warnings, and MCP transport boundaries aligned with existing codebase modules and types

Sponsored by @aptos-labs

Test plan

  • cd gitnexus && npm test — 14,358 passed, 49 skipped
  • focused post-cleanup suite — 181 passed
  • cd gitnexus && npx tsc --noEmit
  • cd gitnexus && npm run build
  • cd gitnexus-web && npm test — 390 passed
  • cd gitnexus-web && npx tsc -b --noEmit
  • cd gitnexus-shared && npm run build
  • npm run lint
  • Prettier check for changed source files
  • live move-flow 2.0.0 analysis on etna/move/perp — 3,193 nodes, 11,818 edges, 2,503 Move nodes
  • live MCP checks — 140 entries, 32 resources, and resolved Move impact

@vercel

vercel Bot commented Jul 16, 2026

Copy link
Copy Markdown

@zwxxb is attempting to deploy a commit to the NexusCore Team on Vercel.

A member of the Team first needs to authorize it.

@zwxxb
zwxxb force-pushed the upstream/move-compiler-ingestion branch from e8c2c5f to e194baf Compare July 16, 2026 22:20
@magyargergo

magyargergo commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

@zwxxb Thanks for submitting this PR and this is certainly an interesting approach. I now understand the what is your motive but our goal is to provide a compilerless solution, so we don't have to drag "heavy" binaries with us. It's already enough to have the tree-sitters as dependencies.


Could you please adopt the tree-sitter solution I shared with you and create and try to maintin the grammar? 🙏 I'm happy to integrate move language but I'd like to avoid having complete compiler integrations as they have part GitNexus will never need such as producing byte codes and other compiler backend solutions.


The main reasing I'm asking for this is becaase we can then integrate it with PDG and other advanced processing that can give LLMs statement level insights.

@abhigyanpatwari

Copy link
Copy Markdown
Owner

@zwxxb thanks for this, went through the full diff. @magyargergo I get the compiler-less concern, but hear me out.

The compiler isn't actually embedded, move-flow runs as a pinned subprocess behind a generic standaloneIngest phase, all the Move code sits in src/core/move/, and if the binary's missing everything else works fine. We honestly already do this pattern with COBOL (standalone phase, no tree-sitter), this just formalizes the seam.

On PDG: fair point, compiler facts are function-level so no statement-level analysis for Move. But tree-sitter alone can't get us resource read/write effects or address resolution anyway, and rebuilding that while mono-move is churning feels like chasing a moving target. I'd rather do compiler facts now, revisit the tree-sitter grammar once Move 2 settles.

Honestly my bar here is simple: if this adds no bloat for non-Move users, doesn't touch existing functionality, and takes our Move coverage from zero to something decently usable, even if it's not 100%, I don't see a problem. That's a net win. The asks below are basically about making sure that's actually true:

  • get the Move columns off the shared node tables (side-table or extended-props, right now every user eats a full re-index for a language they don't use)
  • drop the 'moveIngest' literal from the shared phase enum
  • make the install lazy instead of postinstall so non-Move users never fetch the binary
  • short note documenting the boundary + migration intent

@magyargergo does that address the maintenance worry or am I underweighting something? @zwxxb , is the schema split doable on your end?

Let me know what you guys think.

@magyargergo

Copy link
Copy Markdown
Collaborator

I’m happy to compromise on implementation details, but not on the architecture. Temporary exceptions have a habit of becoming permanent, and I’d rather spend a little more time now than create technical debt that affects every future language integration.

@magyargergo

Copy link
Copy Markdown
Collaborator

COBOL is different because it did not introduce a separate compiler-owned analysis pipeline into GitNexus.

We still use the same GitNexus architecture: parse the source, extract the language structure, and let GitNexus build and enrich the graph. The COBOL support required language-specific parsing and handling, but it did not make GitNexus depend on an external compiler or delegate CFG, CDG, call graph, type resolution, and semantic analysis to another tool.

That distinction matters.

The concern with the current Move proposal is not that Move needs special handling. Every language needs some language-specific logic. The concern is that this would create a second architecture where an external compiler produces most of the semantic model and GitNexus becomes an integration layer around it.

Once we accept that pattern for one language, it becomes difficult to reject the next compiler-specific integration. Over time, GitNexus could become a collection of unrelated analysis backends rather than one consistent analysis platform.

So COBOL is not an exception to the architecture. It is a language implementation within the existing architecture. The proposed Move integration would be an architectural exception.

@magyargergo

Copy link
Copy Markdown
Collaborator

@zwxxb After discussing this further internally, I think we have a path that satisfies both the immediate goal and the long-term architecture.

My concern has never been Move support itself. It’s about keeping the GitNexus core consistent and avoiding language-specific architectural exceptions in the common graph pipeline.

Rather than evolving the OSS core around one ecosystem, I think a better long-term approach is to treat this as an Aptos-specific integration. That gives us the opportunity to validate adoption, provides value to the Aptos ecosystem, and keeps a clear separation between the core platform and ecosystem-specific workflows.

If the integration proves successful, we can evolve it into a dedicated product/Enterprise offering (or even its own distribution) where these kinds of integrations naturally belong. That also makes ownership much clearer and avoids the OSS core gradually accumulating one-off exceptions for individual languages.

I think this gives us the best of both worlds: we can move quickly today while preserving the architectural principles that have made GitNexus consistent across every supported language.

@abhigyanpatwari

Copy link
Copy Markdown
Owner

@magyargergo This seems best. @zwxxb Lets do this then

@magyargergo

Copy link
Copy Markdown
Collaborator

I'd recommend creating a new branch off from main called main-aptos and rebase the merge origin to that new branch. Additionally, we will need to update our release pipeline that will release a specific version such as 1.6.9-aptos.

@zwxxb

zwxxb commented Jul 18, 2026

Copy link
Copy Markdown
Author

Sounds good. I agree on a separate Aptos release .

If you create the main-aptos branch and the related release setup, I can rebase this PR onto it.

@magyargergo
magyargergo changed the base branch from main to main-aptos July 18, 2026 13:33
@magyargergo

Copy link
Copy Markdown
Collaborator

Sounds good. I agree on a separate Aptos release .

If you create the main-aptos branch and the related release setup, I can rebase this PR onto it.

Done :)

zwxxb and others added 3 commits July 18, 2026 15:37
Integrate Move compiler facts across indexing, persistence, and MCP queries while keeping shared ingestion language-agnostic and incremental updates safe.

Co-authored-by: GotenJBZ <marco@freemaker.it>
Move availability and discovery into focused modules, accept literal executable paths, and split resource queries for LadybugDB compatibility.
Document the compiler-first ingestion flow and link contributors to Aptos Core, MoveFlow 2.0.0, and the Move Book.

Co-authored-by: Cursor <cursoragent@cursor.com>
@zwxxb
zwxxb force-pushed the upstream/move-compiler-ingestion branch from ca74a21 to a67ea56 Compare July 18, 2026 13:40
@zwxxb
zwxxb marked this pull request as ready for review July 18, 2026 13:40
@zwxxb
zwxxb requested a review from azizur100389 as a code owner July 18, 2026 13:40
@magyargergo

magyargergo commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

@zwxxb you need to update the release pipeline to support releasing GitNexus with a specific version after tagged, so you'll be able to use gitnexus@x.x.x

Also you'll need to maintain this branch to get the latest updates from us. :-)

@zwxxb

zwxxb commented Jul 18, 2026

Copy link
Copy Markdown
Author

@zwxxb you need to update the release pipeline to support releasing GitNexus with a specific version after tagged, so you'll be able to use gitnexus@x.x.x

Also you'll need to maintain this branch to get the latest updates from us. :-)

should be fine

@abhigyanpatwari

Copy link
Copy Markdown
Owner

went through the diff properly, found a couple things worth fixing before this goes out:

  • mcp-client.ts: the 30s init timeout doesn't kill the child process or reset client state if move-flow is slow to boot or crashes on launch. leaves an orphaned process running and permanently wedges the client for that run (no retry). there's also a race where a late init response can sneak in after the timeout already rejected

  • install-move-flow.cjs uses JSON.stringify to quote a path inside a powershell -Command string, that's wrong escaping for powershell specifically. not exploitable today since the inputs are hardcoded but worth fixing properly

  • also no error handler on the download stream itself, a network error mid-download could throw instead of hitting the soft-fail path the script is supposed to guarantee

rest looks good, checksum-before-extract is correct, no shell injection on the spawn calls, csv escaping for the new columns checks out, no dupe node ids. nice work overall, just want the mcp-client cleanup path fixed before merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants