feat(move): add compiler-first graph ingestion#2524
Conversation
|
@zwxxb is attempting to deploy a commit to the NexusCore Team on Vercel. A member of the Team first needs to authorize it. |
e8c2c5f to
e194baf
Compare
|
@zwxxb Thanks for submitting this PR and this is certainly an interesting approach. I now understand the what is your motive but our goal is to provide a compilerless solution, so we don't have to drag "heavy" binaries with us. It's already enough to have the tree-sitters as dependencies. Could you please adopt the tree-sitter solution I shared with you and create and try to maintin the grammar? 🙏 I'm happy to integrate move language but I'd like to avoid having complete compiler integrations as they have part GitNexus will never need such as producing byte codes and other compiler backend solutions. The main reasing I'm asking for this is becaase we can then integrate it with PDG and other advanced processing that can give LLMs statement level insights. |
|
@zwxxb thanks for this, went through the full diff. @magyargergo I get the compiler-less concern, but hear me out. The compiler isn't actually embedded, move-flow runs as a pinned subprocess behind a generic standaloneIngest phase, all the Move code sits in src/core/move/, and if the binary's missing everything else works fine. We honestly already do this pattern with COBOL (standalone phase, no tree-sitter), this just formalizes the seam. On PDG: fair point, compiler facts are function-level so no statement-level analysis for Move. But tree-sitter alone can't get us resource read/write effects or address resolution anyway, and rebuilding that while mono-move is churning feels like chasing a moving target. I'd rather do compiler facts now, revisit the tree-sitter grammar once Move 2 settles. Honestly my bar here is simple: if this adds no bloat for non-Move users, doesn't touch existing functionality, and takes our Move coverage from zero to something decently usable, even if it's not 100%, I don't see a problem. That's a net win. The asks below are basically about making sure that's actually true:
@magyargergo does that address the maintenance worry or am I underweighting something? @zwxxb , is the schema split doable on your end? Let me know what you guys think. |
|
I’m happy to compromise on implementation details, but not on the architecture. Temporary exceptions have a habit of becoming permanent, and I’d rather spend a little more time now than create technical debt that affects every future language integration. |
|
COBOL is different because it did not introduce a separate compiler-owned analysis pipeline into GitNexus. We still use the same GitNexus architecture: parse the source, extract the language structure, and let GitNexus build and enrich the graph. The COBOL support required language-specific parsing and handling, but it did not make GitNexus depend on an external compiler or delegate CFG, CDG, call graph, type resolution, and semantic analysis to another tool. That distinction matters. The concern with the current Move proposal is not that Move needs special handling. Every language needs some language-specific logic. The concern is that this would create a second architecture where an external compiler produces most of the semantic model and GitNexus becomes an integration layer around it. Once we accept that pattern for one language, it becomes difficult to reject the next compiler-specific integration. Over time, GitNexus could become a collection of unrelated analysis backends rather than one consistent analysis platform. So COBOL is not an exception to the architecture. It is a language implementation within the existing architecture. The proposed Move integration would be an architectural exception. |
|
@zwxxb After discussing this further internally, I think we have a path that satisfies both the immediate goal and the long-term architecture. My concern has never been Move support itself. It’s about keeping the GitNexus core consistent and avoiding language-specific architectural exceptions in the common graph pipeline. Rather than evolving the OSS core around one ecosystem, I think a better long-term approach is to treat this as an Aptos-specific integration. That gives us the opportunity to validate adoption, provides value to the Aptos ecosystem, and keeps a clear separation between the core platform and ecosystem-specific workflows. If the integration proves successful, we can evolve it into a dedicated product/Enterprise offering (or even its own distribution) where these kinds of integrations naturally belong. That also makes ownership much clearer and avoids the OSS core gradually accumulating one-off exceptions for individual languages. I think this gives us the best of both worlds: we can move quickly today while preserving the architectural principles that have made GitNexus consistent across every supported language. |
|
@magyargergo This seems best. @zwxxb Lets do this then |
|
I'd recommend creating a new branch off from main called main-aptos and rebase the merge origin to that new branch. Additionally, we will need to update our release pipeline that will release a specific version such as 1.6.9-aptos. |
|
Sounds good. I agree on a separate Aptos release . If you create the |
Done :) |
Integrate Move compiler facts across indexing, persistence, and MCP queries while keeping shared ingestion language-agnostic and incremental updates safe. Co-authored-by: GotenJBZ <marco@freemaker.it>
Move availability and discovery into focused modules, accept literal executable paths, and split resource queries for LadybugDB compatibility.
Document the compiler-first ingestion flow and link contributors to Aptos Core, MoveFlow 2.0.0, and the Move Book. Co-authored-by: Cursor <cursoragent@cursor.com>
ca74a21 to
a67ea56
Compare
|
@zwxxb you need to update the release pipeline to support releasing GitNexus with a specific version after tagged, so you'll be able to use Also you'll need to maintain this branch to get the latest updates from us. :-) |
should be fine |
|
went through the diff properly, found a couple things worth fixing before this goes out:
rest looks good, checksum-before-extract is correct, no shell injection on the spawn calls, csv escaping for the new columns checks out, no dupe node ids. nice work overall, just want the mcp-client cleanup path fixed before merge |
Summary
move-flow2.0.0, including package-aware symbols, calls, resources, friends, attributes, and source locationsstandaloneIngestphase, with safe full rebuilds when compiler availability or package-wide Move inputs changeSponsored by @aptos-labs
Test plan
cd gitnexus && npm test— 14,358 passed, 49 skippedcd gitnexus && npx tsc --noEmitcd gitnexus && npm run buildcd gitnexus-web && npm test— 390 passedcd gitnexus-web && npx tsc -b --noEmitcd gitnexus-shared && npm run buildnpm run lintmove-flow 2.0.0analysis onetna/move/perp— 3,193 nodes, 11,818 edges, 2,503 Move nodes