Skip to content

Nb/ensdb byte parity #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nikbhintade wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Nb/ensdb byte parity #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1ca35b8
feat(parity): byte-for-byte ENSDb parity + subgraph-compat mode
nikbhintade Jun 11, 2026
451a931
test(validation): DB comparison script + validation configs
nikbhintade Jun 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
379 changes: 379 additions & 0 deletions config.validation-370k.yaml
View file
Open in desktop

Large diffs are not rendered by default.

379 changes: 379 additions & 0 deletions config.validation.yaml
View file
Open in desktop

Large diffs are not rendered by default.

25 changes: 8 additions & 17 deletions config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ contracts:
field_selection:
transaction_fields:
- hash
- from

@coderabbitai coderabbitai Bot Jun 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Search for NewOwner handler usage of transaction.from

# Check Registry and RegistryOld handlers
rg -n 'NewOwner' --type=ts -A 10 -B 2

# Check for transaction.from usage in handlers
rg -n 'transaction\.from' --type=ts src/handlers/

Repository: enviodev/ens-indexer

Length of output: 18455

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Find sharedEventValues definition
rg -n "sharedEventValues" -S src | head -n 50

# 2) Check where subgraph_new_owners is set and whether any `from` field is included directly
rg -n "subgraph_new_owners\.set\(" -S src/handlers

# 3) Search for direct assignment of a `from` property inside subgraph_new_owners writes
rg -n "subgraph_new_owners\.set\\(\\{[^}]*from" -S src/handlers || true

# 4) Inspect config.yaml around line 13
if [ -f config.yaml ]; then
  echo "----- config.yaml (lines 1-60) -----"
  sed -n '1,60p' config.yaml | cat -n
fi

# 5) Look for `from` field in the NewOwner entity/schema if present in repo (best-effort)
rg -n "subgraph_new_owners|NewOwner" -S schema.graphql src | head -n 100 || true

Repository: enviodev/ens-indexer

Length of output: 7489

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Inspect sharedEventValues implementation
echo "----- sharedEventValues (src/lib/helpers.ts around 195) -----"
sed -n '160,260p' src/lib/helpers.ts | cat -n

# 2) Inspect subgraph_new_owners entity in schema.graphql
echo "----- subgraph_new_owners in schema.graphql -----"
# print around the type definition
sed -n '40,110p' schema.graphql | cat -n

# 3) Confirm config.yaml includes transaction_fields.from for ThreeDNSToken.NewOwner
echo "----- ThreeDNSToken NewOwner config (config.yaml) -----"
rg -n "ThreeDNSToken|ThreeDNSToken.*NewOwner|event: \"NewOwner" config.yaml
rg -n "ThreeDNSToken" config.yaml -n --context 2

Repository: enviodev/ens-indexer

Length of output: 6811

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Search schema.graphql for any `from` field on new owners-related entities
echo "----- schema.graphql: any 'from' references -----"
rg -n "\\bfrom\\b" schema.graphql || true

# 2) Show exact config.yaml block for ThreeDNSToken NewOwner (incl transaction_fields if present)
echo "----- config.yaml: ThreeDNSToken NewOwner block (around first occurrence) -----"
# Print around the first NewOwner event for ThreeDNSToken
python3 - <<'PY'
import re, itertools
path="config.yaml"
with open(path,'r',encoding='utf-8') as f:
    lines=f.readlines()
# find line indexes of ThreeDNSToken NewOwner event
idxs=[i for i,l in enumerate(lines) if 'name: ThreeDNSToken' in l]
# find first 'event: "NewOwner' after a ThreeDNSToken name
first=None
for i in idxs:
    for j in range(i, min(i+200, len(lines))):
        if 'event: "NewOwner' in lines[j]:
            first=j
            break
    if first is not None:
        break
if first is None:
    print("Could not find ThreeDNSToken NewOwner event block.")
    raise SystemExit(0)
start=max(0, first-40)
end=min(len(lines), first+80)
for k in range(start,end):
    ln=k+1
    print(f"{ln:4d}\t{lines[k].rstrip()}")
PY

# 3) Check all writes to subgraph_new_owners and see if any include `from`
echo "----- Handlers: subgraph_new_owners.set writes that mention from -----"
rg -n "subgraph_new_owners\\.set\\(" -S src/handlers
echo "----- Handlers: any writes including `from:` -----"
rg -n "subgraph_new_owners\\.set\\(\\{[^}]*from\\s*:" -S src/handlers || true

# 4) Check if any handler reads event.transaction.from inside Registry/ThreeDNS NewOwner flow
echo "----- Handlers: NewOwner handlers that reference transaction.from -----"
rg -n "NewOwner|handleNewOwner|contract: \"RegistryOld\"|contract: \"Registry\"|contract: \"ThreeDNSToken\"" src/handlers/Registry.ts src/handlers/ThreeDNS.ts -S
rg -n "transaction\\.from" src/handlers/Registry.ts src/handlers/ThreeDNS.ts -S || true

Repository: enviodev/ens-indexer

Length of output: 8706

NewOwner transaction.from is selected in config.yaml but not persisted by any NewOwner handler

  • config.yaml requests transaction_fields: - from for RegistryOld.NewOwner, Registry.NewOwner, and ThreeDNSToken.NewOwner, but sharedEventValues() only sets transaction_id (tx hash) and schema.graphql’s subgraph_new_owners has no from field.
  • src/handlers/Registry.ts and src/handlers/ThreeDNS.ts both write context.subgraph_new_owners.set({ ...sharedEventValues(...), ... }) without adding a from value; event.transaction.from is only used in Registry.ts for label healing, not stored on the NewOwner entity.

Update the GraphQL entity/types to include a from field and set it from event.transaction.from in all NewOwner handlers (then run pnpm codegen), or remove transaction_fields: from from config.yaml if it’s intentionally unused.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@config.yaml` at line 13, config.yaml declares transaction_fields: - from for
RegistryOld.NewOwner, Registry.NewOwner, and ThreeDNSToken.NewOwner but no
handler persists it; update either the GraphQL entity and handlers to persist
the field or remove the config entry. Specifically, add a "from" field to the
subgraph_new_owners GraphQL type/schema, run pnpm codegen, and in
src/handlers/Registry.ts and src/handlers/ThreeDNS.ts include
event.transaction.from when calling context.subgraph_new_owners.set (combine it
with sharedEventValues(...) so the saved object contains from), or alternatively
delete the transaction_fields: from lines from config.yaml if you intend not to
store it.

Source: Coding guidelines

- event: "Transfer(bytes32 indexed node, address owner)"
field_selection:
transaction_fields:
Expand All @@ -29,6 +30,7 @@ contracts:
field_selection:
transaction_fields:
- hash
- from
- event: "Transfer(bytes32 indexed node, address owner)"
field_selection:
transaction_fields:
Expand Down Expand Up @@ -218,6 +220,7 @@ contracts:
field_selection:
transaction_fields:
- hash
- from
- event: "Transfer(bytes32 indexed node, address owner)"
field_selection:
transaction_fields:
Expand Down Expand Up @@ -369,17 +372,8 @@ chains:
- "0x00000000000000adc04c56bf30ac9d3c0aaf14dc"
start_block: 17129405
- name: Resolver
address:
# Static reverse resolver addresses (PA plugin)
- "0x231b0ee14048e9dccd1d247744d114a4eb5e8e63" # DefaultReverseResolver2
- "0xa7d635c8de9a58a228aa69353a1699c7cc240dcf" # DefaultReverseResolver3
- "0xc800dbc8ff9796e58efba2d7b35028ddd1997e5e" # BaseReverseResolver
- "0x0ce08a41bdb10420fb5cac7da8ca508ea313aef8" # LineaReverseResolver
- "0xf9edb1a21867ac11b023ce34abad916d29abf107" # OptimismReverseResolver
- "0x4b9572c03aaa8b0efa4b4b0f0cc0f0992bedb898" # ArbitrumReverseResolver
- "0xc4842814ca523e481ca5aa85f719fed1e9cac614" # ScrollReverseResolver
start_block: 16925619
# Also dynamically registered via Registry.NewResolver contractRegister
# No address = wildcard: index resolver events from all contracts
# (matches the reference ENSIndexer Resolver datasource)
- name: StandaloneReverseRegistrar
address:
- "0x283f227c4bd38ece252c4ae7ece650b0e913f1f9"
Expand Down Expand Up @@ -411,9 +405,7 @@ chains:
address:
- "0xbb7b805b257d7c76ca9435b3ffe780355e4c4b17"
- name: Resolver
address:
- "0xf97aac6c8dbaebcb54ff166d79706e3af7a813c8" # ThreeDNS resolver (static)
# Also dynamically registered via Registry.NewResolver contractRegister
# No address = wildcard (covers ThreeDNS resolver too)
- name: StandaloneReverseRegistrar
address:
- "0x0000000000d8e504002cc26e3ec46d81971c1664"
Expand All @@ -437,7 +429,7 @@ chains:
- "0xdb75db974b1f2bd3b5916d503036208064d18295"
start_block: 6682978
- name: Resolver
# Dynamically registered via Registry.NewResolver contractRegister
# No address = wildcard
- name: StandaloneReverseRegistrar
address:
- "0x0000000000d8e504002cc26e3ec46d81971c1664"
Expand All @@ -449,8 +441,7 @@ chains:
address:
- "0xbb7b805b257d7c76ca9435b3ffe780355e4c4b17"
- name: Resolver
address:
- "0xf97aac6c8dbaebcb54ff166d79706e3af7a813c8" # ThreeDNS resolver (static)
# No address = wildcard (covers ThreeDNS resolver too)
- name: StandaloneReverseRegistrar
address:
- "0x0000000000d8e504002cc26e3ec46d81971c1664"
Expand Down
175 changes: 175 additions & 0 deletions docs/ENSDB_PARITY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
# ENSDb DB-Level Parity — Gap Analysis

Context: Step 1 of the Envio↔ENS collaboration scopes Envio as a compatible
**EnsDbWriter** for the ENS Subgraph data model. The ENSDb standard means an
ENSDbReader (e.g. [ENSApi](https://github.com/namehash/ensnode/tree/main/apps/ensapi))
reads the writer's Postgres directly via its drizzle schema. GraphQL-level
parity (verified by `ens-subgraph-transition-tools`) is therefore necessary but
not sufficient — the **physical Postgres surface** (table names, column names,
column types, metadata tables) must also line up, or the standard must evolve
to accommodate writer-specific naming (which lightwalker explicitly offered).

This document maps the current envio output (verified against a live
`envio@3.0.0-rc.0` database) to the ENSNode/Ponder reference (from
`apps/ensindexer`'s schema files) and lists options for closing each gap.

## 1. Table names

Envio derives the physical table name directly from the schema entity name.
Entities were renamed (branch `nb/ensdb-naming-parity`) to match the ENSNode
ponder schema **export** names — but Ponder's physical table names come from
the first argument of `onchainTable(...)`, which is pluralized for most
entities.

| envio table (actual) | ENSDb table (ponder reference) | gap |
| ---------------------------------- | ------------------------------------- | ---------- |
| `subgraph_domain` | `subgraph_domains` | plural |
| `subgraph_account` | `subgraph_accounts` | plural |
| `subgraph_resolver` | `subgraph_resolvers` | plural |
| `subgraph_registration` | `subgraph_registrations` | plural |
| `subgraph_wrapped_domain` | `subgraph_wrapped_domains` | plural |
| `subgraph_transfer` | `subgraph_transfers` | plural |
| `subgraph_new_owner` | `subgraph_new_owners` | plural |
| `subgraph_new_resolver` | `subgraph_new_resolvers` | plural |
| `subgraph_new_ttl` | `subgraph_new_ttls` | plural |
| `subgraph_wrapped_transfer` | `subgraph_wrapped_transfers` | plural |
| `subgraph_name_wrapped` | `subgraph_name_wrapped` | none ✓ |
| `subgraph_name_unwrapped` | `subgraph_name_unwrapped` | none ✓ |
| `subgraph_fuses_set` | `subgraph_fuses_set` | none ✓ |
| `subgraph_expiry_extended` | `subgraph_expiry_extended` | none ✓ |
| `subgraph_name_registered` | `subgraph_name_registered` | none ✓ |
| `subgraph_name_renewed` | `subgraph_name_renewed` | none ✓ |
| `subgraph_name_transferred` | `subgraph_name_transferred` | none ✓ |
| `subgraph_*_changed` (10 tables) | `subgraph_*_changed` | none ✓ |
| `resolver` | `resolvers` | plural |
| `resolver_records` | `resolver_records` | none ✓ |
| `resolver_address_record` | `resolver_address_records` | plural |
| `resolver_text_record` | `resolver_text_records` | plural |
| `domain_resolver_relation` | `domain_resolver_relations` | plural |
| `reverse_name_record` | `reverse_name_records` | plural |
| `migrated_node_by_node` | `migrated_nodes_by_node` | plural |
| — | `migrated_nodes_by_parent` | not built (Ponder prefetch optimization; envio doesn't need it) |
| `subregistry` | `subregistries` | plural |
| `registration_lifecycle` | `registration_lifecycles` | plural |
| `registrar_action` | `registrar_actions` | plural |
| `internal_registrar_action_metadata` | `_ensindexer_registrar_action_metadata` | name + leading underscore |
| `name_token` | `name_tokens` | plural |
| `name_sale` | `name_sales` | plural |

Net: ~19 tables differ only by pluralization; 17 already exact.

Why not just rename entities to the plural table names: envio couples entity
name = GraphQL type = Hasura field = TS context key = table name. Plural
entity names are livable, but the cleaner fix is decoupling (see options).

## 2. Column names

envio uses the schema field name verbatim as the column name (camelCase).
Ponder runs drizzle with snake_case casing, so its physical columns are
snake_case versions of the TS property names.

Verified envio columns vs expected ENSDb columns (subgraph_registration):

| envio column (actual) | ENSDb column (expected) | gap |
| --------------------- | ----------------------- | --- |
| `id` | `id` | ✓ |
| `domain_id` | `domain_id` | ✓ |
| `registrationDate` | `registration_date` | case |
| `expiryDate` | `expiry_date` | case |
| `cost` | `cost` | ✓ |
| `registrant_id` | `registrant_id` | ✓ |
| `labelName` | `label_name` | case |

Pattern: relationship columns (`*_id`) already match because envio's `_id`
convention coincides with drizzle's snake_case of ponder's `*Id` fields.
Multi-word value columns (`subdomainCount`, `createdAt`, `isMigrated`,
`contentHash`, `transactionID`, `blockNumber`, …) differ by case only.

⚠️ TO VERIFY against a live ENSDb dump: drizzle's exact snake_case output for
irregular names (`transactionID` → `transaction_id` vs `transaction_i_d`;
`contentHash` → `content_hash`). Do this before any rename work.

## 3. Column types

| schema type | envio (actual) | ponder (reference) | compatible? |
| ----------- | ------------------ | ------------------------- | ----------- |
| `ID!`/refs | `text` | `text` (`t.hex()`/`t.text()`) | ✓ (hex stored as lowercase text both sides) |
| `BigInt` | `numeric` | `numeric(78)` | ✓ reads fine; precision declaration differs |
| `Int` | `integer` | `t.integer()` → `integer` | ✓ |
| `chainId` | `integer` | `t.int8({mode:"number"})` → `bigint` | type width differs; reads coerce |
| `Boolean` | `boolean` | `boolean` | ✓ |
| `[String!]` | `text[]` | `text[]` | ✓ |
| `[BigInt!]` | `text[]` (!) | `numeric[]` (ponder `t.bigint().array()`) | ✗ element type differs (`subgraph_resolver.coinTypes`) |

Also: graph-node uses collation `"C"` for string ordering; ENSNode
monkeypatches `subgraph_domains.name/label_name` to `COLLATE "C"`. envio
columns use DB default collation — affects `ORDER BY` results on name columns
at the reader layer. Lowercase-hex `id` columns are unaffected.

## 4. Indexing metadata tables

| envio | ponder/ENSDb |
| ---------------------------------------- | ----------------------------- |
| `envio_chains`, `envio_addresses`, `envio_checkpoints`, `envio_info` | `_ponder_meta`, `_ponder_status`, `_ponder_checkpoint` |
| `envio_history_*` (reorg history, per entity) | `_reorg__*` (per table) |
| `chain_metadata`, `_meta`, `raw_events` | — |

No overlap. This is the "first engineering problem" lightwalker scoped:
ENSDb's metadata model is mid-refactor on their side, and they explicitly
offered to evolve the standard so envio's representation fits naturally.
Deliverable here is a proposal, not a rename: define what an ENSDbReader
actually needs (per-chain indexed height, readiness, reorg safety horizon) and
map both writers' native tables onto that contract.

## 5. Options for closing the gaps

**A. Postgres view layer (recommended short-term).** Ship a SQL migration that
creates an `ensdb` schema of views over the envio tables:

```sql
CREATE VIEW ensdb.subgraph_registrations AS
SELECT id,
domain_id,
"registrationDate" AS registration_date,
"expiryDate" AS expiry_date,
cost,
registrant_id,
"labelName" AS label_name
FROM public.subgraph_registration;
```

- Zero envio changes, zero reindex, reversible, testable today (point ENSApi at
`ensdb` schema).
- Covers table names, column names, and the metadata contract (views over
`envio_chains` emulating `_ponder_status`).
- Cost: views are read-only surface; writes stay on envio names (fine — reader
standard only needs reads). Type gaps (`coinTypes text[]` vs `numeric[]`)
can be cast in the view.

**B. envio feature: physical-name mapping.** `@config(table: "...")`-style
directive (or config-level map) decoupling entity name from table name +
snake_case column option. Best long-term DX; needs envio core work —
candidate for the Step-2 feature-prioritization agreement.

**C. Evolve the ENSDb standard.** Per lightwalker: "we want to listen to your
perspectives and actually evolve the ENSDb standard so that it can more
naturally be achieved by Envio." Propose the standard specify a small
**naming manifest** (logical entity → physical table/column map) that readers
like ENSApi consume, instead of hardcoding ponder's drizzle output. Then both
writers comply natively and option A/B become optional.

Recommended sequence: A now (unblocks ENSApi-on-envio experiments), C as the
standards conversation, B if/when C lands on writer-native naming.

## 6. Current status

- [x] GraphQL-level entity naming aligned to ENSNode schema exports
- [x] snapshot-eq harness extended with envio mode (transforms verified on
synthetic data; see `ens-subgraph-transition-tools` clone)
- [x] `config.validation.yaml` — mainnet-only subgraph scope for the
validation sync (excludes L2 chains + PA statics + registrars/tokenscope
contracts so `subgraph_*` tables receive only subgraph-equivalent data)
- [ ] Validation sync + snapshot diff (requires sync to target blockheight)
- [ ] Verify drizzle snake_case output against live ENSDb dump (§2)
- [ ] View-layer migration (option A) if DB-level reads are pursued before
the standard evolves
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
"vitest": "4.0.16"
},
"dependencies": {
"@adraffy/ens-normalize": "1.11.1",
"dns-packet": "^5.6.1",
"envio": "3.0.0-rc.0",
"viem": "^2.46.3"
Expand Down
3 changes: 3 additions & 0 deletions pnpm-lock.yaml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading