Skip to content

check uri blacklist before fetching animation_url in nft parser#20186

Open
madib06ops wants to merge 1 commit into
aptos-labs:mainfrom
madib06ops:animation-uri-blacklist
Open

check uri blacklist before fetching animation_url in nft parser#20186
madib06ops wants to merge 1 commit into
aptos-labs:mainfrom
madib06ops:animation-uri-blacklist

Conversation

@madib06ops

@madib06ops madib06ops commented Jul 3, 2026

Copy link
Copy Markdown

Description

raw_image_uri is checked against uri_blacklist before it is fetched, but the raw_animation_uri branch parses and optimizes the URI without that check. A token whose JSON metadata sets animation_url to a blacklisted entry is still fetched and optimized, so the blacklist is only half enforced. Both image and animation_url come from the same untrusted on-chain metadata, so an operator that blacklists a host to stop the crawler from reaching it can still be driven to it through the animation field. This adds the same check on the animation path, mirroring the image branch (skip, mark do_not_parse, bump the blacklist skip counter, return early).

How Has This Been Tested?

cargo check -p aptos-nft-metadata-crawler and cargo clippy -p aptos-nft-metadata-crawler are clean. The new branch is identical in shape to the existing raw_image_uri blacklist branch, which is exercised by the same parse flow. Worker requires a live Postgres connection, so there is no unit-test harness for this path to extend.

Key Areas to Review

The animation branch returns Ok(()) after setting do_not_parse, matching how a blacklisted image URI is handled (both skip the trailing retry/success bookkeeping). Worth confirming that early-return is the intended behavior for a blacklisted animation, same as for a blacklisted image.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Note

Low Risk
Small, symmetric guard in the NFT metadata crawler parser with no auth or schema changes; behavior aligns with existing image blacklist handling.

Overview
The NFT metadata crawler already blocked raw_image_uri against uri_blacklist before fetch/optimize, but animation_url could still hit blacklisted hosts. This change runs the same is_blacklisted_uri check on raw_animation_uri immediately before URIParser::parse and animation optimization.

On a match, the worker sets do_not_parse, upserts, increments SKIP_URI_COUNT with label blacklist, and returns early—matching the image and top-level asset URI blacklist behavior so operators cannot be driven to blocked hosts via metadata animation_url.

Reviewed by Cursor Bugbot for commit 3c066ba. Bugbot is set up for automated code reviews on this repo. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant