Skip to content

011.md

Latest commit

 

History

History
50 lines (32 loc) · 2.11 KB

011.md

File metadata and controls

50 lines (32 loc) · 2.11 KB

Track a Dataset with DVC

Problem

A teammate has added the transactions dataset to the xFusionCorp Industries fraud-detection repository, but it was committed directly to Git instead of being tracked with DVC. Bring the repository in line with the team standard—every dataset under data/ must be tracked by DVC, not by Git.

  1. A project exists at /root/code/fraud-detection/ with DVC already initialised. The dataset data/raw/transactions.csv is currently tracked by Git, and the team standard requires DVC to own it instead.

  2. Stop Git from tracking the dataset without deleting it from disk.

  3. Track the same dataset with DVC so a .dvc pointer file is produced and data/raw/.gitignore excludes the dataset itself.

  4. Stage the new .dvc pointer and the new .gitignore, then record a Git commit with the message Track transactions dataset with DVC.

Once tracking is moved to DVC, the DVC TRACKED section in the EXPLORER panel will list the dataset, confirming the extension recognises it as a DVC-managed file.

Solution

  1. First remove git tracking of /data/raw/transactions.csv file:

    cd fraud-detection
git rm --cached data/raw/transactions.csv

  2. Add datasets to dvc tracking:

    dvc add data/raw/transactions.csv

  3. Check if .gitignore is created or not under /data/raw directory, otherwise create it manually

    echo "transactions.csv" > /data/raw/.gitignore

  4. Stage the changes and commit:

    git add .
git commit -m "Track transactions dataset with DVC"

Best Practices

  • Keep large or mutable datasets out of Git history; let DVC manage files under data/ so commits stay small and reviewable.
  • Use git rm --cached before dvc add when a file is already tracked by Git, so the file stays on disk but Git stops owning it.
  • Commit the generated .dvc pointer file and the accompanying .gitignore together to keep the repository state consistent.
  • Verify that the dataset appears in the DVC TRACKED section after the change; that is the quickest confirmation that the file is managed correctly.