Skip to content

Commit 63db545

Browse files
lidellidel
authored
Merge pull request #530 from ipfs/cid
Spec for CIDs
2 parents 8721f87 + 9240173 commit 63db545

3 files changed

Lines changed: 264 additions & 4 deletions

File tree

‎src/architecture/principles.md‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,9 @@ editors:
1212
url: https://berjon.com/
1313
github: darobin
1414
twitter: robinberjon
15-
mastodon: "@robin@mastodon.social"
1615
affiliation:
17-
name: Protocol Labs
18-
url: https://protocol.ai/
16+
name: IPFS Foundation
17+
url: https://ipfsfoundation.org/
1918
tags: ['architecture']
2019
order: 0
2120
xref:

‎src/data-formats/cid.md‎

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
---
2+
title: CID (Content IDentifier)
3+
description: >
4+
Self-describing content-addressed identifiers for distributed systems
5+
date: 2026-06-26
6+
maturity: permanent
7+
editors:
8+
- name: Marcin Rataj
9+
github: lidel
10+
affiliation:
11+
name: Interplanetary Shipyard
12+
url: https://ipshipyard.com/
13+
- name: Robin Berjon
14+
email: robin@berjon.com
15+
url: https://berjon.com/
16+
github: darobin
17+
twitter: robinberjon
18+
affiliation:
19+
name: IPFS Foundation
20+
url: https://ipfsfoundation.org/
21+
former_editors:
22+
- name: Juan Benet
23+
github: jbenet
24+
thanks:
25+
- name: Steven Allen
26+
github: Stebalien
27+
- name: Rod Vagg
28+
github: rvagg
29+
- name: bumblefudge
30+
github: bumblefudge
31+
- name: Volker Mische
32+
github: vmx
33+
- name: Joel Thorstensson
34+
github: oed
35+
- name: Oli Evans
36+
github: olizilla
37+
38+
tags: ['data-formats']
39+
order: 1
40+
---
41+
42+
**CID** is a format for referencing content in distributed information systems, like [IPFS](https://ipfs.tech).
43+
It leverages [content addressing](https://en.wikipedia.org/wiki/Content-addressable_storage),
44+
[cryptographic hashing](https://simple.wikipedia.org/wiki/Cryptographic_hash_function), and
45+
[self-describing formats](https://github.com/multiformats/multiformats).
46+
It is the core identifier used by [IPFS](https://ipfs.tech) and [IPLD](https://ipld.io).
47+
It uses a [multicodec](https://github.com/multiformats/multicodec) to indicate its version, making it fully self-describing.
48+
49+
## What is it?
50+
51+
A CID is a self-describing content-addressed identifier.
52+
It uses cryptographic hashes for content addressing and several
53+
[multiformats](https://github.com/multiformats/multiformats) for flexible self-description, namely:
54+
55+
1. [multihash](https://github.com/multiformats/multihash) for content-addressed hashing,
56+
2. [multicodec](https://github.com/multiformats/multicodec) to type that addressed content, and
57+
3. optionally, [multibase](https://github.com/multiformats/multibase) to encode the binary CID as a string.
58+
59+
The first two form a self-contained binary identifier; the third is added only when the CID is written as text.
60+
61+
Concretely, it's a *typed* content address: a tuple of `(content-type, content-address)`.
62+
63+
## How does it work?
64+
65+
Current version: CIDv1.
66+
67+
CIDv1 is a **binary** format composed of [unsigned varints](https://github.com/multiformats/unsigned-varint)
68+
prefixing a hash digest to form a self-describing "content address":
69+
70+
```text
71+
<cidv1> ::= <multicodec-cidv1><multicodec-content-type><multihash-content-address>
72+
73+
# example: a CIDv1 addressing the raw bytes "hello", in hex
74+
01 55 12 20 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
75+
# 01: cidv1 | 55: raw | 12 20: sha2-256, 32 bytes | 2cf2...: sha2-256 digest of "hello"
76+
```
77+
78+
Where
79+
80+
- `<multicodec-cidv1>` is a [multicodec](https://github.com/multiformats/multicodec) representing the version of CID, here for upgradability purposes.
81+
- `<multicodec-content-type>` is a [multicodec](https://github.com/multiformats/multicodec) code representing the content type or format of the data being addressed.
82+
- `<multihash-content-address>` is a [multihash](https://github.com/multiformats/multihash) value, which uses a registry of hash function abbreviations to prefix a cryptographic hash of the content being addressed, thus making it self-describing.
83+
84+
## Stringified Form
85+
86+
Since CIDs have many applications outside binary-only contexts, a CID may need to be base-encoded for different consumers or transports.
87+
In such applications, CIDs are expressed as a Unicode *string* with a [multibase](https://github.com/multiformats/multibase) prefix.
88+
The multibase prefix identifies the string encoding but is not part of the CID itself; the same binary CID can appear in different bases depending on context and needs such as string length and case-sensitivity.
89+
The full string form is:
90+
91+
```text
92+
<cidv1-str> ::= <multibase-prefix><multibase-encoding(<multicodec-cidv1><multicodec-content-type><multihash-content-address>)>
93+
```
94+
95+
Where
96+
97+
- `<multibase-prefix>` is a [multibase prefix](https://github.com/multiformats/multibase/blob/master/multibase.csv) (1 Unicode code point) that makes the string self-describing for conversion back to binary.
98+
99+
IPFS implementations SHOULD support at minimum `base58btc` (`z`), `base32` (`b`), `base16` (`f`), and `base36` (`k`, for ed25519 keys in [IPNS Records](https://specs.ipfs.tech/ipns/ipns-record/)).
100+
101+
## Design Considerations
102+
103+
The design of CIDs takes into account many difficult tradeoffs encountered while building [IPFS](https://ipfs.tech). Most of these come from the multiformats project.
104+
105+
- Compactness: CIDs are binary to keep them as compact as possible, since they're meant to be part of longer path identifiers or URIs.
106+
- Transport friendliness (or "copy-pastability"): CIDs are encoded with multibase to allow choosing the best base for transporting. For example, CIDs can be encoded into base58btc to yield shorter and easily-copy-pastable hashes.
107+
- Versatility: CIDs are meant to be able to represent values of any format with any cryptographic hash.
108+
- Avoid Lock-in: CIDs prevent lock-in to old, potentially-outdated decisions.
109+
- Upgradability: CIDs encode a version to ensure the CID format itself can evolve.
110+
111+
## Versions
112+
113+
### CIDv0
114+
115+
CIDv0 is a backwards-compatible version, where:
116+
- the `multibase` of the string representation is always `base58btc` and implicit (prefix `z` not present)
117+
- the `multicodec` is always `dag-pb` (`0x70`) and implicit (not written)
118+
- the `cid-version` is always `cidv0` (`0`) and implicit (not written)
119+
- the `multihash` is written as is but is always a full (length 32) `sha2-256` (`0x12`) hash.
120+
121+
```text
122+
cidv0 ::= <multihash-content-address>
123+
```
124+
125+
### CIDv1
126+
127+
See the section: [How does it work?](#how-does-it-work)
128+
129+
```text
130+
<cidv1> ::= <multicodec-cidv1><multicodec-content-type><multihash-content-address>
131+
```
132+
133+
## Decoding Algorithm
134+
135+
The binary fields of a CID are [unsigned varints](https://github.com/multiformats/unsigned-varint).
136+
Two rules hold for every CID:
137+
138+
- each varint MUST be minimally encoded, and a decoder MUST reject any overlong (non-minimal) varint;
139+
- a decoder MUST reject any bytes left over after the multihash.
140+
141+
A binary CID has one of two shapes, told apart by its leading bytes:
142+
143+
| Leading bytes | Shape |
144+
| --- | --- |
145+
| `0x01` | a **CIDv1**, where this first varint is the version field |
146+
| `0x12 0x20` | a bare 34-byte `sha2-256` multihash: a **CIDv0** |
147+
| anything else | not a CID; a decoder MUST reject it |
148+
149+
A CIDv0 is identified by the two-byte prefix `0x12 0x20`, not by the leading byte `0x12` alone: `0x12` is the `sha2-256` multihash code and `0x20` is its digest length, 32. A CIDv0 has no version field.
150+
151+
### Decoding a binary CID
152+
153+
To decode a binary CID `bytes`:
154+
155+
1. If `bytes` is exactly 34 bytes long and begins with `0x12 0x20`, it is a **CIDv0**, a bare `sha2-256` multihash (`cidv0 ::= <multihash-content-address>`):
156+
1. The 34 bytes are `0x12` (the `sha2-256` code), `0x20` (the digest length, 32), and a 32-byte digest.
157+
2. The content type is implicitly `dag-pb` (`0x70`) and is not encoded.
158+
2. Otherwise, read the leading varint of `bytes`.
159+
3. If the leading varint is `0x01`, it is a **CIDv1** (`<cidv1> ::= <multicodec-cidv1><multicodec-content-type><multihash-content-address>`):
160+
1. The `<multicodec-cidv1>` version field is the `0x01` just read.
161+
2. Read the next varint as the `<multicodec-content-type>`, which types the content.
162+
3. Read the [`<multihash-content-address>`](https://github.com/multiformats/multihash) that follows, structured as `<hash-code><digest-length><digest>`.
163+
4. The `<digest-length>` MUST consume the remaining bytes exactly; a decoder MUST reject a truncated digest or any trailing bytes.
164+
4. Otherwise, a decoder MUST reject `bytes`. No other leading value is a CID. In particular, reject a leading `0x12` that is not the `0x12 0x20` prefix of a 34-byte input, and reject `0x00`, `0x02` (reserved for the never-deployed CIDv2), and `0x03` (reserved for the never-deployed CIDv3).
165+
166+
### Decoding a CID string
167+
168+
To decode a CID string (ASCII or UTF-8):
169+
170+
1. Convert the string to binary `bytes`:
171+
- If it is 46 characters long and begins with `Qm`, it is a CIDv0; decode it as `base58btc`.
172+
- Otherwise, decode it by its [multibase](https://github.com/multiformats/multibase) prefix.
173+
2. Decode `bytes` as a binary CID (above) and return the result.
174+
175+
# Appendices
176+
177+
:::warning
178+
These sections provide additional context. This is not part of specification,
179+
and is provided here only for extra context.
180+
:::
181+
182+
<!-- TODO: review each implementation for spec conformance before listing here.
183+
A spec should not reference implementations that may not follow it.
184+
185+
## Implementations
186+
187+
- [go-cid](https://github.com/ipfs/go-cid)
188+
- [java-cid](https://github.com/ipld/java-cid)
189+
- [js-multiformats](https://github.com/multiformats/js-multiformats)
190+
- [rust-cid](https://github.com/multiformats/rust-cid)
191+
- [py-multiformats-cid](https://github.com/pinnaculum/py-multiformats-cid)
192+
- [elixir-cid](https://github.com/nocursor/ex-cid)
193+
- [dart_cid](https://github.com/dwyl/dart_cid)
194+
- [zig_cid](https://github.com/zen-eth/multiformats-zig)
195+
196+
-->
197+
198+
## FAQ
199+
200+
> **Q. I have questions on multicodec, multibase, multihash, or unsigned-varint.**
201+
202+
Please check their repositories: [multicodec](https://github.com/multiformats/multicodec), [multibase](https://github.com/multiformats/multibase), [multihash](https://github.com/multiformats/multihash), [unsigned-varint](https://github.com/multiformats/unsigned-varint).
203+
204+
> **Q. Why does CID exist?**
205+
206+
IPFS originally used base58btc-encoded multihashes, but the need to support multiple data formats via IPLD revealed limitations of bare multihashes as identifiers.
207+
CIDs were created to provide a self-describing, versioned, typed content address.
208+
The history of this format is documented at: https://github.com/ipfs/specs/issues/130
209+
210+
> **Q. Is the use of multicodec similar to file extensions?**
211+
212+
Yes. Like a file extension, the multicodec in a CID tells consumers how to interpret the bytes.
213+
And just like file extensions, most users will never change it, but you can swap the codec to change how the same bytes are parsed.
214+
215+
> **Q. What formats (multicodec codes) does CID support?**
216+
217+
CID can reference content of any type registered in the [multicodec table](https://github.com/multiformats/multicodec/blob/master/table.csv).
218+
In practice, IPFS primarily uses [`dag-pb`](https://web.archive.org/web/20260305020653/https://ipld.io/specs/codecs/dag-pb/spec/) (`0x70`), [`raw`](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw) (`0x55`), [`dag-cbor`](https://web.archive.org/web/20260305020653/https://ipld.io/specs/codecs/dag-cbor/spec/) (`0x71`), [`dag-json`](https://web.archive.org/web/20260305020653/https://ipld.io/specs/codecs/dag-json/spec/) (`0x0129`), and [`libp2p-key`](https://github.com/libp2p/specs/blob/4e2c796bc77a2639136b277224468b7c48b9fff1/RFC/0001-text-peerid-cid.md) (`0x72`).
219+
220+
> **Q. What is the process for updating CID specification (e.g., adding a new version)?**
221+
222+
CIDs are a well established standard.
223+
IPFS uses CIDs for content-addressing and IPNS.
224+
Changing such a core protocol requires careful review, including feedback from implementers and stakeholders across the ecosystem.
225+
226+
For this reason, changes to the CID specification MUST be submitted as an improvement proposal to [ipfs/specs](https://github.com/ipfs/specs/tree/main/IPIP) repository (PR with [IPIP document](https://github.com/ipfs/specs/blob/main/ipip-template.md)), and follow the IPIP process described there.
227+
228+
## Historical Design Decisions
229+
230+
You can read an [in-depth discussion on why this format was needed in IPFS](https://github.com/ipfs/specs/issues/130) and the [original CIDv1 proposal](https://github.com/multiformats/cid/blob/f638ca68390758f0d4c7f90ac843091d3973cd02/original-rfc.md).
231+
232+
## Human-Readable Form
233+
234+
This is design guidance for tools that present a human-readable CID inspector to a user, such as a debugger, a block explorer, or the diagnostic UI at https://cid.ipfs.tech.
235+
It is not a wire format: nothing produces or parses it, and it carries no information beyond what the CID already encodes.
236+
237+
When such a UI needs to show what a CID contains, it can expand the already self-describing CID into a labelled listing of its parts:
238+
239+
```text
240+
<hr-cid> ::= <hr-mbc> "-" <hr-cid-mc> "-" <hr-mc> "-" <hr-mh>
241+
```
242+
243+
Where each sub-component is the name of its code in the relevant multiformats registry:
244+
245+
- `<hr-mbc>` is the multibase code name (eg `b` -> `base32`)
246+
- `<hr-cid-mc>` is the CID version multicodec name (eg `0x01` -> `cidv1`)
247+
- `<hr-mc>` is the content-type multicodec name (eg `0x55` -> `raw`)
248+
- `<hr-mh>` is the multihash code name and digest length (eg `sha2-256-256`), then a final dash and the hex digest
249+
250+
For example, the CIDv1 for the raw bytes `hello`:
251+
252+
```text
253+
# example CID
254+
bafkreibm6jg3ux5qumhcn2b3flc3tyu6dmlb4xa7u5bf44yegnrjhc4yeq
255+
# corresponding human readable CID
256+
base32 - cidv1 - raw - sha2-256-256-2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
257+
```
258+
259+
These names come from the multiformats registries and are provisional labels for human eyes only; only the numeric codes are stable.
260+
A UI should show the codes next to the names, and no consumer should rely on a name staying the same.
261+
See: https://cid.ipfs.tech/#bafkreibm6jg3ux5qumhcn2b3flc3tyu6dmlb4xa7u5bf44yegnrjhc4yeq

‎src/unixfs.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ thanks:
6666
github: bumblefudge
6767

6868
tags: ['data-formats']
69-
order: 1
69+
order: 2
7070
---
7171

7272
# Node Types

0 commit comments

Comments
 (0)