Skip to content
VERASPEC
Repository
VER 1.1 — draftdraft

§7 Size limits

1.0 §1 through 1.0 §14 contain no bound of any kind: no maximum Record size, no cap on inline base64, no maxItems anywhere. A 1.4 MB inline metadata segment validates; so does a 100 MB one. Consumers on the receiving end of a federated firehose need a number they can enforce without being accused of non-conformance.

The three quantities are distinct, and an earlier revision of this draft conflated two of them. It capped the serialized Record at 10 MiB and the sum of decoded inline raw segments at 16 MiB — a bound that cannot be reached, because 16 MiB of octets is ≈ 21.3 MiB of base64 text and cannot fit inside a 10 MiB document. A limit that nothing can violate is not a limit. The model below states each quantity in its own units and shows that each is individually reachable inside the one that contains it.

DRAFT limits — numbers require ratification.

#QuantityUnitsLimitLevel
(a)The serialized Record documentoctets of the JSON document as transmitted10 MiBMUST NOT exceed
(b)Sum of all metadata.raw[].bytes after base64 decodingoctets of decoded segment data6 MiBMUST NOT exceed
(c)Any single inline vector (vector, or vector_b64 after decoding)octets of decoded vector data6 MiBMUST NOT exceed
(d)spaces[], embeddings[]entries64 eachSHOULD NOT exceed
(e)provenance.chain[]entries1024SHOULD NOT exceed

Why 6 MiB. Base64 expands by 4/3, so 6 MiB of decoded octets is 8 MiB of text: individually reachable inside (a), with ~2 MiB of headroom for the rest of the document. (b) and (c) are each reachable inside (a); they are not jointly reachable, and that is intended. (a) is the binding constraint on the whole document; (b) and (c) are per-category ceilings that stop one category from consuming the entire budget. A Producer at 6 MiB of raw segments has no room for a 6 MiB inline vector and MUST carry one of them by reference.

Referenced external bytes are governed separately. A bytes_ref, vector_ref or source_bytes_ref payload is not counted against (a) or (b) or (c): what the Record carries is the reference, and the reference's byte_length states the size of what it names. A Consumer MAY enforce its own fetch ceiling and MUST document it; refusing to fetch an oversized referenced payload is a local policy decision and is not a conformance judgement about the Record.

A Producer whose payload would exceed (b) or (c) MUST use by-reference carriage — metadata.raw[].bytes_ref (§10.1) or embeddings[].vector_ref.

A Consumer MAY enforce a lower limit as local policy and MUST document it. A Record rejected under a Consumer's own lower limit is not thereby non-conformant; a Consumer MUST NOT report it as a conformance failure.

These limits are not schema-encoded, and cannot portably be. JSON Schema has no keyword for the byte length of the document being validated; maxLength on a base64 string bounds the encoded text, not the decoded octets, and mixing the two is exactly the incoherence above; and no keyword sums a quantity across array members. All three quantities are therefore prose plus profile, enforced together by VER104, whose defaults are aligned to this model so that the validator and the specification stop stating different numbers. The numbers themselves have no empirical corpus behind them; assembling one is a ratification prerequisite (RELEASE.md).