Index-Only Types

Some documents are nothing but a position: a like says which post, which hashtag and which identity, and nothing else. Stored as an ordinary document, a like pays for a serialized body, a row in the primary tree and a reference in every index, for a fact its index entries already hold. An index-only type stores no body and no row: its index entries are its documents. That cuts the storage of a small document by more than half, and makes each index a uniqueness rule. In exchange, its documents can only be created and deleted, every property must live in an index or in the entry's value, and a query returns documents rebuilt from index entries rather than fetched by $id.

Five keywords shape an index-only type: indexOnly and entryPayload on the document type, and terminal, preallocated and skipIfAbsent on its indexes. All of them arrived at protocol version 14 and are fixed once the type exists.

Example

A like of a social contract whose post type cannot be deleted:

"like": {
  "type": "object",
  "indexOnly": true,
  "documentsMutable": false,
  "canBeDeleted": true,
  "indices": [
    {
      "name": "byHashtagPost",
      "properties": [{ "hashtag": "asc" }, { "postId": "asc" }],
      "countable": "countable",
      "rangeCountable": true,
      "rankedCountable": true,
      "skipIfAbsent": true
    },
    {
      "name": "byPost",
      "properties": [{ "postId": "asc" }],
      "countable": "countable",
      "rangeCountable": true,
      "rankedCountable": true
    },
    { "name": "byLiker", "properties": [{ "$ownerId": "asc" }], "terminal": "postId" }
  ],
  "properties": {
    "hashtag": { "type": "string", "minLength": 1, "maxLength": 63, "position": 0 },
    "postId": {
      "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
      "contentMediaType": "application/x.dash.dpp.identifier",
      "refersTo": {
        "type": "permanentDocument",
        "documentType": "post",
        "propertyAgreement": { "hashtag": "hashtag" }
      },
      "position": 1
    }
  },
  "required": ["postId"],
  "additionalProperties": false
}

byPost holds one entry per post and liker (its terminal defaults to $ownerId), so an identity can like a post once, and it counts and ranks posts by likes. byHashtagPost ranks the posts under each hashtag, and only likes that carry a hashtag enter it. byLiker lists the posts one identity liked. The reference makes sure the post exists and that a like's hashtag is its post's.

indexOnly

true stores the type's documents only as index entries. Each entry sits under the index's values, is keyed by the terminal's values in place of the document id, and holds a 32-byte row commitment: a hash over all of the document's values that ties its entries in the different indexes together as one document.

Create. A create writes one entry into each index. If any of those entries already exists the create is refused with DuplicateUniqueIndexError (40105), so every index is a uniqueness rule over its properties and its terminal. refersTo and the other property checks run as on any document.

Delete. A document has no id to delete by. It is deleted with an indexOnlyDelete transition that carries all of its values (and $createdAt when the type requires it); every entry those values produce must exist and carry the matching row commitment, or the delete is refused with DocumentNotFoundError (40101). Deleting by id on an index-only type, or with indexOnlyDelete on an ordinary type, is refused with InvalidDocumentTransitionActionError (10404). The owner can only ever reach its own entries, since every index holds $ownerId. canBeDeleted: false forbids deletes as on any type.

No other action. A document cannot be replaced, transferred, sold or repriced.

Queries. A query goes through one index and returns documents rebuilt from its entries: the index's properties, the terminal, and $ownerId and $createdAt where the index holds them. A query through an index that holds only some of the properties yields only those. The rebuilt $id is a hash of the entry's position and addresses nothing, so there is no fetch by $id and no startAt cursor; a query pages by the terminal instead (postId > <last seen>, with a limit). A query that sets the terminal with an equality can put an in on the index's last property, with a limit of at least its number of values; a range on an index property in such a query is refused, because its pages could hold fewer rows than exist. List the equality-bound properties first in an index, and page by a range on its terminal instead. The proof that a create or delete took effect is the presence or absence of its entry in the proof index, an index that involves no $createdAt and does not skip.

Rules at registration:

• documentsMutable: false; transferable 0; tradeMode 0; no documentsKeepHistory or keeps*History; no transient properties.
• None of the document type aggregate keywords (documentsCountable and the others): the type has no primary tree. The index keywords of Counts, Sums and Averages and Ranked Indexes are allowed.
• At least one index. No index is unique or contested, and none sets nullSearchable: false.
• Every index holds $ownerId, as a property or in its terminal.
• The only system properties an index may list are $ownerId and $createdAt, and an indexed $createdAt must be in required.
• No index declares integerRange: rows that differ only in the bucketed integer would claim the same entry in the windows they share.
• At least one index involves no $createdAt and does not set skipIfAbsent: the proof index.
• Every property is in required, except a skip property of a skipIfAbsent index. An object holding an indexed property is required too.
• Every required property appears in at least one index that does not skip, as a property or a terminal component, except the entryPayload properties. Every optional property appears in a skip index without a timeRange whose skip set is that property alone.
• The type cannot also set ttl or moderatorAbilities.delete, and a refersTo lookup cannot target it.

entryPayload

The properties stored in each entry's value, after the row commitment, instead of in a key. They are for data the application reads but never queries by, such as a public key or a ciphertext: they need not be indexed, and they come back with every query result.

"indices": [{ "name": "byRequest", "terminal": ["appEphemeralPubKeyHash", "$ownerId"] }],
"entryPayload": ["walletEphemeralPubKey", "encryptedPayload"]

With a flat index keyed by a request hash and the responder, this is a key-value table: a query on the hash returns every responder with its public key and ciphertext.

Rules at registration:

• Only on an indexOnly type.
• Each entry names a top-level property that is required, a scalar (not an object or an array of values), and bounded: maxLength on a string, maxItems on a byte array.
• A payload property appears in no index, neither as a property nor in a terminal.
• The largest size each payload property can take, plus two bytes each, adds up to at most 5120 bytes. With several indexes, the payload is repeated in every entry.

terminal

Where an ordinary index keys each entry by the document id, an index-only index keys it by the terminal's values: the member key. There is one entry per index values and member key, so the terminal decides what the index makes unique. byPost above, with the default terminal, allows one like per post and owner; byLiker, with postId as terminal under $ownerId, holds the same pairs the other way round.

An array is a composite terminal whose values are joined in order. An index with no properties at all is a flat index, keyed by its terminal alone, as byRequest above is.

A query that fixes every property of the index can test one member key ("did I like this post") or walk the member keys in order, a page at a time.

Rules at registration:

• Only on an indexOnly type.
• Each component is $ownerId or a property of the type that could be indexed: not an object or an array of values, a string with maxLength of at most 63, a byte array with maxItems of at most 255. No other system property.
• Every component but the last has a fixed width: a byte array with minItems equal to maxItems, an identifier, an integer or a boolean. A string can only be last.
• The whole member key is at most 255 bytes. On a flat index, the level key, the component names each preceded by a zero byte, is at most 255 bytes as well.
• A component is not one of the index's properties, and not an optional property.
• A flat index takes no count, sum, ranking, timeRange, integerRange, skipIfAbsent or preallocated keyword.

preallocated

The first entry under a new set of values pays for every tree on its path; later ones pay for one entry. When the whole path is decided by a referenced document, preallocated: true creates the trees when that document is created, paid by its creator, so every entry costs the same from the first one on. Deleting the last entry keeps the trees, so a post with no likes still shows in the rankings with a count of zero.

In the example, byHashtagPost could be preallocated: postId is the reference and hashtag agrees with the post's. byLiker could not, since no post decides who likes it.

Rules at registration:

• Only on an indexOnly type.
• Every index property is either a property with a permanentDocument reference to a type of the same contract, or a key of that reference's propertyAgreement. A deletableDocument reference does not qualify, since the trees would outlive a deleted target. $ownerId may only be the terminal.
• The referenced property of each such agreement key holds at most 255 bytes, since creating a referenced document makes its value an index key (40126 when the contract is created or updated).
• Not with timeRange or integerRange.

A referenced document whose agreed value takes more bytes than the referring property can hold preallocates nothing for that index, since no entry could agree with it.

skipIfAbsent

The keyword is described in Indexes: a document that leaves out a property of the index's skip set writes nothing into the index, and its delete looks for nothing there. It is the one way a property of an index-only type can be optional: in the example, a like without a hashtag is not in byHashtagPost and pays nothing for it. A skip property may sit below the first position, which is what lets a windowed ranking skip it:

{
  "name": "byDayHashtagPost",
  "properties": [{ "$createdAt": "asc" }, { "hashtag": "asc" }, { "postId": "asc" }],
  "terminal": "$ownerId",
  "rangeCountable": true,
  "rankedCountable": { "at": ["hashtag", "postId"] },
  "timeRange": { "on": "$createdAt", "range": 86400, "step": 86400, "ttl": 604800 },
  "skipIfAbsent": true
}

An untagged like still enters the type's other indexes over the same day window, but writes nothing under hashtag in it.

What an index-only type adds to the rules of every type:

• An index path has no representation for a missing value, so every index holding an optional property skips on it: the skip set is every optional property of the index, and an array must name them all.
• Each optional property needs a skip index without a timeRange whose skip set is that property alone. A document carrying one optional property but missing another skips every index holding both, and its value would otherwise be written nowhere; a windowed index keeps it only until its windows drain, where document queries do not read it.
• An optional property is never a terminal.
• At least one index that involves no $createdAt does not skip: the proof index.

See also

• Index-Only Document Types for the entry layout, the row commitment, the full constraint list and the query surface.
• Indexes, Counts, Sums and Averages and Ranked Indexes for the index keywords an index-only type uses.
• References (refersTo) for permanentDocument references and propertyAgreement, which preallocated relies on.
• Mutability, Deletion and Creation, Transfers and Trading for the flags an index-only type must set.