Skip to main content

Data Management

Managing the data on your IOTA full node is critical to ensuring a healthy IOTA network. This topic provides a high-level description of data management on IOTA full nodes that you can use to optimize your full node configuration.

Basic IOTA Full node functionality​

The minimal version of an IOTA full node executes all of the transactions IOTA validators commit. IOTA full nodes also orchestrate the submitting of new transactions to the system:

Basic IOTA full node functionality

The preceding image shows how data flows through a full node:

  1. State sync protocol: An IOTA full node performs the following to achieve state synchronization:
    • Retrieves the information about the committed checkpoints via the p2p gossip-like protocol
    • Executes the transactions locally to verify that effects match the effects certified by the quorum of the validators
    • Updates the local state with the latest object values accordingly.
  2. RPCs: An IOTA full node exposes IOTA RPC endpoints for querying the latest state of the system, including both the latest state metadata (such as, iota_getProtocolConfig), and the latest state object data (iota_getObject).
  3. Transaction submission: Each IOTA full node orchestrates transaction submission to the quorum of the IOTA Validators, and, optionally if configured, locally executes the finalized transactions (called fast path execution), which circumvents the wait for checkpoint synchronization.

IOTA full node Data and RPC types​

An IOTA Full Node stores multiple categories of data in its permanent store.

info

The per-epoch IOTA store is beyond the scope of this topic. IOTA uses the per-epoch store (resets at the start of each epoch) internally for authority and consensus operations.

IOTA full nodes store the following types of data:

  1. Transactions with associated effects and events: IOTA uses a unique transaction digest to retrieve information about a transaction, including its effects and emitted events. IOTA full nodes don't require the historic transaction information for basic full node operations. To conserve disk space, you can enable pruning to remove this historical data.
  2. Checkpoints: IOTA groups committed transactions in checkpoints, and then uses those checkpoints to achieve state synchronization. Checkpoints keep transaction digests that contain additional integrity metadata. IOTA full nodes don't require data from checkpoints to execute and submit transactions.
  3. Objects: Transactions that mutate objects create new object versions. Each object has a unique pair of (objectId, version) used to identify the object. IOTA full nodes don't require historic object versions to execute and submit transactions.
  4. Indexing information: A full node default configuration is to post-process the committed transactions: it indexes the committed information to enable efficient aggregation and filtering queries. For example, the indexing can be useful for retrieving all the historic transactions of a given sender, or finding all the objects owned by an address.

IOTA full nodes support more than 40 RPC types that includes the following categories:

  • General metadata, such as iota_getProtocolConfig and iota_getChainIdentifier. These requests don't depend on additional indexing and don't require historic data to process.
  • Direct lookups, such as iota_getObject, iota_getEvents. These requests don't depend on additional indexing, but require historic data in some cases, such as iota_tryGetPastObject and iota_getTransactionBlock.
  • Accumulation and filtering queries,such as iotax_getOwnedObjects and iotax_getCoins. These requests depend on additional indexing, and require historic data in some cases, such as iotax_queryTransactionBlocks.
info

IOTA plans to migrate the RPC endpoints that require additional indexing away from IOTA full nodes in the near future. This plan decouples the storage that is backing transaction execution from the storage that is better suited for data indexing.