How to configure automatic data expiration (TTL) for time-series containers in GridDB using the Java API?

by

Summary

GridDB can automatically purge old rows from a time‑series container by setting a time‑to‑live (TTL) expiration policy. The TTL is defined when the container is created through the ContainerInfo object. It can also be updated later via the setExpirationInfo API, but the change only takes effect for rows written after the update. Using the Java client you:

  1. Declare the expiration period (e.g., 30 days).
  2. Attach it to the container’s ContainerInfo before createContainer.
  3. Optionally adjust it later with setExpirationInfo.

Root Cause

Missing TTL configuration → rows accumulate indefinitely → storage exhaustion and degraded query performance.

  • Default behavior: no expiration, all rows are retained forever.
  • Misconception: assuming GridDB will infer a TTL from a timestamp column. It does not; the policy must be explicitly set.

Why This Happens in Real Systems

  • IoT streams generate millions of points per day.
  • Teams often focus on ingestion logic and forget to add lifecycle management.
  • TTL is metadata, not part of the schema, so it is easy to overlook during container creation.

Real-World Impact

  • Storage cost blow‑up (cloud / on‑prem).
  • Backup windows grow, increasing RPO risk.
  • Query latency rises as scan ranges include stale data.
  • Operational incidents when disk fills and writes are rejected.

Example or Code (if necessary and relevant)

import com.toshiba.mwcloud.gs.*;
import java.sql.Timestamp;
import java.time.Duration;

public class GridDBTTLExample {
    public static void main(String[] args) throws GSException {
        // Connect to GridDB
        GridStore store = StoreFactory.getInstance().getStore(
            new Properties() {{
                setProperty("notificationAddress", "239.0.0.1");
                setProperty("notificationPort", "3199");
                setProperty("clusterName", "myCluster");
                setProperty("username", "admin");
                setProperty("password", "admin");
            }}
        );

        // ----- 1️⃣ Define expiration (30 days) -----
        // TTL is expressed in seconds
        long ttlSec = Duration.ofDays(30).getSeconds();

        // Create container info with TTL
        ContainerInfo info = new ContainerInfo()
            .setName("sensor_data")
            .setType(ContainerInfo.Type.TIME_SERIES)
            .setRowKey(false)                 // timestamp column is the row key
            .setColumnInfoList(Arrays.asList(
                new ColumnInfo("timestamp", GSColumnType.TIMESTAMP),
                new ColumnInfo("temperature", GSColumnType.DOUBLE),
                new ColumnInfo("humidity", GSColumnType.DOUBLE)
            ))
            .setExpirationInfo(new ExpirationInfo(ttlSec, ExpirationInfo.Type.TIME));

        // ----- 2️⃣ Create or get container -----
        TimeSeries container;
        if (!store.hasContainer("sensor_data")) {
            container = store.createContainer(info);
        } else {
            container = store.getContainer("sensor_data");
        }

        // ----- 3️⃣ Insert a row (TTL will be applied automatically) -----
        Row row = container.createRow();
        row.setTimestamp("timestamp", new Timestamp(System.currentTimeMillis()));
        row.setDouble("temperature", 22.5);
        row.setDouble("humidity", 60.0);
        container.put(row);

        // ----- 4️⃣ Update TTL later (optional) -----
        // New TTL = 60 days
        long newTtlSec = Duration.ofDays(60).getSeconds();
        container.setExpirationInfo(new ExpirationInfo(newTtlSec, ExpirationInfo.Type.TIME));
    }
}

How Senior Engineers Fix It

  • Plan TTL at design time – decide retention period before the first write.
  • Encapsulate container creation in a reusable factory that always applies the policy.
  • Add automated tests that verify old rows disappear after the TTL elapses (use a short TTL in CI).
  • Document the policy in runbooks and include a monitoring alert for container size growth.
  • When changing TTL, remember it only affects future rows; schedule a one‑time cleanup for existing data if needed.

Why Juniors Miss It

  • Focus on CRUD code and ignore metadata configuration.
  • Assume that a timestamp column automatically triggers expiration.
  • Unaware of the ExpirationInfo class and its setExpirationInfo / createContainer usage.
  • Lack of experience with long‑term data lifecycle management, leading to reactive (manual) clean‑ups instead of proactive TTL settings.

Leave a Comment