For time-series analytics workloads, recent data typically demands faster access while historical data can be archived or optimized for storage. Easysearch's Index Lifecycle Management (ILM) automates these transitions without manual intervention.

Defining a Lifecycle Strategy

Consider this sample strategy configuration that progressively adjusts replica counts:

PUT _ilm/policy/metrics_retention_policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0m"
      },
      "warm": {
        "min_age": "5m",
        "actions": {
          "replica_count": {
            "number_of_replicas": 0
          }
        }
      },
      "cold": {
        "min_age": "15m",
        "actions": {
          "replica_count": {
            "number_of_replicas": 1
          }
        }
      }
    }
  }
}

This configuration supports additional operations including rollover, forcemerge, and snapshot retention. Consult the official documentation for comprehensive action types.

Adjusting Execution Frequency

Lifecycle tasks execute on a scheduled basis. For testing purposes, accelerate the interval to every 60 seconds:

PUT _cluster/settings
{
  "transient": {
    "ilm.background_task_interval": "60s"
  }
}

Mapping Templates to Policies

Associate index patterns with lifecycle policies through composable templates:

PUT _index_template/metrics_template
{
  "index_patterns": ["app_metrics-*"],
  "template": {
    "settings": {
      "index": {
        "lifecycle": {
          "name": "metrics_retention_policy",
          "rollover_alias": "metrics_write_alias"
        },
        "number_of_replicas": 1
      }
    }
  },
  "priority": 100
}

Initializing Managed Indices

Create the initial index with a write-enabled alias to begin lifecycle tracking:

PUT app_metrics-000001
{
  "aliases": {
    "metrics_write_alias": {
      "is_write_index": true
    }
  }
}

Phase Transition Timeline

Upon creation at 14:23:10, the index starts with one primary and one replica shard distributed across the cluster.

After five minutes (14:28:15), the warm phase triggers, eliminating replicas and retaining only the primary shard for resource optimization.

At the fifteen-minute mark (14:38:20), the cold phase initiates, restoring one replica for improved data durability in archival storage.

Modifying Active Policies

Updates require optimistic concurrency control. Retrieve the current if_seq_no and if_primary_term before applying modifications:

PUT _ilm/policy/metrics_retention_policy?if_seq_no=15230&if_primary_term=8
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0m"
      },
      "warm": {
        "min_age": "5m",
        "actions": {
          "replica_count": {
            "number_of_replicas": 0
          }
        }
      },
      "cold": {
        "min_age": "12m",
        "actions": {
          "replica_count": {
            "number_of_replicas": 2
          }
        }
      }
    }
  }
}

Important: Policy modifications affect only newly created indices. Existing indices continue following the previous policy definition.

Removing Policies

Delete obsolete policies using the dedicated endpoint:

DELETE _ilm/policy/metrics_retention_policy