DynamoDB and TTL Records

Why you might need a TTL

A TTL‘ed record (Time To Live) is basically a record that is configured to expire after a predetermined period of time (usually measured in seconds) after it was stored in a database table. There are a few reasons why a TTL might be a good fit for an application. Maybe you store some sort of state in DynamoDB and you want subsequent requests for the same object to be served immediately rather than trigger a full backend system. Or, maybe you have an application that retrieves information that is only valid for certain amount of time before it becomes irrelevant. What follows is sample code on how to configure DynamoDB with a TTL column and how this can be used from an application.

Deploy DynamoDB

First, we need to create a DynamoDB table with a TTL column.

DynamoDBSampleTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: myapp-data
    AttributeDefinitions:
      - AttributeName: ObjectId
        AttributeType: S
    KeySchema:
      - AttributeName: ObjectId
        KeyType: HASH
    ProvisionedThroughput:
      ReadCapacityUnits: 5
      WriteCapacityUnits: 5
    TimeToLiveSpecification:
      AttributeName: Ttl
      Enabled: true

The above CloudFormation will create a DynamoDB table where ObjectId is the hash key and Ttl is the column that will be configured to expire based on its value.

Application Sample Code

Sample code (in Python) to store data in DynamoDB with a TTL:

def store_to_ddb(boto_session, table_name, event):
    """ Store MyApp data from 'event' to DDB """
    try:
        ddb_client = boto_session.client("dynamodb")
        ttl = int(time.time() + 1800)
        ddb_response = ddb_client.put_item(
            TableName=table_name,
            Item={
                "ObjectId": {
                    "S": event["object_id"]
                },
                "Ttl": {
                    "N": str(ttl)
                },
                "Metadata": {
                    "M": event["metadata"]
                }
            }
        )
    except ClientError as e:
        error_msg = f"Couldn't store information to DDB: {e}"
        log.error(error_msg)
        my_alert_layer(error_msg)

Note that even though we store the TTL value as a number in the Ttl column, we still need to send it across to DynamoDB as a string.

Moving on, let’s see how we can retrieve the data back from DynamoDB.

def check_ddb(ddb_client, table_name, object_id):
    """ Check if we have the requested data cached in DDB """
    _result = {}
    timestamp = int(time.time())
    try:
        ddb_response = ddb_client.query(
            TableName=table_name,
            Select="ALL_ATTRIBUTES",
            KeyConditionExpression="ObjectId = :object_id",
            FilterExpression="#t > :timestamp",
            ExpressionAttributeNames={
                "#t": "Ttl"
            },
            ExpressionAttributeValues={
                ":object_id": {
                    "S": object_id
                },
                ":timestamp": {
                    "N": str(timestamp)
                }
            }
        )
        if "Items" in ddb_response:
            _result["result"] = ddb_response["Items"]
            return _result
        else:
            log.info(f"No data found in DDB for object: {object_id}")
            return None
    except ClientError as e:
        error_msg = f"Couldn't query DDB table {table_name}: {e}"
        _result["error"] = error_msg
        log.error(error_msg)
        my_alert_layer(error_msg)

Note again how we cast the timestamp to int, then to str so that DynamoDB can check the filter expression and only return records with TTLs that have not expired. DynamoDB will expire them automatically but, at the time of writing, the window that this is done depends on the workload and is typically done within 48 hours according to the documentation (from experience this is done much faster, but let’s go with the docs here).

Happy serverless development! :)