Manage the DynamoDB table with AWS CLI

Dang Nhu HieuAWS / Database Administrator

Introduction

You can use the AWS Console Manager to manage the DynamoDB table, alternatively, you can manage the DynamoDB table with AWS CLI in Linux as below.

Prerequisites

Before starting, you should have the following prerequisites configured

  • An AWS account
  • AWS CLI on your computer
  • Download and unzip the sample data, i will use sample data from the DynamoDB Developer Guide

Guide to creating and managing the DynamoDB table with AWS CLI

Topics

  • Create the DynamoDB table with AWS CLI
  • Load the data using the batch-write-item AWS CLI into DynamoDB table
  • DynamoDB table scan CLI command
  • DynamoDB table Query CLI command
  • DynamoDB table Query and Sort CLI command
  • DynamoDB table Scan and filter with CLI command
  • Insert data to DynamoDB table with CLI command
  • Update data to DynamoDB table with CLI command
  • Delete data of DynamoDB table with CLI command
  • Delete a value from a list type with the CLI command
  • DynamoDB transaction with CLI command
  • Using the Global Secondary Index
  • Delete Global Secondary Index with CLI command
  • Amazon DynamoDB point-in-time recovery (PITR) CLI command
  • Delete Amazon DynamoDB table CLI command

Create the DynamoDB table with AWS CLI

The following AWS CLI example creates DynamoDB tables with name ProductCatalog, Forum, Thread, Reply

aws dynamodb create-table \
--table-name ProductCatalog \
--attribute-definitions \
AttributeName=Id,AttributeType=N \
--key-schema \
AttributeName=Id,KeyType=HASH \
--provisioned-throughput \
ReadCapacityUnits=10,WriteCapacityUnits=5
aws dynamodb create-table \
--table-name Forum \
--attribute-definitions \
AttributeName=Name,AttributeType=S \
--key-schema \
AttributeName=Name,KeyType=HASH \
--provisioned-throughput \
ReadCapacityUnits=10,WriteCapacityUnits=5

aws dynamodb create-table \
--table-name Thread \
--attribute-definitions \
AttributeName=ForumName,AttributeType=S \
AttributeName=Subject,AttributeType=S \
--key-schema \
AttributeName=ForumName,KeyType=HASH \
AttributeName=Subject,KeyType=RANGE \
--provisioned-throughput \
ReadCapacityUnits=10,WriteCapacityUnits=5

aws dynamodb create-table \
--table-name Reply \
--attribute-definitions \
AttributeName=Id,AttributeType=S \
AttributeName=ReplyDateTime,AttributeType=S \
--key-schema \
AttributeName=Id,KeyType=HASH \
AttributeName=ReplyDateTime,KeyType=RANGE \
--provisioned-throughput \
ReadCapacityUnits=10,WriteCapacityUnits=5

This command returns the following result.

Manage the DynamoDB table with AWS CLI

To verify that DynamoDB has finished creating, use the command

aws dynamodb describe-table --table-name ProductCatalog | grep TableStatus

This command returns the following result. When DynamoDB finishes creating the table, the value of the TableStatus field is set to ACTIVE

Create the DynamoDB table with AWS CLI

Load the data using the batch-write-item AWS CLI into DynamoDB table

Load the sample data using the batch-write-item CLI

aws dynamodb batch-write-item --request-items file://ProductCatalog.json
aws dynamodb batch-write-item --request-items file://Forum.json
aws dynamodb batch-write-item --request-items file://Thread.json
aws dynamodb batch-write-item --request-items file://Reply.json

This command returns the following result. After each data load, you should get this message saying that there were no Unprocessed Items

Load the data using the batch-write-item AWS CLI into DynamoDB table

DynamoDB table scan CLI command

The scan will do a full table scan and return the items in 1MB chunks. Scanning is the slowest and most expensive way to get data out of DynamoDB. Try running a scan on the DynamoDB table

aws dynamodb scan --table-name Forum

This command returns the following result.

DynamoDB table scan CLI command

Read data from a DynamoDB table with the CLI command

GetItem is the fastest and cheapest way to get data out of DynamoDB as you must specify the full Primary Key so the command is guaranteed to match at most one item in the table.

The default behavior for DynamoDB is eventually consistent reads.

The following AWS CLI example reads an item from the ProductCatalog

aws dynamodb get-item --table-name ProductCatalog --key '{"Id":{"N":"101"}}'

There are many useful options for the get-item command, a few that get used regularly are:

  • –consistent-read : Specifying that you want a strongly consistent read
  • –projection-expression : Specifying that you only want certain attributes returned in the request
  • –return-consume-capacity : Tell us how much capacity was consumed by the request
aws dynamodb get-item \
    --table-name ProductCatalog \
    --key '{"Id":{"N":"101"}}' \
    --consistent-read \
    --projection-expression "ProductCategory, Price, Title" \
    --return-consumed-capacity TOTAL

This command returns the following result.

Read data from a DynamoDB table with CLI command

Performing this request consume 1.0 RCU, because this item is less than 4KB. If we run the command again but remove the –consistent-read option, this command returns the following result.

aws dynamodb get-item \
    --table-name ProductCatalog \
    --key '{"Id":{"N":"101"}}' \
    --projection-expression "ProductCategory, Price, Title" \
    --return-consumed-capacity TOTAL
Read data from a DynamoDB table with CLI command

You can do this either through the DynamoDB API or PartiQL, a SQL-compatible query language for DynamoDB.

PartiQL

aws dynamodb execute-statement --statement "SELECT ProductCategory, Price, Title FROM ProductCatalog WHERE Id=101" --return-consumed-capacity TOTAL
Read data from a DynamoDB table with PartiQL command

DynamoDB table Query CLI command

Just the Partition Key value query

aws dynamodb query \
    --table-name Reply \
    --key-condition-expression 'Id = :Id' \
    --expression-attribute-values '{
        ":Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 1"}
    }' \
    --return-consumed-capacity TOTAL

This command returns the following result.

DynamoDB table Query CLI command

Partition Key and Sort Key Query

aws dynamodb query \
--table-name Reply \
--key-condition-expression 'Id = :Id and ReplyDateTime > :ts' \
--expression-attribute-values '{
":Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 1"},
":ts" : {"S": "2015-09-21"}
}' \
--return-consumed-capacity TOTAL

PartiQL

aws dynamodb execute-statement --statement "SELECT * FROM Reply WHERE Id='Amazon DynamoDB#DynamoDB Thread 1' And ReplyDateTime > '2015-09-21'" --return-consumed-capacity TOTAL

With Id is Primary Key and ReplyDateTime is Sort Key

DynamoDB table Query CLI command

If you want to limit results based on non-key attributes, we can use Filter Expressions

aws dynamodb query \
    --table-name Reply \
    --key-condition-expression 'Id = :Id' \
    --filter-expression 'PostedBy = :user' \
    --expression-attribute-values '{
        ":Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 1"},
        ":user" : {"S": "User B"}
    }' \
    --return-consumed-capacity TOTAL

PartiQL

aws dynamodb execute-statement --statement "SELECT * FROM Reply WHERE Id='Amazon DynamoDB#DynamoDB Thread 1' And PostedBy = 'User B'" --return-consumed-capacity TOTAL

With PostedBy attributes at the filter-expression option, this command returns the following result.

DynamoDB table Query CLI command

DynamoDB table Query and Sort CLI command

  • –scan-index-forward : order items in ascending order of the sort key. This would be analogous in SQL to “ORDER BY ReplyDateTime ASC”
  • –no-scan-index-forward : order items in descending order of the sort key. This would be analogous in SQL to “ORDER BY ReplyDateTime DESC”
  • –max-items : limit the number of items
aws dynamodb query \
    --table-name Reply \
    --key-condition-expression 'Id = :Id' \
    --expression-attribute-values '{
        ":Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 1"}
    }' \
    --max-items 1 \
    --scan-index-forward  \
    --return-consumed-capacity TOTAL

PartiQL

aws dynamodb execute-statement --statement "SELECT * FROM Reply WHERE Id='Amazon DynamoDB#DynamoDB Thread 1' ORDER BY ReplyDateTime ASC" --limit 1  --return-consumed-capacity TOTAL

This command returns the following result.

DynamoDB table Query and Sort CLI command

Now let’s try without setting a limit on the number of items

#DynamoDB API
aws dynamodb query --table-name Reply --key-condition-expression 'Id = :Id' --expression-attribute-values '{":Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 1"}}' --no-scan-index-forward --output table

#PartiQL
aws dynamodb execute-statement --statement "SELECT * FROM Reply WHERE Id='Amazon DynamoDB#DynamoDB Thread 1' ORDER BY ReplyDateTime DESC" --output table

This is sorted in descending order by sort key ReplyDateTime.

DynamoDB table Query and Sort CLI command

DynamoDB table Scan and filter with CLI command

 Scan will do a full table scan, however, we can specify a Filter Expression which will reduce the size of the result set, but it will not reduce the amount of capacity consumed.

#DynamoDB API
aws dynamodb scan \
    --table-name Forum \
    --filter-expression 'Threads >= :threads AND #Views >= :views' \
    --expression-attribute-values '{
        ":threads" : {"N": "1"},
        ":views" : {"N": "50"}
    }' \
    --expression-attribute-names '{"#Views" : "Views"}' \
    --return-consumed-capacity TOTAL
#PartiQL
aws dynamodb execute-statement --statement "SELECT * FROM Forum WHERE Threads >= 1 And Views >= 50"

This command returns the following result.

image 50

Insert data to DynamoDB table with CLI command

Creates a new item, or replaces an old item with a new item. If an item that has the same primary key as the new item already exists in the specified table, the new item completely replaces the existing item.

Let’s insert a new item into the Reply table.

#DynamoDB API
aws dynamodb put-item \
    --table-name Reply \
    --item '{
        "Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 2"},
        "ReplyDateTime" : {"S": "2022-04-27T17:47:30Z"},
        "Message" : {"S": "DynamoDB Thread 2 Reply 3 text"},
        "PostedBy" : {"S": "User C"}
    }' \
    --return-consumed-capacity TOTAL

#PartiQL
aws dynamodb execute-statement --statement "INSERT INTO Reply value {'Id' : 'Amazon DynamoDB#DynamoDB Thread 2','ReplyDateTime' : '2023-04-27T17:47:30Z','Message' : 'DynamoDB Thread 2 Reply 4 text','PostedBy' : 'User C'}"

This command returns the following result.

Insert data to DynamoDB table with CLI command

Let’s insert an existing item with the put-item command

#DynamoDB API
aws dynamodb put-item \
    --table-name Reply \
    --item '{
        "Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 2"},
        "ReplyDateTime" : {"S": "2022-04-27T17:47:30Z"},
        "Message" : {"S": "DynamoDB Thread 2 Reply 5 text"},
        "PostedBy" : {"S": "User C"}
    }' \
    --return-consumed-capacity TOTAL

#PartiQL
aws dynamodb execute-statement --statement "INSERT INTO Reply value {'Id' : 'Amazon DynamoDB#DynamoDB Thread 2','ReplyDateTime' : '2023-04-27T17:47:30Z','Message' : 'DynamoDB Thread 2 Reply 6 text','PostedBy' : 'User C'}"

With put-item, if an item that has the same primary key as the new item already exists in the specified table, the new item completely replaces the existing item. But with PartiQL, if the table already has an item with the same primary key as the primary key of the item being inserted, DuplicateItemException is returned.

image 52

With put-item, to prevent a new item from replacing an existing item, use a conditional expression that contains the attribute_not_exists function with the name of the attribute being used as the partition key for the table. 

aws dynamodb put-item \
    --table-name Reply \
    --item '{
        "Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 2"},
        "ReplyDateTime" : {"S": "2022-04-27T17:47:30Z"},
        "Message" : {"S": "DynamoDB Thread 2 Reply 5 text"},
        "PostedBy" : {"S": "User C"}
    }' \
    --condition-expression "attribute_not_exists(Id) AND attribute_not_exists(ReplyDateTime)"
a example attribute_not_exists function with hash and range combine

Update data to DynamoDB table with CLI command

With the update-item command, we can edit an existing item’s attributes, or add a new item to the table if it does not already exist.

#DynamoDB API
aws dynamodb update-item \
    --table-name Forum \
    --key '{
        "Name" : {"S": "Amazon DynamoDB"}
    }' \
    --update-expression "SET Messages = :newMessages" \
    --condition-expression "Messages = :oldMessages" \
    --expression-attribute-values '{
        ":oldMessages" : {"N": "4"},
        ":newMessages" : {"N": "5"}
    }' \
    --return-consumed-capacity TOTAL

#PartiQL(update and show value before update)
aws dynamodb execute-statement --statement "Update Forum SET Messages = 6 WHERE Name='Amazon DynamoDB' RETURNING ALL OLD *" --return-consumed-capacity TOTAL

This command returns the following result.

Update data to DynamoDB table with CLI command

Let’s update the attribute of the type map with the CLI

#DynamoDB API
aws dynamodb update-item \
    --table-name ProductCatalog \
    --key '{
        "Id" : {"N": "201"}
    }' \
    --update-expression "SET #Color = list_append(#Color, :values)" \
    --expression-attribute-names '{"#Color": "Color"}' \
    --expression-attribute-values '{
        ":values" : {"L": [{"S" : "Blue"}, {"S" : "Yellow"}]}
    }' \
    --return-consumed-capacity TOTAL

#PartiQL
aws dynamodb execute-statement --statement "Update ProductCatalog SET Color = list_append(Color,['White']) WHERE Id = 201 RETURNING ALL NEW *" --return-consumed-capacity TOTAL

This command returns the following result.

Update DynamoDB table attribute of type map with CLI command

Delete data of DynamoDB table with CLI command

The delete-item the command is used command to delete an item from the DynamoDB table. Because it won’t report an error if the key doesn’t exist, we can use the delete-item command to confirm the existence of an item before deleting it.

#Confirm
aws dynamodb get-item \
    --table-name Reply \
    --key '{
        "Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 2"},
        "ReplyDateTime" : {"S": "2023-04-27T17:47:30Z"}
    }'
#Delete
aws dynamodb delete-item \
    --table-name Reply \
    --key '{
        "Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 2"},
        "ReplyDateTime" : {"S": "2023-04-27T17:47:30Z"}
    }' \
 --return-consumed-capacity TOTAL

This command returns the following result.

Delete data of DynamoDB table with CLI command

PartiQL delete statements for DynamoDB

If the DynamoDB table does not have any item with the same primary key as that of the item for which the DELETE is issued, SUCCESS is returned with 0 items deleted. If the table has an item with same primary key, but the condition in the WHERE clause of the DELETE statement evaluates to false, ConditionalCheckFailedException is returned.

#PartiQL delete
aws dynamodb get-item --table-name Reply --key '{
        "Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 2"},
        "ReplyDateTime" : {"S": "2022-04-27T17:47:30Z"}
    }'
aws dynamodb execute-statement --statement "DELETE FROM Reply WHERE Id = 'Amazon DynamoDB#DynamoDB Thread 2' AND ReplyDateTime = '2022-04-27T17:47:30Z'"
PartiQL delete statements for DynamoDB

Delete a value from a list type with the CLI command

Delete from a list type using update-item API

#Confirm
aws dynamodb get-item --table-name ProductCatalog --key '{ "Id" : {"N": "201"} }'

#Delete 'Yellow', 'White' from list
aws dynamodb update-item --table-name ProductCatalog --key '{ "Id" : {"N": "201"} }' \
    --update-expression "REMOVE #Color[3], #Color[4]" \
    --expression-attribute-names '{"#Color": "Color"}' \
    --return-consumed-capacity TOTAL
Delete item from list type use update-item API

Delete from a list type use PartiQL update statements

#Confirm
aws dynamodb get-item --table-name ProductCatalog --key '{ "Id" : {"N": "201"} }'

#Delete 'Blue' from Color list
aws dynamodb execute-statement --statement "UPDATE ProductCatalog REMOVE Color[2] WHERE Id=201" --return-consumed-capacity TOTAL

This command returns the following result.

image 59

DynamoDB transaction with CLI command

DynamoDB API TransactWriteItems is a synchronous write operation that groups up to 100 action requests. These actions can target items in different tables, but not in different Amazon Web Services accounts or Regions, and no two actions can target the same item. For example, you cannot both ConditionCheck and Update the same item. The aggregate size of the items in the transaction cannot exceed 4 MB.

The following example runs multiple statements as a transaction.

#DynamoDB API
aws dynamodb transact-write-items --client-request-token TRANSACTION1 --transact-items '[
    {
        "Put": {
            "TableName" : "Reply",
            "Item" : {
                "Id" : {"S": "Amazon DynamoDB#DynamoDB Thread 2"},
                "ReplyDateTime" : {"S": "2023-04-27T17:47:30Z"},
                "Message" : {"S": "DynamoDB Thread 2 Reply 3 text"},
                "PostedBy" : {"S": "User C"}
            }
        }
    },
    {
        "Update": {
            "TableName" : "Forum",
            "Key" : {"Name" : {"S": "Amazon DynamoDB"}},
            "UpdateExpression": "ADD Messages :inc",
            "ExpressionAttributeValues" : { ":inc": {"N" : "1"} }
        }
    }
]'

#PartiQL
aws dynamodb execute-transaction --transact-statements "[
    {
        \"Statement\": \"INSERT INTO Reply value {'Id':'Amazon DynamoDB#DynamoDB Thread 2','ReplyDateTime':'2023-08-26T17:47:30Z','Message':'DynamoDB Thread 2 Reply 4 text','PostedBy':'User C'}\"
    },
    {
        \"Statement\": \"UPDATE Forum SET Messages=8 where Name='Amazon DynamoDB'\"
    }
]"

#Confirm
aws dynamodb get-item --table-name Forum --key '{"Name" : {"S": "Amazon DynamoDB"}}'

PartiQL allows you to save your JSON code to a file and pass it to the –transact-statements parameter. Alternatively, you can enter it directly on the command line.

This command returns the following result.

image 60

Using the Global Secondary Index

To speed up queries on non-key attributes, you can create a global secondary index. A global secondary index contains a selection of attributes from the base table, but they are organized by a primary key that is different from that of the table. 

#Before create GSI
aws dynamodb scan \
    --table-name Reply \
    --filter-expression 'PostedBy = :user' \
    --expression-attribute-values '{
        ":user" : {"S": "User A"}
    }' \
    --return-consumed-capacity TOTAL

#create GSI
aws dynamodb update-table \
    --table-name Reply \
    --attribute-definitions AttributeName=PostedBy,AttributeType=S AttributeName=ReplyDateTime,AttributeType=S \
    --global-secondary-index-updates '[{
        "Create":{
            "IndexName": "PostedBy-ReplyDateTime-GSI",
            "KeySchema": [
                {
                    "AttributeName" : "PostedBy",
                    "KeyType": "HASH"
                },
                {
                    "AttributeName" : "ReplyDateTime",
                    "KeyType" : "RANGE"
                }
            ],
            "ProvisionedThroughput": {
                "ReadCapacityUnits": 2, "WriteCapacityUnits": 2
            },
            "Projection": {
                "ProjectionType": "ALL"
            }
        }
    }
]'

#GSI confirm
aws dynamodb query \
    --table-name Reply \
    --key-condition-expression 'PostedBy = :pb' \
    --expression-attribute-values '{
        ":pb" : {"S": "User A"}
    }' \
    --index-name PostedBy-ReplyDateTime-GSI \
    --return-consumed-capacity TOTAL

This command returns the following result.

image 61

Delete Global Secondary Index with CLI command

#GSI delete
aws dynamodb update-table \
    --table-name Reply \
    --global-secondary-index-updates '[{
        "Delete":{
            "IndexName": "PostedBy-ReplyDateTime-GSI"
        }
    }
]'

This command returns the following result.

image 62

Amazon DynamoDB point-in-time recovery (PITR) CLI command

The following procedure shows how to use the AWS CLI to restore an existing table named ProductCatalog to the LatestRestorableDateTime.

#Confirm that point-in-time recovery is enabled
aws dynamodb describe-continuous-backups --table-name ProductCatalog

#Enable
aws dynamodb update-continuous-backups --table-name ProductCatalog --point-in-time-recovery-specification PointInTimeRecoveryEnabled=True

#delete data
aws dynamodb delete-item --table-name ProductCatalog --key '{"Id" : {"N": "205"}}'

#Restore the table to the LatestRestorableDateTime
aws dynamodb restore-table-to-point-in-time \
    --source-table-name ProductCatalog \
    --target-table-name ProductCatalogMinutesAgo \
    --use-latest-restorable-time

To enable PITR for the ProductCatalog table, run the following command.

aws dynamodb update-continuous-backups --table-name ProductCatalog --point-in-time-recovery-specification PointInTimeRecoveryEnabled=True
image 63

Confirm that point-in-time recovery is enabled for the ProductCatalog table by using the describe-continuous-backups command.

aws dynamodb describe-continuous-backups --table-name ProductCatalog
image 64

delete data from ProductCatalog

aws dynamodb delete-item --table-name ProductCatalog --key '{"Id" : {"N": "205"}}'
image 65

Restore the table to a point in time. In this case, the ProductCatalog table is restored to the LatestRestorableDateTime (~5 minutes ago) to the same AWS Region.

aws dynamodb restore-table-to-point-in-time \
    --source-table-name ProductCatalog \
    --target-table-name ProductCatalogMinutesAgo \
    --use-latest-restorable-time
image 66

Check PITR of ProductCatalogMinutesAgo table

image 67

We must manually set the following on the restored table:

  • Auto scaling policies
  • AWS Identity and Access Management (IAM) policies
  • Amazon CloudWatch metrics and alarms
  • Tags
  • Stream settings
  • Time to Live (TTL) settings
  • Point-in-time recovery settings

Delete Amazon DynamoDB table CLI command

The delete-table operation deletes a table and all of its items, any indexes on that table are also deleted. If you have DynamoDB Streams enabled on the table, then the corresponding stream on that table goes into the DISABLED state and the stream is automatically deleted after 24 hours.

aws dynamodb delete-table --table-name Forum
image 68

We can use the describe-table action to check the status of the table.

image 69

Conclusion

These steps provide a general AWS CLI (DynamoDB API and PartiQL for DynamoDB) of the process to manage DynamoDB. The specific configuration details may vary depending on your environment and setup. It’s recommended to consult the relevant documentation from AWS for detailed instructions on setting up. I hope will this be helpful. Thank you for reading the DevopsRoles page!

Manage the DynamoDB table with AWS CLI

Refer

https://amazon-dynamodb-labs.workshop.aws/hands-on-labs.html

https://docs.aws.amazon.com/cli/latest/

Related Posts

DevopsRoles.com is written about terms DevOps Tutorials and DevOps Skills: Docker, git, Jenkins, Ansible, Bash Script, Vagrant, and DevOps-related.

DevOps Basics: What is DevOps? An Introduction to DevOps

11/05/2024
aws

Top DevOps Tools for AWS: From Basics to Advanced for 2024

11/04/2024
Oracle devopsroles.com

Oracle CRM in Docker: The Definitive Guide

10/25/2024

About Dang Nhu Hieu

I'm Vietnamese. In the past, I'm a software developer, now working in Japan on an Infra team. Skill : AWS, VMware, HA architech,setting Database : Oracle DB, PostgresDB ,.. Programming language : Java, C#, Python, Bash linux, Batch windows, powershell ,... Hobbies: badminton, film photo, travel. https://www.linkedin.com/in/hieu-dang-15a0561a6/
View all posts by Dang Nhu Hieu →

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.