Query Dynamodb with a Hash Key and a Range Key with Boto3

Question

I am having trouble using AWS Boto3 to query DynamoDB with a hash key and a range key at the same time using the recommend KeyConditionExpression. I have attached an example query:

import boto3
from boto3 import dynamodb
from boto3.session import Session
dynamodb_session = Session(aws_access_key_id=AWS_KEY,
              aws_secret_access_key=AWS_PASS,
              region_name=DYNAMODB_REGION)
dynamodb = dynamodb_session.resource('dynamodb')
table=dynamodb.Table(TABLE_NAME)
request = {
    'ExpressionAttributeNames': {
        '#n0': 'hash_key',
        '#n1': 'range_key'
    },
    'ExpressionAttributeValues': {
        ':v0': {'S': MY_HASH_KEY},
        ':v1': {'N': GT_RANGE_KEY}
    },
    'KeyConditionExpression': '(#n0 = :v0) AND (#n1 > :v1)',
    'TableName': TABLE_NAME
}
response = table.query(**request)

When I run this against a table with the following scheme:

Table Name: TABLE_NAME
Primary Hash Key: hash_key (String)
Primary Range Key: range_key (Number)

I get the following error and I cannot understand why:

ClientError: An error occurred (ValidationException) when calling the Query operation: Invalid KeyConditionExpression: Incorrect operand type for operator or function; operator or function: >, operand type: M

From my understanding the type M would be a map or dictionary type and I am using a type N which is a number type and matches my table scheme for the range key. If someone could explain why this error is happening or I am also open to a different way of accomplishing the same query even if you cannot explain why this error exists.

Accepted Answer

The Boto 3 SDK constructs a Condition Expression for you when you use the Key and Attr functions imported from boto3.dynamodb.conditions:

response = table.query(
    KeyConditionExpression=Key('hash_key').eq(hash_value) & Key('range_key').eq(range_key_value)

)

Reference: Step 4: Query and Scan the Data

Hope it helps

Answer 2

Adding this solution as the accepted answer did not address why the query used did not work.

TLDR: Using query on a Table resource in boto3 has subtle differences as opposed to using client.query(...) and requires a different syntax.

The syntax is valid for a query on a client, but not on a Table. The ExpressionAttributeValues on a table do not require you to specify the data type. Also if you are executing a query on a Table resource you do not have to specify the TableName again.

Working solution:

from boto3.session import Session

dynamodb_session = Session(aws_access_key_id=AWS_KEY,aws_secret_access_key=AWS_PASS,region_name=DYNAMODB_REGION)

dynamodb = dynamodb_session.resource('dynamodb')
table = dynamodb.Table(TABLE_NAME)

request = {
    'ExpressionAttributeNames': {
        '#n0': 'hash_key',
        '#n1': 'range_key'
    },
    'ExpressionAttributeValues': {
        ':v0': MY_HASH_KEY,
        ':v1': GT_RANGE_KEY
    },
    'KeyConditionExpression': '(#n0 = :v0) AND (#n1 > :v1)',
}
response = table.query(**request)

I am the author of a package called botoful which might be useful to avoid dealing with these complexities. The code using botoful will be as follows:

import boto3
from botoful import Query

client = boto3.Session(
    aws_access_key_id=AWS_KEY,
    aws_secret_access_key=AWS_PASS,
    region_name=DYNAMODB_REGION
).client('dynamodb')

results = (
    Query(TABLE_NAME)
        .key(hash_key=MY_HASH_KEY, range_key__gt=GT_RANGE_KEY)
        .execute(client)
)

print(results.items)

Query Dynamodb with a Hash Key and a Range Key with Boto3

2 Answers

Must Read

Your Answer

David Miller

Query Dynamodb with a Hash Key and a Range Key with Boto3

2 Answers

Must Read

Your Answer

Sign up or log in

Post as a guest

David Miller