Aws Sigv4 Request Signature Does Not Match

I am getting Invalid Request with below error message when trying to create a thing-type using Rest API. I am not using aws-sdk rather python rest client.

I followed the steps given here to sign the request using sigv4. The example given there is working for listing the Thing-Types. So the credentials I am using, are correct. Error Message :

Response code: 403

{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}

The code I am using :

# See: 
# This version makes a POST request and passes request parameters
# in the body (payload) of the request. Auth information is passed in
# an Authorization header.
import sys, os, base64, datetime, hashlib, hmac
import requests  # pip install requests

# ************* REQUEST VALUES *************
method = 'POST'
service = 'execute-api'
api = '
request_parameter='{"thingTypeProperties":{"searchableAttributes":  ["gas"],"thingTypeDescription":"Devices that reports gas level readings"}}'
host = api.split('/')[2] if api.count('/') >= 2 else ''
region = api.split('.')[1] if api.count('.') >= 1 else ''
endpoint = api.split('?')[0] if api.count('?') >= 0 else api
#request_parameters = api.split('?')[1] if api.__contains__('?') else ''


# Request parameters for CreateTable--passed in a JSON block.
request_parameters = payload
print(host + ' $ ' + region + ' $ ' + endpoint + ' $ ' + request_parameters)
# Key derivation functions. See:
# 
def sign(key, msg):
    return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()

def getSignatureKey(key, date_stamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), date_stamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = 'XXXXXXXXXXXXX'
secret_key = 'XXXXXXXXXXXXXX'   
if access_key is None or secret_key is None:
    print
    'No access key is available.'
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
date_stamp = t.strftime('%Y%m%d')  # Date w/o time, used in credential scope

# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# 

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
canonical_uri = '/'

## Step 3: Create the canonical query string. In this example, request
# parameters are passed in the body of the request and the query string
# is blank.
canonical_querystring = ''

# Step 4: Create the canonical headers. Header names must be trimmed
# and lowercase, and sorted in code point order from low to high.
# Note that there is a trailing \n.
canonical_headers = 'content-type:' + content_type + '\n' + 'host:' + host + '\n' + 'x-amz-date:' + amz_date + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers include those that you want to be included in the
# hash of the request. "Host" and "x-amz-date" are always required.
# For DynamoDB, content-type and x-amz-target are also required.
signed_headers = 'content-type;host;x-amz-date'

# Step 6: Create payload hash. In this example, the payload (body of
# the request) contains the request parameters.
payload_hash = hashlib.sha256(request_parameters.encode('utf-8')).hexdigest()

# Step 7: Combine elements to create create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash

# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = date_stamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' + amz_date + '\n' + credential_scope + '\n' + hashlib.sha256(
    canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, date_stamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()

# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# Put the signature information in a header named Authorization.
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' + 'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# # Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'Content-Type': content_type,
           'X-Amz-Date': amz_date,
           'Authorization': authorization_header}

# ************* SEND THE REQUEST *************
print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')

print( 'Request URL = ' + endpoint)


r = requests.post(endpoint, data=request_parameters, headers=headers)

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)

1 Answer

For most use cases, I'd recommend leaning on a library for this. Since you're using the requests library, aws-requests-auth is an attractive choice.

If you really are interested in implementing the signing process yourself, this test suite may help you figure out what's going wrong.


The only thing I noticed looking at your code:

When I use aws-requests-auth, my call looks like this:

response = requests.post('
                        auth=auth,
                        data= json.dumps({'payload': 'data'}))

Just wanted to call out the json.dumps(), which stringifies my params object. Amazon's sample also uses a string for the request_parameters.

I'm not sure where payload comes from in your code, but make sure you're turning it into a string if it isn't already one.

Update

I looked at your code more closely, and you need to implement Step 2 under Task 1 - Create canonical URI. This URI is indeed "/" for the dynamo call, but yours should look something like this:

# Step 2: Create canonical URI--the part of the URI from domain to query
# string (use '/' if no path)
canonical_uri = '/thing-types/GasSensor'

Also, this request will succeed with or without a Content-Type header. So this:

content_type='application/json'

canonical_headers = 'content-type:' + content_type + '\n' + 'host:' + host + '\n' + 'x-amz-date:' + amz_date + '\n'

signed_headers = 'content-type;host;x-amz-date'

And this:

canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amz_date + '\n'

signed_headers = 'host;x-amz-date'

Both work for me.

2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Sarah Jenkins

Sarah Jenkins

Senior Technology Editor & AI Specialist

Sarah Jenkins is a veteran tech journalist with over 12 years of experience covering artificial intelligence, mobile innovations, and digital ethics. Her insights have appeared in leading technology publications worldwide.

Share this article
Twitter Facebook Pinterest