Skip to content

fix(idempotency): keep the payload hash when completing records in the Redis layer - #8656

Open
thebadalov wants to merge 1 commit into
aws-powertools:developfrom
thebadalov:fix/idempotency-redis-payload-validation-retry
Open

thebadalov wants to merge 1 commit into
aws-powertools:developfrom
thebadalov:fix/idempotency-redis-payload-validation-retry

Conversation

@thebadalov

Copy link
Copy Markdown

Issue number: closes #8655

Summary

Changes

CachePersistenceLayer._update_record now writes validation_key_attr when payload validation is enabled, the same way _put_in_progress_record and DynamoDBPersistenceLayer._update_record already do. Before this, the COMPLETED record replaced the in-progress one without the payload hash.

Added test_idempotent_function_and_lambda_handler_redis_validation_same_payload, which checks that the completed record keeps the hash and that a retry with the same payload returns the stored response without running the handler again. It fails on the current develop and passes with the fix.

User experience

With payload_validation_jmespath set on the Redis/Valkey layer:

  • Before: a retry with the same payload raised IdempotencyValidationError: Payload does not match stored record for this event key.
  • After: the same-payload retry returns the stored response, and a retry with a different payload still raises IdempotencyValidationError.

Records written without payload validation are unchanged.

Testing

  • tests/functional/idempotency and tests/unit/idempotency: 203 passed
  • make format-check lint-check and mypy on the idempotency package: clean
  • Checked against a local Redis server: same payload returns the stored response, different payload raises, TTL is kept, no validation field is written when validation is off

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…e Redis layer

CachePersistenceLayer._update_record overwrote the record with a new SET
that only had data, status and expiration, dropping the validation hash
written by _put_in_progress_record. With payload_validation_jmespath set,
a retry with the same payload then raised IdempotencyValidationError
instead of returning the stored response.

Write the validation key in _update_record when payload validation is
enabled, as the DynamoDB layer already does.
@thebadalov
thebadalov requested a review from a team as a code owner October 24, 2026 14:18
@thebadalov
thebadalov requested a review from svozza October 24, 2026 14:18
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 24, 2026
@boring-cyborg boring-cyborg Bot added the tests label Oct 24, 2026
@boring-cyborg

boring-cyborg Bot commented Oct 24, 2026

Copy link
Copy Markdown

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@blacksoxx blacksoxx left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against develop; I did not run the suite here, so these are reading notes.

The asymmetry is exactly as described: _put_in_progress_record writes validation_key_attr when payload_validation_enabled, and the DynamoDB layer's _update_record appends #validation_key = :validation_key under the same condition, so the Redis _update_record was the one place that dropped the hash when the record moved to COMPLETED. With the four lines the completed record carries the hash again, which is what _get_record reads back into payload_hash for the next validation. Records written without payload validation stay as they are.

The functional test pins both halves, the stored hash and the second call not re-running the handler. Nothing blocking from my side.

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @thebadalov, confirmed against a real Valkey server: on develop the completed record loses the hash and a same-payload retry raises, with your change the retry returns the stored response and a different payload still raises. Records without validation are unchanged, and old records written before the fix keep behaving as they did until they expire, so nothing breaks on upgrade.

I'm approving now. This goes out in 3.40.0, so it will sit here for a while before I merge it. No rebase needed in the meantime, and you'll be credited in the release notes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

3 participants