Expected Behaviour
BasePersistenceLayer.is_missing_idempotency_key should correctly detect when the JMESPath-extracted idempotency payload is effectively empty (all values absent), regardless of whether event_key_jmespath resolves to a scalar, list/tuple, or dict.
Current Behaviour
For dict input, is_missing_idempotency_key iterates data directly:
@staticmethod
def is_missing_idempotency_key(data) -> bool:
if isinstance(data, (tuple, list, dict)):
return all(x is None for x in data)
elif isinstance(data, (int, float, bool)):
return False
return not data
for x in data over a dict walks its keys, not its values. It works correctly for list/tuple, but silently checks the wrong thing for dict.
This matters because event_key_jmespath can be a multi-select expression that projects several event fields into a dict, e.g. '{user: headers.user_id, order: body.order_id}'. If the event lacks both fields, JMESPath still returns {"user": None, "order": None} — the keys are ordinary non-None strings, so the (wrong) key-based check evaluates False, i.e. "not missing," even though every value is None.
With raise_on_no_idempotency_key=True, the safety check meant to raise IdempotencyKeyError in this situation silently doesn't fire. With the default False, no warning is emitted either, and the persistence layer hashes {"user": None, "order": None} into a real idempotency key — so unrelated invocations that both fail to populate those fields collapse onto the same idempotency key and get incorrectly deduplicated against each other.
The existing unit test for this function only covers {None: None} (None as the key), which happens to still pass under the buggy key-iterating behavior (since it iterates over the single key, which is None) — that's why this went unnoticed.
Code snippet
from aws_lambda_powertools.utilities.idempotency.persistence.base import BasePersistenceLayer
BasePersistenceLayer.is_missing_idempotency_key({"user": None, "order": None})
# False (expected: True)
End-to-end, with the safety check that should reject this:
from aws_lambda_powertools.utilities.idempotency.persistence.base import BasePersistenceLayer
from aws_lambda_powertools.utilities.idempotency.config import IdempotencyConfig
class DummyLayer(BasePersistenceLayer):
def _put_record(self, *a, **kw): pass
def _update_record(self, *a, **kw): pass
def _delete_record(self, *a, **kw): pass
def _get_record(self, *a, **kw): pass
layer = DummyLayer()
layer.configure(
IdempotencyConfig(
event_key_jmespath="{user: headers.user_id, order: body.order_id}",
raise_on_no_idempotency_key=True,
),
function_name="test",
)
layer._get_hashed_idempotency_key(data={"headers": {}, "body": {}})
# Returns 'test-func.test#b7d94249a37b613fc73b975e0d56d7c4' instead of raising IdempotencyKeyError
Possible Solution
Iterate data.values() for dict input instead of data itself:
if isinstance(data, dict):
return all(x is None for x in data.values())
elif isinstance(data, (tuple, list)):
return all(x is None for x in data)
I have a fix and a regression test ready and will open a PR shortly.
Steps to Reproduce
See code snippet above.
AWS Lambda Powertools for Python version
latest (reproduced on current develop)
AWS Lambda function runtime
python3.12
Packaging format used
PyPi
Debugging logs
No response
Expected Behaviour
BasePersistenceLayer.is_missing_idempotency_keyshould correctly detect when the JMESPath-extracted idempotency payload is effectively empty (all values absent), regardless of whetherevent_key_jmespathresolves to a scalar, list/tuple, or dict.Current Behaviour
For
dictinput,is_missing_idempotency_keyiteratesdatadirectly:for x in dataover adictwalks its keys, not its values. It works correctly forlist/tuple, but silently checks the wrong thing fordict.This matters because
event_key_jmespathcan be a multi-select expression that projects several event fields into a dict, e.g.'{user: headers.user_id, order: body.order_id}'. If the event lacks both fields, JMESPath still returns{"user": None, "order": None}— the keys are ordinary non-Nonestrings, so the (wrong) key-based check evaluatesFalse, i.e. "not missing," even though every value isNone.With
raise_on_no_idempotency_key=True, the safety check meant to raiseIdempotencyKeyErrorin this situation silently doesn't fire. With the defaultFalse, no warning is emitted either, and the persistence layer hashes{"user": None, "order": None}into a real idempotency key — so unrelated invocations that both fail to populate those fields collapse onto the same idempotency key and get incorrectly deduplicated against each other.The existing unit test for this function only covers
{None: None}(Noneas the key), which happens to still pass under the buggy key-iterating behavior (since it iterates over the single key, which isNone) — that's why this went unnoticed.Code snippet
End-to-end, with the safety check that should reject this:
Possible Solution
Iterate
data.values()fordictinput instead ofdataitself:I have a fix and a regression test ready and will open a PR shortly.
Steps to Reproduce
See code snippet above.
AWS Lambda Powertools for Python version
latest (reproduced on current
develop)AWS Lambda function runtime
python3.12
Packaging format used
PyPi
Debugging logs
No response