Skip to content

Bug: is_missing_idempotency_key iterates dict keys instead of values, missing all-None-value payloads #8390

Description

@Adityaj0

Expected Behaviour

BasePersistenceLayer.is_missing_idempotency_key should correctly detect when the JMESPath-extracted idempotency payload is effectively empty (all values absent), regardless of whether event_key_jmespath resolves to a scalar, list/tuple, or dict.

Current Behaviour

For dict input, is_missing_idempotency_key iterates data directly:

@staticmethod
def is_missing_idempotency_key(data) -> bool:
    if isinstance(data, (tuple, list, dict)):
        return all(x is None for x in data)
    elif isinstance(data, (int, float, bool)):
        return False
    return not data

for x in data over a dict walks its keys, not its values. It works correctly for list/tuple, but silently checks the wrong thing for dict.

This matters because event_key_jmespath can be a multi-select expression that projects several event fields into a dict, e.g. '{user: headers.user_id, order: body.order_id}'. If the event lacks both fields, JMESPath still returns {"user": None, "order": None} — the keys are ordinary non-None strings, so the (wrong) key-based check evaluates False, i.e. "not missing," even though every value is None.

With raise_on_no_idempotency_key=True, the safety check meant to raise IdempotencyKeyError in this situation silently doesn't fire. With the default False, no warning is emitted either, and the persistence layer hashes {"user": None, "order": None} into a real idempotency key — so unrelated invocations that both fail to populate those fields collapse onto the same idempotency key and get incorrectly deduplicated against each other.

The existing unit test for this function only covers {None: None} (None as the key), which happens to still pass under the buggy key-iterating behavior (since it iterates over the single key, which is None) — that's why this went unnoticed.

Code snippet

from aws_lambda_powertools.utilities.idempotency.persistence.base import BasePersistenceLayer

BasePersistenceLayer.is_missing_idempotency_key({"user": None, "order": None})
# False  (expected: True)

End-to-end, with the safety check that should reject this:

from aws_lambda_powertools.utilities.idempotency.persistence.base import BasePersistenceLayer
from aws_lambda_powertools.utilities.idempotency.config import IdempotencyConfig

class DummyLayer(BasePersistenceLayer):
    def _put_record(self, *a, **kw): pass
    def _update_record(self, *a, **kw): pass
    def _delete_record(self, *a, **kw): pass
    def _get_record(self, *a, **kw): pass

layer = DummyLayer()
layer.configure(
    IdempotencyConfig(
        event_key_jmespath="{user: headers.user_id, order: body.order_id}",
        raise_on_no_idempotency_key=True,
    ),
    function_name="test",
)

layer._get_hashed_idempotency_key(data={"headers": {}, "body": {}})
# Returns 'test-func.test#b7d94249a37b613fc73b975e0d56d7c4' instead of raising IdempotencyKeyError

Possible Solution

Iterate data.values() for dict input instead of data itself:

if isinstance(data, dict):
    return all(x is None for x in data.values())
elif isinstance(data, (tuple, list)):
    return all(x is None for x in data)

I have a fix and a regression test ready and will open a PR shortly.

Steps to Reproduce

See code snippet above.

AWS Lambda Powertools for Python version

latest (reproduced on current develop)

AWS Lambda function runtime

python3.12

Packaging format used

PyPi

Debugging logs

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions