Skip to content

Hierarchical Keyring: cold-cache stampede — N concurrent decrypts → N DynamoDB/KMS calls #1663

@yosefbs

Description

@yosefbs

Security issue notifications

If you discover a potential security issue in the AWS Encryption SDK we ask that you notify AWS Security via our vulnerability reporting page. Please do not create a public GitHub issue.

Problem:

The Node Hierarchical Keyring doesn't de-dupe concurrent branch-key lookups. If I fire a lot of decrypts for the same branch key at once against a cold cache, they all miss the cache together (it's only filled after the keystore call returns), so each one hits the keystore on its own.

So instead of one lookup I get N DynamoDB GetItem + N KMS Decrypt calls. Easy to repro: await Promise.all of ~3000 decrypts for the same key version, and you see ~3000 keystore calls instead of 1. Encrypt has the same problem since it shares the same code path.

Solution:

Add single-flight to getBranchKeyMaterials: on a miss, the first caller starts the keystore fetch and stores the in-flight promise (keyed by cache entry id); everyone else for the same key awaits that promise instead of starting their own. The entry is dropped once it settles, so the materials cache still owns caching and TTL, and a failed request isn't shared — the next call just retries.

Out of scope:

The legacy caching CMM has the same gap but it's a separate path, so I'm not touching it here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions