Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds the ability to query Dynamo DB and return rows querying by their primary keys. It adds the query implementation of the
<table name>_by_keys
function that the schema declares (added in #2).This means you can now perform the following GraphQL queries from v3-engine:
This PR uses some hacks (marked with TODOs) to work around issues in the TypeScript SDK. These issues have since been resolved, but I'd rather perform the SDK upgrade in a subsequent PR.
Solution Design
Enhanced schema information
The query execution pipeline needed a bit more information about the function definition, so the
ByKeysFunction
type inschema-ndc.ts
was enhanced with extra information about the primary key schema and the table row type. This information is already computed during schema generation, and capturing it on the function definition ensures we don't need to recompute it at query time.Query
The new query code lives in
query.ts
. The most important function isperformByKeysFunction
. This function first builds a projection expression from the query (queryFieldsToProjectionExpression
), ensuring we only pull back attributes we are interested in, and then creates a BatchGetItemCommand to query for the rows by key. This is complicated by the need to alias attribute names (seeAttributeNameAliaser
) to avoid projection expression syntax errors caused by weird attribute names that conflict with the expression syntax.We may need to issue multiple
BatchGetItemCommand
queries because Dynamo only accepts 100 keys at a time, and it may not return all the rows for keys you requested (UnprocessedKeys
), requiring you to try again. We maintain a queue of keys (unprocessedKeys
) we haven't got results for and then we repeatedly query dynamo, draining the queue as we receive rows and putting anyunprocessedKeysFromThisResponse
back onto the queue for retry.As we receive rows
mkNdcResponseRowFromDynamoResponseRow
maps the Dynamo response row format into our desired NDC row response format.Other Changes
noUncheckedIndexedAccess
has been turned on in the TypeScript compiler to tighten up map lookups from objects. Various places that needed explicit error handling if the expected key wasn't in the object map have been updated.