Repository navigation
Bug: EventDeserializer.asListOf(String.class) fails when SQS message bodies are plain text #2723
Description
Expected Behaviour
extractDataFrom(sqsEvent).asListOf(String.class) should return the message bodies as-is (e.g. ["hello", "world"]), the same way extractDataFrom(sqsMessage).as(String.class) already returns a single body without JSON parsing. Deserialization of JSON bodies into POJOs via asListOf(MyType.class) should not change.
Current Behaviour
Location (aws-powertools/powertools-lambda-java @ default branch 6455f45):
EventDeserializer.java:96-100— SQS bodies are collected into a list of strings.EventDeserializer.java:258-267—asListOfruns every element throughObjectMapper.readValue(s, clazz), including forString.class.EventDeserializer.java:200-203— theas()shortcut that skips JSON forString.
Current code:
// asListOf, contentList branch
} else {
return contentList.stream().map(s ->
{
try {
return s == null ? null : JsonConfig.get().getObjectMapper().reader().readValue(s, clazz);
} catch (IOException e) {
throw new EventDeserializationException(
"Cannot load the event as a list of " + clazz.getSimpleName(), e);
}
// as(), for comparison
if (content.getClass().equals(clazz)) {
// do not read json when returning String, just return the String
return (T) content;
}Problem: A plain-text body such as hello is not a JSON document, so asListOf(String.class) throws EventDeserializationException: Cannot load the event as a list of String (caused by JsonParseException: Unrecognized token 'hello'). as() has an explicit shortcut for this case, so the single-message and list APIs currently disagree.
Code snippet
import static software.amazon.lambda.powertools.utilities.EventDeserializer.extractDataFrom;
SQSEvent.SQSMessage m1 = new SQSEvent.SQSMessage();
m1.setMessageId("msg-1");
m1.setBody("hello");
SQSEvent.SQSMessage m2 = new SQSEvent.SQSMessage();
m2.setMessageId("msg-2");
m2.setBody("world");
SQSEvent event = new SQSEvent();
event.setRecords(Arrays.asList(m1, m2));
String single = extractDataFrom(m1).as(String.class); // "hello" (works)
List<String> bodies = extractDataFrom(event).asListOf(String.class); // throws EventDeserializationExceptionSteps to Reproduce
Reproduction
- Check out
mainatd767ff5. - Put the snippet above in a JUnit test in
powertools-serialization. mvn -pl powertools-serialization -am test -Dtest=<TestClass> -Dsurefire.failIfNoSpecifiedTests=false
Failing output:
software.amazon.lambda.powertools.utilities.EventDeserializationException: Cannot load the event as a list of String
at software.amazon.lambda.powertools.utilities.EventDeserializer$EventPart.lambda$asListOf$0(EventDeserializer.java:266)
...
at software.amazon.lambda.powertools.utilities.EventDeserializer$EventPart.asListOf(EventDeserializer.java:268)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'hello': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
...
at software.amazon.lambda.powertools.utilities.EventDeserializer$EventPart.lambda$asListOf$0(EventDeserializer.java:263)The as(String.class) control case in the same test class passed.
Equivalent real-world trigger: aws sqs send-message --message-body 'hello' to a queue whose consumer calls extractDataFrom(event).asListOf(String.class) (I reproduced with the constructed event above, not on a deployed function).
Environment:
- Commit:
d767ff5(main); currentmain6455f45differs only by docs dependency bumps and apowertools-batchMultiThreadMDCchange, so the code paths above are unchanged. - Toolchain: OpenJDK 25.0.5, Apache Maven 3.9.16 (unit test, not deployed to Lambda)
Possible Solution
Already fixed in aws-powertools/powertools-lambda-typescript
PR: aws-powertools/powertools-lambda-typescript#3513 — "fix(parser): allow SQS envelopes to handle non-JSON strings" (merged 2025-01-23T14:05:29Z, merge commit 89f0006e9b5).
The TypeScript Parser's SQS envelope also forced JSON parsing of every record body; the fix stops doing that and hands the raw body string to the requested schema, so plain-text bodies work (JSON bodies are opted into explicitly). The relevant part is the SqsEnvelope.parse change in envelopes/sqs.ts; the rest of the PR reworks error reporting, the SNS-in-SQS envelope and tests.
How powertools-lambda-typescript fixed it:
--- a/packages/parser/src/envelopes/sqs.ts
+++ b/packages/parser/src/envelopes/sqs.ts
@@ -1,93 +1,62 @@
...
- parse<T extends ZodSchema>(data: unknown, schema: T): z.infer<T>[] {
- const parsedEnvelope = SqsSchema.parse(data);
-
- return parsedEnvelope.Records.map((record) => {
- return Envelope.parse(record.body, schema);
- });
- },
...
+ return parsedEnvelope.Records.map((record, recordIndex) => {
+ let parsedRecord: z.infer<typeof SqsRecordSchema>;
+ try {
+ parsedRecord = schema.parse(record.body);
+ } catch (error) {Suggested fix
In the contentList branch of asListOf, return the bodies unchanged when clazz == String.class, mirroring the existing as() shortcut (and, like the TypeScript fix, handing raw bodies through instead of forcing JSON parsing); other target types keep going through Jackson. As with as(String.class) today, a body that is itself a JSON string literal is returned verbatim rather than unquoted:
diff --git a/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java b/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java
--- a/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java
+++ b/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java
@@ -256,6 +256,9 @@ public class EventDeserializer {
"Cannot load the event as a list of " + clazz.getSimpleName() +
", consider using 'as' instead", e);
}
+ } else if (clazz == String.class) {
+ // do not read json when returning String, just return the Strings
+ return (List<T>) contentList;
} else {
return contentList.stream().map(s ->
{Verified on d767ff5 (OpenJDK 25.0.5, Maven 3.9.16): the test above fails without this patch and passes with it; no other test results change.
I can submit a PR for this.
Powertools for AWS Lambda (Java) version
2.11.0 (main at d767ff5; same code on current main 6455f45)
AWS Lambda function runtime
Java 25 (reproduced as a unit test on OpenJDK 25.0.5, Apache Maven 3.9.16 — not deployed to Lambda)
Debugging logs
See the failing output under Steps to Reproduce.
Related issues checked
Checked: #757 (original feature), #2614 (CRaC priming; does not touch asListOf) and #2163 (SNS→SQS batch feature) — none cover this.
Activity