Skip to content

Bug: EventDeserializer.asListOf(String.class) fails when SQS message bodies are plain text #2723

Open

Description

@hidingmyname

Expected Behaviour

extractDataFrom(sqsEvent).asListOf(String.class) should return the message bodies as-is (e.g. ["hello", "world"]), the same way extractDataFrom(sqsMessage).as(String.class) already returns a single body without JSON parsing. Deserialization of JSON bodies into POJOs via asListOf(MyType.class) should not change.

Current Behaviour

Location (aws-powertools/powertools-lambda-java @ default branch 6455f45):

Current code:

// asListOf, contentList branch
            } else {
                return contentList.stream().map(s ->
                {
                    try {
                        return s == null ? null : JsonConfig.get().getObjectMapper().reader().readValue(s, clazz);
                    } catch (IOException e) {
                        throw new EventDeserializationException(
                                "Cannot load the event as a list of " + clazz.getSimpleName(), e);
                    }

// as(), for comparison
                    if (content.getClass().equals(clazz)) {
                        // do not read json when returning String, just return the String
                        return (T) content;
                    }

Problem: A plain-text body such as hello is not a JSON document, so asListOf(String.class) throws EventDeserializationException: Cannot load the event as a list of String (caused by JsonParseException: Unrecognized token 'hello'). as() has an explicit shortcut for this case, so the single-message and list APIs currently disagree.

Code snippet

import static software.amazon.lambda.powertools.utilities.EventDeserializer.extractDataFrom;

SQSEvent.SQSMessage m1 = new SQSEvent.SQSMessage();
m1.setMessageId("msg-1");
m1.setBody("hello");
SQSEvent.SQSMessage m2 = new SQSEvent.SQSMessage();
m2.setMessageId("msg-2");
m2.setBody("world");
SQSEvent event = new SQSEvent();
event.setRecords(Arrays.asList(m1, m2));

String single = extractDataFrom(m1).as(String.class);                  // "hello"  (works)
List<String> bodies = extractDataFrom(event).asListOf(String.class);   // throws EventDeserializationException

Steps to Reproduce

Reproduction

  1. Check out main at d767ff5.
  2. Put the snippet above in a JUnit test in powertools-serialization.
  3. mvn -pl powertools-serialization -am test -Dtest=<TestClass> -Dsurefire.failIfNoSpecifiedTests=false

Failing output:

software.amazon.lambda.powertools.utilities.EventDeserializationException: Cannot load the event as a list of String
	at software.amazon.lambda.powertools.utilities.EventDeserializer$EventPart.lambda$asListOf$0(EventDeserializer.java:266)
	...
	at software.amazon.lambda.powertools.utilities.EventDeserializer$EventPart.asListOf(EventDeserializer.java:268)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'hello': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
	...
	at software.amazon.lambda.powertools.utilities.EventDeserializer$EventPart.lambda$asListOf$0(EventDeserializer.java:263)

The as(String.class) control case in the same test class passed.

Equivalent real-world trigger: aws sqs send-message --message-body 'hello' to a queue whose consumer calls extractDataFrom(event).asListOf(String.class) (I reproduced with the constructed event above, not on a deployed function).

Environment:

  • Commit: d767ff5 (main); current main 6455f45 differs only by docs dependency bumps and a powertools-batch MultiThreadMDC change, so the code paths above are unchanged.
  • Toolchain: OpenJDK 25.0.5, Apache Maven 3.9.16 (unit test, not deployed to Lambda)

Possible Solution

Already fixed in aws-powertools/powertools-lambda-typescript

PR: aws-powertools/powertools-lambda-typescript#3513 — "fix(parser): allow SQS envelopes to handle non-JSON strings" (merged 2025-01-23T14:05:29Z, merge commit 89f0006e9b5).

The TypeScript Parser's SQS envelope also forced JSON parsing of every record body; the fix stops doing that and hands the raw body string to the requested schema, so plain-text bodies work (JSON bodies are opted into explicitly). The relevant part is the SqsEnvelope.parse change in envelopes/sqs.ts; the rest of the PR reworks error reporting, the SNS-in-SQS envelope and tests.

How powertools-lambda-typescript fixed it:

--- a/packages/parser/src/envelopes/sqs.ts
+++ b/packages/parser/src/envelopes/sqs.ts
@@ -1,93 +1,62 @@
...
-  parse<T extends ZodSchema>(data: unknown, schema: T): z.infer<T>[] {
-    const parsedEnvelope = SqsSchema.parse(data);
-
-    return parsedEnvelope.Records.map((record) => {
-      return Envelope.parse(record.body, schema);
-    });
-  },
...
+    return parsedEnvelope.Records.map((record, recordIndex) => {
+      let parsedRecord: z.infer<typeof SqsRecordSchema>;
+      try {
+        parsedRecord = schema.parse(record.body);
+      } catch (error) {

Suggested fix

In the contentList branch of asListOf, return the bodies unchanged when clazz == String.class, mirroring the existing as() shortcut (and, like the TypeScript fix, handing raw bodies through instead of forcing JSON parsing); other target types keep going through Jackson. As with as(String.class) today, a body that is itself a JSON string literal is returned verbatim rather than unquoted:

diff --git a/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java b/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java
--- a/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java
+++ b/powertools-serialization/src/main/java/software/amazon/lambda/powertools/utilities/EventDeserializer.java
@@ -256,6 +256,9 @@ public class EventDeserializer {
                             "Cannot load the event as a list of " + clazz.getSimpleName() +
                                     ", consider using 'as' instead", e);
                 }
+            } else if (clazz == String.class) {
+                // do not read json when returning String, just return the Strings
+                return (List<T>) contentList;
             } else {
                 return contentList.stream().map(s ->
                 {

Verified on d767ff5 (OpenJDK 25.0.5, Maven 3.9.16): the test above fails without this patch and passes with it; no other test results change.

I can submit a PR for this.

Powertools for AWS Lambda (Java) version

2.11.0 (main at d767ff5; same code on current main 6455f45)

AWS Lambda function runtime

Java 25 (reproduced as a unit test on OpenJDK 25.0.5, Apache Maven 3.9.16 — not deployed to Lambda)

Debugging logs

See the failing output under Steps to Reproduce.

Related issues checked

Checked: #757 (original feature), #2614 (CRaC priming; does not touch asListOf) and #2163 (SNS→SQS batch feature) — none cover this.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions