Skip to content

Feature request: Opt-in human-readable (indented) JSON log output for local development #2732

Open

Description

@hidingmyname

Use case

When running a function locally (sam local invoke, unit/integration tests in an IDE), logs from the Logback LambdaJsonEncoder are compact single-line JSON. That is exactly right for CloudWatch, but entries with nested structured arguments or MDC values are hard to scan in a terminal:

{"level":"INFO","message":"Test message","cold_start":true,"function_name":"my-func","service":"myService","timestamp":"2023-10-01T12:00:00.000Z"}

I'd like to see the same structure (same fields, same JSON) but indented, without changing the production logging configuration.

Location (aws-powertools/powertools-lambda-java @ default branch 6455f45):

Current code (abridged):

    public byte[] encode(ILoggingEvent event) {
        StringBuilder builder = new StringBuilder();
        try (JsonSerializer serializer = new JsonSerializer(builder)) {
            serializer.writeStartObject();
            serializer.writeStringField(LEVEL_ATTR_NAME, event.getLevel().toString());
            serializer.writeRaw(',');
            serializer.writeStringField(FORMATTED_MESSAGE_ATTR_NAME, event.getFormattedMessage());
            ...
            serializer.writeEndObject();
            serializer.writeRaw('\n');

Problem: There is no way to get indented output, and POWERTOOLS_DEV is not read anywhere in the Java code base. Compact output is the correct default — this is not a report that it is wrong; the request is for an opt-in.

Reproduction: observed on main at d767ff5: with POWERTOOLS_DEV=true set, an encoded event is still one line. Current main (6455f45) differs only by docs dependency bumps and a powertools-batch MultiThreadMDC change, so the code paths above are unchanged.

Solution/User Experience

Already implemented in aws-powertools/powertools-lambda-python

PR: aws-powertools/powertools-lambda-python#1548 — "feat(logger): pretty-print JSON when POWERTOOLS_DEV is set" (merged 2022-10-03T09:05:02Z, merge commit a0ef408c64e).

The Python Logger serializes log JSON with an indent when POWERTOOLS_DEV is truthy and keeps compact output otherwise (still the case today: formatter.py#L156-L158).

How powertools-lambda-python implemented it:

--- a/aws_lambda_powertools/logging/formatter.py
+++ b/aws_lambda_powertools/logging/formatter.py
@@ -111,9 +112,15 @@ def __init__(
             Key-value to be included in log messages
 
         """
+
         self.json_deserializer = json_deserializer or json.loads
         self.json_default = json_default or str
-        self.json_serializer = json_serializer or partial(json.dumps, default=self.json_default, separators=(",", ":"))
+        self.json_indent = (
+            constants.PRETTY_INDENT if strtobool(os.getenv("POWERTOOLS_DEV", "0")) else constants.COMPACT_INDENT
+        )  # indented json serialization when in AWS SAM Local
+        self.json_serializer = json_serializer or partial(
+            json.dumps, default=self.json_default, separators=(",", ":"), indent=self.json_indent
+        )
 
         self.datefmt = datefmt
         self.use_datetime_directive = use_datetime_directive
--- a/aws_lambda_powertools/shared/constants.py
+++ b/aws_lambda_powertools/shared/constants.py
@@ -32,3 +32,7 @@
     "cold_start",
     "xray_trace_id",
 ]
+
+# JSON indentation level
+PRETTY_INDENT: int = 4
+COMPACT_INDENT = None

Suggested fix

This adds an opt-in prettyPrint option to LambdaJsonEncoder that defaults to true only when POWERTOOLS_DEV=true; it re-indents the compact JSON (whitespace only, outside string values, so fields, order and values are identical), and a docs bullet notes that CloudWatch Logs splits multi-line output into separate events:

diff --git a/docs/core/logging.md b/docs/core/logging.md
--- a/docs/core/logging.md
+++ b/docs/core/logging.md
@@ -1665,6 +1665,14 @@ If you need to customize format and timezone, you can change use the following:
     </encoder>
 ```
 
+- For local development, you can indent the JSON output on multiple lines (default is `false`, or `true` when the `POWERTOOLS_DEV` environment variable is `true`). Do not enable it in Lambda: CloudWatch Logs splits multi-line output into separate log events.
+
+```xml
+    <encoder class="software.amazon.lambda.powertools.logging.logback.LambdaJsonEncoder">
+        <prettyPrint>true</prettyPrint>
+    </encoder>
+```
+
 ## Elastic Common Schema (ECS) Support
 
 Utility also supports [Elastic Common Schema(ECS)](https://www.elastic.co/guide/en/ecs/current/ecs-reference.html){target="_blank"} format.
diff --git a/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java b/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java
--- a/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java
+++ b/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java
@@ -51,6 +51,7 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
     protected String timestampFormatTimezoneId = null;
     private boolean includeThreadInfo = false;
     private boolean includePowertoolsInfo = true;
+    private boolean prettyPrint = "true".equals(System.getenv("POWERTOOLS_DEV"));
 
     @Override
     public byte[] headerBytes() {
@@ -93,6 +94,9 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
                     timestampFormat, timestampFormatTimezoneId, TIMESTAMP_ATTR_NAME);
 
             serializer.writeEndObject();
+            if (prettyPrint) {
+                indent(builder);
+            }
             serializer.writeRaw('\n');
         } catch (IOException e) {
             System.err.printf("Failed to encode log event, error: %s.%n", e.getMessage());
@@ -100,6 +104,64 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
         return builder.toString().getBytes(UTF_8);
     }
 
+    /**
+     * Indents the compact JSON contained in the builder. Only whitespace is added, outside of string values.
+     */
+    private static void indent(StringBuilder builder) {
+        String json = builder.toString();
+        builder.setLength(0);
+        int depth = 0;
+        boolean inString = false;
+        for (int i = 0; i < json.length(); i++) {
+            char c = json.charAt(i);
+            if (inString) {
+                builder.append(c);
+                if (c == '\\') {
+                    builder.append(json.charAt(++i));
+                } else if (c == '"') {
+                    inString = false;
+                }
+                continue;
+            }
+            switch (c) {
+                case '"':
+                    inString = true;
+                    builder.append(c);
+                    break;
+                case '{':
+                case '[':
+                    builder.append(c);
+                    if (i + 1 < json.length() && (json.charAt(i + 1) == '}' || json.charAt(i + 1) == ']')) {
+                        builder.append(json.charAt(++i));
+                    } else {
+                        newLine(builder, ++depth);
+                    }
+                    break;
+                case '}':
+                case ']':
+                    newLine(builder, --depth);
+                    builder.append(c);
+                    break;
+                case ',':
+                    builder.append(c);
+                    newLine(builder, depth);
+                    break;
+                case ':':
+                    builder.append(": ");
+                    break;
+                default:
+                    builder.append(c);
+            }
+        }
+    }
+
+    private static void newLine(StringBuilder builder, int depth) {
+        builder.append('\n');
+        for (int i = 0; i < depth; i++) {
+            builder.append("  ");
+        }
+    }
+
     private Map<String, String> getKeyValuePairs(ILoggingEvent event) {
         return Optional.ofNullable(event.getKeyValuePairs())
                 .orElse(Collections.emptyList()).stream()
@@ -255,6 +317,23 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
         this.includePowertoolsInfo = includePowertoolsInfo;
     }
 
+    /**
+     * Specify if the JSON output should be indented on multiple lines, for local development
+     * (default is <b>true</b> if the POWERTOOLS_DEV environment variable is "true", <b>false</b> otherwise).
+     * Do not enable it in CloudWatch Logs, where each line is a separate log event.
+     * <br/>
+     * <pre>{@code
+     *     <encoder class="software.amazon.lambda.powertools.logging.logback.LambdaJsonEncoder">
+     *         <prettyPrint>true</prettyPrint>
+     *     </encoder>
+     * }</pre>
+     *
+     * @param prettyPrint if the JSON output should be indented
+     */
+    public void setPrettyPrint(boolean prettyPrint) {
+        this.prettyPrint = prettyPrint;
+    }
+
     private void serializeException(JsonSerializer serializer, String className, String message, String stackTrace) {
         Map<Object, Object> map = new HashMap<>();
         map.put(EXCEPTION_MSG_ATTR_NAME, message);

Verified on d767ff5 (OpenJDK 25.0.5, Maven 3.9.16): a test encoding an event with POWERTOOLS_DEV=true set gets single-line output without this patch and indented multi-line output with it; tests that <prettyPrint>true</prettyPrint> produces the same JSON apart from whitespace and that prettyPrint=false stays single-line also pass; no other test results change.

I can submit a PR for this.

Alternative solutions

- Use a different Logback encoder/pattern locally: loses the Powertools structure, so local output no longer matches production.
- Pipe output through `jq`: works in a shell, less so in IDE consoles or `sam local` output mixed with runtime lines.

Acknowledgment

Acknowledgment

Related issues checked

Checked: searches for POWERTOOLS_DEV, pretty print logs local and logging readable local development returned no existing issue or PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions