Repository navigation
Feature request: Opt-in human-readable (indented) JSON log output for local development #2732
Description
Use case
When running a function locally (sam local invoke, unit/integration tests in an IDE), logs from the Logback LambdaJsonEncoder are compact single-line JSON. That is exactly right for CloudWatch, but entries with nested structured arguments or MDC values are hard to scan in a terminal:
{"level":"INFO","message":"Test message","cold_start":true,"function_name":"my-func","service":"myService","timestamp":"2023-10-01T12:00:00.000Z"}
I'd like to see the same structure (same fields, same JSON) but indented, without changing the production logging configuration.
Location (aws-powertools/powertools-lambda-java @ default branch 6455f45):
LambdaJsonEncoder.java:71-101—encodewrites fields sequentially into a single line; there is no formatting option.
Current code (abridged):
public byte[] encode(ILoggingEvent event) {
StringBuilder builder = new StringBuilder();
try (JsonSerializer serializer = new JsonSerializer(builder)) {
serializer.writeStartObject();
serializer.writeStringField(LEVEL_ATTR_NAME, event.getLevel().toString());
serializer.writeRaw(',');
serializer.writeStringField(FORMATTED_MESSAGE_ATTR_NAME, event.getFormattedMessage());
...
serializer.writeEndObject();
serializer.writeRaw('\n');Problem: There is no way to get indented output, and POWERTOOLS_DEV is not read anywhere in the Java code base. Compact output is the correct default — this is not a report that it is wrong; the request is for an opt-in.
Reproduction: observed on main at d767ff5: with POWERTOOLS_DEV=true set, an encoded event is still one line. Current main (6455f45) differs only by docs dependency bumps and a powertools-batch MultiThreadMDC change, so the code paths above are unchanged.
Solution/User Experience
Already implemented in aws-powertools/powertools-lambda-python
PR: aws-powertools/powertools-lambda-python#1548 — "feat(logger): pretty-print JSON when POWERTOOLS_DEV is set" (merged 2022-10-03T09:05:02Z, merge commit a0ef408c64e).
The Python Logger serializes log JSON with an indent when POWERTOOLS_DEV is truthy and keeps compact output otherwise (still the case today: formatter.py#L156-L158).
How powertools-lambda-python implemented it:
--- a/aws_lambda_powertools/logging/formatter.py
+++ b/aws_lambda_powertools/logging/formatter.py
@@ -111,9 +112,15 @@ def __init__(
Key-value to be included in log messages
"""
+
self.json_deserializer = json_deserializer or json.loads
self.json_default = json_default or str
- self.json_serializer = json_serializer or partial(json.dumps, default=self.json_default, separators=(",", ":"))
+ self.json_indent = (
+ constants.PRETTY_INDENT if strtobool(os.getenv("POWERTOOLS_DEV", "0")) else constants.COMPACT_INDENT
+ ) # indented json serialization when in AWS SAM Local
+ self.json_serializer = json_serializer or partial(
+ json.dumps, default=self.json_default, separators=(",", ":"), indent=self.json_indent
+ )
self.datefmt = datefmt
self.use_datetime_directive = use_datetime_directive--- a/aws_lambda_powertools/shared/constants.py
+++ b/aws_lambda_powertools/shared/constants.py
@@ -32,3 +32,7 @@
"cold_start",
"xray_trace_id",
]
+
+# JSON indentation level
+PRETTY_INDENT: int = 4
+COMPACT_INDENT = NoneSuggested fix
This adds an opt-in prettyPrint option to LambdaJsonEncoder that defaults to true only when POWERTOOLS_DEV=true; it re-indents the compact JSON (whitespace only, outside string values, so fields, order and values are identical), and a docs bullet notes that CloudWatch Logs splits multi-line output into separate events:
diff --git a/docs/core/logging.md b/docs/core/logging.md
--- a/docs/core/logging.md
+++ b/docs/core/logging.md
@@ -1665,6 +1665,14 @@ If you need to customize format and timezone, you can change use the following:
</encoder>
```
+- For local development, you can indent the JSON output on multiple lines (default is `false`, or `true` when the `POWERTOOLS_DEV` environment variable is `true`). Do not enable it in Lambda: CloudWatch Logs splits multi-line output into separate log events.
+
+```xml
+ <encoder class="software.amazon.lambda.powertools.logging.logback.LambdaJsonEncoder">
+ <prettyPrint>true</prettyPrint>
+ </encoder>
+```
+
## Elastic Common Schema (ECS) Support
Utility also supports [Elastic Common Schema(ECS)](https://www.elastic.co/guide/en/ecs/current/ecs-reference.html){target="_blank"} format.
diff --git a/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java b/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java
--- a/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java
+++ b/powertools-logging/powertools-logging-logback/src/main/java/software/amazon/lambda/powertools/logging/logback/LambdaJsonEncoder.java
@@ -51,6 +51,7 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
protected String timestampFormatTimezoneId = null;
private boolean includeThreadInfo = false;
private boolean includePowertoolsInfo = true;
+ private boolean prettyPrint = "true".equals(System.getenv("POWERTOOLS_DEV"));
@Override
public byte[] headerBytes() {
@@ -93,6 +94,9 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
timestampFormat, timestampFormatTimezoneId, TIMESTAMP_ATTR_NAME);
serializer.writeEndObject();
+ if (prettyPrint) {
+ indent(builder);
+ }
serializer.writeRaw('\n');
} catch (IOException e) {
System.err.printf("Failed to encode log event, error: %s.%n", e.getMessage());
@@ -100,6 +104,64 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
return builder.toString().getBytes(UTF_8);
}
+ /**
+ * Indents the compact JSON contained in the builder. Only whitespace is added, outside of string values.
+ */
+ private static void indent(StringBuilder builder) {
+ String json = builder.toString();
+ builder.setLength(0);
+ int depth = 0;
+ boolean inString = false;
+ for (int i = 0; i < json.length(); i++) {
+ char c = json.charAt(i);
+ if (inString) {
+ builder.append(c);
+ if (c == '\\') {
+ builder.append(json.charAt(++i));
+ } else if (c == '"') {
+ inString = false;
+ }
+ continue;
+ }
+ switch (c) {
+ case '"':
+ inString = true;
+ builder.append(c);
+ break;
+ case '{':
+ case '[':
+ builder.append(c);
+ if (i + 1 < json.length() && (json.charAt(i + 1) == '}' || json.charAt(i + 1) == ']')) {
+ builder.append(json.charAt(++i));
+ } else {
+ newLine(builder, ++depth);
+ }
+ break;
+ case '}':
+ case ']':
+ newLine(builder, --depth);
+ builder.append(c);
+ break;
+ case ',':
+ builder.append(c);
+ newLine(builder, depth);
+ break;
+ case ':':
+ builder.append(": ");
+ break;
+ default:
+ builder.append(c);
+ }
+ }
+ }
+
+ private static void newLine(StringBuilder builder, int depth) {
+ builder.append('\n');
+ for (int i = 0; i < depth; i++) {
+ builder.append(" ");
+ }
+ }
+
private Map<String, String> getKeyValuePairs(ILoggingEvent event) {
return Optional.ofNullable(event.getKeyValuePairs())
.orElse(Collections.emptyList()).stream()
@@ -255,6 +317,23 @@ public class LambdaJsonEncoder extends EncoderBase<ILoggingEvent> {
this.includePowertoolsInfo = includePowertoolsInfo;
}
+ /**
+ * Specify if the JSON output should be indented on multiple lines, for local development
+ * (default is <b>true</b> if the POWERTOOLS_DEV environment variable is "true", <b>false</b> otherwise).
+ * Do not enable it in CloudWatch Logs, where each line is a separate log event.
+ * <br/>
+ * <pre>{@code
+ * <encoder class="software.amazon.lambda.powertools.logging.logback.LambdaJsonEncoder">
+ * <prettyPrint>true</prettyPrint>
+ * </encoder>
+ * }</pre>
+ *
+ * @param prettyPrint if the JSON output should be indented
+ */
+ public void setPrettyPrint(boolean prettyPrint) {
+ this.prettyPrint = prettyPrint;
+ }
+
private void serializeException(JsonSerializer serializer, String className, String message, String stackTrace) {
Map<Object, Object> map = new HashMap<>();
map.put(EXCEPTION_MSG_ATTR_NAME, message);Verified on d767ff5 (OpenJDK 25.0.5, Maven 3.9.16): a test encoding an event with POWERTOOLS_DEV=true set gets single-line output without this patch and indented multi-line output with it; tests that <prettyPrint>true</prettyPrint> produces the same JSON apart from whitespace and that prettyPrint=false stays single-line also pass; no other test results change.
I can submit a PR for this.
Alternative solutions
- Use a different Logback encoder/pattern locally: loses the Powertools structure, so local output no longer matches production.
- Pipe output through `jq`: works in a shell, less so in IDE consoles or `sam local` output mixed with runtime lines.Acknowledgment
- This feature request meets Powertools for AWS Lambda (Java) Tenets
- Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, TypeScript, and .NET
Acknowledgment
- This feature request meets Powertools for AWS Lambda (Java) Tenets
- Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, TypeScript, and .NET
Related issues checked
Checked: searches for POWERTOOLS_DEV, pretty print logs local and logging readable local development returned no existing issue or PR.
Activity