Repository navigation
Logger bug audit: 14 reproduced bugs, issues to be opened for 3.41.0 #8730
leandrodamascena
started this conversation in
Maintainers discussions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Same exercise as #8576 and #8603, now for the Logger. I read the whole package, wrote a reproduction for every suspect and ran all of them on
develop; the two High ones I also confirmed inside a real Lambda function (Python 3.13, 3.39.0 layer). Everything below reproduces. None has an open issue.I'm holding these here and will open one issue per item for 3.41.0, each marked
good first issueunless noted. Severity is my call: High loses logs or crashes in normal use, Medium gives wrong data or contradicts the docs, Low is an edge case.High
1.
SuppressFilterhides unrelated loggers whose name contains the service namelogging/filters.py:16Logger.__init__adds aSuppressFilter(service)to every root handler, and the filter doesself.logger not in record.name, a substring test. Any third-party logger whose name contains the service string is dropped from the root handlers, which in Lambda is the runtime's own handler. Real function,Logger(service="core"):logging.getLogger("botocore").warning(...)never reaches CloudWatch,urllib3does.service="s"would suppress almost everything.Fix: match the logger itself or its children:
record.name == self.logger or record.name.startswith(f"{self.logger}.").2. Log buffering silently drops records when there is no
_X_AMZN_TRACE_IDlogging/logger.py(_add_log_record_to_buffer,flush_buffer)Both methods return early when
get_tracer_id()isNone, so the record is neither buffered nor emitted. In Lambda the variable is not set during the init phase: alogger.debug(...)at module level withbuffer_configis lost even when an error later flushes the buffer. Confirmed in a real function. Outside Lambda (containers, local runs, tests) every buffered-level log disappears.Fix: when there is no tracer id, log immediately instead of buffering (or buffer under a fixed key and flush it on the first invocation). Document the init-phase behaviour.
3. Lowercase log level with Advanced Logging Controls crashes, custom int levels crash
logging/logger.py(_determine_log_level,_get_powertools_log_level)With
AWS_LAMBDA_LOG_LEVELset,Logger(level="debug")orPOWERTOOLS_LOG_LEVEL=inforaisesTypeError: '>' not supported between instances of 'int' and 'str', becauselogging.getLevelName("debug")returns the string"Level debug"and it is compared with an int. Without ALC the same input works.Logger(level=15)raisesValueError: Unknown level: 'LEVEL 15'for any non-standard int.Fix: upper-case before comparing, compare numeric levels (
logging.getLevelNamesMapping()/_nameToLevel), and keep int levels as ints.Medium
4. Buffered records lose their caller information
logging/buffer/functions.py(_create_buffer_record),buffer/handler.py,logger.py(_create_and_flush_log_record)stacklevelis ignored when buffering: the location is always taken from a fixedsys._getframe(3). Directinfo(stacklevel=3)givesbusiness:41, buffered giveswrapper:36.BufferingHandlergetlocation: handle:1027(logging internals) andnameof the Powertools logger, because the original record'spathname,lineno,funcNameandnameare discarded.stack_info=Trueis dropped at flush (makeRecordis called withoutsinfo).Fix: carry
pathname/lineno/funcName/name/sinfoin the buffer record (from the realLogRecordin the handler path, honouringstacklevelin the logger path) and pass them tomakeRecord.5. Buffered records replace a provided
exc_infowith the current exceptionlogging/buffer/functions.py:_create_buffer_recordif exc_info: exc_info = sys.exc_info()throws away a tuple the caller passed.logger.info("later", exc_info=saved)outside theexceptblock flushes with no exception at all. Fix: only resolveexc_info is Truetosys.exc_info(), keep tuples and exception instances.6. Buffered record timestamps mix the original seconds with flush-time milliseconds
logging/logger.py:_create_and_flush_log_recordOnly
record.createdis overwritten;record.msecskeeps the flush time. With the default format a log created at.167and flushed 0.7 s later prints,867.use_rfc3339=Trueis correct since #8631. Fix: setrecord.msecs(andrelativeCreated) from the stored timestamp.7.
setLevel()is undone on every warm invocation whensampling_rateis setlogging/logger.py:refresh_sample_rate_calculationThe refresh resets the level to
initial_log_level(the constructor value) before re-rolling the sample.logger.setLevel("WARNING")after construction holds for the cold start only; from the second invocation INFO logs come back. Fix: track the level set throughsetLevelas the new baseline.8. A child logger without an explicit level gets INFO instead of inheriting the parent's level
logging/logger.py:_init_loggerLogger(child=True)callsself.setLevel(log_level)withNone, which resolves to INFO, so a parent atlevel="DEBUG"and a child with no level drops the child's debug logs. The docs describe child loggers as inheriting the parent's configuration. Fix: leave the child atNOTSETwhen no level was given so stdlib inheritance applies.9.
taskNameleaks into log lines emitted inside asyncio tasks (Python 3.12+)logging/formatter.py:RESERVED_LOG_ATTRSPython 3.12 added
LogRecord.taskName. It is not inRESERVED_LOG_ATTRS, so every log inside a task carries"taskName": "Task-1". Fix: add it to the reserved list, or derive the list fromlogging.LogRecordattributes so the next stdlib addition does not leak either.10. Methods reached through
__getattr__bypass buffering and flush-on-errorlogging/logger.py:__getattr__logger.log(logging.DEBUG, ...),logger.fatal(...)andlogger.warn(...)go straight to the stdlib logger:log()emits immediately whiledebug()buffers,fatal()does not flush the buffer whilecritical()does. Fix: implementlog()on top of the same dispatch, aliasfatal/warn, or document that only the explicit methods take part in buffering.Low
11. String messages that are valid JSON scalars are coerced
logging/formatter.py:_extract_log_messagelogger.info("null")produces a line with nomessagekey at all,"true"becomestrue,"123"becomes123,"1e3"becomes1000.0. Decoding JSON objects is a documented feature; scalars are a surprise. Fix: only keep the decoded value when it is a dict or list.12.
copy_config_to_registered_loggersmutates the caller'sexcludeset and ignoreslog_level=0logging/utils.pyexclude.update([...])adds the source and package logger names to the set the customer passed in.level = log_level or source_logger.log_leveltreatsNOTSET(0) as "not provided". Fix: copy the set; test forNone.13.
LoggerBufferConfig.max_bytescounts characters, not byteslogging/buffer/cache.pySizes use
len(str(item)); fiveécount as 5 while the UTF-8 payload is 10 bytes. Fix: measurelen(str(item).encode())or rename the parameter and document it.14. Appending a reserved attribute with a non-string value breaks every later log line
logging/formatter.py:_extract_log_keyslogger.append_keys(lineno=5)is accepted, then everyformat()raisesValueError, which the stdlib swallows as--- Logging error ---on stderr: no more log lines until the key is removed. Fix: validate inappend_keysand raise there.Cleanup, not bugs
debug/info/warningrepeat the same 30-line buffering block three times; one_log(level, ...)dispatcher would remove the duplication and fix item 10 at the same time._check_minimum_buffer_log_levelkeeps its own{"DEBUG": 10, ...}map;logging.getLevelNamesMapping()exists._get_caller_filenamewalksf_back.f_back.f_backand_create_buffer_recordusessys._getframe(3); both break if a frame is added in between.LambdaContextModelis a hand-written class that should be a dataclass.json_deserializeris typed as returningstrinLoggerand both formatters.exception_notesis emitted as an extra key only when present; worth a line in the docs, people will not find it otherwise.All reactions