[core] support DictConfigurator prefixes for rename_fields and static_fields (#45)

Python logging config supports access to
[external](https://docs.python.org/3/library/logging.config.html#access-to-external-objects)
and
[internal](https://docs.python.org/3/library/logging.config.html#access-to-internal-objects)
objects via
[DictConfiguator](https://docs.python.org/3/library/logging.config.html#configuration-functions)
prefixes.

This works internally by warpping the dict with a
[ConvertingDict](https://github.com/python/cpython/blob/main/Lib/logging/config.py#L337)
that requires an explicit call to it's
[\_\_getitem\_\_](https://github.com/python/cpython/blob/main/Lib/logging/config.py#L340)
dunder method for the
[conversion](https://github.com/python/cpython/blob/main/Lib/logging/config.py#L307)
to take place (using the specific converter for the specified prefix).

This allows, for example, having a *log_config.yaml* file like:

```yaml
version: 1
disable_existing_loggers: False
formatters:
  default:
    "()": pythonjsonlogger.json.JsonFormatter
    format: "%(asctime)s %(levelname)s %(name)s %(module)s %(funcName)s %(lineno)s %(message)s"
    rename_fields:
      "asctime": "timestamp"
      "levelname": "status"
    static_fields:
      "service": ext://logging_config.PROJECT_NAME
      "env": ext://logging_config.ENVIRONMENT
      "version": ext://logging_config.PROJECT_VERSION
      "app_log": "true"
handlers:
  default:
    formatter: default
    class: logging.StreamHandler
    stream: ext://sys.stderr
  access:
    formatter: default
    class: logging.StreamHandler
    stream: ext://sys.stdout
loggers:
  uvicorn.error:
    level: INFO
    handlers:
      - default
    propagate: no
  uvicorn.access:
    level: INFO
    handlers:
      - access
    propagate: no
```

where *service*, *env* and *version* values are taken from the external
resource *logging_config.py*.

The content for *logging_config.py* could be something like (for getting
some values from project metadata or environment variables):

```python
import importlib.metadata
import os


def get_version_metadata():
    # https://stackoverflow.com/a/78082532
    version = importlib.metadata.version(PROJECT_NAME)
    return version


PROJECT_NAME = 'test-api'
PROJECT_VERSION = get_version_metadata()
ENVIRONMENT = os.environ.get('ENVIRONMENT', 'dev')
```

---------

Co-authored-by: Nicholas Hairs <info@nicholashairs.com>

Author: rubensa

docs/changelog.md

@@ -4,6 +4,14 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [UNRELEASED]
+
+### Added
+- Support `DictConfigurator` prefixes for `rename_fields` and `static_fields`. [#45](https://github.com/nhairs/python-json-logger/pull/45)
+  - Allows using values like `ext://sys.stderr` in `fileConfig`/`dictConfig` value fields.
+
+Thanks @rubensa
+
 ## [3.3.0](https://github.com/nhairs/python-json-logger/compare/v3.2.1...v3.3.0) - 2025-03-06
 
 ### Added

docs/cookbook.md

@@ -139,35 +139,61 @@ main_3()
 
 ## Using `fileConfig`
 
-To use the module with a config file using the [`fileConfig` function](https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig), use the class `pythonjsonlogger.json.JsonFormatter`. Here is a sample config file.
-
-```ini
-[loggers]
-keys = root,custom
+To use the module with a yaml config file using the [`fileConfig` function](https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig), use the class `pythonjsonlogger.json.JsonFormatter`. Here is a sample config file:
+
+```yaml title="example_config.yaml"
+version: 1
+disable_existing_loggers: False
+formatters:
+  default:
+    "()": pythonjsonlogger.json.JsonFormatter
+    format: "%(asctime)s %(levelname)s %(name)s %(module)s %(funcName)s %(lineno)s %(message)s"
+    rename_fields:
+      "asctime": "timestamp"
+      "levelname": "status"
+    static_fields:
+      "service": ext://logging_config.PROJECT_NAME
+      "env": ext://logging_config.ENVIRONMENT
+      "version": ext://logging_config.PROJECT_VERSION
+      "app_log": "true"
+handlers:
+  default:
+    formatter: default
+    class: logging.StreamHandler
+    stream: ext://sys.stderr
+  access:
+    formatter: default
+    class: logging.StreamHandler
+    stream: ext://sys.stdout
+loggers:
+  uvicorn.error:
+    level: INFO
+    handlers:
+      - default
+    propagate: no
+  uvicorn.access:
+    level: INFO
+    handlers:
+      - access
+    propagate: no
+```
 
-[logger_root]
-handlers =
+You'll notice that we are using `ext://...` for the `static_fields`. This will load data from other modules such as the one below.
 
-[logger_custom]
-level = INFO
-handlers = custom
-qualname = custom
+```python title="logging_config.py"
+import importlib.metadata
+import os
 
-[handlers]
-keys = custom
 
-[handler_custom]
-class = StreamHandler
-level = INFO
-formatter = json
-args = (sys.stdout,)
+def get_version_metadata():
+    # https://stackoverflow.com/a/78082532
+    version = importlib.metadata.version(PROJECT_NAME)
+    return version
 
-[formatters]
-keys = json
 
-[formatter_json]
-format = %(message)s
-class = pythonjsonlogger.jsonlogger.JsonFormatter
+PROJECT_NAME = 'test-api'
+PROJECT_VERSION = get_version_metadata()
+ENVIRONMENT = os.environ.get('ENVIRONMENT', 'dev')
 ```
 
 ## Logging Expensive to Compute Data

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "python-json-logger"
-version = "3.3.0"
+version = "3.3.1.dev0"
 description = "JSON Log Formatter for the Python Logging Package"
 authors = [
     {name = "Zakaria Zajac", email = "zak@madzak.com"},

src/pythonjsonlogger/core.py

@@ -215,9 +215,18 @@ def __init__(
         ## JSON Logging specific
         ## ---------------------------------------------------------------------
         self.prefix = prefix
-        self.rename_fields = rename_fields if rename_fields is not None else {}
+
+        # We recreate the dict in rename_fields and static_fields to support internal/external
+        # references which require getting the item to do the conversion.
+        # For more details see: https://github.com/nhairs/python-json-logger/pull/45
+        self.rename_fields = (
+            {key: rename_fields[key] for key in rename_fields} if rename_fields is not None else {}
+        )
+        self.static_fields = (
+            {key: static_fields[key] for key in static_fields} if static_fields is not None else {}
+        )
+
         self.rename_fields_keep_missing = rename_fields_keep_missing
-        self.static_fields = static_fields if static_fields is not None else {}
         self.reserved_attrs = set(reserved_attrs if reserved_attrs is not None else RESERVED_ATTRS)
         self.timestamp = timestamp
 

tests/test_dictconfig.py

@@ -0,0 +1,80 @@
+### IMPORTS
+### ============================================================================
+## Future
+from __future__ import annotations
+
+## Standard Library
+from dataclasses import dataclass
+import io
+import json
+import logging
+import logging.config
+from typing import Any, Generator
+
+## Installed
+import pytest
+
+### SETUP
+### ============================================================================
+_LOGGER_COUNT = 0
+EXT_VAL = 999
+
+LOGGING_CONFIG = {
+    "version": 1,
+    "disable_existing_loggers": False,
+    "formatters": {
+        "default": {
+            "()": "pythonjsonlogger.json.JsonFormatter",
+            "static_fields": {"ext-val": "ext://tests.test_dictconfig.EXT_VAL"},
+        }
+    },
+    "handlers": {
+        "default": {
+            "level": "DEBUG",
+            "formatter": "default",
+            "class": "logging.StreamHandler",
+            "stream": "ext://sys.stdout",  # Default is stderr
+        },
+    },
+    "loggers": {
+        "": {"handlers": ["default"], "level": "WARNING", "propagate": False},  # root logger
+    },
+}
+
+
+@dataclass
+class LoggingEnvironment:
+    logger: logging.Logger
+    buffer: io.StringIO
+
+    def load_json(self) -> Any:
+        return json.loads(self.buffer.getvalue())
+
+
+@pytest.fixture
+def env() -> Generator[LoggingEnvironment, None, None]:
+    global _LOGGER_COUNT  # pylint: disable=global-statement
+    _LOGGER_COUNT += 1
+    logging.config.dictConfig(LOGGING_CONFIG)
+    default_formatter = logging.root.handlers[0].formatter
+    logger = logging.getLogger(f"pythonjsonlogger.tests.{_LOGGER_COUNT}")
+    logger.setLevel(logging.DEBUG)
+    buffer = io.StringIO()
+    handler = logging.StreamHandler(buffer)
+    handler.setFormatter(default_formatter)
+    logger.addHandler(handler)
+    yield LoggingEnvironment(logger=logger, buffer=buffer)
+    logger.removeHandler(handler)
+    logger.setLevel(logging.NOTSET)
+    buffer.close()
+    return
+
+
+### TESTS
+### ============================================================================
+def test_external_reference_support(env: LoggingEnvironment):
+    env.logger.info("hello")
+    log_json = env.load_json()
+
+    assert log_json["ext-val"] == EXT_VAL
+    return