JSON Template Layout

JsonTemplateLayout is a customizable, efficient, and garbage-free JSON generating layout. It encodes LogEvents according to the structure described by the JSON template provided. In a nutshell, it shines with its

JSON Template Layout is intended for production deployments where the generated logs are expected to be delivered to an external log ingestion system such as Elasticsearch or Google Cloud Logging. While running tests or developing locally, you can use Pattern Layout for human-readable log output.

Usage

Adding log4j-layout-template-json artifact to your list of dependencies is enough to enable access to JSON Template Layout in your Log4j configuration:

  • Maven

  • Gradle

We assume you use log4j-bom for dependency management.

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-layout-template-json</artifactId>
  <scope>runtime</scope>
</dependency>

We assume you use log4j-bom for dependency management.

runtimeOnly 'org.apache.logging.log4j:log4j-layout-template-json'

JSON Template Layout is primarily configured through an event template describing the structure log events should be JSON-encoded in. Event templates themselves are also JSON documents, where objects containing $resolver members, such as,

{
  "$resolver": "message", (1)
  "stringified": true (2)
}
1 Indicating that this object should be replaced with the output from the message event template resolver
2 Passing a configuration to the message event template resolver

are interpreted by the JSON Template Layout compiler, and replaced with the referenced event or stack trace template resolver rendering that particular item.

For instance, given the following event template stored in MyLayout.json in your classpath:

{
  "instant": { (1)
    "$resolver": "timestamp",
    "pattern": {
      "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
      "timeZone": "UTC"
    }
  },
  "someConstant": 1, (2)
  "message": { (3)
    "$resolver": "message",
    "stringified": true
  }
}
1 Using the timestamp event template resolver to populate the instant field
2 Passing a constant that will be rendered as is
3 Using the message event template resolver to populate the message field

in combination with the below layout configuration:

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<JsonTemplateLayout eventTemplateUri="classpath:MyLayout.json"/>
Snippet from an example log4j2.json
"JsonTemplateLayout": {
  "eventTemplateUri": "classpath:MyLayout.json"
}
Snippet from an example log4j2.yaml
JsonTemplateLayout:
  eventTemplateUri: "classpath:MyLayout.json"
Snippet from an example log4j2.properties
appender.0.layout.type = JsonTemplateLayout
appender.0.layout.eventTemplateUri = classpath:MyLayout.json

JSON Template Layout generates JSON as follows:

{"instant":"2017-05-25T19:56:23.370Z","someConstant":1,"message":"Hello, error!"} (1)
1 JSON pretty-printing is not supported for performance reasons.

Good news is JSON Template Layout is perfectly production-ready without any configuration! The event template defaults to EcsLayout.json, bundled in the classpath, modelling the Elastic Common Schema (ECS) specification. JSON Template Layout bundles several more predefined event templates modeling popular JSON-based log formats.

Configuration

This section explains how to configure JSON Template Layout plugin element in a Log4j configuration file.

Are you trying to implement your own event (or stack trace) template and looking for help on available resolvers? Please refer to Template configuration instead.

Plugin attributes

JSON Template Layout plugin configuration accepts the following attributes:

charset

Type

Charset

Default value

UTF-8

Configuration property

log4j.layout.jsonTemplate.charset

Charset used for encoding the produced JSON into bytes

While RFC 4627, the JSON specification, supports multiple Unicode encodings, we strongly advise you to stick to the layout default, and use UTF-8 for several practical reasons.

locationInfoEnabled

Type

boolean

Default value

false

Configuration property

log4j.layout.jsonTemplate.locationInfoEnabled

Toggles access to the LogEvent source; file name, line number, etc. See also location information.

stackTraceEnabled

Type

boolean

Default value

true

Configuration property

log4j.layout.jsonTemplate.stackTraceEnabled

Toggles access to the stack traces

eventTemplate

Type

String

Default value

null

Configuration property

log4j.layout.jsonTemplate.eventTemplate

Inline event template JSON for rendering LogEvents. If present, this configuration overrides eventTemplateUri.

eventTemplateUri

Type

String

Default value

classpath:EcsLayout.json

Configuration property

log4j.layout.jsonTemplate.eventTemplateUri

URI pointing to the event template JSON for rendering LogEvents. This configuration is overriden by eventTemplate, if present.

eventTemplateRootObjectKey

Type

String

Default value

null

Configuration property

log4j.layout.jsonTemplate.eventTemplateRootObjectKey

If present, the event template is put into a JSON object composed of a single member with the provided key.

stackTraceElementTemplate

Type

String

Default value

null

Configuration property

log4j.layout.jsonTemplate.stackTraceElementTemplate

Inline stack trace element template JSON for rendering StackTraceElements If present, this configuration overrides stackTraceElementTemplateUri.

stackTraceElementTemplateUri

Type

String

Default value

classpath:StackTraceElementLayout.json

Configuration property

log4j.layout.jsonTemplate.stackTraceElementTemplateUri

URI pointing to the stack trace element template JSON for rendering StackTraceElements. This configuration is overriden by stackTraceElementTemplate, if present.

eventDelimiter

Type

String

Default value

System.lineSeparator()

Configuration property

log4j.layout.jsonTemplate.eventDelimiter

Delimiter used for separating rendered LogEvents. if nullEventDelimiterEnabled is true, this value will be suffixed with \0 (null) character.

nullEventDelimiterEnabled

Type

String

Default value

false

Configuration property

log4j.layout.jsonTemplate.nullEventDelimiterEnabled

If true, eventDelimiter will be suffixed with \0 (null) character.

maxStringLength

Type

int

Default value

16384 (16 KiB)

Configuration property

log4j.layout.jsonTemplate.maxStringLength

Causes truncation of string values longer than the specified limit. When a string value is truncated, its length will be shortened to the maxStringLength provided and truncatedStringSuffix will be appended to indicate the truncation.

Note that this doesn’t cap the maximum length of the JSON document produced! Consider a JSON document rendered using a LogEvent containing 20,000 thread context map (aka. MDC) entries such that each key/value is less than 16,384 characters. This document will certainly exceed 16,384 characters, yet it will not be subject to any truncation, since every string value in the JSON document is less than 16,384 characters. That is,

An example JSON document exceeding 16,384 characters in length, yet subject to no truncation
{
  "mdc": {
    "key00001": "value00001",
    "key00002": "value00002",
    "key00003": "value00003",
    // ...
    "key16384": "value16384"
  }
}

truncatedStringSuffix

Type

String

Default value

…

Configuration property

log4j.layout.jsonTemplate.truncatedStringSuffix

Suffix to append to strings truncated due to exceeding maxStringLength

Plugin elements

JSON Template Layout plugin configuration accepts the following elements:

EventTemplateAdditionalField

Additional event template fields are convenient shortcuts to add custom fields to a template or override existing ones. You can specify an additional event template field using an EventTemplateAdditionalField element composed of following attributes:

key

Entry key

value

Entry value

format

Format of the value. Accepted values are:

STRING (default)

Indicates that the entry value will be string-encoded

JSON

Indicates the the entry value is already formatted in JSON and will be appended to the event template verbatim

Below we share an example configuration overriding the GelfLayout.json event template with certain custom fields:

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json">
  <EventTemplateAdditionalField
      key="aString"
      value="foo"/>(1)
  <EventTemplateAdditionalField
      key="marker"
      format="JSON"
      value='{"$resolver": "marker", "field": "name"}'/>
  <EventTemplateAdditionalField
      key="aNumber"
      format="JSON"
      value="1"/>
  <EventTemplateAdditionalField
      key="aList"
      format="JSON"
      value='[1, 2, "three"]'/>
</JsonTemplateLayout>
Snippet from an example log4j2.json
"JsonTemplateLayout": {
  "eventTemplateUri": "classpath:GelfLayout.json",
  "eventTemplateAdditionalField": [
    {
      "key": "aString",
      "value": "foo" (1)
    },
    {
      "key": "marker",
      "value": "{\"$resolver\": \"marker\", \"field\": \"name\"}",
      "format": "JSON"
    },
    {
      "key": "aNumber",
      "value": "1",
      "format": "JSON"
    },
    {
      "key": "aList",
      "value": "[1, 2, \"three\"]",
      "format": "JSON"
    }
  ]
}
Snippet from an example log4j2.yaml
JsonTemplateLayout:
  eventTemplateUri: "classpath:GelfLayout.json"
  eventTemplateAdditionalField:
    - key: "aString"
      value: "foo" (1)
    - key: "marker"
      value: '{"$resolver": "marker", "field": "name"}'
      format: "JSON"
    - key: "aNumber"
      value: "1"
      format: "JSON"
    - key: "aList"
      value: '[1, 2, "three"]'
      format: "JSON"
Snippet from an example log4j2.properties
appender.0.layout.type = JsonTemplateLayout
appender.0.layout.eventTemplateUri = classpath:GelfLayout.json
appender.0.layout.eventTemplateAdditionalField[0].type = EventTemplateAdditionalField
appender.0.layout.eventTemplateAdditionalField[0].key = aString
appender.0.layout.eventTemplateAdditionalField[0].value = foo (1)
appender.0.layout.eventTemplateAdditionalField[1].type = EventTemplateAdditionalField
appender.0.layout.eventTemplateAdditionalField[1].key = marker
appender.0.layout.eventTemplateAdditionalField[1].value = {"$resolver": "marker", "field": "name"}
appender.0.layout.eventTemplateAdditionalField[1].format = JSON
appender.0.layout.eventTemplateAdditionalField[2].type = EventTemplateAdditionalField
appender.0.layout.eventTemplateAdditionalField[2].key = aNumber
appender.0.layout.eventTemplateAdditionalField[2].value = 1
appender.0.layout.eventTemplateAdditionalField[2].format = JSON
appender.0.layout.eventTemplateAdditionalField[3].type = EventTemplateAdditionalField
appender.0.layout.eventTemplateAdditionalField[3].key = aList
appender.0.layout.eventTemplateAdditionalField[3].value = [1, 2, "three"]
appender.0.layout.eventTemplateAdditionalField[3].format = JSON
1 Since the format attribute is not explicitly set, the default (i.e., STRING) will be used

Template configuration

Templates are JSON documents, where objects containing $resolver members, such as,

{
  "$resolver": "message", (1)
  "stringified": true (2)
}
1 Indicating that this object should be replaced with the output from the message event template resolver
2 Passing a configuration to the message event template resolver

are interpreted by the JSON Template Layout compiler, and replaced with the referenced event or stack trace template resolver rendering that particular item.

Templates are configured by means of the following JSON Template Layout plugin attributes:

Event templates

eventTemplate and eventTemplateUri describe the JSON structure JSON Template Layout uses to encode LogEvents. JSON Template Layout contains the following predefined event templates:

EcsLayout.json

The default event template modelling the Elastic Common Schema (ECS) specification

LogstashJsonEventLayoutV1.json

Models Logstash json_event pattern for Log4j

GelfLayout.json

Models the Graylog Extended Log Format (GELF) payload specification with additional _thread and _logger fields.

If used, it is advised to override the obligatory host field with a user provided constant via additional event template fields to avoid hostName property lookup at runtime, which incurs an extra cost.

GcpLayout.json

Models the structure described by Google Cloud Platform structured logging with additional _thread, _logger and _exception fields. The exception trace, if any, is written to the _exception field as well as the message field – the former is useful for explicitly searching/analyzing structured exception information, while the latter is Google’s expected place for the exception, and integrates with Google Error Reporting.

Event template resolvers

Event template resolvers consume a LogEvent and render a certain property of it at the point of the JSON where they are declared. For instance, marker resolver renders the marker of the event, level resolver renders the level, and so on. An event template resolver is denoted with a special object containing a $resolver key:

Example event template demonstrating the usage of level resolver
{
  "version": "1.0",
  "level": {
    "$resolver": "level",
    "field": "name"
  }
}

Here version field will be rendered as is, while level field will be populated by the level resolver. That is, this template will generate JSON similar to the following:

Example JSON generated from the demonstrated event template
{
  "version": "1.0",
  "level": "INFO"
}

The complete list of available event template resolvers are provided below in detail.

counter

Resolves a number from an internal counter

counter event template resolver grammar
config      = [ start ] , [ overflowing ] , [ stringified ]
start       = "start" -> number
overflowing = "overflowing" -> boolean
stringified = "stringified" -> boolean

Unless provided, start and overflowing are respectively set to 0 (zero) and true by default.

When overflowing is set to true, the internal counter is created using a long, which is subject to overflow while incrementing, though faster and garbage-free. Otherwise, a BigInteger is used, which does not overflow, but incurs allocation costs.

When stringified is enabled, which is set to false by default, the resolved number will be converted to a string.

See examples

Resolves a sequence of numbers starting from 0. Once Long.MAX_VALUE is reached, counter overflows to Long.MIN_VALUE.

{
  "$resolver": "counter"
}

Resolves a sequence of numbers starting from 1000. Once Long.MAX_VALUE is reached, counter overflows to Long.MIN_VALUE.

{
  "$resolver": "counter",
  "start": 1000
}

Resolves a sequence of numbers starting from 0 and keeps on doing as long as JVM heap allows.

{
  "$resolver": "counter",
  "overflowing": false
}
caseConverter

Converts the case of string values

caseConverter event template resolver grammar
config                = case , input , [ locale ] , [ errorHandlingStrategy ]
input                 = JSON
case                  = "case" -> ( "upper" | "lower" )
locale                = "locale" -> (
                            language                                   |
                          ( language , "_" , country )                 |
                          ( language , "_" , country , "_" , variant )
                        )
errorHandlingStrategy = "errorHandlingStrategy" -> (
                          "fail"    |
                          "pass"    |
                          "replace"
                        )
replacement           = "replacement" -> JSON

input can be any available template value; e.g., a JSON literal, a lookup string, an object pointing to another resolver.

Unless provided, locale points to the one returned by JsonTemplateLayoutDefaults.getLocale(), which is configured by the log4j.layout.jsonTemplate.locale system property and by default set to the default system locale.

errorHandlingStrategy determines the behavior when either the input doesn’t resolve to a string value or case conversion throws an exception:

fail

Propagates the failure

pass

Causes the resolved value to be passed as is

replace

Suppresses the failure and replaces it with the replacement, which is set to null by default

errorHandlingStrategy is set to replace by default.

Most of the time JSON logs are persisted to a storage solution (e.g., Elasticsearch) that keeps a statically-typed index on fields. Hence, if a field is always expected to be of type string, using non-string replacements or pass in errorHandlingStrategy might result in type incompatibility issues at the storage level.

Unless the input value is passed intact or replaced, case conversion is not garbage-free.

See examples

Convert the resolved log level strings to upper-case:

{
  "$resolver": "caseConverter",
  "case": "upper",
  "input": {
    "$resolver": "level",
    "field": "name"
  }
}

Convert the resolved USER environment variable to lower-case using nl_NL locale:

{
  "$resolver": "caseConverter",
  "case": "lower",
  "locale": "nl_NL",
  "input": "${env:USER}"
}

Convert the resolved sessionId thread context data (MDC) to lower-case:

{
  "$resolver": "caseConverter",
  "case": "lower",
  "input": {
    "$resolver": "mdc",
    "key": "sessionId"
  }
}

Above, if sessionId MDC resolves to a, say, number, case conversion will fail. Since errorHandlingStrategy is set to replace and replacement is set to null by default, the resolved value will be null. One can suppress this behavior and let the resolved sessionId number be left as is:

{
  "$resolver": "caseConverter",
  "case": "lower",
  "input": {
    "$resolver": "mdc",
    "key": "sessionId"
  },
  "errorHandlingStrategy": "pass"
}

or replace it with a custom string:

{
  "$resolver": "caseConverter",
  "case": "lower",
  "input": {
    "$resolver": "mdc",
    "key": "sessionId"
  },
  "errorHandlingStrategy": "replace",
  "replacement": "unknown"
}
endOfBatch

Resolves LogEvent#isEndOfBatch() boolean flag

exception

Resolves fields of the Throwable returned by LogEvent#getThrown()

exception event template resolver grammar
config              = field , [ stringified ] , [ stackTrace ]
field               = "field" -> ( "className" | "message" | "stackTrace" )

stackTrace          = "stackTrace" -> (
                        [ stringified ]
                      , [ elementTemplate ]
                      )

stringified         = "stringified" -> ( boolean | truncation )
truncation          = "truncation" -> (
                        [ suffix ]
                      , [ pointMatcherStrings ]
                      , [ pointMatcherRegexes ]
                      )
suffix              = "suffix" -> string
pointMatcherStrings = "pointMatcherStrings" -> string[]
pointMatcherRegexes = "pointMatcherRegexes" -> string[]

elementTemplate     = "elementTemplate" -> object

stringified is set to false by default. stringified at the root level is deprecated in favor of stackTrace.stringified, which has precedence if both are provided.

pointMatcherStrings and pointMatcherRegexes enable the truncation of stringified stack traces after the given matching point. If both parameters are provided, pointMatcherStrings will be checked first.

If a stringified stack trace truncation takes place, it will be indicated with a`suffix`, which by default is set to the configure truncatedStringSuffix in the layout, unless explicitly provided. Every truncation suffix is prefixed with a newline.

Stringified stack trace truncation operates in Caused by: and Suppressed: label blocks. That is, matchers are executed against each label in isolation.

elementTemplate is an object describing the template to be used while resolving the StackTraceElement array. If stringified is set to true, elementTemplate will be discarded. By default, elementTemplate is set to null and rather populated from the layout configuration. That is, the stack trace element template can also be provided using stackTraceElementTemplate and stackTraceElementTemplateUri layout configuration attributes. The template to be employed is determined in the following order:

  1. elementTemplate provided in the resolver configuration

  2. stackTraceElementTemplate layout configuration attribute (The default is populated from the log4j.layout.jsonTemplate.stackTraceElementTemplate system property.)

  3. stackTraceElementTemplateUri layout configuration attribute (The default is populated from the log4j.layout.jsonTemplate.stackTraceElementTemplateUri system property.)

See Stack trace element templates for the list of available resolvers in a stack trace element template.

Note that this resolver is toggled by the stackTraceEnabled layout configuration attribute.

Since Throwable#getStackTrace() clones the original StackTraceElement[], access to (and hence rendering of) stack traces are not garbage-free.

Each pointMatcherRegexes item triggers a Pattern#matcher() call, which is not garbage-free either.

See examples

Resolve LogEvent#getThrown().getClass().getCanonicalName():

{
  "$resolver": "exception",
  "field": "className"
}

Resolve the stack trace into a list of StackTraceElement objects:

{
  "$resolver": "exception",
  "field": "stackTrace"
}

Resolve the stack trace into a string field:

{
  "$resolver": "exception",
  "field": "stackTrace",
  "stackTrace": {
    "stringified": true
  }
}

Resolve the stack trace into a string field such that the content will be truncated after the given point matcher:

{
  "$resolver": "exception",
  "field": "stackTrace",
  "stackTrace": {
    "stringified": {
      "truncation": {
        "suffix": "... [truncated]",
        "pointMatcherStrings": ["at javax.servlet.http.HttpServlet.service"]
      }
    }
  }
}

Resolve the stack trace into an object described by the provided stack trace element template:

{
  "$resolver": "exception",
  "field": "stackTrace",
  "stackTrace": {
    "elementTemplate": {
      "class": {
       "$resolver": "stackTraceElement",
       "field": "className"
      },
      "method": {
       "$resolver": "stackTraceElement",
       "field": "methodName"
      },
      "file": {
       "$resolver": "stackTraceElement",
       "field": "fileName"
      },
      "line": {
       "$resolver": "stackTraceElement",
       "field": "lineNumber"
      }
    }
  }
}

See Stack trace element templates for further details on resolvers available for StackTraceElement templates.

exceptionRootCause

Resolves the fields of the innermost Throwable returned by LogEvent#getThrown(). Its syntax and garbage-footprint are identical to the exception resolver.

level

Resolves the fields of the LogEvent#getLevel()

level event template resolver grammar
config         = field , [ severity ]
field          = "field" -> ( "name" | "severity" )
severity       = severity-field
severity-field = "field" -> ( "keyword" | "code" )
See examples

Resolve the level name:

{
  "$resolver": "level",
  "field": "name"
}

Resolve the Syslog severity keyword:

{
  "$resolver": "level",
  "field": "severity",
  "severity": {
    "field": "keyword"
  }
}

Resolve the Syslog severity code:

{
  "$resolver": "level",
  "field": "severity",
  "severity": {
    "field": "code"
  }
}
logger

Resolves LogEvent#getLoggerFqcn() and LogEvent#getLoggerName().

logger event template grammar
config = "field" -> ( "name" | "fqcn" )
See examples

Resolve the logger name:

{
  "$resolver": "logger",
  "field": "name"
}

Resolve the logger’s fully qualified class name:

{
  "$resolver": "logger",
  "field": "fqcn"
}
main

Performs Main Argument Lookup for the given index or key

main event template resolver grammar
config = ( index | key )
index  = "index" -> number
key    = "key" -> string
See examples

Resolve the 1st main() method argument:

{
  "$resolver": "main",
  "index": 0
}

Resolve the argument coming right after --userId:

{
  "$resolver": "main",
  "key": "--userId"
}
map

Resolves MapMessages. See Map resolver for details.

marker

Resolves the marker of the event

marker event template resolver grammar
config = "field" -> ( "name" | "parents" )
See examples

Resolve the marker name:

{
  "$resolver": "marker",
  "field": "name"
}

Resolve the names of the marker’s parents:

{
  "$resolver": "marker",
  "field": "parents"
}
mdc

Resolves the thread context map, aka. Mapped Diagnostic Context (MDC). See Map resolver for details.

log4j2.garbagefreeThreadContextMap flag needs to be turned on to iterate the map without allocations.

message

Resolves the message of the event

message event template resolver grammar
config      = [ stringified ] , [ fallbackKey ]
stringified = "stringified" -> boolean
fallbackKey = "fallbackKey" -> string

For simple string messages, the resolution is performed without allocations. For ObjectMessages and MultiformatMessages, it depends.

See examples

Resolve the message into a string:

{
  "$resolver": "message",
  "stringified": true
}

Resolve the message such that if it is an ObjectMessage or a MultiformatMessage with JSON support, its type (string, list, object, etc.) will be retained:

{
  "$resolver": "message"
}

Given the above configuration, a SimpleMessage will generate a "sample log message", whereas a MapMessage will generate a {"action": "login", "sessionId": "87asd97a"}. Certain indexed log storage systems (e.g., Elasticsearch) will not allow both values to coexist due to type mismatch: one is a string while the other is an object. Here one can use a fallbackKey to work around this problem:

{
  "$resolver": "message",
  "fallbackKey": "formattedMessage"
}

Using this configuration, a SimpleMessage will generate a {"formattedMessage": "sample log message"}, and a MapMessage will generate a {"action": "login", "sessionId": "87asd97a"}. Note that both emitted JSONs are of type object and have no type-conflicting fields.

messageParameter

Resolves LogEvent#getMessage().getParameters()

messageParameter event template resolver grammar
config      = [ stringified ] , [ index ]
stringified = "stringified" -> boolean
index       = "index" -> number

Regarding garbage footprint, stringified flag translates to String.valueOf(value), hence mind values that are not String-typed. Further, LogEvent#getMessage() is expected to implement ParameterVisitable interface, which is the case if log4j2.enableThreadlocals property set to true.

See examples

Resolve the message parameters into an array:

{
  "$resolver": "messageParameter"
}

Resolve the string representation of all message parameters into an array:

{
  "$resolver": "messageParameter",
  "stringified": true
}

Resolve the first message parameter:

{
  "$resolver": "messageParameter",
  "index": 0
}

Resolve the string representation of the first message parameter:

{
  "$resolver": "messageParameter",
  "index": 0,
  "stringified": true
}
ndc

Resolves the thread context stack – aka. Nested Diagnostic Context (NDC), aka – String[] returned by LogEvent#getContextStack()

ndc event template resolver grammar
config  = [ pattern ]
pattern = "pattern" -> string
See examples

Resolve all NDC values into a list:

{
  "$resolver": "ndc"
}

Resolve all NDC values matching with the pattern regex:

{
  "$resolver": "ndc",
  "pattern": "user(Role|Rank):\\w+"
}
pattern

Resolver delegating to Pattern Layout

pattern event template resolver grammar
config            = pattern , [ stackTraceEnabled ]
pattern           = "pattern" -> string
stackTraceEnabled = "stackTraceEnabled" -> boolean

Unlike providing the pattern attribute to Pattern Layout in a configuration file, property substitutions found in the pattern will not be resolved.

The default value of stackTraceEnabled is inherited from the parent JSON Template Layout.

This resolver is mostly intended as an emergency lever when all other JSON Template Layout resolvers fall short of addressing your need. If you find yourself using this, it is highly likely you are either doing something wrong, or JSON Template Layout needs some improvement. In either case, you are advised to share your use case with maintainers in one of our support channels.

See examples

Resolve the string produced by %p %c{1.} [%t] %X{userId} %X %m%ex pattern:

{
  "$resolver": "pattern",
  "pattern": "%p %c{1.} [%t] %X{userId} %X %m%ex"
}
source

Resolves the fields of the StackTraceElement returned by LogEvent#getSource()

source event template resolver grammar
config = "field" -> (
           "className"  |
           "fileName"   |
           "methodName" |
           "lineNumber" )

Note that this resolver is toggled by the locationInfoEnabled layout configuration attribute.

Capturing the source location information is an expensive operation, and is not garbage-free. The logger resolver can generally be used as a zero-cost substitute for className. See this section of the layouts page for details.

See examples

Resolve the line number:

{
  "$resolver": "source",
  "field": "lineNumber"
}
thread

Resolves LogEvent#getThreadId(), LogEvent#getThreadName(), LogEvent#getThreadPriority()

thread event template resolver grammar
config = "field" -> ( "name" | "id" | "priority" )
See examples

Resolve the thread name:

{
  "$resolver": "thread",
  "field": "name"
}
timestamp

Resolves LogEvent#getInstant()

timestamp event template resolver grammar
config        = [ patternConfig | epochConfig ]

patternConfig = "pattern" -> ( [ format ] , [ timeZone ] , [ locale ] )
format        = "format" -> string
timeZone      = "timeZone" -> string
locale        = "locale" -> (
                   language                                   |
                 ( language , "_" , country )                 |
                 ( language , "_" , country , "_" , variant )
                )

epochConfig   = "epoch" -> ( unit , [ rounded ] )
unit          = "unit" -> (
                   "nanos"         |
                   "millis"        |
                   "secs"          |
                   "millis.nanos"  |
                   "secs.nanos"    |
                )
rounded       = "rounded" -> boolean
See examples
Table 1. timestamp template resolver examples

Configuration

Output

{
  "$resolver": "timestamp"
}

2020-02-07T13:38:47.098+02:00

{
  "$resolver": "timestamp",
  "pattern": {
    "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
    "timeZone": "UTC",
    "locale": "en_US"
  }
}

2020-02-07T13:38:47.098Z

{
  "$resolver": "timestamp",
  "epoch": {
    "unit": "secs"
  }
}

1581082727.982123456

{
  "$resolver": "timestamp",
  "epoch": {
    "unit": "secs",
    "rounded": true
  }
}

1581082727

{
  "$resolver": "timestamp",
  "epoch": {
    "unit": "secs.nanos"
  }
}

982123456

{
  "$resolver": "timestamp",
  "epoch": {
    "unit": "millis"
  }
}

1581082727982.123456

{
  "$resolver": "timestamp",
  "epoch": {
    "unit": "millis",
    "rounded": true
  }
}

1581082727982

{
  "$resolver": "timestamp",
  "epoch": {
    "unit": "millis.nanos"
  }
}

123456

{
  "$resolver": "timestamp",
  "epoch": {
    "unit": "nanos"
  }
}

1581082727982123456

Map resolver

ReadOnlyStringMap is Log4j’s Map<String, Object> equivalent with garbage-free accessors, and is heavily employed throughout the code base. It is the data structure backing both thread context map (aka., Mapped Diagnostic Context (MDC)) and MapMessage implementations. Hence template resolvers for both of these are provided by a single backend: ReadOnlyStringMapResolver. Put another way, both mdc and map resolvers support identical configuration, behaviour, and garbage footprint, which are detailed below.

Map resolver grammar
config        = singleAccess | multiAccess

singleAccess  = key , [ stringified ]
key           = "key" -> string
stringified   = "stringified" -> boolean

multiAccess   = [ pattern ] , [ replacement ] , [ flatten ] , [ stringified ]
pattern       = "pattern" -> string
replacement   = "replacement" -> string
flatten       = "flatten" -> ( boolean | flattenConfig )
flattenConfig = [ flattenPrefix ]
flattenPrefix = "prefix" -> string

singleAccess resolves a single field, whilst multiAccess resolves a multitude of fields. If flatten is provided, multiAccess merges the fields with the parent, otherwise creates a new JSON object containing the values.

Enabling stringified flag converts each value to its string representation.

Regex provided in the pattern is used to match against the keys. If provided, replacement will be used to replace the matched keys. These two are analogous to Pattern.compile(pattern).matcher(key).matches() and Pattern.compile(pattern).matcher(key).replaceAll(replacement) calls.

Regarding garbage footprint, stringified flag translates to String.valueOf(value), hence mind values that are not String-typed.

pattern and replacement incur pattern matcher allocation costs.

Writing certain non-primitive values (e.g., BigDecimal, Set, etc.) to JSON generates garbage, though most (e.g., int, long, String, List, boolean[], etc.) don’t.

See examples

"$resolver" is left out in the following examples, since it is to be defined by the actual resolver, e.g., map, mdc.

Resolve the value of the field keyed with user:role:

{
  "$resolver": "…",
  "key": "user:role"
}

Resolve the string representation of the user:rank field value:

{
  "$resolver": "…",
  "key": "user:rank",
  "stringified": true
}

Resolve all fields into an object:

{
  "$resolver": "…"
}

Resolve all fields into an object such that values are converted to string:

{
  "$resolver": "…",
  "stringified": true
}

Resolve all fields whose keys match with the user:(role|rank) regex into an object:

{
  "$resolver": "…",
  "pattern": "user:(role|rank)"
}

Resolve all fields whose keys match with the user:(role|rank) regex into an object after removing the user: prefix in the key:

{
  "$resolver": "…",
  "pattern": "user:(role|rank)",
  "replacement": "$1"
}

Merge all fields whose keys are matching with the user:(role|rank) regex into the parent:

{
  "$resolver": "…",
  "flatten": true,
  "pattern": "user:(role|rank)"
}

After converting the corresponding field values to string, merge all fields to parent such that keys are prefixed with _:

{
  "$resolver": "…",
  "stringified": true,
  "flatten": {
    "prefix": "_"
  }
}

Stack trace element templates

exception and exceptionRootCause event template resolvers can encode an exception stack trace (i.e., StackTraceElement returned by Throwable#getStackTrace()) into a JSON array. While doing so, they employ the underlying JSON templating infrastructure again. The stack trace template used by these event template resolvers to encode StackTraceElements can be configured in following ways:

  1. elementTemplate provided in the resolver configuration

  2. stackTraceElementTemplate layout configuration attribute (The default is populated from the log4j.layout.jsonTemplate.stackTraceElementTemplate system property.)

  3. stackTraceElementTemplateUri layout configuration attribute (The default is populated from the log4j.layout.jsonTemplate.stackTraceElementTemplateUri system property.)

By default, stackTraceElementTemplateUri is set to classpath:StackTraceElementLayout.json, which references to a classpath resource bundled with JSON Template Layout:

StackTraceElementLayout.json bundled as a classpath resource
{
  "class": {
    "$resolver": "stackTraceElement",
    "field": "className"
  },
  "method": {
    "$resolver": "stackTraceElement",
    "field": "methodName"
  },
  "file": {
    "$resolver": "stackTraceElement",
    "field": "fileName"
  },
  "line": {
    "$resolver": "stackTraceElement",
    "field": "lineNumber"
  }
}

Stack trace element template resolvers

Similar to Event template resolvers consuming a LogEvent and rendering a certain property of it at the point of the JSON where they are declared, stack trace element template resolvers consume a StackTraceElement.

The complete list of available event template resolvers are provided below in detail.

stackTraceElement

Resolves certain fields of a StackTraceElement

Stack trace template resolver grammar
config = "field" -> (
           "className"  |
           "fileName"   |
           "methodName" |
           "lineNumber" )

All above accesses to StackTraceElement is garbage-free.

Property substitution

Property substitutions (e.g., ${myProperty}), including lookups (e.g., ${java:version}, ${env:USER}, ${date:MM-dd-yyyy}), are supported, but extra care needs to be taken. We strongly advise you to carefully read the configuration manual before using them.

Lookups are intended as a very generic, convenience utility to perform string interpolation for, in particular, configuration files and components (e.g., layouts) lacking this mechanism. JSON Template Layout has a rich template resolver collection, and you should always prefer it whenever possible over lookups.

Which resolvers can I use to replace lookups?

Property substitution in event templates

JSON Template Layout performs property substitution in string literals in templates, except if they are located in configuration object of resolvers. Consider the following event template file provided using the eventTemplateUri attribute:

{
    "java-version": "${java:version}", (1)
    "pid": {
      "$resolver": "pattern",
      "pattern": "${env:NO_SUCH_KEY:-%pid}" (2)
    }
}
1 This works. ${java:version} will be replaced with the corresponding value.
2 This won’t work! That is, ${env:NO_SUCH_KEY:-%pid} literal will not get substituted, since it is located in a configuration object of a resolver.

Property substitution in configuration files

If the very same event template shared above is inlined in a configuration file using the eventTemplate attribute or additional event template fields, then all substitutions will be replaced, once, at configuration-time. This has nothing to do with the JSON Template Layout, but the substitution performed by the configuration mechanism when the configuration is read. Consider the following example:

  • XML

  • JSON

  • YAML

  • Properties

Snippet from an example log4j2.xml
<JsonTemplateLayout eventTemplate='{"instant": {"$resolver": "pattern", "pattern": "${env:LOG4J_DATE_PATTERN:-%d}"}}'> (1)
  <EventTemplateAdditionalField
      key="message"
      format="JSON"
      value='{"$resolver": "pattern", "pattern": "${env:LOG4J_MESSAGE_PATTERN:-%m}"}'/> (2)
</JsonTemplateLayout>
Snippet from an example log4j2.json
"JsonTemplateLayout": {
  "eventTemplate": "{\"instant\": {\"$resolver\": \"pattern\", \"pattern\": \"${env:LOG4J_DATE_PATTERN:-%d}\"}}", (1)
  "eventTemplateAdditionalField": [
    {
      "key": "message",
      "format": "JSON",
      "value": "{\"$resolver\": \"pattern\", \"pattern\": \"${env:LOG4J_MESSAGE_PATTERN:-%m}\"}" (2)
    }
  ]
}
Snippet from an example log4j2.yaml
JsonTemplateLayout:
  eventTemplate: '{"instant": {"$resolver": "pattern", "pattern": "${env:LOG4J_DATE_PATTERN:-%d}"}}' (1)
  eventTemplateAdditionalField:
    - key: "message"
      format: "JSON"
      value: '{"$resolver": "pattern", "pattern": "${env:LOG4J_MESSAGE_PATTERN:-%m}"}' (2)
Snippet from an example log4j2.properties
appender.0.layout.type = JsonTemplateLayout
appender.0.layout.eventTemplate = {"instant": {"$resolver": "pattern", "pattern": "${env:LOG4J_DATE_PATTERN:-%d}"}} (1)
appender.0.layout.eventTemplateAdditionalField[0].type = EventTemplateAdditionalField
appender.0.layout.eventTemplateAdditionalField[0].key = message
appender.0.layout.eventTemplateAdditionalField[0].format = JSON
appender.0.layout.eventTemplateAdditionalField[0].value = {"$resolver": "pattern", "pattern": "${env:LOG4J_MESSAGE_PATTERN:-%m}"} (2)
1 eventTemplate will be passed to the layout with ${env:…​} substituted
2 value will be passed to the layout with ${env:…​} substituted

External values injected this way can corrupt your JSON schema. It is your responsibility to ensure the sanitization and safety of the substitution source.

Extending

JSON Template Layout relies on the Log4j plugin system to compose the features it provides. This makes it possible for users to extend the plugin-based feature set as they see fit. As of this moment, following features are implemented by means of plugins:

  • Event template resolvers (e.g., exception, message, level event template resolvers)

  • Event template interceptors (e.g., injection of eventTemplateAdditionalField)

Following sections cover how to extend these in detail.

While existing features should address most common use cases, you might find yourself needing to implement a custom one. If this is the case, we really appreciate it if you can share your use case in a user support channel.

Plugin preliminaries

Log4j plugin system is the de facto extension mechanism embraced by various Log4j components. Plugins make it possible for extensible components to receive feature implementations without any explicit links in between. It is analogous to a dependency injection framework, but curated for Log4j-specific needs.

In a nutshell, you annotate your classes with @Plugin and their (static) factory methods with @PluginFactory. Last, you inform the Log4j plugin system to discover these custom classes. This is done using running the PluginProcessor annotation processor while building your project. Refer to Plugins for details.

Extending event template resolvers

All available event template resolvers are simple plugins employed by JSON Template Layout. To add new ones, one just needs to create their own EventResolver and instruct its injection via a @Plugin-annotated EventResolverFactory class.

For demonstration purposes, below we will create a randomNumber event template resolver. Let’s start with the actual resolver:

Custom random number event template resolver
package com.acme.logging.log4j.layout.template.json;

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolver;
import org.apache.logging.log4j.layout.template.json.util.JsonWriter;

/**
 * Resolves a random floating point number.
 *
 * <h3>Configuration</h3>
 *
 * <pre>
 * config = ( [ range ] )
 * range  = number[]
 * </pre>
 *
 * {@code range} is a number array with two elements, where the first number
 * denotes the start (inclusive) and the second denotes the end (exclusive).
 * {@code range} is optional and by default set to {@code [0, 1]}.
 *
 * <h3>Examples</h3>
 *
 * Resolve a random number between 0 and 1:
 *
 * <pre>
 * {
 *   "$resolver": "randomNumber"
 * }
 * </pre>
 *
 * Resolve a random number between -0.123 and 0.123:
 *
 * <pre>
 * {
 *   "$resolver": "randomNumber",
 *   "range": [-0.123, 0.123]
 * }
 * </pre>
 */
public final class RandomNumberResolver implements EventResolver {

    private final double loIncLimit;

    private final double hiExcLimit;

    RandomNumberResolver(TemplateResolverConfig config) {
        List<Number> rangeArray = config.getList("range", Number.class);
        if (rangeArray == null) {
            this.loIncLimit = 0D;
            this.hiExcLimit = 1D;
        } else if (rangeArray.size() != 2) {
            throw new IllegalArgumentException(
                    "range array must be of size two: " + config);
        } else {
            this.loIncLimit = rangeArray.get(0).doubleValue();
            this.hiExcLimit = rangeArray.get(1).doubleValue();
            if (loIncLimit > hiExcLimit) {
                throw new IllegalArgumentException("invalid range: " + config);
            }
        }
    }

    static String getName() {
        return "randomNumber";
    }

    @Override
    public void resolve(LogEvent value, JsonWriter jsonWriter) {
        double randomNumber = loIncLimit + (hiExcLimit - loIncLimit) * Math.random();
        jsonWriter.writeNumber(randomNumber);
    }

}

Next, create an EventResolverFactory class to admit RandomNumberResolver into the event resolver factory plugin registry.

Resolver factory class to admit RandomNumberResolver into the event resolver factory plugin registry
package com.acme.logging.log4j.layout.template.json;

import org.apache.logging.log4j.plugins.Plugin;
import org.apache.logging.log4j.plugins.PluginFactory;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverContext;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverFactory;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolver;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverConfig;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverFactory;

/**
 * {@link RandomNumberResolver} factory.
 */
@Category(TemplateResolverFactory.CATEGORY)
@Plugin("RandomNumberResolverFactory")
public final class RandomNumberResolverFactory implements EventResolverFactory {

    private static final RandomNumberResolverFactory INSTANCE = new RandomNumberResolverFactory();

    private RandomNumberResolverFactory() {}

    @PluginFactory
    public static RandomNumberResolverFactory getInstance() {
        return INSTANCE;
    }

    @Override
    public String getName() {
        return RandomNumberResolver.getName();
    }

    @Override
    public RandomNumberResolver create(EventResolverContext context, TemplateResolverConfig config) {
        return new RandomNumberResolver(config);
    }

}

Done! Let’s use our custom event resolver:

Log4j configuration employing the custom randomNumber event resolver
<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns="https://logging.apache.org/xml/ns"
               xsi:schemaLocation="
                       https://logging.apache.org/xml/ns
                       https://logging.apache.org/xml/ns/log4j-config-3.xsd">
  <!-- ... -->
  <JsonTemplateLayout>
    <EventTemplateAdditionalField
        key="id"
        format="JSON"
        value='{"$resolver": "randomNumber", "range": [0, 1000000]}'/>
  </JsonTemplateLayout>
  <!-- ... -->
</Configuration>

All available event template resolvers are located in org.apache.logging.log4j.layout.template.json.resolver package. It is a fairly rich resource for inspiration while implementing new resolvers.

Intercepting the template resolver compiler

JSON Template Layout allows interception of the template resolver compilation, which is the process converting a template into a Java function performing the JSON encoding. This interception mechanism is internally used to implement eventTemplateRootObjectKey and EventTemplateAdditionalField features. In a nutshell, one needs to create a @Plugin-annotated class extending from the EventResolverInterceptor interface.

To see the interception in action, check out the EventRootObjectKeyInterceptor class which is responsible for implementing the eventTemplateRootObjectKey feature:

Event interceptor to add eventTemplateRootObjectKey, if present
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverContext;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverInterceptor;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverInterceptor;

/**
 * Interceptor to add a root object key to the event template.
 */
@Category(TemplateResolverInterceptor.CATEGORY)
@Plugin("EventRootObjectKeyInterceptor")
public class EventRootObjectKeyInterceptor implements EventResolverInterceptor {

    private static final EventRootObjectKeyInterceptor INSTANCE =
            new EventRootObjectKeyInterceptor();

    private EventRootObjectKeyInterceptor() {}

    @PluginFactory
    public static EventRootObjectKeyInterceptor getInstance() {
        return INSTANCE;
    }

    @Override
    public Object processTemplateBeforeResolverInjection(EventResolverContext context, Object node) {
        String eventTemplateRootObjectKey = context.getEventTemplateRootObjectKey();
        return eventTemplateRootObjectKey != null
                ? Collections.singletonMap(eventTemplateRootObjectKey, node)
                : node;
    }

}

Here, processTemplateBeforeResolverInjection() method checks if the user has provided an eventTemplateRootObjectKey. If so, it wraps the root node with a new object; otherwise, returns the node as is. Note that node refers to the root Java object of the event template read by JsonReader.

Features

Below is a feature comparison matrix between JSON Template Layout and alternatives.

Table 2. Feature comparison matrix

Feature

JsonTemplateLayout

EcsLayout

Java version

8

6

Dependencies

None

None

Schema customization?

✓

✕

Timestamp customization?

✓

✕

(Almost) garbage-free?

✓

✓

Custom typed Message serialization?

✓

?[1]

Custom typed MDC value serialization?

✓

✕

Rendering stack traces as array?

✓

✓

Stack trace truncation?

✓

✕

JSON pretty print?

✕

✕

Additional string fields?

✓

✓

Additional JSON fields?

✓

✕

Custom resolvers?

✓

✕

Performance

JSON Template Layout is a heavily optimized piece of software to encode a log event as fast as possible. To get the most out of it, mind the following checklist:

F.A.Q.

Are recursive collections supported?

No. Consider a Message containing a recursive value as follows:

Object[] recursiveCollection = new Object[1];
recursiveCollection[0] = recursiveCollection;

While the exact exception might vary, you will most like get a StackOverflowError for trying to render recursiveCollection into a String. Note that this is also the default behaviour for other Java standard library methods, e.g., Arrays.toString(). Hence mind self references while logging.

Is JSON Template Layout garbage-free?

Yes, if garbage-free logging is enabled. Take into account the following caveats:

  • The configured recycling strategy might not be garbage-free.

  • Since Throwable#getStackTrace() clones the original StackTraceElement[], access to (and hence rendering of) stack traces are not garbage-free.

  • Serialization of MapMessages and ObjectMessages are mostly garbage-free except for certain types (e.g., BigDecimal, BigInteger, Collections, except List).

  • Property substitutions (that is, ${…​} variables) might not be garbage-free.

Event template resolvers contain notes on their garbage footprint. Make sure to check those notes of resolvers you employ in templates.


1. Only for ObjectMessages and if Jackson is in the classpath.