Lookups

Lookups provide a way to add values to the Log4j configuration at arbitrary places. They are a particular type of Plugin that implements the StrLookup interface. Information on how to use Lookups in configuration files can be found in the Property Substitution section of the Configuration page.

Context Map Lookup

The ContextMapLookup allows applications to store data in the Log4j ThreadContext Map and then retrieve the values in the Log4j configuration. In the example below, the application would store the current user’s login id in the ThreadContext Map with the key "loginId". During initial configuration processing the first '$' will be removed. The PatternLayout supports interpolation with Lookups and will then resolve the variable for each event. Note that the pattern "%X{loginId}" would achieve the same result.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${ctx:loginId} %m%n</pattern>
  </PatternLayout>
</File>

Date Lookup

The DateLookup is somewhat unusual from the other lookups as it doesn’t use the key to locate an item. Instead, the key can be used to specify a date format string that is valid for SimpleDateFormat. The current date, or the date associated with the current log event will be formatted as specified.

<RollingFile name="Rolling-${map:type}" fileName="${filename}" filePattern="target/rolling1/test1-$${date:MM-dd-yyyy}.%i.log.gz">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] %m%n</pattern>
  </PatternLayout>
  <SizeBasedTriggeringPolicy size="500" />
</RollingFile>

Docker Lookup

The DockerLookup can be used to lookup attributes from the Docker container the application is running in.

Log4j Docker provides access to the following container attributes:

Key Description

containerId

The full id assigned to the container.

containerName

The name assigned to the container.

imageId

The id assigned to the image.

imageName

The name assigned to the image.

shortContainerId

The first 12 characters of the container id.

shortImageId

The first 12 characters of the image id.

<JsonTemplateLayout>
  <EventTemplateAdditionalField key="containerId" value="${docker:containerId}"/>
  <EventTemplateAdditionalField key="containerName" value="${docker:containerName}"/>
  <EventTemplateAdditionalField key="imageName" value="${docker:imageName}"/>
</JsonTemplateLayout>

This Lookup is subject to the requirements listed at Log4j Docker Support

Environment Lookup

The EnvironmentLookup allows systems to configure environment variables, either in global files such as /etc/profile or in the startup scripts for applications, and then retrieve those variables from within the logging configuration. The example below includes the name of the currently logged in user in the application log.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${env:USER} %m%n</pattern>
  </PatternLayout>
</File>

This lookup also supports default value syntax. In the sample below, when the USER environment variable is undefined, the default value jdoe is used:

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${env:USER:-jdoe} %m%n</pattern>
  </PatternLayout>
</File>

Event Lookup

The EventLookup provides access to fields within the log event from the configuration.

Key Description

Exception

Returns the simple class name of the Exception, if one is included in the event.

Level

Returns the logging Level of the event.

Logger

Returns the name of the Logger.

Marker

Returns the name of the Marker associated with the log event, if one is present.

Message

Returns the formatted Message string.

ThreadId

Returns the thread id associated with the log event.

ThreadName

Returns the name of the thread associate with the log event.

Timestamp

Returns the time in milliseconds when the event occurred.

In this example the RoutingAppender picks a route based on the presence of a Marker named "AUDIT" being present in the log event.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" name="RoutingTest">
  <Appenders>
    <Console name="STDOUT" target="SYSTEM_OUT" />
    <Flume name="AuditLogger" compress="true">
      <Agent host="192.168.10.101" port="8800"/>
      <Agent host="192.168.10.102" port="8800"/>
      <RFC5424Layout enterpriseNumber="18060" includeMDC="true" appName="MyApp"/>
    </Flume>
    <Routing name="Routing">
      <Routes>
        <Route pattern="$${event:Marker}">
          <RollingFile
              name="Rolling-${mdc:UserId}"
              fileName="${mdc:UserId}.log"
              filePattern="${mdc:UserId}.%i.log.gz">
            <PatternLayout>
              <pattern>%d %p %c{1.} [%t] %m%n</pattern>
            </PatternLayout>
            <SizeBasedTriggeringPolicy size="500" />
          </RollingFile>
        </Route>
        <Route ref="AuditLogger" key="AUDIT"/>
        <Route ref="STDOUT" key="STDOUT"/>
      </Routes>
      <IdlePurgePolicy timeToLive="15" timeUnit="minutes"/>
    </Routing>
  </Appenders>
  <Loggers>
    <Root level="error">
      <AppenderRef ref="Routing" />
    </Root>
  </Loggers>
</Configuration>

Java Lookup

The JavaLookup allows Java environment information to be retrieved in convenient preformatted strings using the java: prefix.

Key Description

version

The short Java version, like:

Java version 1.7.0_67

runtime

The Java runtime version, like:

Java™ SE Runtime Environment (build 1.7.0_67-b01) from Oracle Corporation

vm

The Java VM version, like:

Java HotSpot™ 64-Bit Server VM (build 24.65-b04, mixed mode)

os

The OS version, like:

Windows 7 6.1 Service Pack 1, architecture: amd64-64

locale

Hardware information, like:

default locale: en_US, platform encoding: Cp1252

hw

System locale and file encoding information, like:

processors: 4, architecture: amd64-64, instruction sets: amd64

For example:

<File name="Application" fileName="application.log">
  <PatternLayout header="${java:runtime} - ${java:vm} - ${java:os}">
    <Pattern>%d %m%n</Pattern>
  </PatternLayout>
</File>

JNDI Lookup

As of Log4j 2.15.1 JNDI operations require that log4j2.enableJndi=true be set as a system property or the corresponding environment variable for this lookup to function. See the log4j2.enableJndi system property.

The JndiLookup allows variables to be retrieved via JNDI. By default the key will be prefixed with java:comp/env/, however if the key contains a ":" no prefix will be added.

By default the JNDI Lookup only supports the java, ldap, and ldaps protocols or no protocol. Additional protocols may be supported by specifying them on the log4j2.allowedJndiProtocols property. When using LDAP Java classes that implement the Referenceable interface are not supported for security reasons. Only the Java primative classes are supported by default as well as any classes specified by the log4j2.allowedLdapClasses property. When using LDAP only references to the local host name or ip address are supported along with any hosts or ip addresses listed in the log4j2.allowedLdapHosts property.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${jndi:logging/context-name} %m%n</pattern>
  </PatternLayout>
</File>

Java’s JNDI module is not available on Android.

JVM Input Arguments Lookup (JMX)

Maps JVM input arguments — but not main arguments — using JMX to acquire the JVM arguments.

Use the prefix jvmrunargs to access JVM arguments.

Java’s JMX module is not available on Android or on Google App Engine.

Kubernetes Lookup

The KubernetesLookup can be used to lookup attributes from the Kubernetes environment for the container the application is running in.

Log4j Kubernetes provides access to the following container attributes:

Attribute

Description

accountName

The service account name

clusterName

The name of the cluster the application is deployed in

containerId

>The full id assigned to the container

containerName

The name assigned to the container

host

The name assigned to the host operating system

hostIp

The host’s ip address

imageId

The id assigned to the container image

imageName

The name assigned to the container image

labels

All labels formatted in a list

labels.app

The application name

labels.podTemplateHash

The pod’s template hash value

masterUrl

The URL used to access the API server

namespaceId

The id of the namespace the various kubernetes components are located within

namespaceName

The namespace the various kubernetes components are located within

podId

The pod’s ip number

podIp

The pod’s ip address

podName

The name of the pod

This Lookup is subject to the configuration requirements listed at Log4j Kubernetes Support

Log4j Configuration Location Lookup

Log4j configuration properties. The expressions ${log4j:configLocation} and ${log4j:configParentLocation} respectively provide the absolute path to the log4j configuration file and its parent folder.

The example below uses this lookup to place log files in a directory relative to the log4j configuration file.

<File name="Application" fileName="${log4j:configParentLocation}/logs/application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] %m%n</pattern>
  </PatternLayout>
</File>

Lower Lookup

The LowerLookup converts the passed in argument to lower case. Presumably the value will be the result of a nested lookup.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${lower:${spring:spring.application.name}} %m%n</pattern>
  </PatternLayout>
</File>

Main Arguments Lookup (Application)

This lookup requires that you manually provide the main arguments of the application to Log4j:

import org.apache.logging.log4j.core.lookup.MainMapLookup;

public static void main(String args[]) {
  MainMapLookup.setMainArguments(args);
  ...
}

If the main arguments have been set, this lookup allows applications to retrieve these main argument values from within the logging configuration. The key that follows the main: prefix can either be a 0-based index into the argument list, or a string, where ${main:myString} is substituted with the value that follows myString in the main argument list.

Note: Many applications use leading dashes to identify command arguments. Specifying ${main:--file} would result in the lookup failing because it would look for a variable named "main" with a default value of "-file". To avoid this the ":" separating the Lookup name from the key must be followed by a backslash as an escape character as in ${main:\--file}.

For example, suppose the static void main String[] arguments are:

--file foo.txt --verbose -x bar

Then the following substitutions are possible:

Expression Result

${main:0}

--file

${main:1}

foo.txt

${main:2}

--verbose

${main:3}

-x

${main:4}

bar

${main:\--file}

foo.txt

${main:\-x}

bar

${main:bar}

null

|${main:\--quiet:-true} |true

Example usage:

[source,xml] ---- <File name="Application" fileName="application.log"> <PatternLayout header="File: ${main:--file}"> <Pattern>%d %m%n</Pattern> </PatternLayout> </File> ----

[#MapLookup] == Map Lookup

The MapLookup serves several purposes.

1. Provide the base for Properties declared in the configuration file. 2. Retrieve values from MapMessages in LogEvents. 3. Retrieve values set with MapLookup.setMainArguments(String[)]

The first item simply means that the MapLookup is used to substitute properties that are defined in the configuration file. These variables are specified without a prefix - e.g. ${name}. The second usage allows a value from the current MapMessage, if one is part of the current log event, to be substituted. In the example below the RoutingAppender will use a different RollingFileAppender for each unique value of the key named "type" in the MapMessage. Note that when used this way a value for "type" should be declared in the properties declaration to provide a default value in case the message is not a MapMessage or the MapMessage does not contain the key. See the Property Substitution section of the Configuration page for information on how to set the default values.

[source,xml] ---- <Routing name="Routing"> <Routes pattern="{map:type}"> <Route> <RollingFile name="Rolling-${map:type}" fileName="${filename}" filePattern="target/rolling1/test1-${map:type}.%i.log.gz"> <PatternLayout> <pattern>%d %p %c{1.} [%t] %m%n</pattern> </PatternLayout> <SizeBasedTriggeringPolicy size="500" /> </RollingFile> </Route> </Routes> </Routing> ----

The marker lookup allows you to use markers in interesting configurations like a routing appender. Consider the following YAML configuration and code that logs to different files based on markers:

[source,yaml] ---- Configuration: status: debug

Appenders: Console: RandomAccessFile: - name: SQL_APPENDER fileName: logs/sql.log PatternLayout: Pattern: "%d{ISO8601_BASIC} %-5level %logger{1} %X %msg%n" - name: PAYLOAD_APPENDER fileName: logs/payload.log PatternLayout: Pattern: "%d{ISO8601_BASIC} %-5level %logger{1} %X %msg%n" - name: PERFORMANCE_APPENDER fileName: logs/performance.log PatternLayout: Pattern: "%d{ISO8601_BASIC} %-5level %logger{1} %X %msg%n"

Routing: name: ROUTING_APPENDER Routes: pattern: "{marker:}" Route: - key: PERFORMANCE ref: PERFORMANCE_APPENDER - key: PAYLOAD ref: PAYLOAD_APPENDER - key: SQL ref: SQL_APPENDER

Loggers: Root: level: trace AppenderRef: - ref: ROUTING_APPENDER ----

[source,java] ---- public static final Marker SQL = MarkerFactory.getMarker("SQL"); public static final Marker PAYLOAD = MarkerFactory.getMarker("PAYLOAD"); public static final Marker PERFORMANCE = MarkerFactory.getMarker("PERFORMANCE");

final Logger logger = LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME);

logger.info(SQL, "Message in Sql.log"); logger.info(PAYLOAD, "Message in Payload.log"); logger.info(PERFORMANCE, "Message in Performance.log"); ----

Note the key part of the configuration is pattern: "{marker:}"`. This will produce three log files, each with a log event for a specific marker. Log4j will route the log event with the `SQL` marker to `sql.log`, the log event with the `PAYLOAD` marker to `payload.log`, and so on.

You can use the notation `"${marker:name}"` and `"{marker:name}" to check for the existence of a marker where name is the marker name. If the marker exists, the expression returns the name, otherwise null.

[#StructuredDataLookup] == Structured Data Lookup

The StructuredDataLookup is very similar to the MapLookup in that it will retrieve values from StructuredDataMessages. In addition to the Map values it will also return the name portion of the id (not including the enterprise number) and the type field. The main difference between the example below and the example for MapMessage is that the "type" is an attribute of the StructuredDataMessage while "type" would have to be an item in the Map in a MapMessage.

[source,xml] ---- <Routing name="Routing"> <Routes pattern="{sd:type}"> <Route> <RollingFile name="Rolling-${sd:type}" fileName="${filename}" filePattern="target/rolling1/test1-${sd:type}.%i.log.gz"> <PatternLayout> <pattern>%d %p %c{1.} [%t] %m%n</pattern> </PatternLayout> <SizeBasedTriggeringPolicy size="500" /> </RollingFile> </Route> </Routes> </Routing> ----

[#SystemPropertiesLookup] == System Properties Lookup

As it is quite common to define values inside and outside the application by using System Properties, it is only natural that they should be accessible via a Lookup. As system properties are often defined outside the application it would be quite common to see something like:

[source,xml] ---- <Appenders> <File name="ApplicationLog" fileName="${sys:logPath}/app.log"/> </Appenders> ----

This lookup also supports default value syntax. In the sample below, when the `logPath` system property is undefined, the default value `/var/logs` is used:

[source,xml] ---- <Appenders> <File name="ApplicationLog" fileName="${sys:logPath:-/var/logs}/app.log"/> </Appenders> ----

[#UpperLookup] == Upper Lookup

The LowerLookup converts the passed in argument to upper case. Presumably the value will be the result of a nested lookup.

[source,xml] ---- <File name="Application" fileName="application.log"> <PatternLayout> <pattern>%d %p %c{1.} [%t] {upper:{${spring:spring.application.name}} %m%n</pattern> </PatternLayout> </File> ----

[#WebLookup] == Web Lookup

The WebLookup allows applications to retrieve variables that are associated with the ServletContext. In addition to being able to retrieve various fields in the ServletContext, WebLookup supports looking up values stored as attributes or configured as initialization parameters. The following table lists various keys that can be retrieved:

[cols="1m,4"]

|Key |Description

|attr.name |Returns the ServletContext attribute with the specified name

|request.attr.name |Returns the ServletRequest attribute with the specified name - requires Log4jServletFilter

|header.name |Returns the HttpServletRequest header with the specified name - requires Log4jServletFilter

|cookie.name |Returns the HttpServletRequest cookie with the specified name - requires Log4jServletFilter

|header.name |Returns the HttpServletRequest header with the specified name - requires Log4jServletFilter

|request.method |Returns the HttpServletRequest method - requires Log4jServletFilter

|request.uri |Returns the HttpServletRequest URI - requires Log4jServletFilter

|request.url |Returns the HttpServletRequest URL - requires Log4jServletFilter

|request.remoteAddress |Returns the HttpServletRequest remote address - requires Log4jServletFilter

|request.remoteHost |Returns the HttpServletRequest remote host - requires Log4jServletFilter

|request.parameter.name |Returns the HttpServletRequest parameter - requires Log4jServletFilter

|request.principal |Returns the HttpServletRequest principal name - requires Log4jServletFilter

|session.id |Returns the HttpSession id or null if none is started - requires Log4jServletFilter

|session.attr.name |Returns the HttpSession attribute value (using toString() if not null) or null if absent - requires Log4jServletFilter

|contextPath |The context path of the web application

|contextPathName |The first token in the context path of the web application splitting on "/" characters.

|effectiveMajorVersion |Gets the major version of the Servlet specification that the application represented by this ServletContext is based on.

|effectiveMinorVersion |Gets the minor version of the Servlet specification that the application represented by this ServletContext is based on.

|initParam.name |Returns the ServletContext initialization parameter with the specified name

|majorVersion |Returns the major version of the Servlet API that this servlet container supports.

|minorVersion |Returns the minor version of the Servlet API that this servlet container supports.

|rootDir |Returns the result of calling getRealPath with a value of "/".

|serverInfo |Returns the name and version of the servlet container on which the servlet is running.

|servletContextName |Returns the name of the web application as defined in the display-name element of the deployment descriptor

Any other key names specified will first be checked to see if a ServletContext attribute exists with that name and then will be checked to see if an initialization parameter of that name exists. If the key is located then the corresponding value will be returned.

[source,xml] ---- <Appenders> <File name="ApplicationLog" fileName="${web:rootDir}/app.log"/> </Appenders> ----

=== Request lookups and asynchronous calls

Servlet 3.0 supports asynchronous calls, by default the request tracking - and therefore request related lookups, will not work. To make it work you can extract the servlet context attribute log4j.requestExecutor which is a BiConsumer<ServletRequest, Runnable> and call it passing the correct request and task to execute synchronously. During this task execution the lookups will be set up properly:

[source,java] ---- @GET // example using JAX-RS asynchronous feature backed by servlet AsyncContext public void get(@Suspended AsyncResponse response, @Context ServletContext context, @Context ServletRequest request) { final BiConsumer<ServletRequest, Runnable> log4jWrapper = (BiConsumer<ServletRequest, Runnable>) context.getAttribute("log4j.requestExecutor"); myThreadPool.submit) → log4jWrapper.accept(request, () → response.resume(doInternalGet(); } ----