Lookups

Lookups provide a way to add values to the Log4j configuration at arbitrary places. They are a particular type of Plugin that implements the StrLookup interface. Information on how to use Lookups in configuration files can be found in the Property Substitution section of the Configuration page.

Context Map Lookup

The ContextMapLookup allows applications to store data in the Log4j ThreadContext Map and then retrieve the values in the Log4j configuration. In the example below, the application would store the current user's login id in the ThreadContext Map with the key "loginId". During initial configuration processing the first '$' will be removed. The PatternLayout supports interpolation with Lookups and will then resolve the variable for each event. Note that the pattern "%X{loginId}" would achieve the same result.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${ctx:loginId} %m%n</pattern>
  </PatternLayout>
</File>

Date Lookup

The DateLookup is somewhat unusual from the other lookups as it doesn't use the key to locate an item. Instead, the key can be used to specify a date format string that is valid for SimpleDateFormat. The current date, or the date associated with the current log event will be formatted as specified.

<RollingFile name="Rolling-${map:type}" fileName="${filename}" filePattern="target/rolling1/test1-$${date:MM-dd-yyyy}.%i.log.gz">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] %m%n</pattern>
  </PatternLayout>
  <SizeBasedTriggeringPolicy size="500" />
</RollingFile>

Docker Lookup

The DockerLookup can be used to lookup attributes from the Docker container the application is running in.

Log4j Docker provides access to the following container attributes:
containerId The full id assigned to the container.
containerName The name assigned to the container.
imageId The id assigned to the image.
imageName The name assigned to the image.
shortContainerId The first 12 characters of the container id.
shortImageId The first 12 characters of the image id.
<JsonLayout properties="true" compact="true" eventEol="true">
  <KeyValuePair key="containerId" value="${docker:containerId}"/>
  <KeyValuePair key="containerName" value="${docker:containerName}"/>
  <KeyValuePair key="imageName" value="${docker:imageName}"/>
</JsonLayout>

This Lookup is subject to the requirements listed at Log4j Docker Support

Environment Lookup

The EnvironmentLookup allows systems to configure environment variables, either in global files such as /etc/profile or in the startup scripts for applications, and then retrieve those variables from within the logging configuration. The example below includes the name of the currently logged in user in the application log.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${env:USER} %m%n</pattern>
  </PatternLayout>
</File>

This lookup also supports default value syntax. In the sample below, when the USER environment variable is undefined, the default value jdoe is used:

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${env:USER:-jdoe} %m%n</pattern>
  </PatternLayout>
</File>

EventLookup

The EventLookup provides access to fields within the log event from the configuration.

Key Description
Exception Returns the simple class name of the Exception, if one is included in the event.
Level Returns the logging Level of the event.
Logger Returns the name of the Logger.
Marker Returns the name of the Marker associated with the log event, if one is present.
Message Returns the formatted Message string.
ThreadId Returns the thread id associated with the log event.
ThreadName Returns the name of the thread associate with the log event.
Timestamp Returns the time in milliseconds when the event occurred.

In this example the RoutingAppender picks a route based on the presence of a Marker named "AUDIT" being present in the log event.

          <?xml version="1.0" encoding="UTF-8"?>
          <Configuration status="WARN" name="RoutingTest">
            <Appenders>
              <Console name="STDOUT" target="SYSTEM_OUT" />
              <Flume name="AuditLogger" compress="true">
                <Agent host="192.168.10.101" port="8800"/>
                <Agent host="192.168.10.102" port="8800"/>
                <RFC5424Layout enterpriseNumber="18060" includeMDC="true" appName="MyApp"/>
              </Flume>
              <Routing name="Routing">
                <Routes>
                  <Route pattern="$${event:Marker}">
                    <RollingFile
                        name="Rolling-${mdc:UserId}"
                        fileName="${mdc:UserId}.log"
                        filePattern="${mdc:UserId}.%i.log.gz">
                      <PatternLayout>
                        <pattern>%d %p %c{1.} [%t] %m%n</pattern>
                      </PatternLayout>
                      <SizeBasedTriggeringPolicy size="500" />
                    </RollingFile>
                  </Route>
                  <Route ref="AuditLogger" key="AUDIT"/>
                  <Route ref="STDOUT" key="STDOUT"/>
                </Routes>
                <IdlePurgePolicy timeToLive="15" timeUnit="minutes"/>
              </Routing>
            </Appenders>
            <Loggers>
              <Root level="error">
                <AppenderRef ref="Routing" />
              </Root>
            </Loggers>
          </Configuration>

Java Lookup

The JavaLookup allows Java environment information to be retrieved in convenient preformatted strings using the java: prefix.

Key Description
version

The short Java version, like:

Java version 1.7.0_67

runtime

The Java runtime version, like:

Java(TM) SE Runtime Environment (build 1.7.0_67-b01) from Oracle Corporation

vm

The Java VM version, like:

Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

os

The OS version, like:

Windows 7 6.1 Service Pack 1, architecture: amd64-64

locale

Hardware information, like:

default locale: en_US, platform encoding: Cp1252

hw

Hardware information, like:

processors: 4, architecture: amd64-64, instruction sets: amd64

For example:

<File name="Application" fileName="application.log">
  <PatternLayout header="${java:runtime} - ${java:vm} - ${java:os}">
    <Pattern>%d %m%n</Pattern>
  </PatternLayout>
</File>

Jndi Lookup

As of Log4j 2.17.0 JNDI operations require that log4j2.enableJndiLookup=true be set as a system property or the corresponding environment variable for this lookup to function. See the enableJndiLookup system property.

The JndiLookup allows variables to be retrieved via JNDI. By default the key will be prefixed with java:comp/env/, however if the key contains a ":" no prefix will be added.

The JDNI Lookup only supports the java protocol or no protocol (as shown in the example below).

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${jndi:logging/context-name} %m%n</pattern>
  </PatternLayout>
</File>

Java's JNDI module is not available on Android.

JVM Input Arguments Lookup (JMX)

Maps JVM input arguments -- but not main arguments -- using JMX to acquire the JVM arguments.

Use the prefix jvmrunargs to access JVM arguments.

See the Javadocs for java.lang.management.RuntimeMXBean.getInputArguments() .

Java's JMX module is not available on Android or on Google App Engine.

Kubernetes Lookup

The KubernetesLookup can be used to lookup attributes from the Kubernetes environment for the container the application is running in.

Log4j Kubernetes provides access to the following container attributes:
accountName The service account name
clusterName The name of the cluster the application is deployed in
containerId The full id assigned to the container
containerName The name assigned to the container
host The name assigned to the host operating system
hostIp The host's ip address
imageId The id assigned to the container image
imageName The name assigned to the container image
labels All labels formatted in a list
labesl.app The application name
labels.podTemplateHash The pod's template hash value
masterUrl The URL used to access the API server
namespaceId The id of the namespace the various kubernetes components are located within
namespaceName The namespace the various kubernetes components are located within
podId The pod's ip number
podIp The pod's ip address
podName The name of the pod
      <GelfLayout includeStackTrace="true" host="${hostName}" includeThreadContext="true" includeNullDelimiter="true"
                  compressionType="OFF">
        <ThreadContextIncludes>requestId,sessionId,loginId,userId,ipAddress,callingHost</ThreadContextIncludes>
        <MessagePattern>%d [%t] %-5p %X{requestId, sessionId, loginId, userId, ipAddress} %C{1.}.%M:%L - %m%n</MessagePattern>
        <KeyValuePair key="docker.containerId" value="${docker:containerId:-}"/>
        <KeyValuePair key="application" value="$${lower:${spring:spring.application.name}}"/>
        <KeyValuePair key="kubernetes.serviceAccountName" value="${k8s:accountName:-}"/>
        <KeyValuePair key="kubernetes.clusterName" value="${k8s:clusterName:-}/>
        <KeyValuePair key="kubernetes.containerId" value="${k8s:containerId:-}"/>
        <KeyValuePair key="kubernetes.containerName" value="${k8s:containerName:-}"/>
        <KeyValuePair key="kubernetes.host" value="${k8s:host:-}"/>
        <KeyValuePair key="kubernetes.labels.app" value="${k8s:labels.app:-}"/>
        <KeyValuePair key="kubernetes.labels.pod-template-hash" value="${k8s:labels.podTemplateHash:-}"/>
        <KeyValuePair key="kubernetes.master_url" value="${k8s:masterUrl:-}"/>
        <KeyValuePair key="kubernetes.namespaceId" value="${k8s:namespaceId:-}"/>
        <KeyValuePair key="kubernetes.namespaceName" value="${k8s:namespaceName:-}"/>
        <KeyValuePair key="kubernetes.podID" value="${k8s:podId:-}"/>
        <KeyValuePair key="kubernetes.podIP" value="${k8s:podIp:-}"/>
        <KeyValuePair key="kubernetes.podName" value="${k8s:podName:-}"/>
        <KeyValuePair key="kubernetes.imageId" value="${k8s:imageId:-}"/>
        <KeyValuePair key="kubernetes.imageName" value="${k8s:imageName:-}"/>
      </GelfLayout>

This Lookup is subject to the configuration requirements listed at Log4j Kubernetes Support

Log4j Configuration Location Lookup

Log4j configuration properties. The expressions ${log4j:configLocation} and ${log4j:configParentLocation} respectively provide the absolute path to the log4j configuration file and its parent folder.

The example below uses this lookup to place log files in a directory relative to the log4j configuration file.

<File name="Application" fileName="${log4j:configParentLocation}/logs/application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] %m%n</pattern>
  </PatternLayout>
</File>

Lower Lookup

The LowerLookup converts the passed in argument to lower case. Presumably the value will be the result of a nested lookup.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${lower:{${spring:spring.application.name}} %m%n</pattern>
  </PatternLayout>
</File>

Main Arguments Lookup (Application)

This lookup requires that you manually provide the main arguments of the application to Log4j:

import org.apache.logging.log4j.core.lookup.MainMapLookup;

public static void main(String args[]) {
  MainMapLookup.setMainArguments(args);
  ...
}

If the main arguments have been set, this lookup allows applications to retrieve these main argument values from within the logging configuration. The key that follows the main: prefix can either be a 0-based index into the argument list, or a string, where ${main:myString} is substituted with the value that follows myString in the main argument list.

Note: Many applications use leading dashes to identify command arguments. Specifying ${main:--file} would result in the lookup failing because it would look for a variable named "main" with a default value of "-file". To avoid this the ":" separating the Lookup name from the key must be followed by a backslash as an escape character as in ${main:\--file}

For example, suppose the static void main String[] arguments are:

--file foo.txt --verbose -x bar

Then the following substitutions are possible:

Expression Result
${main:0}

--file

${main:1}

foo.txt

${main:2}

--verbose

${main:3}

-x

${main:4}

bar

${main:\--file}

foo.txt

${main:\-x}

bar

${main:bar}

null

${main:\--quiet:-true}

true

Example usage:

<File name="Application" fileName="application.log">
  <PatternLayout header="File: ${main:--file}">
    <Pattern>%d %m%n</Pattern>
  </PatternLayout>
</File>

Map Lookup

The MapLookup serves several purposes.

  1. Provide the base for Properties declared in the configuration file.
  2. Retrieve values from MapMessages in LogEvents.
  3. Retrieve values set with MapLookup.setMainArguments(String[])

The first item simply means that the MapLookup is used to substitute properties that are defined in the configuration file. These variables are specified without a prefix - e.g. ${name}. The second usage allows a value from the current MapMessage, if one is part of the current log event, to be substituted. In the example below the RoutingAppender will use a different RollingFileAppender for each unique value of the key named "type" in the MapMessage. Note that when used this way a value for "type" should be declared in the properties declaration to provide a default value in case the message is not a MapMessage or the MapMessage does not contain the key. See the Property Substitution section of the Configuration page for information on how to set the default values.

<Routing name="Routing">
  <Routes pattern="$${map:type}">
    <Route>
      <RollingFile name="Rolling-${map:type}" fileName="${filename}"
                   filePattern="target/rolling1/test1-${map:type}.%i.log.gz">
        <PatternLayout>
          <pattern>%d %p %c{1.} [%t] %m%n</pattern>
        </PatternLayout>
        <SizeBasedTriggeringPolicy size="500" />
      </RollingFile>
    </Route>
  </Routes>
</Routing>

Marker Lookup

The marker lookup allows you to use markers in interesting configurations like a routing appender. Consider the following YAML configuration and code that logs to different files based on markers:

Configuration:
  status: debug

  Appenders:
    Console:
    RandomAccessFile:
      - name: SQL_APPENDER
        fileName: logs/sql.log
        PatternLayout:
          Pattern: "%d{ISO8601_BASIC} %-5level %logger{1} %X %msg%n"
      - name: PAYLOAD_APPENDER
        fileName: logs/payload.log
        PatternLayout:
          Pattern: "%d{ISO8601_BASIC} %-5level %logger{1} %X %msg%n"
      - name: PERFORMANCE_APPENDER
        fileName: logs/performance.log
        PatternLayout:
          Pattern: "%d{ISO8601_BASIC} %-5level %logger{1} %X %msg%n"

    Routing:
      name: ROUTING_APPENDER
      Routes:
        pattern: "$${marker:}"
        Route:
        - key: PERFORMANCE
          ref: PERFORMANCE_APPENDER
        - key: PAYLOAD
          ref: PAYLOAD_APPENDER
        - key: SQL
          ref: SQL_APPENDER

  Loggers:
    Root:
      level: trace
      AppenderRef:
        - ref: ROUTING_APPENDER
public static final Marker SQL = MarkerFactory.getMarker("SQL");
public static final Marker PAYLOAD = MarkerFactory.getMarker("PAYLOAD");
public static final Marker PERFORMANCE = MarkerFactory.getMarker("PERFORMANCE");

final Logger logger = LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME);

logger.info(SQL, "Message in Sql.log");
logger.info(PAYLOAD, "Message in Payload.log");
logger.info(PERFORMANCE, "Message in Performance.log");

Note the key part of the configuration is pattern: "$${marker:}". This will produce three log files, each with a log event for a specific marker. Log4j will route the log event with the SQL marker to sql.log, the log event with the PAYLOAD marker to payload.log, and so on.

You can use the notation "${marker:name}" and "$${marker:name}" to check for the existence of a marker where name is the marker name. If the marker exists, the expression returns the name, otherwise null.

Spring Boot Lookup

The Spring Boot Lookup retrieves the values of Spring properties from the Spring configuration as well as values of the active and default profiles. Specifying a key of "profiles.active" will reutrn the active profiles while a key of "profiles.default" will return the default profiles. The default and active profiles can be an array. If more than one profile is present they will be returned as a comma separated list. To retrieve a single item from the array append "[{index}]" to the key. For example, to return the first active profile in the list specify "profiles.active[0]".

This Lookup will return null values until Spring Boot initializes application logging.

<File name="Application" fileName="application-${spring:profiles.active[0]}.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $${spring:spring.application.name} %m%n</pattern>
  </PatternLayout>
</File>

This Lookup requires log4j-spring-cloud-config-client be included in the application.

Structured Data Lookup

The StructuredDataLookup is very similar to the MapLookup in that it will retrieve values from StructuredDataMessages. In addition to the Map values it will also return the name portion of the id (not including the enterprise number) and the type field. The main difference between the example below and the example for MapMessage is that the "type" is an attribute of the StructuredDataMessage while "type" would have to be an item in the Map in a MapMessage.

<Routing name="Routing">
  <Routes pattern="$${sd:type}">
    <Route>
      <RollingFile name="Rolling-${sd:type}" fileName="${filename}"
                   filePattern="target/rolling1/test1-${sd:type}.%i.log.gz">
        <PatternLayout>
          <pattern>%d %p %c{1.} [%t] %m%n</pattern>
        </PatternLayout>
        <SizeBasedTriggeringPolicy size="500" />
      </RollingFile>
    </Route>
  </Routes>
</Routing>

System Properties Lookup

As it is quite common to define values inside and outside the application by using System Properties, it is only natural that they should be accessible via a Lookup. As system properties are often defined outside the application it would be quite common to see something like:

<Appenders>
  <File name="ApplicationLog" fileName="${sys:logPath}/app.log"/>
</Appenders>

This lookup also supports default value syntax. In the sample below, when the logPath system property is undefined, the default value /var/logs is used:

<Appenders>
  <File name="ApplicationLog" fileName="${sys:logPath:-/var/logs}/app.log"/>
</Appenders>

Upper Lookup

The UpperLookup converts the passed in argument to upper case. Presumably the value will be the result of a nested lookup.

<File name="Application" fileName="application.log">
  <PatternLayout>
    <pattern>%d %p %c{1.} [%t] $$upper{${spring:spring.application.name}} %m%n</pattern>
  </PatternLayout>
</File>

Web Lookup

The WebLookup allows applications to retrieve variables that are associated with the ServletContext. In addition to being able to retrieve various fields in the ServletContext, WebLookup supports looking up values stored as attributes or configured as initialization parameters. The following table lists various keys that can be retrieved:

Key Description
attr.name Returns the ServletContext attribute with the specified name
contextPath The context path of the web application
contextPathName The first token in the context path of the web application splitting on "/" characters.
effectiveMajorVersion Gets the major version of the Servlet specification that the application represented by this ServletContext is based on.
effectiveMinorVersion Gets the minor version of the Servlet specification that the application represented by this ServletContext is based on.
initParam.name Returns the ServletContext initialization parameter with the specified name
majorVersion Returns the major version of the Servlet API that this servlet container supports.
minorVersion Returns the minor version of the Servlet API that this servlet container supports.
rootDir Returns the result of calling getRealPath with a value of "/".
serverInfo Returns the name and version of the servlet container on which the servlet is running.
servletContextName Returns the name of the web application as defined in the display-name element of the deployment descriptor

Any other key names specified will first be checked to see if a ServletContext attribute exists with that name and then will be checked to see if an initialization parameter of that name exists. If the key is located then the corresponding value will be returned.

<Appenders>
  <File name="ApplicationLog" fileName="${web:rootDir}/app.log"/>
</Appenders>