Access logging

Configuration

Access logs are configured as part of the HTTP connection manager config, TCP Proxy, UDP Proxy or Thrift Proxy.

Format Rules

Access log formats contain command operators that extract the relevant data and insert it. They support two formats: “format strings” and “format dictionaries”. In both cases, the command operators are used to extract the relevant data, which is then inserted into the specified log format. Only one access log format may be specified at a time.

Format Strings

Format strings are plain strings, specified using the format key. They may contain either command operators or other characters interpreted as a plain string. The access log formatter does not make any assumptions about a new line separator, so one has to be specified as part of the format string. See the default format for an example.

Default Format String

If a custom format string is not specified, Envoy uses the following default format:

[%START_TIME%] "%REQUEST_HEADER(:METHOD)% %REQUEST_HEADER(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%"
%RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION%
%RESPONSE_HEADER(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQUEST_HEADER(X-FORWARDED-FOR)%" "%REQUEST_HEADER(USER-AGENT)%"
"%REQUEST_HEADER(X-REQUEST-ID)%" "%REQUEST_HEADER(:AUTHORITY)%" "%UPSTREAM_HOST%"

Example of the default Envoy access log format:

[2016-04-15T20:17:00.310Z] "POST /api/v1/locations HTTP/2" 204 - 154 0 226 100 "10.0.35.28"
"nsq2http" "cc21d9b0-cf5c-432b-8c7e-98aeb7988cd2" "locations" "tcp://10.0.2.1:80"

Format Dictionaries

Format dictionaries are dictionaries that specify a structured access log output format, specified using the json_format or typed_json_format keys. This allows logs to be output in a structured format such as JSON. Similar to format strings, command operators are evaluated and their values inserted into the format dictionary to construct the log output.

For example, the following Envoy configuration snippet shows how to configure json_format:

json-format-config.yaml
 1          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
 2          stat_prefix: ingress_http
 3          access_log:
 4          - name: envoy.access_loggers.stdout
 5            typed_config:
 6              "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
 7              log_format:
 8                json_format:
 9                  protocol: "%PROTOCOL%"
10                  duration: "%DURATION%"
11                  my_custom_header: "%REQUEST_HEADER(MY_CUSTOM_HEADER)%"
12          route_config:
13            name: local_route

The following JSON object would be written to the log file:

{"protocol": "HTTP/1.1", "duration": "123", "my_custom_header": "value_of_MY_CUSTOM_HEADER"}

This allows you to specify a custom key for each command operator.

The typed_json_format differs from json_format in that values are rendered as JSON numbers, booleans, and nested objects or lists where applicable. In the example, the request duration would be rendered as the number 123.

Format dictionaries have the following restrictions:

  • The dictionary must map strings to strings (specifically, strings to command operators). Nesting is supported.

  • When using the typed_json_format command operators will only produce typed output if the command operator is the only string that appears in the dictionary value. For example, "%DURATION%" will log a numeric duration value, but "%DURATION%.0" will log a string value.

Note

When using the typed_json_format, integer values that exceed \(2^{53}\) will be represented with reduced precision as they must be converted to floating point numbers.

Command Operators

Command operators are used to extract values that will be inserted into the access logs. The same operators are used by different types of access logs (such as HTTP and TCP). Some fields may have slightly different meanings, depending on what type of log it is. Differences are noted in the descriptions.

Note

If a value is not set/empty, the logs will contain a - character or, for JSON logs, the string "-". For typed JSON logs unset values are represented as null values and empty strings are rendered as "". The omit_empty_values option could be used to omit empty values entirely.

Unless otherwise noted, command operators produce string outputs for typed JSON logs.

See all the available command operators in the substitution formatter documentation.