Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Log4j 2 example #222

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Improve Log4j 2 example #222

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 35 additions & 34 deletions docs/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,31 +28,41 @@ Like stdin and file inputs, each event is assumed to be one line of text.
Can either accept connections from clients or connect to a server,
depending on `mode`.

===== Accepting log4j2 logs

Log4j2 can send JSON over a socket, and we can use that combined with our tcp
input to accept the logs.

First, we need to configure your application to send logs in JSON over a
socket. The following log4j2.xml accomplishes this task.

Note, you will want to change the `host` and `port` settings in this
configuration to match your needs.

<Configuration>
<Appenders>
<Socket name="Socket" host="localhost" port="12345">
<JsonLayout compact="true" eventEol="true" />
</Socket>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="Socket"/>
</Root>
</Loggers>
</Configuration>

To accept this in Logstash, you will want tcp input and a date filter:
===== Accepting Log4j 2 logs

Log4j 2 can write ECS-compliant JSON-formatted log events to a TCP socket.
We can combine with our TCP input to accept the logs from applications using Log4j 2.

First, we need to configure your application to write JSON-formatted logs to a TCP socket:

.Example `log4j2.xml` configuration for writing JSON-formatted logs to Logstash TCP input
[source,xml]
----
<Configuration xmlns="https://logging.apache.org/xml/ns"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
https://logging.apache.org/xml/ns
https://logging.apache.org/xml/ns/log4j-config-2.xsd">
<Appenders>
<Socket name="SOCKET" host="localhost" port="12345"> <!--1-->
<JsonTemplateLayout <!--2-->
eventTemplateUri="classpath:EcsLayout.json" <!--3-->
nullEventDelimiterEnabled="true"/> <!--4-->
</Socket>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="SOCKET"/>
</Root>
</Loggers>
</Configuration>
----
<1> Using Socket Appender to write logs to a TCP socket – make sure to *change the `host` attribute* to match your setup
<2> Using https://logging.apache.org/log4j/2.x/manual/json-template-layout.html[JSON Template Layout] to encode log events in JSON
<3> Using the ECS (Elastic Common Schema) layout bundled with JSON Template Layout
<4> Configuring that written log events should be terminated with a null (i.e., `\0`) character

To accept this in Logstash, you will want a TCP input:

input {
tcp {
Expand All @@ -61,15 +71,6 @@ To accept this in Logstash, you will want tcp input and a date filter:
}
}

and add a date filter to take log4j2's `timeMillis` field and use it as the
event timestamp

filter {
date {
match => [ "timeMillis", "UNIX_MS" ]
}
}

[id="plugins-{type}s-{plugin}-ecs_metadata"]
==== Event Metadata and the Elastic Common Schema (ECS)

Expand Down