Updating my ELK stack with GELF appender


A couple of days ago I wrote about setting up an ELK stack with docker-compose. I did some small changes to the set up, so I thought it’s worth an update.

First change is in the docker-compose.yml to enable logstashs gelf input, move the Log4j socket to port 12202 and add an UDP port forwarding for docker on port 12201:

elasticsearch:
  image: elasticsearch
  ports:
  - 9200:9200
logstash:
  image: logstash:latest
  links:
  - elasticsearch:elasticsearch
  ports:
  - 12201:12201/udp
  - 12202:12202
  command: logstash agent -e 'input { gelf { port => "12201" } log4j { mode => "server" port => "12202"} } output { elasticsearch { hosts => ["elasticsearch"] } }'
kibana:
  image: kibana
  links:
  - elasticsearch:elasticsearch
  ports:
  - 5601:5601
  environment:
  - ELASTICSEARCH_URL=http://elasticsearch:9200

This way, logstash will listen to ports 12201 with the gelf input and port 12202 with the Log4j socket input.

Now I modified the log4j.properties for my application and replaced the Log4j SocketAppender with a biz.paluch.logging.gelf.log4j.GelfLogAppender:

log4j.appender.gelf=biz.paluch.logging.gelf.log4j.GelfLogAppender
log4j.appender.gelf.Threshold=INFO
log4j.appender.gelf.Host=udp:127.0.0.1
log4j.appender.gelf.Port=12201
#log4j.appender.gelf.Version=1.1
#log4j.appender.gelf.Facility=java-test
log4j.appender.gelf.ExtractStackTrace=true
log4j.appender.gelf.FilterStackTrace=true
log4j.appender.gelf.MdcProfiling=true
log4j.appender.gelf.TimestampPattern=yyyy-MM-dd HH:mm:ss,SSSS
log4j.appender.gelf.MaximumMessageSize=8192

# This are static fields
log4j.appender.gelf.AdditionalFields=environment=local

The gelf appender is much more configurable and the search abilities in logstash are now a bit more useful to me. But I have to add another dependency to my application now:

<dependency>
    <groupId>biz.paluch.logging</groupId>
    <artifactId>logstash-gelf</artifactId>
    <version>1.8.0</version>
</dependency>

You can find the documentation at https://github.com/mp911de/logstash-gelf.

Weitere Artikel

Nach den Crossfit Open

Crossfit Open WOD 17.5

Crossfit Open WOD 17.4

Crossfit Open WOD 17.3

Crossfit Open Workout 17.2

Crossfit Open Workout 17.1

Run Feedbin in your local Kubernetes cluster

Running Threema Web in Docker

Neues Workout Video

Diät Update #1