Skip to content

Latest commit

 

History

History
155 lines (107 loc) · 4.93 KB

README.md

File metadata and controls

155 lines (107 loc) · 4.93 KB
Raw

Log Director

This software is free to use and is licensed under the Apache 2.0 License.

Features:

  • Received messages are written to stdout and/or forwarded to one or more remote logging destinations
  • Log messages are not stored, but can be piped from stdout to local files or through the systemd journal
  • Simple web interface where logs can be monitored live

Supported incoming message formats are:

  • Syslog RFC5424 - TCP and UDP
  • Syslog RFC3164 (BSD) - TCP and UDP
  • Graylog GELF - TCP and UDP (compressed & chunked)

Supported remote logging destinations are:

  • Syslog (RFC5424 over UDP)
  • Graylog (GELF over UDP)
  • Grafana Loki (HTTP over TCP).

architecture

Some of my other related projects are:

  • hmci for agent-less monitoring of IBM Power servers
  • svci for monitoring IBM Storage (Flashsystems / Storwize / SVC)
  • sysmon for monitoring directly from host with a small Java agent

Usage Instructions

  • Install the syslogd package (.deb or .rpm) from releases or build from source.
Usage: syslogd [-dhV] [--[no-]ansi] [--[no-]monitor] [--[no-]stdin] [--[no-]
               stdout] [--[no-]tcp] [--[no-]udp] [-f=<proto>]
               [--monitor-path=<path>] [--monitor-port=<num>] [-p=<num>]
               [--to-gelf=<uri>] [--to-loki=<url>] [--to-syslog=<uri>]
  -d, --debug                Enable debugging [default: false].
  -f, --format=<proto>       Input format: RFC3164, RFC5424, GELF [default:
                               RFC3164].
  -h, --help                 Show this help message and exit.
      --monitor-path=<path>  Monitor context path [default: /].
      --monitor-port=<num>   Monitor listening port [default: 8514].
      --[no-]ansi            Output in ANSI colors [default: true].
      --[no-]monitor         Start Monitor UI on port 8514 [default: true].
      --[no-]stdin           Forward messages from stdin [default: true].
      --[no-]stdout          Output messages to stdout [default: true].
      --[no-]tcp             Listen on TCP [default: true].
      --[no-]udp             Listen on UDP [default: true].
  -p, --port=<num>           Listening port [default: 1514].
      --to-gelf=<uri>        Forward to Graylog <udp://host:port>.
      --to-loki=<url>        Forward to Grafana Loki <http://host:port>.
      --to-syslog=<uri>      Forward to Syslog <udp://host:port> (RFC-5424).
  -V, --version              Print version information and exit.

The default syslog port (514) requires you to run syslogd as root / administrator. Any port number above 1024 does not require privileges and can be selected with the -p or --port option.

Examples

Listening on a non-standard syslog port:

java -jar /path/to/syslogd-x.y.z-all.jar --port 1514

or, if installed as a deb or rpm package:

/opt/syslogd/bin/syslogd --port 1514

Forwarding messages on to another log-system on a non-standard port.

/opt/syslogd/bin/syslogd --to-syslog udp://remotehost:1514

Forwarding messages to a Graylog server in GELF format.

/opt/syslogd/bin/syslogd --to-gelf udp://remotehost:12201

Forwarding to a Grafana Loki server.

/opt/syslogd/bin/syslogd --to-loki http://remotehost:3100

Receive log messages sent with the GELF protocol:

/opt/syslogd/bin/syslogd --port 12201 --format GELF

Receive log messages sent with the GELF protocol and forward to remote syslog server:

/opt/syslogd/bin/syslogd --port 12201 --format GELF --to-syslog udp://remotehost:1514

Started from a tmux session, listening for syslog messages and forwarding to a remote Graylog server:

tmux new-session -d -s "syslogd" "/opt/syslogd/bin/syslogd -p 514 --to-gelf=udp://remotehost:12201"

If you don't want any output locally (only forwarding), you can use the --no-stdout flag.

Notes

IBM AIX and VIO Servers

Syslog messages from AIX (and IBM Power Virtual I/O Servers) can be troublesome with some logging solutions. These can be received with syslogd and then forwarded on to your preferred logging solution.

Forwarding to Grafana Loki

Forwarding is currently done by making HTTP connections to the Loki API, which works fine for low volume messages, but might cause issues for large volume of messages.

Development

Test Grafana Loki

Run Loki and Grafana in local containers to test.

docker run --rm -d --name=loki -p 3100:3100 grafana/loki
docker run --rm -d --name=grafana --link loki:loki -p 3000:3000 grafana/grafana:7.1.3

Testing

while true; do sleep 10; logger -n localhost -P 1514 --rfc3164 test $(date); done