Example: Loki

Loki Logging

paStash can be used to easily ship logs to Grafana Loki or cLoki

Setup

Install paStash next-gen and the Loki output plugin globally using npm:

# npm install -g @pastash/pastash @pastash/output_loki

Recipe

In this super simple recipe we'll use the file input and loki output, with an optional wss socket for live logs:

input {
  file {
    path => "/var/log/*.log"
  }
}

filter {
  compute_field {
    field => type
    value => "my_logs"
  }
}

output {
  loki {
    host => localhost
    port => 3100
    path => "/loki/api/v1/push"
  }
}

Grafana Hosted

When using the Grafana Hosted Loki, use this format as template:

input {
  file {
    path => "/var/log/*.log"
  }
}

output {
  loki {
    basic_auth_password => "some_very_secure_password_hash_here"
    basic_auth_user => "1234"
    host => "logs-us-west1.grafana.net"
    port => 80
    path => "/api/prom/push"
    partition_id => "ABC123"
  }
}

Real-Time Logs via Secure WebSockets for streaming clients

input {
  ws {
    host => 0.0.0.0
    port => 5654
    ssl => true
    ssl_key => /etc/letsencrypt/live/host.domain.ext/privkey.pem
    ssl_cert => /etc/letsencrypt/live/host.domain.ext/fullchain.pem
  }
}

output {
  loki {
    host => localhost
    port => 3100
    path => "/loki/api/v1/push"
  }
}

Scraping Prometheus endpoints to cLoki metrics

input {
  prom_scraper {
    url => "http://prometheus.app/metrics"
    interval => 5000
    flat => true
  }
}

filter {
  compute_field {
    field => type
    value => "prometheus"
  }
}

output {
  stdout {}
  loki {
    host => localhost
    port => 3100
    path => "/loki/api/v1/push"
  }
}

Once ready, save to your preferred location (ie: /etc/pastash_loki.json)

Usage

./bin/pastash --config_file=/etc/pastash_loki.json

Docker

For those using Docker there's a ready to go container for Loki output

pastash:
    image: qxip/pastash-loki
    container_name: pastash
    volumes:
      - ./conf/pastash_loki.json:/pastash.conf
      - /var/log:/var/log:ro