created: 20161105234744185
creator: user
modified: 20180830082851161
modifier: user
tags: docker logstash
title: ELK Stack
type: text/vnd.tiddlywiki

; The ELK stack powered by Docker and Compose.
: https://github.com/aikomastboom/docker-elk

; Docker apps logging with Filebeat and Logstash
: http://stackoverflow.com/questions/33432983/docker-apps-logging-with-filebeat-and-logstash

; Docker ELK stack
: https://github.com/ind3x/rpi-docker-elk

; Graylog Extended Log Format (GELF)
: http://docs.graylog.org/en/2.1/pages/gelf.html

; LogStash input plugins
: https://www.elastic.co/guide/en/logstash/current/input-plugins.html

; LogStash GELF input plugin
: https://github.com/logstash-plugins/logstash-input-gelf

; LogStash working w/ plugins
: https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html

; Docker GELF Logging Example
: https://github.com/francolaiuppa/docker-gelf-logging-example

; A connector that collects data using 'appmetrics' and sends it to ElasticSearch for use with Kibana
: https://github.com/RuntimeTools/appmetrics-elk


https://discuss.elastic.co/t/solved-weekly-indexes-instead-of-daily/56871



! cleanup
* https://hobo.house/2016/02/18/how-to-manually-clean-indexes-from-elasticsearch/
* https://www.elastic.co/guide/en/elasticsearch/client/curator/current/singleton-cli.html

```
virtualenv elk_cleanup
cd elk_cleanup
. bin/activate
pip install elasticsearch-curator
```

!!dev

```
curator_cli --host 192.168.0.238 show_indices --verbose

curator_cli --dry-run --timeout 600 --host 192.168.0.238 delete_indices --filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":31},{"filtertype":"pattern","kind":"prefix","value":"logstash"}]'
```

!!live

```
curator_cli --host 188.166.116.203 show_indices --verbose

curator_cli --dry-run --timeout 600 --host 188.166.116.203 delete_indices --filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":31},{"filtertype":"pattern","kind":"prefix","value":"logstash"}]'
```