In my last post ELK Stack Monitoring On Ubuntu 18.04 I attempted to get hands down on full ELK stack monitoring of your ubuntu machines. I ended up with a brief overview of Kibana, and how to install ElasticSearch and MetricBeats.
Today I want to make up for that and establish Logstash as part of the team.
First, let us bring the os up to date. The last update is a week old. Then, in the next step, install Logstash from the repositories that we already setup last time.
Logstash uses pipelines to receive, enrich, format, and send data to elasticsearch. So in order to have Logstash get to work, we have to setup such a pipeline. Let’s start with an easy one, to get an idea of it.
Let’s create a file called
/etc/logstash/conf.d/system-metrics-pipeline.conf. Put the following text in it:
Restart the service with
systemctl restart logstash.
Now head over to Kibana in your browser, reload the tab and start to panic. Your Dashboard should now not show any metrics anymore. To change this, we have to change MetricBeat, so it sends data to Logstash directly instead of ElasticSearch.
As already said, the default output of MetricBeat sends data to ElasticSearch. Change
/etc/metricbeat/metricbeat.yml like so:
systemctl restart metricbeat, refresh your browser and feel a little relief. Your metrics are working again.
Another thing that Logstash is really great with are filters. Filters let you take values from any field of your input stream, format them, change them.. think “verything” them, then put them back into the stream. I’ll cover that in a later post.
Send to Logstash from multiple hosts
In order to send data to Logstash from more than just “localhost”, we need to set it up accordingly. From my previous post just follow the steps to:
- install Java Runtime Environment
- add the ElasticSearch 6.x Repository
- install MetricBeat
- configure MetricBeat to send data to our ELK Server (make sure to enter the correct hostname or IP in
/etc/metricbeat/modules.d/system.ymland send it to the Logstash output, not the ElasticSearch output)
- restart MetricBeat
Once these steps are complete, data from your other hosts will start pouring into Elasticsearch accordingly.
[Metricbeat System] Overview dashboard should now show number of total hosts that are sending their system log to their master.
We do now know how to setup Logstash on our master server to receive data. Also we have MetricBeat sending data to the right output and can send data from other hosts as well.
From my first post in this series, there are still some things to do, and many more to explore.
Thank you for reading!
If you have anything to add or questions to ask, feel free to shoot me a message!