Skip to main content

Posts

Showing posts with the label Filebeat

Running data analytics on application events and logs using Elasticsearch, Logstash and Kibana

In this post, we will learn how to use Elasticsearch, Logstash and Kibana for running analytics on application events and logs. Firstly, I will install all these applications on my local machine. Installations You can read my previous posts on how to install Elasticsearch , Logstash , Kibana and Filebeat on your local machine. Basic configuration I hope by now you are have installed Elasticsearch, Logstash, Kibana and Filebeat on your system. Now, Let's do few basic configurations required to be able to run analytics on application events and logs. Elasticsearch Open elasticsearch.yml file in [ELASTICSEARCH_INSTLLATION_DIR]/config folder and add properties to it. cluster.name: gauravbytes-event-analyzer node.name: node-1 Cluster name is used by Elasticsearch node to form a cluster. Node name within cluster need to be unique. We are running only single instance of Elasticsearch on our local machine. But, in production grade setup there will be master nodes, data nodes a

Installing Filebeat

Filebeat Filebeat is a light-weight log shipper. It is installed as a agent and listen to your predefined set of log files and locations and forward them to your choice of sink (Logstash, Elasticsearch, database etc.) Installation deb curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-amd64.deb sudo dpkg -i filebeat-6.3.2-amd64.deb rpm curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-x86_64.rpm sudo rpm -vi filebeat-6.3.2-x86_64.rpm mac curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-darwin-x86_64.tar.gz tar xzvf filebeat-6.3.2-darwin-x86_64.tar.gz docker docker pull docker.elastic.co/beats/filebeat:6.3.2 Windows Download the filebeat from official website and do the following configurations. 1) Extract the zip file to your choice of location. e.g. C:\Program Files. 2) Rename the filebeat- -windows directory to Filebeat . 3) Open a PowerShell prompt as an Administrator (right