Skip to main content

Posts

Showing posts with the label ELK stack

Installing Filebeat

Filebeat Filebeat is a light-weight log shipper. It is installed as a agent and listen to your predefined set of log files and locations and forward them to your choice of sink (Logstash, Elasticsearch, database etc.) Installation deb curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-amd64.deb sudo dpkg -i filebeat-6.3.2-amd64.deb rpm curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-x86_64.rpm sudo rpm -vi filebeat-6.3.2-x86_64.rpm mac curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-darwin-x86_64.tar.gz tar xzvf filebeat-6.3.2-darwin-x86_64.tar.gz docker docker pull docker.elastic.co/beats/filebeat:6.3.2 Windows Download the filebeat from official website and do the following configurations. 1) Extract the zip file to your choice of location. e.g. C:\Program Files. 2) Rename the filebeat- -windows directory to Filebeat . 3) Open a PowerShell prompt as an Administrator (right

Installing Kibana

Kibana Kibana is a visualization dashboard for Elasticsearch and you can choose many available charts like graphs, pie, bar, histogram etc. or real time textual data and can gain meaningful analytics. Installation Installating Kibana directly from tar files For Linux installation wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.3-linux-x86_64.tar.gz shasum -a 512 kibana-6.2.3-linux-x86_64.tar.gz tar -xzf kibana-6.2.3-linux-x86_64.tar.gz cd kibana-6.2.3-linux-x86_64/ For Windows installation //Dowload Kibana https://artifacts.elastic.co/downloads/kibana/kibana-6.2.3-windows-x86_64.zip //running kibana /bin/kibana.bat Installation from packages Debian package installation // Import elatic PGP key wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - //install https transport module sudo apt-get install apt-transport-https //save repository definition echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" |

Installing Logstash

Logstash Logstash is data processing pipeline which ingests the data simultaneously from multiple data sources, transform it and send it to different `stash` i.e. Elasticsearch, Redis, database, rest endpoint etc. For example; Ingesting logs files; cleaning and transforming it to machine and human readable formats. There are three components in Logstash i.e. Inputs, Filters and Outputs Inputs It ingests data of any kind, shape and size. For examples: Logs, AWS metrics, Instance health metrics etc. Filters Logstash filters parse each event, build a structure, enrich the data in event and also transform it to desired form. For example: Enriching geo-location from IP using GEO-IP filter, Anonymize PII information from events, transforming unstructured data to structural data using GROK filters etc. Outputs This is the sink layer. There are many output plugins i.e. Elasticsearch, Email, Slack, Datadog, Database persistence etc. Installing Logstash As of writing Logstash(6.2.3) r