Elasticsearch, Logstash, and Kibana form the ELK Stack, an open-source suite for data management, search, and visualization. Elasticsearch serves as a distributed search and analytics engine, Logstash handles data collection and processing, and Kibana provides visualization tools. This guide covers installation and configuration on Ubuntu 20.04 LTS.

System Requiremants

• Operating System: Ubuntu 20.04 LTS
• Hardware: 8 CPU cores, 12 GB RAM, 500 GB storage

Install Java Java is required for running ELK components. Install OpenJDK 16:

sudo apt update
sudo apt install openjdk-16-jre-headless
java --version

Add ELK Repository Add the Elastic repository to install the latest versions:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo sh -c 'echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" > /etc/apt/sources.list.d/elastic-8.x.list'

Update Package List Refresh the package list to include the new repository:

sudo apt update

Install Elasticsearch Install Elasticsearch and set it to start on boot:

sudo apt install elasticsearch
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

Generate a password for the default user 'elastic':

cd /usr/share/elasticsearch
sudo bin/elasticsearch-reset-password -u elastic

Backup the Elasticsearch configuration file:

sudo cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.backup

Create an enrollment token for Kibana integration:

cd /usr/share/elasticsearch
sudo bin/elasticsearch-create-enrollment-token --scope kibana

Install Kibana Install Kibana and manage its service:

sudo apt install kibana
sudo systemctl enable kibana
sudo systemctl start kibana

Generate a verification code for Kibana setup:

cd /usr/share/kibana
sudo bin/kibana-verification-code

Install Filebeat as a Data Collector While Logstash is part of ELK, Filebeat is a lightweight alternative for log collection. Install Filebeat:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.9.0-amd64.deb
sudo dpkg -i filebeat-8.9.0-amd64.deb
sudo systemctl start filebeat
sudo systemctl enable filebeat

Configure Elasticsearch Edit the Elasticsearch configuration file to set network settings:

sudo vi /etc/elasticsearch/elasticsearch.yml

Key settings to modify:

network.host: 127.0.0.1
http.port: 9200
cluster.initial_master_nodes: ["ubuntu"]
xpack.security.enabled: true
http.host: 0.0.0.0

Configure Kibana Adjust Kibana settings for network access and Elasticsearch connection:

sudo vi /etc/kibana/kibana.yml

Essential configurations:

server.host: "123.58.97.169"
elasticsearch.hosts: ['https://123.58.97.169:9200']
i18n.locale: "zh-CN"

Troubleshooting Common Issues

1. Dashboard panels not displaying: Ensure indices are configured correctly and run sudo filebeat setup to initialize dashboards.
2. Filebeat system module shows 'not connected': Verify the configuration in /etc/filebeat/modules.d/system.yml for correct file paths. Check service status and logs with sudo systemctl status filebeat.
3. Unable to delete indices in index management: Stop the data source service first (e.g., sudo systemctl stop filebeat), then delete the data stream in the index management interface.