How to set up Elastic Stack on Debian 11 Bullseye
Tags: elastic stack, elk stack, linux, monitoring, logging, debian, bullseye
Last update: Sept 2022
Testing with: Debian 11, Elastic Stack 8.4
Table of Contents
- Preparation
- Installation Elasticsearch
- Installation Kibana
- Configure Elasticsearch and Kibana
- Connect Kibana with Elasticsearch
- Securing Kibana
- Setting up Fleet
1. Preparation
This is my very personal configuration. All you really need is to have the sudo-package installed. Everything else is optional.
Run as root:
# updating repo
apt update
# installing my own packages, but sudo is mandatory
apt install sudo htop ncdu neofetch mc qemu-guest-agent
# adding my user to the sudo-group
adduser user sudoRun as user:
# oneliner for setting up 4 GB swapfile
sudo fallocate -l 4G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile && echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab2. Installation Elasticsearch
# installing packages
sudo apt update && sudo apt install apt-transport-https gnupg curl wget
# adding repo key
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
# adding repo
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
# install the package
sudo apt update && sudo apt install elasticsearchYou will get an output like:
-------------------------- Security autoconfiguration information ------------------------------
Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.
The generated password for the elastic built-in superuser is : z5I551q=+9G3qaDD3GnT
If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.
You can complete the following actions at any time:
Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.
Generate an enrollment token for Kibana instances with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.
Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.
-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
sudo systemctl start elasticsearch.service3. Installation Kibana
# install package
sudo apt install kibanaYou will get an output like:
Created Kibana keystore in /etc/kibana/kibana.keystore4. Configure Elasticsearch and Kibana
# open the elasticsearch config file
sudo nano /etc/elasticsearch/elasticsearch.yml
# change/add one value
network.host: SERVER_IP
# and restart the service
sudo systemctl start elasticsearch.service
# open the kibana config file
sudo nano /etc/kibana/kibana.yml
# change/add two values
server.host: "SERVER_IP“
server.publicBaseUrl: "http://SERVER_IP:5601"
# and restart the service
sudo systemctl start kibana.service5. Connect Kibana with Elasticsearch
# generate a token
sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibanaYou will get an token like:
eyJ2ZXIiOiI4LjQuMiIsImFkciI6WyIxOTIuMTY4LjU2LjE2MDo5MjAwIl0sImZnciI6IjFmZGE3MjgwM2QzYjI1Y2VkMmY3MmJmODQxYjc4Mjc5MWFjNjZkNDFjODIwMDJjZWEzYTEzMTIwZjBmOTQzOWYiLCJrZXkiOiJBbVY5ZjRNQkdJbmRxMDE5LUlfSjpDRmpVeDF5ZFNPV0dqeEtnTHlPUUd3In0=Point your browser to Kibana: http://SERVER_IP:5601. Paste the token in the browser and verify by using the code via command line:
# get verification code after providing the enrollment token
sudo /usr/share/kibana/bin/kibana-verification-code6. Securing Kibana
# get the import password which is needed for the next step
sudo /usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_passwordYou will get an import password like:
f-Ss0Db5RKhabsd4wKZ82# generate certificates and paste your import password
sudo openssl pkcs12 -in /etc/elasticsearch/certs/http.p12 -out /etc/kibana/server.crt -clcerts -nokeys
sudo openssl pkcs12 -in /etc/elasticsearch/certs/http.p12 -out /etc/kibana/server.key -nocerts -nodes
# change the permissions
sudo chown root:kibana /etc/kibana/server.crt
sudo chown root:kibana /etc/kibana/server.key
sudo chmod g+r /etc/kibana/server.crt
sudo chmod g+r /etc/kibana/server.key
# add more security to kibana
sudo nano /etc/kibana/kibana.yml
# add the following three values
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/server.crt
server.ssl.key: /etc/kibana/server.key
# generate security keys
sudo /usr/share/kibana/bin/kibana-encryption-keysYou will get an output like:
## Kibana Encryption Key Generation Utility
The 'generate' command guides you through the process of setting encryption keys for:
xpack.encryptedSavedObjects.encryptionKey
Used to encrypt stored objects such as dashboards and visualizations
https://www.elastic.co/guide/en/kibana/current/xpack-security-secure-saved-objects.html#xpack-security-secure-saved-objects
xpack.reporting.encryptionKey
Used to encrypt saved reports
https://www.elastic.co/guide/en/kibana/current/reporting-settings-kb.html#general-reporting-settings
xpack.security.encryptionKey
Used to encrypt session information
https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#security-session-and-cookie-settings
Already defined settings are ignored and can be regenerated using the --force flag. Check the documentation links for instructions on how to rotate encryption keys.
Definitions should be set in the kibana.yml used configure Kibana.
Settings:
xpack.encryptedSavedObjects.encryptionKey: c8136c292e1bb8c7ebcfab522ca8cf12
xpack.reporting.encryptionKey: 38718ec714520269b6b116ca9eb3055c
xpack.security.encryptionKey: c84879165b3180bfb9da4f8510779f0eCopy and the last the lines of the output and fill it into the kibana config file.
sudo nano /etc/kibana/kibana.yml
# fill in the encryption keys at the end as mentioned before
xpack.encryptedSavedObjects.encryptionKey: c8136c292e1bb8c7ebcfab522ca8cf12
xpack.reporting.encryptionKey: 38718ec714520269b6b116ca9eb3055c
xpack.security.encryptionKey: c84879165b3180bfb9da4f8510779f0eLogin with your browser at https://SERVER_IP:5601 with your username elastic and your superuser-password as provided in the second step. Change your password via editing the profile.
# enable autostart for both services and reboot to verify all is working
sudo systemctl enable elasticsearch.service
sudo systemctl enable kibana.service
sudo rebootAllow the system some minutes to start all services after reboot. Elasticstack should soon be reachable under https://SERVER_IP:9200 and Kibana under https://SERVER_IP:5601.
7. Setting up Fleet
In order to set up Fleet we need to install the elastic agent in fleet-server mode. Login and go to Management > Fleet > Agents > Add a fleet server. Use https://SERVER_IP:8220 and generate the Fleet Server Policy. Use Linux Tar.
# download the package
curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.4.2-linux-x86_64.tar.gz
# extract it
tar xzvf elastic-agent-8.4.2-linux-x86_64.tar.gz && cd elastic-agent-8.4.2-linux-x86_64
# extend the last command by adding the flag --fleet-server-es-ca
# otherwise the installation will fail with "Error: fleet-server failed: context canceled"
# your final installation command will look like
sudo ./elastic-agent install \
--fleet-server-es=https://SERVER_IP:9200 \
--fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NjQyOTI1NDUzMTM6WWpydUlkNW1Sb0dNUHFrN09oc2xUQQ \
--fleet-server-policy=fleet-server-policy \
--fleet-server-es-ca=/etc/elasticsearch/certs/http_ca.crtYou will get an output like:
Elastic Agent will be installed at /opt/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:
{"log.level":"info","@timestamp":"2022-09-27T17:43:52.544+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":403},"message":"Generating self-signed certificate for Fleet Server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-27T17:43:56.793+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":773},"message":"Fleet Server - Running on policy with Fleet Server integration: fleet-server-policy; missing config fleet.agent.id (expected during bootstrap process)","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-27T17:43:56.933+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":471},"message":"Starting enrollment to URL: https://fs-pve3-elastic:8220/","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-27T17:44:00.885+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":273},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.
Elastic Agent has been successfully installed.Et voila. We are ready to go.
8. Adding a client to Fleet Management
Now it’s time to add our first client to the system in order to be able to monitor it. For demonstration purposes we’ll connect a simple stock Debian machine.
- After having your client ready log in to Kibana and go to Management > Fleet > button Add Agent. You can leave the name “Agent policy 1”. Use the button to create it.
- Leave the default setting “enroll in fleet” untouched.
- Follow the Linux Tar code to install the agent on your client system and to enroll the client automatically:
# download agent
curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.4.2-linux-x86_64.tar.gz
# unpack
tar xzvf elastic-agent-8.4.2-linux-x86_64.tar.gz
# change into directory
cd elastic-agent-8.4.2-linux-x86_64
# important!!! the last install command has to be extended by the flag --insecure
# as we don't have any regular PKI in place.
sudo ./elastic-agent install --url=https://192.168.56.160:8220 --enrollment-token=cFZmSGhJTUI0LTBMVmd4M3cwcG46UzRpSjYtejhTV2FTVW1TMndEV1QtUQ== --insecureYou will get an output like:
Elastic Agent will be installed at /opt/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:
{"log.level":"warn","@timestamp":"2022-09-28T17:51:28.584+0200","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":104},"message":"SSL/TLS verifications disabled.","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-28T17:51:28.603+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":471},"message":"Starting enrollment to URL: https://192.168.56.160:8220/","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-09-28T17:51:28.946+0200","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":104},"message":"SSL/TLS verifications disabled.","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-28T17:51:29.806+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":273},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.
Elastic Agent has been successfully installed.Leave the Kibana tab open and wait until the waiting circle close to “confirm incoming data” vanishes. If nothing changes there are several places you can start troubleshooting:
# check the agent status on the client
sudo systemctl status elastic-agent.service
# the client logs are located at
/opt/Elastic/Agent/data/elastic-agent-xxxxxx/logs
