How to: Install Elastic Stack on Debian 11 Bullseye

1. Preparation

This is my very personal configuration. All you really need is to have the sudo-package installed. Everything else is optional.

# updating repo
apt update

# installing my own packages, but sudo is mandatory
apt install sudo htop ncdu neofetch mc qemu-guest-agent

# adding my user to the sudo-group
adduser user sudo

# oneliner for setting up 4 GB swapfile
sudo fallocate -l 4G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile && sudo swapon /swapfile && echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

2. Elasticsearch

# installing packages
sudo apt update && sudo apt install apt-transport-https gnupg curl wget

# adding repo key
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

# adding repo
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

# install the package
sudo apt update && sudo apt install elasticsearch

You will get an output like:

-------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : PtzeSXs39q6rvbNFctR=

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with 
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with 
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with 
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

Start the Elasticsearch service and check if the service is running after a short period of time:

# start and enable service
sudo systemctl enable elasticsearch.service && sudo systemctl start elasticsearch.service

# check if service responds with username elastic and your superuser password from above
curl -X GET -k "https://elastic:PtzeSXs39q6rvbNFctR=@localhost:9200"

3. Kibana

# install package
sudo apt install kibana

You will get an output like:

Created Kibana keystore in /etc/kibana/kibana.keystore

# connect kibana with elastic
sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

# you'll get a token like:
eyJ2ZXIiOiI4LjQuMyIsImFkciI6LKHkahsdfklALSKDhlkaslhfKJASaASBKJDBJKASLBKJDBJsdxZjRkOTU3M2RkMTk1M2E0ZDYiLCJrZXkiOiJ1akZGeklNQl9oc2J2MjdBbTdDcjptQnFpWm9VaVNJNmx3MXFUWDIxUFZRIn0=

# start the kibana setup
sudo /usr/share/kibana/bin/kibana-setup

# start and enable kibana
sudo systemctl enable kibana && sudo systemctl start kibana

# check if kibana is listening on port 5601
ss -lntp | grep 5601

4. Install NGINX and configure for proxy requests

# install the package
sudo apt install nginx

# edit the default site to enable proxying
sudo nano /etc/nginx/sites-enabled/default

# find "location" and edit this part accordingly
        location / {
                proxy_pass http://127.0.0.1:5601;
        }

# restart nginx webserver
sudo systemctl restart nginx

Kibana is now reachable via web browser at your ip address. Check if you can log in.

5. Further configuration

# edit config file
sudo nano /etc/kibana/kibana.yml

# change is_default and is_default_monitoring to false
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: false, is_default_monitoring: false, type: elasticsearch, hosts: ['https://38.242.252.137:9200'], ca_trusted_fingerprint: d65d4e2fd567f88732a040ead1f579bc95beb3159b57a01f4d9573dd1953a4d6}]

# fleet > settings > outputs
# set host to https://IP:9200
# set fingerprint from kibana.yml
# add to adv yaml config
# ssl.verification_mode: "none"

# fleet > add a fleet server: https://IP:8220
# you ll get instructions like:
# curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.4.3-linux-x86_64.tar.gz
# tar xzvf elastic-agent-8.4.3-linux-x86_64.tar.gz
# cd elastic-agent-8.4.3-linux-x86_64
# sudo ./elastic-agent install \
#  --fleet-server-es=http://localhost:9200 \
#  --fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NjU1ODE0Njk1MzU6ZXc5V3BYUzVTODJ0ZjczUUtLLXpYdw \
#  --fleet-server-policy=fleet-server-policy

# edit the last command

sudo ./elastic-agent install \
  --fleet-server-es=https://38.242.252.137:9200 \
  --fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NjU1ODE0Njk1MzU6ZXc5V3BYUzVTODJ0ZjczUUtLLXpYdw \
  --fleet-server-policy=fleet-server-policy \
  --fleet-server-es-ca-trusted-fingerprint=d65d4e2fd567f88732a040ead1f579bc95beb3159b57a01f4d9573dd1953a4d6 \
  --insecure

6. Configure Elasticsearch and Kibana

# open the elasticsearch config file
sudo nano /etc/elasticsearch/elasticsearch.yml

# change/add one value
network.host: SERVER_IP

# and restart the service
sudo systemctl start elasticsearch.service

# open the kibana config file
sudo nano /etc/kibana/kibana.yml

# change/add two values
server.host: "SERVER_IP“
server.publicBaseUrl: "http://SERVER_IP:5601"

# and restart the service
sudo systemctl start kibana.service

7. Connect Kibana with Elasticsearch

# generate a token
sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

You will get an token like:

eyJ2ZXIiOiI4LjQuMiIsImFkciI6WyIxOTIuMTY4LjU2LjE2MDo5MjAwIl0sImZnciI6IjFmZGE3MjgwM2QzYjI1Y2VkMmY3MmJmODQxYjc4Mjc5MWFjNjZkNDFjODIwMDJjZWEzYTEzMTIwZjBmOTQzOWYiLCJrZXkiOiJBbVY5ZjRNQkdJbmRxMDE5LUlfSjpDRmpVeDF5ZFNPV0dqeEtnTHlPUUd3In0=

Point your browser to Kibana: http://SERVER_IP:5601. Paste the token in the browser and verify by using the code via command line:

# get verification code after providing the enrollment token
sudo /usr/share/kibana/bin/kibana-verification-code

8. Securing Kibana

# get the import password which is needed for the next step
sudo /usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password

# You will get an import password like:
f-Ss0Db5RKhabsd4wKZ82

# generate certificates and paste your import password
sudo openssl pkcs12 -in /etc/elasticsearch/certs/http.p12 -out /etc/kibana/server.crt -clcerts -nokeys

sudo openssl pkcs12 -in /etc/elasticsearch/certs/http.p12 -out /etc/kibana/server.key -nocerts -nodes

# change the permissions
sudo chown root:kibana /etc/kibana/server.crt
sudo chown root:kibana /etc/kibana/server.key
sudo chmod g+r /etc/kibana/server.crt
sudo chmod g+r /etc/kibana/server.key

# add more security to kibana
sudo nano /etc/kibana/kibana.yml

# add the following three values
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/server.crt
server.ssl.key: /etc/kibana/server.key

# generate security keys
sudo /usr/share/kibana/bin/kibana-encryption-keys generate

You will get an output like:

## Kibana Encryption Key Generation Utility

The 'generate' command guides you through the process of setting encryption keys for:

xpack.encryptedSavedObjects.encryptionKey
    Used to encrypt stored objects such as dashboards and visualizations
    https://www.elastic.co/guide/en/kibana/current/xpack-security-secure-saved-objects.html#xpack-security-secure-saved-objects

xpack.reporting.encryptionKey
    Used to encrypt saved reports
    https://www.elastic.co/guide/en/kibana/current/reporting-settings-kb.html#general-reporting-settings

xpack.security.encryptionKey
    Used to encrypt session information
    https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#security-session-and-cookie-settings


Already defined settings are ignored and can be regenerated using the --force flag.  Check the documentation links for instructions on how to rotate encryption keys.
Definitions should be set in the kibana.yml used configure Kibana.

Settings:
xpack.encryptedSavedObjects.encryptionKey: c8136c292e1bb8c7ebcfab522ca8cf12
xpack.reporting.encryptionKey: 38718ec714520269b6b116ca9eb3055c
xpack.security.encryptionKey: c84879165b3180bfb9da4f8510779f0e

Copy and the last the lines of the output and fill it into the kibana config file:

sudo nano /etc/kibana/kibana.yml

# fill in the encryption keys at the end as mentioned before
xpack.encryptedSavedObjects.encryptionKey: c8136c292e1bb8c7ebcfab522ca8cf12
xpack.reporting.encryptionKey: 38718ec714520269b6b116ca9eb3055c
xpack.security.encryptionKey: c84879165b3180bfb9da4f8510779f0e

Login with your browser at https://SERVER_IP:5601 with your username elastic and your superuser-password as provided in the second step. Change your password via editing the profile:

# enable autostart for both services and reboot to verify all is working
sudo systemctl enable elasticsearch.service
sudo systemctl enable kibana.service
sudo reboot

Allow the system some minutes to start all services after reboot. Elasticstack should soon be reachable under https://SERVER_IP:9200 and Kibana under https://SERVER_IP:5601.

9. Setting up Fleet

In order to set up Fleet we need to install the elastic agent in fleet-server mode. Login and go to Management > Fleet > Agents > Add a fleet server. Use https://SERVER_IP:8220 and generate the Fleet Server Policy. Use Linux Tar.

# download the package
curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.4.2-linux-x86_64.tar.gz

# extract it
tar xzvf elastic-agent-8.4.2-linux-x86_64.tar.gz && cd elastic-agent-8.4.2-linux-x86_64

# extend the last command by adding the flag --fleet-server-es-ca
# otherwise the installation will fail with "Error: fleet-server failed: context canceled"

# your final installation command will look like
sudo ./elastic-agent install \
  --fleet-server-es=https://SERVER_IP:9200 \
  --fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NjQyOTI1NDUzMTM6WWpydUlkNW1Sb0dNUHFrN09oc2xUQQ \
  --fleet-server-policy=fleet-server-policy \
  --fleet-server-es-ca=/etc/elasticsearch/certs/http_ca.crt

You will get an output like:

Elastic Agent will be installed at /opt/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:

{"log.level":"info","@timestamp":"2022-09-27T17:43:52.544+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":403},"message":"Generating self-signed certificate for Fleet Server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-27T17:43:56.793+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":773},"message":"Fleet Server - Running on policy with Fleet Server integration: fleet-server-policy; missing config fleet.agent.id (expected during bootstrap process)","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-27T17:43:56.933+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":471},"message":"Starting enrollment to URL: https://myelasticserver:8220/","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-27T17:44:00.885+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":273},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.
Elastic Agent has been successfully installed.

Et voila. We are ready to go.

10. Adding a client to Fleet Management

Now it’s time to add our first client to the system in order to be able to monitor it. For demonstration purposes we’ll connect a simple stock Debian machine.

After having your client ready log in to Kibana and go to Management > Fleet > button Add Agent. You can leave the name “Agent policy 1”. Use the button to create it.
Leave the default setting “enroll in fleet” untouched.
Follow the Linux Tar code to install the agent on your client system and to enroll the client automatically:

# download agent
curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.4.2-linux-x86_64.tar.gz

# unpack
tar xzvf elastic-agent-8.4.2-linux-x86_64.tar.gz

# change into directory
cd elastic-agent-8.4.2-linux-x86_64

# important!!! the last install command has to be extended by the flag --insecure
# as we don't have any regular PKI in place.
sudo ./elastic-agent install --url=https://192.168.56.160:8220 --enrollment-token=cFZmSGhJTUI0LTBMVmd4M3cwcG46UzRpSjYtejhTV2FTVW1TMndEV1QtUQ== --insecure

You will get an output like:

Elastic Agent will be installed at /opt/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:

{"log.level":"warn","@timestamp":"2022-09-28T17:51:28.584+0200","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":104},"message":"SSL/TLS verifications disabled.","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-28T17:51:28.603+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":471},"message":"Starting enrollment to URL: https://192.168.56.160:8220/","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2022-09-28T17:51:28.946+0200","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":104},"message":"SSL/TLS verifications disabled.","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-09-28T17:51:29.806+0200","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":273},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.
Elastic Agent has been successfully installed.

Leave the Kibana tab open and wait until the waiting circle close to “confirm incoming data” vanishes. If nothing changes there are several places you can start troubleshooting:

# check the agent status on the client
sudo systemctl status elastic-agent.service

# the client logs are located at
/opt/Elastic/Agent/data/elastic-agent-xxxxxx/logs

How to: Install Elastic Stack on Debian 11 Bullseye

Meta

1. Preparation

3. Kibana

4. Install NGINX and configure for proxy requests

5. Further configuration

6. Configure Elasticsearch and Kibana

7. Connect Kibana with Elasticsearch

8. Securing Kibana

9. Setting up Fleet