Elasticsearch - ELK stack

======================================================================================================================================================

Upgrade version of elasticsearch on Linux

Stop running elasticsearch
systemctl stop elasticsearch

# With seperated config and logs:
cd /opt
rm -rf elasticsearch
tar xvzf elasticsearch-8.x.x-linux-aarch64.tar.gz
rm -rf elasticsearch-8.x.x/jdk
mv elasticsearch-8.x.x elasticsearch
rm /var/log/elasticsearch/*
chown -R elasticsearch:elasticsearch /opt/elasticsearch


systemctl start elasticsearch

#Check the health of the cluster
curl --user elastic:<PASSWORD> -X GET http://localhost:9200/_cat/health?v

curl --user elastic:<PASSWORD> -X GET http://localhost:9200/_cat/indices?v

======================================================================================================================================================

Upgrade version of Kibana on Windows PC/Laptop

Remove the old folder (like C:\Temp\kibana)

Properties zip file "unblock" content flag
Rename to short name and place directly under c:\ (otherwise there are path lenght problems while extracting)

unzip via windows Zip the file  (location like: C:\Kibana.zip)

Move the Kibana directory to another location/map

Edit config\kibana.yml 
server.port: 5601
server.host: "<Your PC name in WIndows"
#elasticsearch.hosts: ["http://localhost:9200"]
elasticsearch.hosts: ["http://<IPADDRESS ELASTICSEARCH SERVER>:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "<Password>"
# Maybe 15 minutes is not enough and make it 1 hour: 3600000
xpack.reporting.queue.timeout: 3600000
# Maybe 100MB is not enough..
xpack.reporting.csv.maxSizeBytes: 104857600
# And change for reporting the timezone to "UTC" in "Stack Management"->"Advanced settings"

Start Kibana via the bin\kibana.bat

======================================================================================================================================================

New version 8.x.x of ELK
========================
tar xvzf elasticsearch-8.x.x-linux-aarch64.tar.gz
rm -rf jdk
chown -R elasticsearch:elasticsearch /opt/elasticsearch-8..

export ES_JAVA_HOME=/etc/alternatives/jre

Edit elasticsearch.yml (Initial setup...further below the current config)
cluster.name: <Your clustername>
node.name: <Your hostname>
network.host: [127.0.0.1,<IPADDRESS>]
http.port: 9200
discovery.seed_hosts: ["127.0.0.1", "<IPADDRESS>"]
node.roles: [master, data, ingest]
xpack.security.enabled: true
discovery.type: single-node
xpack.ml.enabled: false
path.repo: backups

cd /opt/elasticsearch-8..
cp -p -r /var/lib/elasticsearch data

./bin/elasticsearch -d

Switch off rsyslog module omelasticsearch (_type problem..maybe solved by now..?)
Edit the scipts in /root/ELK/ (look for _doc and remove it; _type is no longer supported)

=========================

-----------------------------------------------------------------------

Place elasticsearch on redberry server.
========================================================================
yum install perl-JSON-PP jq
yum install java-latest-openjdk-headless.armv7hl

 

groupadd -g 124 elasticsearch
useradd -u 116 -g elasticsearch -G users -c "ElasticSearch" -d /opt/elasticsearch -m elasticsearch
tar xvzf elasticsearch-8.x.x-linux-aarch64.tar.gz
mv elasticsearch-8.x.x. /opt/elasticsearch
cd /opt/elasticsearch/
rm -rf jdk
chown -R elasticsearch:elasticsearch /opt/elasticsearch

#vi bin/elasticsearch-env
#ES_JAVA_HOME=/etc/alternatives/jre

vi config/elasticsearch.yml

cluster.name: <CLustername>
node.name: <Hostname>
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: [127.0.0.1,<IPADDRESS>]
http.port: 9200

discovery.seed_hosts: ["127.0.0.1", "<IPADDRESS>"]
node.roles: [master, data, ingest]
xpack.security.enabled: true
discovery.type: single-node
xpack.ml.enabled: false
path.repo: /var/lib/elasticsearch/backups
xpack.security.authc.api_key.enabled: true

vi config/jvm.options
## GC logging
#-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m

vi config/jvm.options.d/unix4life.options
-Xms400m
-Xmx550m
-Dlog4j2.formatMsgNoLookups=true
-Xlog:all=warning:stderr:utctime,level,tags
-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m
-Xlog:disable

vi config/log4j2.properties
#rootLogger.level = info
rootLogger.level = error
#logger.deprecation.level = deprecation   => Gives POSSIBLE shit with shards: .ds-.logs-deprecation.elasticsearch-default-<date>...
logger.deprecation.level = OFF

 

iptables -I IN_public_allow -p tcp -m tcp --destination-port 9200 -j ACCEPT
firewall-cmd --zone=public --add-port=9200/tcp
firewall-cmd --zone=public --add-port=9200/tcp --permanent


vi /usr/lib/systemd/system/elasticsearch.service

[Unit]
Description=Run ElasticSearch as user elasticsearch
DefaultDependencies=no
After=network.target

[Install]
WantedBy=multi-user.target

[Service]
Type=simple
User=elasticsearch
Group=elasticsearch
Environment="ES_PATH_CONF=/etc/elasticsearch/config"
Environment="ES_JAVA_HOME=/etc/alternatives/jre"
WorkingDirectory=/opt/elasticsearch
ExecStart=/opt/elasticsearch/bin/elasticsearch -d -p /opt/elasticsearch/elasticsearch.pid
TimeoutStartSec=60
RemainAfterExit=yes

 


chown -R elasticsearch:elasticsearch /opt/elasticsearch/
chown -R elasticsearch:elasticsearch /var/log/elasticsearch/
chown -R elasticsearch:elasticsearch /var/lib/elasticsearch/

chmod 660 /etc/elasticsearch/config/*.yml

systemctl daemon-reload
systemctl enable elasticsearch

# Set passwords
systemctl start elasticsearch
su - elasticsearch
export ES_PATH_CONF=/etc/elasticsearch/config
export ES_JAVA_HOME=/usr/lib/jvm/java-17-openjdk-17.0.1.0.12-1.rolling.el7.arm
./bin/elasticsearch-setup-passwords interactive

export ES_PATH_CONF=/etc/elasticsearch/config
export ES_JAVA_HOME=/usr/lib/jvm/java-17-openjdk-17.0.1.0.12-1.rolling.el7.arm
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-17.0.1.0.12-1.rolling.el7.arm
bin/elasticsearch-sql-cli http://elastic:<PASSWORD>@localhost:9200

 

 

Enable Elasticsearch security featuresedit
Add the xpack.security.enabled setting to the ES_PATH_CONF/elasticsearch.yml file and set the value to true:
xpack.security.enabled: true

Re-start elasticsearch

Create passwords for built-in usersedit
You only need to set passwords for the elastic and kibana_system users when enabling minimal or basic security.

./bin/elasticsearch-setup-passwords interactive
root@unix4life:/opt/elasticsearch# ./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana_system]:
Reenter password for [kibana_system]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

 

Check of your elasticsearch query to see if it all works...

curl --user elastic:<PASSWORD> -X GET http://localhost:9200/_cat/indices?v
health status index       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .security-7 HIeZfpFgSDexxHucNImLaA   1   0          7            0     25.3kb         25.3kb

 

 

What do you think or any questions?

Send us feedback!

Enter the sum of the numbers.

Menu

  If you like my website, feel free to donate via the Paypal button... A small amount for a cup of coffee is enough ;-) Thank you!