Enterprise Search > Installation technician > Elastic Search > First installation

First installation

Smartsite 7.9 - ...

Purpose

The steps below are for a first installation.

Version

Use Elastic Search v8. Several versions are tested; however it is recommended to use the particular version which has been tested in combination with other software components such as the Java runtime environment or the Enterprise Search indexer. See the version table also.

  • v8.19.3

Elastic Search is not 32 bit or 64 bit specific; this is handled by the Java runtime environment.

Media

The installation media includes folder R12.2\Elastic Search, containing:

  • elasticsearch-8.19.3-windows-x86_64.zip

Prerequisites

The following prerequisites apply to the Elastic Search server.

  • The Java run time environment must be available. The Java as bundled with Elastic Search will be used. A separate installation of the Java run time environment is not required.

Installation

This is a new installation. Instructions for an update will be added separately.

Typically install on multiple servers, forming an Elastic Search cluster. For example install on three servers.

Unpack the zip.

  1. Unpack the zip in a temporary folder, for example E:\Temp\elasticsearch-8.19.3-windows-x86_64.
  2. Move subfolder elasticsearch-8.19.3 to a folder and subfolder under Program Files, for example E:\Program Files\ElasticSearch\8.19.3. Folder 8.19.3 should contain subfolders such as bin and config after moving.

Configure config\elasticsearch.yml. Uncomment and adjust entries:

  1. Set cluster.name. For example: es-cluster.
  2. Set node.name. For example: es-node-1. This is a name for use by Elastic Search, as opposed to a network resolvable server name.
  3. Set path.data with the folder that will contain the Elastic Search data, which can be substantial. For example: E:\ElasticSearch\Data. It is not required to create this folder; this is done by Elastic Search.
  4. Set path.logs with the folder that will contain the Elastic Search logs. For example: E:\ElasticSearch\Logs. It is not required to create this folder; this is done by Elastic Search.
  5. Set the IP address or server name ES will listen to, entry network.host. For example use host-123.example.com. This name should be resolvable on the network. This name is also relevant when securing the cluster, and will for example return as *.example.com in certificates used for https communication.
  6. Consider setting the port; entry http.port. Port 9200 will be used as default if not set.
  7. For a cluster and if this is a side-by-side installation with another instance / version of Elastic Search: set the transport port for the instance, for example 9301.
  8. For a cluster: list the hosts in the cluster, entry discovery.seed_hosts. Use the IP addresses or names as used in entry network.host. For example: ["host-123.example.com", "host-124.example.com", "host-125.example.com"]. The same complete list can be used on each server; the list can include the address of the server itself.
  9. For a cluster: list the master-eligible nodes, entry cluster.initial_master_nodes. Use the node names, not the server addresses. For example: ["es-node-1", "es-node-2", "es-node-3"]. The same complete list can be used on each server; the list can include the name of the node itself. Note that when building-up a cluster and starting with the first node list this first node only; otherwise an error may occur if Elastic Search attempts to contact the other nodes in order to establish the master node. This also means that when done setting up the cluster it is required to return to this configuration, adding the remaining nodes.
  10. Add the following comment and setting:
# Do not automatically create a new index because this can result in default mappings.
# The Enterprise Search solution should create an index and should specify appropriate mappings.
action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*

Temporarily disable security.

  1. This is required for the interactive test. It will be replaced by actual security. Add the following line:
xpack.security.enabled: false

If Elastic Search serves on a non-local address allow incoming traffic.

  1. Add a firewall rule. Use an inbound port rule, specifying 9200 or specifying the selected port, for traffic over TCP. Name the rule, for example Elastic-search-8.19.3. Describe the rule, for example Elastic Search index- and search operations.

If the node is part of a cluster allow internode communication.

  1. Add a firewall rule. Use an inbound port rule, specifying 9300, for traffic over TCP. Name the rule, for example Elastic-search-cluster-8.19.3. Describe the rule, for example Elastic Search cluster operations.

Additional configuration is required to secure the cluster, described below. First test interactively:

  1. Start a command box, as administrator.
  2. Go to the bin folder, for example E:\Program Files\elasticsearch\8.19.3\bin.
  3. Run elasticsearch.bat.
  4. Inspect the feedback. Expect that running continues and does not come to a halt.
  5. Visit Elastic Search, using a browser on the server. Issue http://localhost:9200/, adjusting the host name and port. Expect cluster information. Note that at this point http should be used, until security setup has been completed and https should be used.
  6. In te command box stop Elastic Search with Ctrl-C.

Secure the cluster

Optionally secure the cluster at these levels:

  • Enable Transport Layer Security (TLS) for secure communication between nodes of the cluster.
  • Use TLS / HTTPS for secure communication between clients and the cluster. Clients include the Smartsite Manager, search applications, and Kibana.
  • Require credentials to access the cluster.

Internode communication

Create a certificate authority (CA) for use in the cluster. Each node will be instructed to trust this CA.

  1. On a node of the cluster: start a command box, as administrator.
  2. Go to the folder above the bin, for example E:\Program Files\elasticsearch\8.19.3.
  3. Run: bin\elasticsearch-certutil.bat ca
  4. Accept: elastic-stack-ca.p12
  5. Create and enter strong password #1.
  6. This generates a CA certificate elastic-stack-ca.p12. Create a folder config\certs and move the certificate to folder certs. For information: there is no need to install this certicate, it is found by means of xpack configuration below.

Generate a certificate and private key for use on each node of the cluster. It is possible to use a distinct certificate per node. Here the same certificate will be used for all nodes.

  1. Continue working in the folder above the bin.
  2. Run: bin\elasticsearch-certutil.bat cert --ca config\certs\elastic-stack-ca.p12
  3. Enter the above password #1 for the CA.
  4. Accept: elastic-certificates.p12
  5. Create and enter another strong password #2.
  6. This generates a certificate elastic-certificates.p12. Move the certificate to folder certs. For information: there is no need to install this certicate, it is found by means of xpack configuration below.

Do not directly add the above passwords to the Elastic Search configuration file elasticsearch.yml. Instead use a keystore (file elasticsearch.keystore, containing passwords, as opposed to certificate elastic-certificates.p12 which contains a private key and which is also called a keystore).

  1. Go to the bin.
  2. Run: elasticsearch-keystore.bat add xpack.security.transport.ssl.truststore.secure_password
  3. Enter the node certificate password #2.
  4. Run: elasticsearch-keystore.bat add xpack.security.transport.ssl.keystore.secure_password
  5. Enter the node certificate password #2.

This will create or update file config\elasticsearch.keystore.

  1. Enable internode security in config\elasticsearch.yml. For the moment keep the line xpack.security.enabled: false, adding the remaining lines:
# Cluster security.
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.truststore.path: certs\elastic-certificates.p12
xpack.security.transport.ssl.keystore.path: certs\elastic-certificates.p12
  1. Test interactively, as done earlier. Still use http, and still expect that no credentials are requested.

Client communication

The steps below result in a certificate http.p12 for secure communication between clients and the cluster. The cluster will present the certificate to clients such as browsers, the Smartsite manager, Kibana and curl.

  1. On a node of the cluster: start a command box, as administrator.
  2. Go to the folder above the bin, for example E:\Program Files\elasticsearch\8.19.3.
  3. Run: bin\elasticsearch-certutil.bat http
  4. Generate a CSR? [y/N] => N, the known CA will be used.
  5. Use an existing CA? [y/N] => y
  6. CA Path: certs/elastic-stack-ca.p12. For information: given the current folder one would expect config\certs which however results in an error because config\config\certs is searched. Use certs. Also use a forward slash.
  7. Password for elastic-stack-ca.p12: enter the CA password #1.
  8. For how long should your certificate be valid? [5y] => consider raising this, for example to 30y.
  9. Generate a certificate per node? [y/N] => N, one certificate will be used for all nodes.
  10. Enter all the hostnames that you need, one per line. => *.example.com, the suffix that resolves to the nodes of the cluster.
  11. Enter all the IP addresses that you need, one per line. => enter, no IP addresses
  12. A summary is presented. Do you wish to change any of these options? [y/N] => N
  13. Provide a password for the "http.p12" file: Create and enter another strong password #3.
  14. Repeat password to confirm.
  15. What filename should be used for the output zip file? [C:\Program Files\elasticsearch\7.17.3\elasticsearch-ssl-http.zip] => accept.
  16. Move elasticsearch-ssl-http.zip to config\certs.

The zip contains instructions how to apply the certificate.

  1. Unpack in a temporary folder.
  2. Locate http.p12 and copy this certificate to config\certs.
  3. Enable client security in config\elasticsearch.yml. Add:
# This turns on SSL for the HTTP (Rest) interface
xpack.security.http.ssl.enabled: true
#
# This configures the keystore to use for SSL on HTTP
# Because your keystore has a password, you will also need to add the password to the Elasticsearch keystore
# elasticsearch-keystore add "xpack.security.http.ssl.keystore.secure_password"
xpack.security.http.ssl.keystore.path: certs\http.p12

Also add the password to the keystore:

  1. Go to the bin.
  2. Run: elasticsearch-keystore.bat add xpack.security.http.ssl.keystore.secure_password
  3. Enter the client certificate password #3

Test:

  1. Test interactively, as done earlier. Use http and do not use credentials.

Require credentials

Require that clients supply credentials:

  1. In config\elasticsearch.yml enable security. Change false to true:
# Enable security and require credentials.
xpack.security.enabled: true

Elastic Search uses internal password for its system users, until password are configured. Once configured internal passwords no longer apply. Create and set passwords for all system users.

  1. Make sure Elastic Search is running. Use the above interactive test steps and do not stop with Ctrl-C.
  2. Run (adjust the url): bin\elasticsearch-setup-passwords.bat interactive --url "https://host-123.example.com:9200/"
  3. In turn create and set a password for each system user. For information: this includes [elastic] with password #4 and this includes [kibana_system] with password #5.

 Test with a browser.

  1. Restart Elastic Search interactively.
  2. On a remote computer install the CA certificate elastic-stack-ca.p12 in the Trusted Root Certification Authorities store. The result is a certificate installed under a name such as Elastic Certificate Tool Autogenerated CA.
  3. Browse to https://host-123.example.com:9200/_cluster/health?pretty. Expect no warning about an untrusted certificate. Expect a login request. Perform a login using user elastic and using password #4.
  4. Stop Elastic Search with Ctrl-C.

Enable all master nodes, if disabled. Test once more.

Install as a Windows service

Install and start Elastic Search as a Windows service.

  1. Go to folder bin.
  2. Run elasticsearch-service.bat install es-service-819. The name es-service-819 is optional, however is required if the cluster runs multiple Elastic Search instances / versions side by side.
  3. Run elasticsearch-service.bat start es-service-819.
  4. Start the services management console and locate the service with a name such as Elasticsearch 8.19.3 (es-service-819).
  5. Set the startup type to Automatic.
  6. Set the recovery mode. First failure: restart the service. Second failure: restart the service. Subsequent failures: take no action.

Test using a browser.

  1. If Elastic Search runs on localhost: visit http://localhost:9200/_cluster/health?pretty. Expect a cluster status summary.
  2. If Elastic Search runs for a server url: perform the test from a browser on a remote system.

Server configuration

Elastic Search heavily uses files for its storage and processing. The following general recommendations apply to the involved folder, for example E:\ElasticSearch\Data.

  • A faster disk results in better performance, both indexing time and search time.
  • Disable Microsoft Search and/or other search services, or at least ensure that the ES data folder is excluded from indexing.
  • Consider excluding the ES data folder for monitoring by virus scanners. The folder does not contain executable files, script files and other files with active content. Technically it is possible to store files in ES, using a field of type binary, which results in the file on disk. The Enterprise Search solution however does not use this support and it can be assumed that files in the data folder are all safe files in terms of virus threats.

Add nodes to the cluster

Add nodes to the cluster, for performance and reliability. Typically add a node by adding a modest server and running one node on that server.

  1. Add a server and perform the above installation steps.
  2. Restart the Elastic Search service on the node and perform the above cluster health sanity check. Expect number_of_nodes to specify the number of nodes in the cluster.
Installation technician