FreeBSD OpenSearch Configuring Plugins and TLS certificates with OpenSSL

OpenSearch is an open source utility that is often used as a search engine and is also used as a powerful tool for performing data analysis. The writing of Opensearch is taken from the Elasticsearch 7.10.2 code base. The systems in Opensearch include:
  1. Storage and search engine.
  2. Web interface.
  3. OpenSearch Dashboard data visualization environment.
  4. As well as add-ons that allow you to use a number of Elasticsearch engine features.
OpenSearch was born thanks to community participation and support from large companies such as Red Hat, SAP, Capital One, etc. The Opensearch system is often used for full-text searches on sites because it can store and analyze logs, and to visualize and analyze the information received.

Opensearch can be combined with Beat data delivery platforms (Filebeat, Winlogbeat, etc.), so that OpenSearch users can build a full cycle of log management in the form of collection, systematization and search.



1. System specifications

OS: FreeBSD 13.3
Hostname: ns3
IP address: 192.168.5.2
Logstash version: logstash8-8.11.3
Opensearch version: Opensearch-2.11.1
Dependencies: bash jna
Java version:
a. openjdk version "17.0.9" 2023-10-17
b. OpenJDK Runtime Environment (build 17.0.9+9-1)
c. OpenJDK 64-Bit Server VM (build 17.0.9+9-1, mixed mode, sharing)


2. Setup Transport layer TLS

In this section we will explain how to configure the Opensearch security plugin in FreeBSD. On Opensearch there are lots of plugins that you can use, all of these plugins require an SSL certificate which can support many additional features and configuration methods. You can create a TLS certificate configuration using tools such as OpenSSL.

Adding a TLS Certificate will provide additional security for your Opensearch host. The TLS certificate will grant permission to the client to verify the node's identity and encrypt traffic between the client and the host. The TLS certificate is configured in the opensearch.yml file, and we will save it in the /usr/local/etc/opensearch directory. Follow the steps below to create a self-signed certificate.

Remove certificate
root@ns3:~ # cd /usr/local/etc/opensearch
root@ns3:/usr/local/etc/opensearch # rm -f *pem

a. Generate a root certificate

Before going any further, we first create a private key for the root certificate. The root certificate is used to sign other certificates.

Root certificate
root@ns3:/usr/local/etc/opensearch # openssl genrsa -out root-ca-key.pem 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
.........................................+++++
.............................+++++
e is 65537 (0x010001)
After that, you create a root CA certificate that you sign yourself. Use the -subj parameter to provide your host-specific information.

Root certificate
root@ns3:/usr/local/etc/opensearch # openssl req -new -x509 -sha256 -key root-ca-key.pem -out root-ca.pem -days 730

b. Generate Administrator Certificates

Next, create a certificate for the administrator. To create an administrator certificate, first create a new key so that an administrator can perform administrative tasks related to the security plugin.

Create administrator certificate
root@ns3:/usr/local/etc/opensearch # openssl genrsa -out admin-key-temp.pem 2048
So that the administrator certificate can be used with Java programs, convert the key to PKCS format.

Convert administrator certificate to PKCS
root@ns3:/usr/local/etc/opensearch # openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
Next, create a certificate signing request (CSR) for the administrator's certificate (Certificate Signing request) based on the private key. This file acts as an application to the CA for the certificate you have signed.

Create CSR
root@ns3:/usr/local/etc/opensearch # openssl req -new -key admin-key.pem -out admin.csr
Now that the private key and signing request have been created, sign the administrator certificate using the root certificate and private key you created earlier.

Sign the administrator certificate
root@ns3:/usr/local/etc/opensearch # openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730

c. Create node Certificates

Creating a Certificates node is almost the same as the steps for creating an administrator certificate above. In this section we will create keys and CSRs with new file names for each node and client certificate. To generate a node or client certificate, first generate a new key.

Generate Node Certificates
root@ns3:/usr/local/etc/opensearch # openssl genrsa -out node1-key-temp.pem 2048
Then convert the private key to PKCS format, so that it can be used by Java applications.

Convert private key to PKCS
root@ns3:/usr/local/etc/opensearch # openssl pkcs8 -inform PEM -outform PEM -in node1-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node1-key.pem
Next, create a new CSR for the host certificate based on the private key.

Create CSR
root@ns3:/usr/local/etc/opensearch # openssl req -new -key node1-key.pem -out node1.csr
For all host and client certificates, you must specify a subject alternative name (SAN). Before creating a signed certificate, create a SAN extension file that contains the host name and domain name.

Create a SAN
root@ns3:/usr/local/etc/opensearch # echo 'subjectAltName=DNS:node1.datainchi.com' > node1.ext
Next, generate the certificate.

Sign the CSR
root@ns3:/usr/local/etc/opensearch # openssl x509 -req -in node1.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out node1.pem -days 730 -extfile node1.ext
Next, convert the CA root certificate to .crt format so that the CA root certificate can be installed on the Opensearch server.

Convert CA root
root@ns3:/usr/local/etc/opensearch # openssl x509 -outform der -in root-ca.pem -out root-ca.crt
Change the ownership and permissions of the directory to the opensearch:opensearch.

Change the ownership and permissions
root@ns3:/usr/local/etc/opensearch # chown -R opensearch:opensearch /usr/local/etc/opensearch/

d. Setup Certificates To Opensearch

Once you have created all the certificates, proceed to install and add them to your OpenSearch configuration file namely "opensearch.yml". Open "/usr/local/etc/opensearch/opensearch.yml", and at the very bottom of the script (end of script) you add the script below.

Change the ownership and permissions
root@ns3:/usr/local/etc/opensearch # ee opensearch.yml
plugins.security.ssl.transport.pemcert_filepath: /usr/local/etc/opensearch/node1.pem
plugins.security.ssl.transport.pemkey_filepath: /usr/local/etc/opensearch/node1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /usr/local/etc/opensearch/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /usr/local/etc/opensearch/node1.pem
plugins.security.ssl.http.pemkey_filepath: /usr/local/etc/opensearch/node1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /usr/local/etc/opensearch/root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.nodes_dn:
  - 'CN=node1.datainchi.com,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true


3. Installing Plugins

As we know, Opensearch has several core plugins as part of its release installation. In Opensearch, plugins are used to enhance key functions and features. Apart from the core plugins, you can also write your own custom plugins and there are also community created plugins available on Github.

So that plugin functions can run perfectly and connect with Opensearch, all plugins must be able to access data in the cluster. Before you use a plugin, you must first understand the function and benefits of the plugin. Then, what plugins are there in Opensearch. You can view the available plugins by running the command below.

Check the plugin list
root@ns3:~ # cd /usr/local/lib/opensearch/bin
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin list
analysis-extension
analysis-fess
analysis-icu
configsync
minhash
opensearch-alerting
opensearch-anomaly-detection
opensearch-asynchronous-search
opensearch-cross-cluster-replication
opensearch-custom-codecs
opensearch-geospatial
opensearch-index-management
opensearch-job-scheduler
opensearch-knn
opensearch-ml
opensearch-neural-search
opensearch-notifications
opensearch-notifications-core
opensearch-observability
opensearch-performance-analyzer
opensearch-reports-scheduler
opensearch-security
opensearch-security-analytics
opensearch-sql
Full list of supported additional plugins:

- repository-gcs
Adds support for the Google Cloud Storage service as a snapshot repository.

- analysis-icu
Adds the Lucene ICU module with extended Unicode support and the use of ICU libraries. The module provides improved analysis of Asian languages, Unicode normalization, Unicode case conversion, matching support, and transliteration.

- repository-azure
Adds support for Azure Blob storage as a snapshot repository.

- analysis-kuromoji
Adds the Lucene kuromoji analysis module for Japanese.

- analysis-nori
Adds the Lucene nori analysis module for Korean. 

- analysis-phonetic
Provides token filters that convert expressions to their phonetic representation using Soundex, Metaphone, and other algorithms.

- repository-s3
Adds support for AWS S3 as a snapshot repository.

- analysis-smartcn
Adds Lucene's Smart Chinese analysis module for Chinese or mixed Chinese-English text.

- analysis-stempel
Adds Lucene's Stempel analysis module for Polish.

- ingest-attachment
Extracts file attachments in common formats (such as PPT, XLS, and PDF) using the Apache Tika text extraction library.

- mapper-annotated-text
Indexes text that is a combination of plain text and special markup. This combination is used to identify objects, such as people or organizations.

- mapper-murmur3
Calculates a hash of field values based on index time and stores them in the index.

- mapper-size
Provides a metadata field size, that indexes the size in bytes of the source

- repository-hdfs
Adds support for the HDFS file system as a snapshot repository.

- transport-nio
A server-client non-blocking network library created using Netty.


a. How to use the Opensearch plugin

Opensearch's functionality and features can be enhanced by adding custom plugins. Examples of using plugins that can enhance Opensearch functionality such as plugins can add custom mapping types, engine scripts, etc. In this section we will learn how to use the Opensearch plugin. Below is an example script to install the plugin.

/usr/local/lib/opensearch/bin/opensearch-plugin install plugin-name

To clarify your understanding of plugins, below we show an example of how to install a plugin.

Example of how to install the plugin
root@ns3:~ # cd /usr/local/lib/opensearch/bin
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin install org.codelibs.opensearch:opensearch-analysis-fess:2.9.0
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin install org.codelibs.opensearch:opensearch-analysis-extension:2.9.0
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin install org.codelibs.opensearch:opensearch-minhash:2.9.0
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin install org.codelibs.opensearch:opensearch-configsync:2.9.0
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin install analysis-icu
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin install org.opensearch.plugin:opensearch-anomaly-detection:2.2.0.0

b. Remove plugin

After you learn how to install plugins, you will definitely want to know how to delete plugins. The script is almost the same as installing a plugin, here is the syntax for how to delete the plugin.

/usr/local/lib/opensearch/bin/opensearch-plugin remove plugin-name

Below, we demonstrate how to delete a plugin.

Remove plugin
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin remove org.codelibs.opensearch:opensearch-configsync:2.9.0
root@ns3:/usr/local/lib/opensearch/bin # ./opensearch-plugin remove analysis-icu
In this post we have explained in detail the process of creating an SSL certificate and how to install, remove the Opensearch plugin. We recommend that you read all the Opensearch documentation pages so that your understanding is broader and can make it easier to use Opensearch functions and features.
Iwan Setiawan

I Like Adventure: Mahameru Mount, Rinjani Mount I Like Writer FreeBSD

Post a Comment

Previous Post Next Post