Elasticsearch Containers: Nexcess Cloud & the API

Notice anything different?

We've enhanced the appearance of our portal and we're working on updating screenshots. Things might look different, but the functionality remains the same.

November 10, 2021

By Amy Moruzzi

Elasticsearch Containers: Nexcess Cloud & the API

Nexcess Cloud & Managed Apps: Optimize using Elasticsearch Containers & API! Enable a fast, scalable, open source search engine for large amounts of data!

Elasticsearch Containers - Using the API

What are Elasticsearch Containers?

Elasticsearch is a fast, scalable, open source search engine for large amounts of structured and unstructured data. Elasticsearch can be used for a variety of applications, and it is required for recent versions of Magento.

Elasticsearch containers are external instances of Elasticsearch within the Nexcess cloudhost network that are running in docker containers with minimal overhead. When they are enabled in the Nexcess portal for your application, an endpoint and port are provided that will be needed in order to utilize the API commands described below. Here is an example endpoint and port:

This tutorial will cover some basics of how to use the Elasticsearch API with Nexcess Managed Apps cloud containers. This covers how to connect and check uptime, verify resource usage, and change some settings for the Elasticsearch container.

How to Connect to the Elasticsearch Container

Nexcess Elasticsearch containers are not accessible from outside the internal network, and so it’s necessary to log in via SSH to the cloudhost account to connect to it directly. Next, check for a simple response using the following curl command, replacing with actual endpoint and port found in portal under Plans > Plan Dashboard > Containers:

curl -X GET $endpoint:$port

Example:

curl -X GET cg-8977-elasticsearch.nl-west-1.nxcli.net:63511
{
  "name" : "8487575a9cfc",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "Jzkz1EWFTPeuQFq1gQEf5w",
  "version" : {
  "number" : "7.9.3",
  "build_flavor" : "default",
  "build_type" : "docker",
  "build_hash" : "c4138e51121ef06a6404866cddc601906fe5c868",
  "build_date" : "2020-10-16T10:36:16.141335Z",
  "build_snapshot" : false,
  "lucene_version" : "8.6.2",
  "minimum_wire_compatibility_version" : "6.8.0",
  "minimum_index_compatibility_version" : "6.0.0-beta1"
},
  "tagline" : "You Know, for Search"
}

Container Diagnostics

Heap size: current memory usage and percent of total:

$ curl -s 'cg-9871-elasticsearch.au-south-1.nxcli.net:4250/_cat/nodes?v&h=hc,hp'

Response will look something like the following. Here 624.9mb are in use by the container, or 62% of the total available (1G):

hc hp
624.9mb 62

Uptime

This is pretty self-explanatory; the response should give the total uptime of the container:

curl -s ‘cg-8997-elasticsearch.nl-west-1.nxcli.net:63511/_cat/nodes?v&h=uptime’

10.3d

Elasticsearch “Yellow” Status

If the health statuses of the container indices are reporting as yellow, or logs are showing the error 'the shard cannot be allocated to the same node on which a copy of the shard already exists', the yellow status will not actually cause any issues because the container is running as a single node and so there are no replicas. The number_of_replicas setting is set to 1 by default, but it can be changed to 0 using the Elasticsearch API. Example of yellow status:

$ curl -s 'cg-8997-elasticsearch.nl-west-1.nxcli.net:63511/_cat/indices?v'
health status index   uuid       pri rep docs.count docs.deleted store.size pri.store.size
yellow open test_stg_product_1   rXnBOT2QQuKklj305xNVGw   5   1     111640      0   187.1mb 187.1mb
yellow open test_prod_product_2 jxQOE0KSQ-O4SqFrBt2C6w   5   1     232144     4844   528.5mb 528.5mb
yellow open test_prod_product_3 Xcd_RfRHQgKzXhonA36_Jw   5   1     227178     4895   514.5mb 514.5mb

Here is an example of how to check the current ‘number_of_replicas’ for the indices:

curl -s 'cg-8997-elasticsearch.nl-west-1.nxcli.net:63511/_all/_settings?pretty' | jq 'with_entries(.value |= .settings.index.number_of_replicas)'

{

"magento2m_product_5_v12": "1",

"magento2m_product_1_v12": "1",

"magento2m_product_3_v12": "1"

}

Setting 'number_of_replicas' to 0 for the indices will lead to Elasticsearch reporting green status, as long as there isn't another resource issue causing yellow/red status:

curl -X PUT "cg-8997-elasticsearch.us-west-1.nxcli.net:63511/_all/_settings" -H 'Content-Type: application/json' -d '{"index.number_of_replicas":0}'

{"acknowledged":true}

If yellow or red status is caused by hitting resource limits for the container, a larger size container can be selected in the portal. If the largest size container is already in use, please reach out to the support team for guidance.

Unable to connect at all?

If container was just enabled, wait at least 15 minutes to allow for the DNS record for the elasticsearch endpoint to propagate.
Assuming 15 minutes has elapsed, support may need to assist if connection is failing.

Working further with the Elasticsearch API

There is quite a bit more that can be done with the Elasticsearch API, both in terms of changing default settings and more complex queries. Below are a few links to documentation that can be starting points for a deeper dive.

Resources for More Information

Need more help? The Applications > Control Panel Tools > Client Portal sections within the Nexcess Knowledge Base are important resources for those seeking additional knowledge. Or, check out our related articles below.