What is ElasticSearch?
ElasticSearch is a distributed, RESTful search and analytics engine that can analyze big volumes of data in near real time (because there is an indexing delay of about a second). ElasticSearch also exposes a REST API through which you can index data and make it available for analysis. You don’t need to get your hands dirty with sending and receiving HTTP requests though, as there are two APIs, named Elastic.net and NEST, available for integration in your .net applications. ElasticSearch is available on Azure Marketplace, which you can purchase and deploy on your own cluster of VMs. It is also available as an installer package, which you can use to host it on your laptop or your local data center.
Kibana is an open-source analytics and visualization platform. The data that ElasticSearch indexes can be discovered and visualized with Kibana. The ElasticSearch template on Azure Marketplace comes bundled with Kibana and you only need to enable the option to deploy Kibana to your cluster.
ElasticSearch Cluster
ElasticSearch cluster uses sharding to distribute data across multiple nodes. The data is replicated to ensure high availability. ElasticSearch stores documents in indexes and uses them to locate documents. The ElasticSearch nodes are categorized by the roles that they fulfill, which are as follows.
A data node which can hold one or more shards that contain index data. It houses all the data and does not service any query requests.
A client node that does not hold index data, but that handles incoming requests made by the client applications by routing them to the appropriate data node.
A master node that does not hold index data, but performs cluster management operations, such as maintaining and distributing routing information around the cluster (the list of which nodes contain which shards), determining which nodes are available, relocating shards as nodes appear and disappear, and coordinating recovery after node failure. Multiple nodes can be configured as masters, but only one will actually be elected to perform the master functions. If this node fails, another election takes place and one of the other eligible master nodes will be elected to take over.
