Elasticsearch Search Syntax and Boolean and Aggregation Searches

Here we explain how to do searches in ElasticSearch (ES). ES has a seemingly endless list of search options, which can seem overwhelming. So we will start with some simple examples and build from there.

We look at these items:

Prerequisites
If you have not installed ElasticSearch, you can follow these instructions to do that. And if you could also review this cheat sheet which provides some introductory.

Not that we are not using Kibana in these examples. Kibana is an optional graphical front-end for ElasticSearch. How to use KIbana is a topic unto itself.

Data
We need some data. So:

1. Download this program to create student data. Change the URL in the program to point to your instance of ElasticSearch: http://(your IP):9200/universities/universities/.
2. Then download this list of universities. Select the csv option and unzip the downloaded file. Copy the file InstitutionCampus.csv. Then download this program to load that data.

Run both programs. Both will run for several minutes since we want to create a large amount of data.

These programs create two different types of data, students and universities, in the same index, universities. We do this because with join type nested queries the data must be in the same index.

(This article is part of our ElasticSearch Guide. Use the right-hand menu to navigate.)

Basic Search Syntax

In order to get started, you need to understand something about the ES search syntax, aka Lucene Query. Here is a list of options:

Boolean Searches

With all the available options, there are multiple ways to achieve the same results, like finding schools in North Carolina.

This search finds all universities with both North and Carolina in the ParentName field.

curl -XGET --header 'Content-Type: application/json'  -d '{
"query" : { 
"bool": {
"must": [
{ "match" : { "ParentName": "North" }},
{ "match" : { "ParentName": "Carolina" }}
]
}
}
}'

results in:

      {
"_index" : "universities",
"_type" : "universities",
"_id" : "fc7549cbac32fc5770c9cb7bdc72d30de9fdc5a2",
"_score" : 10.899638,
"_source" : {
"ParentName" : "North Carolina Wesleyan College",
"DapipId" : "133942003",
"OpeId" : "",
"LocationType" : "Additional Location",
"AdminPhone" : "",
"Fax" : "",
"LocationName" : "Raleigh",
"Address" : "2000 Perimeter Park Dr, Morrisville, NC 27560",
"AdminName" : "",
"ParentDapipId" : "133942",
"UpdateDate" : "",
"AdminEmail" : "",
"GeneralPhone" : ""
}
},

You can limit the output to just certain fields using the _source parameter as shown below.

curl -XGET --header 'Content-Type: application/json'  -d '{
"_source" : ["ParentName"],
"query" : { 
"bool": {
"must": [
{ "match" : { "ParentName": "Florida" }},
{ "match" : { "ParentName": "University" }}
]
}
}
}'

Results in:

{
"_index" : "universities",
"_type" : "universities",
"_id" : "455f7ec172260cae0afad78c6b532920d65876d3",
"_score" : 5.5798583,
"_source" : {
"ParentName" : "Florida State University"
}
}