Elasticsearch and Its interworking with Python

If you want to search data, it is inevitable to search, and search is inseparable from search engines. Baidu and Google are both very large and complex search engines, and they almost index all the open web pages and data on the Internet. Elasticsearch is a full text search engine that can quickly store, search, and analyze massive amounts of data.

Elasticsearch is an open source search engine built on top of Apache Lucene™, a full-text search engine library.

What is Lucene? Lucene may be the most advanced, high-performance, and full-featured search engine library in existence, open source or private, but it is just that. To use Lucene, we need to write Java and reference the Lucene package, and we need some understanding of information retrieval to understand how Lucene works, which is not that easy to use.

To solve this problem, Elasticsearch was born. Elasticsearch is also written in Java. It uses Lucene for indexing and searching internally, but its goal is to make full text retrieval easy. It provides a simple and consistent RESTful API for storage and retrieval.

So is Elasticsearch just a simplified Version of Lucene encapsulation? Elasticsearch isn’t just Lucene, and it’s not just a full-text search engine. It can be accurately described as follows:

All in all, it’s a pretty awesome search engine that wikipedia, Stack Overflow, and GitHub are all using to do searches.

Elasticsearch can be downloaded from the official Elasticsearch website:

At the same time, there are installation instructions on the official website.

To start elasticSearch, run bin/ ElasticSearch (Mac or Linux) or bin\ ElasticSearch. bat (Windows).

I am using A Mac and I recommend using Homebrew for Mac installation:

Elasticsearch will run on port 9200 by default

You can see something like this:

Elasticsearch has been installed and started successfully. The version of Elasticsearch is 6.2.4.

Let’s take a look at the basic concepts of Elasticsearch and how it interconnects with Python.

Elasticsearch has several basic concepts, such as nodes, indexes, documents, and so on. These concepts will help you to understand Elasticsearch.

Elasticsearch is essentially a distributed database that allows multiple servers to work together and each server can run multiple instances of Elasticsearch.

A single Instance of Elasticsearch is called a Node. A group of nodes forms a Cluster.

May I have an Inverted Index for Elasticsearch? May I have an Inverted Index for Elasticsearch? When looking for data, look up the index directly.

The top unit of Elasticsearch data management is called an Index, which is similar to the concept of MySQL, MongoDB, etc. It is also worth noting that the name of each Index (that is, database) must be lowercase.

The single record inside Index is called a Document. A number of documents form an Index.

Document is represented in JSON format, and here is an example.

Documents in the same Index are not required to have the same structure (scheme), but it is better to keep the same, so as to improve the search efficiency.

Document can be grouped, for example, in the weather Index, it can be grouped by city (Beijing and Shanghai), or by climate (sunny and rainy days). This grouping is called Type, which is a virtual logical grouping used to filter Document, similar to the data table in MySQL and Collection in MongoDB.

Different types should have similar schemas. For example, an ID field cannot be a string in one group and a number in another. This is a difference from tables in a relational database. Data of completely different natures (such as products and logs) should be stored as two indexes instead of two Types in one Index (although that is possible).

As planned, Elastic 6.x will only allow one Type per Index and will remove Type entirely.

That is, fields, each Document is similar to a JSON structure, it contains many fields, each field has its corresponding value, multiple fields form a Document, in fact, can be analogous to the fields in MySQL data table.

In Elasticsearch, documents belong to a Type that exists in an Index. We can draw some simple comparisons to traditional relational databases:

These are some of the basic concepts of Elasticsearch, compared to relational databases.

Elasticsearch provides a number of Restful apis for accessing and querying Elasticsearch. You can use curl to access Elasticsearch. So let’s go straight to the method of interfacing Elasticsearch with Python.

You can install Elasticsearch using a library of the same name in Python.

The official document is:

, all usage can be found here, and the rest of the article is based on official documentation.

Let’s look at how to create an Index. Here we create an Index named news:

If the creation is successful, the following information is displayed:

Plain text view

Copy the code

1	{ 'acknowledged' : True . 'shards_acknowledged' : True . 'index' : 'news' }

The result is returned in JSON format, with the Acknowledged field indicating that the creation operation was successful.

But if we execute the code again, we will return the following result:

It says create failed, status is 400, error cause Index already exists.

Note that the code uses ignore as 400, which means that if 400 is returned, the error will be ignored and no exception will be thrown.

If we omit the ignore argument:

An error is reported when executed again:

This will cause problems in the execution of the program, so we need to use the ignore parameter to rule out unexpected situations, so that the program can be executed without interruption.

Delete Index as follows:

Here, too, the ignore parameter is used to ignore the problem of the program breaking when the Index does not exist and the deletion fails.

If the deletion is successful, the following information is displayed:

If Index is already deleted, the following output is displayed:

This result indicates that the current Index does not exist and fails to be deleted. The result is also JSON with a status code of 400, but because we added the ignore parameter, the 400 status code is ignored, so the program executes the JSON result normally, instead of throwing an exception.

Elasticsearch, like MongoDB, can insert structured dictionary data directly when inserting data. Create () is used to insert data.

Here we declare a piece of news data, including the title and link, and then insert the data by calling create(). When we call create(), we pass in four parameters, index for the index name, doc_type for the document type, The body represents the content of the document, and the ID is the unique id of the data.

The running results are as follows:

The result field is created, indicating that the data is successfully inserted.

We can also insert data using index(), but unlike create(), create() requires us to specify an ID field to uniquely identify the data, whereas index() does not. If you do not specify an ID, an ID will be generated automatically. Calling the index() method is written as follows:

The create() method also calls index() internally, encapsulating the index() method.

Updating the data is also very simple, we also need to specify the id and content of the data, just call the update() method, the code is as follows:

Here we add a date field to the data and call the update() method, which results in the following:

As you can see, the index() method can do both of these things for us. If the data doesn’t exist, insert it, and if it does, update it, which is very convenient.

If you want to delete a piece of data, you can call the delete() method and specify the id of the data to be deleted.

The running results are as follows:

The result field is deleted, indicating that the deletion is successful. _version changes to 3 and 1 is added.

Elasticsearch is a very simple operation that can be done by a normal database such as MongoDB. What makes Elasticsearch special is its extremely powerful retrieval capabilities.

Install elasticSearch-analysis-IK plugin for ElasticSearch-analysis-IK

Elasticsearch -plugin (version 6.2.4) Elasticsearch (version 6.2.4) Elasticsearch (version 6.2.4)

Replace the version number here with the version number of your Elasticsearch.

Restart Elasticsearch after installation and it will automatically load the installed plug-ins.

First we create a new index and specify the fields to be split, as follows:

Here we first delete the previous index, then create a new index, and then update its mapping information. The mapping information specifies the segmentation field and the type of the field is text. The word analyzer and search word analyzer search_analyzer are ik_max_word, which uses the Chinese word analyzer plug-in we just installed. If not specified, the default English participle is used.

Let’s insert some new data:

Here we specify four data fields with title, URL, and date, and insert them into Elasticsearch using index() with name news and type Politics.

Next, let’s query the relevant content according to the keywords:

You can see that all four inserted data are retrieved:

You can see that the results are returned in the HITS field, with the total field indicating the number of results and max_score indicating the maximum match score.

Full text search is also available, which is where Elasticsearch comes in:

Select * from Elasticsearch (select * from Elasticsearch); select * from Elasticsearch (select * from Elasticsearch); select * from Elasticsearch (select * from Elasticsearch);

Here we see the result of the match has two, the first score of 2.54, the second score is 0.28, this is because the first match of the data contained in “China” and the “consulate” two words, the second match data does not include the “consulate”, but contains the word “China”, so has been retrieved, but had lower scores by comparison.

Therefore, it can be seen that the corresponding field will be retrieved in full text, and the results will be sorted according to the relevance of the keywords, which is a basic search engine prototype.

Elasticsearch also supports a wide variety of query methods.

This is the basic introduction to Elasticsearch and the basic use of The Python operation Elasticsearch, but this is only the basic features of Elasticsearch, there are many more powerful features waiting to be explored

Source:

https://cuiqingcai.com/6214.html