preface

Welcome to our GitHub repository Star: github.com/bin39232820… The best time to plant a tree was ten years ago, followed by now

omg

Last time, I finished the series of Concurrent programming in Java. I was going to write the SERIES of JVM, because this is required in the interview. However, due to the personnel transfer of the company, I have to take over several more services, one of which is search service. Today I write this reason is good to learn the ElasticSearch again, because I have learned before taking over the over again, but for a long time useless, equivalent to forget about the same, just a lot of readers that want to study together, and that I was going to organize a series of articles from 0 begin to learn, to learn together.

What is a search?

Baidu: For example, when we want to find any information, we will search hundreds of times, for example, to find a favorite movie, or to find a favorite book, or to find an interesting news (mentioned the first impression of searching). Baidu! = search

  • Internet search: e-commerce websites, recruitment websites, news websites, various APPS
  • IT system search: OA software, office automation software, conference management, schedule management, project management.

Search, in any situation, is to find the information you want, in this case, you enter a paragraph of the keyword you want to search, and then expect to find some information about that keyword

What is full text search and Lucene?

  • Full text search, inverted index

Full-text retrieval refers to the computer indexing program through scanning every word in the article, to establish an index for each word, indicating the number and position of the word in the article, when the user queries, the retrieval program will be based on the index established in advance to search, and the search results feedback to the user retrieval method. This process is similar to looking up words through the search word table in a dictionary. Full text search data in a search engine database.

  • lucene

Lucene is a JAR package that contains all the code for building inverted indexes and searching, including algorithms. We use Java development, the introduction of Lucene JAR, and then based on Lucene API to carry out development can be.

Vague history

Years ago, an unemployed developer named Shay Banon, who had recently married, went with his wife to London to study cooking. During his job search, he started building an early version of Lucene in order to build a recipe search engine for his wife. Working directly on Lucene was difficult, so Shay began abstracting Lucene code so that Java programmers could add search capabilities to their applications. He released his first open source project, called “Compass.” Then Shay got a job in a distributed environment with high performance and in-memory data grids, so a high performance, real-time, distributed search engine was a natural necessity. He then decided to rewrite the Compass library to make it a standalone service called Elasticsearch.

The first public version appeared in February 2010, and since then Elasticsearch has become one of the most popular projects on Github with over 300 contributors. A company focused on Elasticsearch has been formed to provide commercial support while developing new features, but Elasticsearch will always be open source and available to everyone. Shay’s wife is still waiting for her recipe search…

What is Elasticsearch?

  • Elasticsearch, based on Lucene. Distributed Restful real-time search and analysis engine
  • Distributed real-time file storage where each field is indexed and searchable
  • High scalability, can be extended to hundreds of servers, processing PB level of structured or unstructured data
  • Elasticsearch for full text search, structured search, analysis/merge use

Elasticsearch features:

  • Elasticsearch has no typical transaction meaning (non-transactional)
  • Elasticsearch is a document-oriented database
  • Elasticsearch does not provide authorization or authentication features

What can Elasticsearch do?

  • You have an online marketplace that allows users to search for products you sell. In this example, you can use Elasticsearch to store your entire catalog and inventory and provide search and search autocomplete and search recommendations.
  • You want to collect logs or business data and analyze and mine that data for market trends, statistics, summaries, or anomalies. In this example, you can use the Logstash (part of the Elasticsearch Logstash/Kibana stack) to collect, aggregation, and parse your data, and then through the Logstash data into Elasticsearch. Once the data is in Elasticsearch, you can run searches and aggregations and mine any data you’re interested in.
  • You run a price alert platform that allows price-savvy customers to specify a rule, such as: “I have a preference for an electronic product and I want to alert any seller next month about how much the price of this electronic product is below.” In this example, you can grab all the sellers’ prices, put them into Elasticsearch and use the reverse search (filter/extractor) function of Elasticsearch to match the price changes in response to the user’s query and finally give the user a pop-up when a match is found.
  • You have analytics/business intelligence needs and want to quickly review, analyze, and graphically present, and query point-to-point problems on a large data set (imagine millions or tens of millions of records). In this example, you can use Elasticsearch to store your data and then use the Kibana (part of the Elasticsearch Logstash/Kibana stack) to build customized dashboard. This way you can visualize the data that is important to you. In addition, you can use Elasticsearch integration to present more complex business intelligence queries with your data.

How can I put it? In fact, ELK is a technical solution that can do a lot of things. At present we just put es entry, his search function to learn, use well.

This section describes how to install and configure Elasticsearch

Because bloggers docker, before you play is to use the docker, considering that many readers will not, however, this time I can use the native to install it, there will be several pit, I’ll give you the way, but I want to say is, the first step to learn a thing, is it, perhaps process will be very painful, but you must do not.

Single-node Installation

Elasticsearch is Java based, so install JDK 1.8 or higher before installing elasticSearch

Download address: www.oracle.com/technetwork…

JDK installation: Download — unzip — configure environment variables

Explain, the virtual machine of the blogger is Ubantu, memory to a 3G bar, because the blogger is not a professional tutorial, may say that this installation tutorial you have to combine a few blogs to complete, but the things on the Internet, are east and west gather together, this is you must experience, but the blogger as far as possible to do a good point

The environment variable is configured in /etc/profile. The environment variable is configured in /etc/profile

Vim /etc/profile export JAVA_HOME=/usr/local/jdk1.8.0_1 export CLASSPATH=$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib export PATH=$JAVA_HOME/bin:$PATH source /etc/profileCopy the code

Install ElasticSearch: Download — Unzip — Configure — launch

Wget tar ZXVF - https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.2.tar.gz Elasticsearch - 6.6.2. Tar. Gz CD elasticsearch - 6.6.2Copy the code

Here is the directory structure inside, in fact, do more, you will find that in fact are similar, every middleware is the same

Sudo vi config/elasticsearch.yml

Why don’t we just go ahead and see what happens

cd /bin
./elasticsearch
Copy the code

Start the

Obvious error, he says I can’t boot with root, let’s fix it first

So I need to create a new system run account for ElasticSearch

Groupadd ElasticSearch // Create a new elasticSearch user group useradd -g ElasticSearch ElasticSearch / / below the elasticsearch user group to establish a user elasticsearch chown -r elasticsearch: elasticsearch elasticsearch 6.6.2 /Copy the code

Then switch to the original account and start ElasticSearch

Su elasticsearch. / elasticsearch - 6.6.2 / bin/elasticsearchCopy the code

Max_map_count (65530) is too low, at least increase to 262144, then restart

Modify the configuration file as user root

vi /etc/sysctl.conf

vm.max_map_count=262144 

sysctl -p
Copy the code

Now let’s go ahead and start ElasticSearch with an account

Yeah, it worked. Good for you

Ik installation

Then anyway put ES installed, and then is to install the IK word segmentation bar, because ES is foreign open source, for Chinese, need to use the plug-in

  • First: download address: github.com/medcl/elast… In this case, you need to download the corresponding version of IK according to the version of your Es. Here, I used 6.6.2 Es, so download the iK-6.6.2.zip file.

  • Second: unzip –> copy the files to the es installation directory /plugins.

  • The third step is to restart es

ElasticSearch head installed

This is Google browser plug-in, can be used to query data, this installation is involved in Google plug-in installation, we baidu about ha.

So far, the installation is finished, we do this step first, what do not understand can be asked in the group, maybe not successful, because everyone configuration what is not necessarily, we grope, I installed for the first time, but made a few hours? That’s how it’s always been. Come on, guys.

At the end

Today, I briefly introduced ES, and then I made the things to be installed for you. If you have any questions about installation, you can ask them in the group. If I am free, I will definitely reply when I see them.

Daily for praise

Ok, everybody, that’s all for this article, you can see people here, they are real fans.

Creation is not easy, your support and recognition, is the biggest motivation for my creation, we will see in the next article

Six pulse excalibur | article “original” if there are any errors in this blog, please give criticisms, be obliged!