I. Project background

For a GEO management system, its core point and bottleneck lies in the storage performance and query ability of database. On the one hand, storage services need to deal with the low-latency storage and read of massive data; on the other hand, storage services also need to provide efficient GEO+ multidimensional data retrieval. TableStore, as a Serverless distributed NoSQL database, fully meet the requirements of the system. Below we will build a [100 million GEO management system] based on TableStore;

Demand scenarios

A store search platform, provides a hundred million store information. Through the PC and mobile web pages provided by the platform, users can search for their favorite stores according to the combination of their own requirements. The platform needs to display the specific location of the store, the detailed information of the store and the jump of the store homepage on the map; Dimension 1: [distance within 1km] [per capita within 100] [highest score] [milk tea shop]; Dimension 2: [Hangzhou city] [highest score] [Shenyang *] store; . It is the core function of the GEO management solution to realize the fast and multi-dimensional GEO query function. For example: note: This sample provides [hundred million magnitude] store data. Official website console example address: project example

Store set up shop search system based on the form page view, the sample is embedded in a table to store the console, the user can log on to the console experience system (if stored to form a new user, click on the service experience, after opening, free of charge order data stored in a public instance, user experience consumption storage, flow, Cu).

TableStore scheme

The use of table storage (TableStore) research and development of multiple index (SearchIndex) scheme, can easily build a set of: 100 million magnitude store search system. Multi-index function can create GEO index, segmented string index, provide users with GEO search, multi-dimensional combination search and other capabilities, users can create at any time, the stock and increment data automatic synchronization. TableStore, as a fully hosted distributed NoSql data storage service with zero operation and maintenance provided by Ali Cloud, has the functions of [massive data storage], [automatic fragmentation of hot data], [multi-dimensional retrieval of massive data] and so on, which effectively solves the challenge of large expansion of GEO data. Users can create and open indexes only when needed. The TableStore ensures the consistency of data synchronization, which greatly reduces the workload of users’ scheme design, service operation and maintenance, code development and so on.

Two, construction preparation

If you have a good experience of [hundred million store search system] based on TableStore, and want to start the journey of building your own system, just follow the following steps to set up:

1. Enable table storage

The form storage service is opened through the console. The form storage is out-of-the-box (postpaid), and the free amount of the function test is provided to the users by the pay-per-quantity method. Form storage official website console, free limit description.

2. Create an instance

Use the console to create a table storage instance and select a Region that supports multivariate indexes. (SearchIndex is not yet commercially available, but will be available in Beijing, Shanghai, Hangzhou and Shenzhen, with the rest to follow)

After creating an example, submit work order to apply for multi-index function invitation test (open by default after commercialization, no charge for use).

• Invitation test address: Submit work order, choose “Form storage” > “Product function and feature Consultation” > “Create work Order”, the application content is as follows:
• Problem description: Please fill in the application for SearchIndex Test
• Confidential Information: Please fill in [region + instance name], for example: Shanghai +myInstanceName

3. SDK download

Using the SDK with multi-index (SearchIndex), the official website, temporarily added new features to the Java, Go, node.js SDK

java-SDK

<dependency>
    <groupId>com.aliyun.openservices</groupId>
    <artifactId>tablestore</artifactId>
    <version>4.8.0</version>
</dependency>
Copy the code

go-SDK

$ go get github.com/aliyun/aliyun-tablestore-go-sdk
Copy the code

Nodejs-SDK

$NPM install [email protected]Copy the code

C#-SDK

$ Install-Package Aliyun.TableStore.SDK -Version 4.1.0
Copy the code

4. Table design

Store retrieval system sample, only a simple use of a store table, mainly contains fields: store type, store name, store location, store average score, per capita consumption and so on. Table design is as follows: table name: GEO_positon

The column name	The data type	The index type	Fields that
_ID (primary key column)	String		MD5(pId) To avoid hot spots
pId	Stirng		The store number
type	String	KEYWORD	type
name	String	TEXT	Store name, TEXT type index can be fuzzy query, but can not sort
pos	String	GEO_POINT	Store location: “30.132,120.082”(latitude, accuracy)
point	double	DOUBLE	score
.	.	.	.

Iii. Start building (core code)

1. Create table

Users only need to create “Store Information Table” under the instance that has completed the invited test: create and manage the data table through the console (users can also directly create the data table through the SDK) :

Create table index

TableStore automatic full, incremental index data synchronization: users can create index through the console, manage index (can also create index through SDK)

3. Data import

Insert test data (100 million data were inserted in the console sample, and users can insert a small amount of test data themselves);

The store number	Store (MD5) (Primary key)	type	Shop name	Store location	Store ratings	Per capita consumption
o0057022192	0000000f470ef0f548b925ceffe1a7e3	Hangzhou cuisines	Han Village hangbang dishes	36.76613, 111.41461	2.87	63.67

4. Data reading

Data reads fall into two categories:

The primary key to read

Obtain primary key columns based on native table storage: getRow, getRange, batchGetRow, etc. Primary key read for index (automatic) backlookup, users can also provide primary key (order MD5) single query page, query speed is very fast under the order of billions. Single primary key query does not support multi-dimensional retrieval.

Index reading (store query)

Query: Search interface based on the new SearchIndex feature. Users are free to design multi-dimensional condition combination queries for index fields. By setting and selecting different query parameters, different query conditions and different sorting methods are constructed. Currently support: precise query, range query, prefix query, matching query, wildcard query, phrase matching query, word string query, and through Boolean and, or combination. For example, [milk tea shop within 1km of “36.76613,111.41461”], the query conditions are as follows: #### node.js code

client.search({ tableName: TABLE_NAME, indexName: INDEX_NAME, searchQuery: { offset: 0, limit: 10, query: { queryType: TableStore. QueryType. BOOL_QUERY, query: {mustQueries: [{/ / a query condition: TermQuery, cId the column values to match "c0001 QueryType" : TableStore.QueryType.TERM_QUERY, query: { fieldName: "type", term: QueryType: tableStore.querytype. GEO_DISTANCE_QUERY, query: {fieldName: "pos", centerPoint: "36.76613,111.41461", // Set centerPoint distance: 10000 // Set the distance condition to the central point, not exceeding 10000 m}},],}}, getTotalCount: true}, columnToGet: {returnType: TableStore.ColumnReturnType.RETURN_ALL } }, callback);Copy the code

# # # # Java code:

List<Query> mustQueries = new ArrayList<Query>();

TermQuery termQuery = new TermQuery();
termQuery.setFieldName("type"); TermQuery. SetTerm (ColumnValue. FromString (tea)); mustQueries.add(termQuery); GeoDistanceQuery geoDistanceQuery =new GeoDistanceQuery();
geoDistanceQuery.setFieldName("pos");
geoDistanceQuery.setCenterPoint("36.76613, 111.41461,");
geoDistanceQuery.setDistanceInMeter(1000);
mustQueries.add(geoDistanceQuery);

BoolQuery boolQuery = new BoolQuery();
boolQuery.setMustQueries(mustQueries);
Copy the code

Welcome aboard

In this way, the core code of the system has been completed, based on the table storage build hundreds of millions of “store search system”, is it very simple? If you are interested in TableStore, welcome to join TableStore Open Communication Group, group number: 11789671.