This is the 15th day of my participation in the August More Text Challenge. For details, see:August is more challenging

The advantage of the mongo

  • Easy extension
  • Large data volume, high performance
  • Flexible data model

Installation and startup

Install mongodb: sudo apt-get install -y mongodb-org

Install visual management interface: https://robomongo.org/download

See help: mongod — help

Start the service: sudo service mongod start

To stop the service, run the sudo service mongod stop command

To restart the service, run sudo service mongod restart

Check the process: ps ajx | grep mongod

The configuration file is located in /etc/mongod.conf

Default port: 27017

The location of the log: / var/log/mongo/mongod. Log

Mongodb Database Operations

Database operation

View the current database: db

To view all databases, run the show DBS /show databases command

To switch databases, run the use db_name command

Delete the current database: db.dropDatabase()

Set operations

When the collection does not exist, insert any data set automatically created.

Or create the collection manually: db.createcollection (name,[options])

These options:

Capped: If the default value is false, the capped parameter is not set. If the value is true, the capped parameter is set. Size: If the capped value is true, the capped parameter must be specifiedCopy the code

When the set exists:

View collections: Show Collections

Drop collection: db.collection name. drop()

Mongodb Data Types

Object ID: indicates the ID of the document

String: String, most commonly used, must be valid utF-8

Boolean: Stores a Boolean value,true or false

Integer: The Integer can be either 32-bit or 64-bit, depending on the server

Double: stores floating point values

Arrays: Arrays or lists that store multiple values into one key

Object: For embedded documents, where one value is one document

Null: Stores Null values

Timestamp: indicates the total number of seconds since 1970-1-1

Date: UNIX time format that stores the current Date or time

Note:

Create date statement as follows: The format of the parameter is YYYY-MM-DD new Date('2017-12-20'). Each document has an attribute, _id, to ensure that each document is unique If not, then MongoDB provides a unique _id for each document of type objectID. ObjectID is a 12-byte hexadecimal number: The first 4 bytes are the current timestamp, the next 3 bytes are the machine ID, the next 2 bytes are the MongoDB service process ID, and the last 3 bytes are simple incrementsCopy the code

Mongodb Data Operation

new

Db.set name. Insert (document)

Insert data (field _ID is updated if it exists) : db.collection name. save(document)

Here’s an example:

If you do not specify the _id parameter when you insert a document,MongoDB will assign a unique ObjectId to the document
db.xianyu.insert({name:"xianyuplus",age:"3"})
When inserting a document, you can specify the _id parameter
db.xianyu.insert({_id:"10001",name:"xianyuplus",age:"30"})
# update document with _id 1001
db.xianyu.save({_id:"10001",name:"xianyuplus",age:"40"})
Copy the code

The query

Query data: db.set name. find()

Here’s an example:

db.xianyu.find()
Copy the code

update

Update (

,

,{multi: < Boolean >})

Parameter query: query condition parameter update: update operator parameter multi: Optional. The default value is False, which means that only the first found record is updated. The value true means that all documents meeting the condition are updatedCopy the code

Here’s an example:

Original content: {"_id" : ObjectId("5b66f05f1194e110103bc283"),
    "name": "xianyuplus"."age": "40"
    }
# Replace name as xianyuplus with xianyuplus1
db.xianyu.update({name:"xianyuplus"},{name:"xianyuplus1"}) After operation: {"_id" : ObjectId("5b66f05f1194e110103bc283"),
    "name": "xianyuplus1"
    }
Copy the code

As you can see, simply updating the data with update causes the old data to be replaced with the new data, so we should use $set to specify the key for the update. Here’s an example:

Original content: {"_id" : ObjectId("5b66f05f1194e110103bc283"),
    "name": "xianyuplus"."age": "40"
    }
Xianyuplus = xianyuplus1
db.xianyu.update({name:"xianyuplus"}, {$set:{name:"xianyuplus1"}}) {"_id" : ObjectId("5b66f05f1194e110103bc283"),
    "name": "xianyuplus1"."age": "40"
    }
Copy the code

To update multiple pieces of data, run the multi:true parameter

Here’s an example:

# update all data with name value xianyuplus1
db.stu.update({},{$set:{name:"xianyuplus1"}},{multi:true})
Copy the code

Note: Multi update only works with $Operators, meaning that multi works as long as you use them with $operators.

delete

Remove (

,{justOne: < Boolean >})

Parameter query: optional, condition parameter justOne: optional if set to true or1, only one item is deleted. By default, fals indicates that multiple items are deletedCopy the code

Here’s an example:

Delete all data with name value xianyuplus
db.xianyu.remove({name:"xianyuplus"})
Copy the code

Mongodb Advanced Query

Mongodb Query Methods

Query document: db.collection name. find({conditional document})

Db.set name. findOne({conditional document})

Format query: db.collection name.find ({conditional document}).pretty()

Here’s an example:

Select * from xianyuplus where name is xianyuplus
db.xianyu.find({name:"xianyuplus"})

Select * from xianyuplus where name = xianyuplus
db.xianyu.findOne({name:"xianyuplus"})
Copy the code

Mongodb’s comparison operator

Equal to: as above chestnuts

Greater than: $gt (greater than)

Greater than or equal to: $gte (greater than equal)

$lt (less than)

$lte (less than equal)

$nt (not equal) $nt (not equal)

Here’s an example:

# Query data with age > 20
db.xianyu.find({age:{$gt:20}})
# Query data with age > 20
db.xianyu.find({age:{$gte:20}})
# Query data with age less than 20
db.xianyu.find({age:{$lt:20}})
# Query data with age less than or equal to 20
db.xianyu.find({age:{$lte:20}})
# Query data where age = 20
db.xianyu.find({age:{$ne:20}})
Copy the code

Mongodb logical operators

And: Write multiple field conditions in the find condition document

Or: Use $or

Here’s an example:

Select * from xianyuplus where name is xianyuplus and age is 20
db.xianyu.find({name:"xianyuplus",age:20})
Select * from xianyuplus where name is xianyuplus or age is 20
db.xianyu.find({$or:[{name:"xianyuplus"},{age:20}]})
Select * from xianyuplus where name is xianyuplus or age > 20
db.xianyu.find({$or:[{age:{$gt:20}},{name:"xianyuplus"}]})
# search for data with age = 20 or gender = male and name = xianyuplus
db.xianyu.find({$or:[{gender:"true"},{age:{$gte:18}}],name:"xianyuplus"})
Copy the code

Mongodb range operator

Use $in and $nin to determine whether it is in a certain range

Here’s an example:

Select * from age 18, age 28 where age 18, age 28
db.xianyu.find({age:{$in: []18.28}})
Copy the code

Mongodb uses regular expressions

Write regular expressions using // or $regex

Here’s an example:

# Query data whose name starts with xian
db.xianyu.find({name:/^xianyu/})
db.xianyu.find({name:{$regex:'^xianyu'}})
Copy the code

Mongodb paging and skipping

Db.set name. find().limit(NUMBER)

Skip (NUMBER) : db.set name. find().skip(NUMBER)

Here’s an example:

# Query the first 3 data
db.xianyu.find().limit(3)
# Query data after 3 entries
db.xianyu.find().skip(3)
#skip and limit can be used together to query 4,5 and 6 pieces of data
db.xianyu.find().skip(3).limit(3)
Copy the code

Mongodb custom query

Use $WHERE to customize the query, using JS syntax

Here’s an example:

// Query data with age greater than 30
db.xianyu.find({
    $where:function() {
        return this.age>30;}
})

Copy the code

Directing projection

Projection: Displays only the data fields you want to see in the query results.

Find ({},{field name :1,... })

The field that you want to display is set to 1, and the field that you don’t want to display is not set to 1. The _id field is a special field, so you need to set _id to 0.

# Select * from 'name' where 'age' = 'age'
db.xianyu.find({},{name:1})
Copy the code

Directing a sort

Sort: db.collection name.find ().sort({fields :1,… })

Set the values of the fields to be sorted: the ascending order is 1, and the descending order is -1

Here’s an example:

# Rank in descending order of sex and then ascending order of age
db.xianyu.find().sort({gender:-1,age:1})
Copy the code

Directing a count

Count: db.set name. find({condition}).count() db. Set name. Count ({condition})

Here’s an example:

# Query the number of data with age 20
db.xianyu.find({age:20}).count()
Select * from nan where age > 20 and gender = nan
db.xianyu.count({age:{$gt:20},gender:true})
Copy the code

Directing to heavy

Db. set name. Distinct (‘ deduplicated field ‘,{condition})

Here’s an example:

# Remove data with same hometown and age over 18
db.xianyu.distinct('hometown',{age:{$gt:18}})
Copy the code

Mongodb pipeline and aggregation

Aggregate is an aggregation pipeline based on data processing. Each document passes through a pipeline composed of multiple stages. The pipeline of each stage can be grouped and filtered, and then output corresponding results after a series of processing.

Usage: db.set name. Aggregate ({pipe :{expression}})

Common pipelines:

$match: filter data and output only documents that match the criteria $project: Modify the structure of the output document, such as renaming, adding, deleting fields, and creating calculation results $sort: $skip: skips a specified number of documents and returns the rest of the documents. $unwind: unwinds the array type fieldsCopy the code

Common expressions: Expression: “column name”

$sum: calculate the sum, $sum:1$avg: calculate the average $min: Gets the minimum value $max$push: Inserts values into an array in the result document. $first: gets the first document data according to the resource document ordering. $last: gets the last document data according to the resource document orderingCopy the code

The $group of aggregation

$group: Groups documents for easy counting

_id:”$field name”

Here’s an example:

# Group by hometown and count
db.xianyu.aggregate({$group:{_id:"$hometown", count:{$sum:1}}})
# Divide all contents in the set into a group and count the number
db.xianyu.aggregate({$group:{_id:null, count:{$sum:1}}})
Copy the code

Aggregation of $project

$project: Modify the structure of the input document, such as: rename, add, delete fields, etc

Here’s an example:

# Group by hometown and count
# Group output, display only the count field
 db.xianyu.aggregate(
        {$group:{_id:"$hometown", count:{$sum:1}}},
        {$project:{_id:0,count:1}})Copy the code

Aggregation of $match

$match: filters the data and outputs only the documents that match the criteria. The function is similar to find, but match is a pipe command and can send the results to the next pipe, whereas find cannot.

Here’s an example:

# Query age > 20
# Group by hometown and count
# Group output, display only the count field
 db.xianyu.aggregate(
        {$match:{age:{$gte:20}}},
        {$group:{_id:"$hometown", count:{$sum:1}}},
        {$project:{_id:0,count:1}})Copy the code

Aggregation of $sort

$sort: Output the sorted input document

Here’s an example:

# Query age > 20
# Group by hometown and count
# Group output, display only the count field
# Sort by ascending count order
 db.xianyu.aggregate(
        {$match:{age:{$gte:20}}},
        {$group:{_id:"$hometown", count:{$sum:1}}},
        {$project:{_id:0,count:1}},
        {$sort:{count:1}})Copy the code

The aggregation of
l i m i t with Limit and skip

$limit: Limits the number of documents returned by the aggregation pipeline

$skip: Skip the specified number of documents and return the remaining documents

Here’s an example:

# Query age > 20
# Group by hometown and count
# Sort by ascending count order
# Skip the previous document and return the second
 db.xianyu.aggregate(
        {$match:{age:{$gte:20}}},
        {$group:{_id:"$hometown", count:{$sum:1}}},
        {$sort:{count:1}},
        {$skip:1},
        {$limit:1})Copy the code

Aggregation of $unwind

$unwind: unwinds an array type field in a document into multiple pieces, each containing a value from the array

{$unwind:'$unwind '}}

Here’s an example:

db.xianyu.insert({_id:1,item:'t-shirt',size:['S'.'M'.'L']})
db.xianyu.aggregate({$unwind:'$size'}) Output: {"_id" : 1."item" : "t-shirt"."size" : "S" }
{ "_id" : 1."item" : "t-shirt"."size" : "M" }
{ "_id" : 1."item" : "t-shirt"."size" : "L" }
Copy the code

Precautions for aggregation use

  • $groupThe corresponding dictionary has several keys, and the result has several keys
  • Group according to need to put_idbehind
  • Take different field values needed to use
    . ‘ , `     gender.$age`
  • When you get a value from a dictionary nested within a dictionary$_id.country
  • The ability to group by multiple keys simultaneously{$group:{_id:{country: $field, type :"$field "}}}
    • The result is:{_id:{country:"",province:""}

Directing index

Usage: db.set. ensureIndex({property :1}), 1 for ascending, -1 for descending

Create unique index: db. Collection. EnsureIndex ({” properties “: 1}, {” unique” : true})

EnsureIndex ({” attribute “:1},{“unique”:true,”dropDups”:true})

EnsureIndex ({attribute :1,age:1})

View all indexes for the current collection: db.collection.getIndexes ()

DropIndex (‘ index name ‘)

Backup and restore mongodb data

Backing up mongodb Data

Backup: mongodump -h dbhost -d dbname -o dbdirectory

-d: indicates the name of the database to be backed up. -o: indicates the directory where the backup data is storedCopy the code

Mongodb Data Restoration

Mongorestore -h dbhost -d dbname –dir dbdirectory

-h: indicates the server address. -d: indicates the database instance to be restoreddir: Indicates the location of backup dataCopy the code

Mongodb interacts with Python

Installation and import

PIP install Pymongo

Import module: from Pymongo import MongoClient

instantiation

Instantiate an object to link to the database. The connection object takes two parameters, host and port.

from pymongo import MongoClient
class clientMongo:
    def __init__(self) :
      client = MongoClient(host="127.0.0.1", port=27017)
      # Select database and collection using [] parentheses
      self.cliention = client["xianyu"] ["xianyuplus"]
Copy the code

Insert data

Insert a single piece of data: ObjectId is returned

def item_inser_one(self) :
  ret = self.cliention.insert({"xianyu":"xianyuplus"."age":20})
  print(ret)
Copy the code

Insert multiple pieces of data:

def item_insert_many(self) :
  item_list = [{"name":"xianyuplus{}".format(i)} for i in range(10000)]
  items = self.cliention.insert_many(item_list)
Copy the code

Query data

Query a single piece of data:

def item_find_one(self) :
  ret = self.cliention.find_one({"xianyu":"xianyuplus"})
  print(ret)
Copy the code

Query multiple pieces of data:

def item_find_many(self) :
  ret = self.cliention.find({"xianyu":"xianyuplus"})
  for i in ret:
    print(i)
Copy the code

Update the data

Update a piece of data:

def item_update_one(self) :
  self.cliention.update_one({"xianyu":"xianyuplus"}, {"$set": {"xianyu":"xianyu"}})
Copy the code

Update all data:

def item_update(self) :
  self.cliention.update_many({"xianyu":"xianyuplus"}, {"$set": {"xianyu":"xianyu"}})
Copy the code

Delete the data

Delete a piece of data:

def item_delete_one(self) :
  self.cliention.delete_one({"xianyu":"xianyuplus"})
Copy the code

Delete data that meets the requirements:

def item_delete_many(self) :
  self.cliention.delete_many({"xianyu":"xianyuplus"})
Copy the code