Introduction to the

Minio is an object storage system written by Go and based on the Apache License V2.0 open source protocol. It is designed for massive data storage, artificial intelligence, and big data analysis. It is fully compatible with Amazon S3 apis and is very suitable for storing large unstructured data ranging from tens of KB to a maximum of 5 TB. Is a small and beautiful open source distributed storage software.

The characteristics of

Minimalism but not simplicity: Minio adopts a simple and reliable clustering solution, eliminates complex large-scale cluster scheduling management, reduces risks and performance bottlenecks, and focuses on the core functions of the product to create a highly available cluster, flexible scalability and superior performance. Create multiple small – and medium-sized clusters that are easy to manage and can be aggregated into a large resource pool across data centers rather than a large-scale, centrally managed distributed cluster.

Minio supports cloud native, and can docking well with Kubernetes, Docker, Swarm programming systems to achieve flexible deployment. And the deployment is simple, only one executable file, few parameters, a command can start a Minio system. Minio adopts a metadata free database design for high performance, avoiding the metadata database from becoming a performance bottleneck of the entire system, and limiting faults to a single cluster so that other clusters are not involved. Minio is fully compatible with S3 interfaces and can be used as a gateway to provide S3 access externally.

Set up

1. Use docker containers

docker run -p 9000:9000 \
  -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" \
  -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
  minio/minio server /data
Copy the code

2.macOs

brew install minio/stable/minio
minio server /data
Copy the code

3.Linux

wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
./minio server /data
Copy the code

The default port of Minio is 9000, so you need to set the port on the firewall. If you access it in a browserhttp://127.0.0.1:9000 If yes, the installation is successful

distributed

The process of distributed setup is basically the same as that of single-machine mode. The Minio service automatically switches to single-machine or distributed based on the parameters passed in from the command line.

1. Prerequisites: Minio has been installed

2. To start a distributed Minio instance, you simply pass the disk location as a parameter to the Minio server command. Then, you need to run the same command on all other nodes.

Note:

  • All nodes in the distributed Minio need to have the same Access and Secret keys in order for these nodes to establish a connection.
  • MinIO can create erasure codes for each group of 4 to 16 disks. So the total number of disks you provide must be a multiple of one of these numbers
  • MinIO will select the maximum size of erasure code set according to the given total number of disks or nodes to ensure uniform distribution, that is, each node has the same number of disks in each set.
  • Each object is written to an EC collection, so the object is distributed on no more than 16 disks.
  • It is recommended that all nodes running the distributed MinIO setup be homogeneous, that is, the same operating system, the same number of disks, and the same network interconnection.
  • Distributed Minio uses a clean directory with no data in it. You can also share disks with other programs by giving a separate subdirectory to MinIO. For example, you can attach a disk to /export and pass /export/data to the MinIO Server.
  • The time difference between nodes in distributed Minio cannot exceed 15 minutes. You can use NTP to ensure time consistency.

Linux and MAC

export MINIO_ACCESS_KEY=<ACCESS_KEY> export MINIO_SECRET_KEY=<SECRET_KEY> minio server http://host{1... n}/export{1... Note: n and m represent positive integers in the example above, do not copy and paste them directly, you should change them to your desired values at deployment time. Note: {1... N} has three points! Use 2 points {1.. N} will be parsed by shell and cannot be transmitted to MinIO Server, affecting the sequence of erasure correction codes, and thus affecting performance and high availability. So always use the ellipsis {1... N} (3 points!) In order to obtain the best distribution of erasure codesCopy the code
Extend an existing distributed cluster

Minio supports expansion of existing clusters by specifying new clusters (erasure code mode)

export MINIO_ACCESS_KEY=<ACCESS_KEY> export MINIO_SECRET_KEY=<SECRET_KEY> minio server http://host{1... n}/export{1... m} http://host{o... z}/export{1... m}Copy the code