Project-driven learning, practice to test knowledge
Object storage has a wide range of applications, ranging from map beds and file services to big data applications. Whether it is an individual or an enterprise, it is very necessary to build an object storage server.
Many domestic cloud vendors provide object storage services: Ali OSS, Tencent COS, Huawei OBS and so on. They can store and manage massive data and are very convenient to build and use, but the price is not cheap. In order to save costs, many people start to build their own object storage servers, which can be either poor performance, or difficult to build, which is not satisfactory.
Is there a way to be free, convenient and high-performance? Of course there is, and this is MinIO.
MinIO claims to be the world’s fastest object storage server. On standard hardware, read/write speeds of 183 GB/s and 171 GB/s are self-evident, according to officials.
MinIO is also open source and free. Best of all, it is easy to build and use, making it the perfect choice for building your own object file storage server.
In this article, crab will teach you how to use MinIO’s own object storage server.
- MinIO construction and configuration;
- The realization of external link of file (graph bed);
- Integrates Spring Boot.
All the code in this article is on Github and can be cloned and run.
Installation and deployment
MinIO provides a variety of installation methods, Windows, Mac, Linux, Docker, etc. Here we choose the most simple and convenient Docker way.
If Docker is already installed on the server, we can install and enable MinIO with a single command:
docker run -d -v /data/minio:/data/minio -p 9000:9000 minio/minio server /data/minio
Copy the codeThis command maps the host /data/minio directory to the /data/minio directory in the container and enables port 9000. You can specify other directories and ports if you want.
MinIO also provides a GRAPHICAL user interface (GUI). You can enter IP address :9000 in a browser to view the GUI.
The default account password is minioadmin for both GUI operations and client operations. After passing the verification, you can enter the management interface:
To change the default account password, specify the following parameters when starting the container:
#You can customize the account password. The account password cannot contain less than 3 characters and the password cannot contain less than 8 characters
docker run -d -p 9000:9000 \
-v /data/minio:/data/minio \
-e "MINIO_ROOT_USER=root" \
-e "MINIO_ROOT_PASSWORD=rudecrab" \
minio/minio server /data/minio
Copy the codeThe graphical interface
Before writing the code to call the API, let’s take a look at the graphical operations.
There are two operation buttons in the lower right corner of the screen, one for creating a storage space Bucket and the other for uploading files.
Object storage server data is stored in buckets, so before uploading a file, we need to create a Bucket. Click the yellow button at the bottom to create:
The Bucket name is arbitrary; here we create a Bucket named File. Once created, you can see the created Bucket in the left navigation bar:
Then we can click the first yellow button in the lower left corner to upload the file:
After selecting the file to upload, you can view it in the file list:
Click the file operation button, you can directly preview, download, but also generate share links for others to access. Very convenient to complete the map bed function!
After clicking the share button, the share link can be generated.
Preview duration can be set for sharing links, which can be open for up to 7 days:
After clicking Copy Link to Copy the Link, you can view the effect in the browser:
This is a temporary link that will expire after a period of time. Is there any way to generate permalink? Of course there is.
Each Bucket can set its access permissions: read-only, write only, and read/write. After setting the permission, you can perform related operations even if you do not log in. All you need to do is set your Bucket to read-only and all files under that Bucket can be accessed directly, with no duration limit!
Click Bucket to configure:
In the dialog box that pops up, select Read Only and click the Add button to complete the configuration. Note that the access prefix in the first dialog box must be configured with an asterisk *, otherwise it will not be accessible:
After the configuration is complete, enter the file path in the browser to access:
Integrated Spring Boot
Graphical management interfaces are convenient, but sometimes we don’t want the user to directly manipulate the management interface. For example, if I just want to integrate a file system into my project that only provides an upload and download interface, then we need to write code.
MinIO provides SDKS for almost all major development languages, and Java is no exception.
To use MinIO’s SDK, you have to import its dependencies. To facilitate the operation of the I/O stream, we also import a toolkit:
<! -- MinIO SDK-->
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.0.3</version>
</dependency>
<! -- I/O tool kit, easy I/O operation -->
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.7</version>
</dependency>
Copy the codeEverything about MinIO is done through a MinioClient object, which is created as follows:
MinioClient minioClient = MinioClient.builder()
.endpoint("Service Address")
.credentials("Account"."Password")
.build();
Copy the codeYou can then use this object to specify a Bucket to do the corresponding operation. For example, to determine whether a Bucket already exists:
boolean found = minioClient.bucketExists(BucketExistsArgs.builder().bucket("Bucekt name").build());
Copy the codeIn the actual development, once the system runs, the service address, account number, password and other data generally will not change, even Bucekt will not change. In this case, you can configure the MinioClient object when the application starts, share one object with the entire system, and encapsulate common methods so that they can be called.
First, we define the service address data in the application.yml configuration file to avoid hard coding:
minio:
url: http://rudecrab.com:9000 # Service address
access: minioadmin # account
secret: minioadmin # your password
bucket: file # Bucket
Copy the codeThen create a MinioService class to encapsulate the code. Copy and paste the following code to use in your own project:
@Service
public class MinioService {
Logger log = LoggerFactory.getLogger(MinioService.class);
private final String bucket;
private final MinioClient minioClient;
public MinioService(@Value("${minio.url}") String url,
@Value("${minio.access}") String access,
@Value("${minio.secret}") String secret,
@Value("${minio.bucket}") String bucket) throws Exception {
this.bucket = bucket;
minioClient = MinioClient.builder()
.endpoint(url)
.credentials(access, secret)
.build();
// Initialize the Bucket
initBucket();
}
private void initBucket(a) throws Exception {
// Check whether the Bucket exists when the application starts
boolean found = minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build());
// If the Bucket does not exist, create a Bucket
if(! found) { minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucket).build()); log.info("Successfully created Bucket [{}]", bucket); }}/** * Upload file *@paramIs Input stream *@paramObject The name of the object (file) *@paramContentType File type */
public void putObject(InputStream is, String object, String contentType) throws Exception {
long start = System.currentTimeMillis();
minioClient.putObject(PutObjectArgs.builder()
.bucket(bucket)
.object(object)
.contentType(contentType)
.stream(is, -1.1024 * 1024 * 10) // Not less than 5 Mib
.build());
log.info("Successfully upload file to cloud [{}], time [{} ms]", object, System.currentTimeMillis() - start);
}
/** * get file stream *@paramObject The name of the object (file) *@returnFile stream * /
public GetObjectResponse getObject(String object) throws Exception {
long start = System.currentTimeMillis();
GetObjectResponse response = minioClient.getObject(GetObjectArgs.builder()
.bucket(bucket)
.object(object)
.build());
log.info([{} ms] [{} ms], object, System.currentTimeMillis() - start);
return response;
}
/** * Delete object (file) *@paramObject Object (file name) */
public void removeObject(String object) throws Exception {
minioClient.removeObject(RemoveObjectArgs.builder()
.bucket(bucket)
.object(object)
.build());
log.info("Delete Object [{}] successfully, object); }}Copy the codeWith business methods encapsulated, interfaces are easy to write.
Upload, download, delete interface is written as follows:
@RestController
@RequestMapping("/file")
public class FileController {
@Autowired
private MinioService minioService;
// Upload. The file name will be returned if the upload succeeds
@PostMapping
public String upload(MultipartFile file) throws Exception {
// Get the file name extension
String extension = FilenameUtils.getExtension(file.getOriginalFilename());
// To avoid duplicate file names, rename the file with UUID and remove the bars
String fileName = UUID.randomUUID().toString().replace("-"."") + "." + extension;
/ / upload
minioService.putObject(file.getInputStream(), fileName, file.getContentType());
// Returns the file name
return fileName;
}
// Download the file according to the file name
@GetMapping("{fileName}")
public void download(HttpServletRequest request, HttpServletResponse response, @PathVariable("fileName") String fileName) throws Exception {
// Set the response type
response.setCharacterEncoding(request.getCharacterEncoding());
response.setContentType("application/octet-stream");
response.setHeader("Content-Disposition"."attachment; filename=" + fileName);
// Get the file stream
GetObjectResponse objectResponse = minioService.getObject(fileName);
// Outputs the file stream to the response stream
IOUtils.copy(objectResponse, response.getOutputStream());
/ / end
response.flushBuffer();
objectResponse.close();
}
// Delete the file based on the file name
@DeleteMapping("{fileName}")
public String remove(@PathVariable("fileName") String fileName) throws Exception {
minioService.removeObject(fileName);
return "success"; }}Copy the codeAfter writing the interface, we start the project and test it on PostMan:
The file was successfully uploaded with the file name returned. In MinIO you can also see that the file has been uploaded successfully:
Let’s copy the file name and test the download interface:
After the request is sent, a download box pops up, proving that there is no problem with the download interface!
MinIO also supports JavaScript SDK. The front end can also be directly called. If you are interested, you can check out min.io/ on MinIO’s official website for more information.
In development, whether the front end calls MinIO directly or through the back end calls depends on the business.
finishing
MinIO also supports distributed deployment. If you’re interested, we’ll see you later.
All the code is already on Github and can be cloned and run. There are many other projects in the warehouse oh, welcome star~
I’m RudeCrab, a RudeCrab, and I’ll see you in the next article.
Follow the “RudeCrab” wechat official account to bully with crabs.