Highly available distributed file system components

The file management mode of traditional stand-alone Web applications, such as uploading and using image and video files, is mostly stored locally on the server. However, this management mode cannot be applied to micro-service applications. On the one hand, microservice applications are distributed in a distributed environment where multiple copies can be deployed anytime, anywhere, so their media files must be stored in a unified place. On the other hand, the establishment of an independent and efficient file system is also an integral part of the high availability, high performance application platform.

If we rent a cloud service, we can use a distributed file system provided by a cloud service provider, such as Ali Cloud or Tencent Cloud’s object storage OSs.

Next, to deepen the reader’s understanding of distributed file systems, we use open source FastDFS to build a highly available distributed file system.

FastDFS architecture

FastDFS is a lightweight distributed file system. You can use FastDFS to build a highly available and sustainable distributed file system.

FastDFS consists of a Tracker and a Storage. Tracers are used to schedule requests from clients and record information about storage servers. Storage nodes are used to store files and their properties and synchronize files. File storage is also organized in groups (or volumes). Two or more trackers can form a highly available distributed file system, as shown in Figure 14-1.

Based on the architecture design shown in Figure 14-1, we will use four servers to build a highly available distributed file system, the code is shown below.

Tracker Server1: 10.10.10.22
Tracker Server2:10.10.10.32
Storagel:10.10.10.23
Storage2:10.10.10.33
Copy the code

In this architecture, adding storage nodes can improve and expand file access performance. After the distributed file system is installed, you can use Nginx to set up a load balancing service for file access on the Web side.

This chapter will use the installation packages shown below, which can be downloaded from the web.

Libfastcommon - 1.0.35. Tar. Gz fastdfs - 5.10, tar, gz pcre - 8.40. Tar. Gz nginx - 1.10.3. Tar. GzCopy the code

The installation of FastDFS

The following installation is performed on all four servers. Assume that the downloaded installation packages are stored in the /opt directory.

First, create a FastDFS and Nginx user:

Useradd fastdfs -m -s /sbin/nologinUserAdd nginx -m -s/sbin/nologinCopy the code

Then, install the build environment:

yum -y install gcc gcc+ gcc-c++ openssl openssl-devel pcre pcre-deve
Copy the code

Install libfastCommon and execute the following commands in that order:

Tar -zxvf libfastcommon-1.0.35.tar.gz CD libfastcommon-1.0.35. /makeCopy the code

Once done, create the soft link shown below so that the installer can find the relevant library files:

ln -sv /usr/include/fastcommon /usr/local/include/fastcommon
ln -sv /usr/include/fastdfs /usr/local/include/fastdfs
ln -sv /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
Copy the code

Execute the following commands in order to install FastDFS:

Gz CD fastdfS-5.10./make. Sh./makeCopy the code

After the installation is complete, a success flag is printed and the individual services are configured.

Tracking server configuration

On the Tracker Server machine, create the data store directory:

mkdir -p /data/fastdfs/tracker
Copy the code

Run the following commands in order to edit the tracker.conf configuration:

cd /etc/fdfs
cp tracker.conf.sample tracker.conf
vi /etc/fdfs/tracker.conf
Copy the code

Edit the following configurations in the configuration file:

Bind addr= # port=22122# connect timeout=30# Log data path base Path =/data/fastdfs/tracker# select group from path=/data/fastdfs/tracker# select group from path=/data/fastdfs/tracker 1: Sort by IP address 2: sort by weight store server=0 #storage Reserved storage space =10% HTTP. server port=8080Copy the code

Save and exit, and then start Tracker Server with the following command:

service fdfs trackerd start
Copy the code

Use the following command to view the ports that the Tracker Server is listening to:

netstat -unltplgrep fdfs
Copy the code

Storage Node Configuration

On the Storage server, create a data store directory as follows:

mkdir -p/data/fdfs storage/base
mkdir -p /data/fdfs storage/storage0
mkdir -p /data/fdfs_storage/storage1
Copy the code

Run the following commands in sequence to edit the storage node configuration:

cd /etc/fdfs
cp storage.conf.sample storage.conf
vi storage.conf
Copy the code

Edit the contents with the following commands:

#storage server owning group name=group1 # Bind IP address bind addr= # Storage server owning port=23000 # Connect timeout=30# Log data path Base Path =/data/ FDFS storage/base# Number of storage paths Store PathCount =2 Store Path0 =/data/ FDFS storage/ Storage0Store Pathl =/data/ FDFS storage/storagel# tracker server=10.10.10.22:22122tracker server= 10.10.10.32.22122Copy the code

After saving the editing, run the following command to start the storage node:

service fdfs_storaged start
Copy the code

Use the following command to check the running status of the service:

/usr/bin/fdfs monitor /etc/fdfs/storage.conf
Copy the code

Uploading a File

Now go back to the Tracker Server machine (such as IP address 10.10.10.22) and test uploading files using the following method.

Run the following commands in sequence to edit the client configuration:

cd /etc/fdfs
cp client.conf.sample client.conf
vi /etc/fdfs/client.conf
Copy the code

Amend the following:

Base path=/data/fastdfs/tracker # tracker server=10.10.10.22:22122 # tracker server IP address and port number HTTP. Tracker_server port=8080 # The HTTP port number on the Tracker server, which must be the same as the # setting of TrackerCopy the code

If a 1.png image file exists in /opt, run the following command to upload the file:

/usr/bin/fdfs upload file /etc/fdfs/client.conf /opt/1.png
Copy the code

If the file is uploaded successfully, the following information is displayed:

group1/M00/00/00/wKgBGFk3kUqACu9JAAGmMprynZs674.png
Copy the code

Three things to watch ❤️

If you find this article helpful, I’d like to invite you to do three small favors for me:

  1. Like, forward, have your “like and comment”, is the motivation of my creation.

  2. Follow the public account “Java rotten pigskin” and share original knowledge from time to time.

  3. Also look forward to the follow-up article ing🚀

  4. [666] Scan the code to obtain the learning materials package

Article is not original, and all of us feel while reading the article good remember thumb up forward + attention oh ~ this article source: www.toutiao.com/i6907843685…