SeaTable Developer Edition is a free, easy-to-use, scalable data center product. It combines the ease of use of collaborative tables with the powerful data processing power of databases. SeaTable provides Restful apis that make it easy to write data over HTTP anytime, anywhere.

In this article, we will introduce how to use SeaTable to record error logs of various services on the server to visualize logs and prevent error messages from being ignored. The overall structure is shown as follows:

An example is shown below:

Features include

  • The logs are placed in a code block in Markdown format and recorded to a long text column in SeaTable
  • The SeaTable multi-view feature allows you to create multiple views with specific filtering criteria for each view, allowing you to quickly delete and view data

The following is a detailed introduction to the construction process.

Use FileBeat to send logs to Redis

Install filebeat

The following uses Ubuntu 20.04 as an example

apt update && apt install filebeat -y

Filebeat configuration file

vim /etc/filebeat/filebeat.yml

The value can be modified as required

Github.com/seatable/se…

Filebeat. Inputs: - type: the log paths: - / opt/seatable/Shared/seatable/logs/dtable_web log # log path tags: ["dtable-web"] # Service multiline.pattern '^[0-9]{4}-[0-9]{2}-[0-9]{2}|^[[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.negate: True # Multi-line match, date-starting information, e.g. '2021-10-12' or '[2021-10-12' multiline.match: after-type: log paths: - /opt/seatable/shared/seatable/logs/dtable-server.log tags: ["dtable-server"] multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}|^[[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.negate: true multiline.match: after output.redis: hosts: ["redis host"] password: "redis password" db: 0 timeout: 5 keys: - key: Contains: message: "[error]" # matches the keyword in the logCopy the code

The example configuration above is suitable for log time formats such as 2021-10-12 or [2021-10-12, which can recognize multiple lines of error logs.

Start the filebeat

Filebeat sends matching logs to Redis

service filebeat start

A script reads message records from Redis into SeaTable

SeaTable table

  1. To generate the API Token
  2. Create with the following, case sensitive
The name of the The first column type
Service Square root Text or radio
Time x Date, down to the minute
Log x Long text

The script

Install the software on the host. /opt/ is used as an example

cd /opt/

git clone [email protected]:seatable/seatable-syncer.git
Copy the code

Installing Python dependencies

cd seatable-syncer/log-sync/

pip3 install -r requirements.txt
Copy the code

The configuration file

vim log_syncer_settings.py

The value can be modified as required

# SeaTable server_URL = 'SeaTable server address 'API_token = 'table api_token' table_name = 'table name' # Redis redis_host = 'redis host' redis_db = 0 redis_port = 6379 redis_password = NoneCopy the code

The startup script

The script reads the matching log from Redis and sends it to SeaTable

python3 log_syncer.py

Log_syncer.py can handle logs starting with the following format

2020-01-01 01:02:02

2020-01-01T01:02:02

[2020-01-01 01:02:02

[2020-01-01T01:02:02
Copy the code

If your log format does not meet the above requirements, you can modify the log_syncer.py file.

Sharing and subsequent processing in SeaTable

By syncing the logs to a table in SeaTable, we can easily share the log information with groups and other individuals for collaborative problem analysis. You can also use reminder rules or automation rules to notify collaborators of log information in real time. You can also add automatic processing rules, such as logs that meet certain conditions, with labels to be ignored, and logs that meet certain conditions, with labels to be processed.

conclusion

With FileBeat, Redis, and Python scripts, you can collect logs from multiple servers into a SeaTable table. You can easily analyze and view logs by filtering conditions and multi-view functions. By sharing the collaboration function, we can easily achieve multi-person collaborative processing. I hope you can find more ways to use SeaTable to improve your productivity!