Redo log concept

The redo log is a log file that is used to ensure that data is kept alive and not lost.

  • Redo logs take up very little space: the amount of storage required to store tablespace ids, page numbers, offsets, and values that need to be updated is very small. We’ll talk more about redo log formats later, but for now it’s good to know that a redo log doesn’t take up a lot of space.
  • Redo logs are written to disk sequentially: During a transaction, several redo logs may be written to disk in the order in which each statement is executed, using sequential IO.

Redo log format

There are many incorrect redo logs for different scenarios, but for the most part we use the generic redo log structure:

  • Type: indicates the redo log type. (There are more than 50 types in mysql5)
  • Space ID: indicates the ID of a tablespace.
  • Page number: indicates the page number.
  • Data: indicates the contents of the redo log.

There are so many different formats and concepts for redo logs that it’s not really interesting to read them unless you have to study them yourself. It’s enough to know that redo logs help you recover data.

Redo log writing process

redo log back

For better crash recovery, the creators of InnoDB put all the redo logs generated by THE MTR in 512-byte pages. To distinguish it from the pages in the table space we mentioned earlier, we call the page that stores the redo log a block. A redo log block looks like this:

The actual redo log is stored in a 496-byte log block body. The Log Block header and Log Block Trailer store some management information. Let’s take a look at what all this so-called management information is:

  • LOG_BLOCK_HDR_NO: Each block has a unique identifier greater than 0. This attribute represents the value of this identifier.
  • LOG_BLOCK_HDR_DATA_LEN: Indicates how many bytes have been used in the block, starting with 12 (because the log block body starts at the 12th byte). The value of this property increases as more redo logs are written to the block. If the log block body is fully written, the value of this property is set to 512.
  • LOG_BLOCK_FIRST_REC_GROUP: A single REDO log is also called a single redo log record. A SINGLE MTR produces multiple redo logs, which are called a redo log record group. LOG_BLOCK_FIRST_REC_GROUP represents the offset of the redo log group generated by the first MTR in the block (that is, the offset of the first REDO log generated by the first MTR in the block).
  • LOG_BLOCK_CHECKPOINT_NO: indicates the checkpoint number
  • Log Block Trailer: Check the validity.

Log buffer

Since redo logs are not friendly to disk, we added a buffer called a redo log buffer. The size of the log buffer can be specified with the innodb_log_buffer_size startup parameter, which defaults to 16MB in MySQL 5.7.21.

Redo log file

Redo log flush time

  • When the space of the log buffer is insufficient, the disk will be flushed about half of the vertical capacity. The capacity can be set.
  • Although the data can not be flushed to disk during transaction submission, in order to ensure persistence, data is flushed to disk during transaction submission
  • The background thread keeps brushing. There is a thread in the background that flushes the log buffer about once every second
  • Shut down the server properly
  • When we do something called checkpoint
  • Other special cases

Redo log file group

Redo files exist in groups. A redo file group consists of (ib_logfile0, ib_logFile1, ib_logfile2,…..) Ib_logfilen). The total size of the redo logfile is: innodb_log_file_size x innodb_log_files_in_group.

Redo log file format

A log buffer is essentially a contiguous piece of memory divided into 512-byte blocks. The nature of flushing redo logs to disk is to write a mirror of the block to a log file, so redo log files are actually made up of 512-byte blocks.

Each log file group has the same size and format and is made up of two parts:

  • The first 2048 bytes are the first 4 blocks that manage information
  • From 2048 bytes down the block image is used to store the block image in the log buffer.

The schematic diagram is as follows:Let’s talk about what the first four blocks do.

Log Sequence Number

Log Sequence Number is a global variable, which is the Log Sequence Number (LSN). (Default minimum 8704). We know that when we want to write redo logs to the log buffer, we do not write them one by one. We write them as a group of redo logs generated by MTR. And you actually write the log content in the body of the log block. However, the increment of LSN is calculated according to the log quantity actually written plus the occupied log block header and log Block Trailer. Each set of REDO logs generated by MTR has a unique LSN value. The smaller the LSN value is, the earlier the redo logs are generated.

flushed_to_disk_lsn

Redo logs are redo log files that are written to the log buffer before being flushed to disk. So the uncle who designed InnoDB proposed a global variable called buf_next_to_write, which marks which logs from the current log buffer have been flushed to disk. Let me draw a picture of this:InnoDB uses a global variable called flushed_to_disk_lsn to flushes the number of redo logs that are flushed to disk. InnoDB uses a global variable called flushed_to_disk_lsn to flushes the number of redo logs that are flushed to disk. When the system starts for the first time, the value of this variable is the same as the initial LSN value, which is 8704. As the system runs, redo logs are constantly written to the log buffer, but are not flushed to disk immediately. The LSN value differs from the flushed_to_disk_lsn value.

LSN in flush linked list

Dirty pages in the Flush list are sorted according to the chronological order in which the modification occurred, that is, by the VALUE of the LSN represented by OLdest_MODIFICATION. Pages that have been repeatedly updated are not inserted into the Flush list, but are updated with the value of the NEwest_modification property.