Today, the customer encountered a problem: the background processing was very slow due to the large amount of data for operation with multiple files. The browser showed 504 timeout. To verify whether the problem is caused by SQL statements, the following methods are used:

  • Query SQL execution records
  • The explain analysis
  • MySQL statement execution time

The following describes how to start the three methods.

Query SQL execution records

Check whether the log query function is enabled

show variables LIKE 'general%';
Copy the code

General_log: indicates whether the log recording function is enabled. The default value is OFF. General_log_file: indicates the path for storing logs

Enabling the Log Function

set GLOBAL general_log = 'ON'; 
Copy the code

Then check whether the function is successfully enabled

View SQL records at the specified path

The explain analysis

This command is used for most performance analysis. It can be used to view the execution effect of SQL statements and help you select better indexes and optimize statements.

grammar

Explain + SQL statements

Output:

Argument parsing

id SELECT identifier. This is the query serial number of SELECT
select_type

The SELECT type can be any of the following:

  • SIMPLE: Simple SELECT(no UNION or subquery)
  • PRIMARY: outermost SELECT
  • UNIONThe second or subsequent SELECT statement in :UNION
  • DEPENDENT UNIONThe second or subsequent SELECT statement in :UNION, depending on the query outside
  • UNION RESULT: what was the result of the UNION
  • SUBQUERY: the first SELECT in the subquery
  • DEPENDENT SUBQUERYThe first SELECT in the: subquery, depending on the outside query
  • DERIVEDSELECT(subquery FROM clause) FROM table
table

The table referenced by the output row

type

Join type. The various join types are given below, sorted from best to worst:

  • system: The table has only one row (= system table). This is a special case of the const join type.
  • const: The table has at most one matching row, which will be read at the start of the query. Because there is only one row, the column values on this row are considered constant by the rest of the optimizer. Const tables are fast because they only read once!
  • eq_ref: Reads a row from the table for each combination of rows from the previous table. This is probably the best join type, except for const.
  • ref: For each combination of rows from the previous table, all rows with matching index values are read from the table.
  • ref_or_null: This join type is the same as ref, but with the addition that MySQL can specifically search for rows containing NULL values.
  • index_merge: This join type indicates that the index merge optimization method is used.
  • unique_subqueryThis type replaces the ref IN subquery of the following form: Value IN (SELECT primary_key FROM single_table WHERE some_expr) unique_subquery is an index lookup function that can completely replace subqueries with higher efficiency.
  • index_subquery: This join type is similar to unique_subquery. Can replace IN subqueries, but only for non-unique indexes IN subqueries of the following form: value IN (SELECT key_column FROM single_table WHERE some_expr)
  • range: Retrieves only rows of a given range, using an index to select rows.
  • index: This join type is the same as ALL, except that only the index tree is scanned. This is usually faster than ALL because index files are usually smaller than data files.
  • ALL: Performs a full table scan for each combination of rows from the previous table.
possible_keys

Indicates which index MySQL can use to find rows in this table

key Shows the key (index) that MySQL actually decides to use. If no index is selected, the key is NULL.
key_len Displays the key length that MySQL decides to use. If the key is NULL, the length is NULL.
ref Shows which column or constant is used with the key to select rows from the table.
rows Shows the number of rows that MySQL thinks it must check when executing a query. Multiple rows are multiplied to estimate the number of rows to process.
filtered Shows the percentage estimate for the number of rows filtered by the criteria.
Extra

This column contains details about how MySQL solves queries

  • DistinctWhen MySQL finds the first matching row, it stops searching for more rows for the current row combination.
  • Not existsMySQL can optimize query LEFT JOIN. After finding a row matching the LEFT JOIN standard, no more rows are checked in the table for the previous row combination.
  • range checked for each record (index map: #)MySQL did not find a good index to use, but found that if the column values from the previous table are known, perhaps some indexes can be used.
  • Using filesortMySQL needs an extra pass to figure out how to retrieve rows in sort order.
  • Using index: Retrieves column information in a table from reading the actual rows using only the information in the index tree without further searching.
  • Using temporaryTo solve the query,MySQL needs to create a temporary table to hold the results.
  • Using whereThe :WHERE clause is used to restrict which row matches the next table or is sent to the customer.
  • Using sort_union(…) , Using union(…) , Using intersect(…)These functions explain how to merge index scans for the Index_Merge join type.
  • Using index for group-by: Similar to Using index for group-by,Using index for group-by indicates that MySQL has found an index that can be used to query all columns in a GROUP by or DISTINCT query without having to search hard disks to access the actual table.

MySQL statement execution time

The show profile and show Profiles statements display performance information about SQL statements executed during the current session.

Check whether profile is enabled

show variables like '%profil%';
Copy the code


Setting the Open state

set profiling = 1;
Copy the code

Check whether it is enabled again


Analyze the SQL statement after execution

After the command is executed, enter

show profiles;Copy the code

You can view the execution time of all SQL

show profile for query 1 
Copy the code

View the time details for each operation executing the first SQL statement.

show profile cpu, block io, memory,swaps,context switches,source for query 6;
Copy the code

You can view the resource consumption of an SQL statement, such as CPU and IO

show profile all for query 6Look at the first6All execution information of a statement.Copy the code

When the test is complete, turn off the parameters:

mysql> set profiling=0
Copy the code

The next article will talk about slow queries