Configuration Optimization in MySQL Database
We may need to adjust the MySQL database default settings according to usage. Configuration settings can help the database work more efficiently. There are many tools used for MySQL database optimization. In this blog post, we do optimization with the below tools:
- MySQL Performance Tuning Primer
- MySQL Check
MySQL Performance Tuning Primer
We can easily do MySQL database analysis using the Tuning Primer script.
NOTE: Before running this tool, the MySQL database must be running without interruption for 48 hours.
We can perform the following database tests with Turning Primer:
- Slow Query Log
- Max Connections
- Worker Threads
- Key Buffer
- Query Cache
- Sort Buffer
- Temp Tables
- Table (Open & Definition) Cache
- Table Locking
- Table Scans (read_buffer)
- InnoDB Status
We can begin testing by downloading the tuning-primer.sh script at https://launchpad.net/mysql-tuning-primer/trunk and running:
# chmod u+x tuning-primer.sh
– MYSQL PERFORMANCE TUNING PRIMER — — By: Matthew Montgomery — MySQL Version 5.0.95 x86_64 Uptime = 6 days 1 hrs 19 min 4 sec Avg. qps = 39 Total Questions = 20796802 Threads Connected = 9 Server has been running for over 48hrs. It should be safe to follow these recommendations To find out more information on how each of these runtime variables effects performance visit: http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html Visit http://www.mysql.com/products/enterprise/advisors.html for info about MySQL’s Enterprise Monitoring and Advisory Service SLOW QUERIES The slow query log is NOT enabled. Current long_query_time = 10 sec. You have 3510 out of 20796847 that take longer than 10 sec. to complete Your long_query_time seems to be fine BINARY UPDATE LOG The binary update log is NOT enabled. You will not be able to do point in time recovery See http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html WORKER THREADS Current thread_cache_size = 0 Current threads_cached = 0 Current threads_per_sec = 1 Historic threads_per_sec = 1 Your thread_cache_size is fine MAX CONNECTIONS Current max_connections = 100 Current threads_connected = 9 Historic max_used_connections = 101 The number of used connections is 101% of the configured maximum. You should raise max_connections
The script indicates that the number of max_connection is insufficient in the row marked with red, and this number should be increased. So the test result is: MySQL connections have experienced problems with this parameter.
We can solve this by increasing the number of max_connections in MySQL configuration.
# vi /etc/my.cnf
# Under MySQL header;
Current max_connections = 100
INNODB STATUS Current InnoDB index space = 20 MCurrent InnoDB data space = 24 MCurrent InnoDB buffer pool free = 0 %Current innodb_buffer_pool_size = 8 MDepending on how much space your innodb indexes take up it may be safe to increase this value to up to 2 / 3 of total system memory MEMORY USAGE Max Memory Ever Allocated : 295 M Configured Max Per-thread Buffers : 274 M Configured Max Global Buffers : 17 M Configured Max Memory Limit : 292 M Physical Memory : 3.83 G Max memory limit seem to be within acceptable norms KEY BUFFER Current MyISAM index space = 488 M Current key_buffer_size = 7 M Key cache miss rate is 1 : 11 Key buffer free ratio = 80 % Your key_buffer_size seems to be fine QUERY CACHE Query cache is supported but not enabled Perhaps you should set the query_cache_size
Secondly, we received a warning due to the query_cache_size field. For this; we can set the query_cache_size limit to 128 MB as default, or we can define this field using the MySQL Tuner tool.
# vi /etc/my.cnf
# Under MySQL header;
By continuing to run the script, we continue to optimize.
SORT OPERATIONS Current sort_buffer_size = 2 MCurrent read_rnd_buffer_size = 256 KSort buffer seems to be fineJOINSCurrent join_buffer_size = 132.00 K You have had 7728 queries where a join could not use an index properly You have had 7212 joins without keys that check for key usage after each row You should enable “log-queries-not-using-indexes” Then look for non-indexed joins in the slow query log. If you are unable to optimize your queries you may want to increase your join_buffer_size to accommodate larger joins in one pass. Note! This script will still suggest raising the join_buffer_size when ANY joins not using indexes are found.
The script recommends adding the log-slow-queries parameter.
# vi /etc/mysql/my.cnf
log-slow-queries = /var/log/mysql/slow.log
OPEN FILES LIMIT Current open_files_limit = 1024 filesThe open_files_limit should typically be set to at least 2x-3xthat of table_cache if you have heavy MyISAM usage.Your open_files_limit value seems to be fine TABLE CACHE Current table_cache value = 64 tables You have a total of 12326 tables You have 64 open tables. Current table_cache hit rate is 0% , while 100% of your table cache is in use You should probably increase your table_cache
The warning indicates that there is insufficient space in table_cache.
12326 x 4 = 49304 → It is determined as 64535 because of 2 and multiplies.
NOTE: If we increase the space of table_cache too much, we should take into account that there will be RAM loss.
# vi /etc/mysql/my.cnf
TEMP TABLES Current max_heap_table_size = 16 MCurrent tmp_table_size = 32 MOf 356862 temp tables, 22% were created on disk Effective in-memory tmp_table_size is limited to max_heap_table_size. Created disk tmp tables ratio seems fine TABLE SCANS Current read_buffer_size = 128 K Current table scan ratio = 851 : 1 read_buffer_size seems to be fine TABLE LOCKING Current Lock Wait ratio = 1 : 104 You may benefit from selective use of InnoDB. If you have long running SELECT’s against MyISAM tables and perform frequent updates consider setting ‘low_priority_updates=1′ If you have a high concurrency of inserts on Dynamic row-length tables consider setting ‘concurrent_insert=2′.
thread_concurrency = 8 — → You can enter as CPU x 2.
Activating # Bin file will help repairing the erroneous files in the future.
# binary logging format — mixed recommended
# If there will be replication, we should add the corresponding server information
# remove comment sections. ID for Master is 1, for Slave is 2.
# server-id = 1
# CHANGE MASTER TO MASTER_HOST=’125.564.12.1′
innodb_data_home_dir = /var/lib/mysql innodb_data_file_path = ibdata1:10M:autoextend innodb_log_group_home_dir = /var/lib/mysql
innodb_buffer_pool_size = 256M innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 64M innodb_log_buffer_size = 8M innodb_flush_log_at_trx_commit = 1 innodb_lock_wait_timeout = 50
We should restart MySQL for the settings to take effect.
NOTE: I recommend checking the logs for any errors.
We’ll use MySQLCheck as another tool for MySQL database performance testing.
1. Database Control
# mysqlcheck -cA -u root -p
company.contacts OK mysql.columns_priv OK mysql.db OK mysql.event OK mysql.func OK mysql.general_log Error : You can't use locks with log tables. status : OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.servers OK mysql.slow_log Error : You can't use locks with log tables. status : OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK phpmyadmin.pma_bookmark OK phpmyadmin.pma_column_info OK phpmyadmin.pma_designer_coords OK phpmyadmin.pma_history OK phpmyadmin.pma_pdf_pages OK phpmyadmin.pma_relation OK phpmyadmin.pma_table_coords OK phpmyadmin.pma_table_info OK phpmyadmin.pma_tracking OK phpmyadmin.pma_userconfig OK
$ mysqlcheck -oA -u root -p
(-o for optimize)
$ mysqlcheck -oA -u root -p Enter password:
company.contacts Table is already up to date mysql.columns_priv Table is already up to date mysql.db OK mysql.event Table is already up to date mysql.func Table is already up to date mysql.general_log note : The storage engine for the table doesn't support optimize mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host Table is already up to date mysql.ndb_binlog_index Table is already up to date mysql.plugin Table is already up to date mysql.proc Table is already up to date mysql.procs_priv OK mysql.servers Table is already up to date mysql.slow_log note : The storage engine for the table doesn't support optimize mysql.tables_priv OK mysql.time_zone Table is already up to date mysql.time_zone_leap_second Table is already up to date mysql.time_zone_name Table is already up to date mysql.time_zone_transition Table is already up to date mysql.time_zone_transition_type Table is already up to date mysql.user OK phpmyadmin.pma_bookmark Table is already up to date phpmyadmin.pma_column_info Table is already up to date phpmyadmin.pma_designer_coords Table is already up to date phpmyadmin.pma_history Table is already up to date phpmyadmin.pma_pdf_pages Table is already up to date phpmyadmin.pma_relation Table is already up to date phpmyadmin.pma_table_coords Table is already up to date phpmyadmin.pma_table_info Table is already up to date phpmyadmin.pma_tracking Table is already up to date phpmyadmin.pma_userconfig OK
3. Repair of Damaged Databases
$ mysqlcheck -rA -u root -p
(-r for repair)
$ mysqlcheck -rA -u root -p Enter password:
company.contacts OK mysql.columns_priv OK mysql.db OK mysql.event OK mysql.func OK mysql.general_log OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.servers OK mysql.slow_log OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK phpmyadmin.pma_bookmark OK phpmyadmin.pma_column_info OK phpmyadmin.pma_designer_coords OK phpmyadmin.pma_history OK phpmyadmin.pma_pdf_pages OK phpmyadmin.pma_relation OK phpmyadmin.pma_table_coords OK phpmyadmin.pma_table_info OK phpmyadmin.pma_tracking OK phpmyadmin.pma_userconfig OK