如何在MySQL中分配innodb_buffer_pool_size

时间  2019-11-06
标签 如何 mysql 分配 innodb buffer pool size 栏目 MySQL 繁體版
原文   原文链接

如何在MySQL中分配innodb_buffer_pool_size

innodb_buffer_pool_size是整个MySQL服务器最重要的变量。mysql

1. 为何须要innodb buffer pool？

在MySQL5.5以前，普遍使用的和默认的存储引擎是MyISAM。MyISAM使用操做系统缓存来缓存数据。InnoDB须要innodb buffer pool中处理缓存。因此很是须要有足够的InnoDB buffer pool空间。sql

2. MySQL InnoDB buffer pool 里包含什么？

  • 数据缓存
    InnoDB数据页面数据库

  • 索引缓存
    索引数据缓存

  • 缓冲数据
    脏页（在内存中修改还没有刷新(写入)到磁盘的数据）服务器

  • 内部结构
    如自适应哈希索引，行锁等。架构

3. 如何设置innodb_buffer_pool_size?

innodb_buffer_pool_size默认大小为128M。最大值取决于CPU的架构。在32-bit平台上，最大值为2**32 -1,在64-bit平台上最大值为2**64-1。当缓冲池大小大于1G时，将innodb_buffer_pool_instances设置大于1的值能够提升服务器的可扩展性。dom

大的缓冲池能够减少屡次磁盘I/O访问相同的表数据。在专用数据库服务器上，能够将缓冲池大小设置为服务器物理内存的80%。性能

3.1 配置缓冲池大小时，请注意如下潜在问题

  • 物理内存争用可能致使操做系统频繁的pagingspa

  • InnoDB为缓冲区和control structures保留了额外的内存，所以总分配空间比指定的缓冲池大小大约大10％。操作系统

  • 缓冲池的地址空间必须是连续的，这在带有在特定地址加载的DLL的Windows系统上多是一个问题。

  • 初始化缓冲池的时间大体与其大小成比例。在具备大缓冲池的实例上，初始化时间可能很长。要减小初始化时间，能够在服务器关闭时保存缓冲池状态，并在服务器启动时将其还原。

    • innodb_buffer_pool_dump_pct：指定每一个缓冲池最近使用的页面读取和转储的百分比。 范围是1到100。默认值是25。例如，若是有4个缓冲池，每一个缓冲池有100个page，而且innodb_buffer_pool_dump_pct设置为25，则dump每一个缓冲池中最近使用的25个page。
    • innodb_buffer_pool_dump_at_shutdown：默认启用。指定在MySQL服务器关闭时是否记录在InnoDB缓冲池中缓存的页面，以便在下次从新启动时缩短预热过程。
    • innodb_buffer_pool_load_at_startup：默认启用。指定在MySQL服务器启动时，InnoDB缓冲池经过加载以前保存的相同页面自动预热。 一般与innodb_buffer_pool_dump_at_shutdown结合使用。

增大或减少缓冲池大小时，将以chunk的形式执行操做。chunk大小由innodb_buffer_pool_chunk_size配置选项定义，默认值为128 MB。

缓冲池大小必须始终等于或者是innodb_buffer_pool_chunk_size * innodb_buffer_pool_instances的倍数。
若是将缓冲池大小更改成不等于或等于innodb_buffer_pool_chunk_size * innodb_buffer_pool_instances的倍数的值，
则缓冲池大小将自动调整为等于或者是innodb_buffer_pool_chunk_size * innodb_buffer_pool_instances的倍数的值。

innodb_buffer_pool_size能够动态设置，容许在不从新启动服务器的状况下调整缓冲池的大小。 能够经过状态变量Innodb_buffer_pool_resize_status报告在线调整缓冲池大小操做的状态。

mysql> show status like 'Innodb_buffer_pool_resize%';
+----------------------------------+-------+
| Variable_name                    | Value |
+----------------------------------+-------+
| Innodb_buffer_pool_resize_status |       |
+----------------------------------+-------+

 

3.2 配置示例

在如下示例中，innodb_buffer_pool_size设置为3G，innodb_buffer_pool_instances设置为8。innodb_buffer_pool_chunk_size默认值为128M。

3G是有效的innodb_buffer_pool_size值，由于3G是innodb_buffer_pool_instances = 8 * innodb_buffer_pool_chunk_size = 128M的倍数

# mysqld --innodb_buffer_pool_size=3G --innodb_buffer_pool_instances=8 &

mysql> show variables like 'innodb_buffer_pool%';

+-------------------------------------+----------------+
| Variable_name                       | Value          |
+-------------------------------------+----------------+
| innodb_buffer_pool_chunk_size       | 134217728      |
| innodb_buffer_pool_dump_at_shutdown | ON             |
| innodb_buffer_pool_dump_now         | OFF            |
| innodb_buffer_pool_dump_pct         | 25             |
| innodb_buffer_pool_filename         | ib_buffer_pool |
| innodb_buffer_pool_instances        | 8              |
| innodb_buffer_pool_load_abort       | OFF            |
| innodb_buffer_pool_load_at_startup  | ON             |
| innodb_buffer_pool_load_now         | OFF            |
| innodb_buffer_pool_size             | 3221225472     |
+-------------------------------------+----------------+
10 rows in set (0.01 sec)

 

在如下示例中，innodb_buffer_pool_size设置为3G，innodb_buffer_pool_instances设置为16. innodb_buffer_pool_chunk_size为128M。

3G不是有效的innodb_buffer_pool_size值，由于3G不是innodb_buffer_pool_instances = 16 * innodb_buffer_pool_chunk_size = 128M的倍数，能够看出innodb_buffer_pool_size的值自动调整到4GB

# mysqld --innodb_buffer_pool_size=3G --innodb_buffer_pool_instances=16 &

mysql> show variables like '%innodb_buffer_pool%';
+-------------------------------------+----------------+
| Variable_name                       | Value          |
+-------------------------------------+----------------+
| innodb_buffer_pool_chunk_size       | 134217728      |
| innodb_buffer_pool_dump_at_shutdown | ON             |
| innodb_buffer_pool_dump_now         | OFF            |
| innodb_buffer_pool_dump_pct         | 25             |
| innodb_buffer_pool_filename         | ib_buffer_pool |
| innodb_buffer_pool_instances        | 16             |
| innodb_buffer_pool_load_abort       | OFF            |
| innodb_buffer_pool_load_at_startup  | ON             |
| innodb_buffer_pool_load_now         | OFF            |
| innodb_buffer_pool_size             | 4294967296     |
+-------------------------------------+----------------+
10 rows in set (0.01 sec)

 

3.3 在线调整InnoDB缓冲池大小

mysql> SET GLOBAL innodb_buffer_pool_size = 3221225472

 

3.4 监控在线缓冲池调整进度

mysql> SHOW STATUS WHERE Variable_name='InnoDB_buffer_pool_resize_status';
+----------------------------------+----------------------------------------------------+
| Variable_name                    | Value                                              |
+----------------------------------+----------------------------------------------------+
| Innodb_buffer_pool_resize_status | Completed resizing buffer pool at 180824 15:05:03. |
+----------------------------------+----------------------------------------------------+

 

缓冲池大小调整进度也记录在服务器错误日志中。如下为增大时，记录的日志

2018-08-24T07:05:03.819049Z 2 [Note] InnoDB: Requested to resize buffer pool. (new size: 3221225472 bytes)
2018-08-24T07:05:03.819141Z 0 [Note] InnoDB: Resizing buffer pool from 2684354560 to 3221225472 (unit=134217728).
2018-08-24T07:05:03.819155Z 0 [Note] InnoDB: Disabling adaptive hash index.
2018-08-24T07:05:03.824902Z 0 [Note] InnoDB: disabled adaptive hash index.
2018-08-24T07:05:03.824933Z 0 [Note] InnoDB: Withdrawing blocks to be shrunken.
2018-08-24T07:05:03.824940Z 0 [Note] InnoDB: Latching whole of buffer pool.
2018-08-24T07:05:03.824959Z 0 [Note] InnoDB: buffer pool 0 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.839564Z 0 [Note] InnoDB: buffer pool 0 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.839594Z 0 [Note] InnoDB: buffer pool 1 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.848910Z 0 [Note] InnoDB: buffer pool 1 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.849046Z 0 [Note] InnoDB: buffer pool 2 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.856711Z 0 [Note] InnoDB: buffer pool 2 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.856741Z 0 [Note] InnoDB: buffer pool 3 : resizing with chunks 5 to 6.
2018-08-24T07:05:03.864867Z 0 [Note] InnoDB: buffer pool 3 : 1 chunks (8192 blocks) were added.
2018-08-24T07:05:03.864902Z 0 [Note] InnoDB: Completed to resize buffer pool from 2684354560 to 3221225472.
2018-08-24T07:05:03.864915Z 0 [Note] InnoDB: Re-enabled adaptive hash index.
2018-08-24T07:05:03.864935Z 0 [Note] InnoDB: Completed resizing buffer pool at 180824 15:05:03.

 

如下为减少时，记录的日志

2018-08-24T07:10:20.666816Z 2 [Note] InnoDB: Requested to resize buffer pool. (new size: 2684354560 bytes)
2018-08-24T07:10:20.666880Z 0 [Note] InnoDB: Resizing buffer pool from 3221225472 to 2684354560 (unit=134217728).
2018-08-24T07:10:20.666889Z 0 [Note] InnoDB: Disabling adaptive hash index.
2018-08-24T07:10:20.673416Z 0 [Note] InnoDB: disabled adaptive hash index.
2018-08-24T07:10:20.673508Z 0 [Note] InnoDB: Withdrawing blocks to be shrunken.
2018-08-24T07:10:20.673519Z 0 [Note] InnoDB: buffer pool 0 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.678441Z 0 [Note] InnoDB: buffer pool 0 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.678521Z 0 [Note] InnoDB: buffer pool 0 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.678919Z 0 [Note] InnoDB: buffer pool 0 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.678977Z 0 [Note] InnoDB: buffer pool 1 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.681644Z 0 [Note] InnoDB: buffer pool 1 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.682168Z 0 [Note] InnoDB: buffer pool 1 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.682235Z 0 [Note] InnoDB: buffer pool 1 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.682254Z 0 [Note] InnoDB: buffer pool 2 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.686560Z 0 [Note] InnoDB: buffer pool 2 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.686917Z 0 [Note] InnoDB: buffer pool 2 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.687002Z 0 [Note] InnoDB: buffer pool 2 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.687010Z 0 [Note] InnoDB: buffer pool 3 : start to withdraw the last 8192 blocks.
2018-08-24T07:10:20.690038Z 0 [Note] InnoDB: buffer pool 3 : withdrawing blocks. (8192/8192)
2018-08-24T07:10:20.690373Z 0 [Note] InnoDB: buffer pool 3 : withdrew 8192 blocks from free list. Tried to relocate 0 pages (8192/8192).
2018-08-24T07:10:20.690433Z 0 [Note] InnoDB: buffer pool 3 : withdrawn target 8192 blocks.
2018-08-24T07:10:20.690479Z 0 [Note] InnoDB: Latching whole of buffer pool.
2018-08-24T07:10:20.690498Z 0 [Note] InnoDB: buffer pool 0 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.693293Z 0 [Note] InnoDB: buffer pool 0 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.693357Z 0 [Note] InnoDB: buffer pool 1 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.695947Z 0 [Note] InnoDB: buffer pool 1 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.696011Z 0 [Note] InnoDB: buffer pool 2 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.698977Z 0 [Note] InnoDB: buffer pool 2 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.699288Z 0 [Note] InnoDB: buffer pool 3 : resizing with chunks 6 to 5.
2018-08-24T07:10:20.702088Z 0 [Note] InnoDB: buffer pool 3 : 1 chunks (8192 blocks) were freed.
2018-08-24T07:10:20.702398Z 0 [Note] InnoDB: Completed to resize buffer pool from 3221225472 to 2684354560.
2018-08-24T07:10:20.702413Z 0 [Note] InnoDB: Re-enabled adaptive hash index.
2018-08-24T07:10:20.703896Z 0 [Note] InnoDB: Completed resizing buffer pool at 180824 15:10:20.

 

4. 配置的innodb_buffer_pool_size是否合适？

当前配置的innodb_buffer_pool_size是否合适，能够经过分析InnoDB缓冲池的性能来验证。

可使用如下公式计算InnoDB缓冲池性能：

Performance = innodb_buffer_pool_reads / innodb_buffer_pool_read_requests * 100

innodb_buffer_pool_reads：表示InnoDB缓冲池没法知足的请求数。须要从磁盘中读取。

innodb_buffer_pool_read_requests：表示从内存中读取逻辑的请求数。

例如，在个人服务器上，检查当前InnoDB缓冲池的性能：

root@localhost [(none)] 15:35:31>show status like 'innodb_buffer_pool_read%';
+---------------------------------------+-------------+
| Variable_name                         | Value       |
+---------------------------------------+-------------+
| Innodb_buffer_pool_read_ahead_rnd     | 0           |
| Innodb_buffer_pool_read_ahead         | 0           |
| Innodb_buffer_pool_read_ahead_evicted | 0           |
| Innodb_buffer_pool_read_requests      | 4029033624  |
| Innodb_buffer_pool_reads              | 91661       |
+---------------------------------------+-------------+
5 rows in set (0.00 sec)


Performance = 91661 / 4029033624 * 100 = 0.0022750120389663

 

意味着InnoDB能够知足缓冲池自己的大部分请求。从磁盘完成读取的百分比很是小。所以无需增长innodb_buffer_pool_size值。

4.1 何时减少innodb_buffer_pool_size？

在专用MySQL服务器上，多余的innodb_buffer内存不会有问题，可是当使用共享服务器时，可能会有性能影响。由于空闲内存对其余程序和操做系统颇有用。

可使用SHOW ENGINE INNODB STATUS\G命令检查内存状态：

mysql> show engine innodb status\G

...
Total large memory allocated 26386366464
Dictionary memory allocated 23826297
Buffer pool size   1572672
Free buffers       8192
Database pages     1553364
Old database pages 573246
Modified db pages  36
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 881819, not young 18198964
0.02 youngs/s, 0.05 non-youngs/s
Pages read 681064, created 2749237, written 3988300
0.02 reads/s, 0.12 creates/s, 11.50 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 1553364, unzip_LRU len: 0
I/O sum[5152]:cur[0], unzip sum[0]:cur[0]
...

 

Free buffers :表示有多少空闲buffer。若是 此值长时间都较高，则能够考虑减少InnoDB缓冲池大小。

InnoDB buffer pool 命中率：

InnoDB buffer pool 命中率 = innodb_buffer_pool_read_requests / (innodb_buffer_pool_read_requests + innodb_buffer_pool_reads ) * 100

此值低于99%，则能够考虑增长innodb_buffer_pool_size。

5. InnoDB缓冲池状态变量有哪些？

能够运行如下命令进行查看：

root@localhost [(none)] 16:00:31>show global status like '%innodb_buffer_pool_pages%';
+----------------------------------+--------+
| Variable_name                    | Value  |
+----------------------------------+--------+
| Innodb_buffer_pool_pages_data    | 457    |
| Innodb_buffer_pool_pages_dirty   | 0      |
| Innodb_buffer_pool_pages_flushed | 36     |
| Innodb_buffer_pool_pages_free    | 163363 |
| Innodb_buffer_pool_pages_misc    | 0      |
| Innodb_buffer_pool_pages_total   | 163820 |
+----------------------------------+--------+
6 rows in set (0.00 sec)

 

说明：

  • Innodb_buffer_pool_pages_data
    InnoDB缓冲池中包含数据的页数。 该数字包括脏页面和干净页面。 使用压缩表时，报告的Innodb_buffer_pool_pages_data值可能大于Innodb_buffer_pool_pages_total（Bug＃59550）。

  • Innodb_buffer_pool_pages_dirty
    显示在内存中修改但还没有写入数据文件的InnoDB缓冲池数据页的数量（脏页刷新）。

  • Innodb_buffer_pool_pages_flushed
    表示从InnoDB缓冲池中刷新脏页的请求数。

  • Innodb_buffer_pool_pages_free
    显示InnoDB缓冲池中的空闲页面

  • Innodb_buffer_pool_pages_misc
    InnoDB缓冲池中的页面数量不少，由于它们已被分配用于管理开销，例如行锁或自适应哈希索引。此值也能够计算为Innodb_buffer_pool_pages_total - Innodb_buffer_pool_pages_free - Innodb_buffer_pool_pages_data。

  • Innodb_buffer_pool_pages_total
    InnoDB缓冲池的总大小，以page为单位。

  • innodb_buffer_pool_reads
    表示InnoDB缓冲池没法知足的请求数。须要从磁盘中读取。

  • innodb_buffer_pool_read_requests
    它表示从内存中逻辑读取的请求数。

  • innodb_buffer_pool_wait_free
    一般，对InnoDB缓冲池的写入发生在后台。 当InnoDB须要读取或建立页面而且没有可用的干净页面时，InnoDB首先刷新一些脏页并等待该操做完成。 此计数器计算这些等待的实例。 若是已正确设置innodb_buffer_pool_size，则此值应该很小。若是大于0，则表示InnoDb缓冲池过小。

  • innodb_buffer_pool_write_request
    表示对缓冲池执行的写入次数。

6. InnoDB缓冲池当前使用了多少实际GB内存？

经过将缓冲池中可用的数据与InnoDB页面（InnoDB缓冲池单位）大小相乘，咱们能够发现InnoDB缓冲池此时正在使用的实际内存。

set @ibpdata = (select variable_value from information_schema.global_status where variable_name = 'innodb_buffer_pool_pages_data');

ERROR 3167 (HY000): The 'INFORMATION_SCHEMA.GLOBAL_STATUS' feature is disabled; see the documentation for 'show_compatibility_56'

#从MySQL 5.7.6开始，GLOBAL_STATUS表中提供的信息从Performance Schema获取

mysql> set @ibpdata = (select variable_value from performance_schema.global_status where variable_name = 'innodb_buffer_pool_pages_data');

mysql> select @ibpdata;
+----------+
| @ibpdata |
+----------+
| 568      |
+----------+
1 row in set (0.00 sec)


mysql> set @idbpgsize = (select variable_value from performance_schema.global_status where variable_name = 'innodb_page_size');

mysql> select @idbpgsize;
+------------+
| @idbpgsize |
+------------+
| 16384      |
+------------+
1 row in set (0.00 sec)

mysql> set @ibpsize = @ibpdata * @idbpgsize / (1024*1024*1024);
Query OK, 0 rows affected (0.00 sec)

mysql> select @ibpsize;
+-----------------+
| @ibpsize        |
+-----------------+
| 0.0086669921875 |
+-----------------+
1 row in set (0.00 sec)
相关文章
相关标签/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公众号
   欢迎关注本站公众号,获取更多信息
联系我们 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程