-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[18.0][ADD] database_autovacuum_tuning #3523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 18.0
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| ========================== | ||
| Database Autovacuum Tuning | ||
| ========================== | ||
|
|
||
| .. | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| !! This file is generated by oca-gen-addon-readme !! | ||
| !! changes will be overwritten. !! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| !! source digest: sha256:162a7723c383e0b6239b46e50d6884767430de1a22f1c42a6c90d80e25b1b4fd | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
|
|
||
| .. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png | ||
| :target: https://odoo-community.org/page/development-status | ||
| :alt: Beta | ||
| .. |badge2| image:: https://img.shields.io/badge/licence-LGPL--3-blue.png | ||
| :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html | ||
| :alt: License: LGPL-3 | ||
| .. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github | ||
| :target: https://github.com/OCA/server-tools/tree/18.0/database_autovacuum_tuning | ||
| :alt: OCA/server-tools | ||
| .. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png | ||
| :target: https://translation.odoo-community.org/projects/server-tools-18-0/server-tools-18-0-database_autovacuum_tuning | ||
| :alt: Translate me on Weblate | ||
| .. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png | ||
| :target: https://runboat.odoo-community.org/builds?repo=OCA/server-tools&target_branch=18.0 | ||
| :alt: Try me on Runboat | ||
|
|
||
| |badge1| |badge2| |badge3| |badge4| |badge5| | ||
|
|
||
| Database Autovacuum Tuning helps administrators keep PostgreSQL healthy | ||
| by exposing recommended autovacuum settings in Odoo. It provides | ||
| guidance and documentation for sizing thresholds and scale factors so | ||
| large, busy databases avoid table bloat and excessive vacuum lag. Use it | ||
| to standardize autovacuum configuration across environments and speed up | ||
| maintenance operations without manual tuning. | ||
|
|
||
| This module is mostly useful for PostgreSQL <= 17. PostgreSQL 18.0 | ||
| introduces the ``autovacuum_vacuum_max_threshold`` parameter, which | ||
| already provides the capability this module targets. | ||
|
|
||
| The ``pgstattuple`` extension must be installed on the database. | ||
|
|
||
| **Table of contents** | ||
|
|
||
| .. contents:: | ||
| :local: | ||
|
|
||
| Usage | ||
| ===== | ||
|
|
||
| 1. Install the module on the database you want to tune. | ||
|
|
||
| 2. Go to Settings > Technical > Database Structure > Database Autovacuum | ||
| Tuning and review the recommended thresholds and scale factors. | ||
|
|
||
| 3. If needed, override the defaults using the following system | ||
| parameters: | ||
|
|
||
| - ``database_autovacuum_tuning.autovacuum_vacuum_max_threshold`` | ||
| - ``database_autovacuum_tuning.autovacuum_vacuum_analyze_max_threshold`` | ||
|
|
||
| 4. The configuration parameters are applied to tables by the daily cron | ||
| job. When the number of dead tuples in a table exceeds the vacuum | ||
| threshold, it applies the following configuration: | ||
|
|
||
| .. code:: sql | ||
|
|
||
| ALTER TABLE {schemaname}.{tablename} SET ( | ||
| autovacuum_vacuum_scale_factor = 0, | ||
| autovacuum_vacuum_threshold = %s, | ||
| autovacuum_analyze_scale_factor = 0, | ||
| autovacuum_analyze_threshold = %s | ||
| ) | ||
|
|
||
| 5. Monitor vacuum activity and table bloat, then adjust the settings if | ||
| your workload changes. | ||
|
|
||
| Bug Tracker | ||
| =========== | ||
|
|
||
| Bugs are tracked on `GitHub Issues <https://github.com/OCA/server-tools/issues>`_. | ||
| In case of trouble, please check there if your issue has already been reported. | ||
| If you spotted it first, help us to smash it by providing a detailed and welcomed | ||
| `feedback <https://github.com/OCA/server-tools/issues/new?body=module:%20database_autovacuum_tuning%0Aversion:%2018.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_. | ||
|
|
||
| Do not contact contributors directly about support or help with technical issues. | ||
|
|
||
| Credits | ||
| ======= | ||
|
|
||
| Authors | ||
| ------- | ||
|
|
||
| * Camptocamp | ||
|
|
||
| Contributors | ||
| ------------ | ||
|
|
||
| - Telmo Santos <telmo.santos@camptocamp.com> | ||
| - Alexandre Fayolle <alexandre.fayolle@camptocamp.com> | ||
|
|
||
| Maintainers | ||
| ----------- | ||
|
|
||
| This module is maintained by the OCA. | ||
|
|
||
| .. image:: https://odoo-community.org/logo.png | ||
| :alt: Odoo Community Association | ||
| :target: https://odoo-community.org | ||
|
|
||
| OCA, or the Odoo Community Association, is a nonprofit organization whose | ||
| mission is to support the collaborative development of Odoo features and | ||
| promote its widespread use. | ||
|
|
||
| This module is part of the `OCA/server-tools <https://github.com/OCA/server-tools/tree/18.0/database_autovacuum_tuning>`_ project on GitHub. | ||
|
|
||
| You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| from . import models |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # Copyright 2026 Camptocamp (https://www.camptocamp.com). | ||
| # License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl.html) | ||
|
|
||
| { | ||
| "name": "Database Autovacuum Tuning", | ||
| "summary": "Scheduled checks for Odoo autovacuum thresholds and scale factors", | ||
| "version": "18.0.1.0.1", | ||
| "author": "Camptocamp, Odoo Community Association (OCA)", | ||
| "website": "https://github.com/OCA/server-tools", | ||
| "category": "Tools", | ||
| "depends": [ | ||
| "base_setup", | ||
| ], | ||
| "data": [ | ||
| "data/config_parameter.xml", | ||
| "data/ir_cron.xml", | ||
| "security/ir.model.access.csv", | ||
| "views/res_config_settings_views.xml", | ||
| "views/database_autovacuum_tuning_views.xml", | ||
| ], | ||
| "development_status": "Alpha", | ||
| "license": "LGPL-3", | ||
| "installable": True, | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| <?xml version="1.0" encoding="utf-8" ?> | ||
| <odoo noupdate="1"> | ||
| <record id="config_autovacuum_vacuum_max_threshold" model="ir.config_parameter"> | ||
| <field | ||
| name="key" | ||
| >database_autovacuum_tuning.autovacuum_vacuum_max_threshold</field> | ||
| <field name="value">100000</field> | ||
| </record> | ||
| <record | ||
| id="config_autovacuum_vacuum_analyze_max_threshold" | ||
| model="ir.config_parameter" | ||
| > | ||
| <field | ||
| name="key" | ||
| >database_autovacuum_tuning.autovacuum_vacuum_analyze_max_threshold</field> | ||
| <field name="value">50000</field> | ||
| </record> | ||
| </odoo> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| <?xml version="1.0" encoding="utf-8" ?> | ||
| <odoo> | ||
| <record id="cron_database_autovacuum_tuning" model="ir.cron"> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can't we simply use the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could but we probably will be mixing 2 goals:
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with @santostelmo this would be confusing the purpose of both are totally different. |
||
| <field name="name">Database Autovacuum Tuning</field> | ||
| <field name="model_id" ref="model_database_autovacuum_tuning" /> | ||
| <field name="state">code</field> | ||
| <field name="code">model._db_autovacuum_tune()</field> | ||
| <field name="interval_number">1</field> | ||
| <field name="interval_type">days</field> | ||
| <field name="active">True</field> | ||
| <field name="user_id" ref="base.user_root" /> | ||
| </record> | ||
| </odoo> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| from . import database_autovacuum_tuning | ||
| from . import res_config_settings |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # Copyright 2026 Camptocamp (https://www.camptocamp.com). | ||
| # License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl.html) | ||
|
|
||
|
|
||
| from odoo import api, fields, models | ||
|
|
||
|
|
||
| class DatabaseAutovacuumTuning(models.Model): | ||
| _name = "database.autovacuum.tuning" | ||
| _description = "Database Autovacuum Tuning" | ||
|
|
||
| name = fields.Char(required=True, help="Table name") | ||
| vacuum_threshold = fields.Integer() | ||
| analyze_threshold = fields.Integer() | ||
|
|
||
| @api.model | ||
| def _db_autovacuum_tune(self): | ||
| vacuum_threshold, analyze_threshold = self._get_thresholds() | ||
| if vacuum_threshold <= 0: | ||
| return | ||
| results = self._get_tables_exceeding_dead_tuples(vacuum_threshold) | ||
| for schemaname, tablename, _ in results: | ||
| self.env.cr.execute( | ||
| f""" | ||
| ALTER TABLE {schemaname}.{tablename} SET ( | ||
| autovacuum_vacuum_scale_factor = 0, | ||
| autovacuum_vacuum_threshold = %s, | ||
| autovacuum_analyze_scale_factor = 0, | ||
| autovacuum_analyze_threshold = %s | ||
| ) | ||
| """, | ||
| (vacuum_threshold, analyze_threshold), | ||
| ) | ||
| self.sudo().create( | ||
santostelmo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| { | ||
| "name": f"{schemaname}.{tablename}", | ||
| "vacuum_threshold": vacuum_threshold, | ||
| "analyze_threshold": analyze_threshold, | ||
| } | ||
| ) | ||
|
|
||
| def _get_tables_exceeding_dead_tuples(self, vacuum_threshold): | ||
| query = """ | ||
| SELECT | ||
| t.schemaname, | ||
| t.tablename, | ||
| st.n_dead_tup | ||
| FROM pg_tables AS t | ||
| JOIN pg_stat_all_tables AS st | ||
| ON st.schemaname = t.schemaname | ||
| AND st.relname = t.tablename | ||
| WHERE t.tableowner = current_user | ||
| AND t.schemaname = 'public' | ||
| AND st.n_dead_tup > %s | ||
| ORDER BY t.schemaname, t.tablename | ||
| """ | ||
| self.env.cr.execute(query, (vacuum_threshold,)) | ||
| return self.env.cr.fetchall() | ||
|
|
||
| def _get_thresholds(self): | ||
| try: | ||
| vacuum_threshold = int( | ||
| self.env["ir.config_parameter"] | ||
| .sudo() | ||
| .get_param( | ||
| "database_autovacuum_tuning.autovacuum_vacuum_max_threshold", | ||
| default="0", | ||
| ) | ||
| ) | ||
| except ValueError: | ||
| vacuum_threshold = 0 | ||
|
|
||
| try: | ||
| analyze_threshold = int( | ||
| self.env["ir.config_parameter"] | ||
| .sudo() | ||
| .get_param( | ||
| "database_autovacuum_tuning.autovacuum_vacuum_analyze_max_threshold", | ||
| default="0", | ||
| ) | ||
| ) | ||
| except ValueError: | ||
| analyze_threshold = 0 | ||
|
|
||
| return vacuum_threshold, analyze_threshold | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Copyright 2026 Camptocamp (https://www.camptocamp.com). | ||
| # License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl.html) | ||
| from odoo import fields, models | ||
|
|
||
|
|
||
| class ResConfigSettings(models.TransientModel): | ||
| _inherit = "res.config.settings" | ||
|
|
||
| autovacuum_vacuum_max_threshold = fields.Integer( | ||
| config_parameter="database_autovacuum_tuning.autovacuum_vacuum_max_threshold", | ||
| ) | ||
| autovacuum_vacuum_analyze_max_threshold = fields.Integer( | ||
| config_parameter="database_autovacuum_tuning.autovacuum_vacuum_analyze_max_threshold", | ||
| ) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| [build-system] | ||
| requires = ["whool"] | ||
| build-backend = "whool.buildapi" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| - Telmo Santos \<<telmo.santos@camptocamp.com>\> | ||
| - Alexandre Fayolle \<<alexandre.fayolle@camptocamp.com>\> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| Database Autovacuum Tuning helps administrators keep PostgreSQL healthy by | ||
| exposing recommended autovacuum settings in Odoo. It provides guidance and | ||
| documentation for sizing thresholds and scale factors so large, busy databases | ||
| avoid table bloat and excessive vacuum lag. Use it to standardize autovacuum | ||
| configuration across environments and speed up maintenance operations without | ||
| manual tuning. | ||
|
|
||
| This module is mostly useful for PostgreSQL <= 17. PostgreSQL 18.0 introduces | ||
| the `autovacuum_vacuum_max_threshold` parameter, which already provides the | ||
| capability this module targets. | ||
|
|
||
| The `pgstattuple` extension must be installed on the database. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| 1. Install the module on the database you want to tune. | ||
| 2. Go to Settings > Technical > Database Structure > Database Autovacuum | ||
| Tuning and review the recommended thresholds and scale factors. | ||
| 3. If needed, override the defaults using the following system parameters: | ||
| - `database_autovacuum_tuning.autovacuum_vacuum_max_threshold` | ||
| - `database_autovacuum_tuning.autovacuum_vacuum_analyze_max_threshold` | ||
| 4. The configuration parameters are applied to tables by the daily cron job. | ||
| When the number of dead tuples in a table exceeds the vacuum threshold, it | ||
| applies the following configuration: | ||
|
|
||
| ```sql | ||
| ALTER TABLE {schemaname}.{tablename} SET ( | ||
| autovacuum_vacuum_scale_factor = 0, | ||
| autovacuum_vacuum_threshold = %s, | ||
| autovacuum_analyze_scale_factor = 0, | ||
| autovacuum_analyze_threshold = %s | ||
| ) | ||
| ``` | ||
| 5. Monitor vacuum activity and table bloat, then adjust the settings if your | ||
| workload changes. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink | ||
| access_database_autovacuum_tuning_system,access_database_autovacuum_tuning_system,model_database_autovacuum_tuning,base.group_system,1,0,1,1 |
Uh oh!
There was an error while loading. Please reload this page.