You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/content/stable/additional-features/auto-analyze.md
+13-26Lines changed: 13 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,8 +4,6 @@ headerTitle: Auto Analyze service
4
4
linkTitle: Auto Analyze
5
5
description: Use the Auto Analyze service to keep table statistics up to date
6
6
headcontent: Keep table statistics up to date automatically
7
-
tags:
8
-
feature: early-access
9
7
menu:
10
8
stable:
11
9
identifier: auto-analyze
@@ -22,40 +20,29 @@ Similar to [PostgreSQL autovacuum](https://www.postgresql.org/docs/current/routi
22
20
23
21
## Enable Auto Analyze
24
22
25
-
Before you can use the feature, you must enable it by setting `ysql_enable_auto_analyze_service` to true on all YB-Masters, and both `ysql_enable_auto_analyze_service` and `ysql_enable_table_mutation_counter` to true on all YB-TServers.
23
+
For new universes running v2025.2 or later, Auto Analyze is enabled by default when you deploy using yugabyted, YugabyteDB Anywhere, or YugabyteDB Aeon.
26
24
27
-
For example, to create a single-node [yugabyted](../../reference/configuration/yugabyted/) cluster with Auto Analyze enabled, use the following command:
25
+
In addition, when upgrading a deployment to v2025.2 or later, if the universe has the cost-based optimizer enabled (`on`), YugabyteDB will enable Auto Analyze.
26
+
27
+
You can explicitly enable or disable auto analyze by setting `ysql_enable_auto_analyze` on all yb-tservers.
28
+
29
+
For example, to create a single-node [yugabyted](../../reference/configuration/yugabyted/) cluster with Auto Analyze explicitly enabled, use the following command:
Enabling Auto Analyze on an existing cluster requires a rolling restart to set `ysql_enable_auto_analyze_service` and `ysql_enable_table_mutation_counter` to true.
36
-
37
36
## Configure Auto Analyze
38
37
39
-
You can control how frequently the service updates table statistics using the following YB-TServer flags:
40
-
41
-
-`ysql_auto_analyze_threshold` - the minimum number of mutations (INSERT, UPDATE, and DELETE) needed to run ANALYZE on a table. Default is 50.
42
-
-`ysql_auto_analyze_scale_factor` - a fraction that determines when enough mutations have been accumulated to run ANALYZE for a table. Default is 0.1.
43
-
44
-
Increasing either of these flags reduces the frequency of statistics updates.
45
-
46
-
If the total number of mutations for a table is greater than its analyze threshold, then the service runs ANALYZE on the table. The analyze threshold of a table is calculated as follows:
where `<table_size>` is the current `reltuples` column value stored in the `pg_class` catalog.
38
+
The auto analyze service counts the number of mutations (INSERT, UPDATE, and DELETE) to a table and triggers ANALYZE on the table automatically when certain thresholds are reached. You can configure this behavior using the following settings.
53
39
54
-
`ysql_auto_analyze_threshold` is important for small tables. With default settings, if a table has 100 rows and 20 are mutated, ANALYZE won't run as the threshold is not met, even though 20% of the rows are mutated.
40
+
A table needs to accumulate a minimum number of mutations before it is considered for ANALYZE. This minimum is the sum of:
55
41
56
-
On the other hand, `ysql_auto_analyze_scale_factor` is especially important for big tables. If a table has 1,000,000,000 rows, 10% (100,000,000 rows) would have to be mutated before ANALYZE runs. Set the scale factor to a lower value to allow for more frequent statistics collection for such large tables.
42
+
- A fraction of the table size. This is controlled by [ysql_auto_analyze_scale_factor](../../reference/configuration/yb-tserver/#ysql-auto-analyze-scale-factor). This setting defaults to 0.1, which translates to 10% of the current table size. Current table size is determined by the [reltuples](https://www.postgresql.org/docs/15/catalog-pg-class.html#:~:text=CREATE%20INDEX.-,reltuples,-float4) column value stored in the `pg_class` catalog entry for that table.
43
+
- A static count of [ysql_auto_analyze_threshold](../../reference/configuration/yb-tserver/#ysql-auto-analyze-threshold) (default 50) mutations. This setting ensures that small tables are not aggressively ANALYZED because the scale factor requirement is easily met.
57
44
58
-
In addition, `ysql_auto_analyze_batch_size` controls the maximum number of tables the Auto Analyze service tries to analyze in a single ANALYZE statement. The default is 10. Setting this flag to a larger value can potentially reduce the number of YSQL catalog cache refreshes if Auto Analyze decides to ANALYZE many tables in the same database at the same time.
45
+
Separately, Auto Analyze also considers cooldown settings for a table so as to not trigger ANALYZE aggressively. After every run of ANALYZE on a table, a cooldown period is enforced before the next run of ANALYZE on that table, even if the mutation thresholds are met. The cooldown period starts from [ysql_auto_analyze_min_cooldown_per_table](../../reference/configuration/yb-tserver/#ysql_auto_analyze_min_cooldown_per_table) (default: 10 seconds) and exponentially increases to [ysql_auto_analyze_max_cooldown_per_table](../../reference/configuration/yb-tserver/#ysql_auto_analyze_max_cooldown_per_table) (default: 24 hours). Cooldown values for a table do not reset. This means that in most cases, it is expected that, after a while, a frequently updated table is only analyzed once every `ysql_auto_analyze_max_cooldown_per_table` period.
59
46
60
47
For more information on flags used to configure the Auto Analyze service, refer to [Auto Analyze service flags](../../reference/configuration/yb-tserver/#auto-analyze-service-flags).
61
48
@@ -94,4 +81,4 @@ SELECT reltuples FROM pg_class WHERE relname = 'test';
94
81
95
82
## Limitations
96
83
97
-
Because ANALYZE is a DDL statement, it can cause DDL conflicts when run concurrently with other DDL statements. As Auto Analyze runs ANALYZE in the background, you should turn off Auto Analyze if you want to execute DDL statements. You can do this by setting `ysql_enable_auto_analyze_service`to false on all YB-TServers at runtime.
84
+
ANALYZE is technically considered a DDL statement (schema change) and normally conflicts with other [concurrent DDLs](../best-practices-operations/administration/#concurrent-ddl-during-a-ddl-operation). However, when run via the auto analyze service, ANALYZE can run concurrently with other DDL. In this case, ANALYZE is pre-empted by concurrent DDL and will be retried at a later point. However, when [transactional DDL](../explore/transactions/transactional-ddl/) is enabled (off by default), certain kinds of transactions that contain DDL may face a `kConflict` error when a background ANALYZE from the auto analyze service interrupts this transaction. In such cases, it is recommended to disable the auto analyze service explicitly and trigger ANALYZE manually. Issue {{<issue 28903>}} tracks this scenario.
Copy file name to clipboardExpand all lines: docs/content/stable/reference/configuration/yb-master.md
+4-6Lines changed: 4 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1058,15 +1058,13 @@ Default: `true`
1058
1058
1059
1059
## Auto Analyze service flags
1060
1060
1061
-
{{<tags/feature/ea idea="590">}}To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
1061
+
To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
1062
1062
1063
-
See also [Auto Analyze Service TServer flags](../yb-tserver/#auto-analyze-service-flags).
1063
+
Auto analyze is automatically enabled when the [cost-based optimizer](../../../architecture/query-layer/planner-optimizer/) (CBO) is enabled by setting the [yb_enable_cbo](../tb-tserver/#yb_enable_cbo) flag to `on`.
{{<tags/feature/ea idea="590">}}Enable the Auto Analyze service, which automatically runs ANALYZE to update table statistics for tables that have changed more than a configurable threshold.
1068
-
1069
-
Default: false
1067
+
Use [ysql_enable_auto_analyze](../yb-tserver#ysql_enable_auto_analyze) on yb-tservers instead.
Copy file name to clipboardExpand all lines: docs/content/stable/reference/configuration/yb-tserver.md
+52-28Lines changed: 52 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2131,70 +2131,78 @@ Default: `legacy_mode`
2131
2131
2132
2132
Enables the YugabyteDB [cost-based optimizer](../../../architecture/query-layer/planner-optimizer/) (CBO). Options are `on`, `off`, `legacy_mode`, and `legacy_stats_mode`.
2133
2133
2134
-
When enabling CBO, you must run ANALYZE on user tables to maintain up-to-date statistics.
2134
+
When CBO is enabled (set to `on`), [auto analyze](#auto-analyze-service-flags) is also enabled automatically. If you disable auto analyze explicitly, you are responsible for periodically running ANALYZE on user tables to maintain up-to-date statistics.
2135
2135
2136
2136
For information on using this parameter to configure CBO, refer to [Enable cost-based optimizer](../../../best-practices-operations/ysql-yb-enable-cbo/).
2137
2137
2138
2138
### Auto Analyze service flags
2139
2139
2140
-
{{<tags/feature/ea idea="590">}}To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
2141
-
2142
-
{{< note title="Note" >}}
2140
+
To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
2143
2141
2144
-
To fully enable the Auto Analyze service, you need to enable `ysql_enable_auto_analyze_service` on all YB-Masters and YB-TServers, and `ysql_enable_table_mutation_counter` on all YB-TServers.
2142
+
Auto analyze is automatically enabled when the [cost-based optimizer](../../../best-practices-operations/ysql-yb-enable-cbo/) (CBO) is enabled ([yb_enable_cbo](#yb_enable_cbo) is set to `on`).
2145
2143
2146
-
{{< /note >}}
2144
+
In v2025.2 and later, CBO and Auto Analyze are enabled by default in new universes when you deploy using yugabyted, YugabyteDB Anywhere, or YugabyteDB Aeon. In addition, when upgrading a deployment to v2025.2 or later, if the universe has the cost-based optimizer enabled (`on`), YugabyteDB will enable Auto Analyze.
2147
2145
2148
-
See also [Auto Analyze Service Master flags](../yb-master/#auto-analyze-service-flags).
2146
+
To explicitly control the service, you can set the `ysql_enable_auto_analyze` flag.
2149
2147
2150
-
##### --ysql_enable_auto_analyze_service
2148
+
##### --ysql_enable_auto_analyze
2151
2149
2152
2150
{{% tags/wrap %}}
2153
-
{{<tags/feature/ea idea="590">}}
2154
-
{{<tags/feature/t-server>}}
2155
-
{{<tags/feature/restart-needed>}}
2151
+
2156
2152
Default: `false`
2157
2153
{{% /tags/wrap %}}
2158
2154
2159
2155
Enable the Auto Analyze service, which automatically runs ANALYZE to update table statistics for tables that have changed more than a configurable threshold.
2160
2156
2161
-
##### --ysql_enable_table_mutation_counter
2157
+
##### --ysql_auto_analyze_threshold
2162
2158
2163
2159
{{% tags/wrap %}}
2164
2160
2161
+
Default: `50`
2162
+
{{% /tags/wrap %}}
2165
2163
2166
-
Default: `false`
2164
+
The minimum number of mutations needed to run ANALYZE on a table. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
2165
+
2166
+
##### --ysql_auto_analyze_scale_factor
2167
+
2168
+
{{% tags/wrap %}}
2169
+
2170
+
Default: `0.1`
2167
2171
{{% /tags/wrap %}}
2168
2172
2169
-
Enable per table mutation (INSERT, UPDATE, DELETE) counting. The Auto Analyze service runs ANALYZE when the number of mutations of a table exceeds the threshold determined by the [ysql_auto_analyze_threshold](#ysql-auto-analyze-threshold) and [ysql_auto_analyze_scale_factor](#ysql-auto-analyze-scale-factor) settings.
2173
+
The fraction defining when sufficient mutations have been accumulated to run ANALYZE for a table. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
2170
2174
2171
-
##### --ysql_auto_analyze_threshold
2175
+
##### --ysql_auto_analyze_min_cooldown_per_table
2172
2176
2173
2177
{{% tags/wrap %}}
2174
2178
2175
-
{{<tags/feature/restart-needed>}}
2176
-
Default: `50`
2179
+
Default: `10000` (10 seconds)
2177
2180
{{% /tags/wrap %}}
2178
2181
2179
-
The minimum number of mutations needed to run ANALYZE on a table.
2182
+
The minimum duration (in milliseconds) for the cooldown period between successive runs of ANALYZE on a specific table by the auto analyze service. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
2180
2183
2181
-
##### --ysql_auto_analyze_scale_factor
2184
+
##### --ysql_auto_analyze_max_cooldown_per_table
2182
2185
2183
2186
{{% tags/wrap %}}
2184
2187
2185
-
{{<tags/feature/restart-needed>}}
2186
-
Default: `0.1`
2188
+
Default: `86400000` (24 hours)
2187
2189
{{% /tags/wrap %}}
2188
2190
2189
-
The fraction defining when sufficient mutations have been accumulated to run ANALYZE for a table.
2191
+
The maximum duration (in milliseconds) for the cooldown period between successive runs of ANALYZE on a specific table by the auto analyze service. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
ANALYZE runs when the mutation count exceeds `ysql_auto_analyze_scale_factor * <table_size> + ysql_auto_analyze_threshold`, where table_size is the value of the `reltuples` column in the `pg_class` catalog.
2200
+
The exponential factor by which the per table cooldown period is scaled up each time from the value ysql_auto_analyze_min_cooldown_per_table to the value ysql_auto_analyze_max_cooldown_per_table. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
2192
2201
2193
2202
##### --ysql_auto_analyze_batch_size
2194
2203
2195
2204
{{% tags/wrap %}}
2196
2205
2197
-
{{<tags/feature/restart-needed>}}
2198
2206
Default: `10`
2199
2207
{{% /tags/wrap %}}
2200
2208
@@ -2204,7 +2212,6 @@ The maximum number of tables the Auto Analyze service tries to analyze in a sing
2204
2212
2205
2213
{{% tags/wrap %}}
2206
2214
2207
-
{{<tags/feature/restart-needed>}}
2208
2215
Default: `10000`
2209
2216
{{% /tags/wrap %}}
2210
2217
@@ -2214,7 +2221,6 @@ Interval at which the reported node level table mutation counts are persisted to
2214
2221
2215
2222
{{% tags/wrap %}}
2216
2223
2217
-
{{<tags/feature/restart-needed>}}
2218
2224
Default: `10000`
2219
2225
{{% /tags/wrap %}}
2220
2226
@@ -2224,7 +2230,6 @@ Timeout for the RPCs used to persist mutation counts in the auto-analyze mutatio
2224
2230
2225
2231
{{% tags/wrap %}}
2226
2232
2227
-
{{<tags/feature/restart-needed>}}
2228
2233
Default: `5000`
2229
2234
{{% /tags/wrap %}}
2230
2235
@@ -2234,12 +2239,31 @@ Interval, in milliseconds, at which the node-level table mutation counts are sen
2234
2239
2235
2240
{{% tags/wrap %}}
2236
2241
2237
-
{{<tags/feature/restart-needed>}}
2238
2242
Default: `5000`
2239
2243
{{% /tags/wrap %}}
2240
2244
2241
2245
Timeout, in milliseconds, for the node-level mutation reporting RPC to the Auto Analyze service.
Enable the Auto Analyze service, which automatically runs ANALYZE to update table statistics for tables that have changed more than a configurable threshold.
Enable per table mutation (INSERT, UPDATE, DELETE) counting. The Auto Analyze service runs ANALYZE when the number of mutations of a table exceeds the threshold determined by the [ysql_auto_analyze_threshold](#ysql-auto-analyze-threshold) and [ysql_auto_analyze_scale_factor](#ysql-auto-analyze-scale-factor) settings.
2266
+
2243
2267
### Advisory lock flags
2244
2268
2245
2269
To learn about advisory locks, see [Advisory locks](../../../architecture/transactions/concurrency-control/#advisory-locks).
0 commit comments