Skip to content

Commit 7396403

Browse files
authored
[DOC-1083][2025.2] Auto analyze update (#29514)
1 parent 9bcbf10 commit 7396403

File tree

3 files changed

+69
-60
lines changed

3 files changed

+69
-60
lines changed

docs/content/stable/additional-features/auto-analyze.md

Lines changed: 13 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,6 @@ headerTitle: Auto Analyze service
44
linkTitle: Auto Analyze
55
description: Use the Auto Analyze service to keep table statistics up to date
66
headcontent: Keep table statistics up to date automatically
7-
tags:
8-
feature: early-access
97
menu:
108
stable:
119
identifier: auto-analyze
@@ -22,40 +20,29 @@ Similar to [PostgreSQL autovacuum](https://www.postgresql.org/docs/current/routi
2220

2321
## Enable Auto Analyze
2422

25-
Before you can use the feature, you must enable it by setting `ysql_enable_auto_analyze_service` to true on all YB-Masters, and both `ysql_enable_auto_analyze_service` and `ysql_enable_table_mutation_counter` to true on all YB-TServers.
23+
For new universes running v2025.2 or later, Auto Analyze is enabled by default when you deploy using yugabyted, YugabyteDB Anywhere, or YugabyteDB Aeon.
2624

27-
For example, to create a single-node [yugabyted](../../reference/configuration/yugabyted/) cluster with Auto Analyze enabled, use the following command:
25+
In addition, when upgrading a deployment to v2025.2 or later, if the universe has the cost-based optimizer enabled (`on`), YugabyteDB will enable Auto Analyze.
26+
27+
You can explicitly enable or disable auto analyze by setting `ysql_enable_auto_analyze` on all yb-tservers.
28+
29+
For example, to create a single-node [yugabyted](../../reference/configuration/yugabyted/) cluster with Auto Analyze explicitly enabled, use the following command:
2830

2931
```sh
3032
./bin/yugabyted start \
31-
--master_flags "ysql_enable_auto_analyze_service=true" \
32-
--tserver_flags "ysql_enable_auto_analyze_service=true,ysql_enable_table_mutation_counter=true"
33+
--tserver_flags "ysql_enable_auto_analyze=true"
3334
```
3435

35-
Enabling Auto Analyze on an existing cluster requires a rolling restart to set `ysql_enable_auto_analyze_service` and `ysql_enable_table_mutation_counter` to true.
36-
3736
## Configure Auto Analyze
3837

39-
You can control how frequently the service updates table statistics using the following YB-TServer flags:
40-
41-
- `ysql_auto_analyze_threshold` - the minimum number of mutations (INSERT, UPDATE, and DELETE) needed to run ANALYZE on a table. Default is 50.
42-
- `ysql_auto_analyze_scale_factor` - a fraction that determines when enough mutations have been accumulated to run ANALYZE for a table. Default is 0.1.
43-
44-
Increasing either of these flags reduces the frequency of statistics updates.
45-
46-
If the total number of mutations for a table is greater than its analyze threshold, then the service runs ANALYZE on the table. The analyze threshold of a table is calculated as follows:
47-
48-
```sh
49-
analyze_threshold = ysql_auto_analyze_threshold + (ysql_auto_analyze_scale_factor * <table_size>)
50-
```
51-
52-
where `<table_size>` is the current `reltuples` column value stored in the `pg_class` catalog.
38+
The auto analyze service counts the number of mutations (INSERT, UPDATE, and DELETE) to a table and triggers ANALYZE on the table automatically when certain thresholds are reached. You can configure this behavior using the following settings.
5339

54-
`ysql_auto_analyze_threshold` is important for small tables. With default settings, if a table has 100 rows and 20 are mutated, ANALYZE won't run as the threshold is not met, even though 20% of the rows are mutated.
40+
A table needs to accumulate a minimum number of mutations before it is considered for ANALYZE. This minimum is the sum of:
5541

56-
On the other hand, `ysql_auto_analyze_scale_factor` is especially important for big tables. If a table has 1,000,000,000 rows, 10% (100,000,000 rows) would have to be mutated before ANALYZE runs. Set the scale factor to a lower value to allow for more frequent statistics collection for such large tables.
42+
- A fraction of the table size. This is controlled by [ysql_auto_analyze_scale_factor](../../reference/configuration/yb-tserver/#ysql-auto-analyze-scale-factor). This setting defaults to 0.1, which translates to 10% of the current table size. Current table size is determined by the [reltuples](https://www.postgresql.org/docs/15/catalog-pg-class.html#:~:text=CREATE%20INDEX.-,reltuples,-float4) column value stored in the `pg_class` catalog entry for that table.
43+
- A static count of [ysql_auto_analyze_threshold](../../reference/configuration/yb-tserver/#ysql-auto-analyze-threshold) (default 50) mutations. This setting ensures that small tables are not aggressively ANALYZED because the scale factor requirement is easily met.
5744

58-
In addition, `ysql_auto_analyze_batch_size` controls the maximum number of tables the Auto Analyze service tries to analyze in a single ANALYZE statement. The default is 10. Setting this flag to a larger value can potentially reduce the number of YSQL catalog cache refreshes if Auto Analyze decides to ANALYZE many tables in the same database at the same time.
45+
Separately, Auto Analyze also considers cooldown settings for a table so as to not trigger ANALYZE aggressively. After every run of ANALYZE on a table, a cooldown period is enforced before the next run of ANALYZE on that table, even if the mutation thresholds are met. The cooldown period starts from [ysql_auto_analyze_min_cooldown_per_table](../../reference/configuration/yb-tserver/#ysql_auto_analyze_min_cooldown_per_table) (default: 10 seconds) and exponentially increases to [ysql_auto_analyze_max_cooldown_per_table](../../reference/configuration/yb-tserver/#ysql_auto_analyze_max_cooldown_per_table) (default: 24 hours). Cooldown values for a table do not reset. This means that in most cases, it is expected that, after a while, a frequently updated table is only analyzed once every `ysql_auto_analyze_max_cooldown_per_table` period.
5946

6047
For more information on flags used to configure the Auto Analyze service, refer to [Auto Analyze service flags](../../reference/configuration/yb-tserver/#auto-analyze-service-flags).
6148

@@ -94,4 +81,4 @@ SELECT reltuples FROM pg_class WHERE relname = 'test';
9481

9582
## Limitations
9683

97-
Because ANALYZE is a DDL statement, it can cause DDL conflicts when run concurrently with other DDL statements. As Auto Analyze runs ANALYZE in the background, you should turn off Auto Analyze if you want to execute DDL statements. You can do this by setting `ysql_enable_auto_analyze_service` to false on all YB-TServers at runtime.
84+
ANALYZE is technically considered a DDL statement (schema change) and normally conflicts with other [concurrent DDLs](../best-practices-operations/administration/#concurrent-ddl-during-a-ddl-operation). However, when run via the auto analyze service, ANALYZE can run concurrently with other DDL. In this case, ANALYZE is pre-empted by concurrent DDL and will be retried at a later point. However, when [transactional DDL](../explore/transactions/transactional-ddl/) is enabled (off by default), certain kinds of transactions that contain DDL may face a `kConflict` error when a background ANALYZE from the auto analyze service interrupts this transaction. In such cases, it is recommended to disable the auto analyze service explicitly and trigger ANALYZE manually. Issue {{<issue 28903>}} tracks this scenario.

docs/content/stable/reference/configuration/yb-master.md

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1058,15 +1058,13 @@ Default: `true`
10581058

10591059
## Auto Analyze service flags
10601060

1061-
{{<tags/feature/ea idea="590">}}To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
1061+
To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
10621062

1063-
See also [Auto Analyze Service TServer flags](../yb-tserver/#auto-analyze-service-flags).
1063+
Auto analyze is automatically enabled when the [cost-based optimizer](../../../architecture/query-layer/planner-optimizer/) (CBO) is enabled by setting the [yb_enable_cbo](../tb-tserver/#yb_enable_cbo) flag to `on`.
10641064

1065-
##### ysql_enable_auto_analyze_service
1065+
##### ysql_enable_auto_analyze_service (deprecated)
10661066

1067-
{{<tags/feature/ea idea="590">}}Enable the Auto Analyze service, which automatically runs ANALYZE to update table statistics for tables that have changed more than a configurable threshold.
1068-
1069-
Default: false
1067+
Use [ysql_enable_auto_analyze](../yb-tserver#ysql_enable_auto_analyze) on yb-tservers instead.
10701068

10711069
## Advisory lock flags
10721070

docs/content/stable/reference/configuration/yb-tserver.md

Lines changed: 52 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -2131,70 +2131,78 @@ Default: `legacy_mode`
21312131
21322132
Enables the YugabyteDB [cost-based optimizer](../../../architecture/query-layer/planner-optimizer/) (CBO). Options are `on`, `off`, `legacy_mode`, and `legacy_stats_mode`.
21332133
2134-
When enabling CBO, you must run ANALYZE on user tables to maintain up-to-date statistics.
2134+
When CBO is enabled (set to `on`), [auto analyze](#auto-analyze-service-flags) is also enabled automatically. If you disable auto analyze explicitly, you are responsible for periodically running ANALYZE on user tables to maintain up-to-date statistics.
21352135
21362136
For information on using this parameter to configure CBO, refer to [Enable cost-based optimizer](../../../best-practices-operations/ysql-yb-enable-cbo/).
21372137
21382138
### Auto Analyze service flags
21392139
2140-
{{<tags/feature/ea idea="590">}}To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
2141-
2142-
{{< note title="Note" >}}
2140+
To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
21432141
2144-
To fully enable the Auto Analyze service, you need to enable `ysql_enable_auto_analyze_service` on all YB-Masters and YB-TServers, and `ysql_enable_table_mutation_counter` on all YB-TServers.
2142+
Auto analyze is automatically enabled when the [cost-based optimizer](../../../best-practices-operations/ysql-yb-enable-cbo/) (CBO) is enabled ([yb_enable_cbo](#yb_enable_cbo) is set to `on`).
21452143
2146-
{{< /note >}}
2144+
In v2025.2 and later, CBO and Auto Analyze are enabled by default in new universes when you deploy using yugabyted, YugabyteDB Anywhere, or YugabyteDB Aeon. In addition, when upgrading a deployment to v2025.2 or later, if the universe has the cost-based optimizer enabled (`on`), YugabyteDB will enable Auto Analyze.
21472145
2148-
See also [Auto Analyze Service Master flags](../yb-master/#auto-analyze-service-flags).
2146+
To explicitly control the service, you can set the `ysql_enable_auto_analyze` flag.
21492147
2150-
##### --ysql_enable_auto_analyze_service
2148+
##### --ysql_enable_auto_analyze
21512149
21522150
{{% tags/wrap %}}
2153-
{{<tags/feature/ea idea="590">}}
2154-
{{<tags/feature/t-server>}}
2155-
{{<tags/feature/restart-needed>}}
2151+
21562152
Default: `false`
21572153
{{% /tags/wrap %}}
21582154
21592155
Enable the Auto Analyze service, which automatically runs ANALYZE to update table statistics for tables that have changed more than a configurable threshold.
21602156
2161-
##### --ysql_enable_table_mutation_counter
2157+
##### --ysql_auto_analyze_threshold
21622158
21632159
{{% tags/wrap %}}
21642160
2161+
Default: `50`
2162+
{{% /tags/wrap %}}
21652163
2166-
Default: `false`
2164+
The minimum number of mutations needed to run ANALYZE on a table. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
2165+
2166+
##### --ysql_auto_analyze_scale_factor
2167+
2168+
{{% tags/wrap %}}
2169+
2170+
Default: `0.1`
21672171
{{% /tags/wrap %}}
21682172
2169-
Enable per table mutation (INSERT, UPDATE, DELETE) counting. The Auto Analyze service runs ANALYZE when the number of mutations of a table exceeds the threshold determined by the [ysql_auto_analyze_threshold](#ysql-auto-analyze-threshold) and [ysql_auto_analyze_scale_factor](#ysql-auto-analyze-scale-factor) settings.
2173+
The fraction defining when sufficient mutations have been accumulated to run ANALYZE for a table. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
21702174
2171-
##### --ysql_auto_analyze_threshold
2175+
##### --ysql_auto_analyze_min_cooldown_per_table
21722176
21732177
{{% tags/wrap %}}
21742178
2175-
{{<tags/feature/restart-needed>}}
2176-
Default: `50`
2179+
Default: `10000` (10 seconds)
21772180
{{% /tags/wrap %}}
21782181
2179-
The minimum number of mutations needed to run ANALYZE on a table.
2182+
The minimum duration (in milliseconds) for the cooldown period between successive runs of ANALYZE on a specific table by the auto analyze service. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
21802183
2181-
##### --ysql_auto_analyze_scale_factor
2184+
##### --ysql_auto_analyze_max_cooldown_per_table
21822185
21832186
{{% tags/wrap %}}
21842187
2185-
{{<tags/feature/restart-needed>}}
2186-
Default: `0.1`
2188+
Default: `86400000` (24 hours)
21872189
{{% /tags/wrap %}}
21882190
2189-
The fraction defining when sufficient mutations have been accumulated to run ANALYZE for a table.
2191+
The maximum duration (in milliseconds) for the cooldown period between successive runs of ANALYZE on a specific table by the auto analyze service. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
2192+
2193+
##### --ysql_auto_analyze_cooldown_per_table_scale_factor
2194+
2195+
{{% tags/wrap %}}
2196+
2197+
Default: `2`
2198+
{{% /tags/wrap %}}
21902199
2191-
ANALYZE runs when the mutation count exceeds `ysql_auto_analyze_scale_factor * <table_size> + ysql_auto_analyze_threshold`, where table_size is the value of the `reltuples` column in the `pg_class` catalog.
2200+
The exponential factor by which the per table cooldown period is scaled up each time from the value ysql_auto_analyze_min_cooldown_per_table to the value ysql_auto_analyze_max_cooldown_per_table. For more details, see [Auto Analyze service](../../../additional-features/auto-analyze).
21922201
21932202
##### --ysql_auto_analyze_batch_size
21942203
21952204
{{% tags/wrap %}}
21962205
2197-
{{<tags/feature/restart-needed>}}
21982206
Default: `10`
21992207
{{% /tags/wrap %}}
22002208
@@ -2204,7 +2212,6 @@ The maximum number of tables the Auto Analyze service tries to analyze in a sing
22042212
22052213
{{% tags/wrap %}}
22062214
2207-
{{<tags/feature/restart-needed>}}
22082215
Default: `10000`
22092216
{{% /tags/wrap %}}
22102217
@@ -2214,7 +2221,6 @@ Interval at which the reported node level table mutation counts are persisted to
22142221
22152222
{{% tags/wrap %}}
22162223
2217-
{{<tags/feature/restart-needed>}}
22182224
Default: `10000`
22192225
{{% /tags/wrap %}}
22202226
@@ -2224,7 +2230,6 @@ Timeout for the RPCs used to persist mutation counts in the auto-analyze mutatio
22242230
22252231
{{% tags/wrap %}}
22262232
2227-
{{<tags/feature/restart-needed>}}
22282233
Default: `5000`
22292234
{{% /tags/wrap %}}
22302235
@@ -2234,12 +2239,31 @@ Interval, in milliseconds, at which the node-level table mutation counts are sen
22342239
22352240
{{% tags/wrap %}}
22362241
2237-
{{<tags/feature/restart-needed>}}
22382242
Default: `5000`
22392243
{{% /tags/wrap %}}
22402244
22412245
Timeout, in milliseconds, for the node-level mutation reporting RPC to the Auto Analyze service.
22422246
2247+
##### --ysql_enable_auto_analyze_service (deprecated)
2248+
2249+
{{% tags/wrap %}}
2250+
{{<tags/feature/t-server>}}
2251+
{{<tags/feature/restart-needed>}}
2252+
Default: `false`
2253+
{{% /tags/wrap %}}
2254+
2255+
Enable the Auto Analyze service, which automatically runs ANALYZE to update table statistics for tables that have changed more than a configurable threshold.
2256+
2257+
##### --ysql_enable_table_mutation_counter (deprecated)
2258+
2259+
{{% tags/wrap %}}
2260+
2261+
2262+
Default: `false`
2263+
{{% /tags/wrap %}}
2264+
2265+
Enable per table mutation (INSERT, UPDATE, DELETE) counting. The Auto Analyze service runs ANALYZE when the number of mutations of a table exceeds the threshold determined by the [ysql_auto_analyze_threshold](#ysql-auto-analyze-threshold) and [ysql_auto_analyze_scale_factor](#ysql-auto-analyze-scale-factor) settings.
2266+
22432267
### Advisory lock flags
22442268
22452269
To learn about advisory locks, see [Advisory locks](../../../architecture/transactions/concurrency-control/#advisory-locks).

0 commit comments

Comments
 (0)