Commit 39b28c2
committed
[yugabyte#24951] Create separate memory context for RelationBuildTriggers
Summary:
Currently, there is an issue where on certain customer schemas, PG backend memory usage spikes up to 1GB during connection startup. In short, this is caused by our relcache preloading combined with inefficient memory management. The issue is as follows:
1. First, we prefetch the `pg_trigger` table by loading it from the master leader into the local tserver's memory.
2. For every table, we call `RelationBuildTriggers` to build the triggers section of the table's relcache entry.
3. Inside `RelationBuildTriggers`, we scan `pg_trigger` for the relevant triggers. Normally, we can use the `pg_trigger_tgrelid_tgname_index` index to seek directly to the triggers we want using the table's OID. But we can't do index seeks on the prefetched table, so we do a sequential scan on the entire table, copying every row of `pg_trigger` into PG memory and then filtering for the triggers we want. The entire YbUpdateRelationCache process uses a single memory context (`UpdateRelationCacheContext`), so each call to `RelationBuildTriggers` will allocate memory from this memory context which isn't freed until the entire relation cache is built.
4. As a result, we are using a lot of extra memory. If we have `m` total triggers and `n` total tables, we are allocating memory for `mn` rows in the `UpdateRelationCacheContext`.
As a shorter-term fix for this issue, this revision creates a new memory context every time we invoke `RelationBuildTriggers` which gets freed at the end of `RelationBuildTriggers`. This way, even though we are still copying `mn` rows into memory, we are only allocating memory for `m` rows at a time before freeing the memory, avoiding the spike in memory usage.
This issue does not address the CPU overhead of doing all of these extra copies—this is addressed in the longer-term fix D40003.
Test Plan: Jenkins
Reviewers: mihnea, myang
Reviewed By: myang
Subscribers: yql
Differential Revision: https://phorge.dev.yugabyte.com/D400051 parent 4b99c2e commit 39b28c2
1 file changed
+22
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1924 | 1924 | | |
1925 | 1925 | | |
1926 | 1926 | | |
| 1927 | + | |
| 1928 | + | |
1927 | 1929 | | |
1928 | 1930 | | |
| 1931 | + | |
| 1932 | + | |
| 1933 | + | |
| 1934 | + | |
| 1935 | + | |
| 1936 | + | |
| 1937 | + | |
| 1938 | + | |
| 1939 | + | |
1929 | 1940 | | |
1930 | 1941 | | |
1931 | 1942 | | |
| |||
2042 | 2053 | | |
2043 | 2054 | | |
2044 | 2055 | | |
| 2056 | + | |
| 2057 | + | |
| 2058 | + | |
| 2059 | + | |
| 2060 | + | |
2045 | 2061 | | |
2046 | 2062 | | |
2047 | 2063 | | |
| |||
2059 | 2075 | | |
2060 | 2076 | | |
2061 | 2077 | | |
| 2078 | + | |
| 2079 | + | |
| 2080 | + | |
| 2081 | + | |
| 2082 | + | |
| 2083 | + | |
2062 | 2084 | | |
2063 | 2085 | | |
2064 | 2086 | | |
| |||
0 commit comments