From 042a66291b04f473cbc72f95f07438abd75ae3a9 Mon Sep 17 00:00:00 2001 From: Daniel Gustafsson Date: Tue, 8 Apr 2025 11:06:56 +0200 Subject: Add function to get memory context stats for processes This adds a function for retrieving memory context statistics and information from backends as well as auxiliary processes. The intended usecase is cluster debugging when under memory pressure or unanticipated memory usage characteristics. When calling the function it sends a signal to the specified process to submit statistics regarding its memory contexts into dynamic shared memory. Each memory context is returned in detail, followed by a cumulative total in case the number of contexts exceed the max allocated amount of shared memory. Each process is limited to use at most 1Mb memory for this. A summary can also be explicitly requested by the user, this will return the TopMemoryContext and a cumulative total of all lower contexts. In order to not block on busy processes the caller specifies the number of seconds during which to retry before timing out. In the case where no statistics are published within the set timeout, the last known statistics are returned, or NULL if no previously published statistics exist. This allows dash- board type queries to continually publish even if the target process is temporarily congested. Context records contain a timestamp to indicate when they were submitted. Author: Rahila Syed Reviewed-by: Daniel Gustafsson Reviewed-by: Andres Freund Reviewed-by: Tomas Vondra Reviewed-by: Atsushi Torikoshi Reviewed-by: Fujii Masao Reviewed-by: Alexander Korotkov Discussion: https://postgr.es/m/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw@mail.gmail.com --- doc/src/sgml/func.sgml | 172 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 172 insertions(+) (limited to 'doc/src') diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9ab070adffb..1c5cfee25d1 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -28663,6 +28663,144 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres} + + + + pg_get_process_memory_contexts + + pg_get_process_memory_contexts ( pid integer, summary boolean, timeout float ) + setof record + ( name text, + ident text, + type text, + path integer[], + level integer, + total_bytes bigint, + total_nblocks bigint, + free_bytes bigint, + free_chunks bigint, + used_bytes bigint, + num_agg_contexts integer, + stats_timestamp timestamptz ) + + + This function handles requests to display the memory contexts of a + PostgreSQL process with the specified + process ID. The function can be used to send requests to backends as + well as auxiliary processes. + + + The returned record contains extended statistics per each memory + context: + + + + name - The name of the memory context. + + + + + ident - Memory context ID (if any). + + + + + type - The type of memory context, possible + values are: AllocSet, Generation, Slab and Bump. + + + + + path - Memory contexts are organized in a + tree model with TopMemoryContext as the root, and all other memory + contexts as nodes in the tree. The path + displays the path from the root to the current memory context. The + path is limited to 100 children per node, which each node limited + to a max depth of 100, to preserve memory during reporting. The + printed path will also be limited to 100 nodes counting from the + TopMemoryContext. + + + + + level - The level in the tree of the current + memory context. + + + + + total_bytes - The total number of bytes + allocated to this memory context. + + + + + total_nblocks - The total number of blocks + used for the allocated memory. + + + + + free_bytes - The amount of free memory in + this memory context. + + + + + free_chunks - The number of chunks that + free_bytes corresponds to. + + + + + used_bytes - The total number of bytes + currently occupied. + + + + + num_agg_contexts - The number of memory + contexts aggregated in the displayed statistics. + + + + + stats_timestamp - When the statistics were + extracted from the process. + + + + + + When summary is true, statistics + for memory contexts at levels 1 and 2 are displayed, with level 1 + representing the root node (i.e., TopMemoryContext). + Statistics for contexts on level 2 and below are aggregates of all + child contexts' statistics, where num_agg_contexts + indicate the number aggregated child contexts. When + summary is false, + the num_agg_contexts value is 1, + indicating that individual statistics are being displayed. The levels + are limited to the first 100 contexts. + + + Busy processes can delay reporting memory context statistics, + timeout specifies the number of seconds + to wait for updated statistics. timeout can be + specified in fractions of a second. + + + After receiving memory context statistics from the target process, it + returns the results as one row per context. If all the contexts don't + fit within the pre-determined size limit, the remaining context + statistics are aggregated and a cumulative total is displayed. The + num_agg_contexts column indicates the number of + contexts aggregated in the displayed statistics. When + num_agg_contexts is 1 is means + that the context statistics are displayed separately. + + + @@ -28802,6 +28940,40 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560 because it may generate a large number of log messages. + + pg_get_process_memory_contexts can be used to request + memory contexts statistics of any PostgreSQL + process. For example: + +postgres=# SELECT * FROM pg_get_process_memory_contexts( + (SELECT pid FROM pg_stat_activity + WHERE backend_type = 'checkpointer'), + false, 0.5) LIMIT 1; +-[ RECORD 1 ]----+------------------------------ +name | TopMemoryContext +ident | +type | AllocSet +path | {1} +level | 1 +total_bytes | 90304 +total_nblocks | 3 +free_bytes | 2880 +free_chunks | 1 +used_bytes | 87424 +num_agg_contexts | 1 +stats_timestamp | 2025-03-24 13:55:47.796698+01 + + + + While pg_get_process_memory_contexts can be used to + query memory contexts of the local backend, + pg_backend_memory_contexts + (see for more details) + will be less resource intensive when only the local backend is of interest. + + + + -- cgit v1.2.3