Add function to get memory context stats for processes

This adds a function for retrieving memory context statistics
and information from backends as well as auxiliary processes.
The intended usecase is cluster debugging when under memory
pressure or unanticipated memory usage characteristics.

When calling the function it sends a signal to the specified
process to submit statistics regarding its memory contexts
into dynamic shared memory.  Each memory context is returned
in detail, followed by a cumulative total in case the number
of contexts exceed the max allocated amount of shared memory.
Each process is limited to use at most 1Mb memory for this.

A summary can also be explicitly requested by the user, this
will return the TopMemoryContext and a cumulative total of
all lower contexts.

In order to not block on busy processes the caller specifies
the number of seconds during which to retry before timing out.
In the case where no statistics are published within the set
timeout,  the last known statistics are returned, or NULL if
no previously published statistics exist.  This allows dash-
board type queries to continually publish even if the target
process is temporarily congested.  Context records contain a
timestamp to indicate when they were submitted.

Author: Rahila Syed <rahilasyed90@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: Alexander Korotkov <aekorotkov@gmail.com>
Discussion: https://postgr.es/m/CAH2L28v8mc9HDt8QoSJ8TRmKau_8FM_HKS41NeO9-6ZAkuZKXw@mail.gmail.com

1 files changed, 172 insertions, 0 deletions

diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9ab070adffb..1c5cfee25d1 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -28666,6 +28666,144 @@ acl      | {postgres=arwdDxtm/postgres,foo=r/postgres}
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
+         <primary>pg_get_process_memory_contexts</primary>
+        </indexterm>
+        <function>pg_get_process_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type>, <parameter>summary</parameter> <type>boolean</type>, <parameter>timeout</parameter> <type>float</type> )
+        <returnvalue>setof record</returnvalue>
+        ( <parameter>name</parameter> <type>text</type>,
+        <parameter>ident</parameter> <type>text</type>,
+        <parameter>type</parameter> <type>text</type>,
+        <parameter>path</parameter> <type>integer[]</type>,
+        <parameter>level</parameter> <type>integer</type>,
+        <parameter>total_bytes</parameter> <type>bigint</type>,
+        <parameter>total_nblocks</parameter> <type>bigint</type>,
+        <parameter>free_bytes</parameter> <type>bigint</type>,
+        <parameter>free_chunks</parameter> <type>bigint</type>,
+        <parameter>used_bytes</parameter> <type>bigint</type>,
+        <parameter>num_agg_contexts</parameter> <type>integer</type>,
+        <parameter>stats_timestamp</parameter> <type>timestamptz</type> )
+       </para>
+       <para>
+        This function handles requests to display the memory contexts of a
+        <productname>PostgreSQL</productname> process with the specified
+        process ID.  The function can be used to send requests to backends as
+        well as <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>.
+       </para>
+       <para>
+        The returned record contains extended statistics per each memory
+        context:
+        <itemizedlist spacing="compact">
+         <listitem>
+          <para>
+           <parameter>name</parameter> - The name of the memory context.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>ident</parameter> - Memory context ID (if any).
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>type</parameter> - The type of memory context, possible
+           values are: AllocSet, Generation, Slab and Bump.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>path</parameter> - Memory contexts are organized in a
+           tree model with TopMemoryContext as the root, and all other memory
+           contexts as nodes in the tree. The <parameter>path</parameter>
+           displays the path from the root to the current memory context. The
+           path is limited to 100 children per node, which each node limited
+           to a max depth of 100, to preserve memory during reporting. The
+           printed path will also be limited to 100 nodes counting from the
+           TopMemoryContext.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>level</parameter> - The level in the tree of the current
+           memory context.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>total_bytes</parameter> - The total number of bytes
+           allocated to this memory context.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>total_nblocks</parameter> - The total number of blocks
+           used for the allocated memory.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>free_bytes</parameter> - The amount of free memory in
+           this memory context.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>free_chunks</parameter> - The number of chunks that
+           <parameter>free_bytes</parameter> corresponds to.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>used_bytes</parameter> - The total number of bytes
+           currently occupied.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>num_agg_contexts</parameter> - The number of memory
+           contexts aggregated in the displayed statistics.
+          </para>
+         </listitem>
+         <listitem>
+          <para>
+           <parameter>stats_timestamp</parameter> - When the statistics were
+           extracted from the process.
+          </para>
+         </listitem>
+        </itemizedlist>
+       </para>
+       <para>
+        When <parameter>summary</parameter> is <literal>true</literal>, statistics
+        for memory contexts at levels 1 and 2 are displayed, with level 1
+        representing the root node (i.e., <literal>TopMemoryContext</literal>).
+        Statistics for contexts on level 2 and below are aggregates of all
+        child contexts' statistics, where <literal>num_agg_contexts</literal>
+        indicate the number aggregated child contexts.  When
+        <parameter>summary</parameter> is <literal>false</literal>,
+        <literal>the num_agg_contexts</literal> value is <literal>1</literal>,
+        indicating that individual statistics are being displayed. The levels
+        are limited to the first 100 contexts.
+       </para>
+       <para>
+        Busy processes can delay reporting memory context statistics,
+        <parameter>timeout</parameter> specifies the number of seconds
+        to wait for updated statistics. <parameter>timeout</parameter> can be
+        specified in fractions of a second.
+       </para>
+       <para>
+        After receiving memory context statistics from the target process, it
+        returns the results as one row per context.  If all the contexts don't
+        fit within the pre-determined size limit, the remaining context
+        statistics are aggregated and a cumulative total is displayed.  The
+        <literal>num_agg_contexts</literal> column indicates the number of
+        contexts aggregated in the displayed statistics.  When
+        <literal>num_agg_contexts</literal> is <literal>1</literal> is means
+        that the context statistics are displayed separately.
+       </para></entry>
+      </row>
+
+      <row>
+       <entry role="func_table_entry"><para role="func_signature">
+        <indexterm>
          <primary>pg_log_backend_memory_contexts</primary>
         </indexterm>
         <function>pg_log_backend_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type> )
@@ -28802,6 +28940,40 @@ LOG:  Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
     because it may generate a large number of log messages.
    </para>
 
+   <para>
+    <function>pg_get_process_memory_contexts</function> can be used to request
+    memory contexts statistics of any <productname>PostgreSQL</productname>
+    process.  For example:
+<programlisting>
+postgres=# SELECT * FROM pg_get_process_memory_contexts(
+  (SELECT pid FROM pg_stat_activity
+    WHERE backend_type = 'checkpointer'),
+  false, 0.5) LIMIT 1;
+-[ RECORD 1 ]----+------------------------------
+name             | TopMemoryContext
+ident            |
+type             | AllocSet
+path             | {1}
+level            | 1
+total_bytes      | 90304
+total_nblocks    | 3
+free_bytes       | 2880
+free_chunks      | 1
+used_bytes       | 87424
+num_agg_contexts | 1
+stats_timestamp  | 2025-03-24 13:55:47.796698+01
+</programlisting>
+    <note>
+     <para>
+      While <function>pg_get_process_memory_contexts</function> can be used to
+      query memory contexts of the local backend,
+      <structname>pg_backend_memory_contexts</structname>
+      (see <xref linkend="view-pg-backend-memory-contexts"/> for more details)
+      will be less resource intensive when only the local backend is of interest.
+     </para>
+    </note>
+   </para>
+
   </sect2>
 
   <sect2 id="functions-admin-backup">