Add new wal_level, logical, sufficient for logical decoding.

When wal_level=logical, we'll log columns from the old tuple as
configured by the REPLICA IDENTITY facility added in commit
07cacba983ef79be4a84fcd0e0ca3b5fcb85dd65.  This makes it possible
a properly-configured logical replication solution to correctly
follow table updates even if they change the chosen key columns,
or, with REPLICA IDENTITY FULL, even if the table has no key at
all.  Note that updates which do not modify the replica identity
column won't log anything extra, making the choice of a good key
(i.e. one that will rarely be changed) important to performance
when wal_level=logical is configured.

Each insert, update, or delete to a catalog table will also log
the CMIN and/or CMAX values of stamped by the current transaction.
This is necessary because logical decoding will require access to
historical snapshots of the catalog in order to decode some data
types, and the CMIN/CMAX values that we may need in order to judge
row visibility may have been overwritten by the time we need them.

Andres Freund, reviewed in various versions by myself, Heikki
Linnakangas, KONDO Mitsumasa, and many others.

3 files changed, 31 insertions, 22 deletions

diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml
index 995933c62dd..a2361d780fb 100644
--- a/doc/src/sgml/backup.sgml
+++ b/doc/src/sgml/backup.sgml
@@ -587,7 +587,7 @@ tar -cf backup.tar /usr/local/pgsql/data
 
    <para>
     To enable WAL archiving, set the <xref linkend="guc-wal-level">
-    configuration parameter to <literal>archive</> (or <literal>hot_standby</>),
+    configuration parameter to <literal>archive</> or higher,
     <xref linkend="guc-archive-mode"> to <literal>on</>,
     and specify the shell command to use in the <xref
     linkend="guc-archive-command"> configuration parameter.  In practice
@@ -1259,7 +1259,7 @@ restore_command = 'cp /mnt/server/archivedir/%f %p'
       If more flexibility in copying the backup files is needed, a lower
       level process can be used for standalone hot backups as well.
       To prepare for low level standalone hot backups, set <varname>wal_level</> to
-      <literal>archive</> (or <literal>hot_standby</>), <varname>archive_mode</> to
+      <literal>archive</> or higher, <varname>archive_mode</> to
       <literal>on</>, and set up an <varname>archive_command</> that performs
       archiving only when a <emphasis>switch file</> exists.  For example:
 <programlisting>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index fee83c1496b..f33a16b7aad 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -1648,10 +1648,12 @@ include 'filename'
         <varname>wal_level</> determines how much information is written
         to the WAL. The default value is <literal>minimal</>, which writes
         only the information needed to recover from a crash or immediate
-        shutdown. <literal>archive</> adds logging required for WAL archiving,
-        and <literal>hot_standby</> further adds information required to run
-        read-only queries on a standby server.
-        This parameter can only be set at server start.
+        shutdown. <literal>archive</> adds logging required for WAL archiving;
+        <literal>hot_standby</> further adds information required to run
+        read-only queries on a standby server; and, finally
+        <literal>logical</> adds information necessary to support logical
+        decoding.  Each level includes the information logged at all lower
+        levels.  This parameter can only be set at server start.
        </para>
        <para>
         In <literal>minimal</> level, WAL-logging of some bulk
@@ -1665,24 +1667,30 @@ include 'filename'
          <member><command>COPY</> into tables that were created or truncated in the same
          transaction</member>
         </simplelist>
-        But minimal WAL does not contain
-        enough information to reconstruct the data from a base backup and the
-        WAL logs, so either <literal>archive</> or <literal>hot_standby</>
-        level must be used to enable
-        WAL archiving (<xref linkend="guc-archive-mode">) and streaming
-        replication.
+        But minimal WAL does not contain enough information to reconstruct the
+        data from a base backup and the WAL logs, so <literal>archive</> or
+        higher must be used to enable WAL archiving
+        (<xref linkend="guc-archive-mode">) and streaming replication.
        </para>
        <para>
         In <literal>hot_standby</> level, the same information is logged as
         with <literal>archive</>, plus information needed to reconstruct
         the status of running transactions from the WAL. To enable read-only
         queries on a standby server, <varname>wal_level</> must be set to
-        <literal>hot_standby</> on the primary, and
+        <literal>hot_standby</> or higher on the primary, and
         <xref linkend="guc-hot-standby"> must be enabled in the standby. It is
-        thought that there is
-        little measurable difference in performance between using
-        <literal>hot_standby</> and <literal>archive</> levels, so feedback
-        is welcome if any production impacts are noticeable.
+        thought that there is little measurable difference in performance
+        between using <literal>hot_standby</> and <literal>archive</> levels,
+        so feedback is welcome if any production impacts are noticeable.
+       </para>
+       <para>
+        In <literal>logical</> level, the same information is logged as
+        with <literal>hot_standby</>, plus information needed to allow
+        extracting logical changesets from the WAL. Using a level of
+        <literal>logical</> will increase the WAL volume, particularly if many
+        tables are configured for <literal>REPLICA IDENTITY FULL</literal> and
+        many <command>UPDATE</> and <command>DELETE</> statements are
+        executed.
        </para>
       </listitem>
      </varlistentry>
@@ -2239,9 +2247,9 @@ include 'filename'
         disabled. WAL sender processes count towards the total number
         of connections, so the parameter cannot be set higher than
         <xref linkend="guc-max-connections">.  This parameter can only
-        be set at server start. <varname>wal_level</> must be set
-        to <literal>archive</> or <literal>hot_standby</> to allow
-        connections from standby servers.
+        be set at server start. <varname>wal_level</> must be set to
+        <literal>archive</> or higher to allow connections from standby
+        servers.
        </para>
        </listitem>
       </varlistentry>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index c8f6fa8a54d..e2e5ac93ab9 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -1861,8 +1861,9 @@ LOG:  database system is ready to accept read only connections
     Consistency information is recorded once per checkpoint on the primary.
     It is not possible to enable hot standby when reading WAL
     written during a period when <varname>wal_level</> was not set to
-    <literal>hot_standby</> on the primary.  Reaching a consistent state can
-    also be delayed in the presence of both of these conditions:
+    <literal>hot_standby</> or <literal>logical</> on the primary.  Reaching
+    a consistent state can also be delayed in the presence of both of these
+    conditions:
 
       <itemizedlist>
        <listitem>