tableam: basic documentation.

This adds documentation about the user oriented parts of table access
methods (i.e. the default_table_access_method GUC and the USING clause
for CREATE TABLE etc), adds a basic chapter about the table access
method interface, and adds a note to storage.sgml that it's contents
don't necessarily apply for non-builtin AMs.

Author: Haribabu Kommi and Andres Freund
Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de

12 files changed, 225 insertions, 16 deletions

diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index f4aabf5dc7f..0e38382f319 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -587,8 +587,9 @@
    The catalog <structname>pg_am</structname> stores information about
    relation access methods.  There is one row for each access method supported
    by the system.
-   Currently, only indexes have access methods.  The requirements for index
-   access methods are discussed in detail in <xref linkend="indexam"/>.
+   Currently, only table and indexes have access methods. The requirements for table
+   and index access methods are discussed in detail in <xref linkend="tableam"/> and
+   <xref linkend="indexam"/> respectively.
   </para>
 
   <table>
@@ -634,8 +635,8 @@
       <entry><type>char</type></entry>
       <entry></entry>
       <entry>
-       Currently always <literal>i</literal> to indicate an index access
-       method; other values may be allowed in future
+       <literal>t</literal> = table (including materialized views),
+       <literal>i</literal> = index.
       </entry>
      </row>
     </tbody>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 4cbcc7a8e51..bc1d0f7bfae 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -7294,6 +7294,23 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-default-table-access-method" xreflabel="default_table_access_method">
+      <term><varname>default_table_access_method</varname> (<type>string</type>)
+      <indexterm>
+       <primary><varname>default_table_access_method</varname> configuration parameter</primary>
+      </indexterm>
+      </term>
+      <listitem>
+       <para>
+        This parameter specifies the default table access method to use when
+        creating tables or materialized views if the <command>CREATE</command>
+        command does not explicitly specify an access method, or when
+        <command>SELECT ... INTO</command> is used, which does not allow to
+        specify a table access method. The default is <literal>heap</literal>.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-default-tablespace" xreflabel="default_tablespace">
       <term><varname>default_tablespace</varname> (<type>string</type>)
       <indexterm>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a03ea1427b9..7e37042a55e 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -89,6 +89,7 @@
 <!ENTITY gin        SYSTEM "gin.sgml">
 <!ENTITY brin       SYSTEM "brin.sgml">
 <!ENTITY planstats    SYSTEM "planstats.sgml">
+<!ENTITY tableam    SYSTEM "tableam.sgml">
 <!ENTITY indexam    SYSTEM "indexam.sgml">
 <!ENTITY nls        SYSTEM "nls.sgml">
 <!ENTITY plhandler  SYSTEM "plhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index ff8290da9ff..dd54c688024 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -3,6 +3,14 @@
 <chapter id="indexam">
  <title>Index Access Method Interface Definition</title>
 
+ <indexterm>
+  <primary>Index Access Method</primary>
+ </indexterm>
+ <indexterm>
+  <primary>indexam</primary>
+  <secondary>Index Access Method</secondary>
+ </indexterm>
+
   <para>
    This chapter defines the interface between the core
    <productname>PostgreSQL</productname> system and <firstterm>index access
@@ -50,8 +58,8 @@
    Each index access method is described by a row in the
    <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
    system catalog.  The <structname>pg_am</structname> entry
-   specifies a name and a <firstterm>handler function</firstterm> for the access
-   method.  These entries can be created and deleted using the
+   specifies a name and a <firstterm>handler function</firstterm> for the index
+   access method.  These entries can be created and deleted using the
    <xref linkend="sql-create-access-method"/> and
    <xref linkend="sql-drop-access-method"/> SQL commands.
   </para>
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 96d196d2293..3e115f1c76c 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -250,6 +250,7 @@
   &tablesample-method;
   &custom-scan;
   &geqo;
+  &tableam;
   &indexam;
   &generic-wal;
   &btree;
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63beb..dae43dbaed5 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,8 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
     <listitem>
      <para>
       This clause specifies the type of access method to define.
-      Only <literal>INDEX</literal> is supported at present.
+      Only <literal>TABLE</literal> and <literal>INDEX</literal>
+      are supported at present.
      </para>
     </listitem>
    </varlistentry>
@@ -75,10 +76,13 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
       that represents the access method.  The handler function must be
       declared to take a single argument of type <type>internal</type>,
       and its return type depends on the type of access method;
-      for <literal>INDEX</literal> access methods, it must
-      be <type>index_am_handler</type>.  The C-level API that the handler
-      function must implement varies depending on the type of access method.
-      The index access method API is described in <xref linkend="indexam"/>.
+      for <literal>TABLE</literal> access methods, it must
+      be <type>table_am_handler</type> and for <literal>INDEX</literal>
+      access methods, it must be <type>index_am_handler</type>.
+      The C-level API that the handler function must implement varies
+      depending on the type of access method. The table access method API
+      is described in <xref linkend="tableam"/> and the index access method
+      API is described in <xref linkend="indexam"/>.
      </para>
     </listitem>
    </varlistentry>
diff --git a/doc/src/sgml/ref/create_materialized_view.sgml b/doc/src/sgml/ref/create_materialized_view.sgml
index 7f31ab4d26d..ec8847ed406 100644
--- a/doc/src/sgml/ref/create_materialized_view.sgml
+++ b/doc/src/sgml/ref/create_materialized_view.sgml
@@ -23,6 +23,7 @@ PostgreSQL documentation
 <synopsis>
 CREATE MATERIALIZED VIEW [ IF NOT EXISTS ] <replaceable>table_name</replaceable>
     [ (<replaceable>column_name</replaceable> [, ...] ) ]
+    [ USING <replaceable class="parameter">method</replaceable> ]
     [ WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) ]
     [ TABLESPACE <replaceable class="parameter">tablespace_name</replaceable> ]
     AS <replaceable>query</replaceable>
@@ -86,6 +87,21 @@ CREATE MATERIALIZED VIEW [ IF NOT EXISTS ] <replaceable>table_name</replaceable>
    </varlistentry>
 
    <varlistentry>
+    <term><literal>USING <replaceable class="parameter">method</replaceable></literal></term>
+    <listitem>
+     <para>
+      This optional clause specifies the table access method to use to store
+      the contents for the new materialized view; the method needs be an
+      access method of type <literal>TABLE</literal>. See <xref
+      linkend="tableam"/> for more information.  If this option is not
+      specified, the default table access method is chosen for the new
+      materialized view. See <xref linkend="guc-default-table-access-method"/>
+      for more information.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
     <term><literal>WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
     <listitem>
      <para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 99b95bbdb43..85c0ec1b318 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -29,6 +29,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
 ] )
 [ INHERITS ( <replaceable>parent_table</replaceable> [, ... ] ) ]
 [ PARTITION BY { RANGE | LIST | HASH } ( { <replaceable class="parameter">column_name</replaceable> | ( <replaceable class="parameter">expression</replaceable> ) } [ COLLATE <replaceable class="parameter">collation</replaceable> ] [ <replaceable class="parameter">opclass</replaceable> ] [, ... ] ) ]
+[ USING <replaceable class="parameter">method</replaceable> ]
 [ WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) | WITHOUT OIDS ]
 [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
 [ TABLESPACE <replaceable class="parameter">tablespace_name</replaceable> ]
@@ -40,6 +41,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
     [, ... ]
 ) ]
 [ PARTITION BY { RANGE | LIST | HASH } ( { <replaceable class="parameter">column_name</replaceable> | ( <replaceable class="parameter">expression</replaceable> ) } [ COLLATE <replaceable class="parameter">collation</replaceable> ] [ <replaceable class="parameter">opclass</replaceable> ] [, ... ] ) ]
+[ USING <replaceable class="parameter">method</replaceable> ]
 [ WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) | WITHOUT OIDS ]
 [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
 [ TABLESPACE <replaceable class="parameter">tablespace_name</replaceable> ]
@@ -51,6 +53,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
     [, ... ]
 ) ] { FOR VALUES <replaceable class="parameter">partition_bound_spec</replaceable> | DEFAULT }
 [ PARTITION BY { RANGE | LIST | HASH } ( { <replaceable class="parameter">column_name</replaceable> | ( <replaceable class="parameter">expression</replaceable> ) } [ COLLATE <replaceable class="parameter">collation</replaceable> ] [ <replaceable class="parameter">opclass</replaceable> ] [, ... ] ) ]
+[ USING <replaceable class="parameter">method</replaceable> ]
 [ WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) | WITHOUT OIDS ]
 [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
 [ TABLESPACE <replaceable class="parameter">tablespace_name</replaceable> ]
@@ -1165,6 +1168,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
     </listitem>
    </varlistentry>
 
+   <varlistentry id="sql-createtable-method">
+    <term><literal>USING <replaceable class="parameter">method</replaceable></literal></term>
+    <listitem>
+     <para>
+      This optional clause specifies the table access method to use to store
+      the contents for the new table; the method needs be an access method of
+      type <literal>TABLE</literal>. See <xref linkend="tableam"/> for more
+      information.  If this option is not specified, the default table access
+      method is chosen for the new materialized view. See <xref
+      linkend="guc-default-table-access-method"/> for more information.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
     <listitem>
@@ -1238,7 +1255,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
     </listitem>
    </varlistentry>
 
-   <varlistentry>
+   <varlistentry id="sql-createtable-tablespace">
     <term><literal>TABLESPACE <replaceable class="parameter">tablespace_name</replaceable></literal></term>
     <listitem>
      <para>
diff --git a/doc/src/sgml/ref/create_table_as.sgml b/doc/src/sgml/ref/create_table_as.sgml
index 679e8f521ed..1371261e0a1 100644
--- a/doc/src/sgml/ref/create_table_as.sgml
+++ b/doc/src/sgml/ref/create_table_as.sgml
@@ -23,6 +23,7 @@ PostgreSQL documentation
 <synopsis>
 CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable>table_name</replaceable>
     [ (<replaceable>column_name</replaceable> [, ...] ) ]
+    [ USING <replaceable class="parameter">method</replaceable> ]
     [ WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) | WITHOUT OIDS ]
     [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
     [ TABLESPACE <replaceable class="parameter">tablespace_name</replaceable> ]
@@ -121,6 +122,20 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
    </varlistentry>
 
    <varlistentry>
+    <term><literal>USING <replaceable class="parameter">method</replaceable></literal></term>
+    <listitem>
+     <para>
+      This optional clause specifies the table access method to use to store
+      the contents for the new table; the method needs be an access method of
+      type <literal>TABLE</literal>. See <xref linkend="tableam"/> for more
+      information.  If this option is not specified, the default table access
+      method is chosen for the new materialized view. See <xref
+      linkend="guc-default-table-access-method"/> for more information.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
     <term><literal>WITH ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
     <listitem>
      <para>
diff --git a/doc/src/sgml/ref/select_into.sgml b/doc/src/sgml/ref/select_into.sgml
index 462e3723819..17bed24743d 100644
--- a/doc/src/sgml/ref/select_into.sgml
+++ b/doc/src/sgml/ref/select_into.sgml
@@ -104,6 +104,16 @@ SELECT [ ALL | DISTINCT [ ON ( <replaceable class="parameter">expression</replac
    <command>CREATE TABLE AS</command> offers a superset of the
    functionality provided by <command>SELECT INTO</command>.
   </para>
+
+  <para>
+   In contrast to <command>CREATE TABLE AS</command> <command>SELECT
+   INTO</command> does not allow to specify properties like a table's access
+   method with <xref linkend="sql-createtable-method" /> or the table's
+   tablespace with <xref linkend="sql-createtable-tablespace" />. Use <xref
+   linkend="sql-createtableas"/> if necessary.  Therefore the default table
+   access method is chosen for the new table. See <xref
+   linkend="guc-default-table-access-method"/> for more information.
+  </para>
  </refsect1>
 
  <refsect1>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 5df987f9c9c..62333e31a02 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -189,6 +189,14 @@ there.
 </para>
 
 <para>
+ Note that the following sections describe the way the builtin
+ <literal>heap</literal> <link linkend="tableam">table access method</link>,
+ and the builtin <link linkend="indexam">index access methods</link> work. Due
+ to the extensible nature of <productname>PostgreSQL</productname> other types
+ of access method might work similar or not.
+</para>
+
+<para>
 Each table and index is stored in a separate file.  For ordinary relations,
 these files are named after the table or index's <firstterm>filenode</firstterm> number,
 which can be found in <structname>pg_class</structname>.<structfield>relfilenode</structfield>. But
@@ -695,10 +703,11 @@ erased (they will be recreated automatically as needed).
 This section provides an overview of the page format used within
 <productname>PostgreSQL</productname> tables and indexes.<footnote>
   <para>
-    Actually, index access methods need not use this page format.
-    All the existing index methods do use this basic format,
-    but the data kept on index metapages usually doesn't follow
-    the item layout rules.
+    Actually, neither table nor index access methods need not use this page
+    format.  All the existing index methods do use this basic format, but the
+    data kept on index metapages usually doesn't follow the item layout
+    rules. The <literal>heap</literal> table access method also always uses
+    this format.
   </para>
 </footnote>
 Sequences and <acronym>TOAST</acronym> tables are formatted just like a regular table.
diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml
new file mode 100644
index 00000000000..8d9bfd81307
--- /dev/null
+++ b/doc/src/sgml/tableam.sgml
@@ -0,0 +1,110 @@
+<!-- doc/src/sgml/tableam.sgml -->
+
+<chapter id="tableam">
+ <title>Table Access Method Interface Definition</title>
+
+ <indexterm>
+  <primary>Table Access Method</primary>
+ </indexterm>
+ <indexterm>
+  <primary>tableam</primary>
+  <secondary>Table Access Method</secondary>
+ </indexterm>
+
+ <para>
+  This chapter explains the interface between the core
+  <productname>PostgreSQL</productname> system and <firstterm>table access
+  methods</firstterm>, which manage the storage for tables. The core system
+  knows little about these access methods beyond what is specified here, so
+  it is possible to develop entirely new access method types by writing
+  add-on code.
+ </para>
+
+ <para>
+  Each table access method is described by a row in the <link
+  linkend="catalog-pg-am"><structname>pg_am</structname></link> system
+  catalog. The <structname>pg_am</structname> entry specifies a name and a
+  <firstterm>handler function</firstterm> for the table access method.  These
+  entries can be created and deleted using the <xref
+  linkend="sql-create-access-method"/> and <xref
+  linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+  A table access method handler function must be declared to accept a single
+  argument of type <type>internal</type> and to return the pseudo-type
+  <type>table_am_handler</type>.  The argument is a dummy value that simply
+  serves to prevent handler functions from being called directly from SQL commands.
+
+  The result of the function must be a pointer to a struct of type
+  <structname>TableAmRoutine</structname>, which contains everything that the
+  core code needs to know to make use of the table access method. The return
+  value needs to be of server lifetime, which is typically achieved by
+  defining it as a <literal>static const</literal> variable in global
+  scope. The <structname>TableAmRoutine</structname> struct, also called the
+  access method's <firstterm>API struct</firstterm>, defines the behavior of
+  the access method using callbacks. These callbacks are pointers to plain C
+  functions and are not visible or callable at the SQL level. All the
+  callbacks and their behavior is defined in the
+  <structname>TableAmRoutine</structname> structure (with comments inside the
+  struct defining the requirements for callbacks). Most callbacks have
+  wrapper functions, which are documented for the point of view of a user,
+  rather than an implementor, of the table access method.  For details,
+  please refer to the <ulink url="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/access/tableam.h;hb=HEAD">
+  <filename>src/include/access/tableam.h</filename></ulink> file.
+ </para>
+
+ <para>
+  To implement a access method, an implementor will typically need to
+  implement a AM specific type of tuple table slot (see
+  <ulink url="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/executor/tuptable.h;hb=HEAD">
+   <filename>src/include/executor/tuptable.h</filename></ulink>) which allows
+   code outside the access method to hold references to tuples of the AM, and
+   to access the columns of the tuple.
+ </para>
+
+ <para>
+  Currently the the way an AM actually stores data is fairly
+  unconstrained. It is e.g. possible to use postgres' shared buffer cache,
+  but not required. In case shared buffers are used, it likely makes to
+  postgres' standard page layout described in <xref
+  linkend="storage-page-layout"/>.
+ </para>
+
+ <para>
+  One fairly large constraint of the table access method API is that,
+  currently, if the AM wants to support modifications and/or indexes, it is
+  necessary that each tuple has a tuple identifier (<acronym>TID</acronym>)
+  consisting of a block number and an item number (see also <xref
+  linkend="storage-page-layout"/>).  It is not strictly necessary that the
+  sub-parts of <acronym>TIDs</acronym> have the same meaning they e.g. have
+  for <literal>heap</literal>, but if bitmap scan support is desired (it is
+  optional), the block number needs to provide locality.
+ </para>
+
+ <para>
+  For crash safety an AM can use postgres' <link
+  linkend="wal"><acronym>WAL</acronym></link>, or a custom approach can be
+  implemented.  If <acronym>WAL</acronym> is chosen, either <link
+  linkend="generic-wal">Generic WAL Records</link> can be used &mdash; which
+  implies higher WAL volume but is easy, or a new type of
+  <acronym>WAL</acronym> records can be implemented &mdash; but that
+  currently requires modifications of core code (namely modifying
+  <filename>src/include/access/rmgrlist.h</filename>).
+ </para>
+
+ <para>
+  To implement transactional support in a manner that allows different table
+  access methods be accessed within a single transaction, it likely is
+  necessary to closely integrate with the machinery in
+  <filename>src/backend/access/transam/xlog.c</filename>.
+ </para>
+
+ <para>
+  Any developer of a new <literal>table access method</literal> can refer to
+  the existing <literal>heap</literal> implementation present in
+  <filename>src/backend/heap/heapam_handler.c</filename> for more details of
+  how it is implemented.
+ </para>
+
+</chapter>