aboutsummaryrefslogtreecommitdiff
path: root/doc/src
Commit message (Collapse)AuthorAge
...
* doc: clarify when row-level locks are releasedBruce Momjian2020-03-31
| | | | | | | | | | | They are released just like table-level locks. Also clean up wording. (Uses wording "rolled back to".) Reported-by: me@sillymon.ch Discussion: https://postgr.es/m/158074944048.1095.4309647363871637715@wrigleys.postgresql.org Backpatch-through: 9.5
* Revert erroroneous commit 834b80464d; my apologiesBruce Momjian2020-03-31
| | | | Backpatch-through: master
* dummy commitBruce Momjian2020-03-31
|
* doc: add namespace column to pg_buffercache example queryBruce Momjian2020-03-31
| | | | | | | | | | Without the namespace, the table name could be ambiguous. Reported-by: adunham@arbormetrix.com Discussion: https://postgr.es/m/158155175140.23798.2189464781144503491@wrigleys.postgresql.org Backpatch-through: 9.5
* doc: clarify which table creation is used for inheritance part.Bruce Momjian2020-03-31
| | | | | | | | | | | | Previously people might assume that the partition syntax version of CREATE TABLE is to be used for the inheritance partition table example; mention that the non-partitioned version should be used. Reported-by: mib@nic.at Discussion: https://postgr.es/m/158089540905.1098.15071165437284409576@wrigleys.postgresql.org Backpatch-through: 10
* doc: adjust UPDATE/DELETE's FROM/USING to match SELECT's FROMBruce Momjian2020-03-31
| | | | | | | | | | Previously the syntax and wording were unclear. Reported-by: Alexey Bashtanov Discussion: https://postgr.es/m/968d4724-8e58-788f-7c45-f7b1813824cc@imap.cc Backpatch-through: 9.5
* Fix lquery's NOT handling, and add ability to quantify non-'*' items.Tom Lane2020-03-31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The existing implementation of the ltree ~ lquery match operator is sufficiently complex and undocumented that it's hard to tell exactly what it does. But one thing it clearly gets wrong is the combination of NOT symbols (!) and '*' symbols. A pattern such as '*.!foo.*' should, by any ordinary understanding of regular expression behavior, match any ltree that has at least one label that's not "foo". As best we can tell by experimentation, what it's actually matching is any ltree in which *no* label is "foo". That's surprising, and not at all what the documentation says. Now, that's arguably a useful behavior, so if we rewrite to fix the bug we should provide some other way to get it. To do so, add the ability to attach lquery quantifiers to non-'*' items as well as '*'s. Then the pattern '!foo{,}' expresses "any ltree in which no label is foo". For backwards compatibility, the default quantifier for non-'*' items has to be "{1}", although the default for '*' items is '{,}'. I wouldn't have done it like that in a green field, but it's not totally horrible. Armed with that, rewrite checkCond() from scratch. Treating '*' and non-'*' items alike makes it simpler, not more complicated, so that the function actually gets a lot shorter than it was. Filip Rembiałkowski, Tom Lane, Nikita Glukhov, per a very ancient bug report from M. Palm Discussion: https://postgr.es/m/CAP_rww=waX2Oo6q+MbMSiZ9ktdj6eaJj0cQzNu=Ry2cCDij5fw@mail.gmail.com
* Fix assorted typosMagnus Hagander2020-03-31
| | | | Author: Daniel Gustafsson <daniel@yesql.se>
* Fix INSERT OVERRIDING USER VALUE behaviorPeter Eisentraut2020-03-31
| | | | | | | | | | | | | The original implementation disallowed using OVERRIDING USER VALUE on identity columns defined as GENERATED ALWAYS, which is not per standard. So allow that now. Expand documentation and tests around this. Author: Dean Rasheed <dean.a.rasheed@gmail.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> Reviewed-by: Vik Fearing <vik@postgresfriends.org> Discussion: https://www.postgresql.org/message-id/flat/CAEZATCVrh2ufCwmzzM%3Dk_OfuLhTTPBJCdFkimst2kry4oHepuQ%40mail.gmail.com
* Implement operator class parametersAlexander Korotkov2020-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
* Improve handling of parameter differences in physical replicationPeter Eisentraut2020-03-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When certain parameters are changed on a physical replication primary, this is communicated to standbys using the XLOG_PARAMETER_CHANGE WAL record. The standby then checks whether its own settings are at least as big as the ones on the primary. If not, the standby shuts down with a fatal error. The correspondence of settings between primary and standby is required because those settings influence certain shared memory sizings that are required for processing WAL records that the primary might send. For example, if the primary sends a prepared transaction, the standby must have had max_prepared_transaction set appropriately or it won't be able to process those WAL records. However, fatally shutting down the standby immediately upon receipt of the parameter change record might be a bit of an overreaction. The resources related to those settings are not required immediately at that point, and might never be required if the activity on the primary does not exhaust all those resources. If we just let the standby roll on with recovery, it will eventually produce an appropriate error when those resources are used. So this patch relaxes this a bit. Upon receipt of XLOG_PARAMETER_CHANGE, we still check the settings but only issue a warning and set a global flag if there is a problem. Then when we actually hit the resource issue and the flag was set, we issue another warning message with relevant information. At that point we pause recovery, so a hot standby remains usable. We also repeat the last warning message once a minute so it is harder to miss or ignore. Reviewed-by: Sergei Kornilov <sk@zsrv.org> Reviewed-by: Masahiko Sawada <masahiko.sawada@2ndquadrant.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/4ad69a4c-cc9b-0dfe-0352-8b1b0cd36c7b@2ndquadrant.com
* Doc: correct misstatement about ltree label maximum length.Tom Lane2020-03-29
| | | | | | The documentation says that the max length is 255 bytes, but code inspection says it's actually 255 characters; and relevant lengths are stored as uint16 so that that works.
* Document color supportPeter Eisentraut2020-03-29
| | | | | | | Add a documentation appendix that explains the PG_COLOR and PG_COLORS environment variables. Discussion: https://www.postgresql.org/message-id/flat/bbdcce43-bd2e-5599-641b-9b44b9e0add4@2ndquadrant.com
* Protect against overflow of ltree.numlevel and lquery.numlevel.Tom Lane2020-03-28
| | | | | | | | | | | | | | | | | | | | | | These uint16 fields could be overflowed by excessively long input, producing strange results. Complain for invalid input. Likewise check for out-of-range values of the repeat counts in lquery. (We don't try too hard on that one, notably not bothering to detect if atoi's result has overflowed.) Also detect length overflow in ltree_concat. In passing, be more consistent about whether "syntax error" messages include the type name. Also, clarify the documentation about what the size limit is. This has been broken for a long time, so back-patch to all supported branches. Nikita Glukhov, reviewed by Benjie Gillam and Tomas Vondra Discussion: https://postgr.es/m/CAP_rww=waX2Oo6q+MbMSiZ9ktdj6eaJj0cQzNu=Ry2cCDij5fw@mail.gmail.com
* Trigger autovacuum based on number of INSERTsDavid Rowley2020-03-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Traditionally autovacuum has only ever invoked a worker based on the estimated number of dead tuples in a table and for anti-wraparound purposes. For the latter, with certain classes of tables such as insert-only tables, anti-wraparound vacuums could be the first vacuum that the table ever receives. This could often lead to autovacuum workers being busy for extended periods of time due to having to potentially freeze every page in the table. This could be particularly bad for very large tables. New clusters, or recently pg_restored clusters could suffer even more as many large tables may have the same relfrozenxid, which could result in large numbers of tables requiring an anti-wraparound vacuum all at once. Here we aim to reduce the work required by anti-wraparound and aggressive vacuums in general, by triggering autovacuum when the table has received enough INSERTs. This is controlled by adding two new GUCs and reloptions; autovacuum_vacuum_insert_threshold and autovacuum_vacuum_insert_scale_factor. These work exactly the same as the existing scale factor and threshold controls, only base themselves off the number of inserts since the last vacuum, rather than the number of dead tuples. New controls were added rather than reusing the existing controls, to allow these new vacuums to be tuned independently and perhaps even completely disabled altogether, which can be done by setting autovacuum_vacuum_insert_threshold to -1. We make no attempt to skip index cleanup operations on these vacuums as they may trigger for an insert-mostly table which continually doesn't have enough dead tuples to trigger an autovacuum for the purpose of removing those dead tuples. If we were to skip cleaning the indexes in this case, then it is possible for the index(es) to become bloated over time. There are additional benefits to triggering autovacuums based on inserts, as tables which never contain enough dead tuples to trigger an autovacuum are now more likely to receive a vacuum, which can mark more of the table as "allvisible" and encourage the query planner to make use of Index Only Scans. Currently, we still obey vacuum_freeze_min_age when triggering these new autovacuums based on INSERTs. For large insert-only tables, it may be beneficial to lower the table's autovacuum_freeze_min_age so that tuples are eligible to be frozen sooner. Here we've opted not to zero that for these types of vacuums, since the table may just be insert-mostly and we may otherwise freeze tuples that are still destined to be updated or removed in the near future. There was some debate to what exactly the new scale factor and threshold should default to. For now, these are set to 0.2 and 1000, respectively. There may be some motivation to adjust these before the release. Author: Laurenz Albe, Darafei Praliaskouski Reviewed-by: Alvaro Herrera, Masahiko Sawada, Chris Travers, Andres Freund, Justin Pryzby Discussion: https://postgr.es/m/CAC8Q8t%2Bj36G_bLF%3D%2B0iMo6jGNWnLnWb1tujXuJr-%2Bx8ZCCTqoQ%40mail.gmail.com
* Allow walreceiver configuration to change on reloadAlvaro Herrera2020-03-27
| | | | | | | | | | | | | | | The parameters primary_conninfo, primary_slot_name and wal_receiver_create_temp_slot can now be changed with a simple "reload" signal, no longer requiring a server restart. This is achieved by signalling the walreceiver process to terminate and having it start again with the new values. Thanks to Andres Freund, Kyotaro Horiguchi, Fujii Masao for discussion. Author: Sergei Kornilov <sk@zsrv.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/19513901543181143@sas1-19a94364928d.qloud-c.yandex.net
* Set wal_receiver_create_temp_slot PGC_POSTMASTERAlvaro Herrera2020-03-27
| | | | | | | | | | | | | | | | | | | | | | | | | Commit 329730827848 gave walreceiver the ability to create and use a temporary replication slot, and made it controllable by a GUC (enabled by default) that can be changed with SIGHUP. That's useful but has two problems: one, it's possible to cause the origin server to fill its disk if the slot doesn't advance in time; and also there's a disconnect between state passed down via the startup process and GUCs that walreceiver reads directly. We handle the first problem by setting the option to disabled by default. If the user enables it, its on their head to make sure that disk doesn't fill up. We handle the second problem by passing the flag via startup rather than having walreceiver acquire it directly, and making it PGC_POSTMASTER (which ensures a walreceiver always has the fresh value). A future commit can relax this (to PGC_SIGHUP again) by having the startup process signal walreceiver to shutdown whenever the value changes. Author: Sergei Kornilov <sk@zsrv.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/20200122055510.GH174860@paquier.xyz
* pg_dump: Allow dumping data of specific foreign serversAlvaro Herrera2020-03-25
| | | | | | | | | | | | | | | | | | The new command-line switch --include-foreign-data=PATTERN lets the user specify foreign servers from which to dump foreign table data. This can be refined by further inclusion/exclusion switches, so that the user has full control over which tables to dump. A limitation is that this doesn't work in combination with parallel dumps, for implementation reasons. This might be lifted in the future, but requires shuffling some code around. Author: Luis Carril <luis.carril@swarm64.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Surafel Temesgen <surafel3000@gmail.com> Reviewed-by: vignesh C <vignesh21@gmail.com> Reviewed-by: Álvaro Herrera <alvherre@2ndQuadrant.com> Discussion: https://postgr.es/m/LEJPR01MB0185483C0079D2F651B16231E7FC0@LEJPR01MB0185.DEUPRD01.PROD.OUTLOOK.DE
* Re-implement the ereport() macro using __VA_ARGS__.Tom Lane2020-03-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that we require C99, we can depend on __VA_ARGS__ to work, and revising ereport() to use it has several significant benefits: * The extra parentheses around the auxiliary function calls are now optional. Aside from being a bit less ugly, this removes a common gotcha for new contributors, because in some cases the compiler errors you got from forgetting them were unintelligible. * The auxiliary function calls are now evaluated as a comma expression list rather than as extra arguments to errfinish(). This means that compilers can be expected to warn about no-op expressions in the list, allowing detection of several other common mistakes such as forgetting to add errmsg(...) when converting an elog() call to ereport(). * Unlike the situation with extra function arguments, comma expressions are guaranteed to be evaluated left-to-right, so this removes platform dependency in the order of the auxiliary function calls. While that dependency hasn't caused us big problems in the past, this change does allow dropping some rather shaky assumptions around errcontext() domain handling. There's no intention to make wholesale changes of existing ereport calls, but as proof-of-concept this patch removes the extra parens from a couple of calls in postgres.c. While new code can be written either way, code intended to be back-patched will need to use extra parens for awhile yet. It seems worth back-patching this change into v12, so as to reduce the window where we have to be careful about that by one year. Hence, this patch is careful to preserve ABI compatibility; a followup HEAD-only patch will make some additional simplifications. Andres Freund and Tom Lane Discussion: https://postgr.es/m/CA+fd4k6N8EjNvZpM8nme+y+05mz-SM8Z_BgkixzkA34R+ej0Kw@mail.gmail.com
* Doc: fix broken markup.Tom Lane2020-03-24
| | | | | | | Sloppiness in commit cedffbdb8, noted by Erikjan Rijkers. (It's fairly unfortunate that xmllint doesn't catch this.) Discussion: https://postgr.es/m/2e3dc9e4bfa4802d2c9f5fe15bde44de@xs4all.nl
* Report wait event for cost-based vacuum delay.Andres Freund2020-03-23
| | | | | Author: Justin Pryzby Discussion: https://postgr.es/m/20200321040750.GD13662@telsasoft.com
* Prefer standby promotion over recovery pause.Fujii Masao2020-03-24
| | | | | | | | | | | | | | | | | | | | | | | | | | Previously if a promotion was triggered while recovery was paused, the paused state continued. Also recovery could be paused by executing pg_wal_replay_pause() even while a promotion was ongoing. That is, recovery pause had higher priority over a standby promotion. But this behavior was not desirable because most users basically wanted the recovery to complete as soon as possible and the server to become the master when they requested a promotion. This commit changes recovery so that it prefers a promotion over recovery pause. That is, if a promotion is triggered while recovery is paused, the paused state ends and a promotion continues. Also this commit makes recovery pause functions like pg_wal_replay_pause() throw an error if they are executed while a promotion is ongoing. Internally, this commit adds new internal function PromoteIsTriggered() that returns true if a promotion is triggered. Since the name of this function and the existing function IsPromoteTriggered() are confusingly similar, the commit changes the name of IsPromoteTriggered() to IsPromoteSignaled, as more appropriate name. Author: Fujii Masao Reviewed-by: Atsushi Torikoshi, Sergei Kornilov Discussion: https://postgr.es/m/00c194b2-dbbb-2e8a-5b39-13f14048ef0a@oss.nttdata.com
* Add wait events for WAL archive and recovery pause.Fujii Masao2020-03-24
| | | | | | | | | | | This commit introduces new wait events BackupWaitWalArchive and RecoveryPause. The former is reported while waiting for the WAL files required for the backup to be successfully archived. The latter is reported while waiting for recovery in pause state to be resumed. Author: Fujii Masao Reviewed-by: Michael Paquier, Atsushi Torikoshi, Robert Haas Discussion: https://postgr.es/m/f0651f8c-9c96-9f29-0ff9-80414a15308a@oss.nttdata.com
* Report NULL as total backup size if it's not estimated.Fujii Masao2020-03-24
| | | | | | | | | | | | | | Previously 0 was reported in pg_stat_progress_basebackup.total_backup if the total backup size was not estimated. Per discussion, our consensus is that NULL is better choise as the value in total_backup in that case. So this commit makes pg_stat_progress_basebackup view report NULL in total_backup column if the estimation is disabled. Bump catversion. Author: Fujii Masao Reviewed-by: Amit Langote, Magnus Hagander, Alvaro Herrera Discussion: https://postgr.es/m/CABUevExnhOD89zBDuPvfAAh243RzNpwCPEWNLtMYpKHMB8gbAQ@mail.gmail.com
* docs: add backend_type to file-fdw CSV log exampleBruce Momjian2020-03-23
| | | | | | | | | | backend_type was added to the CVS log output in commit 70a7b4776b. Reported-by: Fabrízio de Royes Mello Discussion: https://postgr.es/m/CAFcNs+ruvRks3BV1j7yQ-MvxsswmKJa0cVh2yK5Dd-xXVM8wPw@mail.gmail.com Backpatch-through: master
* Doc: explain that LIKE et al can be used in ANY (sub-select) etc.Tom Lane2020-03-23
| | | | | | | | This wasn't stated anywhere, and it's perhaps not that obvious, since we get questions about it from time to time. Also undocumented was that the parser actually translates these into operators. Discussion: https://postgr.es/m/CAFj8pRBkvZ71BqGKZnBBG4=0cKG+s50Dy+DYmrizUKEpAtdc+w@mail.gmail.com
* Fix our getopt_long's behavior for a command line argument of just "-".Tom Lane2020-03-23
| | | | | | | | | | | | | | | | | | | src/port/getopt_long.c failed on such an argument, always seeing it as an unrecognized switch. This is unhelpful; better is to treat such an item as a non-switch argument. That behavior is what we find in GNU's getopt_long(); it's what src/port/getopt.c does; and it is required by POSIX for getopt(), which getopt_long() ought to be generally a superset of. Moreover, it's expected by ecpg, which intends an argument of "-" to mean "read from stdin". So fix it. Also add some documentation about ecpg's behavior in this area, since that was miserably underdocumented. I had to reverse-engineer it from the code. Per bug #16304 from James Gray. Back-patch to all supported branches, since this has been broken forever. Discussion: https://postgr.es/m/16304-c662b00a1322db7f@postgresql.org
* Doc: Fix type of some storage parameters in CREATE TABLE pageMichael Paquier2020-03-23
| | | | | | | | | | autovacuum_vacuum_scale_factor and autovacuum_analyze_scale_factor have been documented as "float4", but "floating type" is used in this case for GUCs and relation options in the documentation. Author: Atsushi Torikoshi Discussion: https://postgr.es/m/CACZ0uYFf_p9BpbjLccx3CA=eM1Hk2Te=ULY4iptGLUhL-JxCPA@mail.gmail.com Backpatch-through: 9.5
* Revert "Skip WAL for new relfilenodes, under wal_level=minimal."Noah Misch2020-03-22
| | | | | | | | This reverts commit cb2fd7eac285b1b0a24eeb2b8ed4456b66c5a09f. Per numerous buildfarm members, it was incompatible with parallel query, and a test case assumed LP64. Back-patch to 9.5 (all supported versions). Discussion: https://postgr.es/m/20200321224920.GB1763544@rfd.leadboat.com
* Skip WAL for new relfilenodes, under wal_level=minimal.Noah Misch2020-03-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Until now, only selected bulk operations (e.g. COPY) did this. If a given relfilenode received both a WAL-skipping COPY and a WAL-logged operation (e.g. INSERT), recovery could lose tuples from the COPY. See src/backend/access/transam/README section "Skipping WAL for New RelFileNode" for the new coding rules. Maintainers of table access methods should examine that section. To maintain data durability, just before commit, we choose between an fsync of the relfilenode and copying its contents to WAL. A new GUC, wal_skip_threshold, guides that choice. If this change slows a workload that creates small, permanent relfilenodes under wal_level=minimal, try adjusting wal_skip_threshold. Users setting a timeout on COMMIT may need to adjust that timeout, and log_min_duration_statement analysis will reflect time consumption moving to COMMIT from commands like COPY. Internally, this requires a reliable determination of whether RollbackAndReleaseCurrentSubTransaction() would unlink a relation's current relfilenode. Introduce rd_firstRelfilenodeSubid. Amend the specification of rd_createSubid such that the field is zero when a new rel has an old rd_node. Make relcache.c retain entries for certain dropped relations until end of transaction. Back-patch to 9.5 (all supported versions). This introduces a new WAL record type, XLOG_GIST_ASSIGN_LSN, without bumping XLOG_PAGE_MAGIC. As always, update standby systems before master systems. This changes sizeof(RelationData) and sizeof(IndexStmt), breaking binary compatibility for affected extensions. (The most recent commit to affect the same class of extensions was 089e4d405d0f3b94c74a2c6a54357a84a681754b.) Kyotaro Horiguchi, reviewed (in earlier, similar versions) by Robert Haas. Heikki Linnakangas and Michael Paquier implemented earlier designs that materially clarified the problem. Reviewed, in earlier designs, by Andrew Dunstan, Andres Freund, Alvaro Herrera, Tom Lane, Fujii Masao, and Simon Riggs. Reported by Martijn van Oosterhout. Discussion: https://postgr.es/m/20150702220524.GA9392@svana.org
* docs: use alias in WHERE clause of full text search exampleBruce Momjian2020-03-20
| | | | | | | | | | | The current doc query specified an alias in the FROM clause and used in it the target list, but not in the WHERE clause. Reported-by: axykon@gmail.com Discussion: https://postgr.es/m/158316348159.30450.16075357948244298217@wrigleys.postgresql.org Backpatch-through: 9.5
* Introduce "anycompatible" family of polymorphic types.Tom Lane2020-03-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the pseudo-types anycompatible, anycompatiblearray, anycompatiblenonarray, and anycompatiblerange. They work much like anyelement, anyarray, anynonarray, and anyrange respectively, except that the actual input values need not match precisely in type. Instead, if we can find a common supertype (using the same rules as for UNION/CASE type resolution), then the parser automatically promotes the input values to that type. For example, "myfunc(anycompatible, anycompatible)" can match a call with one integer and one bigint argument, with the integer automatically promoted to bigint. With anyelement in the definition, the user would have had to cast the integer explicitly. The new types also provide a second, independent set of type variables for function matching; thus with "myfunc(anyelement, anyelement, anycompatible) returns anycompatible" the first two arguments are constrained to be the same type, but the third can be some other type, and the result has the type of the third argument. The need for more than one set of type variables was foreseen back when we first invented the polymorphic types, but we never did anything about it. Pavel Stehule, revised a bit by me Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
* Make pg_basebackup ask the server to estimate the total backup size, by default.Fujii Masao2020-03-19
| | | | | | | | | | | | | | | | | This commit changes pg_basebackup so that it specifies PROGRESS option in BASE_BACKUP replication command whether --progress is specified or not. This causes the server to estimate the total backup size and report it in pg_stat_progress_basebackup.backup_total, by default. This is reasonable default because the time required for the estimation would not be so large in most cases. Also this commit adds new option --no-estimate-size to pg_basebackup. This option prevents the server from the estimation, and so is useful to avoid such estimation time if it's too long. Author: Fujii Masao Reviewed-by: Magnus Hagander, Amit Langote Discussion: https://postgr.es/m/CABUevEyDPPSjP7KRvfTXPdqOdY5aWNkqsB5aAXs3bco5ZwtGHg@mail.gmail.com
* Rename the recovery-related wait events.Fujii Masao2020-03-19
| | | | | | | | | | | | | | | | | | | | This commit renames RecoveryWalAll and RecoveryWalStream wait events to RecoveryWalStream and RecoveryRetrieveRetryInterval, respectively, in order to make the names and what they are more consistent. For example, previously RecoveryWalAll was reported as a wait event while the recovery was waiting for WAL from a stream, and which was confusing because the name was very different from the situation where the wait actually could happen. The names of macro variables for those wait events also are renamed accordingly. This commit also changes the category of RecoveryRetrieveRetryInterval to Timeout from Activity because the wait event is reported while waiting based on wal_retrieve_retry_interval. Author: Fujii Masao Reviewed-by: Kyotaro Horiguchi, Atsushi Torikoshi Discussion: https://postgr.es/m/124997ee-096a-5d09-d8da-2c7a57d0816e@oss.nttdata.com
* Disk-based Hash Aggregation.Jeff Davis2020-03-18
| | | | | | | | | | | | | | | | | | | | | | While performing hash aggregation, track memory usage when adding new groups to a hash table. If the memory usage exceeds work_mem, enter "spill mode". In spill mode, new groups are not created in the hash table(s), but existing groups continue to be advanced if input tuples match. Tuples that would cause a new group to be created are instead spilled to a logical tape to be processed later. The tuples are spilled in a partitioned fashion. When all tuples from the outer plan are processed (either by advancing the group or spilling the tuple), finalize and emit the groups from the hash table. Then, create new batches of work from the spilled partitions, and select one of the saved batches and process it (possibly spilling recursively). Author: Jeff Davis Reviewed-by: Tomas Vondra, Adam Lee, Justin Pryzby, Taylor Vesely, Melanie Plageman Discussion: https://postgr.es/m/507ac540ec7c20136364b5272acbcd4574aa76ef.camel@j-davis.com
* Enable BEFORE row-level triggers for partitioned tablesAlvaro Herrera2020-03-18
| | | | | | | | ... with the limitation that the tuple must remain in the same partition. Reviewed-by: Ashutosh Bapat Discussion: https://postgr.es/m/20200227165158.GA2071@alvherre.pgsql
* Implement type regcollationPeter Eisentraut2020-03-18
| | | | | | | | | This will be helpful for a following commit and it's also just generally useful, like the other reg* types. Author: Julien Rouhaud Reviewed-by: Thomas Munro and Michael Paquier Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
* Document pg_statistic_ext.stxstattargetTomas Vondra2020-03-18
| | | | | | | Commit d06215d03b added a new attribute to pg_statistic_ext catalog, but failed to add it to document it properly. Reported-by: Noriyoshi Shinoda <noriyoshi.shinoda@hpe.com>
* Correct the descriptions of recovery-related wait events in docs.Fujii Masao2020-03-18
| | | | | | | | | | | This commit corrects the descriptions of RecoveryWalAll and RecoveryWalStream wait events in the documentation. Back-patch to v10 where those wait events were added. Author: Fujii Masao Reviewed-by: Kyotaro Horiguchi, Atsushi Torikoshi Discussion: https://postgr.es/m/124997ee-096a-5d09-d8da-2c7a57d0816e@oss.nttdata.com
* doc: Update documentation about reg* typesPeter Eisentraut2020-03-18
| | | | | | Add missing index entries, add missing information on pg_upgrade man page, order things alphabetical instead of (apparently) in the order they were implemented, reduce repetitiveness a bit.
* Update the description of type of check_option reloption in docs.Fujii Masao2020-03-18
| | | | | | | | | Commit 773df883e8f7 changed the type of check_option reloption from string to enum. But it forgot to update the description of the type in the documentation. Author: Atsushi Torikoshi Discussion: https://postgr.es/m/CACZ0uYFvHF4n6yxF390YZgr4Q0Z0c2w0ihu=DLb8ipNOnNcqzQ@mail.gmail.com
* Doc: Correct deduplicate_items varlistentry id.Peter Geoghegan2020-03-17
| | | | | | | | | Use a varlistentry id for the deduplicate_items storage parameter that is derived from the name of the parameter itself. This oversight happened because the storage parameter was renamed relatively late during the development of the patch that became commit 0d861bbb.
* Doc: clarify behavior of "anyrange" pseudo-type.Tom Lane2020-03-17
| | | | | | | | | I noticed that we completely failed to document the restriction that an "anyrange" result type has to be inferred from an "anyrange" input. The docs also were less clear than they could be about the relationship between "anyrange" and "anyarray". It's been like this all along, so back-patch.
* Use pkg-config, if available, to locate libxml2 during configure.Tom Lane2020-03-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | If pkg-config is installed and knows about libxml2, use its information rather than asking xml2-config. Otherwise proceed as before. This patch allows "configure --with-libxml" to succeed on platforms that have pkg-config but not xml2-config, which is likely to soon become a typical situation. The old mechanism can be forced by setting XML2_CONFIG explicitly (hence, build processes that were already doing so will certainly not need adjustment). Also, it's now possible to set XML2_CFLAGS and XML2_LIBS explicitly to override both programs. There is a small risk of this breaking existing build processes, if there are multiple libxml2 installations on the machine and pkg-config disagrees with xml2-config about which to use. The only case where that seems really likely is if a builder has tried to select a non-default xml2-config by putting it early in his PATH rather than setting XML2_CONFIG. Plan to warn against that in the minor release notes. Back-patch to v10; before that we had no pkg-config infrastructure, and it doesn't seem worth adding it for this. Hugh McMaster and Tom Lane; Peter Eisentraut also made an earlier attempt at this, from which I lifted most of the docs changes. Discussion: https://postgr.es/m/CAN9BcdvfUwc9Yx5015bLH2TOiQ-M+t_NADBSPhMF7dZ=pLa_iw@mail.gmail.com
* Add the type information for index storage parameters to the documentation.Fujii Masao2020-03-17
| | | | | Author: Atsushi Torikoshi Discussion: https://postgr.es/m/CACZ0uYFQebs4WT5eu3dK4qm_2PurZuvB++8nDvSBG0ebRWmbdg@mail.gmail.com
* Avoid holding a directory FD open across assorted SRF calls.Tom Lane2020-03-16
| | | | | | | | | | | | | | | | | This extends the fixes made in commit 085b6b667 to other SRFs with the same bug, namely pg_logdir_ls(), pgrowlocks(), pg_timezone_names(), pg_ls_dir(), and pg_tablespace_databases(). Also adjust various comments and documentation to warn against expecting to clean up resources during a ValuePerCall SRF's final call. Back-patch to all supported branches, since these functions were all born broken. Justin Pryzby, with cosmetic tweaks by me Discussion: https://postgr.es/m/20200308173103.GC1357@telsasoft.com
* Document pg_ls_*dir hiding of directories and special filesAlvaro Herrera2020-03-16
| | | | | | | | | | | | | | | | | It's strange that a directory-listing function does not list all entries in a directory, so let's at least document it. This involves pg_ls_logdir pg_ls_waldir pg_ls_archive_statusdir pg_ls_tmpdir Backpatch as far back as it applies cleanly (and as far as as each function exists). REL_10_STABLE uses different wording, but hopefully people are not reading docs so old to write new apps anyway. Author: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://postgr.es/m/20200305161838.GJ684@telsasoft.com
* Introduce a maintenance_io_concurrency setting.Thomas Munro2020-03-16
| | | | | | | | | | | | | Introduce a GUC and a tablespace option to control I/O prefetching, much like effective_io_concurrency, but for work that is done on behalf of many client sessions. Use the new setting in heapam.c instead of the hard-coded formula effective_io_concurrency + 10 introduced by commit 558a9165e08. Go with a default value of 10 for now, because it's a round number pretty close to the value used for that existing case. Discussion: https://postgr.es/m/CA%2BhUKGJUw08dPs_3EUcdO6M90GnjofPYrWp4YSLaBkgYwS-AqA%40mail.gmail.com
* Add backend type to csvlog and optionally log_line_prefixPeter Eisentraut2020-03-15
| | | | | | | | | | | | | The backend type, which corresponds to what pg_stat_activity.backend_type shows, is added as a column to the csvlog and can optionally be added to log_line_prefix using the new %b placeholder. Reviewed-by: Julien Rouhaud <rjuju123@gmail.com> Reviewed-by: Kuntal Ghosh <kuntalghosh.2007@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Discussion: https://www.postgresql.org/message-id/flat/c65e5196-4f04-4ead-9353-6088c19615a3@2ndquadrant.com
* Use functional dependencies to estimate ScalarArrayOpExprTomas Vondra2020-03-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Until now functional dependencies supported only simple equality clauses and clauses that can be trivially translated to equalities. This commit allows estimation of some ScalarArrayOpExpr (IN/ANY) clauses. For IN clauses we can do this thanks to using operator with equality semantics, which means an IN clause WHERE c IN (1, 2, ..., N) can be translated to WHERE (c = 1 OR c = 2 OR ... OR c = N) IN clauses are now considered compatible with functional dependencies, and rely on the same assumption of consistency of queries with data (which is an assumption we already used for simple equality clauses). This applies also to ALL clauses with an equality operator, which can be considered equivalent to IN clause. ALL clauses are still considered incompatible, although there's some discussion about maybe relaxing this in the future. Author: Pierre Ducroquet Reviewed-by: Tomas Vondra, Dean Rasheed Discussion: https://www.postgresql.org/message-id/flat/13902317.Eha0YfKkKy%40pierred-pdoc