aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
...
* MergeAttributes code deduplicationPeter Eisentraut2024-01-26
| | | | | | | | The code handling NOT NULL constraints is duplicated in blocks merging the attribute definition incrementally. Deduplicate that code. Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/52a125e4-ff9a-95f5-9f61-b87cf447e4da@eisentraut.org
* Reindex toast before its main relation in reindex_relation()Michael Paquier2024-01-26
| | | | | | | | | | | | | | | | | | | | | | | | This commit changes the order of reindex on a relation so as a toast relation is processed before its main relation. The original order, where a rebuild was first done for the indexes on the main table, could be a problem in the event of a corruption of a toast index, because, as scans of a toast index may be required to rebuild the indexes on the main relation, this could lead to failures with REINDEX TABLE without being able to fix anything. Rebuilding corrupted toast indexes before this change was possible but troublesome, as it was necessary to issue a REINDEX on the toast relation first, followed by a REINDEX on the main relation. Changing the order of these operations should make things easier when rebuilding corrupted indexes, as toast indexes would be rebuilt before they are used for the indexes on the main relation. Per request from Richard Vesely. Author: Gurjeet Singh Reviewed-by: Nathan Bossart, Michael Paquier Discussion: https://postgr.es/m/18016-2bd9b549b1fe49b3@postgresql.org
* De-dupicate Memoize cache keysDavid Rowley2024-01-26
| | | | | | | | | | | | | | | | | It was possible when determining the cache keys for a Memoize path that if the same expr appeared twice in the parameterized path's ppi_clauses and/or in the Nested Loop's inner relation's lateral_vars.  If this happened the Memoize node's cache keys would contain duplicates.  This isn't a problem for correctness, all it means is that the cache lookups will be suboptimal due to having redundant work to do on every hash table lookup and insert. Here we adjust paraminfo_get_equal_hashops() to look for duplicates and ignore them when we find them. Author: David Rowley Reviewed-by: Richard Guo Discussion: https://postgr.es/m/422277.1706207562%40sss.pgh.pa.us
* Fix comment in index.cMichael Paquier2024-01-26
| | | | | | | Extracted from a larger patch by the same author. Author: Gurjeet Singh Discussion: https://postgr.es/m/CABwTF4WX=m5pQvKXvLFJoEH=hSd6O=iZSqxVqHKjFm+iL-AO=w@mail.gmail.com
* Improve NestLoopParam generation for lateral subqueriesDavid Rowley2024-01-26
| | | | | | | | | | | | | | | | | | | | | | | | | It was possible in cases where we had a LATERAL joined subquery that when the same Var is mentioned in both the lateral references and in the outer Vars of the scan clauses that the given Var wouldn't be assigned to the same NestLoopParam. This could cause issues in Memoize as the cache key would reference the Var for the scan clauses but when the parameter for the lateral references changed some code in Memoize would see that some other parameter had changed that's not part of the cache key and end up purging the entire cache as a result, thinking the cache had become stale. This could result in a Nested Loop -> Memoize plan being quite inefficient as, in the worst case, the cache purging could result in never getting a cache hit. In no cases could this problem lead to incorrect query results. Here we switch the order of operations so that we create NestLoopParam for the lateral references first before doing replace_nestloop_params(). replace_nestloop_params() will find and reuse the existing NestLoopParam in cases where the Var exists in both locations. Author: Richard Guo Reviewed-by: Tom Lane, David Rowley Discussion: https://postgr.es/m/CAMbWs48XHJEK1Q1CzAQ7L9sTANTs9W1cepXu8%3DKc0quUL%2Btg4Q%40mail.gmail.com
* Revert "Add support for parsing of large XML data (>= 10MB)"Michael Paquier2024-01-26
| | | | | | | | | | This reverts commit 2197d06224a1, following a discussion over a Coverity report where issues like the "Billion laugh attack" could cause the backend to waste CPU and memory even if a client applied checks on the size of the data given in input, and libxml2 does not offer guarantees that input limits are respected under XML_PARSE_HUGE. Discussion: https://postgr.es/m/ZbHlgrPLtBZyr_QW@paquier.xyz
* Update comment, generation mem contexts have a "keeper" blockHeikki Linnakangas2024-01-26
| | | | | The keeper block was introduced in commit 1b0d9aa4f7, but it forgot to update this comment.
* Support TZ and OF format codes in to_timestamp().Tom Lane2024-01-25
| | | | | | | | | | | | | | | | | | | | | | | | Formerly, these were only supported in to_char(), but there seems little reason for that restriction. We should at least have enough support to permit round-tripping the output of to_char(). In that spirit, TZ accepts either zone abbreviations or numeric (HH or HH:MM) offsets, which are the cases that to_char() can output. In an ideal world we'd make it take full zone names too, but that seems like it'd introduce an unreasonable amount of ambiguity, since the rules for POSIX-spec zone names are so lax. OF is a subset of this, accepting only HH or HH:MM. One small benefit of this improvement is that we can simplify jsonpath's executeDateTimeMethod function, which no longer needs to consider the HH and HH:MM cases separately. Moreover, letting it accept zone abbreviations means it will accept "Z" to mean UTC, which is emitted by JSON.stringify() for example. Patch by me, reviewed by Aleksander Alekseev and Daniel Gustafsson Discussion: https://postgr.es/m/1681086.1686673242@sss.pgh.pa.us
* Clean up a bug in sql/json items commit 66ea94e8e6Andrew Dunstan2024-01-25
| | | | | | | | | Remove a buggy and unnecessary test, along with an unnecessary pstrdup() and a line of dead code. Per report, diagnosis and fix from Tom Lane Discussion: https://postgr.es/m/439811.1706211069@sss.pgh.pa.us
* Implement various jsonpath methodsAndrew Dunstan2024-01-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit implements ithe jsonpath .bigint(), .boolean(), .date(), .decimal([precision [, scale]]), .integer(), .number(), .string(), .time(), .time_tz(), .timestamp(), and .timestamp_tz() methods. .bigint() converts the given JSON string or a numeric value to the bigint type representation. .boolean() converts the given JSON string, numeric, or boolean value to the boolean type representation. In the numeric case, only integers are allowed. We use the parse_bool() backend function to convert a string to a bool. .decimal([precision [, scale]]) converts the given JSON string or a numeric value to the numeric type representation. If precision and scale are provided for .decimal(), then it is converted to the equivalent numeric typmod and applied to the numeric number. .integer() and .number() convert the given JSON string or a numeric value to the int4 and numeric type representation. .string() uses the datatype's output function to convert numeric and various date/time types to the string representation. The JSON string representing a valid date/time is converted to the specific date or time type representation using jsonpath .date(), .time(), .time_tz(), .timestamp(), .timestamp_tz() methods. The changes use the infrastructure of the .datetime() method and perform the datatype conversion as appropriate. Unlike the .datetime() method, none of these methods accept a format template and use ISO DateTime format instead. However, except for .date(), the date/time related methods take an optional precision to adjust the fractional seconds. Jeevan Chalke, reviewed by Peter Eisentraut and Andrew Dunstan.
* Add a const decorationPeter Eisentraut2024-01-25
| | | | | | Useful for a subsequent patch. Discussion: https://www.postgresql.org/message-id/flat/52a125e4-ff9a-95f5-9f61-b87cf447e4da@eisentraut.org
* Remove dummy_spinlockAlvaro Herrera2024-01-25
| | | | It's been unused since 1b468a131bd2 (2015).
* MergeAttributes: convert pg_attribute back to ColumnDef before comparingPeter Eisentraut2024-01-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MergeAttributes() has a loop to merge multiple inheritance parents into a column column definition. The merged-so-far definition is stored in a ColumnDef node. If we have to merge columns from multiple inheritance parents (if the name matches), then we have to check whether various column properties (type, collation, etc.) match. The current code extracts the pg_attribute value of the currently-considered inheritance parent and compares it against the merged-so-far ColumnDef value. If the currently considered column doesn't match any previously inherited column, we make a new ColumnDef node from the pg_attribute information and add it to the column list. This patch rearranges this so that we create the ColumnDef node first in either case. Then the code that checks the column properties for compatibility compares ColumnDef against ColumnDef (instead of ColumnDef against pg_attribute). This makes the code more symmetric and easier to follow. Also, later in MergeAttributes(), there is a similar but separate loop that merges the new local column definition with the combination of the inheritance parents established in the first loop. That comparison is already ColumnDef-vs-ColumnDef. With this change, both of these can use similar-looking logic. (A future project might be to extract these two sets of code into a common routine that encodes all the knowledge of whether two column definitions are compatible. But this isn't currently straightforward because we want to give different error messages in the two cases.) Furthermore, by avoiding the use of Form_pg_attribute here, we make it easier to make changes in the pg_attribute layout without having to worry about the local needs of tablecmds.c. Because MergeAttributes() is hugely long, it's sometimes hard to know where in the function you are currently looking. To help with that, I also renamed some variables to make it clearer where you are currently looking. The first look is "prev" vs. "new", the second loop is "inh" vs. "new". Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/52a125e4-ff9a-95f5-9f61-b87cf447e4da@eisentraut.org
* Fix s_lock_test compileAlvaro Herrera2024-01-25
| | | | | | | | | This is a mostly unused tool, but I discovered while nosing around the Makefile that it hasn't been kept in line with other changes. Fix it. Backpatching doesn't appear to be necessary. Discussion: https://postgr.es/m/202401241114.ied53jcich72@alvherre.pgsql
* Silence compiler warning introduced in 1edb3b491bAmit Langote2024-01-25
| | | | | Reported-by: Richard Guo <guofenglinux@gmail.com> Discussion: https://postgr.es/m/CAMbWs48qEoe9Du5tuUxrkGQ6VC9oy+tQOORQ6jpob14-E1Z+jg@mail.gmail.com
* Add more LOG messages when starting and ending recovery from a backupMichael Paquier2024-01-25
| | | | | | | | | | | | | | Three LOG messages are added in the recovery code paths, providing information that can be useful to track corruption issues depending on the state of the cluster, telling that: - Recovery has started from a backup_label. - Recovery is restarting from a backup start LSN, without a backup_label. - Recovery has completed from a backup. Author: Andres Freund Reviewed-by: David Steele, Laurenz Albe, Michael Paquier Discussion: https://postgr.es/m/20231117041811.vz4vgkthwjnwp2pp@awork3.anarazel.de
* Allow to enable failover property for replication slots via SQL API.Amit Kapila2024-01-25
| | | | | | | | | | | | | | | | | | | | | This commit adds the failover property to the replication slot. The failover property indicates whether the slot will be synced to the standby servers, enabling the resumption of corresponding logical replication after failover. But note that this commit does not yet include the capability to sync the replication slot; the subsequent commits will add that capability. A new optional parameter 'failover' is added to the pg_create_logical_replication_slot() function. We will also enable to set 'failover' option for slots via the subscription commands in the subsequent commits. The value of the 'failover' flag is displayed as part of pg_replication_slots view. Author: Hou Zhijie, Shveta Malik, Ajin Cherian Reviewed-by: Peter Smith, Bertrand Drouvot, Dilip Kumar, Masahiko Sawada, Nisha Moond, Kuroda, Hayato, Amit Kapila Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
* Remove redundant HandleWalWriterInterrupts().Fujii Masao2024-01-25
| | | | | | | | | | | Because of commit 1bdd54e662, the code of HandleWalWriterInterrupts() became the same as HandleMainLoopInterrupts(). So this commit removes HandleWalWriterInterrupts() and makes walwriter use HandleMainLoopInterrupts() for improved code simplicity. Author: Fujii Masao Reviewed-by: Bharath Rupireddy, Nathan Bossart Discussion: https://postgr.es/m/CAHGQGwHUtwCsB4DnqFLiMiVzjcA=zmeCKf9_pgQM-yJopydatw@mail.gmail.com
* jit: Require at least LLVM 10.Thomas Munro2024-01-25
| | | | | | | | Remove support for older LLVM versions. The default on common software distributions will be at least LLVM 10 when PostgreSQL 17 ships. Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Discussion: https://postgr.es/m/CA%2BhUKGLhNs5geZaVNj2EJ79Dx9W8fyWUU3HxcpZy55sMGcY%3DiA%40mail.gmail.com
* Add progress reporting of skipped tuples during COPY FROM.Masahiko Sawada2024-01-25
| | | | | | | | | | | | | | | | 9e2d870119 enabled the COPY command to skip malformed data, however there was no visibility into how many tuples were actually skipped during the COPY FROM. This commit adds a new "tuples_skipped" column to pg_stat_progress_copy view to report the number of tuples that were skipped because they contain malformed data. Bump catalog version. Author: Atsushi Torikoshi Reviewed-by: Masahiko Sawada Discussion: https://postgr.es/m/d12fd8c99adcae2744212cb23feff6ed%40oss.nttdata.com
* Track LLVM 18 changes.Thomas Munro2024-01-25
| | | | | | | | | | A function was given a newly standard name from C++20 in LLVM 16. Then LLVM 18 added a deprecation warning for the old name, and it is about to ship, so it's time to adjust that. Back-patch to all supported releases. Discussion: https://www.postgresql.org/message-id/CA+hUKGLbuVhH6mqS8z+FwAn4=5dHs0bAWmEMZ3B+iYHWKC4-ZA@mail.gmail.com
* Rename index "abc" in aggregates.sqlAlexander Korotkov2024-01-24
| | | | | | | In order to prevent name collision with table "abc" in namespace.sql. Reported-by: Nathan Bossart Discussion: https://postgr.es/m/20240124173735.GA2708416%40nathanxps13
* Add temporal PRIMARY KEY and UNIQUE constraintsPeter Eisentraut2024-01-24
| | | | | | | | | | | | Add WITHOUT OVERLAPS clause to PRIMARY KEY and UNIQUE constraints. These are backed by GiST indexes instead of B-tree indexes, since they are essentially exclusion constraints with = for the scalar parts of the key and && for the temporal part. Author: Paul A. Jungwirth <pj@illuminatedcomputing.com> Reviewed-by: Peter Eisentraut <peter@eisentraut.org> Reviewed-by: jian he <jian.universality@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com
* Improve notation of BuiltinTrancheNamesAlvaro Herrera2024-01-24
| | | | | | | | | | | | | | | Use C99 designated initializer syntax for array elements, instead of writing the position in a comment. This is less verbose and much more readable. Akin to cc150596341e. One disadvantage is that the BuiltinTrancheNames array now has a hole of 51 NULLs -- previously, the array elements were shifted 51 elements downward to avoid this. This can be fixed by merging the IndividualLWLockNames array into BuiltinTrancheNames, which would occupy those 51 pointers, but because it requires some arguably ugly Meson hackery, it's left for later. Discussion: https://postgr.es/m/202401231025.gbv4nnte5fmm@alvherre.pgsql
* pgbench: Add \syncpipelineMichael Paquier2024-01-24
| | | | | | | | | | | | This change adds a new meta-command called \syncpipeline to pgbench, able to send a sync message without flushing using the new libpq function PQsendPipelineSync(). This meta-command is available within a block made of \startpipeline and \endpipeline. Author: Anthonin Bonnefoy Discussion: https://postgr.es/m/CAO6_XqpcNhW6LZHLF-2NpPzdTbyMm4-RVkr3+AP5cOKSm9hrWA@mail.gmail.com
* Refactor code used by jsonpath executor to fetch variablesAmit Langote2024-01-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, getJsonPathVariable() directly extracts a named variable/key from the source Jsonb value. This commit puts that logic into a callback function called by getJsonPathVariable(). Other implementations of the callback may accept different forms of the source value(s), for example, a List of values passed from outside jsonpath_exec.c. Extracted from a much larger patch to add SQL/JSON query functions. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> Author: Teodor Sigaev <teodor@sigaev.ru> Author: Oleg Bartunov <obartunov@gmail.com> Author: Alexander Korotkov <aekorotkov@gmail.com> Author: Andrew Dunstan <andrew@dunslane.net> Author: Amit Langote <amitlangote09@gmail.com> Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby, Álvaro Herrera, Jian He, Peter Eisentraut Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org Discussion: https://postgr.es/m/CA+HiwqHROpf9e644D8BRqYvaAPmgBZVup-xKMDPk-nd4EpgzHw@mail.gmail.com Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
* Adjust populate_record_field() to handle errors softlyAmit Langote2024-01-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a Node *escontext parameter to it and a bunch of functions downstream to it, replacing any ereport()s in that path by either errsave() or ereturn() as appropriate. This also adds code to those functions where necessary to return early upon encountering a soft error. The changes here are mainly intended to suppress errors in the functions of jsonfuncs.c. Functions in any external modules, such as arrayfuncs.c, that those functions may in turn call are not changed here based on the assumption that the various checks in jsonfuncs.c functions should ensure that only values that are structurally valid get passed to the functions in those external modules. An exception is made for domain_check() to allow handling domain constraint violation errors softly. For testing, this adds a function jsonb_populate_record_valid(), which returns true if jsonb_populate_record() would finish without causing an error for the provided JSON object, false otherwise. Note that jsonb_populate_record() internally calls populate_record(), which in turn uses populate_record_field(). Extracted from a much larger patch to add SQL/JSON query functions. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> Author: Teodor Sigaev <teodor@sigaev.ru> Author: Oleg Bartunov <obartunov@gmail.com> Author: Alexander Korotkov <aekorotkov@gmail.com> Author: Andrew Dunstan <andrew@dunslane.net> Author: Amit Langote <amitlangote09@gmail.com> Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby, Álvaro Herrera, Jian He, Peter Eisentraut Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org Discussion: https://postgr.es/m/CA+HiwqHROpf9e644D8BRqYvaAPmgBZVup-xKMDPk-nd4EpgzHw@mail.gmail.com Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
* Add soft error handling to some expression nodesAmit Langote2024-01-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adjusts the code for CoerceViaIO and CoerceToDomain expression nodes to handle errors softly. For CoerceViaIo, this adds a new ExprEvalStep opcode EEOP_IOCOERCE_SAFE, which is implemented in the new accompanying function ExecEvalCoerceViaIOSafe(). The only difference from EEOP_IOCOERCE's inline implementation is that the input function receives an ErrorSaveContext via the function's FunctionCallInfo.context, which it can use to handle errors softly. For CoerceToDomain, this simply entails replacing the ereport() in ExecEvalConstraintNotNull() and ExecEvalConstraintCheck() by errsave() passing it the ErrorSaveContext passed in the expression's ExprEvalStep. In both cases, the ErrorSaveContext to be used is passed by setting ExprState.escontext to point to it before calling ExecInitExprRec() on the expression tree whose errors are to be handled softly. Note that there's no functional change as of this commit as no call site of ExecInitExprRec() has been changed. This is intended for implementing new SQL/JSON expression nodes in future commits. Extracted from a much larger patch to add SQL/JSON query functions. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> Author: Teodor Sigaev <teodor@sigaev.ru> Author: Oleg Bartunov <obartunov@gmail.com> Author: Alexander Korotkov <aekorotkov@gmail.com> Author: Andrew Dunstan <andrew@dunslane.net> Author: Amit Langote <amitlangote09@gmail.com> Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby, Álvaro Herrera, Jian He, Peter Eisentraut Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org Discussion: https://postgr.es/m/CA+HiwqHROpf9e644D8BRqYvaAPmgBZVup-xKMDPk-nd4EpgzHw@mail.gmail.com Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
* Fix ALTER TABLE .. ADD COLUMN with complex inheritance treesMichael Paquier2024-01-24
| | | | | | | | | | | | | | | | This command, when used to add a column on a parent table with a complex inheritance tree, tried to update multiple times the same tuple in pg_attribute for a child table when incrementing attinhcount, causing failures with "tuple already updated by self" because of a missing CommandCounterIncrement() between two updates. This exists for a rather long time, so backpatch all the way down. Reported-by: Alexander Lakhin Author: Tender Wang Reviewed-by: Richard Guo Discussion: https://postgr.es/m/18297-b04cd83a55b51e35@postgresql.org Backpatch-through: 12
* Support shared libraries on Android (using make)Peter Eisentraut2024-01-23
| | | | | | | | | While the rest of the make build system maps Android to Linux, Android uses unversioned shared libraries (like "libpq.so"). This patch makes it so. (Meson already supported it.) Reported-by: Matthias Kuhn <matthias@opengis.ch> Discussion: https://www.postgresql.org/message-id/flat/CAC7zN94TdsHhY88XkroJzSMx7E%3DBQpV9LKKjNSEnTM04ihoWCA%40mail.gmail.com
* Fix makefiles for newly added filesPeter Eisentraut2024-01-23
| | | | | | | The new files added in 9b1a6f50b9 need to be mentioned in a few more places in the makefiles. Discussion: https://www.postgresql.org/message-id/flat/E1rSAY2-002hIk-4y%40gemulon.postgresql.org
* Revert "libpqwalreceiver: Convert to libpq-be-fe-helpers.h"Heikki Linnakangas2024-01-23
| | | | | | | | | | | | | | | | | | | This reverts commit 728f86fec65537eade8d9e751961782ddb527934. The signal handling was a few bricks shy of a load in that commit, which made the walreceiver non-responsive to SIGTERM while it was waiting for the connection to be established. That prevented a standby from being promoted. Since it was non-essential refactoring, let's revert it to make v16 work the same as earlier releases. I reverted it in 'master' too, to keep the branches in sync. The refactoring was a good idea as such, but it needs a bit more work. Once we have developed a complete patch with this issue fixed, let's re-apply that to 'master'. Reported-by: Kyotaro Horiguchi Backpatch-through: 16 Discussion: https://www.postgresql.org/message-id/20231231.200741.1078989336605759878.horikyota.ntt@gmail.com
* Generate syscache info from catalog filesPeter Eisentraut2024-01-23
| | | | | | | | | | Add a new genbki macros MAKE_SYSCACHE that specifies the syscache ID macro, the underlying index, and the number of buckets. From that, we can generate the existing tables in syscache.h and syscache.c via genbki.pl. Reviewed-by: John Naylor <johncnaylorls@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/75ae5875-3abc-dafc-8aec-73247ed41cde@eisentraut.org
* Improve stability of recovery test 035_standby_logical_decodingMichael Paquier2024-01-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit tweaks a couple of things in 035_standby_logical_decoding to hopefully stabilize it: - Autovacuum is now disabled, as it could hold a global xmin with a transaction. - Conflicts are generated with command sequences that removed rows (on catalogs, shared or non-shared, or just plain relations) followed by a VACUUM. This was unstable because these did not check that the horizon moved between the SQL commands and the VACUUM. The logic is refactored as follows, to ensure that VACUUM removes dead rows before testing for slot invalidation on a standby (idea suggested by Andres Freund): -- Grab the current horizon. -- Launch SQL commands removing rows. -- Check that the snapshot horizon has been updated. -- Launch VACUUM on the relation whose rows have been removed by the first step. Note that there are still some issues because of standby snapshot WAL records generated by the bgwriter, but this makes the test much more stable. Per reports from buildfarm members dikkop and skink, with analysis and tests from Alexander Lakhin. While on it, fix a couple of incorrect comments. Author: Bertrand Drouvot Reviewed-by: Alexander Lakhin, Michael Paquier Discussion: https://postgr.es/m/OSZPR01MB6310ED3CEDB531BCEDBC6AF2FD479@OSZPR01MB6310.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/bf67e076-b163-9ba3-4ade-b9fc51a3a8f6@gmail.com Backpatch-through: 16
* Add better handling of redundant IS [NOT] NULL qualsDavid Rowley2024-01-23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Until now PostgreSQL has not been very smart about optimizing away IS NOT NULL base quals on columns defined as NOT NULL. The evaluation of these needless quals adds overhead. Ordinarily, anyone who came complaining about that would likely just have been told to not include the qual in their query if it's not required. However, a recent bug report indicates this might not always be possible. Bug 17540 highlighted that when we optimize Min/Max aggregates the IS NOT NULL qual that the planner adds to make the rewritten plan ignore NULLs can cause issues with poor index choice. That particular case demonstrated that other quals, especially ones where no statistics are available to allow the planner a chance at estimating an approximate selectivity for can result in poor index choice due to cheap startup paths being prefered with LIMIT 1. Here we take generic approach to fixing this by having the planner check for NOT NULL columns and just have the planner remove these quals (when they're not needed) for all queries, not just when optimizing Min/Max aggregates. Additionally, here we also detect IS NULL quals on a NOT NULL column and transform that into a gating qual so that we don't have to perform the scan at all. This also works for join relations when the Var is not nullable by any outer join. This also helps with the self-join removal work as it must replace strict join quals with IS NOT NULL quals to ensure equivalence with the original query. Author: David Rowley, Richard Guo, Andy Fan Reviewed-by: Richard Guo, David Rowley Discussion: https://postgr.es/m/CAApHDvqg6XZDhYRPz0zgOcevSMo0d3vxA9DvHrZtKfqO30WTnw@mail.gmail.com Discussion: https://postgr.es/m/17540-7aa1855ad5ec18b4%40postgresql.org
* Fix possible NULL pointer dereference in GetNamedDSMSegment().Nathan Bossart2024-01-22
| | | | | | | | | | | | GetNamedDSMSegment() doesn't check whether dsm_attach() returns NULL, which creates the possibility of a NULL pointer dereference soon after. To fix, emit an ERROR if dsm_attach() returns NULL. This shouldn't happen, but it would be nice to avoid a segfault if it does. In passing, tidy up the surrounding code. Reported-by: Tom Lane Reviewed-by: Michael Paquier, Bharath Rupireddy Discussion: https://postgr.es/m/3348869.1705854106%40sss.pgh.pa.us
* Fix ERROR message in injection_point.cMichael Paquier2024-01-23
| | | | | | | | | | | | This commit fixes an error message that failed to show the correct function and library names when a function cannot be loaded. While on it, adjust the call to load_external_function() so as this ERROR can be reached, by making load_external_function() return NULL rather than fail if a function cannot be found for a given injection point. Thinkos in d86d20f0ba79.
* Fix two memcpy() bugs in the new injection point codeHeikki Linnakangas2024-01-22
| | | | | | | | | | | | | | 1. The memcpy()s in InjectionPointAttach() would copy garbage from beyond the end of input string to the buffer in shared memory. You won't usually notice, but if there is not enough valid mapped memory beyond the end of the string, the read of unmapped memory will segfault. This was flagged by the Cirrus CI build with address sanitizer enabled. 2. The memcpy() in injection_point_cache_add() failed to copy the NULL terminator. Discussion: https://www.postgresql.org/message-id/0615a424-b726-4157-afa7-4245629f9512%40iki.fi
* Abort pgbench if script end is reached with an open pipelineAlvaro Herrera2024-01-22
| | | | | | | | | | | | | | | | When a pipeline is opened with \startpipeline and not closed, pgbench will either error on the next transaction with a "already in pipeline mode" error or successfully end if this was the last transaction -- despite not sending anything that was piped in the pipeline. Make it an error to reach end of script is reached while there's an open pipeline. Backpatch to 14, where pgbench got support for pipelines. Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com> Reported-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/Za4IObZkDjrO4TcS@paquier.xyz
* Test EXPLAIN (FORMAT JSON) ... XMLTABLEAlvaro Herrera2024-01-22
| | | | | | | | | Also, add an alias to the XMLTABLE expression in an existing test. This covers some code in explain.c that wasn't previously covered. I patched xml_2.out blindly :-( Discussion: https://postgr.es/m/202401181146.fuoeskfzriq7@alvherre.pgsql
* Re-disallow Memoize for parameterized nested loops with join filtersDavid Rowley2024-01-22
| | | | | | | | | | | | | | | | | This was previously fixed in 9e215378d but got broken again as a result of 2489d76c4. It seems that commit causes ppi_clauses to contain duplicate clauses and it's no longer safe to check the list_length of that list to determine if there are join conditions other than what's mentioned in ppi_clauses. Here we adjust the check to count the distinct rinfo_serial mentioned in ppi_clauses. We expect that extra->restrictlist won't have duplicate rinfo_serials. Reported-by: Amadeo Gallardo Author: Richard Guo Discussion: https://postgr.es/m/CADFREbW-BLJd7-a5J%2B5wjVumeFG1ByXiSOFzMtkmY_SDWckTxw%40mail.gmail.com Backpatch-through: 16, where 2489d76c4 was introduced.
* Fix some typosMichael Paquier2024-01-22
| | | | | Author: Yongtao Huang Discussion: https://postgr.es/m/CAOe1Go1F99o5JsphtXdDC5bxm7AzetU8q3AxLh4AAVGKu1AzEQ@mail.gmail.com
* Add test module injection_pointsMichael Paquier2024-01-22
| | | | | | | | | | | | This provides basic coverage for injection points within a single process, while providing some callbacks that can be used for other tests. There are plans to extend this module later with more advanced capabilities for tests. Author: Michael Paquier, with comment fixes from Ashutosh Bapat. Reviewed-by: Ashutosh Bapat, Nathan Bossart, Álvaro Herrera, Dilip Kumar, Amul Sul, Nazir Bilal Yavuz Discussion: https://postgr.es/m/ZTiV8tn_MIb_H2rE@paquier.xyz
* Add backend support for injection pointsMichael Paquier2024-01-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Injection points are a new facility that makes possible for developers to run custom code in pre-defined code paths. Its goal is to provide ways to design and run advanced tests, for cases like: - Race conditions, where processes need to do actions in a controlled ordered manner. - Forcing a state, like an ERROR, FATAL or even PANIC for OOM, to force recovery, etc. - Arbitrary sleeps. This implements some basics, and there are plans to extend it more in the future depending on what's required. Hence, this commit adds a set of routines in the backend that allows developers to attach, detach and run injection points: - A code path calling an injection point can be declared with the macro INJECTION_POINT(name). - InjectionPointAttach() and InjectionPointDetach() to respectively attach and detach a callback to/from an injection point. An injection point name is registered in a shmem hash table with a library name and a function name, which will be used to load the callback attached to an injection point when its code path is run. Injection point names are just strings, so as an injection point can be declared and run by out-of-core extensions and modules, with callbacks defined in external libraries. This facility is hidden behind a dedicated switch for ./configure and meson, disabled by default. Note that backends use a local cache to store callbacks already loaded, cleaning up their cache if a callback has found to be removed on a best-effort basis. This could be refined further but any tests but what we have here was fine with the tests I've written while implementing these backend APIs. Author: Michael Paquier, with doc suggestions from Ashutosh Bapat. Reviewed-by: Ashutosh Bapat, Nathan Bossart, Álvaro Herrera, Dilip Kumar, Amul Sul, Nazir Bilal Yavuz Discussion: https://postgr.es/m/ZTiV8tn_MIb_H2rE@paquier.xyz
* Fix table name collision in tests in 0452b461bcAlexander Korotkov2024-01-21
|
* Explore alternative orderings of group-by pathkeys during optimization.Alexander Korotkov2024-01-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When evaluating a query with a multi-column GROUP BY clause, we can minimize sort operations or avoid them if we synchronize the order of GROUP BY clauses with the ORDER BY sort clause or sort order, which comes from the underlying query tree. Grouping does not imply any ordering, so we can compare the keys in arbitrary order, and a Hash Agg leverages this. But for Group Agg, we simply compared keys in the order specified in the query. This commit explores alternative ordering of the keys, trying to find a cheaper one. The ordering of group keys may interact with other parts of the query, some of which may not be known while planning the grouping. For example, there may be an explicit ORDER BY clause or some other ordering-dependent operation higher up in the query, and using the same ordering may allow using either incremental sort or even eliminating the sort entirely. The patch always keeps the ordering specified in the query, assuming the user might have additional insights. This introduces a new GUC enable_group_by_reordering so that the optimization may be disabled if needed. Discussion: https://postgr.es/m/7c79e6a5-8597-74e8-0671-1c39d124c9d6%40sigaev.ru Author: Andrei Lepikhov, Teodor Sigaev Reviewed-by: Tomas Vondra, Claudio Freire, Gavin Flower, Dmitry Dolgov Reviewed-by: Robert Haas, Pavel Borisov, David Rowley, Zhihong Yu Reviewed-by: Tom Lane, Alexander Korotkov, Richard Guo, Alena Rybakina
* Generalize the common code of adding sort before processing of groupingAlexander Korotkov2024-01-21
| | | | | | | Extract the repetitive code pattern into a new function make_ordered_path(). Discussion: https://postgr.es/m/CAPpHfdtzaVa7S4onKy3YvttF2rrH5hQNHx9HtcSTLbpjx%2BMJ%2Bw%40mail.gmail.com Author: Andrei Lepikhov
* Add hint about not qualifying UPDATE...SET target with relation name.Tom Lane2024-01-20
| | | | | | | | | | | | | | | Target columns in UPDATE ... SET must not be qualified with the target table; we disallow this because it'd create ambiguity about which name is the column name in case of field-qualified names. However, newbies have been seen to expect that they could qualify a target name just like other names. The error message when they do is confusing: "column "foo" of relation "foo" does not exist". To improve matters, issue a HINT if the invalid name is qualified and matches the relation's alias. James Coleman (editorialized a bit by me) Discussion: https://postgr.es/m/CAAaqYe8S2Qa060UV-YF5GoSd5PkEhLV94x-fEi3=TOtpaXCV+w@mail.gmail.com
* Add planner support functions for range operators <@ and @>.Tom Lane2024-01-20
| | | | | | | | | | | | | | | | These support functions will transform expressions with constant range values into direct comparisons on the range bound values, which are frequently better-optimizable. The transformation is skipped however if it would require double evaluation of a volatile or expensive element expression. Along the way, add the range opfamily OID to range typcache entries, since load_rangetype_info has to compute that anyway and it seems silly to duplicate the work later. Kim Johan Andersson and Jian He, reviewed by Laurenz Albe Discussion: https://postgr.es/m/94f64d1f-b8c0-b0c5-98bc-0793a34e0851@kimmet.dk
* Introduce the dynamic shared memory registry.Nathan Bossart2024-01-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | Presently, the most straightforward way for a shared library to use shared memory is to request it at server startup via a shmem_request_hook, which requires specifying the library in shared_preload_libraries. Alternatively, the library can create a dynamic shared memory (DSM) segment, but absent a shared location to store the segment's handle, other backends cannot use it. This commit introduces a registry for DSM segments so that these other backends can look up existing segments with a library-specified string. This allows libraries to easily use shared memory without needing to request it at server startup. The registry is accessed via the new GetNamedDSMSegment() function. This function handles allocating the segment and initializing it via a provided callback. If another backend already created and initialized the segment, it simply attaches the segment. GetNamedDSMSegment() locks the registry appropriately to ensure that only one backend initializes the segment and that all other backends just attach it. The registry itself is comprised of a dshash table that stores the DSM segment handles keyed by a library-specified string. Reviewed-by: Michael Paquier, Andrei Lepikhov, Nikita Malakhov, Robert Haas, Bharath Rupireddy, Zhang Mingli, Amul Sul Discussion: https://postgr.es/m/20231205034647.GA2705267%40nathanxps13
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-15 12:53:29 +0000