aboutsummaryrefslogtreecommitdiff
path: root/src/backend
Commit message (Collapse)AuthorAge
* Fix freeing old values in index_store_float8_orderby_distances()Alexander Korotkov2019-09-20
| | | | | | | | | | | | | | | 6cae9d2c10 has added an error in freeing old values in index_store_float8_orderby_distances() function. It looks for old value in scan->xs_orderbynulls[i] after setting a new value there. This commit fixes that. Also it removes short-circuit in handling distances == NULL situation. Now distances == NULL will be treated the same way as array with all null distances. That is, previous values will be freed if any. Reported-by: Tom Lane, Nikita Glukhov Discussion: https://postgr.es/m/CAPpHfdu2wcoAVAm3Ek66rP%3Duo_C-D84%2B%2Buf1VEcbyi_caBXWCA%40mail.gmail.com Discussion: https://postgr.es/m/426580d3-a668-b9d1-7b8e-f74d1a6524e0%40postgrespro.ru Backpatch-through: 12
* Improve handling of NULLs in KNN-GiST and KNN-SP-GiSTAlexander Korotkov2019-09-19
| | | | | | | | | | | | | | | | | | | | | This commit improves subject in two ways: * It removes ugliness of 02f90879e7, which stores distance values and null flags in two separate arrays after GISTSearchItem struct. Instead we pack both distance value and null flag in IndexOrderByDistance struct. Alignment overhead should be negligible, because we typically deal with at most few "col op const" expressions in ORDER BY clause. * It fixes handling of "col op NULL" expression in KNN-SP-GiST. Now, these expression are not passed to support functions, which can't deal with them. Instead, NULL result is implicitly assumed. It future we may decide to teach support functions to deal with NULL arguments, but current solution is bugfix suitable for backpatch. Reported-by: Nikita Glukhov Discussion: https://postgr.es/m/826f57ee-afc7-8977-c44c-6111d18b02ec%40postgrespro.ru Author: Nikita Glukhov Reviewed-by: Alexander Korotkov Backpatch-through: 9.4
* GSSAPI error message improvementsPeter Eisentraut2019-09-19
| | | | | | | | | | Make the error messages around GSSAPI encryption a bit clearer. Tweak some messages to avoid plural problems. Also make a code change for clarity. Using "conf" for "confidential" is quite confusing. Using "conf_state" is perhaps not much better but that's what the GSSAPI documentation uses, so there is at least some hope of understanding it.
* Remove unused smgrdounlinkfork() function.Fujii Masao2019-09-18
| | | | | | | | | smgrdounlinkfork() became dead code as the result of commit ece01aae47, but it was left in place just in case we want it someday. However no users have appeared in 7 years, so it's time to remove this unused function. Author: Kirk Jamison Discussion: https://www.postgresql.org/message-id/D09B13F772D2274BB348A310EE3027C64E2067@g01jpexmbkw24
* Add some const decorations to array constantsPeter Eisentraut2019-09-17
| | | | | Author: Mark G <markg735@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/CAEeOP_YFVeFjq4zDZLDQbLSRFxBiTpwBQHxCNgGd%2Bp5VztTXyQ%40mail.gmail.com
* Fix bogus handling of XQuery regex option flags.Tom Lane2019-09-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The SQL spec defers to XQuery to define what the option flags are for LIKE_REGEX patterns. XQuery says that: * 's' allows the dot character to match newlines, which by default it will not; * 'm' allows ^ and $ to match at newlines, not only at the start/end of the whole string. Thus, these are *not* inverses as they are for the similarly-named POSIX options, and neither one corresponds to the POSIX 'n' option. Fortunately, Spencer's library does expose these two behaviors as separately twiddlable flags, so we just have to fix the mapping from JSP flag bits to REG flag bits. I also chose to rename the symbol for 's' to DOTALL, to make it clearer that it's not the inverse of MLINE. Also, XQuery says that if the 'q' flag "is used together with the m, s, or x flag, that flag has no effect". I read this as saying that 'q' overrides the other flags; whoever wrote our code seems to have read it backwards. Lastly, while XQuery's 'x' flag is related to what Spencer's code does for REG_EXPANDED, it's not the same or a subset. It seems best to treat XQuery's 'x' as unimplemented for now. Maybe later we can expand our regex code to offer 'x'-style parsing as a separate option. While at it, refactor the jsonpath code so that (a) there's only one copy of the flag transformation logic not two, and (b) the processing of flags is independent of the order in which the flags are written. We need some documentation updates to go with this, but I'll tackle that separately. Back-patch to v12 where this code originated. Discussion: https://postgr.es/m/CAPpHfdvDci4iqNF9fhRkTqhe-5_8HmzeLt56drH%2B_Rv2rNRqfg@mail.gmail.com Reference: https://www.w3.org/TR/2017/REC-xpath-functions-31-20170321/#flags
* Remove mingwcompat.cPeter Eisentraut2019-09-17
| | | | | | | We believe that the issues that this was working around have been fixed in MinGW more than 5 years ago, so this isn't necessary anymore. Discussion: https://www.postgresql.org/message-id/flat/20190719050830.GK1859%40paquier.xyz
* Support for SSSSS datetime format patternAlexander Korotkov2019-09-16
| | | | | | | | | | | | | | SQL Standard 2016 defines SSSSS format pattern for seconds past midnight in jsonpath .datetime() method and CAST (... FORMAT ...) SQL clause. In our datetime parsing engine we currently support it with SSSS name. This commit adds SSSSS as an alias for SSSS. Alias is added in favor of upcoming jsonpath .datetime() method. But it's also supported in to_date()/ to_timestamp() as positive side effect. Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com Author: Nikita Glukhov, Alexander Korotkov Reviewed-by: Anastasia Lubennikova, Peter Eisentraut
* Support for FF1-FF6 datetime format patternsAlexander Korotkov2019-09-16
| | | | | | | | | | | | | | | | | | | | SQL Standard 2016 defines FF1-FF9 format patters for fractions of seconds in jsonpath .datetime() method and CAST (... FORMAT ...) SQL clause. Parsing engine of upcoming .datetime() method will be shared with to_date()/ to_timestamp(). This patch implements FF1-FF6 format patterns for upcoming jsonpath .datetime() method. to_date()/to_timestamp() functions will also get support of this format patterns as positive side effect. FF7-FF9 are not supported due to lack of precision in our internal timestamp representation. Extracted from original patch by Nikita Glukhov, Teodor Sigaev, Oleg Bartunov. Heavily revised by me. Discussion: https://postgr.es/m/fcc6fc6a-b497-f39a-923d-aa34d0c588e8%402ndQuadrant.com Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com Author: Nikita Glukhov, Teodor Sigaev, Oleg Bartunov, Alexander Korotkov Reviewed-by: Anastasia Lubennikova, Peter Eisentraut
* Fix bogus sizeof calculations.Tom Lane2019-09-15
| | | | | Noted by Coverity. Typo in 27cc7cd2b, so back-patch to v12 as that was.
* Make tuplesort_set_bound() assertions more comprehensible, hopefully.Tom Lane2019-09-13
| | | | | | | | Add the comments that I griped were missing. Also re-order tests so that parallelism-related tests aren't randomly separated from each other. Discussion: https://postgr.es/m/CAAaqYe9GD__4Crm=ddz+-XXcNhfY_V5gFYdLdmkFNq=2VHO56Q@mail.gmail.com
* logical decoding: process ASSIGNMENT during snapshot buildAlvaro Herrera2019-09-13
| | | | | | | | | | | | | Most WAL records are ignored in early SnapBuild snapshot build phases. But it's critical to process some of them, so that later messages have the correct transaction state after the snapshot is completely built; in particular, XLOG_XACT_ASSIGNMENT messages are critical in order for sub-transactions to be correctly assigned to their parent transactions, or at least one assert misbehaves, as reported by Ildar Musin. Diagnosed-by: Masahiko Sawada Author: Masahiko Sawada Discussion: https://postgr.es/m/CAONYFtOv+Er1p3WAuwUsy1zsCFrSYvpHLhapC_fMD-zNaRWxYg@mail.gmail.com
* Fix progress reporting of CLUSTER / VACUUM FULLAlvaro Herrera2019-09-13
| | | | | | | | | | | | | | | | | | The progress state was being clobbered once the first index completed being rebuilt, causing the final phases of the operation not show anything in the progress view. This was inadvertently broken in 03f9e5cba0ee, which added progress tracking for REINDEX. (The reason this bugfix is this small is that I had already noticed this problem when writing monitoring for CREATE INDEX, and had already worked around it, as can be seen in discussion starting at https://postgr.es/m/20190329150218.GA25010@alvherre.pgsql Fixing the problem is just a matter of fixing one place touched by the REINDEX monitoring.) Reported by: Álvaro Herrera Author: Álvaro Herrera Discussion: https://postgr.es/m/20190801184333.GA21369@alvherre.pgsql
* Fix nbtree page split rmgr desc routine.Peter Geoghegan2019-09-12
| | | | | | | | | | | | Include newitemoff in rmgr desc output for nbtree page split records. In passing, correct an obsolete comment that claimed that newitemoff is only logged for _L variant nbtree page split WAL records. Both issues were oversights in commit 2c03216d831, which revamped the WAL format. Author: Peter Geoghegan Backpatch: 9.5-, where the WAL format was revamped.
* Remove redundant _bt_truncate() comment paragraph.Peter Geoghegan2019-09-12
|
* Merge two assertions to make comment clearerAlvaro Herrera2019-09-12
| | | | | | Authored by Tom Lane, after a gripe from James Coleman. Discussion: https://postgr.es/m/CAAaqYe9GD__4Crm=ddz+-XXcNhfY_V5gFYdLdmkFNq=2VHO56Q@mail.gmail.com
* Rearrange postmaster's startup sequence for better syslogger results.Tom Lane2019-09-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a second try at what commit 57431a911 tried to do, namely, launch the syslogger before we open postmaster sockets so that our messages about the sockets end up in the syslogger files. That commit fell foul of a bunch of subtle issues caused by trying to launch a postmaster child process before creating shared memory. Rather than messing with that interaction, let's postpone opening the sockets till after we launch the syslogger. This would not have been terribly safe before commit 7de19fbc0, because we relied on socket opening to detect whether any competing postmasters were using the same port number. But now that we choose IPC keys without regard to the port number, there's no interaction to worry about. Also delay creation of the external PID file (if requested) till after the sockets are open, since external code could plausibly be relying on that ordering of events. And postpone most of the work of RemovePgTempFiles() so that that potentially-slow processing still happens after we make the external PID file. We have to be a bit careful about that last though: as noted in the discussion subsequent to bug #15804, EXEC_BACKEND builds still have to clear the parameter-file temp dir before launching the syslogger. Patch by me; thanks to Michael Paquier for review/testing. Discussion: https://postgr.es/m/15804-3721117bf40fb654@postgresql.org
* Allow setting statistics target for extended statisticsTomas Vondra2019-09-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When building statistics, we need to decide how many rows to sample and how accurate the resulting statistics should be. Until now, it was not possible to explicitly define statistics target for extended statistics objects, the value was always computed from the per-attribute targets with a fallback to the system-wide default statistics target. That's a bit inconvenient, as it ties together the statistics target set for per-column and extended statistics. In some cases it may be useful to require larger sample / higher accuracy for extended statics (or the other way around), but with this approach that's not possible. So this commit introduces a new command, allowing to specify statistics target for individual extended statistics objects, overriding the value derived from per-attribute targets (and the system default). ALTER STATISTICS stat_name SET STATISTICS target_value; When determining statistics target for an extended statistics object we first look at this explicitly set value. When this value is -1, we fall back to the old formula, looking at the per-attribute targets first and then the system default. This means the behavior is backwards compatible with older PostgreSQL releases. Author: Tomas Vondra Discussion: https://postgr.es/m/20190618213357.vli3i23vpkset2xd@development Reviewed-by: Kirk Jamison, Dean Rasheed
* Reduce overhead of scanning the backend[] array in LISTEN/NOTIFY.Tom Lane2019-09-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | Up to now, async.c scanned its whole array of per-backend state whenever it needed to find listening backends. That's expensive if MaxBackends is large, so extend the data structure with list links that thread the active entries together. A downside of this change is that asyncQueueUnregister (unregister a listening backend at backend exit) now requires exclusive not shared lock, and it can take awhile if there are many other listening backends. We could improve the latter issue by using a doubly- not singly-linked list, but it's probably not worth the storage space; typical usage patterns for LISTEN/NOTIFY have fairly long-lived listeners. In return for that, Exec_ListenPreCommit (initially register a listening backend), SignalBackends, and asyncQueueAdvanceTail get significantly faster when MaxBackends is much larger than the number of listening backends. If most of the potential backend slots are listening, we don't win, but that's a case where the actual interprocess-signal overhead is going to swamp these considerations anyway. Martijn van Oosterhout, hacked a bit more by me Discussion: https://postgr.es/m/CADWG95vtRBFDdrx1JdT1_9nhOFw48KaeTev6F_LtDQAFVpSPhA@mail.gmail.com
* Add _bt_binsrch() scantid assertion to nbtree.Peter Geoghegan2019-09-09
| | | | | | | | | | Assert that _bt_binsrch() binary searches with scantid set in insertion scankey cannot be performed on leaf pages. Leaf-level binary searches where scantid is set must use _bt_binsrch_insert() instead. _bt_binsrch_insert() is likely to have additional responsibilities in the future, such as searching within GIN-style posting lists using scantid. It seems like a good idea to tighten things up now.
* Reorder EPQ work, to fix rowmark related bugs and improve efficiency.Andres Freund2019-09-09
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In ad0bda5d24ea I changed the EvalPlanQual machinery to store substitution tuples in slot, instead of using plain HeapTuples. The main motivation for that was that using HeapTuples will be inefficient for future tableams. But it turns out that that conversion was buggy for non-locking rowmarks - the wrong tuple descriptor was used to create the slot. As a secondary issue 5db6df0c0 changed ExecLockRows() to begin EPQ earlier, to allow to fetch the locked rows directly into the EPQ slots, instead of having to copy tuples around. Unfortunately, as Tom complained, that forces some expensive initialization to happen earlier. As a third issue, the test coverage for EPQ was clearly insufficient. Fixing the first issue is unfortunately not trivial: Non-locked row marks were fetched at the start of EPQ, and we don't have the type information for the rowmarks available at that point. While we could change that, it's not easy. It might be worthwhile to change that at some point, but to fix this bug, it seems better to delay fetching non-locking rowmarks when they're actually needed, rather than eagerly. They're referenced at most once, and in cases where EPQ fails, might never be referenced. Fetching them when needed also increases locality a bit. To be able to fetch rowmarks during execution, rather than initialization, we need to be able to access the active EPQState, as that contains necessary data. To do so move EPQ related data from EState to EPQState, and, only for EStates creates as part of EPQ, reference the associated EPQState from EState. To fix the second issue, change EPQ initialization to allow use of EvalPlanQualSlot() to be used before EvalPlanQualBegin() (but obviously still requiring EvalPlanQualInit() to have been done). As these changes made struct EState harder to understand, e.g. by adding multiple EStates, significantly reorder the members, and add a lot more comments. Also add a few more EPQ tests, including one that fails for the first issue above. More is needed. Reported-By: yi huang Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/CAHU7rYZo_C4ULsAx_LAj8az9zqgrD8WDd4hTegDTMM1LMqrBsg@mail.gmail.com https://postgr.es/m/24530.1562686693@sss.pgh.pa.us Backpatch: 12-, where the EPQ changes were introduced
* Fix handling of non-key columns get_index_column_opclass()Alexander Korotkov2019-09-09
| | | | | | | | | | | f2e40380 introduces support of non-key attributes in GiST indexes. Then if get_index_column_opclass() is asked by gistproperty() to get an opclass of non-key column, it returns garbage past oidvector value. This commit fixes that by making get_index_column_opclass() return InvalidOid in this case. Discussion: https://postgr.es/m/20190902231948.GA5343%40alvherre.pgsql Author: Nikita Glukhov, Alexander Korotkov Backpatch-through: 12
* Fix RelationIdGetRelation calls that weren't bothering with error checks.Tom Lane2019-09-08
| | | | | | | | | | | | Some of these are quite old, but that doesn't make them not bugs. We'd rather report a failure via elog than SIGSEGV. While at it, uniformly spell the error check as !RelationIsValid(rel) rather than a bare rel == NULL test. The machine code is the same but it seems better to be consistent. Coverity complained about this today, not sure why, because the mistake is in fact old.
* Fix handling of NULL distances in KNN-GiSTAlexander Korotkov2019-09-08
| | | | | | | | | | | | | | In order to implement NULL LAST semantic GiST previously assumed distance to the NULL value to be Inf. However, our distance functions can return Inf and NaN for non-null values. In such cases, NULL LAST semantic appears to be broken. This commit fixes that by introducing separate array of null flags for distances. Backpatch to all supported versions. Discussion: https://postgr.es/m/CAPpHfdsNvNdA0DBS%2BwMpFrgwT6C3-q50sFVGLSiuWnV3FqOJuQ%40mail.gmail.com Author: Alexander Korotkov Backpatch-through: 9.4
* Fix handling Inf and Nan values in GiST pairing heap comparatorAlexander Korotkov2019-09-08
| | | | | | | | | | | | | | | | Previously plain float comparison was used in GiST pairing heap. Such comparison doesn't provide proper ordering for value sets containing Inf and Nan values. This commit fixes that by usage of float8_cmp_internal(). Note, there is remaining problem with NULL distances, which are represented as Inf in pairing heap. It would be fixes in subsequent commit. Backpatch to all supported versions. Reported-by: Andrey Borodin Discussion: https://postgr.es/m/CAPpHfdsNvNdA0DBS%2BwMpFrgwT6C3-q50sFVGLSiuWnV3FqOJuQ%40mail.gmail.com Author: Alexander Korotkov Reviewed-by: Heikki Linnakangas Backpatch-through: 9.4
* Fix behavior of AND CHAIN outside of explicit transaction blocksPeter Eisentraut2019-09-08
| | | | | | | | | | | | | When using COMMIT AND CHAIN or ROLLBACK AND CHAIN not in an explicit transaction block, the previous implementation would leave a transaction block active in the ROLLBACK case but not the COMMIT case. To fix for now, error out when using these commands not in an explicit transaction block. This restriction could be lifted if a sensible definition and implementation is found. Bug: #15977 Author: fn ln <emuser20140816@gmail.com> Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
* Avoid using INFO elevel for what are fundamentally debug messages.Tom Lane2019-09-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 6f6b99d13 stuck an INFO message into the fast path for checking partition constraints, for no very good reason except that it made it easy for the regression tests to verify that that path was taken. Assorted later patches did likewise, increasing the unsuppressable-chatter level from ALTER TABLE even more. This isn't good for the user experience, so let's drop these messages down to DEBUG1 where they belong. So as not to have a loss of test coverage, create a TAP test that runs the relevant queries with client_min_messages = DEBUG1 and greps for the expected messages. This testing method is a bit brute-force --- in particular, it duplicates the execution of a fair amount of the core create_table and alter_table tests. We experimented with other solutions, but running any significant amount of standard testing with client_min_messages = DEBUG1 seems to have a lot of output-stability pitfalls, cf commits bbb96c370 and 5655565c0. Possibly at some point we'll look into whether we can reduce the amount of test duplication. Backpatch into v12, because some of these messages are new in v12 and we don't really want to ship it that way. Sergei Kornilov Discussion: https://postgr.es/m/81911511895540@web58j.yandex.ru Discussion: https://postgr.es/m/4859321552643736@myt5-02b80404fd9e.qloud-c.yandex.net
* Fix issues around strictness of SIMILAR TO.Tom Lane2019-09-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As a result of some long-ago quick hacks, the SIMILAR TO operator and the corresponding flavor of substring() interpreted "ESCAPE NULL" as selecting the default escape character '\'. This is both surprising and not per spec: the standard is clear that these functions should return NULL for NULL input. Additionally, because of inconsistency of the strictness markings of 3-argument substring() and similar_escape(), the planner could not inline the SQL definition of substring(), resulting in a substantial performance penalty compared to the underlying POSIX substring() function. The simplest fix for this would be to change the strictness marking of similar_escape(), but if we do that we risk breaking existing views that depend on that function. Hence, leave similar_escape() as-is as a compatibility function, and instead invent a new function similar_to_escape() that comes in two strict variants. There are a couple of other behaviors in this area that are also not per spec, but they are documented and seem generally at least as sane as the spec's definition, so leave them alone. But improve the documentation to describe them fully. Patch by me; thanks to Álvaro Herrera and Andrew Gierth for review and discussion. Discussion: https://postgr.es/m/14047.1557708214@sss.pgh.pa.us
* Create an API for inserting and deleting rows in TOAST tables.Robert Haas2019-09-06
| | | | | | | | | | | | | | | This moves much of the non-heap-specific logic from toast_delete and toast_insert_or_update into a helper functions accessible via a new header, toast_helper.h. Using the functions in this module, a table AM can implement creation and deletion of TOAST table rows with much less code duplication than was possible heretofore. Some table AMs won't want to use the TOAST logic at all, but for those that do this will make that easier. Patch by me, reviewed and tested by Prabhat Sabu, Thomas Munro, Andres Freund, and Álvaro Herrera. Discussion: http://postgr.es/m/CA+TgmoZv-=2iWM4jcw5ZhJeL18HF96+W1yJeYrnGMYdkFFnEpQ@mail.gmail.com
* When performing a base backup, check for read errors.Robert Haas2019-09-06
| | | | | | | | | | | | | | | | The old code didn't differentiate between a read error and a concurrent truncation. fread reports both of these by returning 0; you have to use feof() or ferror() to distinguish between them, which this code did not do. It might be a better idea to use read() rather than fread() here, so that we can display a less-generic error message, but I'm not sure that would qualify as a back-patchable bug fix, so just do this much for now. Jeevan Chalke, reviewed by Jeevan Ladhe and by me. Discussion: http://postgr.es/m/CA+TgmobG4ywMzL5oQq2a8YKp8x2p3p1LOMMcGqpS7aekT9+ETA@mail.gmail.com
* Make pg_promote() detect postmaster death while waiting for promotion to end.Fujii Masao2019-09-06
| | | | | | | | | | | | | | | | Previously even if postmaster died and WaitLatch() woke up with that event while pg_promote() was waiting for the standby promotion to finish, pg_promote() did nothing special and kept waiting until timeout occurred. This could cause a busy loop. This patch make pg_promote() return false immediately when postmaster dies, to avoid such a busy loop. Back-patch to v12 where pg_promote() was added. Author: Fujii Masao Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/CAHGQGwEs9ROgSp+QF+YdDU+xP8W=CY1k-_Ov-d_Z3JY+to3eXA@mail.gmail.com
* Use data directory inode number, not port, to select SysV resource keys.Tom Lane2019-09-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This approach provides a much tighter binding between a data directory and the associated SysV shared memory block (and SysV or named-POSIX semaphores, if we're using those). Key collisions are still possible, but only between data directories stored on different filesystems, so the situation should be negligible in practice. More importantly, restarting the postmaster with a different port number no longer risks failing to identify a relevant shared memory block, even when postmaster.pid has been removed. A standalone backend is likewise much more certain to detect conflicting leftover backends. (In the longer term, we might now think about deprecating the port as a cluster-wide value, so that one postmaster could support sockets with varying port numbers. But that's for another day.) The hazards fixed here apply only on Unix systems; our Windows code paths already use identifiers derived from the data directory path name rather than the port. src/test/recovery/t/017_shm.pl, which intends to test key-collision cases, has been substantially rewritten since it can no longer use two postmasters with identical port numbers to trigger the case. Instead, use Perl's IPC::SharedMem module to create a conflicting shmem segment directly. The test script will be skipped if that module is not available. (This means that some older buildfarm members won't run it, but I don't think that that results in any meaningful coverage loss.) Patch by me; thanks to Noah Misch and Peter Eisentraut for discussion and review. Discussion: https://postgr.es/m/16908.1557521200@sss.pgh.pa.us
* Split tuptoaster.c into three separate files.Robert Haas2019-09-05
| | | | | | | | | | | | | | | | | | | | | | | | detoast.c/h contain functions required to detoast a datum, partially or completely, plus a few other utility functions for examining the size of toasted datums. toast_internals.c/h contain functions that are used internally to the TOAST subsystem but which (mostly) do not need to be accessed from outside. heaptoast.c/h contains code that is intrinsically specific to the heap AM, either because it operates on HeapTuples or is based on the layout of a heap page. detoast.c and toast_internals.c are placed in src/backend/access/common rather than src/backend/access/heap. At present, both files still have dependencies on the heap, but that will be improved in a future commit. Patch by me, reviewed and tested by Prabhat Sabu, Thomas Munro, Andres Freund, and Álvaro Herrera. Discussion: http://postgr.es/m/CA+TgmoZv-=2iWM4jcw5ZhJeL18HF96+W1yJeYrnGMYdkFFnEpQ@mail.gmail.com
* Use explicit_bzeroPeter Eisentraut2019-09-05
| | | | | | | | | | | | | | Use the explicit_bzero() function in places where it is important that security information such as passwords is cleared from memory. There might be other places where it could be useful; this is just an initial collection. For platforms that don't have explicit_bzero(), provide various fallback implementations. (explicit_bzero() itself isn't standard, but as Linux/glibc, FreeBSD, and OpenBSD have it, it's the most common spelling, so it makes sense to make that the invocation point.) Discussion: https://www.postgresql.org/message-id/flat/42d26bde-5d5b-c90d-87ae-6cab875f73be%402ndquadrant.com
* Fix thinko when ending progress report for a backendMichael Paquier2019-09-04
| | | | | | | | | | | | | | The logic ending progress reporting for a backend entry introduced by b6fb647 causes callers of pgstat_progress_end_command() to do some extra work when track_activities is enabled as the process fields are reset in the backend entry even if no command were started for reporting. This resets the fields only if a command is registered for progress reporting, and only if track_activities is enabled. Author: Masahiho Sawada Discussion: https://postgr.es/m/CAD21AoCry_vJ0E-m5oxJXGL3pnos-xYGCzF95rK5Bbi3Uf-rpA@mail.gmail.com Backpatch-through: 9.6
* Make XLogReaderInvalReadState staticAlvaro Herrera2019-09-03
| | | | | | | | | | | | | This function is only used by xlogreader.c itself, so there's no need to export it. It was introduced by commit 3b02ea4f0780 with the apparent intention that it could be used externally, but I couldn't find any external code calling it. I (Álvaro) couldn't resist the urge to sort nearby function prototypes properly while at it. Author: Antonin Houska Discussion: https://postgr.es/m/14984.1554998742@spoje.net
* Remove 'msg' parameter from convert_tuples_by_nameAlvaro Herrera2019-09-03
| | | | | | | | | | The message was included as a parameter when this function was added in dcb2bda9b704, but I don't think it has ever served any useful purpose. Let's stop spreading it pointlessly. Reviewed by Amit Langote and Peter Eisentraut. Discussion: https://postgr.es/m/20190806224728.GA17233@alvherre.pgsql
* Better error messages for short reads/writes in SLRUPeter Eisentraut2019-09-03
| | | | | | | | | | | | This avoids getting a Could not read from file ...: Success. for a short read or write (since errno is not set in that case). Instead, report a more specific error messages. Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/5de61b6b-8be9-7771-0048-860328efe027%402ndquadrant.com
* Fix memory leak with lower, upper and initcap with ICU-provided collationsMichael Paquier2019-09-03
| | | | | | | | | | | | | | The leak happens in str_tolower, str_toupper and str_initcap, which are used in several places including their equivalent SQL-level functions, and can only be triggered when using an ICU-provided collation when converting the input string. b615920 fixed a similar leak. Backpatch down 10 where ICU collations have been introduced. Author: Konstantin Knizhnik Discussion: https://postgr.es/m/94c0ad0a-cbc2-e4a3-7829-2bdeaf9146db@postgrespro.ru Backpatch-through: 10
* Avoid touching replica identity index in ExtractReplicaIdentity().Tom Lane2019-09-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In what seems like a fit of misplaced optimization, ExtractReplicaIdentity() accessed the relation's replica-identity index without taking any lock on it. Usually, the surrounding query already holds some lock so this is safe enough ... but in the case of a previously-planned delete, there might be no existing lock. Given a suitable test case, this is exposed in v12 and HEAD by an assertion added by commit b04aeb0a0. The whole thing's rather poorly thought out anyway; rather than looking directly at the index, we should use the index-attributes bitmap that's held by the parent table's relcache entry, as the caller functions do. This is more consistent and likely a bit faster, since it avoids a cache lookup. Hence, change to doing it that way. While at it, rather than blithely assuming that the identity columns are non-null (with catastrophic results if that's wrong), add assertion checks that they aren't null. Possibly those should be actual test-and-elog, but I'll leave it like this for now. In principle, this is a bug that's been there since this code was introduced (in 9.4). In practice, the risk seems quite low, since we do have a lock on the index's parent table, so concurrent changes to the index's catalog entries seem unlikely. Given the precedent that commit 9c703c169 wasn't back-patched, I won't risk back-patching this further than v12. Per report from Hadi Moshayedi. Discussion: https://postgr.es/m/CAK=1=Wrek44Ese1V7LjKiQS-Nd-5LgLi_5_CskGbpggKEf3tKQ@mail.gmail.com
* Fix overflow check and comment in GIN posting list encoding.Heikki Linnakangas2019-08-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The comment did not match what the code actually did for integers with the 43rd bit set. You get an integer like that, if you have a posting list with two adjacent TIDs that are more than 2^31 blocks apart. According to the comment, we would store that in 6 bytes, with no continuation bit on the 6th byte, but in reality, the code encodes it using 7 bytes, with a continuation bit on the 6th byte as normal. The decoding routine also handled these 7-byte integers correctly, except for an overflow check that assumed that one integer needs at most 6 bytes. Fix the overflow check, and fix the comment to match what the code actually does. Also fix the comment that claimed that there are 17 unused bits in the 64-bit representation of an item pointer. In reality, there are 64-32-11=21. Fitting any item pointer into max 6 bytes was an important property when this was written, because in the old pre-9.4 format, item pointers were stored as plain arrays, with 6 bytes for every item pointer. The maximum of 6 bytes per integer in the new format guaranteed that we could convert any page from the old format to the new format after upgrade, so that the new format was never larger than the old format. But we hardly need to worry about that anymore, and running into that problem during upgrade, where an item pointer is expanded from 6 to 7 bytes such that the data doesn't fit on a page anymore, is implausible in practice anyway. Backpatch to all supported versions. This also includes a little test module to test these large distances between item pointers, without requiring a 16 TB table. It is not backpatched, I'm including it more for the benefit of future development of new posting list formats. Discussion: https://www.postgresql.org/message-id/33bfc20a-5c86-f50c-f5a5-58e9925d05ff%40iki.fi Reviewed-by: Masahiko Sawada, Alexander Korotkov
* Avoid catalog lookups in RelationAllowsEarlyPruning().Thomas Munro2019-08-28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RelationAllowsEarlyPruning() performed a catalog scan, but is used in two contexts where that was a bad idea: 1. In heap_page_prune_opt(), which runs very frequently in some large scans. This caused major performance problems in a field report that was easy to reproduce. 2. In TestForOldSnapshot(), which runs while we hold a buffer content lock. It's not clear if this was guaranteed to be free of buffer deadlock risk. The check was introduced in commit 2cc41acd8 and defended against a real problem: 9.6's hash indexes have no page LSN and so we can't allow early pruning (ie the snapshot-too-old feature). We can remove the check from all later releases though: hash indexes are now logged, and there is no way to create UNLOGGED indexes on regular logged tables. If a future release allows such a combination, it might need to put a similar check in place, but it'll need some more thought. Back-patch to 10. Author: Thomas Munro Reviewed-by: Tom Lane, who spotted the second problem Discussion: https://postgr.es/m/CA%2BhUKGKT8oTkp5jw_U4p0S-7UG9zsvtw_M47Y285bER6a2gD%2Bg%40mail.gmail.com Discussion: https://postgr.es/m/CAA4eK1%2BWy%2BN4eE5zPm765h68LrkWc3Biu_8rzzi%2BOYX4j%2BiHRw%40mail.gmail.com
* Remove obsolete nbtree page deletion comment.Peter Geoghegan2019-08-27
| | | | | | Commit efada2b8e92, which made the nbtree page deletion algorithm more robust, removed the concept of a half-dead internal page. Remove a comment about half dead parent pages that was overlooked.
* Reject empty names and recursion in config-file include directives.Tom Lane2019-08-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | An empty file name or subdirectory name leads join_path_components() to just produce the parent directory name, which leads to weird failures or recursive inclusions. Let's throw a specific error for that. It takes only slightly more code to detect all-blank names, so do so. Also, detect direct recursion, ie a file calling itself. As coded this will also detect recursion via "include_dir '.'", which is perhaps more likely than explicitly including the file itself. Detecting indirect recursion would require API changes for guc-file.l functions, which seems not worth it since extensions might call them. The nesting depth limit will catch such cases eventually, just not with such an on-point error message. In passing, adjust the example usages in postgresql.conf.sample to perhaps eliminate the problem at the source: there's no reason for the examples to suggest that an empty value is valid. Per a trouble report from Brent Bates. Back-patch to 9.5; the issue is old, but the code in 9.4 is enough different that the patch doesn't apply easily, and it doesn't seem worth the trouble to fix there. Ian Barwick and Tom Lane Discussion: https://postgr.es/m/8c8bcbca-3bd9-dc6e-8986-04a5abdef142@2ndquadrant.com
* Fix postmaster state machine to handle dead_end child crashes better.Tom Lane2019-08-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A report from Alvaro Herrera shows that if we're in PM_STARTUP state, and we spawn a dead_end child to reject some incoming connection request, and that child dies with an unexpected exit code, the postmaster does not respond well. We correctly send SIGQUIT to the startup process, but then: * if the startup process exits with nonzero exit code, as expected, we thought that that indicated a crash and aborted startup. * if the startup process exits with zero exit code, which is possible due to the inherent race condition, we'd advance to PM_RUN state which is fine --- but the code forgot that AbortStartTime would be nonzero in this situation. We'd either die on the Asserts saying that it was zero, or perhaps misbehave later on. (A quick look suggests that the only misbehavior might be busy-waiting due to DetermineSleepTime doing the wrong thing.) To fix the first point, adjust the state-machine logic to recognize that a nonzero exit code is expected after sending SIGQUIT, and have it transition to a state where we can restart the startup process. To fix the second point, change the Asserts to clear the variable rather than just claiming it should be clear already. Perhaps we could improve this further by not treating a crash of a dead_end child as a reason for panic'ing the database. However, since those child processes are connected to shared memory, that seems a bit risky. There are few good reasons for a dead_end child to report failure anyway (the cause of this in Alvaro's report is quite unclear). On balance, therefore, a minimal fix seems best. This is an oversight in commit 45811be94. While that was back-patched, I'm hesitant to back-patch this change. The lack of reasons for a dead_end child to fail suggests that the case should be very rare in the field, which squares with the lack of reports; so it seems like this might not be worth the risk of introducing new issues. In any case we can let it bake awhile in HEAD before considering a back-patch. Discussion: https://postgr.es/m/20190615160950.GA31378@alvherre.pgsql
* Don't rely on llvm::make_unique.Thomas Munro2019-08-25
| | | | | | | | | | | | Bleeding-edge LLVM has stopped supplying replacements for various C++14 library features, for people on older C++ versions. Since we're not ready to require C++14 yet, just use plain old new instead of make_unique. As revealed by buildfarm animal seawasp. Back-patch to 11. Reviewed-by: Andres Freund Discussion: https://postgr.es/m/CA%2BhUKGJWG7unNqmkxg7nC5o3o-0p2XP6co4r%3D9epqYMm8UY4Mw%40mail.gmail.com
* Explain subtlety in nbtree locking protocol.Peter Geoghegan2019-08-23
| | | | | | | The Postgres approach to coupling locks during an ascent of the tree is slightly different to the approach taken by Lehman and Yao. Add a new paragraph to the "Differences to the Lehman & Yao algorithm" section of the nbtree README that explains the similarities and differences.
* Update SQL conformance informationPeter Eisentraut2019-08-22
| | | | | T612 has been fully supported since the major window function enhancements in PostgreSQL 11, but it wasn't updated at the time.
* Make SQL/JSON error code names match SQL standardPeter Eisentraut2019-08-22
| | | | | | There were some minor differences that didn't seem necessary. Discussion: https://www.postgresql.org/message-id/flat/86b67eef-bb26-c97d-3e35-64f1fbd4f9fe%402ndquadrant.com
* Update comments on nbtree stack struct.Peter Geoghegan2019-08-21
| | | | | | | | Adjust the struct comment that describes how page splits use their descent stack to cascade up the tree from the leaf level. In passing, fix up some unrelated nbtree comments that had typos or were obsolete.
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-16 07:19:17 +0000