aboutsummaryrefslogtreecommitdiff
path: root/src/backend/utils
Commit message (Collapse)AuthorAge
...
* Optimise numeric multiplication using base-NBASE^2 arithmetic.Dean Rasheed2024-08-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently mul_var() uses the schoolbook multiplication algorithm, which is O(n^2) in the number of NBASE digits. To improve performance for large inputs, convert the inputs to base NBASE^2 before multiplying, which effectively halves the number of digits in each input, theoretically speeding up the computation by a factor of 4. In practice, the actual speedup for large inputs varies between around 3 and 6 times, depending on the system and compiler used. In turn, this significantly reduces the runtime of the numeric_big regression test. For this to work, 64-bit integers are required for the products of base-NBASE^2 digits, so this works best on 64-bit machines, on which it is faster whenever the shorter input has more than 4 or 5 NBASE digits. On 32-bit machines, the additional overheads, especially during carry propagation and the final conversion back to base-NBASE, are significantly higher, and it is only faster when the shorter input has more than around 50 NBASE digits. When the shorter input has more than 6 NBASE digits (so that mul_var_short() cannot be used), but fewer than around 50 NBASE digits, there may be a noticeable slowdown on 32-bit machines. That seems to be an acceptable tradeoff, given the performance gains for other inputs, and the effort that would be required to maintain code specifically targeting 32-bit machines. Joel Jacobson and Dean Rasheed. Discussion: https://postgr.es/m/9d8a4a42-c354-41f3-bbf3-199e1957db97%40app.fastmail.com
* Extend mul_var_short() to 5 and 6-digit inputs.Dean Rasheed2024-08-15
| | | | | | | | | | | | | | | | | | | Commit ca481d3c9a introduced mul_var_short(), which is used by mul_var() whenever the shorter input has 1-4 NBASE digits and the exact product is requested. As speculated on in that commit, it can be extended to work for more digits in the shorter input. This commit extends it up to 6 NBASE digits (up to 24 decimal digits), for which it also gives a significant speedup. This covers more cases likely to occur in real-world queries, for which using base-NBASE^2 arithmetic provides little benefit. To avoid code bloat and duplication, refactor it a bit using macros and exploiting the fact that some portions of the code are shared between the different cases. Dean Rasheed, reviewed by Joel Jacobson. Discussion: https://postgr.es/m/9d8a4a42-c354-41f3-bbf3-199e1957db97%40app.fastmail.com
* Remove TRACE_SORT macroPeter Eisentraut2024-08-14
| | | | | | | | | | | | The TRACE_SORT macro guarded the availability of the trace_sort GUC setting. But it has been enabled by default ever since it was introduced in PostgreSQL 8.1, and there have been no reports that someone wanted to disable it. So just remove the macro to simplify things. (For the avoidance of doubt: The trace_sort GUC is still there. This only removes the rarely-used macro guarding it.) Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: https://www.postgresql.org/message-id/flat/be5f7162-7c1d-44e3-9a78-74dcaa6529f2%40eisentraut.org
* Use pgBufferUsage for buffer usage tracking in analyze.Masahiko Sawada2024-08-13
| | | | | | | | | | | | | | | | | | | | Previously, (auto)analyze used global variables VacuumPageHit, VacuumPageMiss, and VacuumPageDirty to track buffer usage. However, pgBufferUsage provides a more generic way to track buffer usage with support functions. This change replaces those global variables with pgBufferUsage in analyze. Since analyze was the sole user of those variables, it removes their declarations. Vacuum previously used those variables but replaced them with pgBufferUsage as part of a bug fix, commit 5cd72cc0c. Additionally, it adjusts the buffer usage message in both vacuum and analyze for better consistency. Author: Anthonin Bonnefoy Reviewed-by: Masahiko Sawada, Michael Paquier Discussion: https://postgr.es/m/CAO6_Xqr__kTTCLkftqS0qSCm-J7_xbRG3Ge2rWhucxQJMJhcRA%40mail.gmail.com
* Remove unnecessary check for NULL locale, per Coverity.Jeff Davis2024-08-12
| | | | | Discussion: https://postgr.es/m/3804933.1723394010@sss.pgh.pa.us Reported-by: Tom Lane
* Add user-callable CRC functions.Nathan Bossart2024-08-12
| | | | | | | | | | | | | | We've had code for CRC-32 and CRC-32C for some time (for WAL records, etc.), but there was no way for users to call it, despite apparent popular demand. The new crc32() and crc32c() functions accept bytea input and return bigint (to avoid returning negative values). Bumps catversion. Author: Aleksander Alekseev Reviewed-by: Peter Eisentraut, Tom Lane Discussion: https://postgr.es/m/CAJ7c6TNMTGnqnG%3DyXXUQh9E88JDckmR45H2Q%2B%3DucaCLMOW1QQw%40mail.gmail.com
* Fix a series of typos and outdated referencesDavid Rowley2024-08-12
| | | | | Author: Alexander Lakhin <exclusion@gmail.com> Discussion: https://postgr.es/m/c1d63754-cb85-2d8a-8409-bde2c4d2d04b@gmail.com
* Remove dead codePeter Eisentraut2024-08-12
| | | | | | | | | After e9931bfb751, the locale argument of SB_lower_char() is never NULL, so the branch that deals with NULL can be removed (similar to how e9931bfb751 for example removed those branches in str_tolower()). Reviewed-by: Jeff Davis <pgsql@j-davis.com> Discussion: https://www.postgresql.org/message-id/4f562d84-87f4-44dc-8946-01d6c437936f@eisentraut.org
* Remove "parent" column from pg_backend_memory_contextsDavid Rowley2024-08-12
| | | | | | | | | | | | | | | | | | | | | | | | 32d3ed816 added the "path" column to pg_backend_memory_contexts to allow a stable method of obtaining the parent MemoryContext of a given row in the view. Using the "path" column is now the preferred method of obtaining the parent row. Previously, any queries which were self-joining to this view using the "name" and "parent" columns could get incorrect results due to the fact that names are not unique. Here we aim to explicitly break such queries so that they can be corrected and use the "path" column instead. It is possible that there are more innocent users of the parent column that just need an indication of the parent and having to write out a self-joining CTE may be an unnecessary hassle for those cases. Let's remove the column for now and see if anyone comes back with any complaints. This does seem like a good time to attempt to get rid of the column as we still have around 1 year to revert this if someone comes back with a valid complaint. Plus this view is new to v14 and is quite niche, so perhaps not many people will be affected. Author: Melih Mutlu <m.melihmutlu@gmail.com> Discussion: https://postgr.es/m/CAGPVpCT7NOe4fZXRL8XaoxHpSXYTu6GTpULT_3E-HT9hzjoFRA@mail.gmail.com
* Suppress Coverity warnings about Asserts in get_name_for_var_field.Tom Lane2024-08-11
| | | | | | | | | Coverity thinks dpns->plan could be null at these points. That shouldn't really be possible, but it's easy enough to modify the Asserts so they'd not core-dump if it were true. These are new in b919a97a6. Back-patch to v13; the v12 version of the patch didn't have these Asserts.
* Allow adjusting session_authorization and role in parallel workers.Tom Lane2024-08-10
| | | | | | | | | | | | | | | | | | | | | | | | | | | The code intends to allow GUCs to be set within parallel workers via function SET clauses, but not otherwise. However, doing so fails for "session_authorization" and "role", because the assign hooks for those attempt to set the subsidiary "is_superuser" GUC, and that call falls foul of the "not otherwise" prohibition. We can't switch to using GUC_ACTION_SAVE for this, so instead add a new GUC variable flag GUC_ALLOW_IN_PARALLEL to mark is_superuser as being safe to set anyway. (This is okay because is_superuser has context PGC_INTERNAL and thus only hard-wired calls can change it. We'd need more thought before applying the flag to other GUCs; but maybe there are other use-cases.) This isn't the prettiest fix perhaps, but other alternatives we thought of would be much more invasive. While here, correct a thinko in commit 059de3ca4: when rejecting a GUC setting within a parallel worker, we should return 0 not -1 if the ereport doesn't longjmp. (This seems to have no consequences right now because no caller cares, but it's inconsistent.) Improve the comments to try to forestall future confusion of the same kind. Despite the lack of field complaints, this seems worth back-patching. Thanks to Nathan Bossart for the idea to invent a new flag, and for review. Discussion: https://postgr.es/m/2833457.1723229039@sss.pgh.pa.us
* Lower minimum maintenance_work_mem to 64kBJohn Naylor2024-08-10
| | | | | | | | | | | | | | | | | | Since the introduction of TID store, vacuum uses far less memory in the common case than in versions 16 and earlier. Invoking multiple rounds of index vacuuming in turn requires a much larger table. It'd be a good idea anyway to cover this case in regression testing, and a lower limit is less painful for slow buildfarm animals. The reason to do it now is to re-enable coverage of the bugfix in commit 83c39a1f7f. For consistency, give autovacuum_work_mem the same treatment. Suggested by Andres Freund Tested by Melanie Plageman Backpatch to v17, where TID store was introduced Discussion: https://postgr.es/m/20240516205458.ohvlzis5b5tvejru@awork3.anarazel.de Discussion: https://postgr.es/m/20240722164745.fvaoh6g6zprisqgp%40awork3.anarazel.de
* Fix "failed to find plan for subquery/CTE" errors in EXPLAIN.Tom Lane2024-08-09
| | | | | | | | | | | | | | | | | | | | | | To deparse a reference to a field of a RECORD-type output of a subquery, EXPLAIN normally digs down into the subquery's plan to try to discover exactly which anonymous RECORD type is meant. However, this can fail if the subquery has been optimized out of the plan altogether on the grounds that no rows could pass the WHERE quals, which has been possible at least since 3fc6e2d7f. There isn't anything remaining in the plan tree that would help us, so fall back to printing the field name as "fN" for the N'th column of the record. (This will actually be the right thing some of the time, since it matches the column names we assign to RowExprs.) In passing, fix a comment typo in create_projection_plan, which I noticed while experimenting with an alternative fix for this. Per bug #18576 from Vasya B. Back-patch to all supported branches. Richard Guo and Tom Lane Discussion: https://postgr.es/m/18576-9feac34e132fea9e@postgresql.org
* Add a caveat to hash_seq_init_with_hash_value() header commentAlexander Korotkov2024-08-08
| | | | | | | | | | The typical use-case for hash_seq_init_with_hash_value() is syscache callback. Add a caveat that the default hash function doesn't match syscache hash function. So, one needs to define a custom hash function. Reported-by: Pavel Stehule Discussion: https://postgr.es/m/CAFj8pRAXmv6eyYx%3DE_BTfyK%3DO_%2ByOF8sXB%3D0bn9eOBt90EgWRA%40mail.gmail.com Reviewed-by: Pavel Stehule
* Optimize InvalidateAttoptCacheCallback() and TypeCacheTypCallback()Alexander Korotkov2024-08-07
| | | | | | | | | | | | | | | | | These callbacks are receiving hash values as arguments, which doesn't allow direct lookups for AttoptCacheHash and TypeCacheHash. This is why subject callbacks currently use full iteration over corresponding hashes. This commit avoids full hash iteration in InvalidateAttoptCacheCallback(), and TypeCacheTypCallback(). At first, we switch AttoptCacheHash and TypeCacheHash to use same hash function as syscache. As second, we use hash_seq_init_with_hash_value() to iterate only hash entries with matching hash value. Discussion: https://postgr.es/m/5812a6e5-68ae-4d84-9d85-b443176966a1%40sigaev.ru Author: Teodor Sigaev Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman Zharkov Reviewed-by: Andrei Lepikhov
* Introduce hash_search_with_hash_value() functionAlexander Korotkov2024-08-07
| | | | | | | | | | | This new function iterates hash entries with given hash values. This function is designed to avoid full sequential hash search in the syscache invalidation callbacks. Discussion: https://postgr.es/m/5812a6e5-68ae-4d84-9d85-b443176966a1%40sigaev.ru Author: Teodor Sigaev Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman Zharkov Reviewed-by: Andrei Lepikhov
* Use psprintf to simplify gtsvectorout()Heikki Linnakangas2024-08-06
| | | | | | | | | | | | | | | | | | | | | | | The buffer allocation was correct, but looked archaic and scary: - It was weird to calculate the buffer size before determining which format string was used. With the same effort, we could've used the right-sized buffer for each branch. - Commit aa0d3504560 added one more possible return string ("all true bits"), but didn't adjust the code at the top of the function to calculate the returned string's max size. It was not a live bug, because the new string was smaller than the existing ones, but seemed wrong in principle. - Use of sprintf() is generally eyebrow-raising these days Switch to psprintf(). psprintf() allocates a larger buffer than what was allocated before, 128 bytes vs 80 bytes, which is acceptable as this code is not performance or space critical. Reviewed-by: Andres Freund Discussion: https://www.postgresql.org/message-id/54c29fb0-edf2-48ea-9814-44e918bbd6e8@iki.fi
* Make nullSemAction const, add 'const' decorators to related functionsHeikki Linnakangas2024-08-06
| | | | | | | To make it more clear that these should never be modified. Reviewed-by: Andres Freund Discussion: https://www.postgresql.org/message-id/54c29fb0-edf2-48ea-9814-44e918bbd6e8@iki.fi
* selfuncs.c: use pg_strxfrm() instead of strxfrm().Jeff Davis2024-08-06
| | | | | | | | | | | | pg_strxfrm() takes a pg_locale_t, so it works properly with all providers. This improves estimates for ICU when performing linear interpolation within a histogram bin. Previously, convert_string_datum() always used strxfrm() and relied on setlocale(). That did not produce good estimates for non-default or non-libc collations. Discussion: https://postgr.es/m/89475ee5487d795124f4e25118ea8f1853edb8cb.camel@j-davis.com
* Remove support for null pg_locale_t most places.Jeff Davis2024-08-05
| | | | | | | | | | | | | Previously, passing NULL for pg_locale_t meant "use the libc provider and the server environment". Now that the database collation is represented as a proper pg_locale_t (not dependent on setlocale()), remove special cases for NULL. Leave wchar2char() and char2wchar() unchanged for now, because the callers don't always have a libc-based pg_locale_t available. Discussion: https://postgr.es/m/cfd9eb85-c52a-4ec9-a90e-a5e4de56e57d@eisentraut.org Reviewed-by: Peter Eisentraut, Andreas Karlsson
* Restrict accesses to non-system views and foreign tables during pg_dump.Masahiko Sawada2024-08-05
| | | | | | | | | | | | | | | | | | | | | | | | When pg_dump retrieves the list of database objects and performs the data dump, there was possibility that objects are replaced with others of the same name, such as views, and access them. This vulnerability could result in code execution with superuser privileges during the pg_dump process. This issue can arise when dumping data of sequences, foreign tables (only 13 or later), or tables registered with a WHERE clause in the extension configuration table. To address this, pg_dump now utilizes the newly introduced restrict_nonsystem_relation_kind GUC parameter to restrict the accesses to non-system views and foreign tables during the dump process. This new GUC parameter is added to back branches too, but these changes do not require cluster recreation. Back-patch to all supported branches. Reviewed-by: Noah Misch Security: CVE-2024-7348 Backpatch-through: 12
* Optimize JSON escaping using SIMDDavid Rowley2024-08-05
| | | | | | | | | | | | | | | | | | | | | | | | Here we adjust escape_json_with_len() to make use of SIMD to allow processing of up to 16-bytes at a time rather than processing a single byte at a time. This has been shown to speed up escaping of JSON strings significantly. Escaping is required for both JSON string properties and also the property names themselves, so this should also help improve the speed of the conversion from JSON into text for JSON objects that have property names 16 or more bytes long. Escaping JSON strings was often a significant bottleneck for longer strings. With these changes, some benchmarking has shown a query performing nearly 4 times faster when escaping a JSON object with a 1MB text property. Tests with shorter text properties saw smaller but still significant performance improvements. For example, a test outputting 1024 JSON strings with a text property length ranging from 1 char to 1024 chars became around 2 times faster. Author: David Rowley Reviewed-by: Melih Mutlu Discussion: https://postgr.es/m/CAApHDvpLXwMZvbCKcdGfU9XQjGCDm7tFpRdTXuB9PVgpNUYfEQ@mail.gmail.com
* Introduce pluggable APIs for Cumulative StatisticsMichael Paquier2024-08-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds support in the backend for $subject, allowing out-of-core extensions to plug their own custom kinds of cumulative statistics. This feature has come up a few times into the lists, and the first, original, suggestion came from Andres Freund, about pg_stat_statements to use the cumulative statistics APIs in shared memory rather than its own less efficient internals. The advantage of this implementation is that this can be extended to any kind of statistics. The stats kinds are divided into two parts: - The in-core "builtin" stats kinds, with designated initializers, able to use IDs up to 128. - The "custom" stats kinds, able to use a range of IDs from 128 to 256 (128 slots available as of this patch), with information saved in TopMemoryContext. This can be made larger, if necessary. There are two types of cumulative statistics in the backend: - For fixed-numbered objects (like WAL, archiver, etc.). These are attached to the snapshot and pgstats shmem control structures for efficiency, and built-in stats kinds still do that to avoid any redirection penalty. The data of custom kinds is stored in a first array in snapshot structure and a second array in the shmem control structure, both indexed by their ID, acting as an equivalent of the builtin stats. - For variable-numbered objects (like tables, functions, etc.). These are stored in a dshash using the stats kind ID in the hash lookup key. Internally, the handling of the builtin stats is unchanged, and both fixed and variabled-numbered objects are supported. Structure definitions for builtin stats kinds are renamed to reflect better the differences with custom kinds. Like custom RMGRs, custom cumulative statistics can only be loaded with shared_preload_libraries at startup, and must allocate a unique ID shared across all the PostgreSQL extension ecosystem with the following wiki page to avoid conflicts: https://wiki.postgresql.org/wiki/CustomCumulativeStats This makes the detection of the stats kinds and their handling when reading and writing stats much easier than, say, allocating IDs for stats kinds from a shared memory counter, that may change the ID used by a stats kind across restarts. When under development, extensions can use PGSTAT_KIND_EXPERIMENTAL. Two examples that can be used as templates for fixed-numbered and variable-numbered stats kinds will be added in some follow-up commits, with tests to provide coverage. Some documentation is added to explain how to use this plugin facility. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot Discussion: https://postgr.es/m/Zmqm9j5EO0I4W8dx@paquier.xyz
* Fix incorrect format placeholders in pgstat.cMichael Paquier2024-08-04
| | | | | | | | These should have been switched from %d to %u in 3188a4582a8c in the debugging elogs added in ca1ba50fcb6f. PgStat_Kind should never be higher than INT32_MAX, but let's be clean. Issue noticed while hacking more on this area.
* Implement pg_wal_replay_wait() stored procedureAlexander Korotkov2024-08-02
| | | | | | | | | | | | | | | | | | | | | | | | | | pg_wal_replay_wait() is to be used on standby and specifies waiting for the specific WAL location to be replayed. This option is useful when the user makes some data changes on primary and needs a guarantee to see these changes are on standby. The queue of waiters is stored in the shared memory as an LSN-ordered pairing heap, where the waiter with the nearest LSN stays on the top. During the replay of WAL, waiters whose LSNs have already been replayed are deleted from the shared memory pairing heap and woken up by setting their latches. pg_wal_replay_wait() needs to wait without any snapshot held. Otherwise, the snapshot could prevent the replay of WAL records, implying a kind of self-deadlock. This is why it is only possible to implement pg_wal_replay_wait() as a procedure working without an active snapshot, not a function. Catversion is bumped. Discussion: https://postgr.es/m/eb12f9b03851bb2583adab5df9579b4b%40postgrespro.ru Author: Kartyshov Ivan, Alexander Korotkov Reviewed-by: Michael Paquier, Peter Eisentraut, Dilip Kumar, Amit Kapila Reviewed-by: Alexander Lakhin, Bharath Rupireddy, Euler Taveira Reviewed-by: Heikki Linnakangas, Kyotaro Horiguchi
* Switch PgStat_Kind from an enum to a uint32 typeMichael Paquier2024-08-02
| | | | | | | | | | | | | | | | A follow-up patch is planned to make cumulative statistics pluggable, and using a type is useful in the internal routines used by pgstats as PgStat_Kind may have a value that was not originally in the enum removed here, once made pluggable. While on it, this commit switches pgstat_is_kind_valid() to use PgStat_Kind rather than an int, to be more consistent with its existing callers. Some loops based on the stats kind IDs are switched to use PgStat_Kind rather than int, for consistency with the new time. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot Discussion: https://postgr.es/m/Zmqm9j5EO0I4W8dx@paquier.xyz
* Add redo LSN to pgstats filesMichael Paquier2024-08-02
| | | | | | | | | | | | | | | | This is used in the startup process to check that the pgstats file we are reading includes the redo LSN referring to the shutdown checkpoint where it has been written. The redo LSN in the pgstats file needs to match with what the control file has. This is intended to be used for an upcoming change that will extend the write of the stats file to happen during checkpoints, rather than only shutdown sequences. Bump PGSTAT_FILE_FORMAT_ID. Reviewed-by: Bertrand Drouvot Discussion: https://postgr.es/m/Zp8o6_cl0KSgsnvS@paquier.xyz
* Convert node test compile-time settings into run-time parametersPeter Eisentraut2024-08-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This converts COPY_PARSE_PLAN_TREES WRITE_READ_PARSE_PLAN_TREES RAW_EXPRESSION_COVERAGE_TEST into run-time parameters debug_copy_parse_plan_trees debug_write_read_parse_plan_trees debug_raw_expression_coverage_test They can be activated for tests using PG_TEST_INITDB_EXTRA_OPTS. The compile-time symbols are kept for build farm compatibility, but they now just determine the default value of the run-time settings. Furthermore, support for these settings is not compiled in at all unless assertions are enabled, or the new symbol DEBUG_NODE_TESTS_ENABLED is defined at compile time, or any of the legacy compile-time setting symbols are defined. So there is no run-time overhead in production builds. (This is similar to the handling of DISCARD_CACHES_ENABLED.) Discussion: https://www.postgresql.org/message-id/flat/30747bd8-f51e-4e0c-a310-a6e2c37ec8aa%40eisentraut.org
* Relax check for return value from second call of pg_strnxfrm().Jeff Davis2024-07-30
| | | | | | | | | strxfrm() is not guaranteed to return the exact number of bytes needed to store the result; it may return a higher value. Discussion: https://postgr.es/m/32f85d88d1f64395abfe5a10dd97a62a4d3474ce.camel@j-davis.com Reviewed-by: Heikki Linnakangas Backpatch-through: 16
* Refactor getWeights to write to caller-supplied bufferHeikki Linnakangas2024-07-30
| | | | | | | This gets rid of the static result buffer. Reviewed-by: Robert Haas Discussion: https://www.postgresql.org/message-id/7f86e06a-98c5-4ce3-8ec9-3885c8de0358@iki.fi
* Replace static bufs with a StringInfo in cash_words()Heikki Linnakangas2024-07-30
| | | | | | | | | | | For clarity. The code was correct, and the buffer was large enough, but string manipulation with no bounds checking is scary. This incurs an extra palloc+pfree to every call, but in quick performance testing, it doesn't seem to be significant. Reviewed-by: Robert Haas Discussion: https://www.postgresql.org/message-id/7f86e06a-98c5-4ce3-8ec9-3885c8de0358@iki.fi
* Preserve tz when converting to jsonb timestamptzAndrew Dunstan2024-07-30
| | | | | | | | | | | | | | This removes an inconsistency in the treatment of different datatypes by the jsonpath timestamp_tz() function. Conversions from data types that are not timestamp-aware, such as date and timestamp, are now treated consistently with conversion from those that are such as timestamptz. Author: David Wheeler Reviewed-by: Junwang Zhao and Jeevan Chalke Discussion: https://postgr.es/m/7DE080CE-6D8C-4794-9BD1-7D9699172FAB%40justatheory.com Backpatch to release 17.
* Make collation not depend on setlocale().Jeff Davis2024-07-30
| | | | | | | | | | | | | | | | | Now that the result of pg_newlocale_from_collation() is always non-NULL, then we can move the collate_is_c and ctype_is_c flags into pg_locale_t. That simplifies the logic in lc_collate_is_c() and lc_ctype_is_c(), removing the dependence on setlocale(). This commit also eliminates the multi-stage initialization of the collation cache. As long as we have catalog access, then it's now safe to call pg_newlocale_from_collation() without checking lc_collate_is_c() first. Discussion: https://postgr.es/m/cfd9eb85-c52a-4ec9-a90e-a5e4de56e57d@eisentraut.org Reviewed-by: Peter Eisentraut, Andreas Karlsson
* Fix partitionwise join with partially-redundant join clausesRichard Guo2024-07-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To determine if the two relations being joined can use partitionwise join, we need to verify the existence of equi-join conditions involving pairs of matching partition keys for all partition keys. Currently we do that by looking through the join's restriction clauses. However, it has been discovered that this approach is insufficient, because there might be partition keys known equal by a specific EC, but they do not form a join clause because it happens that other members of the EC than the partition keys are constrained to become a join clause. To address this issue, in addition to examining the join's restriction clauses, we also check if any partition keys are known equal by ECs, by leveraging function exprs_known_equal(). To accomplish this, we enhance exprs_known_equal() to check equality per the semantics of the opfamily, if provided. It could be argued that exprs_known_equal() could be called O(N^2) times, where N is the number of partition key expressions, resulting in noticeable performance costs if there are a lot of partition key expressions. But I think this is not a problem. The number of a joinrel's partition key expressions would only be equal to the join degree, since each base relation within the join contributes only one partition key expression. That is to say, it does not scale with the number of partitions. A benchmark with a query involving 5-way joins of partitioned tables, each with 3 partition keys and 1000 partitions, shows that the planning time is not significantly affected by this patch (within the margin of error), particularly when compared to the impact caused by partitionwise join. Thanks to Tom Lane for the idea of leveraging exprs_known_equal() to check if partition keys are known equal by ECs. Author: Richard Guo, Tom Lane Reviewed-by: Tom Lane, Ashutosh Bapat, Robert Haas Discussion: https://postgr.es/m/CAN_9JTzo_2F5dKLqXVtDX5V6dwqB0Xk+ihstpKEt3a1LT6X78A@mail.gmail.com
* Add more debugging information when failing to read pgstats filesMichael Paquier2024-07-30
| | | | | | | | | | This is useful to know which part of a stats file is corrupted when reading it, adding to the server logs a WARNING with details about what could not be read before giving up with the remaining data in the file. Author: Michael Paquier Reviewed-by: Bertrand Drouvot Discussion: https://postgr.es/m/Zp8o6_cl0KSgsnvS@paquier.xyz
* Do not return NULL from pg_newlocale_from_collation().Jeff Davis2024-07-29
| | | | | | | | | | | | | | | | | | | | Previously, pg_newlocale_from_collation() returned NULL as a special case for the DEFAULT_COLLATION_OID if the provider was libc. In that case the behavior would depend on the last call to setlocale(). Now, consistent with the other providers, it will return a pointer to default_locale, which is not dependent on setlocale(). Note: for the C and POSIX locales, the locale_t structure within the pg_locale_t will still be zero, because those locales are implemented with internal logic and do not use libc at all. lc_collate_is_c() and lc_ctype_is_c() still depend on setlocale() to determine the current locale, which will be removed in a subsequent commit. Discussion: https://postgr.es/m/cfd9eb85-c52a-4ec9-a90e-a5e4de56e57d@eisentraut.org Reviewed-by: Peter Eisentraut, Andreas Karlsson
* Remove dead generators for cyrillic encoding conversion tablesHeikki Linnakangas2024-07-29
| | | | | | | | | | | | | | | | | | | | These tools were used to read the koi-iso.tab, koi-win.tab, and koi-alt.tab files, which contained the mappings between the single-byte cyrillic encodings. However, those data files were removed in commit 4c3c8c048d, back in 2003. These code generators have been unused and unusable ever since. The generated tables live in cyrillic_and_mic.c. There has been one change to the tables since they were generated in 1999, in commit f4b7624eb07a. So if we resurrected the original data tables, that change would need to be taken into account. So this code is very dead. The tables in cyrillic_and_mic.c, which were originally generated by these tools, are now the authoritative source for these mappings. Reviewed-by: Tom Lane, Aleksander Alekseev Discussion: https://www.postgresql.org/message-id/flat/a821c3dc-36ec-4cee-8b41-7ccaa17adb18@iki.fi
* Move cancel key generation to after forking the backendHeikki Linnakangas2024-07-29
| | | | | | | | | | | | | | | | | | | | | | Move responsibility of generating the cancel key to the backend process. The cancel key is now generated after forking, and the backend advertises it in the ProcSignal array. When a cancel request arrives, the backend handling it scans the ProcSignal array to find the target pid and cancel key. This is similar to how this previously worked in the EXEC_BACKEND case with the ShmemBackendArray, just reusing the ProcSignal array. One notable change is that we no longer generate cancellation keys for non-backend processes. We generated them before just to prevent a malicious user from canceling them; the keys for non-backend processes were never actually given to anyone. There is now an explicit flag indicating whether a process has a valid key or not. I wrote this originally in preparation for supporting longer cancel keys, but it's a nice cleanup on its own. Reviewed-by: Jelte Fennema-Nio Discussion: https://www.postgresql.org/message-id/508d0505-8b7a-4864-a681-e7e5edfe32aa@iki.fi
* Add missing pointer dereference in pg_backend_memory_contexts viewDavid Rowley2024-07-29
| | | | | | | | | | | | | | | 32d3ed816 moved the logic for setting the context's name and ident into a reusable function. I missed adding a pointer dereference after copying and pasting the code into that function. The ident parameter is a pointer to the ident variable in the calling function, so the dereference is required to correctly determine if the contents of that variable is NULL or not. In passing, adjust the if condition to include an == NULL to make it more clear that it's not checking for == '\0'. Reported-by: Tom Lane, Coverity Discussion: https://postgr.es/m/2256588.1722184287@sss.pgh.pa.us
* Fix whitespace in commit 005c6b833f.Jeff Davis2024-07-28
|
* Refactor: make default_locale internal to pg_locale.c.Jeff Davis2024-07-28
| | | | | Discussion: https://postgr.es/m/2228884bb1f1a02614b39f71a90c94d2cc8a3a2f.camel@j-davis.com Reviewed-by: Peter Eisentraut, Andreas Karlsson
* Change collation cache to use simplehash.h.Jeff Davis2024-07-28
| | | | | | | | | Speeds up text comparison expressions when using a collation other than the database default collation. Does not affect larger operations such as ORDER BY, because the lookup is only done once. Discussion: https://postgr.es/m/7bb9f018d20a7b30b9a7f6231efab1b5e50c7720.camel@j-davis.com Reviewed-by: John Naylor, Andreas Karlsson
* Fix incorrect return value for pg_size_pretty(bigint)David Rowley2024-07-28
| | | | | | | | | | | | | | | | | pg_size_pretty(bigint) would return the value in bytes rather than PB for the smallest-most bigint value. This happened due to an incorrect assumption that the absolute value of -9223372036854775808 could be stored inside a signed 64-bit type. Here we fix that by instead storing that value in an unsigned 64-bit type. This bug does exist in versions prior to 15 but the code there is sufficiently different and the bug seems sufficiently non-critical that it does not seem worth risking backpatching further. Author: Joseph Koshakow <koshy44@gmail.com> Discussion: https://postgr.es/m/CAAvxfHdTsMZPWEHUrZ=h3cky9Ccc3Mtx2whUHygY+ABP-mCmUw@mail.gmail.com Backpatch-through: 15
* Optimize escaping of JSON stringsDavid Rowley2024-07-27
| | | | | | | | | | | | | | | | | | | | | | | | There were quite a few places where we either had a non-NUL-terminated string or a text Datum which we needed to call escape_json() on. Many of these places required that a temporary string was created due to the fact that escape_json() needs a NUL-terminated cstring. For text types, those first had to be converted to cstring before calling escape_json() on them. Here we introduce two new functions to make escaping JSON more optimal: escape_json_text() can be given a text Datum to append onto the given buffer. This is more optimal as it foregoes the need to convert the text Datum into a cstring. A temporary allocation is only required if the text Datum needs to be detoasted. escape_json_with_len() can be used when the length of the cstring is already known or the given string isn't NUL-terminated. Having this allows various places which were creating a temporary NUL-terminated string to just call escape_json_with_len() without any temporary memory allocations. Discussion: https://postgr.es/m/CAApHDvpLXwMZvbCKcdGfU9XQjGCDm7tFpRdTXuB9PVgpNUYfEQ@mail.gmail.com Reviewed-by: Melih Mutlu, Heikki Linnakangas
* Introduce num_os_semaphores GUC.Nathan Bossart2024-07-26
| | | | | | | | | | | | | | | | | | | The documentation for System V IPC parameters provides complicated formulas to determine the appropriate values for SEMMNI and SEMMNS. Furthermore, these formulas have often been wrong because folks forget to update them (e.g., when adding a new auxiliary process). This commit introduces a new runtime-computed GUC named num_os_semaphores that reports the number of semaphores needed for the configured number of allowed connections, worker processes, etc. This new GUC allows us to simplify the formulas in the documentation, and it should help prevent future inaccuracies. Like the other runtime-computed GUCs, users can view it with "postgres -C" before starting the server, which is useful for preconfiguring the necessary operating system resources. Reviewed-by: Tom Lane, Sami Imseih, Andres Freund, Robert Haas Discussion: https://postgr.es/m/20240517164452.GA1914161%40nathanxps13
* Add test for early backend startup errorsHeikki Linnakangas2024-07-26
| | | | | | | | | | | | | | | | The new test tests the libpq fallback behavior on an early error, which was fixed in the previous commit. This adds an IS_INJECTION_POINT_ATTACHED() macro, to allow writing injected test code alongside the normal source code. In principle, the new test could've been implemented by an extra test module with a callback that sets the FrontendProtocol global variable, but I think it's more clear to have the test code right where the injection point is, because it has pretty intimate knowledge of the surrounding context it runs in. Reviewed-by: Michael Paquier Discussion: https://www.postgresql.org/message-id/CAOYmi%2Bnwvu21mJ4DYKUa98HdfM_KZJi7B1MhyXtnsyOO-PB6Ww%40mail.gmail.com
* Fix using injection points at backend startup in EXEC_BACKEND modeHeikki Linnakangas2024-07-26
| | | | | | | | | | | | Commit 86db52a506 changed the locking of injection points to use only atomic ops and spinlocks, to make it possible to define injection points in processes that don't have a PGPROC entry (yet). However, it didn't work in EXEC_BACKEND mode, because the pointer to shared memory area was not initialized until the process "attaches" to all the shared memory structs. To fix, pass the pointer to the child process along with other global variables that need to be set up early. Backpatch-through: 17
* SQL/JSON: Respect OMIT QUOTES when RETURNING domains over jsonbAmit Langote2024-07-26
| | | | | | | | | | | | populate_domain() didn't take into account the omit_quotes flag passed down to json_populate_type() by ExecEvalJsonCoercion() and that led to incorrect behavior when the RETURNING type is a domain over jsonb. Fix that by passing the flag by adding a new function parameter to populate_domain(). Reported-by: Jian He <jian.universality@gmail.com> Discussion: https://postgr.es/m/CACJufxEo4sUjKCYtda0_qt9tazqqKPmF1cqhW9KBOUeJFqQd2g@mail.gmail.com Backpatch-through: 17
* Fix -Wmissing-variable-declarations warnings for float.c special casePeter Eisentraut2024-07-25
| | | | | | | | | | This adds extern declarations for the global variables defined in float.c but not meant for external use. This is a workaround to be able to add -Wmissing-variable-declarations to the global set of warning options in the near future. Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://www.postgresql.org/message-id/flat/e0a62134-83da-4ba4-8cdb-ceb0111c95ce@eisentraut.org
* Add path column to pg_backend_memory_contexts viewDavid Rowley2024-07-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | "path" provides a reliable method of determining the parent/child relationships between memory contexts. Previously this could be done in a non-reliable way by writing a recursive query and joining the "parent" and "name" columns. This wasn't reliable as the names were not unique, which could result in joining to the wrong parent. To make this reliable, "path" stores an array of numerical identifiers starting with the identifier for TopLevelMemoryContext. It contains an element for each intermediate parent between that and the current context. Incompatibility: Here we also adjust the "level" column to make it 1-based rather than 0-based. A 1-based level provides a convenient way to access elements in the "path" array. e.g. path[level] gives the identifier for the current context. Identifiers are not stable across multiple evaluations of the view. In an attempt to make these more stable for ad-hoc queries, the identifiers are assigned breadth-first. Contexts closer to TopLevelMemoryContext are less likely to change between queries and during queries. Author: Melih Mutlu <m.melihmutlu@gmail.com> Discussion: https://postgr.es/m/CAGPVpCThLyOsj3e_gYEvLoHkr5w=tadDiN_=z2OwsK3VJppeBA@mail.gmail.com Reviewed-by: Andres Freund, Stephen Frost, Atsushi Torikoshi, Reviewed-by: Michael Paquier, Robert Haas, David Rowley
generated by cgit v1.2.3 (git 2.25.1) at 2025-05-31 04:01:56 +0000