aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
...
* Avoid PGDLLIMPORT for simple local references in frontend programs.Tom Lane2016-03-23
| | | | | I was wondering if this would be an issue, and buildfarm member frogmouth says it is.
* Support CREATE ACCESS METHODAlvaro Herrera2016-03-23
| | | | | | | | | | | | | | | This enables external code to create access methods. This is useful so that extensions can add their own access methods which can be formally tracked for dependencies, so that DROP operates correctly. Also, having explicit support makes pg_dump work correctly. Currently only index AMs are supported, but we expect different types to be added in the future. Authors: Alexander Korotkov, Petr Jelínek Reviewed-By: Teodor Sigaev, Petr Jelínek, Jim Nasby Commitfest-URL: https://commitfest.postgresql.org/9/353/ Discussion: https://www.postgresql.org/message-id/CAPpHfdsXwZmojm6Dx+TJnpYk27kT4o7Ri6X_4OSWcByu1Rm+VA@mail.gmail.com
* Move keywords.c/kwlookup.c into src/common/.Tom Lane2016-03-23
| | | | | | | | | | | | | | | | | | | Now that we have src/common/ for code shared between frontend and backend, we can get rid of (most of) the klugy ways that the keyword table and keyword lookup code were formerly shared between different uses. This is a first step towards a more general plan of getting rid of special-purpose kluges for sharing code in src/bin/. I chose to merge kwlookup.c back into keywords.c, as it once was, and always has been so far as keywords.h is concerned. We could have kept them separate, but there is noplace that uses ScanKeywordLookup without also wanting access to the backend's keyword list, so there seems little point. ecpg is still a bit weird, but at least now the trickiness is documented. I think that the MSVC build script should require no adjustments beyond what's done here ... but we'll soon find out.
* Disable abbreviated keys for string-sorting in non-C locales.Robert Haas2016-03-23
| | | | | | | | | | | | | | | | | | | Unfortunately, every version of glibc thus far tested has bugs whereby strcoll() ordering does not match strxfrm() ordering as required by the standard. This can result in, for example, corrupted indexes. Disabling abbreviated keys in these cases slows down non-C-collation string sorting considerably, but there seems to be no practical alternative. Users who are confident that their libc implementations are solid in this regard can re-enable the optimization by compiling with TRUST_STRXFRM. Users who have built indexes using PostgreSQL 9.5 or PostgreSQL 9.5.1 should REINDEX if there is a possibility that they may have been affected by this problem. Report by Marc-Olaf Jaschke. Investigation mostly by Tom Lane, with help from Peter Geoghegan, Noah Misch, Stephen Frost, and me. Patch by me, reviewed by Peter Geoghegan and Tom Lane.
* Partition the freelist for shared dynahash tables.Robert Haas2016-03-23
| | | | | | | | Without this, contention on the freelist can become a pretty serious problem on large servers. Aleksander Alekseev, reviewed by Anastasia Lubennikova, Dilip Kumar, and me.
* Code review for error reports in jsonb_set().Tom Lane2016-03-23
| | | | | | User-facing (even tested by regression tests) error conditions were thrown with elog(), hence had wrong SQLSTATE and were untranslatable. And the error message texts weren't up to project style, either.
* Fix unsafe use of strtol() on a non-null-terminated Text datum.Tom Lane2016-03-23
| | | | | | | | | | jsonb_set() could produce wrong answers or incorrect error reports, or in the worst case even crash, when trying to convert a path-array element into an integer for use as an array subscript. Per report from Vitaly Burovoy. Back-patch to 9.5 where the faulty code was introduced (in commit c6947010ceb42143). Michael Paquier
* Change comment to describe correct lock level usedSimon Riggs2016-03-23
|
* Fix EvalPlanQual bug when query contains both locked and not-locked rels.Tom Lane2016-03-22
| | | | | | | | | | | | | | | | | In commit afb9249d06f47d7a, we (probably I) made ExecLockRows assign null test tuples to all relations of the query while setting up to do an EvalPlanQual recheck for a newly-updated locked row. This was sheerest brain fade: we should only set test tuples for relations that are lockable by the LockRows node, and in particular empty test tuples are only sensible for inheritance child relations that weren't the source of the current tuple from their inheritance tree. Setting a null test tuple for an unrelated table causes it to return NULLs when it should not, as exhibited in bug #14034 from Bronislav Houdek. To add insult to injury, doing it the wrong way required two loops where one would suffice; so the corrected code is even a bit shorter and faster. Add a regression test case based on his example, and back-patch to 9.5 where the bug was introduced.
* Allow the delay in psql's \watch command to be a fractional second.Tom Lane2016-03-21
| | | | | | Instead of just "2" seconds, allow eg. "2.5" seconds. Per request from Alvaro Herrera. No docs change since the docs didn't say you couldn't do this already.
* Improve header output from psql's \watch command.Tom Lane2016-03-21
| | | | | | | | Include the \pset title string if there is one, and shorten the prefab part of the header to be "timestamp (every Ns)". Per suggestion by David Johnston. Michael Paquier and Tom Lane
* Make max_parallel_degree PGC_USERSET.Robert Haas2016-03-21
| | | | | It was intended to be this way all along, just like other planner GUCs such as work_mem. But I goofed.
* Support parallel aggregation.Robert Haas2016-03-21
| | | | | | | | | Parallel workers can now partially aggregate the data and pass the transition values back to the leader, which can combine the partial results to produce the final answer. David Rowley, based on earlier work by Haribabu Kommi. Reviewed by Álvaro Herrera, Tomas Vondra, Amit Kapila, James Sewell, and me.
* Properly declare FeBeWaitSet.Andres Freund2016-03-21
| | | | | Surprising that this worked on a number of systems. Reported by buildfarm member longfin.
* Introduce WaitEventSet API.Andres Freund2016-03-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit ac1d794 ("Make idle backends exit if the postmaster dies.") introduced a regression on, at least, large linux systems. Constantly adding the same postmaster_alive_fds to the OSs internal datastructures for implementing poll/select can cause significant contention; leading to a performance regression of nearly 3x in one example. This can be avoided by using e.g. linux' epoll, which avoids having to add/remove file descriptors to the wait datastructures at a high rate. Unfortunately the current latch interface makes it hard to allocate any persistent per-backend resources. Replace, with a backward compatibility layer, WaitLatchOrSocket with a new WaitEventSet API. Users can allocate such a Set across multiple calls, and add more than one file-descriptor to wait on. The latter has been added because there's upcoming postgres features where that will be helpful. In addition to the previously existing poll(2), select(2), WaitForMultipleObjects() implementations also provide an epoll_wait(2) based implementation to address the aforementioned performance problem. Epoll is only available on linux, but that is the most likely OS for machines large enough (four sockets) to reproduce the problem. To actually address the aforementioned regression, create and use a long-lived WaitEventSet for FE/BE communication. There are additional places that would benefit from a long-lived set, but that's a task for another day. Thanks to Amit Kapila, who helped make the windows code I blindly wrote actually work. Reported-By: Dmitry Vasilyev Discussion: CAB-SwXZh44_2ybvS5Z67p_CDz=XFn4hNAD=CnMEF+QqkXwFrGg@mail.gmail.com 20160114143931.GG10941@awork2.anarazel.de
* Combine win32 and unix latch implementations.Andres Freund2016-03-21
| | | | | | | | | | | | | Previously latches for windows and unix had been implemented in different files. A later patch introduce an expanded wait infrastructure, keeping the implementation separate would introduce too much duplication. This basically just moves the functions, without too much change. The reason to keep this separate is that it allows blame to continue working a little less badly; and to make review a tiny bit easier. Discussion: 20160114143931.GG10941@awork2.anarazel.de
* Second attempt at fixing MSVC build for 68ab8e8ba4a471d9.Andres Freund2016-03-21
| | | | | | | | | | | | After the previous fix in 6f1f34c9 msvc ended up looking for psqlscan.c in the wrong directory. David's fix just forces the path to be adjusted. That's not a particularly pretty fix, but it hopefully will make the buildfarm green again. Author: David Rowley Discussion: CAKJS1f_9CCi_t+LEgV5GWoCj3wjavcMoDc5qfcf_A0UwpQoPoA@mail.gmail.com
* Use %option bison-bridge in psql/pgbench lexers.Tom Lane2016-03-20
| | | | | | | | | | | | | | | | | | | The point of this change is to use %pure-parser in pgbench's exprparse.y. The immediate reason is that it turns out very ancient versions of bison have a bug with the combination of a reentrant lexer and non-reentrant parser. We could consider dropping support for such ancient bisons; but considering that we might well need exprparse.y to be reentrant some day, it seems better to make it so right now than to move the portability goalposts. (AFAICT there's no particular performance consequence to this change, either, so there's no good reason not to do it.) Now, %pure-parser assumes that the called lexer is built with %option bison-bridge. Because we're assuming bitwise compatibility of yyscan_t (yyguts_t) data structures among all the psql/pgbench lexers, that requirement propagates back to psql's lexers as well. But it's just a few lines of change on that side too; and if psqlscan.l is to set the baseline for a possibly-large family of lexers, it should err on the side of including not omitting useful features.
* Best-guess attempt at fixing MSVC build for 68ab8e8ba4a471d9.Tom Lane2016-03-20
| | | | | | pgbench now needs to use src/bin/psql/psqlscan.l, but it's not very clear how to fit that into the MSVC build system. If this doesn't work I'm going to need some help from somebody who actually understands those scripts ...
* SQL commands in pgbench scripts are now ended by semicolons, not newlines.Tom Lane2016-03-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To allow multiline SQL commands in scripts, adopt the same rules psql uses to decide what is the end of a SQL command, to wit, an unquoted semicolon not encased in parentheses. Do this by importing the same flex lexer that psql uses, since coping with stuff like dollar-quoted literals is hard to get right without going the full nine yards. This makes use of the infrastructure added in commit 0ea9efbe9ec1bf07 to support independently-written flex lexers scanning the same PsqlScanState input-buffer data structure. Since that infrastructure isn't very friendly to ad-hoc parsing code such as strtok(), improve exprscan.l so that it can parse either whitespace-separated words or expression tokens, on demand, and rewrite pgbench.c's backslash-command parsing code to always use the lexer to fetch tokens. It's still the case that pgbench backslash commands extend to the end of the line, no more and no less. That could be changed in a fairly localized way now, and there was some interest in doing so, but it seems like material for a separate patch. In passing, make some marginal cleanups in syntax error reporting, const-ify a few data structures that could use it, and run some of this code through pgindent. I can't tell whether the MSVC build scripts need to be taught explicitly about the changes here or not, but the buildfarm will soon tell us. Kyotaro Horiguchi and Tom Lane
* Remove dependency on psed for MSVC builds.Andrew Dunstan2016-03-19
| | | | | | | | | | | | Modern Perl has removed psed from its core distribution, so it might not be readily available on some build platforms. We therefore replace its use with a Perl script generated by s2p, which is equivalent to the sed script. The latter is retained for non-MSVC builds to avoid creating a new hard dependency on Perl for non-Windows tarball builds. Backpatch to all live branches. Michael Paquier and me.
* Fix phony .PHONY.Tom Lane2016-03-19
| | | | A couple makefiles had misspelled the magic .PHONY target as PHONY.
* Make pgbench's expression lexer reentrant.Tom Lane2016-03-19
| | | | | | | | | | | | | | This is a necessary preliminary step for making it play with psqlscan.l given the way I set up the lexer input-buffer sharing mechanism in commit 0ea9efbe9ec1bf07. I've not tried to make it *actually* reentrant; there's still some static variables laying about. But flex thinks it's reentrant, and that's what counts. In support of that, fix exprparse.y to pass through the yyscan_t from the caller. Also do some minor code beautification, like not casting away const.
* pgbench: Silence new compiler warningsAlvaro Herrera2016-03-19
| | | | | | | The original coding in 7bafffea647 and previous wasn't all that great anyway. Reported by Jeff Janes and Tom Lane
* Typo fix.Tom Lane2016-03-19
|
* Sync backend/parser/scan.l with bin/psql/psqlscan.l.Tom Lane2016-03-19
| | | | | | | | | | | | | | Make some minor formatting adjustments to make it easier to diff these files and see that they indeed implement the same flex rules (at least to the extent that we want them to be the same). (Someday it'd be nice to make ecpg's pgc.l more easily diff'able too, but today is not that day.) Also run relevant parts of these files and psqlscanslash.l through pgindent. No actual behavioral changes here, just obsessive neatnik-ism.
* Build backend/parser/scan.l and interfaces/ecpg/preproc/pgc.l standalone.Tom Lane2016-03-19
| | | | | | | | | | | | | Now that we know about the %top{} trick, we can revert to building flex lexers as separate .o files. This is worth doing for a couple of reasons besides sheer cleanliness. We can narrow the scope of the -Wno-error flag that's forced on scan.c. Also, since these grammar and lexer files are so large, splitting them into separate build targets should have some advantages in build speed, particularly in parallel or ccache'd builds. We have quite a few other .l files that could be changed likewise, but the above arguments don't apply to them, so the benefit of fixing them seems pretty minimal. Leave the rest for some other day.
* pgbench: Allow changing weights for scriptsAlvaro Herrera2016-03-19
| | | | | | | | | | | | | | | | | | | | | Previously, all scripts had the same probability of being chosen when multiple of them were specified via -b, -f, -N, -S. With this commit, -b and -f now search for an "@" in the script name and use the integer found after it as the drawing probability for that script. (One disadvantage is that if you have script whose names contain @, you are now forced to specify "@1" at the end; otherwise the name's @ is confused with a weight separator. We don't expect many pgbench script with @ in their names in the wild, so this shouldn't be too serious a problem.) While at it, rework the interface between addScript, process_file, process_builtin, and findBuiltin. It had gotten a bit out of hand with recent commits. Author: Fabien Coelho Reviewed-By: Andres Freund, Robert Haas, Álvaro Herrera, Michaël Paquier Discussion: http://www.postgresql.org/message-id/alpine.DEB.2.10.1603160721240.1666@sto
* With ancient gcc, skip pg_attribute_printf() on function pointer.Tom Lane2016-03-19
| | | | | | Buildfarm results show that the ability to attach pg_attribute_printf decoration to a function pointer appeared somewhere between gcc 2.95.3 and gcc 4.0.1. Guess that it was there in 4.0.
* Allow SSL server key file to have group read access if owned by rootPeter Eisentraut2016-03-19
| | | | | | | | | | | | | We used to require the server key file to have permissions 0600 or less for best security. But some systems (such as Debian) have certificate and key files managed by the operating system that can be shared with other services. In those cases, the "postgres" user is made a member of a special group that has access to those files, and the server key file has permissions 0640. To accommodate that kind of setup, also allow the key file to have permissions 0640 but only if owned by root. From: Christoph Berg <myon@debian.org> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
* Fix stupid omission in c4901a1e.Andres Freund2016-03-18
| | | | | Reported-By: Jeff Janes Discussion: CAMkU=1zGxREwoyaCrp_CHadEB+dPgpVyKBysCJ+6xP9gCOvAuw@mail.gmail.com
* Fix missed update in _readForeignScan().Tom Lane2016-03-19
| | | | | Blatant fail in 0bf3ae88af330496517722e391e7c975e6bad219. Caught by buildfarm member mandrill.
* Use yylex_init not yylex_init_extra().Tom Lane2016-03-19
| | | | Older versions of flex don't have the latter. Per buildfarm.
* Suppress FLEX_NO_BACKUP check for psqlscanslash.l.Tom Lane2016-03-19
| | | | | | | | | The existing infrastructure for FLEX_NO_BACKUP doesn't work reliably when two lexers are built in parallel in the same directory. We can probably fix that, but as a short-term workaround, just don't make the check for psqlscanslash.l. Per buildfarm.
* Split psql's lexer into two separate .l files for SQL and backslash cases.Tom Lane2016-03-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This gets us to a point where psqlscan.l can be used by other frontend programs for the same purpose psql uses it for, ie to detect when it's collected a complete SQL command from input that is divided across line boundaries. Moreover, other programs can supply their own lexers for backslash commands of their own choosing. A follow-on patch will use this in pgbench. The end result here is roughly the same as in Kyotaro Horiguchi's 0001-Make-SQL-parser-part-of-psqlscan-independent-from-ps.patch, although the details of the method for switching between lexers are quite different. Basically, in this patch we share the entire PsqlScanState, YY_BUFFER_STATE stack, *and* yyscan_t between different lexers. The only thing we need to do to switch to a different lexer is to make sure the start_state is valid for the new lexer. This works because flex doesn't keep any other persistent state that depends on the specific lexing tables generated for a particular .l file. (We are assuming that both lexers are built with the same flex version, or at least versions that are compatible with respect to the contents of yyscan_t; but that doesn't seem likely to be a big problem in practice, considering how slowly flex changes.) Aside from being more efficient than Horiguchi-san's original solution, this avoids possible corner-case changes in semantics: the original code was capable of popping the input buffer stack while still staying in backslash-related parsing states. I'm not sure that that equates to any useful user-visible behaviors, but I'm not sure it doesn't either, so I'm loath to assume that we only need to consider the topmost buffer when parsing a backslash command. I've attempted to update the MSVC build scripts for the added .l file, but will rely on the buildfarm to see if I missed anything. Kyotaro Horiguchi and Tom Lane
* Convert psql's flex lexer to be re-entrant, and make it compile standalone.Tom Lane2016-03-18
| | | | | | | | | | | | | | | | | | | | | | | | | Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
* Merge wal_level "archive" and "hot_standby" into new name "replica"Peter Eisentraut2016-03-18
| | | | | | | | | | | | | | | | | The distinction between "archive" and "hot_standby" existed only because at the time "hot_standby" was added, there was some uncertainty about stability. This is now a long time ago. We would like to move forward with simplifying the replication configuration, but this distinction is in the way, because a primary server cannot tell (without asking a standby or predicting the future) which one of these would be the appropriate level. Pick a new name for the combined setting to make it clearer that it covers all (non-logical) backup and replication uses. The old values are still accepted but are converted internally. Reviewed-by: Michael Paquier <michael.paquier@gmail.com> Reviewed-by: David Steele <david@pgmasters.net>
* Decouple psqlscan.l from surrounding program.Tom Lane2016-03-18
| | | | | | | | | | | | | | | | | | | Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
* Use INT64_FORMAT instead of %ld for int64.Robert Haas2016-03-18
| | | | | | | | Commit 0011c0091e886b874e485a46ff2c94222ffbf550 introduced this mistake. Patch by me. Reported by Andres Freund, who also reviewed the patch.
* Only clear latch self-pipe/event if there is a pending notification.Andres Freund2016-03-18
| | | | | | | | | | | | | | | | This avoids a good number of, individually quite fast, system calls in scenarios with many quick queries. Besides the aesthetic benefit of seing fewer superflous system calls with strace, it also improves performance by ~2% measured by pgbench -M prepared -c 96 -j 8 -S (scale 100). Without having benchmarked it, this patch also adjust the windows code, as that makes it easier to unify the unix/windows codepaths in a later patch. There's little reason to diverge in behaviour between the platforms. Discussion: CA+TgmoYc1Zm+Szoc_Qbzi92z2c1vRHZmjhfPn5uC=w8bXv6Avg@mail.gmail.com Reviewed-By: Robert Haas
* Make it easier to choose the used waiting primitive in unix_latch.c.Andres Freund2016-03-18
| | | | | | | | This allows for easier testing of the different primitives; in preparation for adding a new primitive. Discussion: 20160114143931.GG10941@awork2.anarazel.de Reviewed-By: Robert Haas
* Error out if waiting on socket readiness without a specified socket.Andres Freund2016-03-18
| | | | | | | | | Previously we just ignored such an attempt, but that seems to serve no purpose but making things harder to debug. Discussion: 20160114143931.GG10941@awork2.anarazel.de 20151230173734.hx7jj2fnwyljfqek@alap3.anarazel.de Reviewed-By: Robert Haas
* Remove unused, and dangerous, TestLatch() macro.Andres Freund2016-03-18
| | | | | The macro has not seen any in-tree use since latches had been introduced in 2746e5f, in 2010.
* Directly modify foreign tables.Robert Haas2016-03-18
| | | | | | | | | postgres_fdw can now sent an UPDATE or DELETE statement directly to the foreign server in simple cases, rather than sending a SELECT FOR UPDATE statement and then updating or deleting rows one-by-one. Etsuro Fujita, reviewed by Rushabh Lathia, Shigeru Hanada, Kyotaro Horiguchi, Albe Laurenz, Thom Brown, and me.
* Clean up some misplaced #includes.Tom Lane2016-03-18
| | | | | | | Random .h files have no business including postgres-fe.h (or postgres.h). If that wasn't the first #include done by the calling .c file, it's the .c file that's broken. Noted while prepping Kyotaro Horiguchi's psql lexer refactoring patch.
* Introduce parse_ident()Teodor Sigaev2016-03-18
| | | | | | SQL-layer function to split qualified identifier into array parts. Author: Pavel Stehule with minor editorization by me and Jim Nasby
* Push scan/join target list beneath Gather when possible.Robert Haas2016-03-18
| | | | | | | | | | | | | | | | | This means that, for example, "SELECT expensive_func(a) FROM bigtab WHERE something" can compute expensive_func(a) in the workers rather than the leader if it happens to be parallel-safe, which figures to be a big win in some practical cases. Currently, we can only do this if the entire target list is parallel-safe. If we worked harder, we might be able to evaluate parallel-safe targets in the worker and any parallel-restricted targets in the leader, but that would be more complicated, and there aren't that many parallel-restricted functions that people are likely to use in queries anyway. I think. So just do the simple thing for the moment. Robert Haas, Amit Kapila, and Tom Lane
* Various minor corrections of and improvements to comments.Robert Haas2016-03-18
| | | | Aleksander Alekseev
* Remove useless double calls of make_parsestate().Tom Lane2016-03-17
| | | | Aleksander Alekseev
* Update tuplesort.c comments for memory mangement improvements.Robert Haas2016-03-17
| | | | | | | I'm committing these changes separately so that it's clear what is Peter's original work versus what I changed. This is a followup to commit 0011c0091e886b874e485a46ff2c94222ffbf550, and these changes are all by me.
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-20 12:04:46 +0000