aboutsummaryrefslogtreecommitdiff
path: root/src/backend/storage/buffer/bufmgr.c
Commit message (Collapse)AuthorAge
...
* Implement "distributed" checkpoints in which the checkpoint I/O is spreadTom Lane2007-06-28
| | | | | | | | | | | | | over a fairly long period of time, rather than being spat out in a burst. This happens only for background checkpoints carried out by the bgwriter; other cases, such as a shutdown checkpoint, are still done at full speed. Remove the "all buffers" scan in the bgwriter, and associated stats infrastructure, since this seems no longer very useful when the checkpoint itself is properly throttled. Original patch by Itagaki Takahiro, reworked by Heikki Linnakangas, and some minor API editorialization by me.
* Update obsolete comment: it's no longer the case that mdread() will allowTom Lane2007-06-18
| | | | reads beyond EOF, except by special coercion.
* Make large sequential scans and VACUUMs work in a limited-size "ring" ofTom Lane2007-05-30
| | | | | | | | | | | | | | | | | | | | | | | buffers, rather than blowing out the whole shared-buffer arena. Aside from avoiding cache spoliation, this fixes the problem that VACUUM formerly tended to cause a WAL flush for every page it modified, because we had it hacked to use only a single buffer. Those flushes will now occur only once per ring-ful. The exact ring size, and the threshold for seqscans to switch into the ring usage pattern, remain under debate; but the infrastructure seems done. The key bit of infrastructure is a new optional BufferAccessStrategy object that can be passed to ReadBuffer operations; this replaces the former StrategyHintVacuum API. This patch also changes the buffer usage-count methodology a bit: we now advance usage_count when first pinning a buffer, rather than when last unpinning it. To preserve the behavior that a buffer's lifetime starts to decrease when it's released, the clock sweep code is modified to not decrement usage_count of pinned buffers. Work not done in this commit: teach GiST and GIN indexes to use the vacuum BufferAccessStrategy for vacuum-driven fetches. Original patch by Simon, reworked by Heikki and again by Tom.
* Fix up pgstats counting of live and dead tuples to recognize that committedTom Lane2007-05-27
| | | | | | | | | | | and aborted transactions have different effects; also teach it not to assume that prepared transactions are always committed. Along the way, simplify the pgstats API by tying counting directly to Relations; I cannot detect any redeeming social value in having stats pointers in HeapScanDesc and IndexScanDesc structures. And fix a few corner cases in which counts might be missed because the relation's pgstat_info pointer hadn't been set.
* Dept. of second thoughts: add comments cautioning against usingTom Lane2007-05-02
| | | | | | | ReadOrZeroBuffer to fetch pages from beyond physical EOF. This would usually work, but would cause problems for md.c if writes occurred beyond a segment boundary when the previous segment file hadn't been fully extended.
* During WAL recovery, when reading a page that we intend to overwrite completelyTom Lane2007-05-02
| | | | | | | | | | | | from the WAL data, don't bother to physically read it; just have bufmgr.c return a zeroed-out buffer instead. This speeds recovery significantly, and also avoids unnecessary failures when a page-to-be-overwritten has corrupt page headers on disk. This replaces a former kluge that accomplished the latter by pretending zero_damaged_pages was always ON during WAL recovery; which was OK when the kluge was put in, but is unsafe when restoring a WAL log that was written with full_page_writes off. Heikki Linnakangas
* Add some instrumentation to the bgwriter, through the stats collector.Magnus Hagander2007-03-30
| | | | New view pg_stat_bgwriter, and the functions required to build it.
* Wording cleanup for error messages. Also change can't -> cannot.Bruce Momjian2007-02-01
| | | | | | | | | | | | | | Standard English uses "may", "can", and "might" in different ways: may - permission, "You may borrow my rake." can - ability, "I can lift that log." might - possibility, "It might rain today." Unfortunately, in conversational English, their use is often mixed, as in, "You may use this variable to do X", when in fact, "can" is a better choice. Similarly, "It may crash" is better stated, "It might crash".
* Update CVS HEAD for 2007 copyright. Back branches are typically notBruce Momjian2007-01-05
| | | | back-stamped for this.
* Remove an unnecessary HOLD_INTERRUPTS/RESUME_INTERRUPTS pair.Tom Lane2006-10-22
| | | | | This was required back when RESUME_INTERRUPTS could actually execute ProcessInterrupts, but that hasn't been true since 2001...
* pgindent run for 8.2.Bruce Momjian2006-10-04
|
* Add a check to prevent overwriting valid data if smgrnblocks() gives aTom Lane2006-09-25
| | | | | | wrong answer, as has been seen to occur with a buggy Linux kernel. Not really our bug, but it's a simple test in a seldom-used control path, so might as well have a defense.
* Marginal cleanup in arrangements for ensuring StrategyHintVacuum is clearedTom Lane2006-09-17
| | | | | | | | | after an error during VACUUM. We have a PG_TRY block anyway around the only call sites, so just reset it in the CATCH clause instead of having AtEOXact_Buffers blindly do it during xact end. I think the old code was actively wrong for the case of a failure during ANALYZE inside a subtransaction --- the flag wouldn't get cleared until main transaction end. Probably not worth back-patching though.
* Split the buffer mapping table into multiple separately lockableTom Lane2006-07-23
| | | | | partitions, as per discussion. Passes functionality checks, but I don't have any performance data yet.
* Remove 576 references of include files that were not needed.Bruce Momjian2006-07-14
|
* Repair a low-probability race condition identified by Qingqing Zhou.Tom Lane2006-04-14
| | | | | | | | | | | | If a process abandons a wait in LockBufferForCleanup (in practice, only happens if someone cancels a VACUUM) just before someone else sends it a signal indicating the buffer is available, it was possible for the wakeup to remain in the process' semaphore, causing misbehavior next time the process waited for an lmgr lock. Rather than try to prevent the race condition directly, it seems best to make the lock manager robust against leftover wakeups, by having it repeat waiting on the semaphore if the lock has not actually been granted or denied yet.
* Clean up WAL/buffer interactions as per my recent proposal. Get rid of theTom Lane2006-03-31
| | | | | | | | | | | | | | | | misleadingly-named WriteBuffer routine, and instead require routines that change buffer pages to call MarkBufferDirty (which does exactly what it says). We also require that they do so before calling XLogInsert; this takes care of the synchronization requirement documented in SyncOneBuffer. Note that because bufmgr takes the buffer content lock (in shared mode) while writing out any buffer, it doesn't matter whether MarkBufferDirty is executed before the buffer content change is complete, so long as the content change is completed before releasing exclusive lock on the buffer. So it's OK to set the dirtybit before we fill in the LSN. This eliminates the former kluge of needing to set the dirtybit in LockBuffer. Aside from making the code more transparent, we can also add some new debugging assertions, in particular that the caller of MarkBufferDirty must hold the buffer content lock, not merely a pin.
* Clean up and document the API for XLogOpenRelation and XLogReadBuffer.Tom Lane2006-03-29
| | | | | | | | This commit doesn't make much functional change, but it does eliminate some duplicated code --- for instance, PageIsNew tests are now done inside XLogReadBuffer rather than by each caller. The GIST xlog code still needs a lot of love, but I'll worry about that separately.
* Update copyright for 2006. Update scripts.Bruce Momjian2006-03-05
|
* Arrange to call AbsorbFsyncRequests every so often while performing aTom Lane2006-03-03
| | | | | | | checkpoint in the bgwriter. This forestalls overflow of the fsync request queue, which is not fatal but causes considerable performance degradation when it occurs (because backends then have to do their own fsyncs). Per patch from Itagaki Takahiro, modified a little bit by me.
* Fix ReadBuffer() to correctly handle the case where it's trying to extendTom Lane2006-01-06
| | | | | | | | | the relation but it finds a pre-existing valid buffer. The buffer does not correspond to any page known to the kernel, so we *must* do smgrextend to ensure that the space becomes allocated. The 7.x branches all do this correctly, but the corner case got lost somewhere during 8.0 bufmgr rewrites. (My fault no doubt :-( ... I think I assumed that such a buffer must be not-BM_VALID, which is not so.)
* Get rid of the SpinLockAcquire/SpinLockAcquire_NoHoldoff distinctionTom Lane2005-12-29
| | | | | | | | | | in favor of having just one set of macros that don't do HOLD/RESUME_INTERRUPTS (hence, these correspond to the old SpinLockAcquire_NoHoldoff case). Given our coding rules for spinlock use, there is no reason to allow CHECK_FOR_INTERRUPTS to be done while holding a spinlock, and also there is no situation where ImmediateInterruptOK will be true while holding a spinlock. Therefore doing HOLD/RESUME_INTERRUPTS while taking/releasing a spinlock is just a waste of cycles. Qingqing Zhou and Tom Lane.
* Re-run pgindent, fixing a problem where comment lines after a blankBruce Momjian2005-11-22
| | | | | | | | | comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
* DropRelFileNodeBuffers failed to fix the state of the lookup hash tableTom Lane2005-11-17
| | | | | | | | that was added to localbuf.c in 8.1; therefore, applying it to a temp table left corrupt lookup state in memory. The only case where this had a significant chance of causing problems was an ON COMMIT DELETE ROWS temp table; the other possible paths left bogus state that was unlikely to be used again. Per report from Csaba Nagy.
* Tweak buffer manager so that 'internal' accesses to a buffer do notTom Lane2005-10-27
| | | | | | | | advance its usage_count. This includes writes of dirty buffers triggered by bgwriter, checkpoint, or FlushRelationBuffers, as well as various corner cases that really ought not count as accesses to the page. Should make for some marginal improvement in the quality of our decisions about when to recycle buffers. Per suggestion from ITAGAKI Takahiro.
* Standard pgindent run for 8.1.Bruce Momjian2005-10-15
|
* Do all accesses to shared buffer headers through volatile-qualifiedTom Lane2005-10-12
| | | | | | | pointers, to ensure that compilers won't rearrange accesses to occur while we're not holding the buffer header spinlock. It's probably not necessary to mark volatile in every single place in bufmgr.c, but better safe than sorry. Per trouble report from Kevin Grittner.
* Reverse out Assert addition.Bruce Momjian2005-08-12
|
* Improve documention on loading large data sets into plperl.Bruce Momjian2005-08-12
| | | | David Fetter
* Remove BufferBlockPointers array in favor of a base + (bufnum) * BLCKSZTom Lane2005-08-12
| | | | | | | | computation. On modern machines this is as fast if not faster, and we don't have to clog the CPU's L2 cache with a tens-of-KB pointer array. If we ever decide to adopt a more dynamic allocation method for shared buffers, we'll probably have to revert this patch, but in the meantime we might as well save a few bytes and nanoseconds. Per Qingqing Zhou.
* Avoid useless loop overhead in AtEOXact routines when the backend isTom Lane2005-08-08
| | | | compiled with USE_ASSERT_CHECKING but is running with assert_enabled false.
* Cause ShutdownPostgres to do a normal transaction abort during backendTom Lane2005-08-08
| | | | | | | | exit, instead of trying to take shortcuts. Introduce some additional shutdown callback routines to eliminate kluges like having ProcKill be responsible for shutting down the buffer manager. Ensure that the order of operations during shutdown is predictable and what you would expect given the module layering.
* Tweak BgBufferSync() so that a persistent write error on a dirty bufferTom Lane2005-08-02
| | | | | | | doesn't block the bgwriter from making progress writing out other buffers. This was a hard problem in the context of the ARC/2Q design, but it's trivial in the context of clock sweep ... just advance the sweep counter before we try to write not after.
* Split the shared-memory array of PGPROC pointers out of the sinvalTom Lane2005-05-19
| | | | | | communication structure, and make it its own module with its own lock. This should reduce contention at least a little, and it definitely makes the code seem cleaner. Per my recent proposal.
* Remove unnecessary calls of FlushRelationBuffers: there is no needTom Lane2005-03-20
| | | | | | | | | | to write out data that we are about to tell the filesystem to drop. smgr_internal_unlink already had a DropRelFileNodeBuffers call to get rid of dead buffers without a write after it's no longer possible to roll back the deleting transaction. Adding a similar call in smgrtruncate simplifies callers and makes the overall division of labor clearer. This patch removes the former behavior that VACUUM would write all dirty buffers of a relation unconditionally.
* Avoid infinite loop in InvalidateBuffer if we ourselves are holdingTom Lane2005-03-18
| | | | a pin on the victim buffer.
* Replace the BufMgrLock with separate locks on the lookup hashtable andTom Lane2005-03-04
| | | | | | | | the freelist, plus per-buffer spinlocks that protect access to individual shared buffer headers. This requires abandoning a global freelist (since the freelist is a global contention point), which shoots down ARC and 2Q as well as plain LRU management. Adopt a clock sweep algorithm instead. Preliminary results show substantial improvement in multi-backend situations.
* Phase 1 of fix for 'SMgrRelation hashtable corrupted' problem. ThisTom Lane2005-01-10
| | | | | | is the minimum required fix. I want to look next at taking advantage of it by simplifying the message semantics in the shared inval message queue, but that part can be held over for 8.1 if it turns out too ugly.
* Repair bufmgr deadlock problem reported by Michael Wildpaner. Must takeTom Lane2005-01-03
| | | | | | | | | | | share lock on a buffer being written out before releasing BufMgrLock in the BufferAlloc code path; if we do it later we might block on someone who's re-pinned the buffer. I believe this is only an issue for BufferAlloc and not the other places that call FlushBuffer. BufferSync must continue to do it the old way since it may well be trying to write buffers that other backends have pinned; but it should not be holding any conflicting locks. FlushRelationBuffers is okay since it's got exclusive lock at the relation level.
* Tag appropriate files for rc3PostgreSQL Daemon2004-12-31
| | | | | | | | Also performed an initial run through of upgrading our Copyright date to extend to 2005 ... first run here was very simple ... change everything where: grep 1996-2004 && the word 'Copyright' ... scanned through the generated list with 'less' first, and after, to make sure that I only picked up the right entries ...
* Assert that BufferIsPinned() in IncrBufferRefCount(), rather than usingNeil Conway2004-11-24
| | | | | a home-brewed combination of assertions that boiled down to the same thing.
* Allow background writing to be shut down by setting limit values to zero.Tom Lane2004-10-17
| | | | | | | This does not disable the bgwriter process: it still has to wake up often enough to collect fsync requests from backends in a timely fashion. But it responds to the recent gripe about not being able to prevent the disk from being spun up constantly.
* Give the ResourceOwner mechanism full responsibility for releasing bufferTom Lane2004-10-16
| | | | | | | | pins at end of transaction, and reduce AtEOXact_Buffers to an Assert cross-check that this was done correctly. When not USE_ASSERT_CHECKING, AtEOXact_Buffers is a complete no-op. This gets rid of an O(NBuffers) bottleneck during transaction commit/abort, which recent testing has shown becomes significant above a few tens of thousands of shared buffers.
* Remove BufferLocks[] array in favor of a single pointer to the bufferTom Lane2004-10-16
| | | | | | (if any) currently waited for by LockBufferForCleanup(), which is all that we were using it for anymore. Saves some space and eliminates proportional-to-NBuffers slowdown in UnlockBuffers().
* Repair possible failure to update hint bits back to disk, perTom Lane2004-10-15
| | | | | | | | | | http://archives.postgresql.org/pgsql-hackers/2004-10/msg00464.php. This fix is intended to be permanent: it moves the responsibility for calling SetBufferCommitInfoNeedsSave() into the tqual.c routines, eliminating the requirement for callers to test whether t_infomask changed. Also, tighten validity checking on buffer IDs in bufmgr.c --- several routines were paranoid about out-of-range shared buffer numbers but not about out-of-range local ones, which seems a tad pointless.
* I can't see any good reason for DropRelFileNodeBuffers to be issuingTom Lane2004-09-06
| | | | FATAL when it detects a nonzero reference count. Reduce to ERROR.
* FlushRelationBuffers was also being a bit cavalier about whether theTom Lane2004-08-31
| | | | relation is already opened by smgr.
* Pgindent run for 8.0.Bruce Momjian2004-08-29
|
* Update copyright to 2004.Bruce Momjian2004-08-29
|
* Invent ResourceOwner mechanism as per my recent proposal, and use it toTom Lane2004-07-17
| | | | | | | | keep track of portal-related resources separately from transaction-related resources. This allows cursors to work in a somewhat sane fashion with nested transactions. For now, cursor behavior is non-subtransactional, that is a cursor's state does not roll back if you abort a subtransaction that fetched from the cursor. We might want to change that later.
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-25 21:29:10 +0000