PostgreSQL La base de donnees la plus sophistiquee au monde.

La planete francophone de PostgreSQL

mercredi 7 décembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 4 décembre 2016

[ndt: MeetUp en préparation pour le 20 décembre à Lyon : http://www.meetup.com/fr-FR/PostgreSQL-Lyon-User-Group/]

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en décembre

PostgreSQL Local

  • CHAR(16) aura lieu à New York le 6 décembre 2016 : http://charconference.org/
  • Le PGDay.IT 2016 aura lieu à Prato le 13 décembre 2016 : http://pgday.it
  • La PGConf India 2017 aura lieu les 2 & 3 mars 2017 à Bengalore (État du Karnataka en Inde). Les propositions sont attendues sur <papers AT pgconf DOT in> jusqu'au 31 décembre 2016 : http://pgconf.in/
  • PostgreSQL@SCaLE aura lieu les 2 & 3 mars 2017 au centre des conventions de Pasadena, comme partie du SCaLE 15X : http://www.socallinuxexpo.org/scale/15x/

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161205004347.GA329@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Tom Lane pushed:

  • Code review for early drop of orphaned temp relations in autovacuum. Commit a734fd5d1 exposed some race conditions that existed previously in the autovac code, but were basically harmless because autovac would not try to delete orphaned relations immediately. Specifically, the test for orphaned-ness was made on a pg_class tuple that might be dead by now, allowing autovac to try to remove a table that the owning backend had just finished deleting. This resulted in a hard crash due to inadequate caution about accessing the table's catalog entries without any lock. We must take a relation lock and then recheck whether the table is still present and still looks deletable before we do anything. Also, it seemed to me that deleting multiple tables per transaction, and trying to continue after errors, represented unjustifiable complexity. We do not expect this code path to be taken often in the field, nor even during testing, which means that prioritizing performance over correctness is a bad tradeoff. Rip all that out in favor of just starting a new transaction after each successful temp table deletion. If we're unlucky enough to get an error, which shouldn't happen anyway now that we're being more cautious, let the autovacuum worker fail as it normally would. In passing, improve the order of operations in the initial scan loop. Now that we don't care about whether a temp table is a wraparound hazard, there's no need to perform extract_autovac_opts, get_pgstat_tabentry_relid, or relation_needs_vacanalyze for temp tables. Also, if GetTempNamespaceBackendId returns InvalidBackendId (indicating it doesn't recognize the schema as temp), treat that as meaning it's NOT an orphaned temp table, not that it IS one, which is what happened before because BackendIdGetProc necessarily failed. The case really shouldn't come up for a table that has RELPERSISTENCE_TEMP, but the consequences if it did seem undesirable. (This might represent a back-patchable bug fix; not sure if it's worth the trouble.) Discussion: https://postgr.es/m/21299.1480272347@sss.pgh.pa.us http://git.postgresql.org/pg/commitdiff/dafa0848da11260e9510c699e7060171336cb550
  • Fix busted tab-completion pattern for ALTER TABLE t ALTER c DROP ... Evidently a thinko in commit 9b181b036. Kyotaro Horiguchi http://git.postgresql.org/pg/commitdiff/404e667580fec0ab9a89a38eaa8643ee7227fdb8
  • Fix estimate_expression_value to constant-fold SQLValueFunction nodes. Oversight in my commit 0bb51aa96. Noted while poking at a recent bug report --- HEAD's estimates for a query using CURRENT_DATE were unexpectedly much worse than 9.6's. http://git.postgresql.org/pg/commitdiff/4e20511d5b8ba427730e09be45f9458f667f9c1e
  • Fix incorrect variable type in set_rel_consider_parallel(). func_parallel() returns char not Oid. Harmless, but still wrong. Amit Langote http://git.postgresql.org/pg/commitdiff/d6c8b34e956864d52780f0db9f8cfe3b2f4411b0
  • Add uuid to the set of types supported by contrib/btree_gist. Paul Jungwirth, reviewed and hacked on by Teodor Sigaev, Ildus Kurbangaliev, Adam Brusselback, Chris Bandy, and myself. Discussion: https://postgr.es/m/CA+renyUEE29=X01JXdz8_TQvo6n9=2XoEBBRnQ8rkLyr+kjPxQ@mail.gmail.com Discussion: https://postgr.es/m/55F6EE82.8080209@sigaev.ru http://git.postgresql.org/pg/commitdiff/11da83a0e70d32ed0e06a5c948cd8343f8ad5102
  • Test all contrib-created operator classes with amvalidate. I'd supposed that people would do this manually when creating new operator classes, but the folly of that was exposed today. The tests seem fast enough that we can just apply them during the normal regression tests. contrib/isn fails the checks for lack of complete sets of cross-type operators. That's a nice-to-have policy rather than a functional requirement, so leave it as-is, but insert ORDER BY in the query to ensure consistent cross-platform output. Discussion: https://postgr.es/m/7076.1480446837@sss.pgh.pa.us http://git.postgresql.org/pg/commitdiff/ade49c605f1d8f517497829363f8b83f69c73a60
  • Improve eqjoinsel_semi's behavior for small inner relations with no stats. If we don't have any MCV statistics for the inner relation, and we don't trust its numdistinct estimate either, eqjoinsel_semi falls back to a very conservative estimate (that 50% of the outer rows have matches). This is particularly problematic if the inner relation is completely empty, since then even an explicit ANALYZE won't produce any pg_statistic entries, so there's no way to budge the planner off the bad estimate. We'd produce a better estimate in such cases if we used the nd2/nd1 selectivity heuristic, so an easy fix is to treat the nd2 estimate as non-default if we derive it from clamping to the inner rel's rowcount estimate. This won't fix every related case (mainly because the rowcount estimate might be larger than DEFAULT_NUM_DISTINCT), but it seems like a sane extension of the existing logic, so let's apply the change in HEAD and see if anyone complains. Per bug #14438 from Nikolay Nikitin. Report: https://postgr.es/m/20161128182113.6527.58926@wrigleys.postgresql.org Discussion: https://postgr.es/m/31089.1480384713@sss.pgh.pa.us http://git.postgresql.org/pg/commitdiff/ca5f88502096a39f831da74eb02ec9eb3310616d
  • Fix bogus handling of JOIN_UNIQUE_OUTER/INNER cases for parallel joins. consider_parallel_nestloop passed the wrong jointype down to its subroutines for JOIN_UNIQUE_INNER cases (it should pass JOIN_INNER), and it thought that it could pass paths other than innerrel->cheapest_total_path to create_unique_path, which create_unique_path is not on board with. These bugs would lead to assertion failures or other errors, suggesting that this code path hasn't been tested much. hash_inner_and_outer's code for parallel join effectively treated both JOIN_UNIQUE_OUTER and JOIN_UNIQUE_INNER the same as JOIN_INNER (for different reasons :-(), leading to incorrect plans that treated a semijoin as if it were a plain join. Michael Day submitted a test case demonstrating that hash_inner_and_outer failed for JOIN_UNIQUE_OUTER, and I found the other cases through code review. Report: https://postgr.es/m/D0E8A029-D1AC-42E8-979A-5DE4A77E4413@rcmail.com http://git.postgresql.org/pg/commitdiff/41e2b84ce1b89ae0125dd34318c56aa51386e2a2
  • Doc: improve description of trim() and related functions. Per bug #14441 from Mark Pether, the documentation could be misread, mainly because some of the examples failed to show what happens with a multicharacter "characters to trim" string. Also, while the text description in most of these entries was fairly clear that the "characters" argument is a set of characters not a substring to match, some of them used variant wording that was a bit less clear. trim() itself suffered from both deficiencies and was thus pretty misinterpretable. Also fix failure to explain which of LEADING/TRAILING/BOTH is the default. Discussion: https://postgr.es/m/20161130011710.6539.53657@wrigleys.postgresql.org http://git.postgresql.org/pg/commitdiff/213c0f2d7880f78c710127920cf4bf7017e0fa57
  • Delete deleteWhatDependsOn() in favor of more performDeletion() flag bits. deleteWhatDependsOn() had grown an uncomfortably large number of assumptions about what it's used for. There are actually only two minor differences between what it does and what a regular performDeletion() call can do, so let's invent additional bits in performDeletion's existing flags argument that specify those behaviors, and get rid of deleteWhatDependsOn() as such. (We'd probably have done it this way from the start, except that performDeletion didn't originally have a flags argument, IIRC.) Also, add a SKIP_EXTENSIONS flag bit that prevents ever recursing to an extension, and use that when dropping temporary objects at session end. This provides a more general solution to the problem addressed in a hacky way in commit 08dd23cec: if an extension script creates temp objects and forgets to remove them again, the whole extension went away when its contained temp objects were deleted. The previous solution only covered temp relations, but this solves it for all object types. These changes require minor additions in dependency.c to pass the flags to subroutines that previously didn't get them, but it's still a net savings of code, and it seems cleaner than before. Having done this, revert the special-case code added in 08dd23cec that prevented addition of pg_depend records for temp table extension membership, because that caused its own oddities: dropping an extension that had created such a table didn't automatically remove the table, leading to a failure if the table had another dependency on the extension (such as use of an extension data type), or to a duplicate-name failure if you then tried to recreate the extension. But we keep the part that prevents the pg_temp_nnn schema from becoming an extension member; we never want that to happen. Add a regression test case covering these behaviors. Although this fixes some arguable bugs, we've heard few field complaints, and any such problems are easily worked around by explicitly dropping temp objects at the end of extension scripts (which seems like good practice anyway). So I won't risk a back-patch. Discussion: https://postgr.es/m/e51f4311-f483-4dd0-1ccc-abec3c405110@BlueTreble.com http://git.postgresql.org/pg/commitdiff/b3427dade14cc31eb48740bc9ea98b5954470b24
  • Fix broken wait-for-previous-process-to-exit loop in regression test. Must do pg_stat_clear_snapshot() inside test's loop, or our snapshot of pg_stat_activity will never change :-(. Thinko in b3427dade -- evidently my workstation never really iterated the loop in testing. Per buildfarm. http://git.postgresql.org/pg/commitdiff/19fcc0058ecc8e5eb756547006bc1b24a93cbb80
  • Don't mess up pstate->p_next_resno in transformOnConflictClause(). transformOnConflictClause incremented p_next_resno while generating the phony targetlist for the EXCLUDED pseudo-rel. Then that field got incremented some more during transformTargetList, possibly leading to free_parsestate concluding that we'd overrun the allowed length of a tlist, as reported by Justin Pryzby. We could fix this by resetting p_next_resno to 1 after using it for the EXCLUDED pseudo-rel tlist, but it seems easier and less coupled to other places if we just don't use that field at all in this loop. (Note that this doesn't change anything about the resnos that end up appearing in the main target list, because those are all replaced with target-column numbers by updateTargetListEntry.) In passing, fix incorrect type OID assigned to the whole-row Var for "EXCLUDED.*" (somehow this escaped having any bad consequences so far, but it's certainly wrong); remove useless assignment to var->location; pstrdup the column names in case of a relcache flush; and improve nearby comments. Back-patch to 9.5 where ON CONFLICT was introduced. Report: https://postgr.es/m/20161204163237.GA8030@telsasoft.com http://git.postgresql.org/pg/commitdiff/3850723208888a6fe90b75ebf692af79119f4158

Ãlvaro Herrera pushed:

  • Fix get_relation_info name typo'ed in a comment. Plus add a missing comment about this in get_relation_info itself. Author: Amit Langote. Discussion: https://postgr.es/m/e46c0569-0449-afa0-e2fe-f3776e4b3fd5@lab.ntt.co.jp http://git.postgresql.org/pg/commitdiff/eb6814168862ef0ff44716467aacdebcef87415a
  • Permit dump/reload of not-too-large >1GB tuples. Our documentation states that our maximum field size is 1 GB, and that our maximum row size of 1.6 TB. However, while this might be attainable in theory with enough contortions, it is not workable in practice; for starters, pg_dump fails to dump tables containing rows larger than 1 GB, even if individual columns are well below the limit; and even if one does manage to manufacture a dump file containing a row that large, the server refuses to load it anyway. This commit enables dumping and reloading of such tuples, provided two conditions are met: 1. no single column is larger than 1 GB (in output size -- for bytea this includes the formatting overhead) 2. the whole row is not larger than 2 GB There are three related changes to enable this: a. StringInfo's API now has two additional functions that allow creating a string that grows beyond the typical 1GB limit (and "long" string). ABI compatibility is maintained. We still limit these strings to 2 GB, though, for reasons explained below. b. COPY now uses long StringInfos, so that pg_dump doesn't choke trying to emit rows longer than 1GB. c. heap_form_tuple now uses the MCXT_ALLOW_HUGE flag in its allocation for the input tuple, which means that large tuples are accepted on input. Note that at this point we do not apply any further limit to the input tuple size. The main reason to limit to 2 GB is that the FE/BE protocol uses 32 bit length words to describe each row; and because the documentation is ambiguous on its signedness and libpq does consider it signed, we cannot use the highest-order bit. Additionally, the StringInfo API uses "int" (which is 4 bytes wide in most platforms) in many places, so we'd need to change that API too in order to improve, which has lots of fallout. Backpatch to 9.5, which is the oldest that has MemoryContextAllocExtended, a necessary piece of infrastructure. We could apply to 9.4 with very minimal additional effort, but any further than that would require backpatching "huge" allocations too. This is the largest set of changes we could find that can be back-patched without breaking compatibility with existing systems. Fixing a bigger set of problems (for example, dumping tuples bigger than 2GB, or dumping fields bigger than 1GB) would require changing the FE/BE protocol and/or changing the StringInfo API in an ABI-incompatible way, neither of which would be back-patchable. Authors: Daniel Vérité, Ãlvaro Herrera. Reviewed by: Tomas Vondra. Discussion: https://postgr.es/m/20160229183023.GA286012@alvherre.pgsql http://git.postgresql.org/pg/commitdiff/fa2fa995528023b2e6ba1108f2f47558c6b66dcd
  • Fix Windows build for 78c8c814390f. Author: Petr Jelínek http://git.postgresql.org/pg/commitdiff/5714931b075b2dc8994b6e464ea3845c33ecf416
  • Fix outdated comments Commit 597a87ccc9a6f neglected to update some comments; fix. Report and patch by Thomas Munro. Reviewed by Petr Jelínek. http://git.postgresql.org/pg/commitdiff/5e5986b6cbebcb57e6c95463031eef01e099e7e1

Stephen Frost pushed:

Robert Haas pushed:

  • libpq: Add target_session_attrs parameter. Commit 274bb2b3857cc987cfa21d14775cae9b0dababa5 made it possible to specify multiple IPs in a connection string, but that's not good enough for the case where you have a read-write master and a bunch of read-only standbys and want to connect to whichever server is the master at the current time. This commit allows that, by making it possible to specify target_session_attrs=read-write as a connection parameter. There was extensive discussion of the best name for the connection parameter and its values as well as the best way to distinguish master and standbys. For now, adopt the same solution as JDBC: if the user wants a read-write connection, issue 'show transaction_read_only' and rejection the connection if the result is 'on'. In the future, we could add additional values of this new target_session_attrs parameter that issue different queries; or we might have some way of distinguishing the server type without resorting to an SQL query; but right now, we have this, and that's (hopefully) a good start. Victor Wagner and Mithun Cy. Design review by Ãlvaro Herrera, Catalin Iacob, Takayuki Tsunakawa, and Craig Ringer; code review by me. I changed Mithun's patch to skip all remaining IPs for a host if we reject a connection based on this new parameter, rewrote the documentation, and did some other cosmetic cleanup. Discussion: http://postgr.es/m/CAD__OuhqPRGpcsfwPHz_PDqAGkoqS1UvnUnOnAB-LBWBW=wu4A@mail.gmail.com http://git.postgresql.org/pg/commitdiff/721f7bd3cbccaf8c07cad2707826b83f84694832
  • Improve hash index bucket split behavior. Previously, the right to split a bucket was represented by a heavyweight lock on the page number of the primary bucket page. Unfortunately, this meant that every scan needed to take a heavyweight lock on that bucket also, which was bad for concurrency. Instead, use a cleanup lock on the primary bucket page to indicate the right to begin a split, so that scans only need to retain a pin on that page, which is they would have to acquire anyway, and which is also much cheaper. In addition to reducing the locking cost, this also avoids locking out scans and inserts for the entire lifetime of the split: while the new bucket is being populated with copies of the appropriate tuples from the old bucket, scans and inserts can happen in parallel. There are minor concurrency improvements for vacuum operations as well, though the situation there is still far from ideal. This patch also removes the unworldly assumption that a split will never be interrupted. With the new code, a split is done in a series of small steps and the system can pick up where it left off if it is interrupted prior to completion. While this patch does not itself add write-ahead logging for hash indexes, it is clearly a necessary first step, since one of the things that could interrupt a split is the removal of electrical power from the machine performing it. Amit Kapila. I wrote the original design on which this patch is based, and did a good bit of work on the comments and README through multiple rounds of review, but all of the code is Amit's. Also reviewed by Jesper Pedersen, Jeff Janes, and others. Discussion: http://postgr.es/m/CAA4eK1LfzcZYxLoXS874Ad0+S-ZM60U9bwcyiUZx9mHZ-KCWhw@mail.gmail.com http://git.postgresql.org/pg/commitdiff/6d46f4783efe457f74816a75173eb23ed8930020
  • libpq: Fix inadvertent change in PQhost() behavior. Commit 274bb2b3857cc987cfa21d14775cae9b0dababa5 caused PQhost() to return the value of the hostaddr parameter rather than the relevant host when the latter parameter was specified. That's wrong. Commit 9a1d0af4ad2cbd419115b453d811c141b80d872b then amplified the damage by using PQhost() in more places, so that the SSL test suite started failing. Report by Andreas Karlsson; patch by me. http://git.postgresql.org/pg/commitdiff/11003eb55658df0caf183eef69c7a97d56a4f2d7
  • Add max_parallel_workers GUC. Increase the default value of the existing max_worker_processes GUC from 8 to 16, and add a new max_parallel_workers GUC with a maximum of 8. This way, even if the maximum amount of parallel query is happening, there is still room for background workers that do other things, as originally envisioned when max_worker_processes was added. Julien Rouhaud, reviewed by Amit Kapila and by revised by me. http://git.postgresql.org/pg/commitdiff/b460f5d6693103076dc554aa7cbb96e1e53074f9
  • Clarify that pg_stat_activity.query has a length limit. There was always documentation of the GUC that controlled what the limit actually was, but previously the documentation of the field itself made no mention of that limit. Ian Barwick http://git.postgresql.org/pg/commitdiff/e63d41498837667a4e2e0a4b9416bfda28c722d6
  • Add a crude facility for dealing with relative pointers. C doesn't have any sort of built-in understanding of a pointer relative to some arbitrary base address, but dynamic shared memory segments can be mapped at different addresses in different processes, so any sort of shared data structure stored within a dynamic shared memory segment can't use absolute pointers. We could use something like Size to represent a relative pointer, but then the compiler provides no type-checking. Use stupid macro tricks to get some type-checking. Patch originally by me. Concept suggested by Andres Freund. Recently resubmitted as part of Thomas Munro's work on dynamic shared memory allocation. Discussion: 20131205144434.GG12398@alap2.anarazel.de Discussion: CAEepm=1z5WLuNoJ80PaCvz6EtG9dN0j-KuHcHtU6QEfcPP5-qA@mail.gmail.com http://git.postgresql.org/pg/commitdiff/fbc1c12a94a638cf4f577fef158175e22ab824a3
  • Management of free memory pages. This is intended as infrastructure for a full-fledged allocator for dynamic shared memory. The interface looks a bit like a real allocator, but only supports allocating and freeing memory in multiples of the 4kB page size. Further, to free memory, you must know the size of the span you wish to free, in pages. While these are make it unsuitable as an allocator in and of itself, it still serves as very useful scaffolding for a full-fledged allocator. Robert Haas and Thomas Munro. This code is mostly the same as my 2014 submission, but Thomas fixed quite a few bugs and made some changes to the interface. Discussion: CA+TgmobkeWptGwiNa+SGFWsTLzTzD-CeLz0KcE-y6LFgoUus4A@mail.gmail.com Discussion: CAEepm=1z5WLuNoJ80PaCvz6EtG9dN0j-KuHcHtU6QEfcPP5-qA@mail.gmail.com http://git.postgresql.org/pg/commitdiff/13e14a78ea15f4c581cae408b5010c13961c96de
  • Introduce dynamic shared memory areas. Programmers discovered decades ago that it was useful to have a simple interface for allocating and freeing memory, which is why malloc() and free() were invented. Unfortunately, those handy tools don't work with dynamic shared memory segments because those are specific to PostgreSQL and are not necessarily mapped at the same address in every cooperating process. So invent our own allocator instead. This makes it possible for processes cooperating as part of parallel query execution to allocate and free chunks of memory without having to reserve them prior to the start of execution. It could also be used for longer lived objects; for example, we could consider storing data for pg_stat_statements or the stats collector in shared memory using these interfaces, rather than writing them to files. Basically, anything that needs shared memory but can't predict in advance how much it's going to need might find this useful. Thomas Munro and Robert Haas. The original code (of mine) on which Thomas based his work was actually designed to be a new backend-local memory allocator for PostgreSQL, but that hasn't gone anywhere - or not yet, anyway. Thomas took that work and performed major refactoring and extensive modifications to make it work with dynamic shared memory, including the addition of appropriate locking. Discussion: CA+TgmobkeWptGwiNa+SGFWsTLzTzD-CeLz0KcE-y6LFgoUus4A@mail.gmail.com Discussion: CAEepm=1z5WLuNoJ80PaCvz6EtG9dN0j-KuHcHtU6QEfcPP5-qA@mail.gmail.com http://git.postgresql.org/pg/commitdiff/13df76a537cca3b8884911d8fdf7c89a457a8dd3
  • Fix thinko in b3427dade14cc31eb48740bc9ea98b5954470b24. http://git.postgresql.org/pg/commitdiff/767a9039d79e35c78afa39cf5ffa3b485a2e3a5b

Peter Eisentraut pushed:

Heikki Linnakangas pushed:

Andres Freund pushed:

  • Perform one only projection to compute agg arguments. Previously we did a ExecProject() for each individual aggregate argument. That turned out to be a performance bottleneck in queries with multiple aggregates. Doing all the argument computations in one ExecProject() is quite a bit cheaper because ExecProject's fastpath can do the work at once in a relatively tight loop, and because it can get all the required columns with a single slot_getsomeattr and save some other redundant setup costs. Author: Andres Freund. Reviewed-By: Heikki Linnakangas. Discussion: https://postgr.es/m/20161103110721.h5i5t5saxfk5eeik@alap3.anarazel.de http://git.postgresql.org/pg/commitdiff/8ed3f11bb045ad7a3607690be668dbd5b3cc31d7
  • User narrower representative tuples in the hash-agg hashtable. So far the hashtable stored representative tuples in the form of its input slot, with all columns in the hashtable that are not needed (i.e. not grouped upon or functionally dependent) set to NULL. Thats good for saving memory, but it turns out that having tuples full of NULL isn't free. slot_deform_tuple is faster if there's no NULL bitmap even if no NULLs are encountered, and skipping over leading NULLs isn't free. So compute a separate tuple descriptor that only contains the needed columns. As columns have already been moved in/out the slot for the hashtable that does not imply additional per-row overhead. Author: Andres Freund. Reviewed-By: Heikki Linnakangas. Discussion: https://postgr.es/m/20161103110721.h5i5t5saxfk5eeik@alap3.anarazel.de http://git.postgresql.org/pg/commitdiff/fc4b3dea2950e4f6081f1ed2380f82c9efd672e0

Michael Meskes pushed:

Noah Misch pushed:

  • Remove wrong CloseHandle() call. In accordance with its own documentation, invoke CloseHandle() only when directed in the documentation for the function that furnished the handle. GetModuleHandle() does not so direct. We have been issuing this call only in the rare event that a CRT DLL contains no "_putenv" symbol, so lack of bug reports is uninformative. Back-patch to 9.2 (all supported versions). Christian Ullrich, reviewed by Michael Paquier. http://git.postgresql.org/pg/commitdiff/b37da1e8a0e46ae12415fafd3ea441fc3546cf2f
  • Refine win32env.c cosmetics. Replace use of plain 0 as a null pointer constant. In comments, update terminology and lessen redundancy. Back-patch to 9.2 (all supported versions) for the convenience of back-patching the next two commits. Christian Ullrich and Noah Misch, reviewed (in earlier versions) by Michael Paquier. http://git.postgresql.org/pg/commitdiff/a9d9208c15de4933f89e5b6ac1d9ef0efd299162
  • Make pgwin32_putenv() visit debug CRTs. This has no effect in the most conventional case, where no relevant DLL uses a debug build. For an example where it does matter, given a debug build of MIT Kerberos, the krb_server_keyfile parameter usually had no effect. Since nobody wants a Heisenbug, back-patch to 9.2 (all supported versions). Christian Ullrich, reviewed by Michael Paquier. http://git.postgresql.org/pg/commitdiff/95b9b8a3977f854e0bfb2a5614b699b7774ae673
  • Make pgwin32_putenv() follow DLL loading and unloading. Until now, the first putenv() call of a given postgres.exe process would cache the set of loaded CRTs. If a CRT unloaded after that call, the next putenv() would crash. That risk was largely theoretical, because the first putenv() precedes all PostgreSQL-initiated module loading. However, this might explain bad interactions with antivirus and other software that injects threads asynchronously. If an additional CRT loaded after the first putenv(), pgwin32_putenv() would not discover it. That CRT would have all environment changes predating its load, but it would not receive later PostgreSQL-initiated changes. An additional CRT loading concurrently with the first putenv() might miss that change in addition to missing later changes. Fix all those problems. This removes the cache mechanism from pgwin32_putenv(); the cost, less than 100 μs per backend startup, is negligible. No resulting misbehavior was known to be user-visible given the core distribution alone, but one can readily construct an affected extension module. No back-patch given the lack of complaints and the potential for behavior changes in non-PostgreSQL code running in the backend. Christian Ullrich, reviewed by Michael Paquier. http://git.postgresql.org/pg/commitdiff/202dbdbe41e1b1085a4d69c96bca9a52e634b196
  • Make pgwin32_putenv() probe every known CRT, regardless of compiler. This extends to MinGW builds the provision for MSVC-built libraries to see putenv() effects. Doing so repairs, for example, the handling of the krb_server_keyfile parameter when linked with MSVC-built MIT Kerberos. Like the previous commit, no back-patch. http://git.postgresql.org/pg/commitdiff/54aa6ccfc51414b94a2363be6302efb0f911b692
  • Document recipe for testing compatibility with old Perl. Craig Ringer, reviewed by Kyotaro HORIGUCHI and Michael Paquier. http://git.postgresql.org/pg/commitdiff/d61aa6ae655a37d757b68d20ad18a4683c280c14

Correctifs en attente

Pavel Stěhule sent in another revision of a patch to add session server side variables.

Kyotaro HORIGUCHI sent in another revision of a patch to refactor psql's tab completion code for more sanity and add some new tab completions based on this.

Etsuro Fujita sent in another revision of a patch to push down more full joins in the postgres_fdw.

Matheus de Oliveira sent in another revision of a patch to add GRANT/REVOKE ON SCHEMAS to ALTER DEFAULT PRIVILEGES.

Masahiko Sawada sent in another revision of a patch to add quorum commit for multiple synchronous replicas.

Julian Markwort sent in another revision of a patch to add a PGPASSFILE connection option to libpq.

Mithun Cy sent in another revision of a patch to add pg_autoprewarm.

Andrew Borodin sent in two revisions of a patch to make a more concurrency-friendly vacuum of GIN indexes.

Tom Lane sent in a patch to fix the order of tests in findDependentObjects().

Dilip Kumar sent in two more revisions of a patch to push scan keys down to the heap.

Petr Jelínek and Peter Eisentraut traded patches to add logical replication.

Haribabu Kommi sent in another revision of a patch to add a 64-bit (EUI-64) mac address type.

Stephen Frost sent in a patch to fix some compiler warnings.

Emre Hasegeli sent in another revision of a patch to add the customer contains/is contained by operators for the inet data type.

Dilip Kumar sent in another revision of a patch to add parallel bitmap heap scan.

Peter Eisentraut sent in a patch to move collation import to the backend.

Amit Langote sent in a doc patch to clarify a point in ALTER TABLE.

Simon Riggs sent in a patch to make EvalPlanQual() follow the LockTuple() pattern.

Heikki Linnakangas and Michaël Paquier traded patches to add secure random number generation.

Pavel StÄ›hule and Ãlvaro Herrera traded patches to add xmltable().

Etsuro Fujita sent in another revision of a patch to ensure that altering a foreign table invalidats prepared statement execution plans in which it is involved.

Amit Langote sent in another revision of a patch to implement declarative partitioning.

Amit Langote sent in two revisions of a patch to clarify the behavior of ALTER TABLE with parallelism enabled.

Peter Eisentraut sent in another revision of a patch to implement a pg_sequence catalog.

Tomas Vondra sent in another revision of a patch to add two slab-like memory allocators.

Ian Barwick sent in a doc patch to clarify the length of query text displayed in pg_stat_statements.

Stephen Frost sent in another revision of a patch to add support for restrictive RLS policies.

Ashutosh Bapat sent in another revision of a patch to implement partition-wise join for join between (declaratively) partitioned tables.

Thomas Munro sent in another revision of a patch to create a DSA area to provide work space for parallel execution.

Peter Eisentraut sent in another revision of a patch intended to support making DROP FUNCTION able to drop multiple functions per statement.

Mithun Cy sent in a patch to fix a bug in failover at the libpq connect level.

Michaël Paquier sent in another revision of a patch to fix checkpoint skip logic on idle systems by tracking LSN progress.

Amit Kapila sent in two more revisions of a patch to fix an infelicity between parallel execution and prepared statements.

Andreas Karlsson sent in another revision of a patch to reload SSL certificates on SIGHUP.

Pavan Deolasee sent in another revision of a patch to implement WARM.

Thomas Munro sent in a patch to provide the right format string for dsa_pointer to printf-like functions.

Amit Kapila sent in a patch to fix a failed assertion in _hash_splitbucket_guts.

Fabien COELHO sent in another revision of a patch to add backslash-return (well newline really) continuations to all pgbench backslash-commands.

Peter Geoghegan sent in another revision of a patch to add parallel tuplesort.

par N Bougain le mercredi 7 décembre 2016 à 21h56

mardi 29 novembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 27 novembre 2016

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en novembre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161127230258.GE21874@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Tom Lane pushed:

  • Fix test for subplans in force-parallel mode. We mustn't force parallel mode if the query has any subplans, since ExecSerializePlan doesn't transmit them to workers. Testing top_plan->initPlan is inadequate because (1) there might be initPlans attached to lower plan nodes, and (2) non-initPlan subplans don't work either. There's certainly room for improvement in those restrictions, but for the moment that's what we've got. Amit Kapila, per report from Andreas Seltenreich Discussion: <8737im6pmh.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/f24cf960d7ae3503e21fcb59dca652575619d9d4
  • Fix optimization for skipping searches for parallel-query hazards. Fix thinko in commit da1c91631: even if the original query was free of parallel hazards, we might introduce such a hazard by adding PARAM_EXEC Param nodes. Adjust is_parallel_safe() so that it will scan the given expression whenever any such nodes have been created. Per report from Andreas Seltenreich. Discussion: <878tse6yvf.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/4324ade9a6880113b08070305482ace2e8a2617c
  • Fix PGLC_localeconv() to handle errors better. The code was intentionally not very careful about leaking strdup'd strings in case of an error. That was forgivable probably, but it also failed to notice strdup() failures, which could lead to subsequent null-pointer-dereference crashes, since many callers unsurprisingly didn't check for null pointers in the struct lconv fields. An even worse problem is that it could throw error while we were setlocale'd to a non-C locale, causing unwanted behavior in subsequent libc calls. Rewrite to ensure that we cannot throw elog(ERROR) until after we've restored the previous locale settings, or at least attempted to. (I'm sorely tempted to make restore failure be a FATAL error, but will refrain for the moment.) Having done that, it's not much more work to ensure that we clean up strdup'd storage on the way out, too. This code is substantially the same in all supported branches, so back-patch all the way. Michael Paquier and Tom Lane Discussion: <CAB7nPqRMbGqa_mesopcn4MPyTs34eqtVEK7ELYxvvV=oqS00YA@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/a4930e7ca2aa49c13ccba089f5fd2c416d87e96a
  • Fix uninitialized variable. Oversight in a734fd5d1. Michael Paquier http://git.postgresql.org/pg/commitdiff/ae92a9a3806c025653140ee6316a82d55e24b82c
  • Make contrib/test_decoding regression tests safe for CZ locale. A little COLLATE "C" goes a long way. Pavel Stehule, per suggestion from Craig Ringer Discussion: <CAFj8pRA8nJZcozgxN=RMSqMmKuHVOkcGAAKPKdFeiMWGDSUDLA@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/e2a0ee69006bf31f24958b72c93455412cba035c
  • Doc: add a section in Part II concerning RETURNING. There are assorted references to RETURNING in Part II, but nothing that would qualify as an explanation of the feature, which seems like an oversight considering how useful it is. Add something. Noted while looking for a place to point a cross-reference to ... http://git.postgresql.org/pg/commitdiff/1c7861e81b4220364bef75d2445e9c0619f3f3f8
  • Improve handling of "UPDATE ... SET (column_list) = row_constructor". Previously, the right-hand side of a multiple-column assignment, if it wasn't a sub-SELECT, had to be a simple parenthesized expression list, because gram.y was responsible for "bursting" the construct into independent column assignments. This had the minor defect that you couldn't write ROW (though you should be able to, since the standard says this is a row constructor), and the rather larger defect that unlike other uses of row constructors, we would not expand a "foo.*" item into multiple columns. Fix that by changing the RHS to be just "a_expr" in the grammar, leaving it to transformMultiAssignRef to separate the elements of a RowExpr; which it will do only after performing standard transformation of the RowExpr, so that "foo.*" behaves as expected. The key reason we didn't do that before was the hard-wired handling of DEFAULT tokens (SetToDefault nodes). This patch deals with that issue by allowing DEFAULT in any a_expr and having parse analysis throw an error if SetToDefault is found in an unexpected place. That's an improvement anyway since the error can be more specific than just "syntax error". The SQL standard suggests that the RHS could be any a_expr yielding a suitable row value. This patch doesn't really move the goal posts in that respect --- you're still limited to RowExpr or a sub-SELECT --- but it does fix the grammar restriction, so it provides some tangible progress towards a full implementation. And the limitation is now documented by an explicit error message rather than an unhelpful "syntax error". Discussion: <8542.1479742008@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/906bfcad7ba7cb3863fe0e2a7810be8e3cd84fbd
  • Doc: improve documentation about composite-value usage. Create a section specifically for the syntactic rules around whole-row variable usage, such as expansion of "foo.*". This was previously documented only haphazardly, with some critical info buried in unexpected places like xfunc-sql-composite-functions. Per repeated questions in different mailing lists. Discussion: <16288.1479610770@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/e1320266edd7df53c60af10b4c33ab2754278b3e
  • Make sure ALTER TABLE preserves index tablespaces. When rebuilding an existing index, ALTER TABLE correctly kept the physical file in the same tablespace, but it messed up the pg_class entry if the index had been in the database's default tablespace and "default_tablespace" was set to some non-default tablespace. This led to an inaccessible index. Fix by fixing pg_get_indexdef_string() to always include a tablespace clause, whether or not the index is in the default tablespace. The previous behavior was installed in commit 537e92e41, and I think it just wasn't thought through very clearly; certainly the possible effect of default_tablespace wasn't considered. There's some risk in changing the behavior of this function, but there are no other call sites in the core code. Even if it's being used by some third party extension, it's fairly hard to envision a usage that is okay with a tablespace clause being appended some of the time but can't handle it being appended all the time. Back-patch to all supported versions. Code fix by me, investigation and test cases by Michael Paquier. Discussion: <1479294998857-5930602.post@n3.nabble.com> http://git.postgresql.org/pg/commitdiff/bd673e8e864a1987eca8d40c593e857ab8d0a5ba
  • Avoid masking a function parameter name with a local variable name. No actual bug here, but it might confuse readers, so change the name of the local variable. Ashutosh Bapat http://git.postgresql.org/pg/commitdiff/6fa391be4e83139cd134d5ccfc1499809bb8c98c
  • Check that default_tablespace affects ALTER TABLE ADD UNIQUE/PRIMARY KEY. Seems like a good thing to test, considering that we nearly broke it yesterday. Michael Paquier http://git.postgresql.org/pg/commitdiff/4cc6a3f1106034187be3a371e61a02915bb93c11
  • Check for pending trigger events on far end when dropping an FK constraint. When dropping a foreign key constraint with ALTER TABLE DROP CONSTRAINT, we refuse the drop if there are any pending trigger events on the named table; this ensures that we won't remove the pg_trigger row that will be consulted by those events. But we should make the same check for the referenced relation, else we might remove a due-to-be-referenced pg_trigger row for that relation too, resulting in "could not find trigger NNN" or "relation NNN has no triggers" errors at commit. Per bug #14431 from Benjie Gillam. Back-patch to all supported branches. Report: <20161124114911.6530.31200@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/4e026b32d4024b03856b4981b26c747b7fef7afb
  • Mark a query's topmost Paths parallel-unsafe if they will have initPlans. Andreas Seltenreich found another case where we were being too optimistic about allowing a plan to be considered parallelizable despite it containing initPlans. It seems like the real issue here is that if we know we are going to tack initPlans onto the topmost Plan node for a subquery, we had better mark that subquery's result Paths as not-parallel-safe. That fixes this problem and allows reversion of a kluge (added in commit 7b67a0a49 and extended in f24cf960d) to not trust the parallel_safe flag at top level. Discussion: <874m2w4k5d.fsf@ex.ansel.ydns.eu> http://git.postgresql.org/pg/commitdiff/ab77a5a4561fad847af4a101a29c922c66449870
  • Bring some clarity to the defaults for the xxx_flush_after parameters. Instead of confusingly stating platform-dependent defaults for these parameters in the comments in postgresql.conf.sample (with the main entry being a lie on Linux), teach initdb to install the correct platform-dependent value in postgresql.conf, similarly to the way we handle other platform-dependent defaults. This won't do anything for existing 9.6 installations, but since it's effectively only a documentation improvement, that seems OK. Since this requires initdb to have access to the default values, move the #define's for those to pg_config_manual.h; the original placement in bufmgr.h is unworkable because that file can't be included by frontend programs. Adjust the default value for wal_writer_flush_after so that it is 1MB regardless of XLOG_BLCKSZ, conforming to what is stated in both the SGML docs and postgresql.conf. (We could alternatively make it scale with XLOG_BLCKSZ, but I'm not sure I see the point.) Copy-edit related SGML documentation. Fabien Coelho and Tom Lane, per a gripe from Tomas Vondra. Discussion: <30ebc6e3-8358-09cf-44a8-578252938424@2ndquadrant.com> http://git.postgresql.org/pg/commitdiff/dbdfd114f34443f1e4ad16ce2721f9817d3b3d80
  • Fix test about ignoring extension dependencies during extension scripts. Commit 08dd23cec introduced an exception to the rule that extension member objects can only be dropped as part of dropping the whole extension, intending to allow such drops while running the extension's own creation or update scripts. However, the exception was only applied at the outermost recursion level, because it was modeled on a pre-existing check to ignore dependencies on objects listed in pendingObjects. Bug #14434 from Philippe Beaudoin shows that this is inadequate: in some cases we can reach an extension member object by recursion from another one. (The bug concerns the serial-sequence case; I'm not sure if there are other cases, but there might well be.) To fix, revert 08dd23cec's changes to findDependentObjects() and instead apply the creating_extension exception regardless of stack level. Having seen this example, I'm a bit suspicious that the pendingObjects logic is also wrong and such cases should likewise be allowed at any recursion level. However, changing that would interact in subtle ways with the recursion logic (at least it would need to be moved to after the recursing-from check). Given that the code's been like that a long time, I'll refrain from touching it without a clear example showing it's wrong. Back-patch to all active branches. In HEAD and 9.6, where suitable test infrastructure exists, add a regression test case based on the bug report. Report: <20161125151448.6529.33039@wrigleys.postgresql.org> Discussion: <13224.1480177514@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/182db070403fb33566da156a3a77cbcda16583b4

Robert Haas pushed:

  • autovacuum: Drop orphan temp tables more quickly but with more caution. Previously, we only dropped an orphan temp table when it became old enough to threaten wraparound; instead, doing it immediately. The only value of waiting is that someone might be able to examine the contents of the orphan temp table for forensic purposes, but it's pretty difficult to actually do that and few users will wish to do so. On the flip side, not performing the drop immediately generates log spam and bloats pg_class. In addition, per a report from Grigory Smolkin, if a temporary schema contains a very large number of temporary tables, a backend attempting to clear the temporary schema might fail due to lock table exhaustion. It's helpful for autovacuum to clean up after such cases, and we don't want it to wait for wraparound to threaten before doing so. To prevent autovacuum from failing in the same manner as a backend trying to drop an entire temp schema, remove orphan temp tables in batches of 50, committing after each batch, so that we don't accumulate an unbounded number of locks. If a drop fails, retry other orphan tables that need to be dropped up to 10 times before giving up. With this system, if a backend does fail to clean a temporary schema due to lock table exhaustion, autovacuum should hopefully put things right the next time it processes the database. Discussion: CAB7nPqSbYT6dRwsXVgiKmBdL_ARemfDZMPA+RPeC_ge0GK70hA@mail.gmail.com Michael Paquier, with a bunch of comment changes by me. http://git.postgresql.org/pg/commitdiff/a734fd5d1c309cc553b7c8c79fba96218af090f7
  • Support condition variables. Condition variables provide a flexible way to sleep until a cooperating process causes an arbitrary condition to become true. In simple cases, this can be accomplished with a WaitLatch/ResetLatch loop; the cooperating process can call SetLatch after performing work that might cause the condition to be satisfied, and the waiting process can recheck the condition each time. However, if the process performing the work doesn't have an easy way to identify which processes might be waiting, this doesn't work, because it can't identify which latches to set. Condition variables solve that problem by internally maintaining a list of waiters; a process that may have caused some waiter's condition to be satisfied must "signal" or "broadcast" on the condition variable. Robert Haas and Thomas Munro http://git.postgresql.org/pg/commitdiff/e8ac886c24776295dd9b025386a821061da8e4d1
  • Code review for commit 274bb2b3857cc987cfa21d14775cae9b0dababa5. Avoid memory leak in conninfo_uri_parse_options. Use the current host rather than the comma-separated list of host names when the host name is needed for GSS, SSPI, or SSL authentication. Document the way connect_timeout interacts with multiple host specifications. Takayuki Tsunakawa http://git.postgresql.org/pg/commitdiff/9a1d0af4ad2cbd419115b453d811c141b80d872b
  • Remove barrier.h. A new thing also called a "barrier" is proposed, but whether we decide to take that patch or not, this file seems to have outlived its usefulness. Thomas Munro http://git.postgresql.org/pg/commitdiff/e343dfa42bff35983c582da3916b205763aeac90
  • Mark IsPostmasterEnvironment and IsBackgroundWorker as PGDLLIMPORT. Per request from Craig Ringer. http://git.postgresql.org/pg/commitdiff/273270593f42bdbd5422923eb70fbd0fb0f65bf0

Ãlvaro Herrera pushed:

Magnus Hagander pushed:

Correctifs en attente

Craig Ringer sent in two revisions of a patch to document that test must pass with Perl 5.8.0 and show how to ensure that they do.

Michaël Paquier sent in another revision of a patch to fix the checkpoint skip logic on idle systems by tracking LSN progress.

Corey Huinker sent in a patch to ensure that dblink knows about valid PostgreSQL FDW options.

Thomas Munro sent in another revision of a patch to add delta relations in AFTER triggers, complete with exposure to PL/pgsql and PL/PythonU.

KaiGai Kohei sent in another revision of a patch to enable passing LIMIT to FDWs AND CUSTOMSCANS.

Daniel Vérité sent in two more revisions of a patch to improve psql hooks for variables.

Amit Kapila sent in a patch to prevent cascading standbys from getting stuck.

Etsuro Fujita sent in three more revisions of a patch to push down more full joins to the PostgreSQL FDW.

Craig Ringer sent in a patch to add logical decoding on standbys.

Thomas Munro sent in another revision of a patch to measure replay lag.

Matheus de Oliveira sent in a patch to extend ALTER DEFAULT PRIVILEGES to include schemas.

Thomas Munro sent in another revision of a patch to add barriers for synchronizing cooperating processes.

Peter Geoghegan sent in another revision of a patch to add amcheck, a B-Tree integrity checking tool.

Mithun Cy sent in two more revisions of a patch to implement failover on the libpq connect level.

Michaël Paquier sent in another revision of a patch to forbid the use of LF and CR characters in database and role names.

Amit Langote sent in two more revisions of a patch to add declarative partitioning.

Erik Rijkers sent in a patch to fix the CREATE SUBSCRIPTION documents, which broke the oldhtml target.

Andres Freund sent in a patch to fix a performance degradation in Bitmapscan.

Amul Sul sent in another revision of a patch to add a pg_background contrib extension.

Amit Kapila sent in another revision of a patch to make hash indexes concurrency-safe.

Ãlvaro Herrera and Pavel StÄ›hule traded patches to add xmltable().

Thomas Munro sent in two more revisions of a patch to add dynamic shared memory areas (DSAs).

Thomas Munro sent in two more revisions of a patch to add a DSA for the parallel execution.

Karl O. Pinc and Gilles Darold traded patches to implement pg_current_logfile().

Rushabh Lathia sent in another revision of a patch to implement Gather Merge.

Andreas Karlsson sent in a WIP patch to make PQHost() return the value of the host parameter rather than the hostaddr.

Amit Kapila sent in two revisions of a patch to allow safe subquery with parallel workers.

Haribabu Kommi sent in another revision of a patch to add an 8-byte MAC address type called macaddr8 which conforms to EUI-64.

Pavel Stěhule sent in a PoC patch to add session server side variables.

Ming Li sent in a patch to fix a pg_class refcache leak when the meta tuple in pg_class is invalid.

Amit Kapila sent in another revision of a patch to add parallel index scans.

Michaël Paquier sent in another revision of a patch to add quorum commit for multiple synchronous replication.

Artur Zakirov sent in a patch to fix a bug in which caused a pg_event_trigger_ddl_commands() error with ALTER TEXT SEARCH CONFIGURATION.

par N Bougain le mardi 29 novembre 2016 à 02h08

samedi 26 novembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 20 novembre 2016

Le nouveau magazine PostgreSQL est disponible ! http://www.pgmag.org/

La PGconf.ASIA 2016 aura lieu les 1, 2 & 3 décembre à Tokyo (Japon). Les inscriptions sont ouvertes. Il y aura une UnConference le 1er décembre avec une inscription séparée et requise : http://www.pgconf.asia/EN/registration/ http://www.pgconf.asia/EN/day-0/

La PGConf India 2017 aura lieu les 2 & 3 mars 2017 à Bengalore (État du Karnataka en Inde). Les propositions sont attendues sur <papers AT pgconf DOT in> jusqu'au 31 décembre 2016 : http://pgconf.in/

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en novembre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161120225904.GB23469@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Ãlvaro Herrera pushed:

  • Fix duplication in ALTER MATERIALIZE VIEW synopsis. Commit 3c4cf080879b should have removed SET TABLESPACE from the synopsis of ALTER MATERIALIZE VIEW as a possible "action" when it added a separate line for it in the main command listing, but failed to. Repair. Backpatch to 9.4, like the aforementioned commit. http://git.postgresql.org/pg/commitdiff/8ce4f597abc530b3b59bcf3a3964f31e50054bcd
  • Avoid pin scan for replay of XLOG_BTREE_VACUUM in all cases Replay of XLOG_BTREE_VACUUM during Hot Standby was previously thought to require complex interlocking that matched the requirements on the master. This required an O(N) operation that became a significant problem with large indexes, causing replication delays of seconds or in some cases minutes while the XLOG_BTREE_VACUUM was replayed. This commit skips the “pin scan†that was previously required, by observing in detail when and how it is safe to do so, with full documentation. The pin scan is skipped only in replay; the VACUUM code path on master is not touched here. No tests included. Manual tests using an additional patch to view WAL records and their timing have shown the change in WAL records and their handling has successfully reduced replication delay. This is a back-patch of commits 687f2cd7a015, 3e4b7d87988f, b60284261375 by Simon Riggs, to branches 9.4 and 9.5. No further backpatch is possible because this depends on catalog scans being MVCC. I (Ãlvaro) additionally updated a slight problem in the README, which explains why this touches the 9.6 and master branches. http://git.postgresql.org/pg/commitdiff/f65b94f63962e9f7e144a469bc1750286ddaee27

Peter Eisentraut pushed:

  • Allow individual TAP tests to be run via PROVE_TESTS. Add a new optional Makefile variable PROVE_TESTS that, if passed as a space-separated list of paths relative to the Makefile invoking $(prove_check) or $(prove_installcheck), runs just those tests instead of t/*.pl . From: Craig Ringer <craig@2ndquadrant.com> Reviewed-by: Michael Paquier <michael.paquier@gmail.com> http://git.postgresql.org/pg/commitdiff/9ca7b0bf016364c74d38f66c7050be915bfea908
  • Build HTML documentation using XSLT stylesheets by default. The old DSSSL build is still available for a while using the make target "oldhtml". http://git.postgresql.org/pg/commitdiff/e36ddab11735052841b4eff96642187ec9a8a7bc
  • doc: Further XSLT HTML build performance optimization. Cut out some expensive stuff from the HTML head element that we don't really need. This was previously discussed as part of e8306745e3504c642f7abad411139d5630e29fac, but ended up separate because it changes the output contents slightly. http://git.postgresql.org/pg/commitdiff/380895f2deb18ed9e7a8be69961af2ed221ba9d3
  • Add pg_sequences view. Like pg_tables, pg_views, and others, this view contains information about sequences in a way that is independent of the system catalog layout but more comprehensive than the information schema. To help implement the view, add a new internal function pg_sequence_last_value() to return the last value of a sequence. This is kept separate from pg_sequence_parameters() to separate querying run-time state from catalog-like information. Reviewed-by: Andreas Karlsson <andreas@proxel.se> http://git.postgresql.org/pg/commitdiff/67dc4ccbb2e1c27da823eced66d9217a5652cbb0

Magnus Hagander pushed:

Andres Freund pushed:

Robert Haas pushed:

  • pgbench: Increase maximum size of log filename from 64 to MAXPGPATH. Commit 41124a91e61fc6d9681c1e8b15ba30494e84d643 allowed the transaction log file prefix to be changed but left in place the existing 64-character limit on the total length of a log file name. It's possible that could be inconvenient for somebody, so increase the limit to MAXPGPATH, which ought to be enough for anybody. Per a suggestion from Tom Lane. http://git.postgresql.org/pg/commitdiff/56eba9b8a1120c861868dd3d86d927a9e3182880
  • Fix broken statement in UCS_to_most.pl. This has been wrong for a very long time, and it's puzzling to me how it ever worked for anyone. Kyotaro Horiguchi http://git.postgresql.org/pg/commitdiff/00c6d8077f39191a6f61a847ce7d55073d8f5a6f
  • Limit the number of number of tapes used for a sort to 501. Gigantic numbers of tapes don't work out well. Original patch by Peter Geoghegan; comments entirely rewritten by me. http://git.postgresql.org/pg/commitdiff/fc19c1801bd2dbee1043b0c0b62e07747d30ea1c
  • Reserve zero as an invalid DSM handle. Previously, the handle for the control segment could not be zero, but some other DSM segment could potentially have a handle value of zero. However, that means that if someone wanted to store a dsm_handle that might or might not be valid, they would need a separate boolean to keep track of whether the associated value is legal. That's annoying, so change things so that no DSM segment can ever have a handle of 0 - or as we call it here, DSM_HANDLE_INVALID. Thomas Munro. This was submitted as part of a much larger patch to add an malloc-like allocator for dynamic shared memory, but this part seems like a good idea independently of the rest of the patch. http://git.postgresql.org/pg/commitdiff/b40b4dd9e10ea701c8d47ccba9407fc32ed384e5
  • Remove or reduce verbosity of some debug messages. The debug messages that merely print StartTransactionCommand, CommitTransactionCommand, ProcessUtilty, or ProcessQuery with no additional details seem to be useless. Get rid of them. The transaction status messages produced by ShowTransactionState are occasionally useful, but they are extremely verbose, producing multiple lines of log output every time they fire, which can happens multiple times per transaction. So, reduce the level to DEBUG5; avoid emitting an extra line just to explain which debug point is at issue; and tighten up the rest of the message so it doesn't use quite so much horizontal space. With these changes, it's possible to run a somewhat busy system with a log level even as high as DEBUG4, whereas previously anything above DEBUG2 would flood the log with output that probably wasn't really all that useful. http://git.postgresql.org/pg/commitdiff/a43f1939d5dcd02f4df1604a68392332168e4be0

Tom Lane pushed:

  • Account for catalog snapshot in PGXACT->xmin updates. The CatalogSnapshot was not plugged into SnapshotResetXmin()'s accounting for whether MyPgXact->xmin could be cleared or advanced. In normal transactions this was masked by the fact that the transaction snapshot would be older, but during backend startup and certain utility commands it was possible to re-use the CatalogSnapshot after MyPgXact->xmin had been cleared, meaning that recently-deleted rows could be pruned even though this snapshot could still see them, causing unexpected catalog lookup failures. This effect appears to be the explanation for a recent failure on buildfarm member piculet. To fix, add the CatalogSnapshot to the RegisteredSnapshots heap whenever it is valid. In the previous logic, it was possible for the CatalogSnapshot to remain valid across waits for client input, but with this change that would mean it delays advance of global xmin in cases where it did not before. To avoid possibly causing new table-bloat problems with clients that sit idle for long intervals, add code to invalidate the CatalogSnapshot before waiting for client input. (When the backend is busy, it's unlikely that the CatalogSnapshot would be the oldest snap for very long, so we don't worry about forcing early invalidation of it otherwise.) In passing, remove the CatalogSnapshotStale flag in favor of using "CatalogSnapshot != NULL" to represent validity, as we do for the other special snapshots in snapmgr.c. And improve some obsolete comments. No regression test because I don't know a deterministic way to cause this failure. But the stress test shown in the original discussion provokes "cache lookup failed for relation 1255" within a few dozen seconds for me. Back-patch to 9.4 where MVCC catalog scans were introduced. (Note: it's quite easy to produce similar failures with the same test case in branches before 9.4. But MVCC catalog scans were supposed to fix that.) Discussion: <16447.1478818294@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/ffaa44cb559db332baeee7d25dedd74a61974203
  • Allow DOS-style line endings in ~/.pgpass files. On Windows, libc will mask \r\n line endings for us, since we read the password file in text mode. But that doesn't happen on Unix. People who share password files across both systems might have \r\n line endings in a file they use on Unix, so as a convenience, ignore trailing \r. Per gripe from Josh Berkus. In passing, put the existing check for empty line somewhere where it's actually useful, ie after stripping the newline not before. Vik Fearing, adjusted a bit by me Discussion: <0de37763-5843-b2cc-855e-5d0e5df25807@agliodbs.com> http://git.postgresql.org/pg/commitdiff/0a7481930c788e9d74a154aac0c8b401fc6a81f9
  • Check that result tupdesc has exactly 1 column in return_next scalar case. This should always be true, but since we're relying on a tuple descriptor passed from outside pltcl itself, let's check. Per a gripe from Coverity. http://git.postgresql.org/pg/commitdiff/4ecd1974377ffb4d6d72874ba14fcd23965b1792
  • Re-pgindent src/bin/pg_dump/* Cleanup for recent patches --- it's not much change, but I got annoyed while re-indenting the view-rule fix I'm working on. http://git.postgresql.org/pg/commitdiff/fcf70e0dbca1432959be5f3557acd546d639c9ba
  • Improve pg_dump/pg_restore --create --if-exists logic. Teach it not to complain if the dropStmt attached to an archive entry is actually spelled CREATE OR REPLACE VIEW, since that will happen due to an upcoming bug fix. Also, if it doesn't recognize a dropStmt, have it print a WARNING and then emit the dropStmt unmodified. That seems like a much saner behavior than Assert'ing or dumping core due to a null-pointer dereference, which is what would happen before :-(. Back-patch to 9.4 where this option was introduced. Discussion: <19092.1479325184@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/ac888986fc656c0badb0a4918b3581e2ebcb154a
  • Fix pg_dump's handling of circular dependencies in views. pg_dump's traditional solution for breaking a circular dependency involving a view was to create the view with CREATE TABLE and then later issue CREATE RULE "_RETURN" ... to convert the table to a view, relying on the backend's very very ancient code that supports making views that way. We've wanted to get rid of that kluge for a long time, but the thing that finally motivates doing something about it is the recognition that this method fails with the --clean option, because it leads to issuing DROP RULE "_RETURN" followed by DROP TABLE --- and the backend won't let you drop a view's _RETURN rule. Instead, let's break circular dependencies by initially creating the view using CREATE VIEW AS SELECT NULL::columntype AS columnname, ... (so that it has the right column names and types to support external references, but no dependencies beyond the column data types), and then later dumping the ON SELECT rule using the spelling CREATE OR REPLACE VIEW. This method wasn't available when this code was originally written, but it's been possible since PG 7.3, so it seems fine to start relying on it now. To solve the --clean problem, make the dropStmt for an ON SELECT rule be CREATE OR REPLACE VIEW with the same dummy target list as above. In this way, during the DROP phase, we first reduce the view to have no extra dependencies, and then we can drop it entirely when we've gotten rid of whatever had a circular dependency on it. (Note: this should work adequately well with the --if-exists option, since the CREATE OR REPLACE VIEW will go through whether the view exists or not. It could fail if the view exists with a conflicting column set, but we don't really support --clean against a non-matching database anyway.) This allows cleaning up some other kluges inside pg_dump, notably that we don't need a notion of reloptions attached to a rule anymore. Although this is a bug fix, commit to HEAD only for now. The problem's existed for a long time and we've had relatively few complaints, so it doesn't really seem worth taking risks to fix it in the back branches. We might revisit that choice if no problems emerge. Discussion: <19092.1479325184@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/d8c05aff56be92dda889abc89e3f3921d1c29c30
  • Code review for GUC serialization/deserialization code. The serialization code dumped core for a string-valued GUC whose value is NULL, which is a legal state. The infrastructure isn't capable of transmitting that state exactly, but fortunately, transmitting an empty string instead should be close enough (compare, eg, commit e45e990e4). The code potentially underestimated the space required to format a real-valued variable, both because it made an unwarranted assumption that %g output would never be longer than %e output, and because it didn't count right even for %e format. In practice this would pretty much always be masked by overestimates for other variables, but it's still wrong. Also fix boundary-case error in read_gucstate, incorrect handling of the case where guc_sourcefile is non-NULL but zero length (not clear that can happen, but if it did, this code would get totally confused), and confusingly useless check for a NULL result from read_gucstate. Andreas Seltenreich discovered the core dump; other issues noted while reading nearby code. Back-patch to 9.5 where this code was introduced. Michael Paquier and Tom Lane Discussion: <871sy78wno.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/13671b4b22ae4bd345c62e7c0b41d717b8a2e19b
  • Fix latent costing error in create_merge_append_path. create_merge_append_path should use the path rowcount it just computed, not rel->tuples, for costing purposes. Those numbers should always be the same at present, but if we ever support parameterized MergeAppend paths (a case this function is otherwise prepared for), the former would be right and the latter wrong. No need for back-patch since the problem is only latent. Ashutosh Bapat Discussion: <CAFjFpRek+cLCnTo24youuGtsq4zRphEB8EUUPjDxZjnL4n4HYQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/0832f2db68cc43524a240db47d0428cc9525723e
  • Prevent multicolumn expansion of "foo.*" in an UPDATE source expression. Because we use transformTargetList() for UPDATE as well as SELECT tlists, the code accidentally tried to expand a "*" reference into several columns. This is nonsensical, because the UPDATE syntax provides exactly one target column to put the value into. The immediate result was that transformUpdateTargetList() got confused and reported "UPDATE target count mismatch --- internal error". It seems better to treat such a reference as a plain whole-row variable, as it would be in other contexts. (This could produce useful results when the target column is of composite type.) Fix by tweaking transformTargetList() to perform *-expansion only conditionally, depending on its exprKind parameter. Back-patch to 9.3. The problem exists further back, but a fix would be much more invasive before that, because transformTargetList() wasn't told what kind of list it was working on. Doesn't seem worth the trouble given the lack of field reports. (I only noticed it because I was checking the code while trying to improve the documentation about how we handle "foo.*".) Discussion: <4308.1479595330@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/c5f365f3ab21a345b7a109bc0411895536e9fa69

Stephen Frost pushed:

  • Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this. http://git.postgresql.org/pg/commitdiff/8f91f323b4feef0371cd3db51be3007e44abd5e8

Correctifs en attente

Tomas Vondra sent in another revision of a patch to add two slab-like memory allocators.

Laurenz Albe sent in another revision of a patch to ensure that writes are synced to disk.

Michaël Paquier and Robert Haas traded patches to add SCRAM auth with provisions for more types.

Takayuki Tsunakawa sent in another revision of a patch to fix a situation where a cascaded standby gets stuck and can't catch up.

Kyotaro HORIGUCHI sent in another revision of a patch to add asynchronous execution.

Kyotaro HORIGUCHI sent in another revision of a patch to rework psql's tab completion system.

Magnus Hagander sent in a patch to add a capability to jsonb_delete to accept an array of keys rather than a singleton one.

Kuntal Ghosh sent in another revision of a patch to add a WAL consistency check facility.

Thomas Munro sent in another revision of a patch to enable SERIALIZABLE isolation mode on standby servers.

Thomas Munro sent in another revision of a patch to create dynamic shared memory areas.

Masahiko Sawada sent in another revision of a patch to enable quorum commit for multiple synchronous standbys.

Craig Ringer sent in two more revisions of a patch to enable logical decoding timeline following.

Andrew Borodin sent in another revision of a patch to implement lazy GiST indexing.

Mark Kirkwood sent in another revision of a patch to switch to the CMake build system.

Pavel Stěhule sent in three more revisions of a patch to implement \setfileref in psql.

Etsuro Fujita and Ashutosh Bapat traded patches to push down more full joins to FDWs.

Amit Langote sent in three more revisions of a patch to implement declarative partitioning.

Magnus Hagander sent in a patch to log "snapshot too old" conditions.

Laurenz Albe and Tobias Bussmann traded patches to enable parallel prepared statement execution.

Karl O. Pinc and Gilles Darold traded patches to implement pg_current_logfile().

Daniel Vérité sent in another revision of a patch to add capabilities to psql hooks for variables.

Ramon Lawrence sent in another revision of a patch to implement Gather Merge.

Christian Ullrich sent in another revision of a patch to add putenv support for msvcrt from Visual Studio 2013.

Rahila Syed sent in a patch to fix some issues that can result in an invalid collation error.

Haribabu Kommi sent in another revision of a patch to add a pg_hba_file_settings view.

Michaël Paquier sent in two more revisions of a patch to fix cleanup of unlogged tables.

Claudio Freire sent in another revision of a patch to enable vacuum to use more than 1GB of work mem.

Craig Ringer sent in a patch to document that perl 5.8.4 or higher is required for TAP tests.

Craig Ringer sent in a patch to allow walsender to exit on conflict with recovery.

Etsuro Fujita sent in a patch to push down more UPDATEs/DELETEs in the postgres_fdw.

Kyotaro HORIGUCHI sent in two approaches to fix an issue where WALs were being recycled too aggressively in synchronous replication. The first injects a pause. The second blocks WAL insertion until sync replication reaches the first surviving segments.

Michaël Paquier sent in two more revisions of a patch to clean up orphan temp tables with autovacuum.

Mithun Cy sent in two more revisions of a patch to implement failover on the libpq connect level.

Ashutosh Bapat sent in a patch to fix estimated tuple counts for appendrel.

Pavel Stěhule sent in another revision of a patch to implement xmltable().

Stephen Frost sent in a doc patch to clarify that blobs are considered data and not schema for pg_dump purposes.

John Gorman sent in a patch to speed up dynamic shared memory hash tables.

Gilles Darold sent in a patch to add psql tab completion for ALTER DEFAULT PRIVILEGES.

Petr Jelínek sent in another revision of a patch to add logical replication.

par N Bougain le samedi 26 novembre 2016 à 16h15

vendredi 4 novembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 30 octobre 2016

Versions correctives 9.6.1, 9.5.5, 9.4.10, 9.3.15, 9.2.19 et 9.1.24 disponibles. Mettez à jour dès que possible. La 9.1.24 est la dernière publication de la série 9.1 : https://www.postgresql.org/about/news/1712/

Offres d'emplois autour de PostgreSQL en octobre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161031023727.GE23627@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Tom Lane pushed:

  • Don't throw serialization errors for self-conflicts in INSERT ON CONFLICT. A transaction that conflicts against itself, for example INSERT INTO t(pk) VALUES (1),(1) ON CONFLICT DO NOTHING; should behave the same regardless of isolation level. It certainly shouldn't throw a serialization error, as retrying will not help. We got this wrong due to the ON CONFLICT logic not considering the case, as reported by Jason Dusek. Core of this patch is by Peter Geoghegan (based on an earlier patch by Thomas Munro), though I didn't take his proposed code refactoring for fear that it might have unexpected side-effects. Test cases by Thomas Munro and myself. Report: <CAO3NbwOycQjt2Oqy2VW-eLTq2M5uGMyHnGm=RNga4mjqcYD7gQ@mail.gmail.com> Related-Discussion: <57EE93C8.8080504@postgrespro.ru> http://git.postgresql.org/pg/commitdiff/a6c0a5b6e8a9498540c6a7bb1b6d68958acc9bc6
  • Avoid testing tuple visibility without buffer lock. INSERT ... ON CONFLICT (specifically ExecCheckHeapTupleVisible) contains another example of this unsafe coding practice. It is much harder to get a failure out of it than the case fixed in commit 6292c2339, because in most scenarios any hint bits that could be set would have already been set earlier in the command. However, Konstantin Knizhnik reported a failure with a custom transaction manager, and it's clearly possible to get a failure via a race condition in async-commit mode. For lack of a reproducible example, no regression test case in this commit. I did some testing with Asserts added to tqual.c's functions, and can say that running "make check-world" exposed these two bugs and no others. The Asserts are messy enough that I've not added them to the code for now. Report: <57EE93C8.8080504@postgrespro.ru> Related-Discussion: <CAO3NbwOycQjt2Oqy2VW-eLTq2M5uGMyHnGm=RNga4mjqcYD7gQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/8f1fb7d621b0e6bd2eb0ba2ac9634c5b5a03564b
  • Release notes for 9.6.1, 9.5.5, 9.4.10, 9.3.15, 9.2.19, 9.1.24. http://git.postgresql.org/pg/commitdiff/7d80417d3dfc88b0c03b5c08a18b29f9d430e217
  • Update release notes for last-minute commit timestamp fix. http://git.postgresql.org/pg/commitdiff/30a6f98ed8a2223ef876197b5dea0d94e5807b51
  • Suppress unused-variable warning in non-assert builds. Introduced in commit 7012b132d. Kyotaro Horiguchi http://git.postgresql.org/pg/commitdiff/8529686ccbb9f1c2fe350920ad324c223135a957
  • Doc: improve documentation about inheritance. Clarify documentation about inheritance of check constraints, in particular mentioning the NO INHERIT option, which didn't exist when this text was written. Document that in an inherited query, the applicable row security policies are those of the explicitly-named table, not its children. This is the intended behavior (per off-list discussion with Stephen Frost), and there are regression tests for it, but it wasn't documented anywhere user-facing as far as I could find. Do a bit of wordsmithing on the description of inherited access-privilege checks. Back-patch to 9.5 where RLS was added. http://git.postgresql.org/pg/commitdiff/162477a63d3c0fd1c31197717140a88077a8d0aa
  • Fix not-HAVE_SYMLINK code in zic.c. I broke this in commit f3094920a. Apparently it's dead code anyway, at least as far as our buildfarm is concerned (and the upstream IANA code doesn't worry at all about symlink() not being present). But as long as the rest of our code is willing to guard against not having symlink(), this should too. Noted while investigating a tangentially-related complaint from Sandeep Thakkar. Back-patch to keep branches in sync. http://git.postgresql.org/pg/commitdiff/19b2094d96807e43d29687b3860e8fffb9da61b4
  • Fix incorrect trigger-property updating in ALTER CONSTRAINT. The code to change the deferrability properties of a foreign-key constraint updated all the associated triggers to match; but a moment's examination of the code that creates those triggers in the first place shows that only some of them should track the constraint's deferrability properties. This leads to odd failures in subsequent exercise of the foreign key, as the triggers are fired at the wrong times. Fix that, and add a regression test comparing the trigger properties produced by ALTER CONSTRAINT with those you get by creating the constraint as-intended to begin with. Per report from James Parks. Back-patch to 9.4 where this ALTER functionality was introduced. Report: <CAJ3Xv+jzJ8iNNUcp4RKW8b6Qp1xVAxHwSXVpjBNygjKxcVuE9w@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/a522fc3d806ca6ebe3f66e55b3f8cecb85116711
  • Improve speed of aggregates that use array_append as transition function. In the previous coding, if an aggregate's transition function returned an expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus force it into the flat representation. This led to ping-ponging between flat and expanded formats, which costs a lot. For an aggregate using array_append as transition function, I measured about a 15X slowdown compared to the pre-9.5 code, when working on simple int[] arrays. Of course, the old code was already O(N^2) in this usage due to copying flat arrays all the time, but it wasn't quite this inefficient. To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition values without copying, so long as the transition function takes care to return the transition value already properly parented under the aggcontext. That puts a bit of extra responsibility on the transition function, but doing it this way allows us to not need any extra logic in the fast path of advance_transition_function (ie, with a pass-by-value transition value, or with a modified-in-place pass-by-reference value). We already know that that's a hot spot so I'm loath to add any cycles at all there. Also, while only array_append currently knows how to follow this convention, this solution allows other transition functions to opt-in without needing to have a whitelist in the core aggregation code. (The reason we would need a whitelist is that currently, if you pass a R/W expanded-object pointer to an arbitrary function, it's allowed to do anything with it including deleting it; that breaks the core agg code's assumption that it should free discarded values. Returning a value under aggcontext is the transition function's signal that it knows it is an aggregate transition function and will play nice. Possibly the API rules for expanded objects should be refined, but that would not be a back-patchable change.) With this fix, an aggregate using array_append is no longer O(N^2), so it's much faster than pre-9.5 code rather than much slower. It's still a bit slower than the bespoke infrastructure for array_agg, but the differential seems to be only about 10%-20% rather than orders of magnitude. Discussion: <6315.1477677885@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/9a00f03e479c2d4911eed5b4bd1994315d409938
  • Fix bogus tree-flattening logic in QTNTernary(). QTNTernary() contains logic to flatten, eg, '(a & b) & c' into 'a & b & c', which is all well and good, but it tries to do that to NOT nodes as well, so that '!!a' gets changed to '!a'. Explicitly restrict the conversion to be done only on AND and OR nodes, and add a test case illustrating the bug. In passing, provide some comments for the sadly naked functions in tsquery_util.c, and simplify some baroque logic in QTNFree(), which I think may have been leaking some items it intended to free. Noted while investigating a complaint from Andreas Seltenreich. Back-patch to all supported versions. http://git.postgresql.org/pg/commitdiff/24ebc444c61306f50777f674544e8559e765ad81
  • Fix nasty performance problem in tsquery_rewrite(). tsquery_rewrite() tries to find matches to subsets of AND/OR conditions; for example, in the query 'a | b | c' the substitution subquery 'a | c' should match and lead to replacement of the first and third items. That's fine, but the matching algorithm apparently takes about O(2^N) for an N-clause query (I say "apparently" because the code is also both unintelligible and uncommented). We could probably do better than that even without any extra assumptions --- but actually, we know that the subclauses are sorted, indeed are depending on that elsewhere in this very same function. So we can just scan the two lists a single time to detect matches, as though we were doing a merge join. Also do a re-flattening call (QTNTernary()) in tsquery_rewrite_query, just to make sure that the tree fits the expectations of the next search cycle. I didn't try to devise a test case for this, but I'm pretty sure that the oversight could have led to failure to match in some cases where a match would be expected. Improve comments, and also stick a CHECK_FOR_INTERRUPTS into dofindsubquery, just in case it's still too slow for somebody. Per report from Andreas Seltenreich. Back-patch to all supported branches. Discussion: <8760oasf2y.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/5ec81aceec8cc5a0edcc4609ee8edbc43a47e878

Álvaro Herrera pushed:

Magnus Hagander pushed:

Robert Haas pushed:

  • postgres_fdw: Try again to stabilize aggregate pushdown regression tests. A query that only aggregates one row isn't a great argument for pushdown, and buildfarm member brolga decides against it. Adjust the query a bit in the hopes of getting remote aggregation to win consistently. Jeevan Chalke, per suggestion from Tom Lane http://git.postgresql.org/pg/commitdiff/f5d6bce63ceb3c59a964814bb0df5a0648e750e5
  • Fix possible pg_basebackup failure on standby with "include WAL". If a restartpoint flushed no dirty buffers, it could fail to update the minimum recovery point, leading to a minimum recovery point prior to the starting REDO location. perform_base_backup() would interpret that as meaning that no WAL files at all needed to be included in the backup, failing an internal sanity check. To fix, have restartpoints always update the minimum recovery point to just after the checkpoint record itself, so that the file (or files) containing the checkpoint record will always be included in the backup. Code by Amit Kapila, per a design suggestion by me, with some additional work on the code comment by me. Test case by Michael Paquier. Report by Kyotaro Horiguchi. http://git.postgresql.org/pg/commitdiff/f267c1c2447bb8da6e4a6b2fcbb612762c3579a8
  • If the stats collector dies during Hot Standby, restart it. This bug exists as far back as 9.0, when Hot Standby was introduced, so back-patch to all supported branches. Report and patch by Takayuki Tsunakawa, reviewed by Michael Paquier and Kuntal Ghosh. http://git.postgresql.org/pg/commitdiff/4f714b2fd2a580d909607de889ac822956eb8299
  • Fix leftover reference to background writer performing checkpoints. This was changed in PostgreSQL 9.2, but somehow this comment never got updated. http://git.postgresql.org/pg/commitdiff/33839b5ffbd7be56681f31d107ec8238c4a0494a
  • pgstattuple: Don't take heavyweight locks when examining a hash index. It's currently necessary to take a heavyweight lock when scanning a hash bucket, but pgstattuple only examines individual pages, so it doesn't need to do this. If, for some hypothetical reason, it did need to do any heavyweight locking here, this logic would probably still be incorrect, because most of the locks that it is taking are meaningless. Only a heavyweight lock on a primary bucket page has any meaning, but this takes heavyweight locks on all pages regardless of function - and in particular overflow pages, where you might imagine that we'd want to lock the primary bucket page if we needed to lock anything at all. This is arguably a bug that has existed since this code was added in commit dab42382f483c3070bdce14a4d93c5d0cf61e82b, but I'm not going to bother back-patching it because in most cases the only consequence is that running pgstattuple() on a hash index is a little slower than it otherwise might be, which is no big deal. Extracted from a vastly larger patch by Amit Kapila which heavyweight locking for hash indexes entirely; analysis of why this can be done independently of the rest by me. http://git.postgresql.org/pg/commitdiff/d4b5d4caddb73f954d9ef86641201dc99677719d

Bruce Momjian pushed:

Peter Eisentraut pushed:

Heikki Linnakangas pushed:

  • Support multi-dimensional arrays in PL/python. Multi-dimensional arrays can now be used as arguments to a PL/python function (used to throw an error), and they can be returned as nested Python lists. This makes a backwards-incompatible change to the handling of composite types in arrays. Previously, you could return an array of composite types as "[[col1, col2], [col1, col2]]", but now that is interpreted as a two- dimensional array. Composite types in arrays must now be returned as Python tuples, not lists, to resolve the ambiguity. I.e. "[(col1, col2), (col1, col2)]". To avoid breaking backwards-compatibility, when not necessary, () is still accepted for arrays at the top-level, but it is always treated as a single-dimensional array. Likewise, [] is still accepted for composite types, when they are not in an array. Update the documentation to recommend using [] for arrays, and () for composite types, with a mention that those other things are also accepted in some contexts. This needs to be mentioned in the release notes. Alexey Grishchenko, Dave Cramer and me. Reviewed by Pavel Stehule. Discussion: <CAH38_tmbqwaUyKs9yagyRra=SMaT45FPBxk1pmTYcM0TyXGG7Q@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/94aceed317730953476bec490ce0148b2af3c383
  • Give a hint, when [] is incorrectly used for a composite type in array. That used to be accepted, so let's try to give a hint to users on why their PL/python functions no longer work. Reviewed by Pavel Stehule. Discussion: <CAH38_tmbqwaUyKs9yagyRra=SMaT45FPBxk1pmTYcM0TyXGG7Q@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/510e1b8ecf2a6f0d91d50f41f6b7fd75242273a0
  • Fix typos in comments. Vinayak Pokale http://git.postgresql.org/pg/commitdiff/56f39009c53e752493ca4478449a1311865dd51a
  • Fix typo in comment. Daniel Gustafsson http://git.postgresql.org/pg/commitdiff/8a2f08fbeaf75f67da122b49f05f96257df3faed
  • Fix regression test to also work with Python 2. Per buildfarm. http://git.postgresql.org/pg/commitdiff/e131ba4fe58123ce5726c1405486913b429c068c
  • Only treat Python Lists as array dimensions. Instead of treating all python sequence types as array dimensions, except for tuples and various kinds of strings, only treat Python lists as dimensions. The PyBytes_Check() function used previously is only available on Python 2.6 and newer, and it was a bit fiddly anyway. The list of exceptions would require adjustment if Python got a new kind of a sequence similar to bytes/unicodes/strings, so only checking for Lists seems more future-proof. The documentation only mentioned using Lists, so this is closer to what was documented, anyway. This should fix the buildfarm failures on systems building with Python 2.5, although I don't have Python 2.5 installed myself to test with. http://git.postgresql.org/pg/commitdiff/cfd9c87a54b0d414a59a912710c6633313174e51
  • Avoid using platform-dependent floats in test case. The number of decimals printed for floats varied in this test case, as noted by several buildfarm members. There's nothing special about floats and arrays in the code being tested, so replace the floats with numerics to make the output platform-independent. http://git.postgresql.org/pg/commitdiff/73c8e8506cd1933ccf5c5d61088ca171a5f414c0
  • Remove platform-dependent PL/python test case. Turns out that the output format of Python Decimal isn't totally platform- independent either. There are other tests for multi-dimensional arrays, so rather than try to fix this test case, just remove it. Per buildfarm member prairiedog. http://git.postgresql.org/pg/commitdiff/8eb6337f9f53c85e707f60157e42fcacfe927668

Tatsuo Ishii pushed:

Correctifs en attente

Samuel D. Leslie sent in a patch to add a radiustimeout parameter for RADIUS HBA.

Tsutomu Yamada and Ashutosh Bapat traded patches to change a returned NULL to NIL in create_foreignscan_path(), and a similar issue in a create_foreignscan_path() call in add_foreign_grouping_paths().

Petr Jelinek sent in another revision of a patch to implement logical replication.

Petr Jelinek sent in another revision of a patch to follow timeline switches in logical decoding and add a test harness for same.

Laurenz Albe sent in two more revisions of a patch to add PGDLLEXPORT to PG_FUNCTION_INFO_V1 function declarations.

Haribabu Kommi sent in another revision of a patch to add macaddr 64 bit (EUI-64) datatype support.

Robert Haas and Victor Wagner traded patches to implement failover on the libpq connect level.

Peter Geoghegan sent in another revision of a patch to cap the number of tapes used by external sorts, add parallel B-tree index build sorting, and add a force_btree_randomaccess GUC for testing.

Amit Kapila sent in two more revisions of a patch to extend the bufmgr api to include hash index.

Ashutosh Sharma sent in a patch to add microvacuum support for hash indexes.

Amit Kapila sent in another revision of a patch to add group update clog.

Kyotaro HORIGUCHI sent in three more revisions of a patch to implement asynchronous execution.

Noah Misch sent in a patch to firmly recommend PG_GETARG_TEXT_PP() over PG_GETARG_TEXT_P().

Tomas Vondra sent in another revision of a patch to add two slab-like memory allocators.

Venkata B Nagothi sent in a patch to add a recovery_target_incomplete GUC.

Julian Markwort sent in another revision of a patch to allow specifying a password file location for libpq-based applications.

Karl O. Pinc and Gilles Darold traded patches to implement a pg_current_logfile() function.

Heikki Linnakangas and Kyotaro HORIGUCHI traded patches to implement a radix tree for character conversion.

Thomas Munro sent in another revision of a patch to add a replay_lag column to the pg_stat_replication view.

Claudio Freire sent in another revision of a patch to refetch buffers on backward scan and allow using more than 1GB work mem in VACUUM.

Peter Eisentraut sent in a patch to add NLS to the remaining bin programs which were moved from contrib a while ago.

Rushabh Lathia sent in another revision of a patch to implement gather merge.

Etsuro Fujita sent in another revision of a patch to push down more full joins to the postgres FDW.

Ashutosh Bapat sent in another revision of a patch to pgstat to avoid writing on SIGQUIT.

Peter Eisentraut sent in a patch to add a user-callable function to import operation system collations.

Michaël Paquier sent in a patch to apply XLR_INFO_MASK correctly when looking at WAL record information.

Peter Eisentraut sent in a patch to add rules to download raw UNICODE mapping files.

Peter Eisentraut sent in a patch to split up psql's \d modifiers column into more legible pieces.

Haribabu Kommi sent in another revision of a patch to add a pg_hba_file_settings view.

Michaël Paquier sent in another revision of a patch to add a wal_consistency GUC with associated machinery.

Mithun Cy sent in two revisions of a patch to add auto_prewarm.

Peter Eisentraut sent in a patch to save redundant code for pseudotype I/O functions.

Ashutosh Bapat sent in a patch to free unused paths in the case of partition-wise join for join between (declaratively) partitioned tables.

Amit Langote sent in another revision of a patch to implement declarative partitioning for tables.

Julien Rouhaud sent in a patch to remove pq_putmessage_hook and pq_flush_hook.

Magnus Hagander and Michaël Paquier traded patches to fix an issue where symlinks in the wrong place could cause streaming backups to fail.

Tomas Vondra sent in another revision of a patch to implement multivariate statistics.

Michaël Paquier sent in another revision of a patch to mention column name in error messages.

Kevin Grittner sent in another revision of a patch to construct and enable the use of delta relations in AFTER triggers.

Masahiko Sawada sent in another revision of a patch to support 2PC across FDWs.

Kyotaro HORIGUCHI sent in another revision of a patch to refactor tab completion in psql to use a more principled approach to grammar.

par N Bougain le vendredi 4 novembre 2016 à 13h12

mardi 1 novembre 2016

Damien Clochard

PostgreSQL Magazine présente : The Paper Elephant #01

Le projet PostgreSQL Magazine revien avec un concept original nommé project is back The Paper Elephant : un journal pliant et dynamique.

Comme d’habitude, le contenu provient directement de membres de la communauté PostgreSQL : Josh Berkus parle de la nouvelle politique de numérotation, Hans-Jürgen Schönig écrit sur les performances et Craig Kierstens présente un tour d’horizon JSON, JSONB et hstore. Vous trouverez également une interview de Paul Ramsey à propos de PostGIS.

Cette édition a été créée grace à l’aide financière de PostgreSQL Europe et de nombreux contributeurs bénévoles ( rédacteurs, éditeurs, relecteurs ). Un grand merci à tous !

Comme son l’indique The Paper Elephant est avant tout un “media papier” destiné à être imprimé et distribué pendant les événements informatiques (conférences, meetup, etc.) pour promouvoir PostgreSQL.

La première édition sera distribuée à PostgreSQL Conference Europe 2016, Paris Open Source Summit 2016 et FOSDEM 2017. Si vous voulez la distribuer à l’occasion d’une rencontre informatique près de chez vous, Contactez nous !

Pour avoir un aperçu du journal, vous pouvez télécharger la version PDF ici : http://pgmag.org/download

The Paper Elephant est un projet ouvert et collectif. Rejoignez-nous via http://www.pgmag.org/contribute

par Damien Clochard le mardi 1 novembre 2016 à 15h17

jeudi 27 octobre 2016

Actualités PostgreSQL.fr

Sortie de PostgreSQL 9.6.1 et Fin de Vie de PostgreSQL 9.1

Paris, le 27 octobre 2017

Le PostgreSQL Global Development Group a publié une mise à jour pour toutes les versions supportées de notre système de base de données, les versions 9.6.1, 9.5.5, 9.4.10, 9.3.15, 9.2.19 et 9.1.24, où la version 9.1.24 est la dernière de la branche 9.1. Cette version corrige un problème d’enregistrement dans les journaux de transaction (WAL) des relations tronquées pouvant conduire à une corruption de données. Elle résout aussi un certain nombre de bugs rapportés sur les trois derniers mois. Les utilisateurs qui sont affectés par cette corruption de données doivent faire la mise à jour immédiatement. Les autres devraient planifier cette mise à jour à la prochaine interruption de service planifiée.

Enregistrement dans les WAL des relations tronquées

Cette mise à jour corrige l’enregistrement dans les journaux de transaction (WAL) des relations tronquées, et assure maintenant que la carte des epaces libres (Free Space Map ou FSM ) est elle aussi tronquée lorsqu’une commande TRUNCATE est envoyée, qui conduisait à la corruption de données. Si la FSM n’était pas tronquée, une base PostgreSQL en mode réparation (recovery) pouvait retourner une page qui avait déjà été tronquées et retourner une erreur du type :

ERROR:  could not read block 28991 in file "base/16390/572026": read only 0 of 8192 bytes

Si les checksum sont activés, des échecs de checksum sur la carte de visibilité (Visibility Map ou VM) peuvent aussi survenir.

Ce problème est présent dans les séries 9.3, 9.4, 9.5 et 9.6 des publications de PostgreSQL.

Problèmes avec pg_upgrade sur les machines big-endian

Sur les machines big-endian (ex : plusieurs architectures non-Intel), pg_upgrade pourrait écrire de manière incorrecte les octets de la carte de visibilité conduisant pg_upgrade à l’échec.

Ce problème est présent uniquement dans les versions 9.6.0 de PostgreSQL.

Correctifs de bug et améliorations

En plus de ce qui énoncé ci-dessus, cette mise à jour corrige aussi un certain nombre de bugs rapportés sur les derniers mois. Certains problèmes ne touchent que la version 9.6, mais certains touchent toutes les versions supportées. Il y a plus de 50 correctifs dans cette version, incluant :

  • Correction d’un risque d’utilisation de variable après libération (“use-after-free”) dans l’exécution des fonctions d’agrégats utilisant DISTINCT, pouvant conduire à un crash.
  • Correction d’une manipulation incorrecte des agrégats polymorphes utilisés comme fonctions fenêtrées, pouvant conduire à des crashs.
  • Correction de la création incorrecte des index GIN dans les enregistrements WAL sur les machines big-endian
  • Correction de la perte de descripteur de fichier lorsqu’une relation temporaire de plus d’1 Go est tronquée
  • Correction d’une fuite sur la durée de vie d’une requête en mémoire lors d’un UPDATE de masse sur une table avec une PRIMARY KEY ou un index REPLICA IDENTITY
  • Correction des SELECT FOR UPDATE/SHARE pour verrouiller correctement les enregistrements qui ont été mis à jour par une transaction qui a été annulée par la suite
  • Correction de COPY sur la liste de nom de colonne d’une table qui a la sécurité au niveau ligne (RLS) activée
  • Correction de la suppression d’enregistrements TOAST spéculativement insérés en retour d’un INSERT … ON CONFLICT
  • Correction de la longueur du timeout quand un VACUUM est en attente d’un verrou exclusif sur une table pour qu’il puisse tronquer la table
  • Correction de problèmes dans la fusion de contraintes CHECK héritées lorsque une table est créée ou altérée
  • Correction du remplacement d’un tableau d’éléments dans jsonb_set()
  • Correction d’une possible erreur de tri lors de l’interruption de l’utilisation des clés abrégées dans les index btree
  • Sur Windows, nouvel essai de création du segment de control en mémoire partagée dynamique après une erreur sur accès refusé
  • Correction d’une erreur de calcul de la latence moyenne dans pgbench
  • Fait en sorte que pg_receivexlog fonctionne correctement avec –synchronous sans slots
  • Force pg_rewind à désactiver synchronous_commit dans sa session sur le serveur source
  • Suppression des tentatives de partage des contextes SSL au travers de multiples connections dans la libpq
  • Support de OpenSSL 1.1.0
  • Installation de l’infrastructure de tests TAP de façon à ce qu’elle soit disponible pour le test des extensions
  • Plusieurs corrections pour le décodage logique des WAL et les slots de réplication
  • Plusieurs corrections de problèmes mineurs dans pg_dump, pg_xlogdump, et pg_upgrade
  • Plusieurs corrections de problèmes mineurs dans le planificateur de requêtes et dans la sortie d’EXPLAIN
  • Plusieurs corrections dans le support de timezone

Cette mise à jour contient aussi le tzdata 2016h pour les changements légaux de DST en Palestine et Turquie, et des corrections historiques pour la Turquie et quelques régions de Russie. Basculement sur des abbréviations numériques pour quelques time zones en Antarctica, l’ancienne Union Soviétique, et le Sri Lanka.

La base de données des time zones de l’IANA renvoyait précédemment des abréviations textuelles pour les time zones, faisant que parfois ces abréviations n’avaient que peu ou pas cours auprès de la population locale. L’IANA est en cours de renversement de cette politique en faveur de l’utilisation des offsets UTC numériques dans les zones où il n’y a pas d’évidence de l’utilisation d’abréviation anglaise dans le monde réel. Pour le moment tout au moins, PostgreSQL continuera d’accepter les abréviations supprimées pour l’entrée de timestamp. Mais elles ne seront pas montrées dans la vue pg_timezone_names ni utilisés en sortie.

Dans cette mise à jour, AMT n’est plus montrée comme étant en cours d’utilisation pour signifier le temps en Arménie. Par conséquent, nous avons changé l’abréviation pour quelle soit interprétée par défaut comme Amazon Time, donc UTC-4 et pas UTC+4.

Notification d’EOL (End Of Life, fin de vie) pour la version 9.1.

PostgreSQL 9.1 est maintenant en fin de vie (EOL). Le projet prévoit de ne plus sortir aucune version supplémentaire pour cette version. Nous encourageons fortement les utilisateurs à commencer à planifier une mise à jour vers une version plus récente dès que possible. Voyez la politique de version pour plus d’informations

Mise à jour

Toutes les mises à jour de PostgreSQL sont cumulatives. Comme pour les autres versions mineures, les utilisateurs n’ont pas besoin d’exporter et réimporter leur base, ni d’utiliser pg_upgrade pour appliquer cette mise à jour. Vous pouvez simplement éteindre PostgreSQL et mettre à jour les binaires.

Si votre système était affecté par le bug pg_upgrade sur big-endian, merci de lire https://wiki.postgresql.org/wiki/Visibility_Map_Problems et suivez les instructions sur comment corriger ce problème sur vos instances PostgreSQL.

Les utilisateurs qui ont sauté une ou plusieurs mises à jour mineures pourraient avoir besoin de lancer des étapes supplémentaires après la mise à jour; merci de consulter les notes de version des versions précédentes pour plus de détails.

Liens

par daamien le jeudi 27 octobre 2016 à 13h14

mercredi 26 octobre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 23 octobre 2016

PostgreSQL@SCaLE est un événement de 2 jours à double programmes qui aura lieu les 2 & 3 mars 2017 au centre de convention de Pasadena, intégré au SCaLE 15X. L'appel à conférenciers court jusqu'au 15 novembre 2016 : http://www.socallinuxexpo.org/scale/15x/cfp

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en octobre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161023230118.GA30097@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

  • Replace PostmasterRandom() with a stronger way of generating randomness. This adds a new routine, pg_strong_random() for generating random bytes, for use in both frontend and backend. At the moment, it's only used in the backend, but the upcoming SCRAM authentication patches need strong random numbers in libpq as well. pg_strong_random() is based on, and replaces, the existing implementation in pgcrypto. It can acquire strong random numbers from a number of sources, depending on what's available: - OpenSSL RAND_bytes(), if built with OpenSSL - On Windows, the native cryptographic functions are used - /dev/urandom - /dev/random Original patch by Magnus Hagander, with further work by Michael Paquier and me. Discussion: <CAB7nPqRy3krN8quR9XujMVVHYtXJ0_60nqgVc6oUk8ygyVkZsA@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/9e083fd4683294f41544e6d0d72f6e258ff3a77c
  • Fix use-after-free around DISTINCT transition function calls. Have tuplesort_gettupleslot() copy the contents of its current table slot as needed. This is based on an approach taken by tuplestore_gettupleslot(). In the future, tuplesort_gettupleslot() may also be taught to avoid copying the tuple where caller can determine that that is safe (the tuplestore_gettupleslot() interface already offers this option to callers). Patch by Peter Geoghegan. Fixes bug #14344, reported by Regina Obe. Report: <20160929035538.20224.39628@wrigleys.postgresql.org> Backpatch-through: 9.6 http://git.postgresql.org/pg/commitdiff/d8589946ddd5c43e1ebd01c5e92d0e177cbfc198
  • Revert "Replace PostmasterRandom() with a stronger way of generating randomness." This reverts commit 9e083fd4683294f41544e6d0d72f6e258ff3a77c. That was a few bricks shy of a load: * Query cancel stopped working * Buildfarm member pademelon stopped working, because the box doesn't have /dev/urandom nor /dev/random. This clearly needs some more discussion, and a quite different patch, so revert for now. http://git.postgresql.org/pg/commitdiff/faae1c918e8aaae034eaf3ea103fcb6ba9adc5ab
  • Fix WAL-logging of FSM and VM truncation. When a relation is truncated, it is important that the FSM is truncated as well. Otherwise, after recovery, the FSM can return a page that has been truncated away, leading to errors like: ERROR: could not read block 28991 in file "base/16390/572026": read only 0 of 8192 bytes We were using MarkBufferDirtyHint() to dirty the buffer holding the last remaining page of the FSM, but during recovery, that might in fact not dirty the page, and the FSM update might be lost. To fix, use the stronger MarkBufferDirty() function. MarkBufferDirty() requires us to do WAL-logging ourselves, to protect from a torn page, if checksumming is enabled. Also fix an oversight in visibilitymap_truncate: it also needs to WAL-log when checksumming is enabled. Analysis by Pavan Deolasee. Discussion: <CABOikdNr5vKucqyZH9s1Mh0XebLs_jRhKv6eJfNnD2wxTn=_9A@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/917dc7d2393ce680dea7a59418be9ff341df3c14
  • Use OpenSSL EVP API for symmetric encryption in pgcrypto. The old "low-level" API is deprecated, and doesn't support hardware acceleration. And this makes the code simpler, too. Discussion: <561274F1.1030000@iki.fi> http://git.postgresql.org/pg/commitdiff/5ff4a67f63fd6d3eb01ff9707d4674ed54a89f3b

Robert Haas pushed:

  • By default, set log_line_prefix = '%m [%p] '. This value might not be to everyone's taste; in particular, some people might prefer %t to %m, and others may want %u, %d, or other fields. However, it's a vast improvement on the old default of ''. Christoph Berg http://git.postgresql.org/pg/commitdiff/7d3235ba42f8d5fc70c58e242702cc5e2e3549a6
  • Fix typo in comment. Amit Langote http://git.postgresql.org/pg/commitdiff/fca41acb86902b90218dcc3bc0ffc462850a090f
  • Improve regression test coverage for hash indexes. On my system, this improves coverage for src/backend/access/hash from 61.3% of lines to 88.2% of lines, and from 83.5% of functions to 97.5% of functions, which is pretty good for 36 lines of tests. Mithun Cy, reviewing by Amit Kapila and Álvaro Herrera http://git.postgresql.org/pg/commitdiff/b801e120080de836b834c1b756c4c4d81ce841b5
  • Remove a comment which is now incorrect. Before 5d305d86bd917723f09ab4f15c075d90586a210a, this comment was correct, but now it says we do something which we don't actually do. Accordingly, remove the comment. http://git.postgresql.org/pg/commitdiff/ec7db2b483e0ff247ed41612cdb5716022401fe6
  • ename "pg_xlog" directory to "pg_wal". "xlog" is not a particularly clear abbreviation for "write-ahead log", and it sometimes confuses users into believe that the contents of the "pg_xlog" directory are not critical data, leading to unpleasant consequences. So, rename the directory to "pg_wal". This patch modifies pg_upgrade and pg_basebackup to understand both the old and new directory layouts; the former is necessary given the purpose of the tool, while the latter merely avoids an unnecessary backward-compatibility break. We may wish to consider renaming other programs, switches, and functions which still use the old "xlog" naming to also refer to "wal". However, that's still under discussion, so let's do just this much for now. Discussion: CAB7nPqTeC-8+zux8_-4ZD46V7YPwooeFxgndfsq5Rg8ibLVm1A@mail.gmail.com Michael Paquier http://git.postgresql.org/pg/commitdiff/f82ec32ac30ae7e3ec7c84067192535b2ff8ec0e
  • postgres_fdw: Push down aggregates to remote servers. Now that the upper planner uses paths, and now that we have proper hooks to inject paths into the upper planning process, it's possible for foreign data wrappers to arrange to push aggregates to the remote side instead of fetching all of the rows and aggregating them locally. This figures to be a massive win for performance, so teach postgres_fdw to do it. Jeevan Chalke and Ashutosh Bapat. Reviewed by Ashutosh Bapat with additional testing by Prabhat Sahu. Various mostly cosmetic changes by me. http://git.postgresql.org/pg/commitdiff/7012b132d07c2b4ea15b0b3cb1ea9f3278801d98
  • Fix comment formatting. http://git.postgresql.org/pg/commitdiff/919c811ca1e2a545cb1db243af93d55270d84469
  • postgres_fdw: Attempt to stabilize regression results. Set enable_hashagg to false for tests involving least_agg(), so that we get the same plan regardless of local costing variances. Also, remove a test involving sqrt(); it's there to test deparsing of HAVING clauses containing expressions, but that's tested elsewhere anyway, and sqrt(2) deparses with different amounts of precision on different machines. Per buildfarm. http://git.postgresql.org/pg/commitdiff/ad13a09d762f0c903a52ed0ec668a0ba51a61047

Tom Lane pushed:

  • Fix cidin() to handle values above 2^31 platform-independently. CommandId is declared as uint32, and values up to 4G are indeed legal. cidout() handles them properly by treating the value as unsigned int. But cidin() was just using atoi(), which has platform-dependent behavior for values outside the range of signed int, as reported by Bart Lengkeek in bug #14379. Use strtoul() instead, as xidin() does. In passing, make some purely cosmetic changes to make xidin/xidout look more like cidin/cidout; the former didn't have a monopoly on best practice IMO. Neither xidin nor cidin make any attempt to throw error for invalid input. I didn't change that here, and am not sure it's worth worrying about since neither is really a user-facing type. The point is just to ensure that indubitably-valid inputs work as expected. It's been like this for a long time, so back-patch to all supported branches. Report: <20161018152550.1413.6439@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/6f13a682c86801cfb9ae4f3126888b42f3cb5c46
  • Update time zone data files to tzdata release 2016g. DST law changes in Turkey. Historical corrections for America/Los_Angeles, Europe/Kirov, Europe/Moscow, Europe/Samara, and Europe/Ulyanovsk. Rename Asia/Rangoon to Asia/Yangon, with a backward compatibility link. The IANA crew continue their campaign to replace invented time zone abbrevations with numeric GMT offsets. This update changes numerous zones in Antarctica and the former Soviet Union, for instance Antarctica/Casey now reports "+08" not "AWST" in the pg_timezone_names view. I kept these abbreviations in the tznames/ data files, however, so that we will still accept them for input. (We may want to start trimming those files someday, but today is not that day.) An exception is that since IANA no longer claims that "AMT" is in use in Armenia for GMT+4, I replaced it in the Default file with GMT-4, corresponding to Amazon Time which is in use in South America. It may be that that meaning is also invented and IANA will drop it in a future update; but for now, it seems silly to give pride of place to a meaning not traceable to IANA over one that is. http://git.postgresql.org/pg/commitdiff/ecbac3e6e038e990f24a2e0eacdcd6738292105f
  • Suppress "Factory" zone in pg_timezone_names view for tzdata >= 2016g. IANA got rid of the really silly "abbreviation" and replaced it with one that's only moderately silly. But it's still pointless, so keep on not showing it. http://git.postgresql.org/pg/commitdiff/a3215431ab7c667bf581728f10c80a36abbe1d5a
  • Windows portability fix. Per buildfarm. http://git.postgresql.org/pg/commitdiff/ad90ac4d671d320ade3c127f215e97cd49c307fb
  • Sync our copy of the timezone library with IANA release tzcode2016g. This is mostly to absorb some corner-case fixes in zic for year-2037 timestamps. The other changes that have been made are unlikely to affect our usage, but nonetheless we may as well take 'em. http://git.postgresql.org/pg/commitdiff/f3094920a567cde6c86adf36a1a033d7431b11ff
  • Another portability fix for tzcode2016g update. clang points out that SIZE_MAX wouldn't fit into an int, which means this comparison is pretty useless. Per report from Thomas Munro. http://git.postgresql.org/pg/commitdiff/23ed2ba8121178474f8c51774c6c258cb165a562
  • Update time zone data files to tzdata release 2016h. (Didn't I just do this? Oh well.) DST law changes in Palestine. Historical corrections for Turkey. Switch to numeric abbreviations for Asia/Colombo. http://git.postgresql.org/pg/commitdiff/d8fc45bd0f62fcebac80c63840b753f8e3b737ff
  • Sync our copy of the timezone library with IANA release tzcode2016h. This absorbs a fix for a symlink-manipulation bug in zic that was introduced in 2016g. It probably isn't interesting for our use-case, but I'm not quite sure, so let's update while we're at it. http://git.postgresql.org/pg/commitdiff/5e21b6811148fdc1fce9dcdcdc777418cc901fe4
  • Fix EXPLAIN so that it doesn't emit invalid XML in corner cases. With track_io_timing = on, EXPLAIN (ANALYZE, BUFFERS) will emit fields named like "I/O Read Time". The slash makes that invalid as an XML element name, so that adding FORMAT XML would produce invalid XML. We already have code in there to translate spaces to dashes, so let's generalize that to convert anything that isn't a valid XML name character, viz letters, digits, hyphens, underscores, and periods. We could just reject slashes, which would run a bit faster. But the fact that this went unnoticed for so long doesn't give me a warm feeling that we'd notice the next creative violation, so let's make it a permanent fix. Reported by Markus Winand, though this isn't his initial patch proposal. Back-patch to 9.2 where track_io_timing was added. The problem is only latent in 9.1, so I don't feel a need to fix it there. Discussion: <E0BF6A45-68E8-45E6-918F-741FB332C6BB@winand.at> http://git.postgresql.org/pg/commitdiff/709e461befa8a4999c4ccdbfc7260ef8092e805c
  • Doc: wording tweak for PERL, PYTHON, TCLSH configuration variables. Replace "Full path to ..." with "Full path name of ...". At least one user has misinterpreted the existing wording as meaning "Directory containing ...". http://git.postgresql.org/pg/commitdiff/7aa2c10ac6785a2de683609b98da607e588a6d02
  • First-draft release notes for 9.6.1. As usual, the release notes for other branches will be made by cutting these down, but put them up for community review first. http://git.postgresql.org/pg/commitdiff/eacaf6e29fd2a3047aff9738a35a8e9b05e55375
  • Improve documentation about use of Linux huge pages. Show how to get the system's huge page size, rather than misleadingly referring to PAGE_SIZE (which is usually understood to be the regular page size). Show how to confirm whether huge pages have been allocated. Minor wordsmithing. Back-patch to 9.4 where this section appeared. http://git.postgresql.org/pg/commitdiff/1885c88459698251eca64f095d9942c540ba0fa8
  • Avoid testing tuple visibility without buffer lock in RI_FKey_check(). Despite the argumentation I wrote in commit 7a2fe85b0, it's unsafe to do this, because in corner cases it's possible for HeapTupleSatisfiesSelf to try to set hint bits on the target tuple; and at least since 8.2 we have required the buffer content lock to be held while setting hint bits. The added regression test exercises one such corner case. Unpatched, it causes an assertion failure in assert-enabled builds, or otherwise would cause a hint bit change in a buffer we don't hold lock on, which given the right race condition could result in checksum failures or other data consistency problems. The odds of a problem in the field are probably pretty small, but nonetheless back-patch to all supported branches. Report: <19391.1477244876@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/6292c2339186bac215bab5a1f01370f9735582c1

Andres Freund pushed:

Peter Eisentraut pushed:

Magnus Hagander pushed:

Correctifs en attente

Haribabu Kommi and Vinayak Pokale traded patches to add a pg_stat_sql system view.

Dilip Kumar sent in two more revisions of a patch to add parallel bitmap heap scan.

Masahiko Sawada sent in another revision of a patch to add quorum commit for multiple synchronous replication.

Kyotaro HORIGUCHI sent in another revision of a patch to implement asynchronous execution.

Laurenz Albe sent in a patch to add PGDLLEXPORT to a sample C function.

Ashutosh Bapat and Etsuro Fujita traded patches to ensure that altering a foreign table invalidates plans involving same.

Aleksander Alekseev sent in two revisions of a patch to enable logging the contents of COPY statements.

Heikki Linnakangas and Michaël Paquier traded patches to fix an FSM corruption leading to errors.

Gilles Darold sent in another revision of a patch to implement pg_current_logfile().

Dmitry Dolgov sent in another revision of a patch to implement generic type subscripting.

Michaël Paquier sent in two more revisions of a patch to implement SCRAM auth.

Rushabh Lathia sent in another revision of a patch to add Gather Merge.

Thom Brown sent in another revision of a patch to implement failover on the libpq connect level.

Jeevan Chalke and Ashutosh Bapat traded patches to implement aggregate pushdown.

Peter Geoghegan sent in a patch to fix ON CONFLICT bugs at higher isolation levels.

Oleksandr Shulgin sent in a patch to prevent psql from sending commands after a connection reset.

Vinayak Pokale sent in a patch to fix a typo in pgstat.h.

Vinayak Pokale sent in a patch to fix a typo in pgstat.c.

Constantin S. Pan and Michaël Paquier traded patches to fix the fact that there can be a lot of orphan temp tables.

Tomas Vondra sent in two more revisions of a patch to add two slab-like memory allocators.

Kyotaro HORIGUCHI sent in another revision of a patch to implement a radix tree for character conversion.

Masahiko Sawada sent in a patch to allow specifying the log file name of pgbench -l option.

Julien Rouhaud sent in a patch to fix the fact that when track_commit_timestamp is enabled, the oldestCommitTsXid and newestCommitTsXid don't persist after a server restart.

Bruce Momjian sent in a patch to mention pg_reload_conf() as a way to reload configurations in the docs.

Ashutosh Bapat sent in another revision of a patch to allow pushing down more FULL JOINs to the the PostgreSQL FDW.

Michaël Paquier sent in another revision of a patch to rename pg_clog to pg_xact and pg_subtrans to pg_subxact.

par N Bougain le mercredi 26 octobre 2016 à 21h18

vendredi 21 octobre 2016

Nicolas Gollet

PG Session #8 les videos à télécharger

Les vidéos de la "PostgreSQL Session #8" organisée par Dalibo et Oslandia viennent d'être mise en ligne sur youtube !

Afin de vous permettre de visionner ces vidéos partout (dans le train, l'avions, aux fins font de l'Ardèche ;) ), je vous propose de pouvoir les télécharger au format "mp4" directement depuis ce partage one drive.

Au programme 8 conférences :
1. Paul Leroux - Postgres comme NoSql Db - le meilleur des deux mondes
2. Nicolas Ribot - Traitements spatiaux parallélisés pour les gros volumes de données
3. Ludovic Delauné - Point Clouds - modèles, chargement, stockage et manipulation de données
4. Tom van Tilburg - From point clouds and vectordata to 3D models with the help of postgres
5. Giuseppe Broccolo & Julien Rouhaud - BRIN indexes on geospatial database
6. Giulio Calacoci - Step up and face the disaster (pgbarman)
7. Julien Tachoires - Temboard - supervision, administration, dashboard et bien plus
8. Olivier Courtin - De l'analyse spatiale avancée à la GéoDataScience en environnement PostgreSQL - PostGIS

Ces vidéos ont été produites par Dalibo et sont sous licence Creative Commons BY-NC-SA n'hésitez donc pas à les partager!

Merci à Télé Millevaches pour leur travail de captation vidéo et de montage ;)

Bon visionnage à toutes et à tous!

par Nicolas GOLLET le vendredi 21 octobre 2016 à 10h34

mardi 18 octobre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 16 octobre 2016

[ndt: Meetup à Paris le 10 novembre : http://www.meetup.com/fr-FR/PostgreSQL-User-Group-Paris/]

Offres d'emplois autour de PostgreSQL en octobre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161016222644.GB13559@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Peter Eisentraut pushed:

Heikki Linnakangas pushed:

Tom Lane pushed:

  • In PQsendQueryStart(), avoid leaking any left-over async result. Ordinarily there would not be an async result sitting around at this point, but it appears that in corner cases there can be. Considering all the work we're about to launch, it's hardly going to cost anything noticeable to check. It's been like this forever, so back-patch to all supported branches. Report: <CAD-Qf1eLUtBOTPXyFQGW-4eEsop31tVVdZPu4kL9pbQ6tJPO8g@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/886f6c5ccdb500eeeec7e0abdf1500e20a304c45
  • Update user docs for switch to POSIX semaphores. Since commit ecb0d20a9 hasn't crashed and burned, here's the promised docs update for it. In addition to explaining that Linux and FreeBSD ports now use POSIX semaphores, I did some wordsmithing on pre-existing wording; in particular trying to clarify which SysV parameters need to be set with an eye to total usage across all applications. http://git.postgresql.org/pg/commitdiff/3d21f08bccd316c3850a1943c1ee1e381dab1588
  • Improve documentation for CREATE RECURSIVE VIEW. It was perhaps not entirely clear that internal self-references shouldn't be schema-qualified even if the view name is written with a schema. Spell it out. Discussion: <871sznz69m.fsf@metapensiero.it> http://git.postgresql.org/pg/commitdiff/e34318725ca5b274efd6f57ea7460e89f4dca9f9
  • Docs: grammatical fix. Fix poor grammar introduced in 741ccd501. http://git.postgresql.org/pg/commitdiff/c7e56811fa38cbc39efd6bdd4bb45f2f0444803e
  • Remove "sco" and "unixware" ports. SCO OpenServer and SCO UnixWare are more or less dead platforms. We have never had a buildfarm member testing the "sco" port, and the last "unixware" member was last heard from in 2012, so it's fair to doubt that the code even compiles anymore on either one. Remove both ports. We can always undo this if someone shows up with an interest in maintaining and testing these platforms. Discussion: <17177.1476136994@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/2b860f52ed1b1784cdf3f03886805f5bf250ea74
  • Drop server support for FE/BE protocol version 1.0. While this isn't a lot of code, it's been essentially untestable for a very long time, because libpq doesn't support anything older than protocol 2.0, and has not since release 6.3. There's no reason to believe any other client-side code still uses that protocol, either. Discussion: <2661.1475849167@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/2f1eaf87e868a1c42f2b159958623daa6a666de4
  • Provide DLLEXPORT markers for C functions via PG_FUNCTION_INFO_V1 macro. This isn't really necessary for our own code, because we use a .DEF file in MSVC builds (see gendef.pl), or --export-all-symbols in MinGW and Cygwin builds, to ensure that all global symbols in loadable modules will be exported on Windows. However, third-party authors might use different build processes that need this marker, and it's harmless enough for our own builds. To some extent, this is an oversight in commit e7128e8db, so back-patch to 9.4 where that was added. Laurenz Albe. Discussion: <A737B7A37273E048B164557ADEF4A58B539300BD@ntex2010a.host.magwien.gv.at> http://git.postgresql.org/pg/commitdiff/8518583cdb10340bab3464121629a1a9ec387afa
  • Remove unnecessary int2vector-specific hash function and equality operator. These functions were originally added in commit d8cedf67a to support use of int2vector columns as catcache lookup keys. However, there are no catcaches that use such columns. (Indeed I now think it must always have been dead code: a catcache with such a key column would need an underlying unique index on the column, but we've never had an int2vector btree opclass.) Getting rid of the int2vector-specific operator and function does not lose any functionality, because operations on int2vectors will now fall back to the generic anyarray support. This avoids a wart that a btree index on an int2vector column (made using anyarray_ops) would fail to match equality searches, because int2vectoreq wasn't a member of the opclass. We don't really care much about that, since int2vector is not meant as a type for users to use, but it's silly to have extra code and less functionality. If we ever do want a catcache to be indexed by an int2vector column, we'd need to put back full btree and hash opclasses for int2vector, comparable to the support for oidvector. (The anyarray code can't be used at such a low level, because it needs to do catcache lookups.) But we'll deal with that if/when the need arises. Also worth noting is that removal of the hash int2vector_ops opclass will break any user-created hash indexes on int2vector columns. While hash anyarray_ops would serve the same purpose, it would probably not compute the same hash values and thus wouldn't be on-disk-compatible. Given that int2vector isn't a user-facing type and we're planning other incompatible changes in hash indexes for v10 anyway, this doesn't seem like something to worry about, but it's probably worth mentioning here. Amit Langote Discussion: <d9bb74f8-b194-7307-9ebd-90645d377e45@lab.ntt.co.jp> http://git.postgresql.org/pg/commitdiff/5c80642aa8de8393b08cd3cbf612b325cedd98dc
  • Remove pg_dump/pg_dumpall support for dumping from pre-8.0 servers. The need for dumping from such ancient servers has decreased to about nil in the field, so let's remove all the code that catered to it. Aside from removing a lot of boilerplate variant queries, this allows us to not have to cope with servers that don't have (a) schemas or (b) pg_depend. That means we can get rid of assorted squishy code around that. There may be some nonobvious additional simplifications possible, but this patch already removes about 1500 lines of code. I did not remove the ability for pg_restore to read custom-format archives generated by these old versions (and light testing says that that does still work). If you have an old server, you probably also have a pg_dump that will work with it; but you have an old custom-format backup file, that might be all you have. It'd be possible at this point to remove fmtQualifiedId()'s version argument, but I refrained since that would affect code outside pg_dump. Discussion: <2661.1475849167@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/64f3524e2c8deebc02808aa5ebdfa17859473add
  • pg_dump's getTypes() needn't retrieve typinput or typoutput anymore. Commit 64f3524e2 removed the stanza of code that examined these values. I failed to notice they were unnecessary because my compiler didn't warn about the un-read variables. Noted by Peter Eisentraut. http://git.postgresql.org/pg/commitdiff/c0a3b211bcb790a8d76022cb2b3ffe9795aaf5e9
  • Revert addition of PGDLLEXPORT in PG_FUNCTION_INFO_V1 macro. This turns out not to be as harmless as I thought: MSVC will complain if it sees an "extern" declaration without PGDLLEXPORT and then one with. (Seems fairly silly, given that this can be changed after the fact by the linker, but there you have it.) Therefore, contrib modules that have extern's for V1 functions in header files are falling over in the buildfarm, since none of those externs are marked PGDLLEXPORT. We might or might not conclude that we're willing to plaster those declarations with PGDLLEXPORT in HEAD, but in any case there's no way we're going to ship this change in the back branches. Third-party authors would not thank us for breaking their code in a minor release. Hence, revert the addition of PGDLLEXPORT (but let's keep the extra info in the comment). If we do the other changes we can revert this commit in HEAD. Per buildfarm. http://git.postgresql.org/pg/commitdiff/4f52fd3c6d28a4380a5afc51ae6f774c91837a38
  • Fix broken jsonb_set() logic for replacing array elements. Commit 0b62fd036 did a fairly sloppy job of refactoring setPath() to support jsonb_insert() along with jsonb_set(). In its defense, though, there was no regression test case exercising the case of replacing an existing element in a jsonb array. Per bug #14366 from Peng Sun. Back-patch to 9.6 where bug was introduced. Report: <20161012065349.1412.47858@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/9c4cc9e2c75e84a8267396be5cccbfe25b8f63f6
  • Fix pg_dumpall regression test to be locale-independent. The expected results in commit b4fc64578 seem to have been generated in a non-C locale, which just points up the fact that the ORDER BY clause was locale-sensitive. Per buildfarm. http://git.postgresql.org/pg/commitdiff/0a4bf6b192377bef8e69c92d0c95434a91509f12
  • Clean up handling of anonymous mmap'd shared-memory segment. Fix detaching of the mmap'd segment to have its own on_shmem_exit callback, rather than piggybacking on the one for detaching from the SysV segment. That was confusing, and given the distance between the two attach calls, it was trouble waiting to happen. Make the detaching calls idempotent by clearing AnonymousShmem to show we've already unmapped. I spent quite a bit of time yesterday trying to find a path that would allow the munmap()'s to be done twice, and while I did not succeed, it seems silly that there's even a question. Make the #ifdef logic less confusing by separating "do we want to use anonymous shmem" from EXEC_BACKEND. Even though there's no current scenario where those conditions are different, it is not helpful for different places in the same file to be testing EXEC_BACKEND for what are fundamentally different reasons. Don't do on_exit_reset() in StartBackgroundWorker(). At best that's useless (InitPostmasterChild would have done it already) and at worst it could zap some callback that's unrelated to shared memory. Improve comments, and simplify the huge_pages enablement logic slightly. Back-patch to 9.4 where hugepage support was introduced. Arguably this should go into 9.3 as well, but the code looks significantly different there, and I doubt it's worth the trouble of adapting the patch given I can't show a live bug. http://git.postgresql.org/pg/commitdiff/15fc5e15811337f5a81d4ae44c6149256f6dd15f
  • Try to find out the actual hugepage size when making a MAP_HUGETLB request. Even if Linux's mmap() is okay with a partial-hugepage request, munmap() is not, as reported by Chris Richards. Therefore it behooves us to try a bit harder to find out the actual hugepage size, instead of assuming that we can skate by with a guess. For the moment, just look into /proc/meminfo to find out the default hugepage size, and use that. Later, on kernels that support requests for nondefault sizes, we might try to consider other alternatives. But that smells more like a new feature than a bug fix, especially if we want to provide any way for the DBA to control it, so leave it for another day. I set this up to allow easy addition of platform-specific code for non-Linux platforms, if needed; but right now there are no reports suggesting that we need to work harder on other platforms. Back-patch to 9.4 where hugepage support was introduced. Discussion: <31056.1476303954@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/cb775768e3e37d466d69b7177a92508b81c1c204
  • Remove dead code in pg_dump. I'm not sure if this provision for "pg_backup" behaving a bit differently from "pg_dump" ever did anything useful in a released version. But it's definitely dead code now. Michael Paquier http://git.postgresql.org/pg/commitdiff/c08521eb55135c493cee23541c233870cdff14b7
  • Fix another bug in merging of inherited CHECK constraints. It's not good for an inherited child constraint to be marked connoinherit; that would result in the constraint not propagating to grandchild tables, if any are created later. The code mostly prevented this from happening but there was one case that was missed. This is somewhat related to commit e55a946a8, which also tightened checks on constraint merging. Hence, back-patch to 9.2 like that one. This isn't so much because there's a concrete feature-related reason to stop there, as to avoid having more distinct behaviors than we have to in this area. Amit Langote Discussion: <b28ee774-7009-313d-dd55-5bdd81242c41@lab.ntt.co.jp> http://git.postgresql.org/pg/commitdiff/3cca13cbfcc5312f7ae1728213e197c6f37ca62a
  • Fix handling of pgstat counters for TRUNCATE in a prepared transaction. pgstat_twophase_postcommit is supposed to duplicate the math in AtEOXact_PgStat, but it had missed out the bit about clearing t_delta_live_tuples/t_delta_dead_tuples for a TRUNCATE. It's harder than you might think to replicate the issue here, because those counters would only be nonzero when a previous transaction in the same backend had added/deleted tuples in the truncated table, and those counts hadn't been sent to the stats collector yet. Evident oversight in commit d42358efb. I've not added a regression test for this; we tried to add one in d42358efb, and had to revert it because it was too timing-sensitive for the buildfarm. Back-patch to 9.5 where d42358efb came in. Stas Kelvich Discussion: <EB57BF68-C06D-4737-BDDC-4BA778F4E62B@postgrespro.ru> http://git.postgresql.org/pg/commitdiff/81e82a2bd48865f9a294b63d9492f9fde6a32787
  • Fix assorted integer-overflow hazards in varbit.c. bitshiftright() and bitshiftleft() would recursively call each other infinitely if the user passed INT_MIN for the shift amount, due to integer overflow in negating the shift amount. To fix, clamp to -VARBITMAXLEN. That doesn't change the results since any shift distance larger than the input bit string's length produces an all-zeroes result. Also fix some places that seemed inadequately paranoid about input typmods exceeding VARBITMAXLEN. While a typmod accepted by anybit_typmodin() will certainly be much less than that, at least some of these spots are reachable with user-chosen integer values. Andreas Seltenreich and Tom Lane Discussion: <87d1j2zqtz.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/32fdf42cf546f613aab9ca98935c40a046187fa9

Andres Freund pushed:

  • Make regression tests less dependent on hash table order. Upcoming changes to the hash table code used, among others, for grouping and set operations will change the output order for a few queries. To make it less likely that actual bugs are hidden between regression test ordering changes, and to make the tests robust against platform dependant ordering, add ORDER BYs guaranteeing the output order. As it's possible that some of the changes expose platform dependant ordering, push this earlier, to let the buildfarm shake out potentially unstable results. Discussion: <20160727004333.r3e2k2y6fvk2ntup@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/0137caf273f4297c4d36df3a542d7c0c853e75be
  • Fix further hash table order dependent tests. Similar to 0137caf273, this makes contrib and pl tests less dependant on hash-table order. After this commit, at least some order affecting changes to execGrouping.c don't result in regression test changes anymore. http://git.postgresql.org/pg/commitdiff/ccbb852cd6101e93976111676a90f1e5d9268951
  • Make pg_dumpall's database ACL query independent of hash table order. Previously GRANT order on databases was not well defined, due to the use of EXCEPT without an ORDER BY. Add an ORDER BY, adapt test output. I don't, at the moment, see reason to backpatch this. http://git.postgresql.org/pg/commitdiff/b4fc645787cc7c614c0c97fc9fffacf2bdc6a388
  • Use more efficient hashtable for tidbitmap.c to speed up bitmap scans. Use the new simplehash.h to speed up tidbitmap.c uses. For bitmap scan heavy queries speedups of over 100% have been measured. Both lossy and exact scans benefit, but the wins are bigger for mostly exact scans. The conversion is mostly trivial, except that tbm_lossify() now restarts lossifying at the point it previously stopped. Otherwise the hash table becomes unbalanced because the scan in done in hash-order, leaving the end of the hashtable more densely filled then the beginning. That caused performance issues with dynahash as well, but due to the open chaining they were less pronounced than with the linear adressing from simplehash.h. Reviewed-By: Tomas Vondra Discussion: <20160727004333.r3e2k2y6fvk2ntup@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/75ae538bc3168bf44475240d4e0487ee2f3bb376
  • Add likely/unlikely() branch hint macros. These are useful for very hot code paths. Because it's easy to guess wrongly about likelihood, and because such likelihoods change over time, they should be used sparingly. Past tests have shown it'd be a good idea to use them in some places, e.g. in error checks around ereports that ERROR out, but that's work for later. Discussion: <20160727004333.r3e2k2y6fvk2ntup@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/aa3ca5e3dd60bf0b992b74f955378f28e601292a
  • Add a macro templatized hashtable. dynahash.c hash tables aren't quite fast enough for some use-cases. There are several reasons for lacking performance: The use of chaining for collision handling makes them cache inefficient, that's especially an issue when the tables get bigger. As the element sizes for dynahash are only determined at runtime, offset computations are somewhat expensive Hash and element comparisons are indirect function calls, causing unnecessary pipeline stalls It's two level structure has some benefits (somewhat natural partitioning), but increases the number of indirections to fix several of these the hash tables have to be adjusted to the individual use-case at compile-time. C unfortunately doesn't provide a good way to do compile code generation (like e.g. c++'s templates for all their weaknesses do). Thus the somewhat ugly approach taken here is to allow for code generation using a macro-templatized header file, which generates functions and types based on a prefix and other parameters. Later patches use this infrastructure to use such hash tables for tidbitmap.c (bitmap scans) and execGrouping.c (hash aggregation, ...). In queries where these use up a large fraction of the time, this has been measured to lead to performance improvements of over 100%. There are other cases where this could be useful (e.g. catcache.c). The hash table design chosen is a variant of linear open-addressing. The biggest disadvantage of simple linear addressing schemes are highly variable lookup times due to clustering, and deletions leaving a lot of tombstones around. To address these issues a variant of "robin hood" hashing is employed. Robin hood hashing optimizes chaining lengths by moving elements close to their optimal bucket ("rich" elements), out of the way if a to-be-inserted element is further away from its optimal position (i.e. it's "poor"). While that can make insertions slower, the average lookup performance is a lot better, and higher fill factors can be used in a still performant manner. To avoid tombstones - which normally solve the issue that a deleted node's presence is relevant to determine whether a lookup needs to continue looking or is done - buckets following a deleted element are shifted backwards, unless they're empty or already at their optimal position. There's further possible improvements that can be made to this implementation. Amongst others: Use distance as a termination criteria during searches. This is generally a good idea, but I've been able to see the overhead of distance calculations in some cases. Consider combining the 'empty' status into the hashvalue, and enforce storing the hashvalue. That could, in some cases, increase memory density and remove a few instructions. Experiment further with the, very conservatively choosen, fillfactor. Make maximum size of hashtable configurable, to allow storing very very large tables. That'd require 64bit hash values to be more common than now, though. some smaller memcpy calls could be optimized to copy larger chunks. But since the new implementation is already considerably faster than dynahash it seem sensible to start using it. Reviewed-By: Tomas Vondra Discussion: <20160727004333.r3e2k2y6fvk2ntup@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/b30d3ea824c5ccba43d3e942704f20686e7dbab8
  • Use more efficient hashtable for execGrouping.c to speed up hash aggregation. The more efficient hashtable speeds up hash-aggregations with more than a few hundred groups significantly. Improvements of over 120% have been measured. Due to the the different hash table queries that not fully determined (e.g. GROUP BY without ORDER BY) may change their result order. The conversion is largely straight-forward, except that, due to the static element types of simplehash.h type hashes, the additional data some users store in elements (e.g. the per-group working data for hash aggregaters) is now stored in TupleHashEntryData->additional. The meaning of BuildTupleHashTable's entrysize (renamed to additionalsize) has been changed to only be about the additionally stored size. That size is only used for the initial sizing of the hash-table. Reviewed-By: Tomas Vondra Discussion: <20160727004333.r3e2k2y6fvk2ntup@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5

Robert Haas pushed:

Tatsuo Ishii pushed:

Correctifs en attente

Michaël Paquier sent in a patch to use pg_ctl promote -w in TAP tests.

Michaël Paquier sent in a patch to fix some FSM corruption.

Pavel Stěhule sent in another revision of a patch to add \setfileref to psql.

Heikki Linnakangas sent in two more revisions of a patch to simplify the tape block format, and add pause/resume support for tapes.

Thomas Munro sent in another revision of a patch to enable DISTINCT with btree skip scan.

Mithun Cy sent in another revision of a patch to add some tests to cover hash_index.

Amit Langote sent in a patch to remove the int2vector operator hash infrastructure.

Haribabu Kommi sent in a patch to add a 64-bit (EUI-64) MAC address data type.

Dilip Kumar sent in a patch to enable scan key push down to heap.

Haribabu Kommi sent in a patch to add a pg_stat_sql system view.

Jim Nasby sent in a patch to add support for SRFs and returning composites from pl/tcl.

Robert Haas sent in another revision of a patch to bump up max-parallel-workers.

Amit Kapila sent in a patch to enables parallelism for btree scans.

Prabhat Kumar Sahu and Jeevan Chalke traded patches to do aggregate pushdown to a foreign server.

Aleksander Alekseev sent in a patch to fix some infelicities in pg_filedump.

Michaël Paquier sent in another revision of a patch to rename pg_xlog to pg_wal and two alternatives for renaming pg_clog: pg_transaction, and pg_xact.

Heikki Linnakangas sent in a patch to replace the polyphase merge algorithm with a simple balanced k-way merge.

Michaël Paquier and Heikki Linnakangas traded patches to add SCRAM auth.

Michaël Paquier sent in a patch to improve OOM handling in pg_locale.c.

Mithun Cy sent in another revision of a patch to implement failover on libpq connect level.

Etsuro Fujita sent in another revision of a patch to push down more full joins in postgres_fdw.

Peter Eisentraut sent in a patch to simplify internal archive version handling in pg_dump.

Etsuro Fujita sent in a patch to clarify the mention of self-joins in the DELETE documentation.

Pavel Stěhule sent in another revision of a patch to add xmltable().

Ashutosh Bapat sent in another revision of a patch to optimize partition-wise JOINs in tables with declarative partitions.

Michaël Paquier sent in a patch to improve the durability of pg_dump(all).

Stas Kelvich sent in a patch to fix some of the logging for 2PC.

Christoph Berg sent in a patch to choose a non-empty default log_line_prefix.

Peter Eisentraut sent in a patch to make getrusage() output a little more readable.

Heikki Linnakangas sent in another revision of a patch to support multi-dimensional arrays in PL/python and give a hint when [] is incorrectly used for a composite type in array.

Magnus Hagander sent in another revision of a patch to enable pg_basebackup to stream xlog to tar.

Jeff Janes sent in a patch to change the auth check in postgres_fdw to something a little stricter.

Jim Nasby sent in a patch to return a bitmask of NULL fields in a record.

par N Bougain le mardi 18 octobre 2016 à 20h51

lundi 10 octobre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 9 octobre 2016

[ndt: Meetup à Nantes le 22 novembre : http://www.meetup.com/fr-FR/PostgreSQL-User-Group-Nantes/]

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en octobre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161009225109.GB15525@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Tom Lane pushed:

  • Enforce a specific order for probing library loadability in pg_upgrade. pg_upgrade checks whether all the shared libraries used in the old cluster are also available in the new one by issuing LOAD for each library name. Previously, it cared not what order it did the LOADs in. Ideally it should not have to care, but currently the transform modules in contrib fail unless both the language and datatype modules they depend on are loaded first. A backend-side solution for that looks possible but probably not back-patchable, so as a stopgap measure, let's do the LOAD tests in order by library name length. That should fix the problem for reasonably-named transform modules, eg "hstore_plpython" will be loaded after both "hstore" and "plpython". (Yeah, it's a hack.) In a larger sense, having a predictable order of these probes is a good thing, since it will make upgrades predictably work or not work in the face of inter-library dependencies. Also, this patch replaces O(N^2) de-duplication logic with O(N log N) logic, which could matter in installations with very many databases. So I don't foresee reverting this even after we have a proper fix for the library-dependency problem. In passing, improve a couple of SQL queries used here. Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib modules failed. Back-patch to 9.5 where transform modules were introduced. Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net> http://git.postgresql.org/pg/commitdiff/83c2492002162bf79d2a0811bff5724e395909d7
  • Show a sensible value in pg_settings.unit for GUC_UNIT_XSEGS variables. Commit 88e982302 invented GUC_UNIT_XSEGS for min_wal_size and max_wal_size, but neglected to make it display sensibly in pg_settings.unit (by adding a case to the switch in GetConfigOptionByNum). Fix that, and adjust said switch to throw a run-time error the next time somebody forgets. In passing, avoid using a static buffer for the output string --- the rest of this function pstrdup's from a local buffer, and I see no very good reason why the units code should do it differently and less safely. Per report from Otar Shavadze. Back-patch to 9.5 where the new unit type was added. Report: <CAG-jOyA=iNFhN+yB4vfvqh688B7Tr5SArbYcFUAjZi=0Exp-Lg@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/6bc811c992a804bdb8d228ce0be9f0a8e7198df6
  • Convert contrib/hstore_plpython to not use direct linking to other modules. Previously, on most platforms, we allowed hstore_plpython's references to hstore and plpython to be unresolved symbols at link time, trusting the dynamic linker to resolve them when the module is loaded. This has a number of problems, the worst being that the dynamic linker does not know where the references come from and can do nothing but fail if those other modules haven't been loaded. We've more or less gotten away with that for the limited use-case of datatype transform modules, but even there, it requires some awkward hacks, most recently commit 83c249200. Instead, let's not treat these references as linker-resolvable at all, but use function pointers that are manually filled in by the module's _PG_init function. There are few enough contact points that this doesn't seem unmaintainable, at least for these use-cases. (Note that the same technique wouldn't work at all for decoupling from libpython itself, but fortunately that's just a standard shared library and can be linked to normally.) This is an initial patch that just converts hstore_plpython. If the buildfarm doesn't find any fatal problems, I'll work on the other transform modules soon. Tom Lane, per an idea of Andres Freund's. Discussion: <2652.1475512158@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/d51924be886c2a05e691fa05b16cb6b30ab8370f
  • Fix hstore_plpython for Python 3. In commit d51924be8, I overlooked the need to provide linkage for PLyUnicode_FromStringAndSize, because that's only used (and indeed only exists) in Python 3 builds. In light of the need to #if this item, rearrange the ordering of the code related to each function pointer, so as not to need more #if's than absolutely necessary. Per buildfarm. http://git.postgresql.org/pg/commitdiff/490ed1ebb9b26fb342a1e3240107092e9c5aec02
  • Improve (I hope) our autolocation of the Python shared library. Older versions of Python produce garbage (or at least useless) values of get_config_vars('LDLIBRARY'). Newer versions produce garbage (or at least useless) values of get_config_vars('SO'), which was defeating our configure logic that attempted to identify where the Python shlib really is. The net result, at least with a stock Python 3.5 installation on macOS, was that we were linking against a static library in the mistaken belief that it was a shared library. This managed to work, if you count statically absorbing libpython into plpython.so as working. But it no longer works as of commit d51924be8, because now we get separate static copies of libpython in plpython.so and hstore_plpython.so, and those can't interoperate on the same data. There are some other infelicities like assuming that nobody ever installs a private version of Python on a macOS machine. Hence, forget about looking in $python_configdir for the Python shlib; as far as I can tell no version of Python has ever put one there, and certainly no currently-supported version does. Also, rather than relying on get_config_vars('SO'), just try all the possibilities for shlib extensions. Also, rather than trusting Py_ENABLE_SHARED, believe we've found a shlib only if it has a recognized extension. Last, explicitly cope with the possibility that the shlib is really in /usr/lib and $python_libdir is a red herring --- this is the actual situation on older macOS, but we were only accidentally working with it. Discussion: <5300.1475592228@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/46ddbbb1177a7e6ce5a4fe0ef8fa8ac49f36d0bb
  • Avoid direct cross-module links in hstore_plperl and ltree_plpython, too. Just turning the crank on the project started in commit d51924be8. These cases turn out to be exact subsets of the boilerplate needed for hstore_plpython. Discussion: <2652.1475512158@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/eda04886c1e048d695728206504ab4198462168e
  • Huh, we do need to look in $python_configdir for the Python shlib. Debian does it that way, for no doubt what seems to them a good reason. Thanks to Aidan Van Dyk for confirmation. http://git.postgresql.org/pg/commitdiff/fc76259f5b8473dbd3d2009b0e4a267cf3a7e704
  • In python shlib probe, cater for OpenBSD, which omits the .so symlink. Most Unix-oid platforms provide a symlink "libfoo.so" -> "libfoo.so.n.n" to allow the linker to resolve a reference "-lfoo" to a version-numbered shared library. OpenBSD has apparently hacked ld(1) to do this internally, because there are no such symlinks to be found in their library directories. Tweak the new code in PGAC_CHECK_PYTHON_EMBED_SETUP to cope. Per buildfarm member curculio. http://git.postgresql.org/pg/commitdiff/ddd4f82cb6f65354776541dfac3bedf680e0e303
  • Try to fix python shlib probe for MinGW. Per discussion with Andrew Dunstan, it seems there are three peculiarities of the Python installation on MinGW (or at least, of the instance on buildfarm member frogmouth). First, the library name doesn't contain "2.7" but just "27". It looks like we can deal with that by consulting get_config_vars('VERSION') to see whether a dot should be used or not. Second, the library might be in c:/Windows/System32, analogously to it possibly being in /usr/lib on Unix-oid platforms. Third, it's apparently not standard to use the prefix "lib" on the file name. This patch will accept files with or without "lib", but the first part of that may well be dead code. http://git.postgresql.org/pg/commitdiff/11c0e743b67acc9a0406b6058ed24de5e5527cf0
  • Remove -Wl,-undefined,dynamic_lookup in macOS build. We don't need this anymore, and it prevents build-time error checking that's usually good to have, so remove it. Undoes one change of commit cac765820. Unfortunately, it's much harder to get a similar effect on other common platforms, because we don't want the linker to throw errors for symbols that will be resolved in the core backend. Only macOS and AIX expect the core backend executable to be available while linking loadable modules, so only these platforms can usefully throw errors for unresolved symbols at link time. Discussion: <2652.1475512158@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/bfe2e847811a4d0a46c8cdf16195c0e5929b6f83
  • Fix pg_dump to work against pre-9.0 servers again. getBlobs' queries for pre-9.0 servers were broken in two ways: the 7.x/8.x query uses DISTINCT so it can't have unspecified-type NULLs in the target list, and both that query and the 7.0 one failed to provide the correct output column labels, so that the subsequent code to extract data from the PGresult would fail. Back-patch to 9.6 where the breakage was introduced (by commit 23f34fa4b). Amit Langote and Tom Lane Discussion: <0a3e7a0e-37bd-8427-29bd-958135862f0a@lab.ntt.co.jp> http://git.postgresql.org/pg/commitdiff/4806f26f9e4273888cbf85f42b3bdc5e9d950ba6
  • Fix python shlib probe for Cygwin. On buildfarm member cockatiel, that library is in /usr/bin. (Possibly we should look at $PATH on this platform?) Per off-list report from Andrew Dunstan. http://git.postgresql.org/pg/commitdiff/17a3a1eb0efc5d84c81e46a26fe6bd21dbe90de9
  • libpqwalreceiver needs to link with libintl when using --enable-nls. The need for this was previously obscured even on picky platforms by the hack we used to support direct cross-module references in the transforms contrib modules. Now that that hack is gone, the undefined symbol is exposed, as reported by Robert Haas. Back-patch to 9.5 where we started to use -Wl,-undefined,dynamic_lookup. I'm a bit surprised that the older branches don't seem to contain any gettext references in this module, but since they don't fail at build time, they must not. (We might be able to get away with leaving this alone in 9.5/9.6, but I think it's cleaner if the reference gets resolved at link time.) Report: <CA+TgmoaHJKU5kcWZcYduATYVT7Mnx+8jUnycaYYL7OtCwCigug@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/8811f5d3a4a1ebf79ccb00da336d70041b003dd2
  • Fix two bugs in merging of inherited CHECK constraints. Historically, we've allowed users to add a CHECK constraint to a child table and then add an identical CHECK constraint to the parent. This results in "merging" the two constraints so that the pre-existing child constraint ends up with both conislocal = true and coninhcount > 0. However, if you tried to do it in the other order, you got a duplicate constraint error. This is problematic for pg_dump, which needs to issue separated ADD CONSTRAINT commands in some cases, but has no good way to ensure that the constraints will be added in the required order. And it's more than a bit arbitrary, too. The goal of complaining about duplicated ADD CONSTRAINT commands can be served if we reject the case of adding a constraint when the existing one already has conislocal = true; but if it has conislocal = false, let's just make the ADD CONSTRAINT set conislocal = true. In this way, either order of adding the constraints has the same end result. Another problem was that the code allowed creation of a parent constraint marked convalidated that is merged with a child constraint that is !convalidated. In this case, an inheritance scan of the parent table could emit some rows violating the constraint condition, which would be an unexpected result given the marking of the parent constraint as validated. Hence, forbid merging of constraints in this case. (Note: valid child and not-valid parent seems fine, so continue to allow that.) Per report from Benedikt Grundmann. Back-patch to 9.2 where we introduced possibly-not-valid check constraints. The second bug obviously doesn't apply before that, and I think the first doesn't either, because pg_dump only gets into this situation when dealing with not-valid constraints. Report: <CADbMkNPT-Jz5PRSQ4RbUASYAjocV_KHUWapR%2Bg8fNvhUAyRpxA%40mail.gmail.com> Discussion: <22108.1475874586@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/e55a946a81c6648c5ff3a0ccdda242f639e33c6f
  • Fix incorrect handling of polymorphic aggregates used as window functions. The transfunction was told that its first argument and result were of the window function output type, not the aggregate state type. This'd only matter if the transfunction consults get_fn_expr_argtype, which typically only polymorphic functions would do. Although we have several regression tests around polymorphic aggs, none of them detected this mistake --- in fact, they still didn't fail when I injected the same mistake into nodeAgg.c. So add some more tests covering both plain agg and window-function-agg cases. Per report from Sebastian Luque. Back-patch to 9.6 where the error was introduced (by sloppy refactoring in commit 804163bc2, looks like). Report: <87int2qkat.fsf@gmail.com> http://git.postgresql.org/pg/commitdiff/ac4a9d92fcb6869e757cc729dca2ca5ccf94b185
  • Use unnamed POSIX semaphores, if available, on Linux and FreeBSD. We've had support for using unnamed POSIX semaphores instead of System V semaphores for quite some time, but it was not used by default on any platform. Since many systems have rather small limits on the number of SysV semaphores allowed, it seems desirable to switch to POSIX semaphores where they're available and don't create performance or kernel resource problems. Experimentation by me shows that unnamed POSIX semaphores are at least as good as SysV semaphores on Linux, and we previously had a report from Maksym Sobolyev that FreeBSD is significantly worse with SysV semaphores than POSIX ones. So adjust those two platforms to use unnamed POSIX semaphores, if configure can find the necessary library functions. If this goes well, we may switch other platforms as well, but it would be advisable to test them individually first. It's not currently contemplated that we'd encourage users to select a semaphore API for themselves, but anyone who wants to experiment can add PREFERRED_SEMAPHORES=UNNAMED_POSIX (or NAMED_POSIX, or SYSV) to their configure command line to do so. I also tweaked configure to report which API it's selected, mainly so that we can tell that from buildfarm reports. I did not touch the user documentation's discussion about semaphores; that will need some adjustment once the dust settles. Discussion: <8536.1475704230@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/ecb0d20a9d2e09b7112d3b192047f711f9ff7e59

Stephen Frost pushed:

  • Fix RLS with COPY (col1, col2) FROM tab Attempting to COPY a subset of columns from a table with RLS enabled would fail due to an invalid query being constructed (using a single ColumnRef with the list of fields to exact in 'fields', but that's for the different levels of an indirection for a single column, not for specifying multiple columns). Correct by building a ColumnRef and then RestTarget for each column being requested and then adding those to the targetList for the select query. Include regression tests to hopefully catch if this is broken again in the future. Patch-By: Adam Brightwell Reviewed-By: Michael Paquier http://git.postgresql.org/pg/commitdiff/814b9e9b8edf36cac65e0d8fcef17e50a04b1617

Andres Freund pushed:

  • Correct logical decoding restore behaviour for subtransactions. Before initializing iteration over a subtransaction's changes, the last few changes were not spilled to disk. That's correct if the transaction didn't spill to disk, but otherwise... This bug can lead to missed or misorderd subtransaction contents when they were spilled to disk. Move spilling of the remaining in-memory changes to ReorderBufferIterTXNInit(), where it can easily be applied to the top transaction and, if present, subtransactions. Since this code had too many bugs already, noticeably increase test coverage. Fixes: #14319 Reported-By: Huan Ruan Discussion: <20160909012610.20024.58169@wrigleys.postgresql.org> Backport: 9,4-, where logical decoding was added http://git.postgresql.org/pg/commitdiff/61633f79048d040aefeed68dcac0e0b996a06189
  • Fix fallback implementation of pg_atomic_write_u32(). I somehow had assumed that in the spinlock (in turn possibly using semaphores) based fallback atomics implementation 32 bit writes could be done without a lock. As far as the write goes that's correct, since postgres supports only platforms with single-copy atomicity for aligned 32bit writes. But writing without holding the spinlock breaks read-modify-write operations like pg_atomic_compare_exchange_u32(), since they'll potentially "miss" a concurrent write, which can't happen in actual hardware implementations. In 9.6+ when using the fallback atomics implementation this could lead to buffer header locks not being properly marked as released, and potentially some related state corruption. I don't see a related danger in 9.5 (earliest release with the API), because pg_atomic_write_u32() wasn't used in a concurrent manner there. The state variable of local buffers, before this change, were manipulated using pg_atomic_write_u32(), to avoid unnecessary synchronization overhead. As that'd not be the case anymore, introduce and use pg_atomic_unlocked_write_u32(), which does not correctly interact with RMW operations. This bug only caused issues when postgres is compiled on platforms without atomics support (i.e. no common new platform), or when compiled with --disable-atomics, which explains why this wasn't noticed in testing. Reported-By: Tom Lane Discussion: <14947.1475690465@sss.pgh.pa.us> Backpatch: 9.5-, where the atomic operations API was introduced. http://git.postgresql.org/pg/commitdiff/b0779abb3add11d4dd745779dd81ea8ecdd00a1d

Heikki Linnakangas pushed:

  • Update comment. mergepreread()/mergeprereadone() don't exist anymore, the function that does roughly the same is now called mergereadnext(). http://git.postgresql.org/pg/commitdiff/c86c2d9d57c1e6c5fec860873734cccaf31df5b0
  • Fix another outdated comment. Preloading is done by logtape.c now. http://git.postgresql.org/pg/commitdiff/d4fca5e6c7363ba6ee4de7b8d72d68064fa864ca
  • Change the way pre-reading in external sort's merge phase works. Don't pre-read tuples into SortTuple slots during merge. Instead, use the memory for larger read buffers in logtape.c. We're doing the same number of READTUP() calls either way, but managing the pre-read SortTuple slots is much more complicated. Also, the on-tape representation is more compact than SortTuples, so we can fit more pre-read tuples into the same amount of memory this way. And we have better cache-locality, when we use just a small number of SortTuple slots. Now that we only hold one tuple from each tape in the SortTuple slots, we can greatly simplify the "batch memory" management. We now maintain a small set of fixed-sized slots, to hold the tuples, and fall back to palloc() for larger tuples. We use this method during all merge phases, not just the final merge, and also when randomAccess is requested, and also in the TSS_SORTEDONTAPE case. In other words, it's used whenever we do an external sort. Reviewed by Peter Geoghegan and Claudio Freire. Discussion: <CAM3SWZTpaORV=yQGVCG8Q4axcZ3MvF-05xe39ZvORdU9JcD6hQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/e94568ecc10f2638e542ae34f2990b821bbf90ac
  • Fix excessive memory consumption in the new sort pre-reading code. LogicalTapeRewind() should not allocate large read buffer, if the tape is completely empty. The calling code relies on that, for its calculation of how much memory to allocate for the read buffers. That lead to massive overallocation of memory, if maxTapes was high, but only a few tapes were actually used. Reported by Tomas Vondra Discussion: <7303da46-daf7-9c68-3cc1-9f83235cf37e@2ndquadrant.com> http://git.postgresql.org/pg/commitdiff/b56fb691b0033f9b86e0552bd5adfd485f05eef6
  • Disable synchronous commits in pg_rewind. If you point pg_rewind to a server that is using synchronous replication, with "pg_rewind --source-server=...", and the replication is not working for some reason, pg_rewind will get stuck because it creates a temporary table, which needs to be replicated. You could call broken replication a pilot error, but pg_rewind is often used in special circumstances, when there are changes to the replication setup. We don't do any "real" updates, and we don't care about fsyncing or replicating the operations on the temporary tables, so fix that by setting synchronous_commit off. Michael Banck, Michael Paquier. Backpatch to 9.5, where pg_rewind was introduced. Discussion: <20161005143938.GA12247@nighthawk.caipicrew.dd-dns.de> http://git.postgresql.org/pg/commitdiff/d7eb76b908a3aac68c23c7d3553c65c80e92b823
  • Don't share SSL_CTX between libpq connections. There were several issues with the old coding: 1. There was a race condition, if two threads opened a connection at the same time. We used a mutex around SSL_CTX_* calls, but that was not enough, e.g. if one thread SSL_CTX_load_verify_locations() with one path, and another thread set it with a different path, before the first thread got to establish the connection. 2. Opening two different connections, with different sslrootcert settings, seemed to fail outright with "SSL error: block type is not 01". Not sure why. 3. We created the SSL object, before calling SSL_CTX_load_verify_locations and SSL_CTX_use_certificate_chain_file on the SSL context. That was wrong, because the options set on the SSL context are propagated to the SSL object, when the SSL object is created. If they are set after the SSL object has already been created, they won't take effect until the next connection. (This is bug #14329) At least some of these could've been fixed while still using a shared context, but it would've been more complicated and error-prone. To keep things simple, let's just use a separate SSL context for each connection, and accept the overhead. Backpatch to all supported versions. Report, analysis and test case by Kacper Zuk. Discussion: <20160920101051.1355.79453@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/8bb14cdd33deecc6977cf5638f73e80f76ab658b
  • Clear OpenSSL error queue after failed X509_STORE_load_locations() call. Leaving the error in the error queue used to be harmless, because the X509_STORE_load_locations() call used to be the last step in initialize_SSL(), and we would clear the queue before the next SSL_connect() call. But previous commit moved things around. The symptom was that if a CRL file was not found, and one of the subsequent initialization steps, like loading the client certificate or private key, failed, we would incorrectly print the "no such file" error message from the earlier X509_STORE_load_locations() call as the reason. Backpatch to all supported versions, like the previous patch. http://git.postgresql.org/pg/commitdiff/275bf98601b230e530003ef20193d095b9309c24
  • Don't allow both --source-server and --source-target args to pg_rewind. They are supposed to be mutually exclusive, but there was no check for that. Michael Banck Discussion: <20161007103414.GD12247@nighthawk.caipicrew.dd-dns.de> http://git.postgresql.org/pg/commitdiff/0d4d7d61850f4f4bc5f6fd0b7a9adb70232aed61
  • Make TAP test suites to work, when @INC does not contain current dir. Recent Perl and/or new Linux distributions are starting to remove "." from the @INC list by default. That breaks pg_rewind and ssl test suites, which use helper perl modules that reside in the same directory. To fix, add the current source directory explicitly to prove's include dir. The vcregress.pl script probably also needs something like this, but I wasn't able to remove '.' from @INC on Windows to test this, and don't want to try doing that blindly. Discussion: <20160908204529.flg6nivjuwp5vaoy@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/d668b03378c28b927d0ba458681ca1b4c1e18e53
  • Remove bogus mapping from UTF-8 to SJIS conversion table. 0xc19c is not a valid UTF-8 byte sequence. It doesn't do any harm, AFAICS, but it's surely not intentional. No backpatching though, just to be sure. In the passing, also add a file header comment to the file, like the UCS_to_SJIS.pl script would produce. (The file was originally created with UCS_to_SJIS.pl, but has been modified by hand since then. That's questionable, but I'll leave fixing that for later.) Kyotaro Horiguchi Discussion: <20160907.155050.233844095.horiguchi.kyotaro@lab.ntt.co.jp> http://git.postgresql.org/pg/commitdiff/0aec7f9aec8b828e074b8f2f3cbea2ec3e5c0209

Robert Haas pushed:

Correctifs en attente

Fabien COELHO sent in a patch to allow backslash continuations in \set expressions in pgbench.

Amit Langote sent in a patch to un-include access/heapam.h.

Michaël Paquier sent in another revision of a patch to rename pg_xlog to pg_wal and rename pg_clog to pg_transaction.

Anastasia Lubennikova sent in two more revisions of a patch to add covering + unique indexes.

Julien Rouhaud sent in another revision of a patch to add the parallel worker class and change max_worker_processes default to 16 and max_parallel_workers to 8.

Thomas Munro sent in another revision of a patch to add condition variables.

Thomas Munro sent in a patch to create a DSA area to provide work space for parallel execution.

Rushabh Lathia sent in a patch to implement Gather Merge.

Michaël Paquier sent in another revision of a patch to fix hot standby checkpoints.

Thomas Munro sent in two more revisions of a patch to add dynamic shared memory areas.

Tomas Vondra sent in another revision of a patch to add two slab-like memory allocators.

Takayuki Tsunakawa sent in another revision of a patch to close listen ports earlier in the shutdown process.

Thomas Munro sent in two revisions of a patch to use DSA memory so that hash tables can be shared by multiple backends and yet grow dynamically. Development name: "DHT".

Etsuro Fujita sent in three more revisions of a patch to fix invalidation for prepared statements with FDWs.

Amit Langote sent in three more revisions of a patch to implement declarative partitioning.

Fabien COELHO sent in another revision of a patch to add more operators and functions to pgbench.

Robert Haas sent in a patch to improve on pgrminclude / pgcompinclude.

Daniel Gustafsson sent in a WIP patch to add Secure Transport support as OpenSSL alternative on macOS.

Vitaly Burovoy sent in a patch to fix wording in the docs on parallel queries.

Franck Verrot sent in another revision of a patch to mntion column name in error messages.

Ashutosh Bapat sent in a patch to pass input_rel->relids to fetch_upper_rel() in create_grouping_path().

Heikki Linnakangas sent in a patch to always use 2048 bit DH parameters for OpenSSL ephemeral DH ciphers.

Dilip Kumar sent in a patch to implement parallel bitmap heap scan.

Pavan Deolasee sent in a patch to prevent certain types of FSM corruption.

Gilles Darold sent in another revision of a patch to implement a pg_current_logfile() function.

Peter Geoghegan sent in another revision of a patch to add parallel tuplesort for parallel B-Tree index creation.

Jeff Janes sent in a patch to add a pgbench transaction function.

Pavel Stěhule sent in a patch to add a parallel status to psql's \df+.

Pavel Stěhule sent in another revision of a patch to add \setfileref to psql.

par N Bougain le lundi 10 octobre 2016 à 22h00

Nouvelles hebdomadaires de PostgreSQL - 2 octobre 2016

PostgreSQL 9.6 est disponible ! https://www.postgresql.org/docs/current/static/release-9-6.html

Les nouveautés des produits dérivés

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20161002231634.GB1769@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Tom Lane pushed:

  • Refer to OS X as "macOS", except for the port name which is still "darwin". We weren't terribly consistent about whether to call Apple's OS "OS X" or "Mac OS X", and the former is probably confusing to people who aren't Apple users. Now that Apple has rebranded it "macOS", follow their lead to establish a consistent naming pattern. Also, avoid the use of the ancient project name "Darwin", except as the port code name which does not seem desirable to change. (In short, this patch touches documentation and comments, but no actual code.) I didn't touch contrib/start-scripts/osx/, either. I suspect those are obsolete and due for a rewrite, anyway. I dithered about whether to apply this edit to old release notes, but those were responsible for quite a lot of the inconsistencies, so I ended up changing them too. Anyway, Apple's being ahistorical about this, so why shouldn't we be? http://git.postgresql.org/pg/commitdiff/da6c4f6ca88df346573bdada2aa2544510bf167e
  • Document has_type_privilege(). Evidently an oversight in commit 729205571. Back-patch to 9.2 where privileges for types were introduced. Report: <20160922173517.8214.88959@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/a4afb2b5c0b409bb175c20104b2ae9d47cf71be6
  • Replace the built-in GIN array opclasses with a single polymorphic opclass. We had thirty different GIN array opclasses sharing the same operators and support functions. That still didn't cover all the built-in types, nor did it cover arrays of extension-added types. What we want is a single polymorphic opclass for "anyarray". There were two missing features needed to make this possible: 1. We have to be able to declare the index storage type as ANYELEMENT when the opclass is declared to index ANYARRAY. This just takes a few more lines in index_create(). Although this currently seems of use only for GIN, there's no reason to make index_create() restrict it to that. 2. We have to be able to identify the proper GIN compare function for the index storage type. This patch proceeds by making the compare function optional in GIN opclass definitions, and specifying that the default btree comparison function for the index storage type will be looked up when the opclass omits it. Again, that seems pretty generically useful. Since the comparison function lookup is done in initGinState(), making use of the second feature adds an additional cache lookup to GIN index access setup. It seems unlikely that that would be very noticeable given the other costs involved, but maybe at some point we should consider making GinState data persist longer than it now does --- we could keep it in the index relcache entry, perhaps. Rather fortuitously, we don't seem to need to do anything to get this change to play nice with dump/reload or pg_upgrade scenarios: the new opclass definition is automatically selected to replace existing index definitions, and the on-disk data remains compatible. Also, if a user has created a custom opclass definition for a non-builtin type, this doesn't break that, since CREATE INDEX will prefer an exact match to opcintype over a match to ANYARRAY. However, if there's anyone out there with handwritten DDL that explicitly specifies _bool_ops or one of the other replaced opclass names, they'll need to adjust that. Tom Lane, reviewed by Enrique Meneses Discussion: <14436.1470940379@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/fdc9186f7ed1ead827509584f3b763f8dc332c43
  • Fix newly-introduced issues in pgbench. The result of FD_ISSET() doesn't necessarily fit in a bool, though assigning it to one might accidentally work depending on platform and which socket FD number is being inquired of. Rewrite to test it with if(), rather than making any specific assumption about the result width, to match the way every other such call in PG is written. Don't break out of the input_mask-filling loop after finding the first client that we're waiting for results from. That mostly breaks parallel query management. Also, if we choose not to call select(), be sure to clear out any bits the mask-filling loop might have set, so that we don't accidentally call doCustom for clients we don't know have input. Doing so would likely be harmless, but it's a waste of cycles and doesn't seem to be intended. Make this_usec wide enough. (Yeah, the value would usually fit in an int, but then why are we using int64 everywhere else?) Minor cosmetic adjustments, mostly comment improvements. Problems introduced by commit 12788ae49. The first issue was discovered by buildfarm testing, the others by code review. http://git.postgresql.org/pg/commitdiff/9779bda86c026e645773a3308f9169c7c0791f7c
  • Improve contrib/cube's handling of zero-D cubes, infinities, and NaNs. It's always been possible to create a zero-dimensional cube by converting from a zero-length float8 array, but cube_in failed to accept the '()' representation that cube_out produced for that case, resulting in a dump/reload hazard. Make it accept the case. Also fix a couple of other places that didn't behave sanely for zero-dimensional cubes: cube_size would produce 1.0 when surely the answer should be 0.0, and g_cube_distance risked a divide-by-zero failure. Likewise, it's always been possible to create cubes containing float8 infinity or NaN coordinate values, but cube_in couldn't parse such input, and cube_out produced platform-dependent spellings of the values. Convert them to use float8in_internal and float8out_internal so that the behavior will be the same as for float8, as we recently did for the core geometric types (cf commit 50861cd68). As in that commit, I don't pretend that this patch fixes all insane corner-case behaviors that may exist for NaNs, but it's a step forward. (This change allows removal of the separate cube_1.out and cube_3.out expected-files, as the platform dependency that previously required them is now gone: an underflowing coordinate value will now produce an error not plus or minus zero.) Make errors from cube_in follow project conventions as to spelling ("invalid input syntax for cube" not "bad cube representation") and errcode (INVALID_TEXT_REPRESENTATION not SYNTAX_ERROR). Also a few marginal code cleanups and comment improvements. Tom Lane, reviewed by Amul Sul Discussion: <15085.1472494782@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/f31a931fade868d788ef4480c59753a2d5059246
  • Redesign parallel dump/restore's wait-for-workers logic. The ListenToWorkers/ReapWorkerStatus APIs were messy and hard to use. Instead, make DispatchJobForTocEntry register a callback function that will take care of state cleanup, doing whatever had been done by the caller of ReapWorkerStatus in the old design. (This callback is essentially just the old mark_work_done function in the restore case, and a trivial test for worker failure in the dump case.) Then we can have ListenToWorkers call the callback immediately on receipt of a status message, and return the worker to WRKR_IDLE state; so the WRKR_FINISHED state goes away. This allows us to design a unified wait-for-worker-messages loop: WaitForWorkers replaces EnsureIdleWorker and EnsureWorkersFinished as well as the mess in restore_toc_entries_parallel. Also, we no longer need the fragile API spec that the caller of DispatchJobForTocEntry is responsible for ensuring there's an idle worker, since DispatchJobForTocEntry can just wait until there is one. In passing, I got rid of the ParallelArgs struct, which was a net negative in terms of notational verboseness, and didn't seem to be providing any noticeable amount of abstraction either. Tom Lane, reviewed by Kevin Grittner Discussion: <1188.1464544443@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/b7b8cc0cfcf1c956b752f3e25894f9ad607583b7
  • Rationalize parallel dump/restore's handling of worker cmd/status messages. The existing APIs for creating and parsing command and status messages are rather messy; for example, archive-format modules have to provide code for constructing command messages, which is entirely pointless since the code to read them is hard-wired in WaitForCommands() and hence no format-specific variation is actually possible. But there's little foreseeable reason to need format-specific variation anyway. The situation for status messages is no better; at least those are both constructed and parsed by format-specific code, but said code is quite redundant since there's no actual need for format-specific variation. To add insult to injury, the first API involves returning pointers to static buffers, which is bad, while the second involves returning pointers to malloc'd strings, which is safer but randomly inconsistent. Hence, get rid of the MasterStartParallelItem and MasterEndParallelItem APIs, and instead write centralized functions that construct and parse command and status messages. If we ever do need more flexibility, these functions can be the standard implementations of format-specific callback methods, but that's a long way off if it ever happens. Tom Lane, reviewed by Kevin Grittner Discussion: <17340.1464465717@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/fb03d08a89e81a68585f17fd8e7f21c618f4e851
  • Make struct ParallelSlot private within pg_dump/parallel.c. The only field of this struct that other files have any need to touch is the pointer to the TocEntry a worker is working on. (Well, pg_backup_archiver.c is actually looking at workerStatus too, but that can be finessed by specifying that the TocEntry pointer is NULL for a non-busy worker.) Hence, move out the TocEntry pointers to a separate array within struct ParallelState, and then we can make struct ParallelSlot private. I noted the possibility of this previously, but hadn't got round to actually doing it. Discussion: <1188.1464544443@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/0109ab27609c0d58c1eddc6b799077d0968083de
  • Disallow pushing volatile quals past set-returning functions. Pushing an upper-level restriction clause into an unflattened subquery-in-FROM is okay when the subquery contains no SRFs in its targetlist, or when it does but the SRFs are unreferenced by the clause *and the clause is not volatile*. Otherwise, we're changing the number of times the clause is evaluated, which is bad for volatile quals, and possibly changing the result, since a volatile qual might succeed for some SRF output rows and not others despite not referencing any of the changing columns. (Indeed, if the clause is something like "random() > 0.5", the user is probably expecting exactly that behavior.) We had most of these restrictions down, but not the one about the upper clause not being volatile. Fix that, and add a regression test to illustrate the expected behavior. Although this is definitely a bug, it doesn't seem like back-patch material, since possibly some users don't realize that the broken behavior is broken and are relying on what happens now. Also, while the added test is quite cheap in the wake of commit a4c35ea1c, it would be much more expensive (or else messier) in older branches. Per report from Tom van Tilburg. Discussion: <CAP3PPDiucxYCNev52=YPVkrQAPVF1C5PFWnrQPT7iMzO1fiKFQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/72daabc7a3e75788df862104b8f723513c2471ae
  • Make to_timestamp() and to_date() range-check fields of their input. Historically, something like to_date('2009-06-40','YYYY-MM-DD') would return '2009-07-10' because there was no prohibition on out-of-range month or day numbers. This has been widely panned, and it also turns out that Oracle throws an error in such cases. Since these functions are nominally Oracle-compatibility features, let's change that. There's no particular restriction on year (modulo the fact that the scanner may not believe that more than 4 digits are year digits, a matter to be addressed separately if at all). But we now check month, day, hour, minute, second, and fractional-second fields, as well as day-of-year and second-of-day fields if those are used. Currently, no checks are made on ISO-8601-style week numbers or day numbers; it's not very clear what the appropriate rules would be there, and they're probably so little used that it's not worth sweating over. Artur Zakirov, reviewed by Amul Sul, further adjustments by me Discussion: <1873520224.1784572.1465833145330.JavaMail.yahoo@mail.yahoo.com> See-Also: <57786490.9010201@wars-nicht.de> http://git.postgresql.org/pg/commitdiff/d3cd36a133d96ad5578b6c10279b55fd5b538093
  • Rationalize format-picture caching logic in formatting.c. Add a validity flag to DCHCacheEntry and NUMCacheEntry entries, and do not set it true until after we've parsed the supplied format string. This allows dealing with possible errors while parsing the format without the baroque hack that was there before (which only covered errors within NUMDesc_prepare, anyway). We can get rid of the PG_TRY in NUMDesc_prepare, as well as last_NUMCacheEntry and NUM_cache_remove. (Essentially, this reverts commit ff783fbae in favor of a less fragile solution; the problems with that approach are well illustrated by later hacking such as 55f927a46.) In passing, define the size of these caches as DCH_CACHE_ENTRIES not DCH_CACHE_FIELDS + 1 (whoever thought that was a good definition?) and likewise for the NUM cache. Also const-ify format string parameters where convenient, and merge duplicated cache lookup logic. This is primarily driven by a proposed patch from Artur Zakirov, which introduced some ereport's into format string parsing for the datetime case. He proposed preventing the creation of invalid cache entries by parsing the format string first into a local-variable array, and then copying that to a cache entry. That seemed a bit ugly to me, and anyway randomly different from the way the identical problem had been solved for the numeric case. Let's make the two sets of code more similar not less so. I'm not sure whether we'll adopt the new error conditions Artur proposes, but this patch seems like good code cleanup and future-proofing in any case. The existing code is critically (and undocumented-ly) dependent on no elog being thrown out of several nontrivial functions, which is trouble waiting to happen, though it doesn't seem to be actively broken today. Discussion: <b2a39359-3282-b402-f4a3-057aae500ee7@postgrespro.ru> http://git.postgresql.org/pg/commitdiff/83bed06be4e808f3da30f99b6c91e9efda3961ad
  • Allow contrib/file_fdw to read from a program, like COPY FROM PROGRAM. This patch just exposes COPY's FROM PROGRAM option in contrib/file_fdw. There don't seem to be any security issues with that that are any worse than what already exist with file_fdw and COPY; as in the existing cases, only superusers are allowed to control what gets executed. A regression test case might be nice here, but choosing a 100% portable command to run is hard. (We haven't got a test for COPY FROM PROGRAM itself, either.) Corey Huinker and Adam Gomaa, reviewed by Amit Langote Discussion: <CADkLM=dGDGmaEiZ=UDepzumWg-CVn7r8MHPjr2NArj8S3TsROQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/8e91e12bc3af85ba2287866669268f6825d2cc03
  • Fix multiple portability issues in pg_upgrade's rewriteVisibilityMap(). This is new code in 9.6, and evidently we missed out testing it as thoroughly as it should have been. Bugs fixed here: 1. Use binary not text mode to open the files on Windows. Before, if the visibility map chanced to contain two bytes that looked like \r\n, Windows' read() would convert that to \n, which both corrupts the map data and causes the file to look shorter than it should. Unless you were *very* unlucky and had an exact multiple of 8K such occurrences in each VM file, this would cause pg_upgrade to report a failure, though with a rather obscure error message. 2. The code for copying rebuilt bytes into the output was simply wrong. It chanced to work okay on little-endian machines but would emit the bytes in the wrong order on big-endian, leading to silent corruption of the visibility map data. 3. The code was careless about alignment of the working buffers. Given all three of an alignment-picky architecture, a compiler that chooses to put the new_vmbuf[] local variable at an odd starting address, and a checksum-enabled database, pg_upgrade would dump core. Point one was reported by Thomas Kellerer, the other two detected by code-reading. Point two is much the nastiest of these issues from an impact standpoint, though fortunately it affects only a minority of users. The Windows issue will definitely bite people, but it seems quite unlikely that there would be undetected corruption from that. In addition, I failed to resist the temptation to do some minor cosmetic adjustments, mostly improving the comments. It would be a good idea to try to improve the error reporting here, but that seems like material for a separate patch. ------- http://git.postgresql.org/pg/commitdiff/5afcd2aa740f87fa6cec7c6693a4891b9df2b67d
  • Improve error reporting in pg_upgrade's file copying/linking/rewriting. The previous design for this had copyFile(), linkFile(), and rewriteVisibilityMap() returning strerror strings, with the caller producing one-size-fits-all error messages based on that. This made it impossible to produce messages that described the failures with any degree of precision, especially not short-read problems since those don't set errno at all. Since pg_upgrade has no intention of continuing after any error in this area, let's fix this by just letting these functions call pg_fatal() for themselves, making it easy for each point of failure to have a suitable error message. Taking this approach also allows dropping cleanup code that was unnecessary and was often rather sloppy about preserving errno. To not lose relevant info that was reported before, pass in the schema name and table name of the current table so that they can be included in the error reports. An additional problem was the use of getErrorText(), which was flat out wrong for all but a couple of call sites, because it unconditionally did "_dosmaperr(GetLastError())" on Windows. That's only appropriate when reporting an error from a Windows-native API, which only a couple of the callers were actually doing. Thus, even the reported strerror string would be unrelated to the actual failure in many cases on Windows. To fix, get rid of getErrorText() altogether, and just have call sites do strerror(errno) instead, since that's the way all the rest of our frontend programs do it. Add back the _dosmaperr() calls in the two places where that's actually appropriate. In passing, make assorted messages hew more closely to project style guidelines, notably by removing initial capitals in not-complete-sentence primary error messages. (I didn't make any effort to clean up places I didn't have another reason to touch, though.) Per discussion of a report from Thomas Kellerer. Back-patch to 9.6, but no further; given the relative infrequency of reports of problems here, it's not clear it's worth adapting the patch to older branches. Patch by me, but with credit to Alvaro Herrera for spotting the issue with getErrorText's misuse of _dosmaperr(). Discussion: <nsjrbh$8li$1@blaine.gmane.org> http://git.postgresql.org/pg/commitdiff/f002ed2b8e45286fe73a36e119cba2016ea0de19
  • Fix misplacement of submake-generated-headers prerequisites. The sequence "configure; cd src/pl/plpython; make -j" failed due to trying to compile plpython's .o files before the generated headers finished building. (This is an important real-world case, since it's the typical second step when building both plpython2 and plpython3.) This happens because the submake-generated-headers target is not placed in a way to make it a prerequisite to compiling the .o files. Fix that. Checking other uses of submake-generated-headers, I noted that the one attached to pg_regress was similarly misplaced; but it's actually not needed at all for pg_regress.o, rather regress.o, so move it to be a prerequisite of that. Back-patch to 9.6 where submake-generated-headers was introduced (by commit 548af97fc). It's not immediately clear to me why the previous coding didn't have the same issue; but since we've not had field reports of plpython make failing, leave it alone in the older branches. Pavel Raiskup and Tom Lane Discussion: <1925924.izSMJEZO3x@unused-4-107.brq.redhat.com> http://git.postgresql.org/pg/commitdiff/7107d58ec5a3c45967e77525809612a5f89b97f3
  • Fix misstatement in comment in Makefile.shlib. There is no need for "all: all-lib" to be placed before inclusion of Makefile.shlib. Makefile.global is what ensures that "all" is the default target, and we already document that that has to be included first. Per comment from Pavel Raiskup. Discussion: <1925924.izSMJEZO3x@unused-4-107.brq.redhat.com> http://git.postgresql.org/pg/commitdiff/ea046f08d1bcee56c3bedfa16a05c38a0515b41d
  • Copy-editing for contrib/pg_visibility documentation. Add omitted names for some function parameters. Fix some minor grammatical issues. http://git.postgresql.org/pg/commitdiff/33596edf09516a7cab65914e16cfd6adf9fc55d1
  • Fix bugs in contrib/pg_visibility. collect_corrupt_items() failed to initialize tuple.t_self. While HeapTupleSatisfiesVacuum() doesn't actually use that value, it does Assert that it's valid, so that the code would dump core if ip_posid chanced to be zero. (That's somewhat unlikely, which probably explains how this got missed. In any case it wouldn't matter for field use.) Also, collect_corrupt_items was returning the wrong TIDs, that is the contents of t_ctid rather than the tuple's own location. This would be the same thing in simple cases, but it could be wrong if, for example, a past update attempt had been rolled back, leaving a live tuple whose t_ctid doesn't point at itself. Also, in pg_visibility(), guard against trying to read a page past the end of the rel. The VM code handles inquiries beyond the end of the map by silently returning zeroes, and it seems like we should do the same thing here. I ran into the assertion failure while using pg_visibility to check pg_upgrade's behavior, and then noted the other problems while reading the code. Report: <29043.1475288648@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/9a109452da1b923e183f20fcf5516984d448ece9
  • Do ClosePostmasterPorts() earlier in SubPostmasterMain(). In standard Unix builds, postmaster child processes do ClosePostmasterPorts immediately after InitPostmasterChild, that is almost immediately after being spawned. This is important because we don't want children holding open the postmaster's end of the postmaster death watch pipe. However, in EXEC_BACKEND builds, SubPostmasterMain was postponing this responsibility significantly, in order to make it slightly more convenient to pass the right flag value to ClosePostmasterPorts. This is bad, particularly seeing that process_shared_preload_libraries() might invoke nearly-arbitrary code. Rearrange so that we do it as soon as we've fetched the socket FDs via read_backend_variables(). Also move the comment explaining about randomize_va_space to before the call of PGSharedMemoryReAttach, which is where it's relevant. The old placement was appropriate when the reattach happened inside CreateSharedMemoryAndSemaphores, but that was a long time ago. Back-patch to 9.3; the patch doesn't apply cleanly before that, and it doesn't seem worth a lot of effort given that we've had no actual field complaints traceable to this. Discussion: <4157.1475178360@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/3b90e38c5d592ea8ec8236287dd5c749fc041728
  • Avoid leaking FDs after an fsync failure. Fixes errors introduced in commit bc34223bc, as detected by Coverity. In passing, report ENOSPC for a short write while padding a new wal file in open_walfile, make certain that close_walfile closes walfile in all cases, and improve a couple of comments. Michael Paquier and Tom Lane http://git.postgresql.org/pg/commitdiff/728ceba938dfadb204a4854bb76ae3b11b635401
  • Add ALTER EXTENSION ADD/DROP ACCESS METHOD, and use it in pg_upgrade. Without this, an extension containing an access method is not properly dumped/restored during pg_upgrade --- the AM ends up not being a member of the extension after upgrading. Another oversight in commit 473b93287, reported by Andrew Dunstan. Report: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net> http://git.postgresql.org/pg/commitdiff/e8bdee2770ff52aab208bc6f8965a4a01979d0aa

Peter Eisentraut pushed:

Ãlvaro Herrera pushed:

Heikki Linnakangas pushed:

  • Refactor script execution state machine in pgbench. The doCustom() function had grown into quite a mess. Rewrite it, in a more explicit state machine style, for readability. This also fixes one minor bug: if a script consisted entirely of meta commands, doCustom() never returned to the caller, so progress reports with the -P option were not printed. I don't want to backpatch this refactoring, and the bug is quite insignificant, so only commit this to master, and leave the bug unfixed in back-branches. Review and original bug report by Fabien Coelho. Discussion: <alpine.DEB.2.20.1607090850120.3412@sto> http://git.postgresql.org/pg/commitdiff/12788ae49e1933f463bc59a6efe46c4a01701b76
  • Turn password_encryption GUC into an enum. This makes the parameter easier to extend, to support other password-based authentication protocols than MD5. (SCRAM is being worked on.) The GUC still accepts on/off as aliases for "md5" and "plain", although we may want to remove those once we actually add support for another password hash type. Michael Paquier, reviewed by David Steele, with some further edits by me. Discussion: <CAB7nPqSMXU35g=W9X74HVeQp0uvgJxvYOuA4A-A3M+0wfEBv-w@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/babe05bc2b781eb3eb84a18d7010d08277e2e399
  • Don't bother to lock bufmgr partitions in pg_buffercache. That makes the view a lot less disruptive to use on a production system. Without the locks, you don't get a consistent snapshot across all buffers, but that's OK. It wasn't a very useful guarantee in practice. Ivan Kartyshov, reviewed by Tomas Vondra and Robert Haas. Discusssion: <f9d6cab2-73a7-7a84-55a8-07dcb8516ae5@postgrespro.ru> http://git.postgresql.org/pg/commitdiff/6e654546fb61f62cc982d0c8f62241b3b30e7ef8

Robert Haas pushed:

Stephen Frost pushed:

  • Remove superuser checks in pgstattuple. Now that we track initial privileges on extension objects and changes to those permissions, we can drop the superuser() checks from the various functions which are part of the pgstattuple extension and rely on the GRANT system to control access to those functions. Since a pg_upgrade will preserve the version of the extension which existed prior to the upgrade, we can't simply modify the existing functions but instead need to create new functions which remove the checks and update the SQL-level functions to use the new functions (and to REVOKE EXECUTE rights on those functions from PUBLIC). Thanks to Tom and Andres for adding support for extensions to follow update paths (see: 40b449a), allowing this patch to be much smaller since no new base version script needed to be included. Approach suggested by Noah. Reviewed by Michael Paquier. http://git.postgresql.org/pg/commitdiff/fd321a1dfd64d30bf1652ea6b39b654304f68ae4

Magnus Hagander pushed:

Correctifs en attente

Takayuki Tsunakawa sent in two revisions of a patch to implement huge_pages on Windows.

Michaël Paquier and Heikki Linnakangas traded patch flocks to implement SCRAM password auth.

Pavel Stěhule sent in another revision of a patch to add a \setfileref command to psql.

Pavel Stěhule sent in another revision of a patch to add xmltable().

Vitaly Burovoy sent in a patch to detect supported SET parameters when pg_restore is run.

Vinayak Pokale support prepared transactions with foreign servers.

Peter Eisentraut and Michaël Paquier traded patches to fix CRC check handling in get_controlfile.

Stephen Frost and Jeevan Chalke traded patches to add support for restrictive RLS policies.

Jesper Pedersen and Peter Eisentraut traded patches to add hash index support to pageinspect.

Kyotaro HORIGUCHI and Michaël Paquier traded patches to track LSN on lightly loaded systems to reduce the number of checkpoints.

Tomas Vondra sent in another revision of a patch to add two slab-like memory allocators.

Fabien COELHO sent in three more revisions of a patch to add more operators and functions to pgbench.

Fabien COELHO sent in another revision of a patch to enable storing select results into variables.

Rajkumar Raghuwanshi sent in a patch to add test cases for partition-wise joins on declarative partitions.

Daniel Gustafsson sent in a patch to make flex/bison checks stricter in Git trees.

Mark Dilger sent in a patch to stop casting away const in comparators.

Mark Dilger sent in a patch to stop casting away const in regex.

Mark Dilger sent in a patch to avoid casting away const when using the FileName typedef.

Takayuki Tsunakawa sent in a patch to allow SIGQUIT to prompt a fast shutdown of the stats collector without leaving partly-written stats files behind.

Mark Dilger sent in a WIP patch to const-ify xlog_outdesc() and rm_redo_error_callback().

David Steele and Michaël Paquier traded patches to exclude additional directories in pg_basebackup.

Amit Langote sent in three more revisions of a patch to implement declarative partitions.

Michaël Paquier sent in four more revisions of a patch to track wait event for latches.

Victor Wagner sent in two more revisions of a patch to implement failover at the libpq connect level.

Mark Dilger sent in a patch to tidy up some handling of pointers in rangetypes_spgist.c, tsgistidx.c, and tsrank.c.

David Cramer and Heikki Linnakangas traded patches to add support for multi-dimensional arrays to PL/PythonU.

Mithun Cy sent in another revision of a patch to cache hash index meta pages.

Emre Hasegeli sent in another revision of a patch to fix floating point comparison inconsistencies in the geometric types.

Ivan Kartyshov sent in another revision of a patch to make pg_buffercache more efficient with large shared memory.

Etsuro Fujita sent in two more revisions of a patch to push down more full joins to the PostgreSQL FDW.

David Fetter and Thomas Munro traded patches to add a hook and corresponding GUCs to disable UPDATEs and DELETEs that lack a WHERE clause.

Michaël Paquier sent in another revision of a patch to rename pg_xlog to pg_wal and rename pg_clog to pg_transaction.

Heikki Linnakangas sent in two more revisions of a patch to change the way use memory for larger read buffers in logtape.c rather than pre-reading tuples into SortTuple slots during merge.

Magnus Hagander sent in two more revisions of a patch to enable pg_basebackup to stream xlog to tar.

Kyotaro HORIGUCHI sent in another revision of a patch to fix wal level minimal.

Ashutosh Bapat sent in another revision of a patch to fix an issue where altering foreign table was not invalidating prepare statement execution plans that it should have.

Jeevan Chalke sent in another revision of a patch to enable pushing aggregates to a FDW.

Aleksander Alekseev sent in a patch to rename md5Salt to pwsalt.

Corey Huinker sent in a PoC patch to make a copy() set-returning function.

Stephen Frost sent in another revision of a patch to fix some infelicities between COPY and RLS.

Andres Freund sent in another revision of a patch to add a macro-customizable hashtable for bitmapscan and aggregation performance.

Daniel Vérité sent in another revision of a patch to fix some pg_dump / copy bugs with long lines.

Dmitry Dolgov sent in another revision of a patch to add a generic type subscription.

Emre Hasegeli sent in a patch to add <@, @>, <<@, and @>> operator symbols for the inet datatype to replace <<=, >>=, <<, and >>.

Christoph Berg sent in a patch to ensure that the default log_line prefix is not empty.

par N Bougain le lundi 10 octobre 2016 à 21h49

lundi 3 octobre 2016

Damien Clochard

6 raisons de choisir PostgreSQL 9.6

PostgreSQL 9.6 est sortie il y a quelques jours et encore fois c’est une version majeure avec des fonctionnalités très attendues ! Les notes de versions sont très longues alors j’ai décidé de sélectionner 6 nouveautés importantes qui peuvent changer la manière dont vous utiliser PostgreSQL.

  1. Le “Parallelisme” est probablement l’attraction principale de cette version : une fonctionnalité réclamée depuis des années. Pour résumer : avec les versions précédentes, Postgres utilisait uniquement un seul cœur par requête, même si d’autres processeurs étaient disponibles. Cette limitation vient d’être supprimée et différentes formes d’opérations peuvent désormais être effectuée en parallèle : parcours séquentiels, jointures et agregats peuvent être distribuée sur plusieurs cœurs, si nécessaire.

  2. Un meilleur suivi des verrous : la vue pg_stat_activity fournit désormais des infos plus précises sur les blocages (“wait events”). Lorsqu’un processus est en attente sur un verrou, vous pouvez voir le type du verrou et des détails sur le contexte du blocage. Par ailleurs, avec la fonction pg_blocking_pids() vous pouvez voir ce qui bloque un processus donné. Ces outils de monitoring vont permettre aux DBA de savoir combien de temps un processus a attendu un certain événement et donc mieux identifier les goulots d’étranglement.

  3. La réplication synchrone multi-noeuds vous garantit qu’une transaction est appliquées simultanément sur plusieurs instances. Cela permet de renforcer la durabilité des données et en dupliquant les données sur plusieurs instances parfaitement identiques.

  4. Moins d’espace perdu ! Jusqu’à présent les requêtes très longues ou les curseurs pouvait empêcher le nettoyage des lignes mortes et provoquer un gonflement (“bloat”) des volumes de stockage, ce qui impactait les performances et causait une surconsommation d’espace disque. La nouvelle option old_snapshot_threshold autorise PostgreSQL à nettoyer une ligne morte lorsque la transaction qui l’a modifiée est terminée et que les snapshots qui peuvent encore la voir ont dépassé un certain age.

  5. Des améliorations sur les Data Wrappers : Avec plus de 80 connecteurs externes (FDW) vous pouvez déjà brancher PostgreSQL sur n’importe quel système de stockage de données. La version 9.6 apporte des nouveautés importantes sur le connecteur postgres_fdw, notamment le transfert ( “push down”) des jointures et des tris sur le nœud distant. Une avancée fondamentale pour agréger des données à partir de plusieurs instances PostgreSQL.

  6. Le mode remote_apply : PostgreSQL 9.6 propose une nouvelle méthode de réplication dans laquelle le nœud primaire attend que les changements soient appliqués sur le nœeud secondaire. C’est un mode un peu plus lent que la réplication classique mais cela garantit que les “données commitées” sur un nœud secondaire sont bien visibles. Idéal pour distribuer les lectures sur plusieurs instances !

Bien sûr retenir uniquement 6 items dans la liste des nouveautés de Postgres 9.6 est un choix difficile. Il y a beaucoup d’autres améliorations dans cette version, par exemple : le rechercher plein texte sur des phrases entières, l’extension pg_visibility, un meilleur nettoyage (VACUUM) des pages gelées, les parcours uniques d’index pour les index partiels, le suivi de la progression des commandes, et comme d’habitude…. une flopée de gains de performance !

Pour un tour d’horizon plus détaillé, la page de wiki What’s New in 9.6 est un bon point de départ !

NB : Merci à Hervé Piedvache et Jean-Louis Louër pour la relecture

Crédit Photo : duncan @ Flickr CC BY-NC

par Damien Clochard le lundi 3 octobre 2016 à 11h17

Nicolas Gollet

Surveillance de logged/unlogged pour PostgreSQL 9.5

Depuis la version 9.1 de PostgreSQL, il est possible de créer une table non journalisée (unlogged) , cette définition est permanente est ne peut être changé.

Lors d'une opération de mise à jour ou d'insertion sur une table, le journal des modifications est écrit (WAL). Dans le mode Logged, le COMMIT de l'utilisateur va attendre jusqu'à ce que l'écriture du WAL soit complète.

Aujourd'hui, la version 9.5 et supérieur de PostgreSQL offre la possibilité de pouvoir passer d'une table non journalisée à journalisée et vice et versa :

ALTER TABLE < nomdelatable > SET [LOGGED|UNLOGGED];

Dans le cas où une table devient unlogged, celle-ci ne bénéficie plus de la sécurité qu'offrent les journaux de transaction; et dans le cas d'une architecture haute disponibilité , celle-ci ne sera plus répliqué.
En cas de "crash", les données de cette table seront perdues.

Faites donc attention lorsque vous changez cette propriété sur une table!

Afin de pouvoir être alerté sur le changement logged/unlogged, un nouveau service a été rajouté sur la sonde de surveillance PostgreSQL compatible Nagios "check_pgactivity".

Ce service est disponible dans la version 2.0 disponible depuis le 29 aout 2016.

Site du projet https://github.com/OPMDG/check_pgactivity

par Nicolas GOLLET le lundi 3 octobre 2016 à 09h27

Actualités PostgreSQL.fr

Sortie de PostgreSQL 9.6

29 septembre 2016 : le PostgreSQL Global Development Group a publié aujourd'hui PostgreSQL 9.6, dernière version du système libre de gestion de bases de données SQL de référence. Cette version va permettre aux utilisateurs de réaliser à la fois une expansion interne (scale up) et externe (scale out, répartition de la charge) de leurs bases de données haute-performance. Les nouvelles fonctionnalités incluent notamment le parallélisme des requêtes, des améliorations sur la réplication synchrone, la recherche par phrase, ainsi que des améliorations sur la performance et la facilité d'utilisation.

Expansion interne avec le parallélisme des requêtes

La version 9.6 ajoute le support du parallélisme pour certaines opérations dans les requêtes. Ce parallélisme active l'utilisation de tout ou partie des cœurs de processeur d'un serveur pour retourner les résultats plus rapidement. Sur cette version, le parallélisme concerne le parcours séquentiel (des tables), les agrégats et les jointures. En fonction des caractéristiques et du nombre de cœurs, le parallélisme permet d'espérer des gains jusqu'à un facteur 32 sur des requêtes traitant des volumes importants de données.

"J'ai migré l'intégralité de notre plateforme génomique - soit un héritage de 25 milliards de lignes MySQL - vers une seule base de données Postgres, en utilisant les possibilités de compression du type JSONB, les excellentes méthodes d'indexation GIN, BRIN et B-tree. Aujourd'hui, avec la version 9.6, j'attends de pouvoir exploiter le parallélisme des requêtes pour améliorer encore plus les performances des requêtes sur les tables volumineuses" indique Mike Sofen, Chief Database Architect chez Synthetic Genomics.

Répartition de charge avec la réplication synchrone et postgres_fdw

Deux nouvelles options ont été ajoutées à la réplication synchrone de PostgreSQL. Elles permettent de maintenir des lectures cohérentes de données sur les nœuds d'un cluster de bases de données. En premier lieu, il est désormais possible de configurer des groupes de réplication synchrone. En second lieu, le mode "remote_apply" (application distante) crée une vue plus cohérente des données sur les différents nœuds. Ces fonctionnalités utilisent la réplication interne pour maintenir un groupe de nœuds "identiques" afin d'équilibrer la charge de lecture (load-balancing).

Le pilote de fédération de données entre bases PostgreSQL, postgres_fdw, dispose de nouvelles fonctionnalités d'exécution du travail sur le serveur distant. En "t;externalisant" les tris, les jointures et les mises à jour par lot, la charge peut être distribuée sur plusieurs serveurs PostgreSQL. Ces possibilités seront bientôt ajoutées aux autres pilotes FDW.

"Avec les possibilités d'externaliser les jointures, les mises à jour et les suppressions, les Foreign Data Wrappers offrent maintenant une solution complète de partage de données entre PostgreSQL et les autres bases de données. Par exemple, PostgreSQL peut être utilisé pour gérer en entrée des données qui vont vers des bases de données différentes" indique Julyanto Sutandang, Director of Business Solutions chez Equnix.

Meilleure recherche de texte avec des phrases

La fonctionnalité de recherche plein texte de PostgreSQL supporte désormais les recherche par phrases. Cela permet une recherche de phrases exactes ou de mots proches les uns des autres, en utilisant l'indexation GIN. Additionné aux nouvelles fonctionnalités d'optimisation de la recherche textuelle, cet apport fait de PostgreSQL un outil de choix pour la "recherche hybride" mêlant relationnel, JSON et recherche plein texte.

Plus agréable, plus rapide et plus facile à utiliser

Grâce au retours d'expérience et aux tests effectués par les utilisateurs de PostgreSQL disposant de bases de données de production à fort volume, le projet a pu améliorer de nombreux aspects des performances et de la maniabilité dans cette version. La réplication, les agrégats, l'indexation, les tris et les procédures stockées sont plus efficaces, et PostgreSQL utilise désormais mieux les ressources lorsqu'il est installé sur des noyaux Linux récents. Le surcoût d’administration des tables volumineuses et des charges complexes a été réduit, notamment par des améliorations du VACUUM.

Détail des fonctionnalités

Plus d'informations sur les fonctionnalités ci-dessus et les autres dans les liens suivants :

Téléchargement

Documentation

La documentation au format HTML et les pages de manuel sont installées avec PostgreSQL. La documentation en ligne, exhaustive et interactive, peut être parcourue, interrogée et commentée librement.

Licence

PostgreSQL utilise la licence PostgreSQL, une licence permissive de type BSD. Cette licence certifiée par l'OSI est largement appréciée pour sa flexibilité et sa compatibilité avec le monde des affaires, puisqu'elle ne restreint pas l'utilisation de PostgreSQL dans les applications propriétaires ou commerciales. Associée à un support proposé par de multiples sociétés et une propriété publique du code, sa licence rend PostgreSQL très populaire parmi les revendeurs souhaitant embarquer une base de données dans leurs produits sans avoir à se soucier des prix de licence, des verrous commerciaux ou modifications des termes de licence.

Contacts

Pages Web

Contact presse

France et pays francophones
Stéphane Schildknecht
fr at postgresql dot org
+33 (0) 617 11 37 42

Pour les contacts d'autres régions, consulter la liste des contacts internationaux.

Informations concernant les sociétés citées et texte intégral des citations

"En tant qu'architecte en chef de bases de données chez Synthetic Genomics, j'ai été réellement impressionné par la puissance, la flexibilité et la richesse des interfaces de programmation de Postgres 9.5. J'ai migré l'intégralité de notre plateforme génomique — soit 25 milliards de lignes issues de MySQL — vers une seule base de données Postgres, en utilisant les possibilités de compression du type JSONB, les excellentes méthodes d'indexation GIN, BRIN et B-tree, ainsi que l'élégant langage de programmation de fonctions stockées PL/pgSQL. Aujourd'hui, avec la version 9.6, j'attends de pouvoir exploiter le parallélisme des requêtes pour améliorer encore plus les performances des requêtes sur les tables volumineuses. Postgres 9.5 a prouvé par lui-même qu'il est complètement stable, remarquablement performant, et selon moi, le meilleur moteur à ce jour de stockage de données hybride relationnel-document sur le marché." indique Mike Sofen, Chief Database Architect chez Synthetic Genomics.

Synthetic Genomics, Inc. (SGI) est l'un des leaders mondiaux dans les domaines de la biologie de synthèse et la génomique de synthèse. SGI se fonde sur l'utilisation de la propriété intellectuelle dans cette discipline qui évolue rapidement, pour concevoir et réaliser des systèmes biologiques qui permettent de résoudre les défis de développement durable à l'échelle de la planète. SGI fournit des solutions dans les domaines de la recherche génomique, de la bio-production et des produits appliqués. La recherche est focalisée sur les solutions génomiques pour tout organisme de recherche et développement, académique ou commercial. Les produits et services commerciaux incluent l'instrumentation, les réactifs, les services de synthèse d'ADN et les logiciels et services en bio-informatique. Contact : Synthetic Genomics, media@syntheticgenomics.com (anglais)

"Avec les possibilités d'externaliser les jointures, les mises à jour et les suppressions, les Foreign Data Wrappers offrent maintenant une solution complète de partage de données entre PostgreSQL et les autres bases de données. Par exemple, PostgreSQL peut être utilisé pour gérer en entrée des données qui vont vers des bases de données différentes" indique Julyanto Sutandang, Director of Business Solutions chez Equnix.

Equnix est fournisseur de solutions informatiques pour les entreprises. Cette société met en place des solutions alternatives complètes, basées sur la recherche, le développement et du code libre. Equnix fournit du service autour de PostgreSQL. Contact en anglais ou en indonésien : Info@equnix.co.id ou téléphoner à Johan D. au +6221-22866662.

Support Professionnel

PostgreSQL bénéficie du support de nombreuses sociétés, qui financent des développeurs, fournissent l'hébergement ou un support financier. Les plus fervents supporters sont listés sur la page des mécènes du développement.

Il existe également une très grande communauté de sociétés fournissant du support PostgreSQL, des consultants indépendants aux sociétés multinationales.

Les dons sont acceptés avec plaisir.

Vous pouvez également acheter des produits dérivés de qualité sur la Boutique Zazzle PostgreSQL.

par SAS le lundi 3 octobre 2016 à 05h33

jeudi 29 septembre 2016

Sébastien Lardière

PostgreSQL 9.6.0

La version 9.6.0 de PostgreSQL est publiée aujourd'hui. La fonctionnalité la plus notable de cette version majeure est la parallélisation des requêtes. De nombreuses autres fonctionnalités permettent de travailler sur des volumes de données toujours plus important. Quelques informations sur le site de Loxodata :... Lire PostgreSQL 9.6.0

par Sébastien Lardière le jeudi 29 septembre 2016 à 14h16

mercredi 28 septembre 2016

Cédric Villemain

pgFincore 1.2, une extension PostgreSQL

pgFincore 1.2 est une extension PostgreSQL pour auditer et manipuler le cache de pages de données du système d’exploitation. L’extension a déjà une histoire de 7 ans d’utilisation, avec des évolutions correspondant aux besoins de production.

Télécharger ici la dernière version 1.2, compatible avec PostgreSQL 9.6.

Cache de données

Cache de données

Le cache de pages de données est une opération qui se réalise «naturellement», à plusieurs niveaux dans la gestion des données. L’objet est simple: une multitude de couches se superposent entre les données physiquement enregistrées sur disque et la restitution à l’utilisateur. Actuellement quasiment chaque couche de données possède une abstraction pour servir plus rapidement des ordres de lecture et d’écriture. Ainsi la majorité des disques durs proposent un cache en écriture, qui permet de retarder l’écriture physique, et un cache en lecture qui permet d’anticiper sur des prochaines demandes et servir des données plus rapidement. Un système équivalent existe dans les SAN, les cartes RAID, les système d’exploitation, les logiciels, etc.

PostgreSQL possède bien sûr son propre système de gestion pour les écritures et les lectures, les shared buffers, que l’on peut auditer avec l’extension pg_buffercache.

Il est possible d’auditer le cache du système d’exploitation avec des outils systèmes, et pgFincore porte cela dans PostgreSQL.

Read Ahead

read aheadLa plupart des systèmes d’exploitation optimisent les parcours de données en proposant une fenêtre de lecture en avance, cela permet de pré-charger des données dans le cache et ainsi les fournir plus rapidement aux applications. PostgreSQL contient plusieurs optimisations pour favoriser ce comportement au niveau système, et porte également une fonctionnalité similaire avec l’option effective_io_concurrency.

Une solution pour faciliter ces optimisations consiste à utiliser des appels systèmes POSIX_FADVISE. Là-aussi pgFincore porte cette solution dans PostgreSQL.

pgFincore 1.2

Cette extension permet donc:

  • d’obtenir des informations précises sur l’occupation d’une table ou d’un index (et quelques autres fichiers utilisés par PostgreSQL) dans le cache du système supportant POSIX (linux, BSD, …),
  • de manipuler ce cache: en faire une carte et la restaurer ultérieurement ou sur un autre serveur,
  • d’optimiser les parcours via les appels posix_fadvise.pgfincore looks at that

Obtenir pgFincore

Paquets Debian et Red Hat disponibles dans les distributions, et pour chaque version de PostgreSQL sur les dépôts Apt PGDG et RPM PGDG.

Et les sources sur le dépôt git pgfincore.

Besoin d’aide ?

En plus du support communautaire, vous pouvez contacter 2ndQuadrant.


Exemples d’utilisation

Installation

$ sudo apt-get install postgresql-9.6-pgfincore
$ psql -c 'CREATE EXTENSION pgfincore;'

Information système

# select * from pgsysconf_pretty();
 os_page_size | os_pages_free | os_total_pages 
--------------+---------------+----------------
 4096 bytes   | 314 MB        | 16 GB

Optimiser le parcours aléatoire (réduction de la fenêtre de read-ahead)

# select * from pgfadvise_random('pgbench_accounts_pkey');
          relpath | os_page_size | rel_os_pages | os_pages_free 
------------------+--------------+--------------+---------------
 base/16385/24980 | 4096         | 2            | 1853808

Optimiser le parcours séquentiel (augmentation de la fenêtre de read-ahead)

# select * from pgfadvise_sequential('pgbench_accounts');
 relpath          | os_page_size | rel_os_pages | os_pages_free 
------------------+--------------+--------------+---------------
 base/16385/25676 | 4096         | 3176         | 1829288

Audit du cache

# select * from pgfincore('pgbench_accounts');
      relpath       | segment | os_page_size | rel_os_pages | pages_mem | group_mem | os_pages_free | databit 
--------------------+---------+--------------+--------------+-----------+-----------+---------------+---------
 base/11874/16447   |       0 |         4096 |       262144 |         3 |         1 |        408444 | 
 base/11874/16447.1 |       1 |         4096 |        65726 |         0 |         0 |        408444 | 

Charger une table en mémoire

# select * from pgfadvise_willneed('pgbench_accounts');
      relpath       | os_page_size | rel_os_pages | os_pages_free 
--------------------+--------------+--------------+---------------
 base/11874/16447   |         4096 |       262144 |         80650
 base/11874/16447.1 |         4096 |        65726 |         80650

Vider le cache d’une table

# select * from pgfadvise_dontneed('pgbench_accounts');
      relpath       | os_page_size | rel_os_pages | os_pages_free
--------------------+--------------+--------------+---------------
 base/11874/16447   |         4096 |       262144 |        342071
 base/11874/16447.1 |         4096 |        65726 |        408103

Restaurer des pages en cache

On utilise ici un paramètre de type bit-string représentant les pages à charger et décharger de la mémoire.

# select * 
  from pgfadvise_loader('pgbench_accounts', 0, true, true, 
                       B'101001'); -- Varbit décrivant les pages à manipuler
     relpath      | os_page_size | os_pages_free | pages_loaded | pages_unloaded 
------------------+--------------+---------------+--------------+----------------
 base/11874/16447 |         4096 |        408376 |            3 |              3

NOTE: pour la démo, seul 6 pages de données sont manipulées ci-dessus, 1 charge la page, 0 décharge la page.

par Cédric Villemain le mercredi 28 septembre 2016 à 07h52

mardi 27 septembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 25 septembre 2016

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en septembre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160925203116.GB17775@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Robert Haas pushed:

Heikki Linnakangas pushed:

Peter Eisentraut pushed:

Tom Lane pushed:

  • Be sure to rewind the tuplestore read pointer in non-leader CTEScan nodes. ExecInitCteScan supposed that it didn't have to do anything to the extra tuplestore read pointer it gets from tuplestore_alloc_read_pointer. However, it needs this read pointer to be positioned at the start of the tuplestore, while tuplestore_alloc_read_pointer is actually defined as cloning the current position of read pointer 0. In normal situations that accidentally works because we initialize the whole plan tree at once, before anything gets read. But it fails in an EvalPlanQual recheck, as illustrated in bug #14328 from Dima Pavlov. To fix, just forcibly rewind the pointer after tuplestore_alloc_read_pointer. The cost of doing so is negligible unless the tuplestore is already in TSS_READFILE state, which wouldn't happen in normal cases. We could consider altering tuplestore's API to make that case cheaper, but that would make for a more invasive back-patch and it doesn't seem worth it. This has been broken probably for as long as we've had CTEs, so back-patch to all supported branches. Discussion: <32468.1474548308@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/96dd77d349424f270d129f8f40e75f762ddcca7d
  • Remove nearly-unused SizeOfIptrData macro. Past refactorings have removed all but one reference to SizeOfIptrData (and that one place was in a pretty noncritical spot). Since nobody's complained, it seems probable that there are no supported compilers that don't think sizeof(ItemPointerData) is 6. If there are, we're wasting MAXALIGN per heap tuple anyway, so it's rather silly to worry about whether we can shave space in places like WAL records. Pavan Deolasee Discussion: <CABOikdOOawDda4hwLOT6zdA6MFfPLu3Z2YBZkX0JdayNS6JOeQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/8023b5827fbada6815ce269db4f3373ac77ec7c3
  • Avoid using PostmasterRandom() for DSM control segment ID. Commits 470d886c3 et al intended to fix the problem that the postmaster selected the same "random" DSM control segment ID on every start. But using PostmasterRandom() for that destroys the intended property that the delay between random_start_time and random_stop_time will be unpredictable. (Said delay is probably already more predictable than we could wish, but that doesn't mean that reducing it by a couple orders of magnitude is OK.) Revert the previous patch and add a comment warning against misuse of PostmasterRandom. Fix the original problem by calling srandom() early in PostmasterMain, using a low-security seed that will later be overwritten by PostmasterRandom. Discussion: <20789.1474390434@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/49a91b88e6c4afb840745c78942dd99ce125a6d6
  • Don't trust CreateFileMapping() to clear the error code on success. We must test GetLastError() even when CreateFileMapping() returns a non-null handle. If that value were left over from some previous system call, we might be fooled into thinking the segment already existed. Experimentation on Windows 7 suggests that CreateFileMapping() clears the error code on success, but it is not documented to do so, so let's not rely on that happening in all Windows releases. Amit Kapila Discussion: <20811.1474390987@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/8e6b4ee21f486e6800aaa6955ff3d98e1a521b49
  • Remove useless code. Apparent copy-and-pasteo in standby_desc_invalidations() had two entries for msg->id == SHAREDINVALRELMAP_ID. Aleksander Alekseev Discussion: <20160923090814.GB1238@e733> http://git.postgresql.org/pg/commitdiff/959ea7fa76120449efa5d85ae351abd1d6e5480c
  • Fix incorrect logic for excluding range constructor functions in pg_dump. Faulty AND/OR nesting in the WHERE clause of getFuncs' SQL query led to dumping range constructor functions if they are part of an extension and we're in binary-upgrade mode. Actually, we don't want to dump them separately even then, since CREATE TYPE AS RANGE will create the range's constructor functions regardless. Per report from Andrew Dunstan. It looks like this mistake was introduced by me, in commit b985d4877, in perhaps-overzealous refactoring to reduce code duplication. I'm suitably embarrassed. Report: <34854939-02d7-f591-5677-ce2994104599@dunslane.net> http://git.postgresql.org/pg/commitdiff/12f6eadffd075e39570a0170f6791de0b96ddfde
  • Doc: fix examples of # operators so they actually work. These worked as-is until around 7.0, but fail in newer versions because there are more operators named "#". Besides it's a bit inconsistent that only two of the examples on this page lack type names on their constants. Report: <20160923081530.1517.75670@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/5a7bae0699657315b487896810a30bd7425f6a08
  • Install TAP test infrastructure so it's available for extension testing. When configured with --enable-tap-tests, "make install" will now install the Perl support files for TAP testing where PGXS will find them. This allows extensions to rely on $(prove_check) even when being built out-of-tree. Back-patch to 9.4 where we first started to support TAP testing, to reduce the number of cases extension makefiles need to consider. Craig Ringer Discussion: <CAMsr+YFXv+2qne6xJW7z_25mYBtktRX5rpkrgrb+DRgQ_FxgHQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/c3a0818460a8f4a5e1a9109b035ac30e84b7e06f
  • Do a final round of updates on the 9.6 release notes. Set release date, document a few recent commits, do one last pass of copy-editing. http://git.postgresql.org/pg/commitdiff/98c2d3332b30006ff71add99bc9d619c9457e71f

Bruce Momjian pushed:

Correctifs en attente

David Fetter sent in another revision of a patch to add a contrib hook and associated GUCs for disallowing UPDATE and/or DELETE when no WHERE clause is present.

Vladimir Gordiychuk and Craig Ringer traded patches to permit pausing replication.

Thomas Munro sent in a patch to check the return code for postmaster death.

Takayuki Tsunakawa sent in a patch to make large shared_buffers settings effective on Windows.

Ashutosh Bapat sent in a patch to cache the result of code in add_paths_to_joinrel() which computes the set of target relations that should overlap parameterization of any proposed join path rather than re-computing it each time it's used in a query.

Daniel Vérité sent in another revision of a patch to enable psql variable hooks to return a boolean indicating whether a change is allowed.

Mithun Cy sent in another revision of a patch to add some tests to cover hash indexes.

Craig Ringer and Robert Haas traded patches to add a txid_status(bigint) to get status of an xact.

Jesper Pedersen sent in four more revisions of a patch to add hash index support to pageinspect.

Heikki Linnakangas and Fabien COELHO traded patches to refactor the pgbench meta commands into a state machine.

Mark Dilger sent in a patch to fix a place that shouldn't have cast away const.

Michaël Paquier sent in a patch to add suitable OOM warnings to palloc_extended() and pg_malloc_internal().

Haribabu Kommi sent in another revision of a patch to enable tracking the number of commands executed grouped by command tag.

Ashutosh Bapat sent in two more revisions of a patch to enable optimizations of partition-wise join for join between (declaratively) partitioned tables.

Michaël Paquier sent in three more revisions of a patch to track wait events for latches.

Amit Kapila sent in another revision of a patch to implement concurrent hash indexes.

David Cramer and Pavel Stěhule traded patches to add PL/PythonU support for multidimensional arrays.

Pavel Stěhule and Craig Ringer traded patches to add xmltable().

Anastasia Lubennikova sent in another revision of a patch to add covering & unique indexes.

Heikki Linnakangas sent in a patch to refactor the LogicalTapeSet/LogicalTape interface.

Ashutosh Bapat sent in a patch to refactor index creation so it plays better with declaratively partitioned tables.

Julian Markwort sent in a patch to allow specifying a 'custom' pgpassfile within the connection string along the lines of the existing parameters sslkey and sslcert.

Michaël Paquier sent in another revision of a patch to move sync routines of initdb into src/common, issue fsync more carefully in pg_receivexlog and pg_basebackup [-X] stream, add --nosync option to pg_basebackup, and switch pg_basebackup commands in Postgres.pm to use --no-sync.

Peter Eisentraut sent in another revision of a patch to change pg_dump by separate table and sequence data object types and fixing pg_upgrade to use this facility from pg_dump.

David Steele sent in another revision of a patch to exclude additional directories in pg_basebackup.

Robert Haas sent in a WIP patch to enable asynchronous execution.

Amit Kapila sent in another revision of a patch to do SetLastError(0) before calling CreateFileMapping.

Thomas Munro sent in a patch to add psql tab completions for LOCK TABLE ... IN ACCESS|ROW|SHARE.

Thomas Munro sent in a patch to refactor StartupXLOG.

Masahiko Sawada sent in another revision of a patch to add quorum commit.

Amit Kapila sent in another revision of a patch to add WAL for hash indexes.

par N Bougain le mardi 27 septembre 2016 à 21h34

lundi 19 septembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 18 septembre 2016

La 10ème édition annuelle des Prague PostgreSQL Developer Day 2017 (P2D2 2017) est une session de 2 jours qui se tiendra les 15 & 16 février 2017 à Prague (République Tchèque). Site en tchèque : http://www.p2d2.cz/

L'appel à conférenciers pour la PGConf US 2017 court jusqu'au 15 novembre 2016, 23h59 EST. Les notifications seront envoyées le 2 décembre. Le programme de l'événement sera publié le 2 janvier 2017 : http://www.pgconf.us/2017/submit/

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en septembre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160918222815.GB28319@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Kevin Grittner pushed:

Simon Riggs pushed:

Peter Eisentraut pushed:

  • pg_basebackup: Clean created directories on failure. Like initdb, clean up created data and xlog directories, unless the new -n/--noclean option is specified. Tablespace directories are not cleaned up, but a message is written about that. Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com> http://git.postgresql.org/pg/commitdiff/9083353b15c3cf8e7bbac104a81ad42281178cdf
  • Add overflow checks to money type input function. The money type input function did not have any overflow checks at all. There were some regression tests that purported to check for overflow, but they actually checked for the overflow behavior of the int8 type before casting to money. Remove those unnecessary checks and add some that actually check the money input function. Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr> http://git.postgresql.org/pg/commitdiff/656df624c0d7b50e1714f2a3a14e143e63799a80

Tom Lane pushed:

  • Docs: assorted minor cleanups. Standardize on "user_name" for a field name in related examples in ddl.sgml; before we had variously "user_name", "username", and "user". The last is flat wrong because it conflicts with a reserved word. Be consistent about entry capitalization in a table in func.sgml. Fix a typo in pgtrgm.sgml. Back-patch to 9.6 and 9.5 as relevant. Alexander Law http://git.postgresql.org/pg/commitdiff/42fd984c0b7b53d1bc961c9ed6bb84fe28eae52b
  • Fix executor/README to reflect disallowing SRFs in UPDATE. The parenthetical comment here is obsoleted by commit a4c35ea1c. Noted by Andres Freund. http://git.postgresql.org/pg/commitdiff/fdc79e1909dc3866a385ffb74bdd6ce6a082a300
  • Improve parser's and planner's handling of set-returning functions. Teach the parser to reject misplaced set-returning functions during parse analysis using p_expr_kind, in much the same way as we do for aggregates and window functions (cf commit eaccfded9). While this isn't complete (it misses nesting-based restrictions), it's much better than the previous error reporting for such cases, and it allows elimination of assorted ad-hoc expression_returns_set() error checks. We could add nesting checks later if it seems important to catch all cases at parse time. There is one case the parser will now throw error for although previous versions allowed it, which is SRFs in the tlist of an UPDATE. That never behaved sensibly (since it's ill-defined which generated row should be used to perform the update) and it's hard to see why it should not be treated as an error. It's a release-note-worthy change though. Also, add a new Query field hasTargetSRFs reporting whether there are any SRFs in the targetlist (including GROUP BY/ORDER BY expressions). The parser can now set that basically for free during parse analysis, and we can use it in a number of places to avoid expression_returns_set searches. (There will be more such checks soon.) In some places, this allows decontorting the logic since it's no longer expensive to check for SRFs in the tlist --- so I made the checks parallel to the handling of hasAggs/hasWindowFuncs wherever it seemed appropriate. catversion bump because adding a Query field changes stored rules. Andres Freund and Tom Lane Discussion: <24639.1473782855@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/a4c35ea1c2f05dd5b56739fcd0dc36a4870ea0c0
  • Be pickier about converting between Name and Datum. We were misapplying NameGetDatum() to plain C strings in some places. This worked, because it was just a pointer cast anyway, but it's a type cheat in some sense. Use CStringGetDatum instead, and modify the NameGetDatum macro so it won't compile if applied to something that's not a pointer to NameData. This should result in no changes to generated code, but it is logically cleaner. Mark Dilger, tweaked a bit by me Discussion: <EFD8AC94-4C1F-40C1-A5EA-304080089C1B@gmail.com> http://git.postgresql.org/pg/commitdiff/55c3391d1e6a201b5b891781d21fe682a8c64fe6
  • Tweak targetlist-SRF tests. Add a test case showing that we don't support SRFs in window-function arguments. Remove a duplicate test case for SRFs in aggregate arguments. http://git.postgresql.org/pg/commitdiff/a163c006ca3e6026546ee0f6c487a0dcfc66f82b
  • Tweak targetlist-SRF tests some more. Seems like it would be good to have a test case documenting the existing behavior for non-top-level SRFs. http://git.postgresql.org/pg/commitdiff/0dac5b5174bde3d6fb4b444a2aa4ca1f0091e258
  • Make min_parallel_relation_size's default value platform-independent. The documentation states that the default value is 8MB, but this was only true at BLCKSZ = 8kB, because the default was hard-coded as 1024. Make the code match the docs by computing the default as 8MB/BLCKSZ. Oversight in commit 75be66464, noted pursuant to a gripe from Peter E. Discussion: <90634e20-097a-e4fd-67d5-fb2c42f0dd71@2ndquadrant.com> http://git.postgresql.org/pg/commitdiff/5472ed3e9bc23eff0b4e457fe564ac667cb69441
  • Add debugging aid "bmsToString(Bitmapset *bms)". This function has no direct callers at present, but it's convenient for manual use in a debugger, rather than having to inspect memory and do bit-counting in your head. In passing, get rid of useless outBitmapset() wrapper around _outBitmapset(); let's just export the function that does the work. Likewise for outToken(). Ashutosh Bapat, tweaked a bit by me Discussion: <CAFjFpRdiht8e1HTVirbubr4YzaON5iZTzFJjq909y4sU8M_6eA@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/d8c61c9765339351409f06bbf964dcb8c1929e8b

Andres Freund pushed:

  • Add more tests for targetlist SRFs. We're considering changing the implementation of targetlist SRFs considerably, and a lot of the current behaviour isn't tested in our regression tests. Thus it seems useful to increase coverage to avoid accidental behaviour changes. It's quite possible that some of the plans here will require adjustments to avoid falling afoul of ordering differences (e.g. hashed group bys). The buildfarm will tell us. Reviewed-By: Tom Lane Discussion: <20160827214829.zo2dfb5jaikii5nw@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/bfe16d1a5dec0d23c917c37de646256c71e07ee8
  • Address portability issues in bfe16d1a5 test output. http://git.postgresql.org/pg/commitdiff/9f478b4f19d8e26300ae19e42c26343f5791e32a
  • Remove user_relns() SRF from regression tests. The output of the function changes whenever previous (or, as in this case, concurrent) tests leave a table in place. That causes unneeded churn. This should fix failures due to the tests added bfe16d1a5, like on lapwing, caused by the tsrf test running concurrently with misc. Those could also have been addressed by using temp tables, but that test has annoyed me before. Discussion: <27626.1473729905@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/0dba54f1666ead71c54ce100b39efda67596d297

Robert Haas pushed:

Heikki Linnakangas pushed:

  • Fix and clarify comments on replacement selection. These were modified by the patch to only use replacement selection for the first run in an external sort. http://git.postgresql.org/pg/commitdiff/c99dd5bfed23d9787dcf7d00197c1ed42bcfdb02
  • Support OpenSSL 1.1.0. Changes needed to build at all: - Check for SSL_new in configure, now that SSL_library_init is a macro. - Do not access struct members directly. This includes some new code in pgcrypto, to use the resource owner mechanism to ensure that we don't leak OpenSSL handles, now that we can't embed them in other structs anymore. - RAND_SSLeay() -> RAND_OpenSSL() Changes that were needed to silence deprecation warnings, but were not strictly necessary: - RAND_pseudo_bytes() -> RAND_bytes(). - SSL_library_init() and OpenSSL_config() -> OPENSSL_init_ssl() - ASN1_STRING_data() -> ASN1_STRING_get0_data() - DH_generate_parameters() -> DH_generate_parameters() - Locking callbacks are not needed with OpenSSL 1.1.0 anymore. (Good riddance!) Also change references to SSLEAY_VERSION_NUMBER with OPENSSL_VERSION_NUMBER, for the sake of consistency. OPENSSL_VERSION_NUMBER has existed since time immemorial. Fix SSL test suite to work with OpenSSL 1.1.0. CA certificates must have the "CA:true" basic constraint extension now, or OpenSSL will refuse them. Regenerate the test certificates with that. The "openssl" binary, used to generate the certificates, is also now more picky, and throws an error if an X509 extension is specified in "req_extensions", but that section is empty. Backpatch to all supported branches, per popular demand. In back-branches, we still support OpenSSL 0.9.7 and above. OpenSSL 0.9.6 should still work too, but I didn't test it. In master, we only support 0.9.8 and above. Patch by Andreas Karlsson, with additional changes by me. Discussion: <20160627151604.GD1051@msg.df7cb.de> http://git.postgresql.org/pg/commitdiff/593d4e47db7af1a3a5dd6b6b1971f181b5566dbd
  • Fix building with LibreSSL. LibreSSL defines OPENSSL_VERSION_NUMBER to claim that it is version 2.0.0, but it doesn't have the functions added in OpenSSL 1.1.0. Add autoconf checks for the individual functions we need, and stop relying on OPENSSL_VERSION_NUMBER. Backport to 9.5 and 9.6, like the patch that broke this. In the back-branches, there are still a few OPENSSL_VERSION_NUMBER checks left, to check for OpenSSL 0.9.8 or 0.9.7. I left them as they were - LibreSSL has all those functions, so they work as intended. Per buildfarm member curculio. Discussion: <2442.1473957669@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/5c6df67e0c961f68e73e7c1e6312211ed59da00a
  • Fix ecpg -? option on Windows, add -V alias for --version. This makes the -? and -V options work consistently with other binaries. --help and --version are now only recognized as the first option, i.e. "ecpg --foobar --help" no longer prints the help, but that's consistent with most of our other binaries, too. Backpatch to all supported versions. Haribabu Kommi Discussion: <CAJrrPGfnRXvmCzxq6Dy=stAWebfNHxiL+Y_z7uqksZUCkW_waQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/3fcc98c990ede7d3b415cc06eb6664aac6e7bbbc

Correctifs en attente

Michaël Paquier sent in a patch to add a pgstat_report_activity() call to walsender.

Craig Ringer and Pavel Stěhule traded patches to add xmltable().

Dagfinn Ilmari Mannsåker sent in a patch to add psql tab completion for the recently-added ALTER TYPE … RENAME VALUE.

Corey Huinker sent in two more revisions of a patch to let file_fdw access COPY FROM PROGRAM.

Craig Ringer sent in two revisions of a patch to install the Perl TAP tests, add install rules for the isolation tester, and note that src/test/Makefile is not called from src/Makefile.

Michaël Paquier sent in a patch to move the fsync routines of initdb into src/common, issue fsync more carefully in pg_receivexlog and pg_basebackup [-X] stream, add a --no-sync option to pg_basebackup, and switch pg_basebackup commands in Postgres.pm to use --no-sync.

Etsuro Fujita sent in another revision of a patch to push more FULL JOINs to the PostgreSQL FDW.

Heikki Linnakangas sent in two revisions of a patch to change the way pre-reading in external sort's merge phase works.

Alexander Korotkov sent in another revision of a patch to implement partial sorting.

Mithun Cy sent in another revision of a patch to cache hash index meta page.

Haribabu Kommi sent in a patch to replace most of the getimeofday function calls, except timeofday(user callable) and GetCurrentTimestamp functions with clock_gettime.

Andrew Borodin sent in two more revisions of a patch to implement penalty functions for GiST in the cube contrib extension.

Michaël Paquier sent in a patch to make PostgresNode.pm the new --noclean option in its call.

KaiGai Kohei sent in another revision of a patch to implement PassDownLimitBound for ForeignScan/CustomScan.

Amit Kapila sent in two more revisions of a patch to implement WAL for hash indexes.

Amit Kapila sent in two more revisions of a patch to implement concurrent hash indexes.

Fabien COELHO sent in another revision of a patch to enable pgbench to store results into variables.

Anastasia Lubennikova sent in another revision of a patch to add covering + unique indexes.

Andrew Borodin sent in a patch to change the interpretation of NaN returned from the GiST penalty function from "best fit" to "worst fit."

Tom Lane sent in a PoC patch to put srfs in separate result nodes.

Pavan Deolasee and Claudio Freire traded patches to allow VACUUM to use over 1GB of work_mem.

Thomas Munro sent in another revision of a patch to use kqueue on platforms where it helps.

Craig Ringer sent in another revision of a patch to install the Perl TAP tests, add install rules for isolation tester, and add txid_status(bigint).

Kuntal Ghosh sent in another revision of a patch to add a WAL consistency check facility.

Marco Pfatschbacher sent in a patch to keep one postmaster monitoring pipe per process.

Amit Langote sent in another revision of a patch to implement declarative partitioning.

Rajkumar Raghuwanshi sent in another revision of a patch to enable piecewise joins of partitioned tables.

Daniel Vérité sent in a patch to create hooks into psql variables to return a boolean indicating whether a change is allowed.

Jeevan Chalke sent in another revision of a patch to enable pushing aggregates to a foreign server.

Masahiko Sawada sent in another revision of a patch to enable quorum commit for multiple synchronous replication.

Stas Kelvich sent in another revision of a patch to speed up 2PC transactions.

Julien Rouhaud sent in another revision of a patch to rename the max-worker-processes GUC to max-parallel-workers.

Kyotaro HORIGUCHI sent in another revision of a patch to refactor psql's tab completion and use that refactoring to implement IF (NOT) EXISTS completion.

MauMau sent in a patch to fix the omission of pg_recvlogical from the Windows installation.

Yury Zhuravlev sent in another revision of a patch to CMake-ify PostgreSQL.

par N Bougain le lundi 19 septembre 2016 à 21h04

Nouvelles hebdomadaires de PostgreSQL - 11 septembre 2016

Le groupe d'utilisateurs coréens tiendra son premier PGDay le 15 octobre à Séoul : http://pgday.postgresql.kr/

Le PGDay 2016 à Austin aura lieu le 12 novembre 2016. Date limite de candidature au 21 septembre 2016 minuit CST. Détails et formulaire de candidature : https://www.postgresql.us/events/2016/austin

Le PGDay.IT 2016 aura lieu à Prato le 13 décembre 2016 : http://pgday.it

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en septembre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160911220711.GA25430@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Simon Riggs pushed:

Tom Lane pushed:

  • Relax transactional restrictions on ALTER TYPE ... ADD VALUE. To prevent possibly breaking indexes on enum columns, we must keep uncommitted enum values from getting stored in tables, unless we can be sure that any such column is new in the current transaction. Formerly, we enforced this by disallowing ALTER TYPE ... ADD VALUE from being executed at all in a transaction block, unless the target enum type had been created in the current transaction. This patch removes that restriction, and instead insists that an uncommitted enum value can't be referenced unless it belongs to an enum type created in the same transaction as the value. Per discussion, this should be a bit less onerous. It does require each function that could possibly return a new enum value to SQL operations to check this restriction, but there aren't so many of those that this seems unmaintainable. Andrew Dunstan and Tom Lane Discussion: <4075.1459088427@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/15bc038f9bcd1a9af3f625caffafc7c20322202d
  • Make locale-dependent regex character classes work for large char codes. Previously, we failed to recognize Unicode characters above U+7FF as being members of locale-dependent character classes such as [[:alpha:]]. (Actually, the same problem occurs for large pg_wchar values in any multibyte encoding, but UTF8 is the only case people have actually complained about.) It's impractical to get Spencer's original code to handle character classes or ranges containing many thousands of characters, because it insists on considering each member character individually at regex compile time, whether or not the character will ever be of interest at run time. To fix, choose a cutoff point MAX_SIMPLE_CHR below which we process characters individually as before, and deal with entire ranges or classes as single entities above that. We can actually make things cheaper than before for chars below the cutoff, because the color map can now be a simple linear array for those chars, rather than the multilevel tree structure Spencer designed. It's more expensive than before for chars above the cutoff, because we must do a binary search in a list of high chars and char ranges used in the regex pattern, plus call iswalpha() and friends for each locale-dependent character class used in the pattern. However, multibyte encodings are normally designed to give smaller codes to popular characters, so that we can expect that the slow path will be taken relatively infrequently. In any case, the speed penalty appears minor except when we have to apply iswalpha() etc. to high character codes at runtime --- and the previous coding gave wrong answers for those cases, so whether it was faster is moot. Tom Lane, reviewed by Heikki Linnakangas Discussion: <15563.1471913698@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/c54159d44ceaba26ceda9fea1804f0de122a8f30
  • Cosmetic code cleanup in commands/extension.c. Some of the comments added by the CREATE EXTENSION CASCADE patch were a bit sloppy, and I didn't care for redeclaring the same local variable inside a nested block either. No functional changes. http://git.postgresql.org/pg/commitdiff/25794e841e5b86a0f90fac7f7f851e5d950e51e2
  • Repair whitespace in initdb message. What used to be four spaces somehow turned into a tab and a couple of spaces in commit a00c58314, no doubt from overhelpful emacs autoindent. Noted by Peter Eisentraut. http://git.postgresql.org/pg/commitdiff/a2ee579b6def8e0bde876c6c2fc9d4b8ec2b6b67
  • Teach appendShellString() to not quote strings containing "-". Brain fade in commit a00c58314: I was thinking that a string starting with "-" could be taken as a switch depending on command line syntax. That's true, but having appendShellString() quote it will not help, so we may as well not do so. Per complaint from Peter Eisentraut. http://git.postgresql.org/pg/commitdiff/cdc70597c9ba62aad08a46e55c0c783bf4c21c9c
  • Guard against possible memory allocation botch in batchmemtuples(). Negative availMemLessRefund would be problematic. It's not entirely clear whether the case can be hit in the code as it stands, but this seems like good future-proofing in any case. While we're at it, insist that the value be not merely positive but not tiny, so as to avoid doing a lot of repalloc work for little gain. Peter Geoghegan Discussion: <CAM3SWZRVkuUB68DbAkgw=532gW0f+fofKueAMsY7hVYi68MuYQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/f032722f86a709277d44a317a3bc8acd74861a72
  • Doc: small improvements for documentation about VACUUM freezing. Mostly, explain how row xmin's used to be replaced by FrozenTransactionId and no longer are. Do a little copy-editing on the side. Per discussion with Egor Rogov. Back-patch to 9.4 where the behavioral change occurred. Discussion: <575D7955.6060209@postgrespro.ru> http://git.postgresql.org/pg/commitdiff/975768f8eae2581b89ceafe8b16a77ff375207fe
  • Add a HINT for client-vs-server COPY failure cases. Users often get confused between COPY and \copy and try to use client-side paths with COPY. The server then cannot find the file (if remote), or sees a permissions problem (if local), or some variant of that. Emit a hint about this in the most common cases. In future we might want to expand the set of errnos for which the hint gets printed, but be conservative for now. Craig Ringer, reviewed by Christoph Berg and Tom Lane Discussion: <CAMsr+YEqtD97qPEzQDqrCt5QiqPbWP_X4hmvy2pQzWC0GWiyPA@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/4f405c8ef4704b7fa7fd8ac14a66c5f5d13722c4
  • Doc: minor documentation improvements about extensions. Document the formerly-undocumented behavior that schema and comment control-file entries for an extension are honored only during initial installation, whereas other properties are also honored during updates. While at it, do some copy-editing on the recently-added docs for CREATE EXTENSION ... CASCADE, use links for some formerly vague cross references, and make a couple other minor improvements. Back-patch to 9.6 where CASCADE was added. The other parts of this could go further back, but they're probably not important enough to bother. http://git.postgresql.org/pg/commitdiff/bd180b607927c7757af17cd6fce0e545e5c48584
  • Support renaming an existing value of an enum type. Not much to be said about this patch: it does what it says on the tin. In passing, rename AlterEnumStmt.skipIfExists to skipIfNewValExists to clarify what it actually does. In the discussion of this patch we considered supporting other similar options, such as IF EXISTS on the type as a whole or IF NOT EXISTS on the target name. This patch doesn't actually add any such feature, but it might happen later. Dagfinn Ilmari MannsÃ¥ker, reviewed by Emre Hasegeli Discussion: <CAO=2mx6uvgPaPDf-rHqG8=1MZnGyVDMQeh8zS4euRyyg4D35OQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/0ab9c56d0fe3acc9d4717a9cbac6ef3369275b90
  • Don't print database's tablespace in pg_dump -C --no-tablespaces output. If the database has a non-default tablespace, we emitted a TABLESPACE clause in the CREATE DATABASE command emitted by -C, even if --no-tablespaces was also specified. This seems wrong, and it's inconsistent with what pg_dumpall does, so change it. Per bug #14315 from Danylo Hlynskyi. Back-patch to 9.5. The bug is much older, but it'd be a more invasive change before 9.5 because dumpDatabase() hasn't got an easy way to get to the outputNoTablespaces flag. Doesn't seem worth the work given the lack of previous complaints. Report: <20160908081953.1402.75347@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/e97e9c57bd22b2dfbfaf41f7d5c69789f7fad554
  • Allow pg_dump to dump non-extension members of an extension-owned schema. Previously, if a schema was created by an extension, a normal pg_dump run (not --binary-upgrade) would summarily skip every object in that schema. In a case where an extension creates a schema and then users create other objects within that schema, this does the wrong thing: we want pg_dump to skip the schema but still create the non-extension-owned objects. There's no easy way to fix this pre-9.6, because in earlier versions the "dump" status for a schema is just a bool and there's no way to distinguish "dump me" from "dump my members". However, as of 9.6 we do have enough state to represent that, so this is a simple correction of the logic in selectDumpableNamespace. In passing, make some cosmetic fixes in nearby code. Martín Marqués, reviewed by Michael Paquier Discussion: <99581032-71de-6466-c325-069861f1947d@2ndquadrant.com> http://git.postgresql.org/pg/commitdiff/df5d9bb8d5074138e6fea63ac8acd9b95a0eb859
  • Avoid reporting "cache lookup failed" for some user-reachable cases. We have a not-terribly-thoroughly-enforced-yet project policy that internal errors with SQLSTATE XX000 (ie, plain elog) should not be triggerable from SQL. record_in, domain_in, and PL validator functions all failed to meet this standard, because they threw plain elog("cache lookup failed for XXX") errors on bad OIDs, and those are all invokable from SQL. For record_in, the best fix is to upgrade typcache.c (lookup_type_cache) to throw a user-facing error for this case. That seems consistent because it was more than halfway there already, having user-facing errors for shell types and non-composite types. Having done that, tweak domain_in to rely on the typcache to throw an appropriate error. (This costs little because InitDomainConstraintRef would fetch the typcache entry anyway.) For the PL validator functions, we already have a single choke point at CheckFunctionValidatorAccess, so just fix its error to be user-facing. Dilip Kumar, reviewed by Haribabu Kommi Discussion: <87wpxfygg9.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/967a7b0fc9c8f4e07b697148238566203cb060de
  • In PageIndexTupleDelete, don't assume stored item lengths are MAXALIGNed. PageAddItem stores the item length as-is. It MAXALIGN's the amount of space actually allocated for each tuple, but not the stored length. PageRepairFragmentation, PageIndexMultiDelete, and PageIndexDeleteNoCompact are all on board with this and MAXALIGN item lengths after fetching them. But PageIndexTupleDelete expects the stored length to be a MAXALIGN multiple already. This accidentally works for existing index AMs because they all maxalign their tuple sizes internally; but we don't do that for heap tuples, and it shouldn't be a requirement for index tuples either. So, sync PageIndexTupleDelete with the rest of bufpage.c by having it maxalign the item size after fetching. Also add a check that pd_special is maxaligned, to ensure that the test "(offset + size) > phdr->pd_special" is still doing the right thing. (If offset and pd_special are aligned, it doesn't matter whether size is.) Again, this is in sync with the rest of the routines here, except for PageAddItem which doesn't test because it doesn't actually do anything for which pd_special alignment matters. This shouldn't have any immediate functional impact; it just adds the flexibility to use PageIndexTupleDelete on index tuples with non-aligned lengths. Discussion: <3814.1473366762@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/984d0a14e8d0141a68da5bd56ce6821042298904
  • Invent PageIndexTupleOverwrite, and teach BRIN and GiST to use it. PageIndexTupleOverwrite performs approximately the same function as PageIndexTupleDelete (or PageIndexDeleteNoCompact) followed by PageAddItem targeting the same item pointer offset. But in the case where the new tuple is the same size as the old, it avoids shuffling other data around on the page, because the new tuple is placed where the old one was rather than being appended to the end of the page. This has been shown to provide a substantial speedup for some GiST use-cases. Also, this change allows some API simplifications: we can get rid of the rather klugy and error-prone PAI_ALLOW_FAR_OFFSET flag for PageAddItemExtended, since that was used only to cover a corner case for BRIN that's better expressed by using PageIndexTupleOverwrite. Note that this patch causes a rather subtle WAL incompatibility: the physical page content change represented by certain WAL records is now different than it was before, because while the tuples have the same itempointer line numbers, the tuples themselves are in different places. I have not bumped the WAL version number because I think it doesn't matter unless you are trying to do bitwise comparisons of original and replayed pages, and in any case we're early in a devel cycle and there will probably be more WAL changes before v10 gets out the door. There is probably room to make use of PageIndexTupleOverwrite in SP-GiST and GIN too, but that is left for a future patch. Andrey Borodin, reviewed by Anastasia Lubennikova, whacked around a bit by me Discussion: <CAJEAwVGQjGGOj6mMSgMwGvtFd5Kwe6VFAxY=uEPZWMDjzbn4VQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/b1328d78f88cdf4f7504004159e530b776f0de16
  • Convert PageAddItem into a macro to save a few cycles. Nowadays this is just a backwards-compatibility wrapper around PageAddItemExtended, so let's avoid the extra level of function call. In addition, because pretty much all callers are passing constants for the two bool arguments, compilers will be able to constant-fold the conversion to a flags bitmask. Discussion: <552.1473445163@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/1a4be103a523db8d47b464463ba175cc664442b2
  • Rewrite PageIndexDeleteNoCompact into a form that only deletes 1 tuple. The full generality of deleting an arbitrary number of tuples is no longer needed, so let's save some code and cycles by replacing the original coding with an implementation based on PageIndexTupleDelete. We can always get back the old code from git if we need it again for new callers (though I don't care for its willingness to mess with line pointers it wasn't told to mess with). Discussion: <552.1473445163@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/24992c6db9fd40f342db1f22747ec9e56483796d
  • Fix miserable coding in pg_stat_get_activity(). Commit dd1a3bccc replaced a test on whether a subroutine returned a null pointer with a test on whether &pointer->backendStatus was null. This accidentally failed to fail, at least on common compilers, because backendStatus is the first field in the struct; but it was surely trouble waiting to happen. Commit f91feba87 then messed things up further, changing the logic to local_beentry = pgstat_fetch_stat_local_beentry(curr_backend); if (!local_beentry) continue; beentry = &local_beentry->backendStatus; if (!beentry) { where the second "if" is now dead code, so that the intended behavior of printing a row with "<backend information not available>" cannot occur. I suspect this is all moot because pgstat_fetch_stat_local_beentry will never actually return null in this function's usage, but it's still very poor coding. Repair back to 9.4 where the original problem was introduced. http://git.postgresql.org/pg/commitdiff/ddc889317912fd8b654439701195a43cecfd4e79
  • Improve unreachability recognition in elog() macro. Some experimentation with an older version of gcc showed that it is able to determine whether "if (elevel_ >= ERROR)" is compile-time constant if elevel_ is declared "const", but otherwise not so much. We had accounted for that in ereport() but were too miserly with braces to make it so in elog(). I don't know how many currently-interesting compilers have the same quirk, but in case it will save some code space, let's make sure that elog() is on the same footing as ereport() for this purpose. Back-patch to 9.3 where we introduced pg_unreachable() calls into elog/ereport. http://git.postgresql.org/pg/commitdiff/f2717c79eeecaf5997016d52fd81881301dcfc5e
  • Fix and simplify MSVC build's handling of xml/xslt/uuid dependencies. Solution.pm mistakenly believed that the xml option requires the xslt option, when actually the dependency is the other way around; and it believed that libxml requires libiconv, which is not necessarily so, so we shouldn't enforce it here. Fix the option cross-checking logic. Also, since AddProject already takes care of adding libxml and libxslt include and library dependencies to every project, there's no need for the custom code that did that in mkvcbuild. While at it, let's handle the similar dependencies for uuid in a similar fashion. Given the lack of field complaints about these overly strict build dependency requirements, there seems no need for a back-patch. Michael Paquier Discussion: <CAB7nPqR0+gpu3mRQvFjf-V-bMxmiSJ6NpTg9_WzVDL+a31cV2g@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/28e5e5648cc3666537c393b2636c4aa34fdb22c1
  • Allow CREATE EXTENSION to follow extension update paths. Previously, to update an extension you had to produce both a version-update script and a new base installation script. It's become more and more obvious that that's tedious, duplicative, and error-prone. This patch attempts to improve matters by allowing the new base installation script to be omitted. CREATE EXTENSION will install a requested version if it can find a base script and a chain of update scripts that will get there. As in the existing update logic, shorter chains are preferred if there's more than one possibility, with an arbitrary tie-break rule for chains of equal length. Also adjust the pg_available_extension_versions view to show such versions as installable. While at it, refactor the code so that CASCADE processing works for extensions requested during ApplyExtensionUpdates(). Without this, addition of a new requirement in an updated extension would require creating a new base script, even if there was no other reason to do that. (It would be easy at this point to add a CASCADE option to ALTER EXTENSION UPDATE, to allow the same thing to happen during a manually-commanded version update, but I have not done that here.) Tom Lane, reviewed by Andres Freund Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de> http://git.postgresql.org/pg/commitdiff/40b449ae84dcf71177d7749a7b0c582b64dc15f0

Bruce Momjian pushed:

Ãlvaro Herrera pushed:

  • Have "make coverage" recurse into contrib as well http://git.postgresql.org/pg/commitdiff/2093f6630500c9d5e122748ac7d3719855d71174
  • Fix two src/test/modules Makefiles. commit_ts and test_pg_dump were declaring targets before including the PGXS stanza, which meant that the "all" target customarily defined as the first (and therefore default target) was not the default anymore. Fix that by moving those target definitions to after PGXS. commit_ts was initially good, but I broke it in commit 9def031bd2; test_pg_dump was born broken, probably copying from commit_ts' mistake. In passing, fix a comment mistake in test_pg_dump/Makefile. Backpatch to 9.6. Noted by Tom Lane. http://git.postgresql.org/pg/commitdiff/19acee8c5adb68b96222e41c084efbc9b31d397a
  • Fix locking a tuple updated by an aborted (sub)transaction. When heap_lock_tuple decides to follow the update chain, it tried to also lock any version of the tuple that was created by an update that was subsequently rolled back. This is pointless, since for all intents and purposes that tuple exists no more; and moreover it causes misbehavior, as reported independently by Marko Tiikkaja and Marti Raudsepp: some SELECT FOR UPDATE/SHARE queries may fail to return the tuples, and assertion-enabled builds crash. Fix by having heap_lock_updated_tuple test the xmin and return success immediately if the tuple was created by an aborted transaction. The condition where tuples become invisible occurs when an updated tuple chain is followed by heap_lock_updated_tuple, which reports the problem as HeapTupleSelfUpdated to its caller heap_lock_tuple, which in turn propagates that code outwards possibly leading the calling code (ExecLockRows) to believe that the tuple exists no longer. Backpatch to 9.3. Only on 9.5 and newer this leads to a visible failure, because of commit 27846f02c176; before that, heap_lock_tuple skips the whole dance when the tuple is already locked by the same transaction, because of the ancient HeapTupleSatisfiesUpdate behavior. Still, the buggy condition may also exist in more convoluted scenarios involving concurrent transactions, so it seems safer to fix the bug in the old branches too. Discussion: https://www.postgresql.org/message-id/CABRT9RC81YUf1=jsmWopcKJEro=VoeG2ou6sPwyOUTx_qteRsg@mail.gmail.com https://www.postgresql.org/message-id/48d3eade-98d3-8b9a-477e-1a8dc32a724d@joh.to http://git.postgresql.org/pg/commitdiff/5c609a742f294907512b946dbaf1feaa3b71ddc7

Peter Eisentraut pushed:

  • Add location field to DefElem. Add a location field to the DefElem struct, used to parse many utility commands. Update various error messages to supply error position information. To propogate the error position information in a more systematic way, create a ParseState in standard_ProcessUtility() and pass that to interested functions implementing the utility commands. This seems better than passing the query string and then reassembling a parse state ad hoc, which violates the encapsulation of the ParseState type. Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> http://git.postgresql.org/pg/commitdiff/49eb0fd0972d14014dd3533b1f1bf8c94c899883
  • Make better use of existing enums in plpgsql plpgsql.h defines a number of enums, but most of the code passes them around as ints. Update structs and function prototypes to take enum types instead. This clarifies the struct definitions in plpgsql.h in particular. Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> http://git.postgresql.org/pg/commitdiff/e0013deb5983303d945aacd56909ac4ce227fde1

Noah Misch pushed:

Andres Freund pushed:

  • Fix mdtruncate() to close fd.c handle of deleted segments. mdtruncate() forgot to FileClose() a segment's mdfd_vfd, when deleting it. That lead to a fd.c handle to a truncated file being kept open until backend exit. The issue appears to have been introduced way back in 1a5c450f3024ac5, before that the handle was closed inside FileUnlink(). The impact of this bug is limited - only VACUUM and ON COMMIT TRUNCATE for temporary tables, truncate files in place (i.e. TRUNCATE itself is not affected), and the relation has to be bigger than 1GB. The consequences of a leaked fd.c handle aren't severe either. Discussion: <20160908220748.oqh37ukwqqncbl3n@alap3.anarazel.de> Backpatch: all supported releases http://git.postgresql.org/pg/commitdiff/769fd9d8e06bf862334a0e04134a3d2c665e5e5b
  • Improve scalability of md.c for large relations. So far md.c used a linked list of segments. That proved to be a problem when processing large relations, because every smgr.c/md.c level access to a page incurred walking through a linked list of all preceding segments. Thus making accessing pages O(#segments). Replace the linked list of segments hanging off SMgrRelationData with an array of opened segments. That allows O(1) access to individual segments, if they've previously been opened. Discussion: <20140331101001.GE13135@alap3.anarazel.de> Reviewed-By: Peter Geoghegan, Tom Lane (in an older version) http://git.postgresql.org/pg/commitdiff/45e191e3aa62d47a8bc1a33f784286b2051f45cb
  • Faster PageIsVerified() for the all zeroes case. That's primarily useful for testing very large relations, using sparse files. Discussion: <20140331101001.GE13135@alap3.anarazel.de> Reviewed-By: Peter Geoghegan http://git.postgresql.org/pg/commitdiff/417fefaf089fc0b73607cbbe8bcd0bc9e89d08ef

Heikki Linnakangas pushed:

  • Implement binary heap replace-top operation in a smarter way. In external sort's merge phase, we maintain a binary heap holding the next tuple from each input tape. On each step, the topmost tuple is returned, and replaced with the next tuple from the same tape. We were doing the replacement by deleting the top node in one operation, and inserting the next tuple after that. However, you can do a "replace-top" operation more efficiently, in one "sift-up". A deletion will always walk the heap from top to bottom, but in a replacement, we can stop as soon as we find the right place for the new tuple. This is particularly helpful, if the tapes are not in completely random order, so that the next tuple from a tape is likely to land near the top of the heap. Peter Geoghegan, reviewed by Claudio Freire, with some editing by me. Discussion: <CAM3SWZRhBhiknTF_=NjDSnNZ11hx=U_SEYwbc5vd=x7M4mMiCw@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/24598337c8d214ba8dcf354130b72c49636bba69

Correctifs en attente

Michaël Paquier and Christian Ullrich traded patches to enable parallel builds with MSVC.

Haribabu Kommi sent in another revision of a patch to add a pg_hba_file_settings system view.

Craig Ringer sent in a patch to send catalog_xmin in hot standby feedback protocol, make walsender respect catalog_xmin in hot standby feedback messages, allow GetOldestXmin(...) to optionally disregard the catalog_xmin, and send catalog_xmin separately in hot_standby_feedback messages.

Heikki Linnakangas sent in a patch to fix compilation with OpenSSL 1.1 and silence deprecation warnings with OpenSSL 1.1.

Abhijit Menon-Sen sent in another revision of a patch to convert recovery.conf settings to GUCs.

KaiGai Kohei sent in another revision of a patch to implement PassDownLimitBound for ForeignScan/CustomScan.

Michaël Paquier sent in another revision of a patch to speed up 2PC transactions.

Mithun Cy sent in another revision of a patch to cache hash index meta pages.

Craig Ringer and Vladimir Gordiychuk traded patches to respect client-initiated CopyDone in walsender and allow streaming to be interrupted during ReorderBufferCommit.

Claudio Freire, Masahiko Sawada, and Simon Riggs traded patches to enable using over 1GB of work_mem in VACUUM.

Tom Lane sent in two revisions of a patch to let CREATE EXTENSION follow update chains.

Simon Riggs sent in another revision of a patch to add a "patient" mode to LOCK TABLE.

Daniel Vérité sent in another revision of a patch to add batch/pipelining support for libpq.

Michaël Paquier and Christian Ullrich traded patches to do some cleanups in pgwin32_putenv, fix the load race in pgwin32_putenv() (and open the unload race), and pin any DLL as soon as seen when looking for _putenv on Windows.

Vinayak Pokale sent in a patch to add pg_fdw_xact_resolver().

Gerdan Rezende dos Santos sent in another revision of a patch to add an option to pg_dumpall to exclude tables from the dump.

Kyotaro HORIGUCHI sent in a patch to fix an illegal SJIS mapping.

Etsuro Fujita sent in another revision of a patch to push more UPDATEs and DELETEs to the PostgreSQL FDW.

Ashutosh Bapat and Amit Langote traded patches to add declarative table partitioning.

Kyotaro HORIGUCHI sent in another revision of a patch to add IF NOT EXISTS support to psql's tab completion.

David Steele sent in two more revisions of a patch to exclude additional directories in pg_basebackup.

Pavan Deolasee sent in a patch to override compile time log levels of specific messages/modules.

Heikki Linnakangas sent in four revisions of a patch to remove tuplesort merge pre-reading.

Fabien COELHO sent in another revision of a patch to pgbench to permit storing select results into variables.

Peter Geoghegan sent in another revision of a patch to add amcheck, a B-Tree integrity checking tool.

Jim Nasby sent in another revision of a patch to fix a mistaken comment in nodeCtescan.c.

Vik Fearing sent in two more revisions of a patch to add long options for pg_ctl waiting.

Kyotaro HORIGUCHI sent in a PoC patch to use a radix tree to encoding characters of Shift JIS.

Michaël Paquier sent in another revision of a patch to forbid the use of LF and CR characters in database and role names.

Ildar Musin sent in another revision of a patch to enable index-only scans for some expressions.

Amit Kapila sent in another revision of a patch to enable WAL logging for hash indexes.

Michaël Paquier sent in three revisions of a patch to remove a useless dependency assumption libxml2 -> libxslt in MSVC scripts.

Kuntal Ghosh sent in two more revisions of a patch to add a WAL consistency check facility.

Masahiko Sawada sent in a patch to make lazy_scan_heap output how many all_frozen pages are skipped and add cheat_vacuum_table_size.

Rahila Syed sent in another revision of a patch to get psql to error out when someone tries to set autocommit at the wrong spot.

Ashutosh Bapat sent in another revision of a patch to enable pushing aggregates to a foreign server.

Masahiko Sawada sent in a patch to add a --disable-page-skipping option to the vacuumdb command.

Peter Eisentraut sent in another revision of a patch to add overflow checks to the money type input function.

Victor Wagner sent in another revision of a patch to implement failover on libpq connect level.

Stephen Frost sent in another revision of a patch to add support for restrictive RLS policies.

Petr Jelinek sent in another revision of a patch to implement logical replication.

Adam Brightwell sent in a patch to fix a mismatch between RLS and COPY.

Andrew Borodin and Mikhail Bakhterev traded patches to improve the GiST penalty functions.

Peter Eisentraut sent in another revision of a patch to add CREATE SEQUENCE AS <data type> clause.

Ashutosh Bapat sent in a patch to support partition-wise joins for partitioned tables.

Etsuro Fujita sent in another revision of a patch to push down more full joins in postgres_fdw.

Pavel Stehule sent in two more revisions of a patch to add xmltable().

Kevin Grittner sent in another revision of a patch to add trigger transition tables.

Thomas Munro sent in another revision of a patch to use kqeue when available.

Dmitry Dolgov sent in a patch to add generic type subscription.

Heikki Linnakangas sent in a patch to allow temp file access tracing.

Vitaly Burovoy sent in another revision of a patch to add tab completion for CREATE DATABASE ... TEMPLATE ... to psql.

Amit Kapila sent in a patch to add some simple tests for pg_stat_statements.

Mattia sent in a patch to allow to_date() and to_timestamp() to accept localized month names.

Corey Huinker sent in another revision of a patch to let file_fdw access COPY FROM PROGRAM.

Peter Eisentraut sent in another revision of a patch to add a pg_sequences view.

Peter Geoghegan sent in another revision of a patch to add parallel tuplesort for parallel B-tree index creation.

par N Bougain le lundi 19 septembre 2016 à 20h46

mercredi 7 septembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 4 septembre 2016

PostgreSQL 9.6 Release Candidate 1 est disponible, à vos tests ! http://www.postgresql.org/docs/devel/static/release-9-6.html

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en septembre

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160905054301.GA25294@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Tom Lane pushed:

  • Make another editorial pass over the 9.6 release notes. I think they're pretty much release-quality now. http://git.postgresql.org/pg/commitdiff/39d866433e6fb1c385eee8dc67843097b8703add
  • Fix stray reference to the old genbki.sh script. Per Tomas Vondra. http://git.postgresql.org/pg/commitdiff/b899ccbb49cbcf8431b3af43fcf3faf91e6a28c6
  • Make AllocSetContextCreate throw an error for bad context-size parameters. The previous behavior was to silently change them to something valid. That obscured the bugs fixed in commit ea268cdc9, and generally seems less useful than complaining. Unlike the previous commit, though, we'll do this in HEAD only --- it's a bit too late to be possibly breaking third-party code in 9.6. Discussion: <CA+TgmobNcELVd3QmLD3tx=w7+CokRQiC4_U0txjz=WHpfdkU=w@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/cf34fdbbe1452b9e19c0956bc48494889e1b2777
  • Fix initdb misbehavior when user mis-enters superuser password. While testing simple_prompt() revisions, I happened to notice that current initdb behaves rather badly when --pwprompt is specified and the user miskeys the second password. It complains about the mismatch, does "rm -rf" on the data directory, and exits. The problem is that since commit c4a8812cf, there's a standalone backend sitting waiting for commands at that point. It gets unhappy about its datadir having gone away, and spews a PANIC message at the user, which is not nice. (And the shell then adds to the mess with meaningless bleating about a core dump...) We don't really want that sort of thing to happen unless there's an internal failure in initdb, which this surely is not. The best fix seems to be to move the collection of the password earlier, so that it's done essentially as part of argument collection, rather than at the rather ad-hoc time it was done before. Back-patch to 9.6 where the problem was introduced. http://git.postgresql.org/pg/commitdiff/37f6fd1eaab698983ca1fb2a036d52381347ac71
  • Simplify correct use of simple_prompt(). The previous API for this function had it returning a malloc'd string. That meant that callers had to check for NULL return, which few of them were doing, and it also meant that callers had to remember to free() the string later, which required extra logic in most cases. Instead, make simple_prompt() write into a buffer supplied by the caller. Anywhere that the maximum required input length is reasonably small, which is almost all of the callers, we can just use a local or static array as the buffer instead of dealing with malloc/free. A fair number of callers used "pointer == NULL" as a proxy for "haven't requested the password yet". Maintaining the same behavior requires adding a separate boolean flag for that, which adds back some of the complexity we save by removing free()s. Nonetheless, this nets out at a small reduction in overall code size, and considerably less code than we would have had if we'd added the missing NULL-return checks everywhere they were needed. In passing, clean up the API comment for simple_prompt() and get rid of a very-unnecessary malloc/free in its Windows code path. This is nominally a bug fix, but it does not seem worth back-patching, because the actual risk of an OOM failure in any of these places seems pretty tiny, and all of them are client-side not server-side anyway. This patch is by me, but it owes a great deal to Michael Paquier who identified the problem and drafted a patch for fixing it the other way. Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/9daec77e165de461fca9d5bc3ece86a91aba5804
  • Fix a bunch of places that called malloc and friends with no NULL check. Where possible, use palloc or pg_malloc instead; otherwise, insert explicit NULL checks. Generally speaking, these are places where an actual OOM is quite unlikely, either because they're in client programs that don't allocate all that much, or they're very early in process startup so that we'd likely have had a fork() failure instead. Hence, no back-patch, even though this is nominally a bug fix. Michael Paquier, with some adjustments by me Discussion: <CAB7nPqRu07Ot6iht9i9KRfYLpDaF2ZuUv5y_+72uP23ZAGysRg@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/052cc223d5ce1b727f62afff75797c88d82f880b
  • Prevent starting a standalone backend with standby_mode on. This can't really work because standby_mode expects there to be more WAL arriving, which there will not ever be because there's no WAL receiver process to fetch it. Moreover, if standby_mode is on then hot standby might also be turned on, causing even more strangeness because that expects read-only sessions to be executing in parallel. Bernd Helmle reported a case where btree_xlog_delete_get_latestRemovedXid got confused, but rather than band-aiding individual problems it seems best to prevent getting anywhere near this state in the first place. Back-patch to all supported branches. In passing, also fix some omissions of errcodes in other ereport's in readRecoveryCommandFile(). Michael Paquier (errcode hacking by me) Discussion: <00F0B2CEF6D0CEF8A90119D4@eje.credativ.lan> http://git.postgresql.org/pg/commitdiff/0e0f43d6fdc2e1fbd5261245ed4cf85302a3f653
  • Remove no-longer-useful SSL-specific Port.count field. Since we removed SSL renegotiation, there's no longer any reason to keep track of the amount of data transferred over the link. Daniel Gustafsson Discussion: <FEA7F89C-ECDF-4799-B789-2F8DDCBA467F@yesql.se> http://git.postgresql.org/pg/commitdiff/679226337ac3f44e784de0a95a8599dfd86401e8
  • Try to fix portability issue in enum renumbering (again). The hack embodied in commit 4ba61a487 no longer works after today's change to allow DatumGetFloat4/Float4GetDatum to be inlined (commit 14cca1bf8). Probably what's happening is that the faulty compilers are deciding that the now-inlined assignment is a no-op and so they're not required to round to float4 width. We had a bunch of similar issues earlier this year in the degree-based trig functions, and eventually settled on using volatile intermediate variables as the least ugly method of forcing recalcitrant compilers to do what the C standard says (cf commit 82311bcdd). Let's see if that method works here. Discussion: <4640.1472664476@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/65a588b4c3b06aefe0b00d6222598d4cff3542c2
  • Improve memory management for PL/Tcl functions. Formerly, the memory used to represent a PL/Tcl function was allocated with malloc() or in TopMemoryContext, and we'd leak it all if the function got redefined during the session. Instead, create a per-function context and keep everything in or under that context. Add a reference-counting mechanism (like the one plpgsql has long had) so that we can safely clean up an old function definition, either immediately if it's not being executed or at the end of the outermost execution. Currently, we only detect that a cached function is obsolete when we next attempt to call that function. So this covers the updated-definition case but leaves cruft around after DROP FUNCTION. It's not clear whether it's worth installing a syscache invalidation callback to watch for drops; none of the other PLs do, so for now we won't do it here either. Michael Paquier and Tom Lane Discussion: <CAB7nPqSOyAsHC6jL24J1B+oK3p=yyNoFU0Vs_B6fd2kdd5g5WQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/d062245b5bd591edf6f78bab8d6b8bb3ff69c7a6
  • Improve memory management for PL/Perl functions. Unlike PL/Tcl, PL/Perl at least made an attempt to clean up after itself when a function gets redefined. But it was still using TopMemoryContext for the fn_mcxt of argument/result I/O functions, resulting in the potential for memory leaks depending on what those functions did, and the retail alloc/free logic was pretty bulky as well. Fix things to use a per-function memory context like the other PLs now do. Tweak a couple of places where things were being done in a not-very-safe order (on the principle that a memory leak is better than leaving global state inconsistent after an error). Also make some minor cosmetic adjustments, mostly in field names, to make the code look similar to the way PL/Tcl does now wherever it's essentially the same logic. Michael Paquier and Tom Lane Discussion: <CAB7nPqSOyAsHC6jL24J1B+oK3p=yyNoFU0Vs_B6fd2kdd5g5WQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/6f7c0ea32f808a7dad3ec07db7e5fdf6514d2af0
  • Change API of ShmemAlloc() so it throws error rather than returning NULL. A majority of callers seem to have believed that this was the API spec already, because they omitted any check for a NULL result, and hence would crash on an out-of-shared-memory failure. The original proposal was to just add such error checks everywhere, but that does nothing to prevent similar omissions in future. Instead, let's make ShmemAlloc() throw the error (so we can remove the caller-side checks that do exist), and introduce a new function ShmemAllocNoError() that has the previous behavior of returning NULL, for the small number of callers that need that and are prepared to do the right thing. This also lets us remove the rather wishy-washy behavior of printing a WARNING for out-of-shmem, which never made much sense: either the caller has a strategy for dealing with that, or it doesn't. It's not ShmemAlloc's business to decide whether a warning is appropriate. The v10 release notes will need to call this out as a significant source-code change. It's likely that it will be a bug fix for extension callers too, but if not, they'll need to change to using ShmemAllocNoError(). This is nominally a bug fix, but the odds that it's fixing any live bug are actually rather small, because in general the requests being made by the unchecked callers were already accounted for in determining the overall shmem size, so really they ought not fail. Between that and the possible impact on extensions, no back-patch. Discussion: <24843.1472563085@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/6c03d981a6b64ed8caaed4e94b54ef926202c9f3
  • Don't require dynamic timezone abbreviations to match underlying time zone. Previously, we threw an error if a dynamic timezone abbreviation did not match any abbreviation recorded in the referenced IANA time zone entry. That seemed like a good consistency check at the time, but it turns out that a number of the abbreviations in the IANA database are things that Olson and crew made up out of whole cloth. Their current policy is to remove such names in favor of using simple numeric offsets. Perhaps unsurprisingly, a lot of these made-up abbreviations have varied in meaning over time, which meant that our commit b2cbced9e and later changes made them into dynamic abbreviations. So with newer IANA database versions that don't mention these abbreviations at all, we fail, as reported in bug #14307 from Neil Anderson. It's worse than just a few unused-in-the-wild abbreviations not working, because the pg_timezone_abbrevs view stops working altogether (since its underlying function tries to compute the whole view result in one call). We considered deleting these abbreviations from our abbreviations list, but the problem with that is that we can't stay ahead of possible future IANA changes. Instead, let's leave the abbreviations list alone, and treat any "orphaned" dynamic abbreviation as just meaning the referenced time zone. It will behave a bit differently than it used to, in that you can't any longer override the zone's standard vs. daylight rule by using the "wrong" abbreviation of a pair, but that's better than failing entirely. (Also, this solution can be interpreted as adding a small new feature, which is that any abbreviation a user wants can be defined as referencing a time zone name.) Back-patch to all supported branches, since this problem affects all of them when using tzdata 2016f or newer. Report: <20160902031551.15674.67337@wrigleys.postgresql.org> Discussion: <6189.1472820913@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/39b691f251167bbb3d49203abfb39d430f68f411
  • Fix corrupt GIN_SEGMENT_ADDITEMS WAL records on big-endian hardware. computeLeafRecompressWALData() tried to produce a uint16 WAL log field by memcpy'ing the first two bytes of an int-sized variable. That accidentally works on little-endian hardware, but not at all on big-endian. Replay then thinks it's looking at an ADDITEMS action with zero entries, and reads the first two bytes of the first TID therein as the next segno/action, typically leading to "unexpected GIN leaf action" errors during replay. Even if replay failed to crash, the resulting GIN index page would surely be incorrect. To fix, just declare the variable as uint16 instead. Per bug #14295 from Spencer Thomason (much thanks to Spencer for turning his problem into a self-contained test case). This likely also explains a previous report of the same symptom from Bernd Helmle. Back-patch to 9.4 where the problem was introduced (by commit 14d02f0bb). Discussion: <20160826072658.15676.7628@wrigleys.postgresql.org> Possible-Report: <2DA7350F7296B2A142272901@eje.land.credativ.lan> http://git.postgresql.org/pg/commitdiff/60893786d5180f5dd5aefd44d9cb6955d77b0473
  • Fix multiple bugs in numeric_poly_deserialize(). These were evidently introduced by yesterday's commit 9cca11c91, which perhaps needs more review than it got. Per report from Andreas Seltenreich and additional examination of nearby code. Report: <87oa45qfwq.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/600dc4c0da3b8c094ccc1ae75b47c8320898c714
  • Improve readability of the output of psql's \timing command. In addition to the existing decimal-milliseconds output value, display the same value in mm:ss.fff format if it exceeds one second. Tack on hours and even days fields if the interval is large enough. This avoids needing mental arithmetic to convert the values into customary time units. Corey Huinker, reviewed by Gerdan Santos; bikeshedding by many Discussion: <CADkLM=dbC4R8sbbuFXQVBFWoJGQkTEW8RWnC0PbW9nZsovZpJQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/6591f4226c81104f7746da6a5c00519919c560ae
  • Remove useless pg_strdup() operations. split_to_stringlist() doesn't modify its first argument nor expect it to remain valid after exit, so there's no need to duplicate the optarg string at the call sites. Per Coverity. (This has been wrong all along, but commit 052cc223d changed the useless calls from "strdup" to "pg_strdup", which apparently made Coverity think it's a new bug. It's not, but it's also not worth back-patching.) http://git.postgresql.org/pg/commitdiff/a2d75b67bccd24d17e328360080e4877d23bc369
  • Update release notes to mention need for ALTER EXTENSION UPDATE. Maybe we ought to make pg_upgrade do this for you, but it won't happen in 9.6, so call out the need for it as a migration consideration. http://git.postgresql.org/pg/commitdiff/5a072244919a92b2c757b2e3985191f02d674627
  • Remove vestigial references to "zic" in favor of "IANA database". Commit b2cbced9e instituted a policy of referring to the timezone database as the "IANA timezone database" in our user-facing documentation. Propagate that wording into a couple of places that were still using "zic" to refer to the database, which is definitely not right (zic is the compilation tool, not the data). Back-patch, not because this is very important in itself, but because we routinely cherry-pick updates to the tznames files and I don't want to risk future merge failures. http://git.postgresql.org/pg/commitdiff/da6ea70c32bed99ca040a5e04d83c0edd5dfc615
  • Add regression test coverage for non-default timezone abbreviation sets. After further reflection about the mess cleaned up in commit 39b691f25, I decided the main bit of test coverage that was still missing was to check that the non-default abbreviation-set files we supply are usable. Add that. Back-patch to supported branches, just because it seems like a good idea to keep this all in sync. http://git.postgresql.org/pg/commitdiff/c7f68bea22bf680a4eab4b8b1592b3c90bc634ac
  • Remove duplicate code from ReorderBufferCleanupTXN(). Andres is apparently the only hacker who thinks this code is better as-is. I (tgl) follow some of his logic, but the fact that it's setting off warnings from static code analyzers seems like a sufficient reason to put the complexity into a comment rather than the code. Aleksander Alekseev Discussion: <20160404190345.54d84ee8@fujitsu> http://git.postgresql.org/pg/commitdiff/b6182081be4a795d70b966be2542ad32d1ffbc20

Fujii Masao pushed:

  • Fix pg_xlogdump so that it handles cross-page XLP_FIRST_IS_CONTRECORD record. Previously pg_xlogdump failed to dump the contents of the WAL file if the file starts with the continuation WAL record which spans more than one pages. Since pg_xlogdump assumed that the continuation record always fits on a page, it could not find the valid WAL record to start reading from in that case. This patch changes pg_xlogdump so that it can handle a continuation WAL record which crosses a page boundary and find the valid record to start reading from. Back-patch to 9.3 where pg_xlogdump was introduced. Author: Pavan Deolasee Reviewed-By: Michael Paquier and Craig Ringer Discussion: CABOikdPsPByMiG6J01DKq6om2+BNkxHTPkOyqHM2a4oYwGKsqQ@mail.gmail.com http://git.postgresql.org/pg/commitdiff/bab7823a49bb210b8920ae59e5126d27e4d63833
  • Fix typos in comments. http://git.postgresql.org/pg/commitdiff/bd082231edbaf25626a023913394b611fe7928e8

Simon Riggs pushed:

Heikki Linnakangas pushed:

  • Remove support for OpenSSL versions older than 0.9.8. OpenSSL officially only supports 1.0.1 and newer. Some OS distributions still provide patches for 0.9.8, but anything older than that is not interesting anymore. Let's simplify things by removing compatibility code. Andreas Karlsson, with small changes by me. http://git.postgresql.org/pg/commitdiff/9b7cd59af1afcfbd786921d5cf73befb5fefa2f7
  • Use static inline functions for float <-> Datum conversions. Now that we are OK with using static inline functions, we can use them to avoid function call overhead of pass-by-val versions of Float4GetDatum, DatumGetFloat8, and Float8GetDatum. Those functions are only a few CPU instructions long, but they could not be written into macros previously, because we need a local union variable for the conversion. I kept the pass-by-ref versions as regular functions. They are very simple too, but they call palloc() anyway, so shaving a few instructions from the function call doesn't seem so important there. Discussion: <dbb82a4a-2c15-ba27-dd0a-009d2aa72b77@iki.fi> http://git.postgresql.org/pg/commitdiff/14cca1bf8e31ed39dbc26dd6c610f1113e759972
  • Support multiple iterators in the Red-Black Tree implementation. While we don't need multiple iterators at the moment, the interface is nicer and less dangerous this way. Aleksander Alekseev, with some changes by me. http://git.postgresql.org/pg/commitdiff/9f85784cae4d057f307b83b0d33edede33434f04
  • Speed up SUM calculation in numeric aggregates. This introduces a numeric sum accumulator, which performs better than repeatedly calling add_var(). The performance comes from using wider digits and delaying carry propagation, tallying positive and negative values separately, and avoiding a round of palloc/pfree on every value. This speeds up SUM(), as well as other standard aggregates like AVG() and STDDEV() that also calculate a sum internally. Reviewed-by: Andrey Borodin Discussion: <c0545351-a467-5b76-6d46-4840d1ea8aa4@iki.fi> http://git.postgresql.org/pg/commitdiff/9cca11c915e458323d0e746c68203f2c11da0302
  • Move code shared between libpq and backend from backend/libpq/ to common/. When building libpq, ip.c and md5.c were symlinked or copied from src/backend/libpq into src/interfaces/libpq, but now that we have a directory specifically for routines that are shared between the server and client binaries, src/common/, move them there. Some routines in ip.c were only used in the backend. Keep those in src/backend/libpq, but rename to ifaddr.c to avoid confusion with the file that's now in common. Fix the comment in src/common/Makefile to reflect how libpq actually links those files. There are two more files that libpq symlinks directly from src/backend: encnames.c and wchar.c. I don't feel compelled to move those right now, though. Patch by Michael Paquier, with some changes by me. Discussion: <69938195-9c76-8523-0af8-eb718ea5b36e@iki.fi> http://git.postgresql.org/pg/commitdiff/ec136d19b21791c845b1deeff43df137add0639e
  • Clarify the new Red-Black post-order traversal code a bit. Coverity complained about the for(;;) loop, because it never actually iterated. It was used just to be able to use "break" to exit it early. I agree with Coverity, that's a bit confusing, so refactor the code to use if-else instead. While we're at it, use a local variable to hold the "current" node. That's shorter and clearer than referring to "iter->last_visited" all the time. http://git.postgresql.org/pg/commitdiff/e21db14b8a6696a2b704b89df9c4be9cd0ea8a33

Ãlvaro Herrera pushed:

Robert Haas pushed:

Kevin Grittner pushed:

Correctifs en attente

Vik Fearing sent in a patch to fix some misbehaviors by using autovacuum better.

Haribabu Kommi sent in a patch to ensure that ecpg -? option works on Windows.

Craig Ringer sent in two revisions of a patch to report server_version_num alongside server_version in startup packet.

Andrew Borodin sent in three revisions of a patch to improve the performance of GiST penalty functions.

Vik Fearing sent in a patch to update sample configuration files to more modern settings.

KaiGai Kohei sent in a patch to add an optional callback to support a special optimization when a ForeignScan/CustomScan is located under the Limit node in plan tree.

Craig Ringer sent in six more revisions of a patch to add txid_status(bigint).

Michaël Paquier sent in two more revisions of a patch to rename pg_xlog and pg_clog to things that look less removable.

Tom Lane sent in a patch to fix some corner cases that cube_in rejects.

Michaël Paquier sent in another revision of a patch to fix a case that crashed pg_basebackup.

Andreas Karlsson sent in a patch to allow compiling with OpenSSL 1.1, remove OpenSSL 1.1 deprecation warnings, and remove px_get_pseudo_random_bytes.

Martín Marqués sent in another revision of a patch to fix the fact that extensions' tables are reloaded twice: once when restoring the extension, the other when the tables themselves get picked up.

Michaël Paquier sent in another revision of a patch to fix pg_receivelog.

Kyotaro HORIGUCHI sent in another revision of a patch to start on asynchronous and vectorized execution.

Kyotaro HORIGUCHI sent in a patch to fix some comments in GatherPath.single_copy.

Fujii Masao sent in three revisions of a patch to fix a GIN bug.

Maksim Milyutin sent in a patch to enable extraction of the state of a query execution from an external session via signals.

Peter Eisentraut sent in a patch to upgrade SERIALs to the new format.

Heikki Linnakangas sent in two revisions of a patch to reflect his conclusion that pinning a buffer in TupleTableSlot is unnecessary.

Jesper Pedersen sent in a patch to add hash index support in pageinspect.

Peter Eisentraut sent in another revision of a patch to add IDENTITY columns.

Mithun Cy sent in a patch to ensure that ecpg passes make.

Jeevan Chalke sent in a patch to push aggregates to a foreign server under some circumstances.

Peter Eisentraut sent in a patch to allow the use of ICU for sorting and other locale things.

Peter Eisentraut sent in a patch to add CREATE SEQUENCE AS <data type> clause.

Peter Eisentraut sent in a patch to add autonomous transactions.

Heikki Linnakangas sent in a patch to use static inline functions for Float <-> Datum conversions.

Ivan Kartyshov sent in a patch to make async slave wait for lsn to be replayed.

Peter Eisentraut sent in a PoC patch to support C++ builds of PostgreSQL.

Heikki Linnakangas and Andres Freund traded patches to optimize aggregates.

Pavel Stehule sent in a patch to add a \setfileref to psql.

Kevin Grittner sent in another revision of a patch to add trigger transition tables.

Craig Ringer sent in another revision of a patch to enable logical decoding to follow timelines.

Craig Ringer sent in another revision of a patch to add an optional --endpos LSN argument to pg_reclogical.

Simon Riggs and Abhijit Menon-Sen traded patches to change the recovery.conf API.

Pavan Deolasee sent in three revisions of a patch to add WARM, a technique to reduce write amplification when an indexed column of a table is updated.

Michael Banck sent in a patch to add -N (exclude schema(s)) to pg_restore.

Ivan Kartyshov sent in a patch to make pg_buffercache cheaper with large shmem.

Magnus Hagander sent in two revisions of a patch to pg_basebackup stream xlog to tar.

Marko Tiikkaja sent in another revision of a patch to add an INSERT ... SET syntax.

Petr Jelinek and Erik Rijkers traded patches to add logical replication.

Stephen Frost sent in a patch to add support for restrictive RLS policies.

Peter Eisentraut sent in a patch to add a pg_sequence catalog.

Simon Riggs sent in a patch to reduce lock levels with some comments as to how and why.

Kurt Roeckx sent in a patch to allow compiling with OpenSSL 1.1.

Amit Kapila sent in another revision of a patch to implement WAL-logged hash indexes.

Michaël Paquier sent in another revision of a patch to disallow database and role names with carriage return or linefeed in them.

Haribabu Kommi sent in a patch to add a new SQL counter statistics view, pg_stat_sql.

Michaël Paquier sent in two more revisions of a patch to fix some data durability issues in pg_basebackup and pg_receivexlog.

Rahila Syed sent in two revisions of a patch to cause psql to error when autocommit is turned on inside a transaction.

Tom Lane sent in another revision of a patch to add a \timing interval to psql.

Craig Ringer sent in three revisions of a patch to emit a HINT when COPY can't find a file.

Simon Riggs sent in a patch to fix docs' incorrect assumption that decoding slots cannot go backwards from SQL.

Etsuro Fujita sent in another revision of a patch to push down more full joins in the PostgreSQL FDW.

Andy Grundman sent in a patch to avoid use of __attribute__ when building with old Sun compiler versions.

Fabien COELHO sent in another revision of a patch to allow storing select results into variables in pgbench.

Claudio Freire and Simon Riggs traded patches to allow setting maintainance_work_mem or autovacuum_work_mem higher than 1GB (and be effective).

Craig Ringer sent in two revisions of a patch to dirty replication slots when accessed via sql interface.

Michaël Paquier sent in another revision of a patch to support SCRAM-SHA-256 for passwords, along with supporting syntax.

Tomas Vondra sent in another revision of a patch to allow index-only scans for expressions.

Vik Fearing sent in a patch to add --wait and --no-wait long options to match the shorter corresponding -w and -W in pg_ctl.

Tom Lane sent in another revision of a patch to enable transaction enum alters and renames.

Pavel Stehule sent in another revision of a patch to add xmltable().

Emre Hasegeli sent in another revision of a patch to fix some floating point comparison inconsistencies for the geometric types.

Jim Nasby sent in a patch to fix some formatting in the comments of execnodes.h.

Andreas Karlsson sent in two more revisions of a patch to fix compilation with OpenSSL 1.1.

Amit Kapila sent in another revision of a patch to use granular locking in CLOG.

Victor Wagner sent in another revision of a patch to implement failover on libpq connect level.

Craig Ringer sent in a patch to send catalog_xmin separately in hot standby feedback.

par N Bougain le mercredi 7 septembre 2016 à 19h11

dimanche 4 septembre 2016

Actualités PostgreSQL.fr

Sortie de PostgreSQL 9.6 RC 1

Le groupe de développement de PostgreSQL (PGDG) vient d'annoncer la sortie de la première version candidate de PostgreSQL 9.6. Cette version est théoriquement identique à la version finale qui sortira dans quelques semaines. Elle contient des corrections pour tous les problèmes identifiés lors de la phase de test.

Changements depuis la version Beta 4

PostgreSQL 9.6 RC 1 corrige des bugs découverts par les utilisateurs qui ont testé la Betat 4, notamment :

 * Ajout de fonctions SQL pour inspecter les méthodes d'accès d'index
 * Corrections de bugs sur les index bloom
 * Ajout d'un test de régression pour l'instertion dans les TOAST
 * Meilleure gestion des locales pour les erreurs sur les requêtes parallèles
 * Beaucoup de mises à jour dans la documentation

A partir de la version RC 1, le traitement parallèle des requêtes est désactivé par défaut dans le fichier postgresql.conf. Les utilisateurs qui souhaitent paralleliser leurs requêtes doivent augmenter le paramètre max_parallel_workers_per_gather.

Feuille de route

Cette version est la première candidate de PostgreSQL 9.6. D'autres versions candidates seront mises en ligne jusqu'à ce que tous les bugs remontés soient résolus et que la version 9.6.0 finale soit livrée. Pour plus d'information, rendez-vous sur la page dédiée aux tests sur les versions Beta.

Liens

par daamien le dimanche 4 septembre 2016 à 18h09

jeudi 1 septembre 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 28 août 2016

[ndt: MeetUp en cours d'organisation à Nantes.]

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en août

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160828231710.GA19651@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Noah Misch pushed:

Tom Lane pushed:

  • initdb now needs submake-libpq and submake-libpgfeutils. More fallout from commit a00c58314. Pointed out by Michael Paquier. http://git.postgresql.org/pg/commitdiff/234309fa87739f7a3ac99de815d181b50f2542e7
  • Refactor some network.c code to create cidr_set_masklen_internal(). Merge several copies of "copy an inet value and adjust the mask length" code to create a single, conveniently C-callable function. This function is exported for future use by inet SPGiST support, but it's good cleanup anyway since we had three slightly-different-for-no-good-reason copies. (Extracted from a larger patch, to separate new code from refactoring of old code) Emre Hasegeli http://git.postgresql.org/pg/commitdiff/7b405b3e04779fc0a026c9c6ac3e06194948b253
  • Improve SP-GiST opclass API to better support unlabeled nodes. Previously, the spgSplitTuple action could only create a new upper tuple containing a single labeled node. This made it useless for opclasses that prefer to work with fixed sets of nodes (labeled or otherwise), which meant that restrictive prefixes could not be used with such node definitions. Change the output field set for the choose() method to allow it to specify any valid node set for the new upper tuple, and to specify which of these nodes to place the modified lower tuple in. In addition to its primary use for fixed node sets, this feature could allow existing opclasses that use variable node sets to skip a separate spgAddNode action when splitting a tuple, by setting up the node needed for the incoming value as part of the spgSplitTuple action. However, care would have to be taken to add the extra node only when it would not make the tuple bigger than before. (spgAddNode can enlarge the tuple, spgSplitTuple can't.) This is a prerequisite for an upcoming SP-GiST inet opclass, but is being committed separately to increase the visibility of the API change. In passing, improve the documentation about the traverse-values feature that was added by commit ccd6eb49a. Emre Hasegeli, with cosmetic adjustments and documentation rework by me Discussion: <CAE2gYzxtth9qatW_OAqdOjykS0bxq7AYHLuyAQLPgT7H9ZU0Cw@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/d2ddee63b43b27d6c6af169342af10db19bd3a1a
  • Create an SP-GiST opclass for inet/cidr. This seems to offer significantly better search performance than the existing GiST opclass for inet/cidr, at least on data with a wide mix of network mask lengths. (That may suggest that the data splitting heuristics in the GiST opclass could be improved.) Emre Hasegeli, with mostly-cosmetic adjustments by me Discussion: <CAE2gYzxtth9qatW_OAqdOjykS0bxq7AYHLuyAQLPgT7H9ZU0Cw@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/77e2906821e2aec3c0807866a84c2934feeac8be
  • Fix network_spgist.c build failures from missing AF_INET definition. AF_INET is apparently defined in something that's pulled in automatically on Linux, but the buildfarm says that's not true everywhere. Comparing to network_gist.c suggests that including <sys/socket.h> ought to fix it, and the POSIX standard concurs. http://git.postgresql.org/pg/commitdiff/32909a57f9fb131eab8971a6d9845b55bbcb9091
  • Suppress compiler warnings in non-cassert builds. With Asserts off, these variables are set but never used, resulting in warnings from pickier compilers. Fix that with our standard solution. Per report from Jeff Janes. http://git.postgresql.org/pg/commitdiff/71e006f031310f77ab72881c47a7d8f41df748bb
  • Fix improper repetition of previous results from a hashed aggregate. ExecReScanAgg's check for whether it could re-use a previously calculated hashtable neglected the possibility that the Agg node might reference PARAM_EXEC Params that are not referenced by its input plan node. That's okay if the Params are in upper tlist or qual expressions; but if one appears in aggregate input expressions, then the hashtable contents need to be recomputed when the Param's value changes. To avoid unnecessary performance degradation in the case of a Param that isn't within an aggregate input, add logic to the planner to determine which Params are within aggregate inputs. This requires a new field in struct Agg, but fortunately we never write plans to disk, so this isn't an initdb-forcing change. Per report from Jeevan Chalke. This has been broken since forever, so back-patch to all supported branches. Andrew Gierth, with minor adjustments by me Report: <CAM2+6=VY8ykfLT5Q8vb9B6EbeBk-NGuLbT6seaQ+Fq4zXvrDcA@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/2c00fad2864350508f666da1a2c04e0cbe9cbf58
  • Fix small query-lifespan memory leak in bulk updates. When there is an identifiable REPLICA IDENTITY index on the target table, heap_update leaks the id_attrs bitmapset. That's not many bytes, but it adds up over enough rows, since the code typically runs in a query-lifespan context. Bug introduced in commit e55704d8b, which did a rather poor job of cloning the existing use-pattern for RelationGetIndexAttrBitmap(). Per bug #14293 from Zhou Digoal. Back-patch to 9.4 where the bug was introduced. Report: <20160824114320.15676.45171@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/ae4760d667c71924932ab32e14996b5be1831fc6
  • Fix instability in parallel regression tests. Commit f0c7b789a added a test case in case.sql that creates and then drops both an '=' operator and the type it's for. Given the right timing, that can cause a "cache lookup failed for type" failure in concurrent sessions, which see the '=' operator as a potential match for '=' in a query, but then the type is gone by the time they inquire into its properties. It might be nice to make that behavior more robust someday, but as a back-patchable solution, adjust the new test case so that the operator is never visible to other sessions. Like the previous commit, back-patch to all supported branches. Discussion: <5983.1471371667@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/2533ff0aa518d4d31391db279cf08e538fae5931
  • Fix logic for adding "parallel worker" context line to worker errors. The previous coding here was capable of adding a "parallel worker" context line to errors that were not, in fact, returned from a parallel worker. Instead of using an errcontext callback to add that annotation, just paste it onto the message by hand; this looks uglier but is more reliable. Discussion: <19757.1472151987@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/fbf28b6b52c269188262a87247adb2c359acd6c5
  • Fix assorted small bugs in ThrowErrorData(). Copy the palloc'd strings into the correct context, ie ErrorContext not wherever the source ErrorData is. This would be a large bug, except that it appears that all catchers of thrown errors do either EmitErrorReport or CopyErrorData before doing anything that would cause transient memory contexts to be cleaned up. Still, it's wrong and it will bite somebody someday. Fix failure to copy cursorpos and internalpos. Utter the appropriate incantations involving recursion_depth, so that we'll behave sanely if we get an error inside pstrdup. (In general, the body of this function ought to act like, eg, errdetail().) Per code reading induced by Jakob Egger's report. http://git.postgresql.org/pg/commitdiff/8529036b53298c0555670b4a81ed7349c44aeeb4
  • Put static forward declarations in elog.c back into same order as code. The guiding principle for the last few patches in this area apparently involved throwing darts. Cosmetic only, but back-patch to 9.6 because there is no reason for 9.6 and HEAD to diverge yet in this file. http://git.postgresql.org/pg/commitdiff/45a36e68539dcd7095a257b49f6f38ae77dec30d
  • Fix potential memory leakage from HandleParallelMessages(). HandleParallelMessages leaked memory into the caller's context. Since it's called from ProcessInterrupts, there is basically zero certainty as to what CurrentMemoryContext is, which means we could be leaking into long-lived contexts. Over the processing of many worker messages that would grow to be a problem. Things could be even worse than just a leak, if we happened to service the interrupt while ErrorContext is current: elog.c thinks it can reset that on its own whim, possibly yanking storage out from under HandleParallelMessages. Give HandleParallelMessages its own dedicated context instead, which we can reset during each call to ensure there's no accumulation of wasted memory. Discussion: <16610.1472222135@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/78dcd027e8f7ed213f69da932853dc4b7cb9cb44
  • Add a nonlocalized version of the severity field to client error messages. This has been requested a few times, but the use-case for it was never entirely clear. The reason for adding it now is that transmission of error reports from parallel workers fails when NLS is active, because pq_parse_errornotice() wrongly assumes that the existing severity field is nonlocalized. There are other ways we could have fixed that, but the other options were basically kluges, whereas this way provides something that's at least arguably a useful feature along with the bug fix. Per report from Jakob Egger. Back-patch into 9.6, because otherwise parallel query is essentially unusable in non-English locales. The problem exists in 9.5 as well, but we don't want to risk changing on-the-wire behavior in 9.5 (even though the possibility of new error fields is specifically called out in the protocol document). It may be sufficient to leave the issue unfixed in 9.5, given the very limited usefulness of pq_parse_errornotice in that version. Discussion: <A88E0006-13CB-49C6-95CC-1A77D717213C@eggerapps.at> http://git.postgresql.org/pg/commitdiff/26fa446da64716f12ab3a623434c644fcb344b2e
  • Add macros to make AllocSetContextCreate() calls simpler and safer. I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <21072.1472321324@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/ea268cdc9a2631da4a5748b00059a9fd43470d0e
  • Update 9.6 release notes through today. http://git.postgresql.org/pg/commitdiff/a6f0dc701b2f84839761783e87c49d43cd3e31df

Peter Eisentraut pushed:

Robert Haas pushed:

Bruce Momjian pushed:

Kevin Grittner pushed:

Heikki Linnakangas pushed:

  • Support OID system column in postgres_fdw. You can use ALTER FOREIGN TABLE SET WITH OIDS on a foreign table, but the oid column read out as zeros, because the postgres_fdw didn't know about it. Teach postgres_fdw how to fetch it. Etsuro Fujita, with an additional test case by me. Discussion: <56E90A76.5000503@lab.ntt.co.jp> http://git.postgresql.org/pg/commitdiff/ae025a15988f5491903cd3a2075f308c2773f711

Correctifs en attente

Michaël Paquier sent in another revision of a patch to avoid crashes in pg_basebackup.

Michaël Paquier sent in a patch to clean up parts of the make system.

Heikki Linnakangas sent in a patch to fix a race condition in GetOldestActiveTransactionId().

Tom Lane sent in a patch to arrange better locale-specific-character-class handling for regexps.

Michaël Paquier sent in a patch to disallow \r and \n characters in database and role names.

Craig Ringer sent in another revision of a patch to add pipelining (batch) support for libpq.

Gabriele Bartolini, Michaël Paquier, and Simon Riggs traded patches to fix an issue where pg_receivexlog does not report flush position with --synchronous.

Thomas Munro sent in a patch to fix an error in Linux's fallocate() that could a bus error.

Amit Kapila and Thomas Munro traded patches to add dsm_unpin_segment().

Craig Ringer sent in two more revisions of a patch to add txid_status(bigint) for transaction traceability.

Heikki Linnakangas sent in another revision of a patch to add CSN-based snapshots.

Ashutosh Sharma sent in another revision of a patch to add some tests for hash index coverage.

Amit Kapila sent in two more revisions of a patch to add WAL logging for hash indexes.

Aleksander Alekseev and Heikki Linnakangas traded patches to fix the RBTree iteration interface.

Kuntal Ghosh sent in two more revisions of a patch to add a WAL consistency check facility.

Michaël Paquier sent in another revision of a patch to track wait events for latches.

Fabien COELHO sent in a patch to fix pgbench stats when using \sleep.

Stephen Frost sent in another revision of a patch to remove all superuser() checks from pgstattuple.

Masahiko Sawada sent in a WIP patch to add block level parallel vacuum.

Michaël Paquier sent in two more revisions of a patch to add LSN as a recovery target.

Pavel Stehule sent in two more revisions of a patch to add xmltable().

Martín Marqués and Michaël Paquier traded patches to fix some issues where pg_dump would dump objects created by extensions separately from the extensions themselves.

Tom Lane sent in a patch to wire into the planner the assumption that VALUES() provided are distinct.

Dagfinn Ilmari Mannsåker sent in another revision of a patch to add ALTER TYPE RENAME VALUE.

Andrew Gierth sent in a patch to fix an issue with some LATERAL queries.

Masahiko Sawada sent in another revision of a patch to make lazy_scan_heap faster.

Artur Zakirov sent in two more revisions of a patch to fix a bug in to_timestamp().

Petr Jelinek sent in another revision of a patch to implement logical replication.

Takayuki Tsunakawa sent in a patch to fix an issue that could cause cascaded replicas to stick.

Mithun Cy sent in another revision of a patch to implement failover at the libpq connect level.

Masahiko Sawada sent in a patch to support 2PC with foreign servers and use that infrastructure to update the PostgreSQL FDW.

Amit Langote sent in another revision of a patch to add declarative partitioning.

Dilip Kumar sent in another revision of a patch to fix a cache lookup failure.

Brandur sent in a patch to add SortSupport for Postgres' macaddr.

Tom Lane sent in a patch to add a nonlocalized error severity diagnostic.

Andreas Karlsson sent in a patch to remove OpenSSL < 0.9.8 on the grounds that it's no longer supported even by OpenSSL, and supporting it breaks newer versions.

Andres Freund sent in another revision of a patch to implement targetlist SRFs using ROWS FROM().

Joe Conway sent in another revision of a patch to add some RLS-related docs.

par N Bougain le jeudi 1 septembre 2016 à 23h29

mardi 23 août 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 21 août 2016

Offres d'emplois autour de PostgreSQL en août

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. . La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160822003514.GA19487@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Tom Lane pushed:

  • Simplify the process of perltidy'ing our Perl files. Wrap the perltidy invocation into a shell script to reduce the risk of copy-and-paste errors. Include removal of *.bak files in the script, so they don't accidentally get committed. Improve the directions in the README file. http://git.postgresql.org/pg/commitdiff/05d8dec690e9719ff9a1830f5492864104275b5e
  • Final pgindent + perltidy run for 9.6. http://git.postgresql.org/pg/commitdiff/b5bce6c1ec6061c8a4f730d927e162db7e2ce365
  • Stamp HEAD as 10devel. This is a good bit more complicated than the average new-version stamping commit, because it includes various adjustments in pursuit of changing from three-part to two-part version numbers. It's likely some further work will be needed around that change; but this is enough to get through the regression tests, at least in Unix builds. Peter Eisentraut and Tom Lane http://git.postgresql.org/pg/commitdiff/ca9112a424ff68ec4f2ef67b47122f7d61412964
  • Stamp shared-library minor version numbers for v10. http://git.postgresql.org/pg/commitdiff/0b9358d4406b9b45a06855d53f491cc7ce9550a9
  • Update git_changelog to know that there's a 9.6 branch. Missed this in the main 10devel version stamping patch. http://git.postgresql.org/pg/commitdiff/3149a12166120d0b476f5ca7837ebcf0e7124703
  • Allow .so minor version numbers above 9 in .gitignore. Needed now that libpq.so's minor version has reached 10. http://git.postgresql.org/pg/commitdiff/2bf06f756142f4c398270cdc30bbba9b9dfbd38b
  • Doc: remove out-of-date claim that pg_am rows must be inserted by hand. Commit 473b93287 added a sentence about that, but neglected to remove the adjacent sentence it had falsified. Per Alexander Law. http://git.postgresql.org/pg/commitdiff/8fc571b7dd9fa1659536a26bb085584b50a65a51
  • Doc: copy-editing in create_access_method.sgml. Improve shaky English grammar. And markup. http://git.postgresql.org/pg/commitdiff/9b002cc9fec557fcfe17d67f55b53804447230e5
  • Remove separate version numbering for ecpg preprocessor. Once upon a time, it made sense for the ecpg preprocessor to have its own version number, because it used a manually-maintained grammar that wasn't always in sync with the core grammar. But those days are thankfully long gone, leaving only a maintenance nuisance behind. Let's use the PG v10 version numbering changeover as an excuse to get rid of the ecpg version number and just have ecpg identify itself by PG_VERSION. From the user's standpoint, ecpg will go from "4.12" in the 9.6 branch to "10" in the 10 branch, so there's no failure of monotonicity. Discussion: <1471332659.4410.67.camel@postgresql.org> http://git.postgresql.org/pg/commitdiff/a7b5573d665c8a37fad9bc69f44c5b4e8760a73b
  • Automate the maintenance of SO_MINOR_VERSION for our shared libraries. Up to now we've manually adjusted these numbers in several different Makefiles at the start of each development cycle. While that's not much work, it's easily forgotten, so let's get rid of it by setting the SO_MINOR_VERSION values directly from $(MAJORVERSION). In the case of libpq, this dev cycle's value of SO_MINOR_VERSION happens to be "10" anyway, so this switch is transparent. For ecpg's shared libraries, this will result in skipping one or two minor version numbers between v9.6 and v10, which seems like no big problem; and it was a bit inconsistent that they didn't have equal minor version numbers anyway. Discussion: <21969.1471287988@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/a3bce17ef1ca6408e8d1e7b10fb767aef1729be6
  • Fix assorted places in psql to print version numbers >= 10 in new style. This is somewhat cosmetic, since as long as you know what you are looking at, "10.0" is a serviceable substitute for "10". But there is a potential for confusion between version numbers with minor numbers and those without --- we don't want people asking "why is psql saying 10.0 when my server is 10.2". Therefore, back-patch as far as practical, which turns out to be 9.3. I could have redone the patch to use fprintf(stderr) in place of psql_error(), but it seems more work than is warranted for branches that will be EOL or nearly so by the time v10 comes out. Although only psql seems to contain any code that needs this, I chose to put the support function into fe_utils, since it seems likely we'll need it in other client programs in future. (In 9.3-9.5, use dumputils.c, the predecessor of fe_utils/string_utils.c.) In HEAD, also fix the backend code that whines about loadable-library version mismatch. I don't see much need to back-patch that. http://git.postgresql.org/pg/commitdiff/7f61fd10ceb715eceece49451f6dfe9058044e15
  • Suppress -Wunused-result warning for strtol(). I'm not sure which bozo thought it's a problem to use strtol() only for its endptr result, but silence the warning using same method used elsewhere. Report: <f845d3a6-5328-3e2a-924f-f8e91aa2b6d2@2ndquadrant.com> http://git.postgresql.org/pg/commitdiff/4bc4cfe3bd186b4a1d1b01279bfd0e6ab11268b2
  • Improve parsetree representation of special functions such as CURRENT_DATE. We implement a dozen or so parameterless functions that the SQL standard defines special syntax for. Up to now, that was done by converting them into more or less ad-hoc constructs such as "'now'::text::date". That's messy for multiple reasons: it exposes what should be implementation details to users, and performance is worse than it needs to be in several cases. To improve matters, invent a new expression node type SQLValueFunction that can represent any of these parameterless functions. Bump catversion because this changes stored parsetrees for rules. Discussion: <30058.1463091294@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/0bb51aa96783e8a6c473c2b5e3725e23e95db834
  • Improve plpgsql's memory management to fix some function-lifespan leaks. In some cases, exiting out of a plpgsql statement due to an error, then catching the error in a surrounding exception block, led to leakage of temporary data the statement was working with, because we kept all such data in the function-lifespan SPI Proc context. Iterating such behavior many times within one function call thus led to noticeable memory bloat. To fix, create an additional memory context meant to have statement lifespan. Since many plpgsql statements, particularly the simpler/more common ones, don't need this, create it only on demand. Reset this context at the end of any statement that uses it, and arrange for exception cleanup to reset it too, thereby fixing the memory-leak issue. Allow a stack of such contexts to exist to handle cases where a compound statement needs statement-lifespan data that persists across calls of inner statements. While at it, clean up code and improve comments referring to the existing short-term memory context, which by plpgsql convention is the per-tuple context of the eval_econtext ExprContext. We now uniformly refer to that as the eval_mcontext, whereas the new statement-lifespan memory contexts are called stmt_mcontext. This change adds some context-creation overhead, but on the other hand it allows removal of some retail pfree's in favor of context resets. On balance it seems to be about a wash performance-wise. In principle this is a bug fix, but it seems too invasive for a back-patch, and the infrequency of complaints weighs against taking the risk in the back branches. So we'll fix it only in HEAD, at least for now. Tom Lane, reviewed by Pavel Stehule Discussion: <17863.1469142152@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/bfaaacc805aef9f07b4805c6a274256340b9cddf
  • Fix -e option in contrib/intarray/bench/bench.pl. As implemented, -e ran an EXPLAIN but then discarded the output, which certainly seems pointless. Make it print to stdout instead. It's been like that forever, so back-patch to all supported branches. Daniel Gustafsson, reviewed by Andreas Scherbaum. Patch: <B97BDCB7-A3B3-4734-90B5-EDD586941629@yesql.se> http://git.postgresql.org/pg/commitdiff/6657acc0100ec79304b868a6346db4d8eb2f6b83
  • Implement regexp_match(), a simplified alternative to regexp_matches(). regexp_match() is like regexp_matches(), but it disallows the 'g' flag and in consequence does not need to return a set. Instead, it returns a simple text array value, or NULL if there's no match. Previously people usually got that behavior with a sub-select, but this way is considerably more efficient. Documentation adjusted so that regexp_match() is presented first and then regexp_matches() is introduced as a more complicated version. This is a bit historically revisionist but seems pedagogically better. Still TODO: extend contrib/citext to support this function. Emre Hasegeli, reviewed by David Johnston Discussion: <CAE2gYzy42sna2ME_e3y1KLQ-4UBrB-eVF0SWn8QG39sQSeVhEw@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/cf9b0fea5f6d1bcc9b2c66f5c30ecb04684a0919
  • Support the new regexp_match() function for citext. Emre Hasegeli Patch: <CAE2gYzzF24ZHWqkMukkHwqa0otbES9Rex22LrjQUNbi=oKziNQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/f9d747a4e908ad09520cad4c0e8c915074e9964c
  • Improve psql's tab completion for ALTER EXTENSION foo UPDATE ... Offer a list of available versions for that extension. Formerly, since there was no special support for this, it triggered off the UPDATE keyword and offered a list of table names --- not too helpful. Jeff Janes, reviewed by Gerdan Santos Patch: <CAMkU=1z0gxEOLg2BWa69P4X4Ot8xBxipGUiGkXe_tC+raj79-Q@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/49917dbd76ba0b4179a82fcf033ef5a10b8e3488
  • Improve psql's tab completion for \l. Offer a list of database names; formerly no help was offered. Ian Barwick, reviewed by Gerdan Santos Patch: <5724132E.1030804@2ndquadrant.com> http://git.postgresql.org/pg/commitdiff/8019b5a89c3cefbaa69ab58c00281419f7e46601
  • In plpgsql, don't try to convert int2vector or oidvector to expanded array. These types are storage-compatible with real arrays, but they don't support toasting, so of course they can't support expansion either. Per bug #14289 from Michael Overmeyer. Back-patch to 9.5 where expanded arrays were introduced. Report: <20160818174414.1529.37913@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/5697522d8411135d26a5d807f9e4afa182502f64
  • Update line count totals for psql help displays. As usual, we've been pretty awful about maintaining these counts. They're not all that critical, perhaps, but let's get them right at release time. Also fix 9.5, which I notice is just as bad. It's probably wrong further back, but the lack of --help=foo options before 9.5 makes it too painful to count. http://git.postgresql.org/pg/commitdiff/c5d4f40cb5e231eb2cbc533b5f094f3a4829e2ef
  • Remove typedef celt from the regex library, along with macro NOCELT. The regex library used to have a notion of a "collating element" that was distinct from a "character", but Henry Spencer never actually implemented his planned support for multi-character collating elements, and the Tcl crew ripped out most of the stubs for that years ago. The only thing left that distinguished the "celt" typedef from the "chr" typedef was that "celt" was supposed to also be able to hold the not-a-character "NOCELT" value. However, NOCELT was not used anywhere after the MCCE stub removal changes, which means there's no need for celt to be different from chr. Removing the separate typedef simplifies matters and also removes a trap for the unwary, in that celt is signed while chr may not be, so comparisons could mean different things. There's no bug there today because we restrict CHR_MAX to be less than INT_MAX, but I think there may have been such bugs before we did that, and there could be again if anyone ever decides to fool with the range of chr. This patch also removes assorted unnecessary casts to "chr" of values that are already chrs. Many of these seem to be leftover from days when the code was compatible with pre-ANSI C. http://git.postgresql.org/pg/commitdiff/6eefd2422ef232aec2fe12465d9ec4018c63814d
  • Clean up another pre-ANSI-C-ism in regex code: get rid of pcolor typedef. pcolor was used to represent function arguments that are nominally of type color, but when using a pre-ANSI C compiler would be passed as the promoted integer type. We really don't need that anymore. http://git.postgresql.org/pg/commitdiff/a859e640035680db31531ccd19a67292dd726baf
  • Speed up planner's scanning for parallel-query hazards. We need to scan the whole parse tree for parallel-unsafe functions. If there are none, we'll later need to determine whether particular subtrees contain any parallel-restricted functions. The previous coding retained no knowledge from the first scan, even though this is very wasteful in the common case where the query contains only parallel-safe functions. We can bypass all of the later scans by remembering that fact. This provides a small but measurable speed improvement when the case applies, and shouldn't cost anything when it doesn't. Patch by me, reviewed by Robert Haas Discussion: <3740.1471538387@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/da1c91631e3577ea5818f855ebb5bd206d559006
  • Guard against parallel-restricted functions in VALUES expressions. Obvious brain fade in set_rel_consider_parallel(). Noticed it while adjusting the adjacent RTE_FUNCTION case. In 9.6, also make the code look more like what I just did in HEAD by removing the unnecessary function_rte_parallel_ok subroutine (it does nothing that expression_tree_walker wouldn't do). http://git.postgresql.org/pg/commitdiff/65a603e90328a7a8fb3ab30ed96f24bf8eb4cf84
  • Use LEFT JOINs in some system views in case referenced row doesn't exist. In particular, left join to pg_authid so that rows in pg_stat_activity don't disappear if the session's owning user has been dropped. Also convert a few joins to pg_database to left joins, in the same spirit, though that case might be harder to hit. We were doing this in other views already, so it was a bit inconsistent that these views didn't. Oskari Saarenmaa, with some further tweaking by me Discussion: <56E87CD8.60007@ohmu.fi> http://git.postgresql.org/pg/commitdiff/8299471c37fff0b0f5a777a12f920125310c0efe
  • Allow empty queries in pgbench. This might have been too much of a foot-gun before 9.6, but with the new commands-end-at-semicolons parsing rule, the only way to get an empty query into a script is to explicitly write an extra ";". So we may as well allow the case. Fabien Coelho Patch: <alpine.DEB.2.20.1607090922170.3412@sto> http://git.postgresql.org/pg/commitdiff/6471045230f5d891ad724c54d406e2214f3c96d9
  • Make initdb's suggested "pg_ctl start" command line more reliable. The original coding here was not nearly careful enough about quoting special characters, and it didn't get corner cases right for constructing the pg_ctl path either. Use join_path_components() and appendShellString() to do it honestly, so that the string will more likely work if blindly copied-and-pasted. While at it, teach appendShellString() not to quote strings that clearly don't need it, so that the output from initdb doesn't become uglier than it was before in typical cases where quoting is not needed. Ryan Murphy, reviewed by Michael Paquier and myself Discussion: <CAHeEsBeAe1FeBypT3E8R1ZVZU0e8xv3A-7BHg6bEOi=jZny2Uw@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/a00c58314745772f6c6a49b6d02a9572cd600bda
  • initdb now needs to reference libpq include files in MSVC builds. Fallout from commit a00c58314. Per buildfarm. http://git.postgresql.org/pg/commitdiff/04164deb7cb8e572302e2b43786fa24de3c40da3

Robert Haas pushed:

  • Once again allow LWLocks to be used within DSM segments. Prior to commit 7882c3b0b95640e361f1533fe0f2d02e4e5d8610, it was possible to use LWLocks within DSM segments, but that commit broke this use case by switching from a doubly linked list to a circular linked list. Switch back, using a new bit of general infrastructure for maintaining lists of PGPROCs. Thomas Munro, reviewed by me. http://git.postgresql.org/pg/commitdiff/b25b6c9701e5c18e3ad3b701df62380f8d138ba0
  • Fix possible crash due to incorrect allocation context. Commit af33039317ddc4a0e38a02e2255c2bf453115fd2 aimed to reduce leakage from tqueue.c, which is good. Unfortunately, by changing the memory context in which all of gather_readnext() executes, it also changed the context in which ExecShutdownGatherWorkers executes, which is not good, because that function eventually causes a call to ExecParallelRetrieveInstrumentation, which proceeds to allocate planstate->worker_instrument in a short-lived context, causing a crash. Rushabh Lathia, reviewed by Amit Kapila and by me. http://git.postgresql.org/pg/commitdiff/41fb35fabf03bffa812caddf24323d4d06f811ba

Peter Eisentraut pushed:

  • Fix typos. From: Alexander Law <exclusion@gmail.com> http://git.postgresql.org/pg/commitdiff/f0fe1c8f70bacb65513f1cbaea14eb384d346ee8
  • doc: Remove some confusion from pg_archivecleanup doc. From: Jeff Janes <jeff.janes@gmail.com> http://git.postgresql.org/pg/commitdiff/639166641102871e09f9c4aebc71df57566a0a4a
  • Improve formatting of comments in plpgsql.h. This file had some unusual comment layout. Most of the comments introducing structs ended up to the right of the screen and following the start of the struct. Some comments for struct members ended up after the member definition. Fix that by moving comments consistently before what they are describing. Also add missing struct tags where missing so that it is easier to tell what the struct is. http://git.postgresql.org/pg/commitdiff/9f31e45a6d36a2564423a20601d5066939ea83c1
  • doc: Speed up XSLT builds. The upstream XSLT stylesheets use some very general XPath expressions in some places that end up being very slow. We can optimize them with knowledge about the DocBook document structure and our particular use thereof. For example, when counting preceding chapters to get a number for the current chapter, we only need to count preceding sibling nodes (more or less) instead of searching through the entire node tree for chapter elements. This change attacks the slowest pieces as identified by xsltproc --profile. This makes the HTML build roughly 10 times faster, resulting in the new total build time being about the same as the old DSSSL-based build. Some of the non-HTML build targets (especially FO) will also benefit a bit, but they have not been specifically analyzed. With this, also remove the pg.fast parameter, which was previously a hack to get the build to a manageable speed. Alexander Lakhin <a.lakhin@postgrespro.ru>, with some additional tweaking by me http://git.postgresql.org/pg/commitdiff/e8306745e3504c642f7abad411139d5630e29fac
  • Remove obsolete replacement system() on darwin. Per comment in the file, this was fixed around OS X 10.2. http://git.postgresql.org/pg/commitdiff/1d2e73a3dfdbd7168b323fa39879c60df6076412

Bruce Momjian pushed:

Magnus Hagander pushed:

Andres Freund pushed:

  • Properly re-initialize replication slot shared memory upon creation. Slot creation did not clear all fields upon creation. After start the memory is zeroed, but when a physical replication slot was created in the shared memory of a previously existing logical slot, catalog_xmin would not be cleared. That in turn would prevent vacuum from doing its duties. To fix initialize all the fields. To make similar future bugs less likely, zero all of ReplicationSlotPersistentData, and re-order the rest of the initialization to be in struct member order. Analysis: Andrew Gierth. Reported-By: md@chewy.com. Author: Michael Paquier. Discussion: <20160705173502.1398.70934@wrigleys.postgresql.org>. Backpatch: 9.4, where replication slots were introduced http://git.postgresql.org/pg/commitdiff/2d7e591007a6f44e5e27e2b6c1098483105c0d10
  • Fix deletion of speculatively inserted TOAST on conflict. INSERT .. ON CONFLICT runs a pre-check of the possible conflicting constraints before performing the actual speculative insertion. In case the inserted tuple included TOASTed columns the ON CONFLICT condition would be handled correctly in case the conflict was caught by the pre-check, but if two transactions entered the speculative insertion phase at the same time, one would have to re-try, and the code for aborting a speculative insertion did not handle deleting the speculatively inserted TOAST datums correctly. TOAST deletion would fail with "ERROR: attempted to delete invisible tuple" as we attempted to remove the TOAST tuples using simple_heap_delete which reasoned that the given tuples should not be visible to the command that wrote them. This commit updates the heap_abort_speculative() function which aborts the conflicting tuple to use itself, via toast_delete, for deleting associated TOAST datums. Like before, the inserted toast rows are not marked as being speculative. This commit also adds a isolationtester spec test, exercising the relevant code path. Unfortunately 9.5 cannot handle two waiting sessions, and thus cannot execute this test. Reported-By: Viren Negi, Oskari Saarenmaa. Author: Oskari Saarenmaa, edited a bit by me. Bug: #14150. Discussion: <20160519123338.12513.20271@wrigleys.postgresql.org>. Backpatch: 9.5, where ON CONFLICT was introduced http://git.postgresql.org/pg/commitdiff/07ef035129586ca26a713c4cd15e550dfe35e643
  • Add alternative output for ON CONFLICT toast isolation test. On some buildfarm animals the isolationtest added in 07ef0351 failed, as the order in which processes are run after unlocking is not guaranteed. Add an alternative output for that. Discussion: <7969.1471484738@sss.pgh.pa.us> Backpatch: 9.6, like the test in the aforementioned commit http://git.postgresql.org/pg/commitdiff/9595383bc6fc24d25970374e2eddd5ce6f977f9e

Heikki Linnakangas pushed:

  • Refactor sendAuthRequest. This way sendAuthRequest doesn't need to know the details of all the different authentication methods. This is in preparation for adding SCRAM authentication, which will add yet another authentication request message type, with different payload. Reviewed-By: Michael Paquier. Discussion: <CAB7nPqQvO4sxLFeS9D+NM3wpy08ieZdAj_6e117MQHZAfxBFsg@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/8d3b9cce81c173da55b9500353e5d773f8449a66
  • Refactor RandomSalt to handle salts of different lengths. All we need is 4 bytes at the moment, for MD5 authentication. But in upcomint patches for SCRAM authentication, SCRAM will need a salt of different length. It's less scary for the caller to pass the buffer length anyway, than assume a certain-sized output buffer. Author: Michael Paquier. Discussion: <CAB7nPqQvO4sxLFeS9D+NM3wpy08ieZdAj_6e117MQHZAfxBFsg@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/fa878703f456b804b01b61a9d94008f57967cdd0

Álvaro Herrera pushed:

  • reorderbuffer: preserve errno while reporting error. Clobbering errno during cleanup after an error is an oft-repeated, easy to make mistake. Deal with it here as everywhere else, by saving it aside and restoring after cleanup, before ereport'ing. In passing, add a missing errcode declaration in another ereport() call in the same file, which I noticed while skimming the file looking for similar problems. Backpatch to 9.4, where this code was introduced. http://git.postgresql.org/pg/commitdiff/6f79ae7fe549bed8bbd1f54ddd9b98f8f9a315f5

Correctifs en attente

Thomas Munro sent in another revision of a patch to add condition variables.

Haribabu Kommi sent in another revision of a patch to add a pg_hba_file_settings view.

Artur Zakirov sent in a patch to add a to_date_valid() function.

Peter Geoghegan sent in another revision of a patch to displace heap's root during tuplesort merge.

Ildar Musin sent in a patch to add index-only scans for expressions.

Anastasia Lubennikova sent in another revision of a patch to add covering + unique indexes.

Thomas Munro sent in two more revisions of a patch to add barriers.

David Steele sent in a patch to exclude additional directories in pg_basebackup.

Haribabu Kommi sent in a patch to add a function to control the number of parallel workers that can be generated for a parallel query, and a contrib extension to use same to consider system load while generating parallel workers.

Peter Eisentraut sent in another revision of a patch to set log_line_prefix and application name in test drivers.

Peter Eisentraut sent in another revision of a patch to add NEXT VALUE FOR.

Peter Eisentraut sent in a patch to run the select_parallel test by itself.

Jeff Janes sent in a patch to mention that hash indexes can be used in uniqueness constraints via EXCLUDE constraints.

Craig Ringer sent in a patch to add a function to get the 32-bit xid from a bigint extended xid-with-epoch.

Oskari Saarenmaa sent in two revisions of a patch to use pread and pwrite instead of lseek + write and read.

Artur Zakirov sent in another revision of a patch to add to_timestamp() format checking.

Claudio Freire sent in a patch to keep indexes sorted by heap physical location.

Andrew Borodin sent in another revision of a patch to optimize memmoves in gistplacetopage for fixed size updates.

Emre Hasegeli sent in a patch to add a regexp_match() returning text for citext.

Andres Freund sent in another revision of a patch to improve scalability of md.c for large relations and speed up PageIsVerified() for the all zeroes case.

Amit Langote and Robert Haas traded patches to fix the ALTER TABLE docs to mention that VALIDATE CONSTRAINT will fail if ONLY is specified and there are descendant tables.

Michaël Paquier sent in another revision of a patch to add checks for the case when malloc() returns NULL.

Michaël Paquier sent in a patch to move the md5 and ip implementations to src/common, where they're usable by more things.

Peter Eisentraut sent in a patch to allow forcing pg_basebackup to clean created directories on failure.

Bruce Momjian sent in two revisions of a patch to make pg_hba.conf case-insensitive.

Alexander Korotkov sent in a patch to cacheline align PGXACT.

Dagfinn Ilmari Mannsåker sent in a patch to add ALTER TYPE ... RENAME VALUE for enums.

Thomas Munro sent in a patch to add a new subsystem: Dynamic Shared Memory Areas.

Pavel Stehule sent in another revision of a patch to add xmltable().

Etsuro Fujita sent in a patch to push down more full joins in postgres_fdw.

Peter Eisentraut sent in a patch to make better use of existing enums in plpgsql.

Peter Geoghegan sent in a patch to fix a bug in abbreviated keys abort handling, that bug having been found with amcheck.

Adrien Nayrat and Michaël Paquier traded patches to add LSN as a recovery target.

Aleksander Alekseev sent in another revision of a patch to make PostgreSQL sanitizers-friendly and prevent information disclosure.

Haribabu Kommi sent in a POC patch based on ProcessUtilityHook to restrict the CREATE TRIGGER, CREATE FUNCTION, CREATE AGGREGATE, CREATE OPERATOR, CREATE VIEW and CREATE POLICY commands from normal users.

Craig Ringer sent in two revisions of a patch to implement a txid_status(bigint) function to report the commit status of a function.

Petr Jelinek sent in another revision of a patch to implement logical replication in core.

Thomas Munro sent in another revision of a patch to implement dsm_unpin_segment().

par N Bougain le mardi 23 août 2016 à 21h37

mercredi 17 août 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 14 août 2016

Les mises à jour de sécurité 9.5.4, 9.4.9, 9.3.14, 9.2.18 et 9.1.23 ont été publiées. Mettez à jour dès que possible ! https://www.postgresql.org/about/news/1688/

PostgreSQL 9.6 Beta 4 disponible. À vos tests ! https://www.postgresql.org/docs/devel/static/release-9-6.html

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en août

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160814214512.GB10448@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

  • Fix TOAST access failure in RETURNING queries. Discussion of commit 3e2f3c2e4 exposed a problem that is of longer standing: since we don't detoast data while sticking it into a portal's holdStore for PORTAL_ONE_RETURNING and PORTAL_UTIL_SELECT queries, and we release the query's snapshot as soon as we're done loading the holdStore, later readout of the holdStore can do TOAST fetches against data that can no longer be seen by any of the session's live snapshots. This means that a concurrent VACUUM could remove the TOAST data before we can fetch it. Commit 3e2f3c2e4 exposed the problem by showing that sometimes we had *no* live snapshots while fetching TOAST data, but we'd be at risk anyway. I believe this code was all right when written, because our management of a session's exposed xmin was such that the TOAST references were safe until end of transaction. But that's no longer true now that we can advance or clear our PGXACT.xmin intra-transaction. To fix, copy the query's snapshot during FillPortalStore() and save it in the Portal; release it only when the portal is dropped. This essentially implements a policy that we must hold a relevant snapshot whenever we access potentially-toasted data. We had already come to that conclusion in other places, cf commits 08e261cbc94ce9a7 and ec543db77b6b72f2. I'd have liked to add a regression test case for this, but I didn't see a way to make one that's not unreasonably bloated; it seems to require returning a toasted value to the client, and those will be big. In passing, improve PortalRunUtility() so that it positively verifies that its ending PopActiveSnapshot() call will pop the expected snapshot, removing a rather shaky assumption about which utility commands might do their own PopActiveSnapshot(). There's no known bug here, but now that we're actively referencing the snapshot it's almost free to make this code a bit more bulletproof. We might want to consider back-patching something like this into older branches, but it would be prudent to let it prove itself more in HEAD beforehand. Discussion: <87vazemeda.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/9ee1cf04ab6bcefe03a11837b53f29ca9dc24c7a
  • Fix crash when pg_get_viewdef_name_ext() is passed a non-view relation. Oversight in commit 976b24fb4. Andreas Seltenreich Report: <87y448l3ag.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/8a8c6b53810026641a1e12f60f873a7bd3cea5e3
  • Fix misestimation of n_distinct for a nearly-unique column with many nulls. If ANALYZE found no repeated non-null entries in its sample, it set the column's stadistinct value to -1.0, intending to indicate that the entries are all distinct. But what this value actually means is that the number of distinct values is 100% of the table's rowcount, and thus it was overestimating the number of distinct values by however many nulls there are. This could lead to very poor selectivity estimates, as for example in a recent report from Andreas Joseph Krogh. We should discount the stadistinct value by whatever we've estimated the nulls fraction to be. (That is what will happen if we choose to use a negative stadistinct for a column that does have repeated entries, so this code path was just inconsistent.) In addition to fixing the stadistinct entries stored by several different ANALYZE code paths, adjust the logic where get_variable_numdistinct() forces an "all distinct" estimate on the basis of finding a relevant unique index. Unique indexes don't reject nulls, so there's no reason to assume that the null fraction doesn't apply. Back-patch to all supported branches. Back-patching is a bit of a judgment call, but this problem seems to affect only a few users (else we'd have identified it long ago), and it's bad enough when it does happen that destabilizing plan choices in a worse direction seems unlikely. Patch by me, with documentation wording suggested by Dean Rasheed Report: <VisenaEmail.26.df42f82acae38a58.156463942b8@tc7-visena> Discussion: <16143.1470350371@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/95bee941be4c009ebbc29162a0dc9664f40de12f
  • Release notes for 9.5.4, 9.4.9, 9.3.14, 9.2.18, 9.1.23. http://git.postgresql.org/pg/commitdiff/19322c0a785f656b82a780c3db44920dcc9bd658
  • Update 9.6 release notes through today. http://git.postgresql.org/pg/commitdiff/be7f7ee5ea73626c037600b515087e8f98038140
  • Update 9.6 release notes through today. http://git.postgresql.org/pg/commitdiff/de4b3ea16d8b053e95eb9d862075b0de123e45b9
  • Stamp 9.6beta4. http://git.postgresql.org/pg/commitdiff/67c08c0d704a5f828492642bf9d133cbb2af3661
  • Doc: clarify description of CREATE/ALTER FUNCTION ... SET FROM CURRENT. Per discussion with David Johnston. http://git.postgresql.org/pg/commitdiff/e775d35317590793863327fedcf3737c3ab838d4
  • Doc: write some for adminpack. Previous contents of adminpack.sgml were rather far short of project norms. Not to mention being outright wrong about the signature of pg_file_read(). http://git.postgresql.org/pg/commitdiff/ff2fd6b06ac00a3c5d451063e0a87dca6156db4e
  • Fix busted Assert for CREATE MATVIEW ... WITH NO DATA. Commit 874fe3aea changed the command tag returned for CREATE MATVIEW/CREATE TABLE AS ... WITH NO DATA, but missed that there was code in spi.c that expected the command tag to always be "SELECT". Fortunately, the consequence was only an Assert failure, so this oversight should have no impact in production builds. Since this code path was evidently un-exercised, add a regression test. Per report from Shivam Saxena. Back-patch to 9.3, like the previous commit. Michael Paquier Report: <97218716-480B-4527-B5CD-D08D798A0C7B@dresources.com> http://git.postgresql.org/pg/commitdiff/0f249fe5f5cb3c83fd8e05743740b35ff5d34196
  • Trivial cosmetic cleanup in bloom/blutils.c. Don't spell "InvalidOid" as "0". Initialize method fields in the same order as amapi.h declares them (and every other AM handler initializes them). http://git.postgresql.org/pg/commitdiff/e3049285a3b8790e26e584fdf1ca31ea2e2eb4bc
  • Last-minute updates for release notes. Security: CVE-2016-5423, CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/9b8271c5a655ef1c35141b266d5039da8d3b2337
  • Fix two errors with nested CASE/WHEN constructs. ExecEvalCase() tried to save a cycle or two by passing &econtext->caseValue_isNull as the isNull argument to its sub-evaluation of the CASE value expression. If that subexpression itself contained a CASE, then *isNull was an alias for econtext->caseValue_isNull within the recursive call of ExecEvalCase(), leading to confusion about whether the inner call's caseValue was null or not. In the worst case this could lead to a core dump due to dereferencing a null pointer. Fix by not assigning to the global variable until control comes back from the subexpression. Also, avoid using the passed-in isNull pointer transiently for evaluation of WHEN expressions. (Either one of these changes would have been sufficient to fix the known misbehavior, but it's clear now that each of these choices was in itself dangerous coding practice and best avoided. There do not seem to be any similar hazards elsewhere in execQual.c.) Also, it was possible for inlining of a SQL function that implements the equality operator used for a CASE comparison to result in one CASE expression's CaseTestExpr node being inserted inside another CASE expression. This would certainly result in wrong answers since the improperly nested CaseTestExpr would be caused to return the inner CASE's comparison value not the outer's. If the CASE values were of different data types, a crash might result; moreover such situations could be abused to allow disclosure of portions of server memory. To fix, teach inline_function to check for "bare" CaseTestExpr nodes in the arguments of a function to be inlined, and avoid inlining if there are any. Heikki Linnakangas, Michael Paquier, Tom Lane Report: https://github.com/greenplum-db/gpdb/pull/327 Report: <4DDCEEB8.50602@enterprisedb.com> Security: CVE-2016-5423 http://git.postgresql.org/pg/commitdiff/f0c7b789ab12fbc8248b671c7882dd96ac932ef4
  • Fix inappropriate printing of never-measured times in EXPLAIN. EXPLAIN (ANALYZE, TIMING OFF) would print an elapsed time of zero for a trigger function, because no measurement has been taken but it printed the field anyway. This isn't what EXPLAIN does elsewhere, so suppress it. In the same vein, EXPLAIN (ANALYZE, BUFFERS) with non-text output format would print buffer I/O timing numbers even when no measurement has been taken because track_io_timing is off. That seems not per policy, either, so change it. Back-patch to 9.2 where these features were introduced. Maksim Milyutin Discussion: <081c0540-ecaa-bd29-3fd2-6358f3b359a9@postgrespro.ru> http://git.postgresql.org/pg/commitdiff/4b234fd8bf21cd6f5ff44f1f1c613bf40860998d
  • Doc: clarify that DROP ... CASCADE is recursive. Apparently that's not obvious to everybody, so let's belabor the point. In passing, document that DROP POLICY has CASCADE/RESTRICT options (which it does, per gram.y) but they do nothing (I assume, anyway). Also update some long-obsolete commentary in gram.y. Discussion: <20160805104837.1412.84915@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/499787819309293f3d2cd7219aee334a0e7d5069
  • Add SQL-accessible functions for inspecting index AM properties. Per discussion, we should provide such functions to replace the lost ability to discover AM properties by inspecting pg_am (cf commit 65c5fcd35). The added functionality is also meant to displace any code that was looking directly at pg_index.indoption, since we'd rather not believe that the bit meanings in that field are part of any client API contract. As future-proofing, define the SQL API to not assume that properties that are currently AM-wide or index-wide will remain so unless they logically must be; instead, expose them only when inquiring about a specific index or even specific index column. Also provide the ability for an index AM to override the behavior. In passing, document pg_am.amtype, overlooked in commit 473b93287. Andrew Gierth, with kibitzing by me and others Discussion: <87mvl5on7n.fsf@news-spur.riddles.org.uk> http://git.postgresql.org/pg/commitdiff/ed0097e4f9e6b1227935e01fa67f12a238b66064
  • Fix assorted bugs in contrib/bloom. In blinsert(), cope with the possibility that a page we pull from the notFullPage list is marked BLOOM_DELETED. This could happen if VACUUM recently marked it deleted but hasn't (yet) updated the metapage. We can re-use such a page safely, but we *must* reinitialize it so that it's no longer marked deleted. Fix blvacuum() so that it updates the notFullPage list even if it's going to update it to empty. The previous "optimization" of skipping the update seems pretty dubious, since it means that the next blinsert() will uselessly visit whatever pages we left in the list. Uniformly treat PageIsNew pages the same as deleted pages. This should allow proper recovery if a crash occurs just after relation extension. Properly use vacuum_delay_point, not assorted ad-hoc CHECK_FOR_INTERRUPTS calls, in the blvacuum() main loop. Fix broken tuple-counting logic: blvacuum.c counted the number of live index tuples over again in each scan, leading to VACUUM VERBOSE reporting some multiple of the actual number of surviving index tuples after any vacuum that removed any tuples (since they'd be counted in blvacuum, maybe more than once, and then again in blvacuumcleanup, without ever zeroing the counter). It's sufficient to count them in blvacuumcleanup. stats->estimated_count is a boolean, not a counter, and we don't want to set it true, so don't add tuple counts to it. Add a couple of Asserts that we don't overrun available space on a bloom page. I don't think there's any bug there today, but the way the FreeBlockNumberArray size calculation is set up is scarily fragile, and BloomPageGetFreeSpace isn't much better. The Asserts should help catch any future mistakes. Per investigation of a report from Jeff Janes. I think the first item above may explain his report; the other changes were things I noticed while casting about for an explanation. Report: <CAMkU=1xEUuBphDwDmB1WjN4+td4kpnEniFaTBxnk1xzHCw8_OQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/d6c9e05cb7db64239887fac65b243229594f331d
  • Remove bogus dependencies on NUMERIC_MAX_PRECISION. NUMERIC_MAX_PRECISION is a purely arbitrary constraint on the precision and scale you can write in a numeric typmod. It might once have had something to do with the allowed range of a typmod-less numeric value, but at least since 9.1 we've allowed, and documented that we allowed, any value that would physically fit in the numeric storage format; which is something over 100000 decimal digits, not 1000. Hence, get rid of numeric_in()'s use of NUMERIC_MAX_PRECISION as a limit on the allowed range of the exponent in scientific-format input. That was especially silly in view of the fact that you can enter larger numbers as long as you don't use 'e' to do it. Just constrain the value enough to avoid localized overflow, and let make_result be the final arbiter of what is too large. Likewise adjust ecpg's equivalent of this code. Also get rid of numeric_recv()'s use of NUMERIC_MAX_PRECISION to limit the number of base-NBASE digits it would accept. That created a dump/restore hazard for binary COPY without doing anything useful; the wire-format limit on number of digits (65535) is about as tight as we would want. In HEAD, also get rid of pg_size_bytes()'s unnecessary intimacy with what the numeric range limit is. That code doesn't exist in the back branches. Per gripe from Aravind Kumar. Back-patch to all supported branches, since they all contain the documentation claim about allowed range of NUMERIC (cf commit cabf5d84b). Discussion: <2895.1471195721@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/9389fbd0385776adf3252eb8cfe6e37a640fdff4

Peter Eisentraut pushed:

Bruce Momjian pushed:

Noah Misch pushed:

  • Promote pg_dumpall shell/connstr quoting functions to src/fe_utils. Rename these newly-extern functions with terms more typical of their new neighbors. No functional changes; a subsequent commit will use them in more places. Back-patch to 9.1 (all supported versions). Back branches lack src/fe_utils, so instead rename the functions in place; the subsequent commit will copy them into the other programs using them. Security: CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/41f18f021a0882eccbeca62e2ed4b66c6b96e9c9
  • Sort out paired double quotes in \connect, \password and \crosstabview. In arguments, these meta-commands wrongly treated each pair as closing the double quoted string. Make the behavior match the documentation. This is a compatibility break, but I more expect to find software with untested reliance on the documented behavior than software reliant on today's behavior. Back-patch to 9.1 (all supported versions). Reviewed by Tom Lane and Peter Eisentraut. Security: CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/984e5beb38a7c79a5a9243865d9598c405df17f6
  • Reject, in pg_dumpall, names containing CR or LF. These characters prematurely terminate Windows shell command processing, causing the shell to execute a prefix of the intended command. The chief alternative to rejecting these characters was to bypass the Windows shell with CreateProcess(), but the ability to use such names has little value. Back-patch to 9.1 (all supported versions). This change formally revokes support for these characters in database names and roles names. Don't document this; the error message is self-explanatory, and too few users would benefit. A future major release may forbid creation of databases and roles so named. For now, check only at known weak points in pg_dumpall. Future commits will, without notice, reject affected names from other frontend programs. Also extend the restriction to pg_dumpall --dbname=CONNSTR arguments and --file arguments. Unlike the effects on role name arguments and database names, this does not reflect a broad policy change. A migration to CreateProcess() could lift these two restrictions. Reviewed by Peter Eisentraut. Security: CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/142c24c23447f212e642a0ffac9af878b93f490d
  • Field conninfo strings throughout src/bin/scripts. These programs nominally accepted conninfo strings, but they would proceed to use the original dbname parameter as though it were an unadorned database name. This caused "reindexdb dbname=foo" to issue an SQL command that always failed, and other programs printed a conninfo string in error messages that purported to print a database name. Fix both problems by using PQdb() to retrieve actual database names. Continue to print the full conninfo string when reporting a connection failure. It is informative there, and if the database name is the sole problem, the server-side error message will include the name. Beyond those user-visible fixes, this allows a subsequent commit to synthesize and use conninfo strings without that implementation detail leaking into messages. As a side effect, the "vacuuming database" message now appears after, not before, the connection attempt. Back-patch to 9.1 (all supported versions). Reviewed by Michael Paquier and Peter Eisentraut. Security: CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/c400717172d77e5b07e51e04c5e5e13da181572e
  • Fix Windows shell argument quoting. The incorrect quoting may have permitted arbitrary command execution. At a minimum, it gave broader control over the command line to actors supposed to have control over a single argument. Back-patch to 9.1 (all supported versions). Security: CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/bd65371851b7a9964b4b265d06fe1304315e37c1
  • Obstruct shell, SQL, and conninfo injection via database and role names. Due to simplistic quoting and confusion of database names with conninfo strings, roles with the CREATEDB or CREATEROLE option could escalate to superuser privileges when a superuser next ran certain maintenance commands. The new coding rule for PQconnectdbParams() calls, documented at conninfo_array_parse(), is to pass expand_dbname=true and wrap literal database names in a trivial connection string. Escape zero-length values in appendConnStrVal(). Back-patch to 9.1 (all supported versions). Nathan Bossart, Michael Paquier, and Noah Misch. Reviewed by Peter Eisentraut. Reported by Nathan Bossart. Security: CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/fcd15f13581f6d75c63d213220d5a94889206c1b
  • Introduce a psql "\connect -reuse-previous=on|off" option. The decision to reuse values of parameters from a previous connection has been based on whether the new target is a conninfo string. Add this means of overriding that default. This feature arose as one component of a fix for security vulnerabilities in pg_dump, pg_dumpall, and pg_upgrade, so back-patch to 9.1 (all supported versions). In 9.3 and later, comment paragraphs that required update had already-incorrect claims about behavior when no connection is open; fix those problems. Security: CVE-2016-5424 http://git.postgresql.org/pg/commitdiff/9d924e9a64b91571e04252424c01210fc0f6f6d9

Álvaro Herrera pushed:

Simon Riggs pushed:

Correctifs en attente

Thomas Munro sent in a patch to implement dsm_unpin_segment.

Peter Eisentraut sent in a patch to set log_line_prefix and application name in test drivers for pg_regress.

David G. Johnston sent in a patch to make comprehension of the code in "TS_phrase_execute" a bit easier.

Amit Langote sent in another revision of the patch set to implement declarative table partitioning.

Robert Haas sent in a patch to run pgindent before the branch.

Tom Lane sent in a patch to allow GIN array_ops to work on anyarray.

Michaël Paquier sent in a patch to forbid use of LF and CR characters in database and role names.

Robert Haas sent in a patch to introduce condition variables. https://en.wikipedia.org/wiki/Monitor_(synchronization)#Condition_variables_2

Craig Ringer sent in a patch to give a more helpful hint as to when psql's \copy or similar trickery might be needed, as opposed to COPY.

Martín Marqués sent in a patch to add a test which will fail if it finds tables created by EXTENSIONs in pg_dump output.

Thomas Munro sent in a patch to create jumbo DSM segments, producing a SIGBUS if overallocated, which is not the right error.

Rushabh Lathia sent in a patch to fix a crash caused by a run of TPC-H.

Andreas Scherbaum sent in another revision of a patch to add to_date_valid().

Thomas Munro sent in a patch to add "barriers" to aid parallel computation.

Ryan Murphy sent in two revisions of a patch to make it possible for initdb to work with paths that contain spaces.

par N Bougain le mercredi 17 août 2016 à 22h03

samedi 13 août 2016

Adrien Nayrat

PostgreSQL : Retarder la vérification des contraintes

Retarder la vérification des contraintes

Remarque : Cet article a été rédigé durant le cadre de mon activité chez Dalibo.

Postgres respecte le modèle ACID, ainsi il garantie la cohérence de la base : une transaction amène la base d’un état stable à un autre.

Les données dans les différentes tables ne sont pas indépendantes mais obéissent à des règles sémantiques mises en place au moment de la conception du modèle conceptuel des données. Les contraintes d’intégrité ont pour principal objectif de garantir la cohérence des données entre elles, et donc de veiller à ce qu’elles respectent ces règles sémantiques. Si une insertion, une mise à jour ou une suppression viole ces règles, l’opération est purement et simplement annulée.

Le moteur effectue la vérification des contraintes à chaque modification (lorsque des contraintes ont été définies). Il est également possible de retarder la vérification des contraintes à la fin de la transaction, au moment du commit. Ainsi, les vérifications ne seront produites que sur les changements effectifs entre les opérations de delete, update et insert de la transaction.

Exemple :

CREATE TABLE t1 (c1 INT PRIMARY KEY, c2 text);
 CREATE TABLE t2 (c1 INT REFERENCES t1(c1), c2 text);

INSERT INTO t1  VALUES(1,'a');
 INSERT INTO t1  VALUES(2,'b');

INSERT INTO t2  VALUES(3,'a');
 ERROR:  INSERT OR UPDATE ON TABLE "t2" violates FOREIGN KEY CONSTRAINT "t2_c1_fkey"
 DETAIL:  KEY (c1)=(3) IS NOT present IN TABLE "t1".

La ligne insérée dans t2 doit respecter la contrainte d’intégrité référentielle, la table t1 ne contient aucune ligne où c1=3. Insérons une ligne correcte :

INSERT INTO t2  VALUES(1,'a');

SELECT * FROM t1;
 c1 | c2
 ----+----
 1 | a
 2 | b
 (2 ROWS)

SELECT * FROM t2;
 c1 | c2
 ----+----
 1 | a
 (1 ROW)

Que se passe t-il si on souhaite modifier la clé primaire de t1?

BEGIN;
 UPDATE t1 SET c1=3 WHERE c1=1;
 ERROR:  UPDATE OR DELETE ON TABLE "t1" violates FOREIGN KEY CONSTRAINT "t2_c1_fkey" ON TABLE "t2"
 DETAIL:  KEY (c1)=(1) IS still referenced FROM TABLE "t2".

La vérification de la contrainte se fait lors de l’UPDATE et déclenche une erreur. Il est possible de demander au moteur d’effectuer la vérification des contraintes à la fin de la transaction avec l’ordre SET CONSTRAINTS ALL DEFERRED;. A noter également que le mot clef ALL peut-être remplacé par le nom d’une contrainte (si elle est nommée et est DEFERRABLE)

BEGIN;
 SET CONSTRAINTS ALL DEFERRED;
 UPDATE t1 SET c1=3 WHERE c1=1;
 ERROR:  UPDATE OR DELETE ON TABLE "t1" violates FOREIGN KEY CONSTRAINT "t2_c1_fkey" ON TABLE "t2"
 DETAIL:  KEY (c1)=(1) IS still referenced FROM TABLE "t2".

Ça ne fonctionne toujours pas, en effet il faut préciser que que l’application de la contrainte peut être retardée avec le mot clé DEFERRABLE

ALTER TABLE t2 ALTER CONSTRAINT t2_c1_fkey DEFERRABLE;

BEGIN;
 SET CONSTRAINTS ALL DEFERRED;
 UPDATE t1 SET c1=3 WHERE c1=1;
 UPDATE t2 SET c1=3 WHERE c1=1;
 commit;

SELECT * FROM t1;
 c1 | c2
 ----+----
 2 | b
 3 | a
 (2 ROWS)

SELECT * FROM t2;
 c1 | c2
 ----+----
 3 | a
 (1 ROW)

Dans ce cas le moteur accepte de faire la vérification en fin de transaction.

Autre intérêt, si une ligne est effacée et réinsérée dans la même transaction, les vérifications sur cette ligne ne sont pas exécutées (car inutiles).

Exemple, on vide les tables puis on insère 1 million de lignes.

TRUNCATE t1 cascade;
 NOTICE:  TRUNCATE cascades TO TABLE "t2"
 TRUNCATE TABLE

EXPLAIN analyse INSERT INTO T1 (c1,c2) (SELECT  *,md5(i::text) FROM (SELECT * FROM generate_series(1,1000000)) i);

Ensuite on insère 100 000 lignes, puis on les supprime pour les réinsérer à nouveau (sans indiquer au moteur de différer la vérification de contraintes).

BEGIN;

EXPLAIN analyse  INSERT INTO T2 (c1,c2) (SELECT  *,md5(i::text) FROM (SELECT * FROM generate_series(1,1000000)) i);

QUERY PLAN
 ------------------------------------------------------------------------------------------------------------------------------------
 INSERT ON t2  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=3451.308..3451.308 ROWS=0 loops=1)
 -&gt;  FUNCTION Scan ON generate_series  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=170.218..1882.406 ROWS=1000000 loops=1)
 Planning TIME: 0.054 ms
 TRIGGER FOR CONSTRAINT t2_c1_fkey: TIME=16097.543 calls=1000000
 Execution TIME: 19654.595 ms
 (5 ROWS)

TIME: 19654.971 ms

DELETE FROM t2 WHERE c1 &lt;= 1000000;
 DELETE 1000000
 TIME: 2088.318 ms

EXPLAIN analyse  INSERT INTO T2 (c1,c2) (SELECT  *,md5(i::text) FROM (SELECT * FROM generate_series(1,1000000)) i);
 QUERY PLAN
 ------------------------------------------------------------------------------------------------------------------------------------
 INSERT ON t2  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=3859.265..3859.265 ROWS=0 loops=1)
 -&gt;  FUNCTION Scan ON generate_series  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=169.826..1845.421 ROWS=1000000 loops=1)
 Planning TIME: 0.054 ms
 TRIGGER FOR CONSTRAINT t2_c1_fkey: TIME=14600.258 calls=1000000
 Execution TIME: 18558.108 ms
 (5 ROWS)

TIME: 18558.386 ms
 commit;
 COMMIT
 TIME: 8.563 ms

Le moteur va effectuer les vérifications à chaque insertion (environ 18 secondes à chaque insertion).

Effectuons la même opération en retardant la vérification des contraintes :

BEGIN;
 BEGIN
 TIME: 0.130 ms

SET CONSTRAINTS ALL deferred ;
 SET CONSTRAINTS

EXPLAIN analyse  INSERT INTO T2 (c1,c2) (SELECT  *,md5(i::text) FROM (SELECT * FROM generate_series(1,1000000)) i);

QUERY PLAN
 ------------------------------------------------------------------------------------------------------------------------------------
 INSERT ON t2  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=3241.172..3241.172 ROWS=0 loops=1)
 -&gt;  FUNCTION Scan ON generate_series  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=169.770..1831.893 ROWS=1000000 loops=1)
 Planning TIME: 0.096 ms
 Execution TIME: 3269.624 ms
 (4 ROWS)

TIME: 3270.018 ms

DELETE FROM t2 WHERE c1 &lt;= 1000000;
 DELETE 2000000
 TIME: 2932.070 ms

EXPLAIN analyse  INSERT INTO T2 (c1,c2) (SELECT  *,md5(i::text) FROM (SELECT * FROM generate_series(1,1000000)) i);
 QUERY PLAN
 ------------------------------------------------------------------------------------------------------------------------------------
 INSERT ON t2  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=3181.294..3181.294 ROWS=0 loops=1)
 -&gt;  FUNCTION Scan ON generate_series  (cost=0.00..17.50 ROWS=1000 width=36) (actual TIME=170.137..1811.889 ROWS=1000000 loops=1)
 Planning TIME: 0.055 ms
 Execution TIME: 3209.712 ms
 (4 ROWS)

TIME: 3210.067 ms

commit;
 COMMIT
 TIME: 16630.155 ms

Les insertions sont plus rapides, en revanche le commit est plus long car le moteur effectue la vérification des contraintes. Au final, le moteur effectue une seule vérification à la fin de la transaction (commit). L’opération est donc plus rapide.

Il est possible de créer la contrainte avec un autre attribut DEFERRABLE INITIALLY DEFERRED qui permet de s’affranchir du SET CONSTRAINTS ALL DEFERRED. A noter également que le mot clef ALL peut-être remplacé par le nom d’une contrainte (si elle est nommée et est DEFERRABLE)

Si les enregistrements modifiés ne respectent pas les contraintes, la transaction est annulée au moment du commit :

SELECT * FROM t1;
 c1 | c2
 ----+----
 1 | un
 (1 ROW)

BEGIN;
 SET constraints ALL deferred ;
 SET CONSTRAINTS
 anayrat=# INSERT INTO t2 VALUES ('2','un');
 INSERT 0 1
 anayrat=# commit;
 ERROR:  INSERT OR UPDATE ON TABLE "t2" violates FOREIGN KEY CONSTRAINT "t2_c1_fkey"
 DETAIL:  KEY (c1)=(2) IS NOT present IN TABLE "t1".

D’autre part, la solution de désactivation des contraintes peut effectivement poser des problèmes si on souhaite les réactiver et que les données ne le permettent pas (contrainte effectivement rompue), cette solution reste possible, au sein d’une transaction, toutefois cela provoque un verrouillage exclusif sur la table modifiée pendant toute la transaction ce qui peut poser de sérieux problèmes de performance.

Il est également possible de déclarer la contrainte en NOT VALID. La création de la contrainte sera quasi immédiate, les données actuellement présentes ne seront pas validées. Cependant, toutes les données insérées ou mises à jour par la suite seront validées vis à vis de ces contraintes.

Ensuite on peut demander au moteur de faire la vérification des contraintes pour l’intégralité des enregistrements avec l’ordre VALIDATE CONSTRAINT. Cet ordre entraîne un verrou exclusif sur la table. A partir de la version 9.4 le verrou est plus léger : SHARE UPDATE EXCLUSIVE sur la table modifiée. Si la contrainte est une clé étrangère, le verrou est de type ROW SHARE sur la table référente.

ALTER TABLE t2 DROP CONSTRAINT t2_c1_fkey ;
 ALTER TABLE
 SELECT * FROM t1;
 c1 | c2
 ----+----
 1 | un
 (1 ROW)

SELECT * FROM t2;
 c1 | c2
 ----+----
 (0 ROWS)

INSERT INTO t2 VALUES (2,'deux');
 INSERT 0 1
 SELECT * FROM t2;
 c1 |  c2
 ----+------
 2 | deux
 (1 ROW)

ALTER TABLE t2 ADD CONSTRAINT t2_c1_fkey FOREIGN KEY (c1) REFERENCES t1(c1) NOT VALID;
 ALTER TABLE

La table t2 contient un enregistrement ne respectant pas la contrainte t2_c1_fkey. Aucune erreur n’est remontée car la vérification se fait seulement pour les nouvelles modifications :

INSERT INTO t2 VALUES (3,'trois');
 ERROR:  INSERT OR UPDATE ON TABLE "t2" violates FOREIGN KEY CONSTRAINT "t2_c1_fkey"
 DETAIL:  KEY (c1)=(3) IS NOT present IN TABLE "t1".

De même, une erreur est bien remontée lors de la vérification de contrainte :

ALTER TABLE t2 VALIDATE CONSTRAINT t2_c1_fkey ;
 ERROR:  INSERT OR UPDATE ON TABLE "t2" violates FOREIGN KEY CONSTRAINT "t2_c1_fkey"
 DETAIL:  KEY (c1)=(2) IS NOT present IN TABLE "t1".

En supprimant l’enregistrement posant problème, les contraintes peuvent être validées :

DELETE FROM t2 WHERE t2.c1 NOT IN (SELECT c1 FROM t1);
 DELETE 1
 ALTER TABLE t2 VALIDATE CONSTRAINT t2_c1_fkey ;
 ALTER TABLE

FacebookTwitterGoogle+ViadeoPrintEmailGoogle GmailLinkedInPartager

par Adrien Nayrat le samedi 13 août 2016 à 20h42

mardi 9 août 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 7 août 2016

La PGconf.ASIA 2016 aura lieu les 2 & 3 décembre à Tokyo. Envoyez vos propositions à pgconf-asia-2016-submission AT pgconf DOT asia (date limite des candidatures au 31 août à minuit) : http://www.pgconf.asia/EN/

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en août

PostgreSQL Local

  • Le PostgresOpen 2016 aura lieu à Dallas (Texas, USA) du 13 au 16 septembre. L'appel à conférenciers à été lancé : https://2016.postgresopen.org/callforpapers/
  • Session PostgreSQL le 22 septembre 2016 à Lyon (France). La date limite de candidature est le 20 mai : envoyez vos propositions à call-for-paper AT postgresql-sessions DOT org [ndt: http://www.postgresql-sessions.org/ ]
  • La PostgreSQL Conference Europe 2016 aura lieu à Tallin, Estonie, du 1er au 4 novembre 2016. Les inscriptions spéciales « lève-tôt » sont ouvertes jusqu'au 14 septembre : http://2016.pgconf.eu/registration/
  • La PgConf Silicon Valley 2016 aura lieu du 14 au 16 novembre 2016 : http://www.pgconfsv.com/
  • CHAR(16) aura lieu à New York le 6 décembre 2016. L'appel à conférenciers court jusqu'au 13 septembre, minuit (EDT) : http://charconference.org/

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160807222712.GC7429@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Fujii Masao pushed:

  • Fix pg_basebackup so that it accepts 0 as a valid compression level. The help message for pg_basebackup specifies that the numbers 0 through 9 are accepted as valid values of -Z option. But, previously -Z 0 was rejected as an invalid compression level. Per discussion, it's better to make pg_basebackup treat 0 as valid compression level meaning no compression, like pg_dump. Back-patch to all supported versions. Reported-By: Jeff Janes Reviewed-By: Amit Kapila Discussion: CAMkU=1x+GwjSayc57v6w87ij6iRGFWt=hVfM0B64b1_bPVKRqg@mail.gmail.com http://git.postgresql.org/pg/commitdiff/74d8c95b7456faefdd4244acf854361711fb42ce
  • Remove unused arguments from pg_replication_origin_xact_reset function. The document specifies that pg_replication_origin_xact_reset function doesn't have any argument variables. But previously it was actually defined so as to have two argument variables, though they were not used at all. That is, the pg_proc entry for that function was incorrect. This patch fixes the pg_proc entry and removes those two arguments from the function definition. No back-patch because this change needs a catalog version bump although the issue exists in 9.5 as well. Instead, a note about those unused argument variables will be added to 9.5 document later. Catalog version bumped due to the change of pg_proc. http://git.postgresql.org/pg/commitdiff/dd5eb805d5e5384963f09c9986845a544ef41810

Michael Meskes pushed:

Bruce Momjian pushed:

Tom Lane pushed:

  • Don't CHECK_FOR_INTERRUPTS between WaitLatch and ResetLatch. This coding pattern creates a race condition, because if an interesting interrupt happens after we've checked InterruptPending but before we reset our latch, the latch-setting done by the signal handler would get lost, and then we might block at WaitLatch in the next iteration without ever noticing the interrupt condition. You can put the CHECK_FOR_INTERRUPTS before WaitLatch or after ResetLatch, but not between them. Aside from fixing the bugs, add some explanatory comments to latch.h to perhaps forestall the next person from making the same mistake. In HEAD, also replace gather_readnext's direct call of HandleParallelMessages with CHECK_FOR_INTERRUPTS. It does not seem clean or useful for this one caller to bypass ProcessInterrupts and go straight to HandleParallelMessages; not least because that fails to consider the InterruptPending flag, resulting in useless work both here (if InterruptPending isn't set) and in the next CHECK_FOR_INTERRUPTS call (if it is). This thinko seems to have been introduced in the initial coding of storage/ipc/shm_mq.c (commit ec9037df2), and then blindly copied into all the subsequent parallel-query support logic. Back-patch relevant hunks to 9.4 to extirpate the error everywhere. Discussion: <1661.1469996911@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/887feefe87b9099eeeec2967ec31ce20df4dfa9b
  • Minor cleanup for access/transam/parallel.c. ParallelMessagePending *must* be marked volatile, because it's set by a signal handler. On the other hand, it's pointless for HandleParallelMessageInterrupt to save/restore errno; that must be, and is, done at the outer level of the SIGUSR1 signal handler. Calling CHECK_FOR_INTERRUPTS() inside HandleParallelMessages, which itself is called from CHECK_FOR_INTERRUPTS(), seems both useless and hazardous. The comment claiming that this is needed to handle the error queue going away is certainly misguided, in any case. Improve a couple of error message texts, and use ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE to report loss of parallel worker connection, since that's what's used in e.g. tqueue.c. (Maybe it would be worth inventing a dedicated ERRCODE for this type of failure? But I do not think ERRCODE_INTERNAL_ERROR is appropriate.) Minor stylistic cleanups. http://git.postgresql.org/pg/commitdiff/a5fe473ad79d8d2c85a625621c165e8c601274e4
  • Fix pg_dump's handling of public schema with both -c and -C options. Since -c plus -C requests dropping and recreating the target database as a whole, not dropping individual objects in it, we should assume that the public schema already exists and need not be created. The previous coding considered only the state of the -c option, so it would emit "CREATE SCHEMA public" anyway, leading to an unexpected error in restore. Back-patch to 9.2. Older versions did not accept -c with -C so the issue doesn't arise there. (The logic being patched here dates to 8.0, cf commit 2193121fa, so it's not really wrong that it didn't consider the case at the time.) Note that versions before 9.6 will still attempt to emit REVOKE/GRANT on the public schema; but that happens without -c/-C too, and doesn't seem to be the focus of this complaint. I considered extending this stanza to also skip the public schema's ACL, but that would be a misfeature, as it'd break cases where users intentionally changed that ACL. The real fix for this aspect is Stephen Frost's work to not dump built-in ACLs, and that's not going to get back-ported. Per bugs #13804 and #14271. Solution found by David Johnston and later rediscovered by me. Report: <20151207163520.2628.95990@wrigleys.postgresql.org> Report: <20160801021955.1430.47434@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/e2e95f5ef3c17197e319e4bbee70486f6a33e7d1
  • Block interrupts during HandleParallelMessages(). As noted by Alvaro, there are CHECK_FOR_INTERRUPTS() calls in the shm_mq.c functions called by HandleParallelMessages(). I believe they're all unreachable since we always pass nowait = true, but it doesn't seem like a great idea to assume that no such call will ever be reachable from HandleParallelMessages(). If that did happen, there would be a risk of a recursive call to HandleParallelMessages(), which it does not appear to be designed for --- for example, there's nothing that would prevent out-of-order processing of received messages. And certainly such cases cannot easily be tested. So let's prevent it by holding off interrupts for the duration of the function. Back-patch to 9.5 which contains identical code. Discussion: <14869.1470083848@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/b6a97b91ffe8e0c6b6557eb4aef85bcbd423ad5f
  • Remove duplicate InitPostmasterChild() call while starting a bgworker. This is apparently harmless on Windows, but on Unix it results in an assertion failure. We'd not noticed because this code doesn't get used on Unix unless you build with -DEXEC_BACKEND. Bug was evidently introduced by sloppy refactoring in commit 31c453165. Thomas Munro Discussion: <CAEepm=1VOnbVx4wsgQFvj94hu9jVt2nVabCr7QiooUSvPJXkgQ@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/c6ea616ff702862fc6923323a49dd24a0e0ae2d9
  • Do not let PostmasterContext survive into background workers. We don't want postmaster child processes to contain a copy of the postmaster's PostmasterContext. That would be a waste of memory at least, and at worst a security issue, since there are copies of the semi-sensitive pg_hba and pg_ident data in there. All other child process types delete the PostmasterContext after forking, but the original coding of the background worker patch (commit da07a1e85) did not do so. It appears that the only reason for that was to avoid copying the bgworker's MyBgworkerEntry out of that context; but the couple of additional statements needed to do so are hardly good justification for it. Hence, copy that data and then clear the context as other child processes do. Because this patch changes the memory context in which a bgworker function gains control, back-patching it would be a bit risky, so we won't fix this in back branches. The "security" complaint is pretty thin anyway for generic bgworkers; only with the introduction of parallel query is there any question of running untrusted code in a bgworker process. Discussion: <14111.1470082717@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/ef1b5af82339a49564037be656a3ff657fb2a246
  • Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better. Previously, if an INSERT with multiple rows of VALUES had indirection (array subscripting or field selection) in its target-columns list, the parser handled that by applying transformAssignedExpr() to each element of each VALUES row independently. This led to having ArrayRef assignment nodes or FieldStore nodes in each row of the VALUES RTE. That works for simple cases, but in bug #14265 Nuri Boardman points out that it fails if there are multiple assignments to elements/fields of the same target column. For such cases to work, rewriteTargetListIU() has to nest the ArrayRefs or FieldStores together to produce a single expression to be assigned to the column. But it failed to find them in the top-level targetlist and issued an error about "multiple assignments to same column". We could possibly fix this by teaching the rewriter to apply rewriteTargetListIU to each VALUES row separately, but that would be messy (it would change the output rowtype of the VALUES RTE, for example) and inefficient. Instead, let's fix the parser so that the VALUES RTE outputs are just the user-specified values, cast to the right type if necessary, and then the ArrayRefs or FieldStores are applied in the top-level targetlist to Vars representing the RTE's outputs. This is the same parsetree representation already used for similar cases with INSERT/SELECT syntax, so it allows simplifications in ruleutils.c, which no longer needs to treat INSERT-from-multiple-VALUES as its own special case. This implementation works by applying transformAssignedExpr to the VALUES entries as before, and then stripping off any ArrayRefs or FieldStores it adds. With lots of VALUES rows it would be noticeably more efficient to not add those nodes in the first place. But that's just an optimization not a bug fix, and there doesn't seem to be any good way to do it without significant refactoring. (A non-invasive answer would be to apply transformAssignedExpr + stripping to just the first VALUES row, and then just forcibly cast remaining rows to the same data types exposed in the first row. But this way would lead to different, not-INSERT-specific errors being reported in casting failure cases, so it doesn't seem very nice.) So leave that for later; this patch at least isn't making the per-row parsing work worse, and it does make the finished parsetree smaller, saving rewriter and planner work. Catversion bump because stored rules containing such INSERTs would need to change. Because of that, no back-patch, even though this is a very long-standing bug. Report: <20160727005725.7438.26021@wrigleys.postgresql.org> Discussion: <9578.1469645245@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/a3c7a993d5eb29df4d33075b83c75ae25f257897
  • Fix bogus coding in WaitForBackgroundWorkerShutdown(). Some conditions resulted in "return" directly out of a PG_TRY block, which left the exception stack dangling, and to add insult to injury failed to restore the state of set_latch_on_sigusr1. This is a bug only in 9.5; in HEAD it was accidentally fixed by commit db0f6cad4, which removed the surrounding PG_TRY block. However, I (tgl) chose to apply the patch to HEAD as well, because the old coding was gratuitously different from WaitForBackgroundWorkerStartup(), and there would indeed have been no bug if it were done like that to start with. Dmitry Ivanov Discussion: <1637882.WfYN5gPf1A@abook> http://git.postgresql.org/pg/commitdiff/8d498a5c8a4c702ca71463a5c76bb4f319872378
  • Update time zone data files to tzdata release 2016f. DST law changes in Kemerovo and Novosibirsk. Historical corrections for Azerbaijan, Belarus, and Morocco. Asia/Novokuznetsk and Asia/Novosibirsk now use numeric time zone abbreviations instead of invented ones. Zones for Antarctic bases and other locations that have been uninhabited for portions of the time span known to the tzdata database now report "-00" rather than "zzz" as the zone abbreviation for those time spans. Also, I decided to remove some of the timezone/data/ files that we don't use. At one time that subdirectory was a complete copy of what IANA distributes in the tzdata tarballs, but that hasn't been true for a long time. There seems no good reason to keep shipping those specific files but not others; they're just bloating our tarballs. http://git.postgresql.org/pg/commitdiff/a629330b2990c2e76cc8e45a78ede0920c16e0bf
  • Re-pgindent tsvector_op.c. Messed up by recent commits --- this is annoying me while trying to fix some bugs here. http://git.postgresql.org/pg/commitdiff/33fe7360afdc0bb1820743ea5bfe3fc7d522f6c4
  • Fix ts_delete(tsvector, text[]) to cope with duplicate array entries. Such cases either failed an Assert, or produced a corrupt tsvector in non-Assert builds, as reported by Andreas Seltenreich. The reason is that tsvector_delete_by_indices() just assumed that its input array had no duplicates. Fix by explicitly de-duping. In passing, improve some comments, and fix a number of tests for null values to use ERRCODE_NULL_VALUE_NOT_ALLOWED not ERRCODE_INVALID_PARAMETER_VALUE. Discussion: <87invhoj6e.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/c50d192ce33c10fa06411306f8644b4f47ce9a06
  • Make array_to_tsvector() sort and de-duplicate the given strings. This is required for the result to be a legal tsvector value. Noted while fooling with Andreas Seltenreich's ts_delete() crash. Discussion: <87invhoj6e.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/f10eab73df2b94c860dea4a906c54e3c903f42e2
  • Fix copy-and-pasteo in 81c766b3fd41c78c634d78ebae8d316808dfc630. Report: <57A4E6DF.8070209@dunslane.net> http://git.postgresql.org/pg/commitdiff/fc509cd82443a4cf338032492f6b1bd6e8698f8d
  • Teach libpq to decode server version correctly from future servers. Beginning with the next development cycle, PG servers will report two-part not three-part version numbers. Fix libpq so that it will compute the correct numeric representation of such server versions for reporting by PQserverVersion(). It's desirable to get this into the field and back-patched ASAP, so that older clients are more likely to understand the new server version numbering by the time any such servers are in the wild. (The results with an old client would probably not be catastrophic anyway for a released server; for example "10.1" would be interpreted as 100100 which would be wrong in detail but would not likely cause an old client to misbehave badly. But "10devel" or "10beta1" would result in sversion==0 which at best would result in disabling all use of modern features.) Extracted from a patch by Peter Eisentraut; comments added by me Patch: <802ec140-635d-ad86-5fdf-d3af0e260c22@2ndquadrant.com> http://git.postgresql.org/pg/commitdiff/69dc5ae408f68c302029a6b43912a2cc16b1256c
  • In B-tree page deletion, clean up properly after page deletion failure. In _bt_unlink_halfdead_page(), we might fail to find an immediate left sibling of the target page, perhaps because of corruption of the page sibling links. The code intends to cope with this by just abandoning the deletion attempt; but what actually happens is that it fails outright due to releasing the same buffer lock twice. (And error recovery masks a second problem, which is possible leakage of a pin on another page.) Seems to have been introduced by careless refactoring in commit efada2b8e. Since there are multiple cases to consider, let's make releasing the buffer lock in the failure case the responsibility of _bt_unlink_halfdead_page() not its caller. Also, avoid fetching the leaf page's left-link again after we've dropped lock on the page. This is probably harmless, but it's not exactly good coding practice. Per report from Kyotaro Horiguchi. Back-patch to 9.4 where the faulty code was introduced. Discussion: <20160803.173116.111915228.horiguchi.kyotaro@lab.ntt.co.jp> http://git.postgresql.org/pg/commitdiff/e89526d4f3567c58c2a69fa1b1d9e44df89349fb
  • First-draft release notes for 9.5.4. As usual, the release notes for other branches will be made by cutting these down, but put them up for community review first. http://git.postgresql.org/pg/commitdiff/3676631c687009c2fadcb35e7d312e9eb8a98182
  • Don't propagate a null subtransaction snapshot up to parent transaction. This oversight could cause logical decoding to fail to decode an outer transaction containing changes, if a subtransaction had an XID but no actual changes. Per bug #14279 from Marko Tiikkaja. Patch by Marko based on analysis by Andrew Gierth. Discussion: <20160804191757.1430.39011@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/bcbecbce2fde3c6dfa9080db11663877808a007d
  • Avoid crashing in GetOldestSnapshot() if there are no known snapshots. The sole caller expects NULL to be returned in such a case, so make it so and document it. Per reports from Andreas Seltenreich and Regina Obe. This doesn't really fix their problem, as now their RETURNING queries will say "ERROR: no known snapshots", but in any case this function should not dump core in a reasonably-foreseeable situation. Report: <87vazemeda.fsf@credativ.de> Report: <20160807051854.1427.32414@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/07a601eedab7c5fa4d62055fa3efacef2f38e446

Peter Eisentraut pushed:

Kevin Grittner pushed:

Álvaro Herrera pushed:

  • Fix assorted problems in recovery tests. In test 001_stream_rep we're using pg_stat_replication.write_location to determine catch-up status, but we care about xlog having been applied not just received, so change that to apply_location. In test 003_recovery_targets, we query the database for a recovery target specification and later for the xlog position supposedly corresponding to that recovery specification. If for whatever reason more WAL is written between the two queries, the recovery specification is earlier than the xlog position used by the query in the test harness, so we wait forever, leading to test failures. Deal with this by using a single query to extract both items. In 2a0f89cd717 we tried to deal with it by giving them more tests to run, but in hindsight that was obviously doomed to failure (no revert of that, though). Per hamster buildfarm failures. Author: Michaël Paquier http://git.postgresql.org/pg/commitdiff/b26f7fa6ae2b4e5d64525b3d5bc66a0ddccd9e24

Robert Haas pushed:

  • Prevent "snapshot too old" from trying to return pruned TOAST tuples. Previously, we tested for MVCC snapshots to see whether they were too old, but not TOAST snapshots, which can lead to complaints about missing TOAST chunks if those chunks are subject to early pruning. Ideally, the threshold lsn and timestamp for a TOAST snapshot would be that of the corresponding MVCC snapshot, but since we have no way of deciding which MVCC snapshot was used to fetch the TOAST pointer, use the oldest active or registered snapshot instead. Reported by Andres Freund, who also sketched out what the fix should look like. Patch by me, reviewed by Amit Kapila. http://git.postgresql.org/pg/commitdiff/3e2f3c2e423b3ae906668c186bac79522b8e3e29
  • Change InitToastSnapshot to a macro. tqual.h is included in some front-end compiles, and a static inline breaks on buildfarm member castoroides. Since the macro is never referenced, it should dodge that problem, although this doesn't seem like the cleanest way of hiding things from front-end compiles. Report and review by Tom Lane; patch by me. http://git.postgresql.org/pg/commitdiff/81c766b3fd41c78c634d78ebae8d316808dfc630

Andres Freund pushed:

  • Fix hard to hit race condition in heapam's tuple locking code. As mentioned in its commit message, eca0f1db left open a race condition, where a page could be marked all-visible, after the code checked PageIsAllVisible() to pin the VM, but before the page is locked. Plug that hole. Reviewed-By: Robert Haas, Andres Freund Author: Amit Kapila Discussion: CAEepm=3fWAbWryVW9swHyLTY4sXVf0xbLvXqOwUoDiNCx9mBjQ@mail.gmail.com Backpatch: - http://git.postgresql.org/pg/commitdiff/e7caacf733f3ee77a555aa715ab6fbf4434e6b52

Correctifs en attente

Etsuro Fujita sent in a patch to clarify the documentation around PlanForeignModify.

Karan Sikka sent in a patch to implement Boyer-Moore searching in LIKE queries.

Vladimir Sitnikov sent in a WIP patch to try CreateOneShotCachedPlan for unnamed statements in exec_parse_message.

Peter Geoghegan sent in a patch to implement parallel tuplesort.

Bruce Momjian sent in a patch to change usages of prefixes (k- to K, for example).

Tomas Vondra sent in a patch to add two slab-like memory allocators.

Masahiko Sawada sent in a patch to add a quorum commit for multiple synchronous replication.

Etsuro Fujita sent in two revisions of a patch to fix an oddity in EXPLAIN for foreign/custom join pushdown plans.

Tomas Vondra sent in another revision of a patch to add multivariate statistics.

Alexey Grishchenko sent in a patch to add upport for multi-dimensional arrays to PL/PythonU.

Andres Freund sent in a PoC patch to fix some messed-up-ness in SRF handling in target lists.

Masahiko Sawada sent in a patch to ensure that lazy_scan_heap accesses the visibility map only once.

Peter Eisentraut sent in a WIP patch to stamp version 10 for development.

Peter Eisentraut sent in another revision of a patch to add a -w (wait) pg_ctl promote option.

Ildar Musin sent in a patch to remove a now-unused argument in the PinBuffer function.

Mithun Cy sent in a patch to add coverage tests for hash indexes.

Michaël Paquier sent in two revisions of a patch to fix a crash in pg_basebackup uncovered by Andreas Seltenreich's sqlsmith.

Peter Eisentraut sent in a patch to improve DefElem list processing.

Aleksander Alekseev sent in another revision of a patch add a "fast" variant of temp tables which don't bloat the catalog.

Tom Lane sent in two revisions of a patch to fix an issue that resulted in bogus ANALYZE results for an otherwise-unique column with many nulls.

Amit Kapila sent in another revision of a patch to implement WAL-logged hash indexes.

Jeff Janes sent in a patch to add a new GUC, notice_lock_waits, which acts like log_lock_waits but sends the message as NOTICE so it will show up on interactive connections like psql.

Petr Jelinek sent in WIP a patch to add logical replication, and there was much rejoicing.

Peter Eisentraut sent in a patch to fix an overflow in the MONEY type.

Anastasia Lubennikova sent in a patch to refactor the heapam code.

Thomas Munro sent in a patch to consolidate the 'unique array values' logic into a reusable function.

Pavel Stehule sent in a PoC patch to add an xmltable function.

Andreas Seltenreich sent in a patch to fix a crash in pg_get_viewdef_name_ext().

par N Bougain le mardi 9 août 2016 à 22h23

lundi 1 août 2016

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 31 juillet 2016

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en juillet

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160801033117.GC11839@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Álvaro Herrera pushed:

Fujii Masao pushed:

Peter Eisentraut pushed:

Tom Lane pushed:

  • Fix constant-folding of ROW(...) IS [NOT] NULL with composite fields. The SQL standard appears to specify that IS [NOT] NULL's tests of field nullness are non-recursive, ie, we shouldn't consider that a composite field with value ROW(NULL,NULL) is null for this purpose. ExecEvalNullTest got this right, but eval_const_expressions did not, leading to weird inconsistencies depending on whether the expression was such that the planner could apply constant folding. Also, adjust the docs to mention that IS [NOT] DISTINCT FROM NULL can be used as a substitute test if a simple null check is wanted for a rowtype argument. That motivated reordering things so that IS [NOT] DISTINCT FROM is described before IS [NOT] NULL. In HEAD, I went a bit further and added a table showing all the comparison-related predicates. Per bug #14235. Back-patch to all supported branches, since it's certainly undesirable that constant-folding should change the semantics. Report and patch by Andrew Gierth; assorted wordsmithing and revised regression test cases by me. Report: <20160708024746.1410.57282@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/4452000f310b8c1c947ee724618c1bc31ed20242
  • Allow functions that return sets of tuples to return simple NULLs. ExecMakeTableFunctionResult(), which is used in SELECT FROM function(...) cases, formerly treated a simple NULL output from a function that both returnsSet and returnsTuple as a violation of the SRF protocol. What seems better is to treat a NULL output as equivalent to ROW(NULL,NULL,...). Without this, cases such as SELECT FROM unnest(...) on an array of composite are vulnerable to unexpected and not-very-helpful failures. Old code comments here suggested an alternative of just ignoring simple-NULL outputs, but that doesn't seem very principled. This change had been hung up for a long time due to uncertainty about how much we wanted to buy into the equivalence of simple NULL and ROW(NULL,NULL,...). I think that's been mostly resolved by the discussion around bug #14235, so let's go ahead and do it. Per bug #7808 from Joe Van Dyk. Although this is a pretty old report, fixing it smells a bit more like a new feature than a bug fix, and the lack of other similar complaints suggests that we shouldn't take much risk of destabilization by back-patching. (Maybe that could be revisited once this patch has withstood some field usage.) Andrew Gierth and Tom Lane Report: <E1TurJE-0006Es-TK@wrigleys.postgresql.org> http://git.postgresql.org/pg/commitdiff/d8411a6c8b6e5f74b362ef2496723f7f88593737
  • Fix cost_rescan() to account for multi-batch hashing correctly. cost_rescan assumed that we don't need to rebuild the hash table when rescanning a hash join. However, that's currently only true for single-batch joins; for a multi-batch join we must charge full freight. This probably has escaped notice because we'd be unlikely to put a hash join on the inside of a nestloop anyway. Nonetheless, it's wrong. Fix in HEAD, but don't backpatch for fear of destabilizing plans in stable releases. http://git.postgresql.org/pg/commitdiff/69995c3b3fd64361bb4d3938315f3e88ccc01e53
  • tqueue.c's record-typmod hashtables need the HASH_BLOBS option. The keys are integers, not strings. The code accidentally worked on little-endian machines, at least up to 256 distinct record types within a session, but failed utterly on big-endian. This was unexpectedly exposed by a test case added by commit 4452000f3, which apparently is the only parallelizable query in the regression suite that uses more than one anonymous record type. Fortunately, buildfarm member mandrill is big-endian and is running with force_parallel_mode on, so it failed. http://git.postgresql.org/pg/commitdiff/e1a93dd6ae114669669e3a77167dc3d3bd91e035
  • Register atexit hook only once in pg_upgrade. start_postmaster() registered stop_postmaster_atexit as an atexit(3) callback each time through, although the obvious intention was to do so only once per program run. The extra registrations were harmless, so long as we didn't exceed ATEXIT_MAX, but still it's a bug. Artur Zakirov, with bikeshedding by Kyotaro Horiguchi and me Discussion: <d279e817-02b5-caa6-215f-cfb05dce109a@postgrespro.ru> http://git.postgresql.org/pg/commitdiff/d9e74959a7fabe57e38bdda430aa662445bd1dd6
  • Improve documentation about CREATE TABLE ... LIKE. The docs failed to explain that LIKE INCLUDING INDEXES would not preserve the names of indexes and associated constraints. Also, it wasn't mentioned that EXCLUDE constraints would be copied by this option. The latter oversight seems enough of a documentation bug to justify back-patching. In passing, do some minor copy-editing in the same area, and add an entry for LIKE under "Compatibility", since it's not exactly a faithful implementation of the standard's feature. Discussion: <20160728151154.AABE64016B@smtp.hushmail.com> http://git.postgresql.org/pg/commitdiff/46b773d4fe0f0c880a1073cb5366efa02efa8ef8
  • Fix assorted fallout from IS [NOT] NULL patch. Commits 4452000f3 et al established semantics for NullTest.argisrow that are a bit different from its initial conception: rather than being merely a cache of whether we've determined the input to have composite type, the flag now has the further meaning that we should apply field-by-field testing as per the standard's definition of IS [NOT] NULL. If argisrow is false and yet the input has composite type, the construct instead has the semantics of IS [NOT] DISTINCT FROM NULL. Update the comments in primnodes.h to clarify this, and fix ruleutils.c and deparse.c to print such cases correctly. In the case of ruleutils.c, this merely results in cosmetic changes in EXPLAIN output, since the case can't currently arise in stored rules. However, it represents a live bug for deparse.c, which would formerly have sent a remote query that had semantics different from the local behavior. (From the user's standpoint, this means that testing a remote nested-composite column for null-ness could have had unexpected recursive behavior much like that fixed in 4452000f3.) In a related but somewhat independent fix, make plancat.c set argisrow to false in all NullTest expressions constructed to represent "attnotnull" constructs. Since attnotnull is actually enforced as a simple null-value check, this is a more accurate representation of the semantics; we were previously overpromising what it meant for composite columns, which might possibly lead to incorrect planner optimizations. (It seems that what the SQL spec expects a NOT NULL constraint to mean is an IS NOT NULL test, so arguably we are violating the spec and should fix attnotnull to do the other thing. If we ever do, this part should get reverted.) Back-patch, same as the previous commit. Discussion: <10682.1469566308@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/9492cf86e40288395a2ec6d81f7f5417e0e1b4fa
  • Teach parser to transform "x IS [NOT] DISTINCT FROM NULL" to a NullTest. Now that we've nailed down the principle that NullTest with !argisrow is fully equivalent to SQL's IS [NOT] DISTINCT FROM NULL, let's teach the parser about it. This produces a slightly more compact parse tree and is much more amenable to optimization than a DistinctExpr, since the planner knows a good deal about NullTest and next to nothing about DistinctExpr. I'm not sure that there are all that many queries in the wild that could be improved by this, but at least one source of such cases is the patch just made to postgres_fdw to emit IS [NOT] DISTINCT FROM NULL when IS [NOT] NULL isn't semantically correct. No back-patch, since to the extent that this does affect planning results, it might be considered undesirable plan destabilization. http://git.postgresql.org/pg/commitdiff/8d19d0e139238cdcb3f1f7e1adc4ff959562822f
  • Guard against empty buffer in gets_fromFile()'s check for a newline. Per the fgets() specification, it cannot return without reading some data unless it reports EOF or error. So the code here assumed that the data buffer would necessarily be nonempty when we go to check for a newline having been read. However, Agostino Sarubbo noticed that this could fail to be true if the first byte of the data is a NUL (\0). The fgets() API doesn't really work for embedded NULs, which is something I don't feel any great need for us to worry about since we generally don't allow NULs in SQL strings anyway. But we should not access off the end of our own buffer if the case occurs. Normally this would just be a harmless read, but if you were unlucky the byte before the buffer would contain '\n' and we'd overwrite it with '\0', and if you were really unlucky that might be valuable data and psql would crash. Agostino reported this to pgsql-security, but after discussion we concluded that it isn't worth treating as a security bug; if you can control the input to psql you can do far more interesting things than just maybe-crash it. Nonetheless, it is a bug, so back-patch to all supported versions. http://git.postgresql.org/pg/commitdiff/ed0b228d7a6b5186adc099f6a31dc33c499ff077
  • Fix pq_putmessage_noblock() to not block. An evident copy-and-pasteo in commit 2bd9e412f broke the non-blocking aspect of pq_putmessage_noblock(), causing it to behave identically to pq_putmessage(). That function is nowadays used only in walsender.c, so that the net effect was to cause walsenders to hang up waiting for the receiver in situations where they should not. Kyotaro Horiguchi Patch: <20160728.185228.58375982.horiguchi.kyotaro@lab.ntt.co.jp> http://git.postgresql.org/pg/commitdiff/80b346c2084928c11b6f9e495a7e9d559d96703d
  • Fix tqueue.c's range-remapping code. It's depressingly clear that nobody ever tested this. http://git.postgresql.org/pg/commitdiff/bf4ae685ae6f37b7fe83336abacf44298431b2f0
  • Fix worst memory leaks in tqueue.c. TupleQueueReaderNext() leaks like a sieve if it has to do any tuple disassembly/reconstruction. While we could try to clean up its allocations piecemeal, it seems like a better idea just to insist that it should be run in a short-lived memory context, so that any transient space goes away automatically. I chose to have nodeGather.c switch into its existing per-tuple context before the call, rather than inventing a separate context inside tqueue.c. This is sufficient to stop all leakage in the simple case I exhibited earlier today (see link below), but it does not deal with leaks induced in more complex cases by tqueue.c's insistence on using TopMemoryContext for data that it's not actually trying hard to keep track of. That issue is intertwined with another major source of inefficiency, namely failure to cache lookup results across calls, so it seems best to deal with it separately. In passing, improve some comments, and modify gather_readnext's method for deciding when it's visited all the readers so that it's more obviously correct. (I'm not actually convinced that the previous code *is* correct in the case of a reader deletion; it certainly seems fragile.) Discussion: <32763.1469821037@sss.pgh.pa.us> http://git.postgresql.org/pg/commitdiff/af33039317ddc4a0e38a02e2255c2bf453115fd2
  • Code review for tqueue.c: fix memory leaks, speed it up, other fixes. When doing record typmod remapping, tqueue.c did fresh catalog lookups for each tuple it processed, which was pretty horrible performance-wise (it seemed to about halve the already none-too-quick speed of bulk reads in parallel mode). Worse, it insisted on putting bits of that data into TopMemoryContext, from where it never freed them, causing a session-lifespan memory leak. (I suppose this was coded with the idea that the sender process would quit after finishing the query --- but the receiver uses the same code.) Restructure to avoid repetitive catalog lookups and to keep that data in a query-lifespan context, in or below the context where the TQueueDestReceiver or TupleQueueReader itself lives. Fix some other bugs such as continuing to use a tupledesc after releasing our refcount on it. Clean up cavalier datatype choices (typmods are int32, please, not int, and certainly not Oid). Improve comments and error message wording. http://git.postgresql.org/pg/commitdiff/a9ed875fdc2c44b5793a07727274786b417fc924
  • Doc: remove claim that hash index creation depends on effective_cache_size. This text was added by commit ff213239c, and not long thereafter obsoleted by commit 4adc2f72a (which made the test depend on NBuffers instead); but nobody noticed the need for an update. Commit 9563d5b5e adds some further dependency on maintenance_work_mem, but the existing verbiage seems to cover that with about as much precision as we really want here. Let's just take it all out rather than leaving ourselves open to more errors of omission in future. (That solution makes this change back-patchable, too.) Noted by Peter Geoghegan. Discussion: <CAM3SWZRVANbj9GA9j40fAwheQCZQtSwqTN1GBTVwRrRbmSf7cg@mail.gmail.com> http://git.postgresql.org/pg/commitdiff/11653cd87f66fc55ab79683a3ba7e6fe1a299596

Robert Haas pushed:

  • Repair damage done by citext--1.1--1.2.sql. That script is incorrect in that it sets the combine function for max(citext) twice instead of setting the combine function for max(citext) once and the combine functon for min(citext) once. The consequence is that if you install 1.0 or 1.1 and then update to 1.2, you end up with min(citext) not having a combine function, contrary to what was intended. If you install 1.2 directly, you're OK. Fix things up by defining a new 1.3 version. Upgrading from 1.2 to 1.3 won't change anything for people who first installed the 1.2 version, but people upgrading from 1.0 or 1.1 will get the right catalog contents once they reach 1.3. Report and patch by David Rowley, reviewed by Andreas Karlsson. http://git.postgresql.org/pg/commitdiff/fe5e3fce798dccf3f298b65c5d9a132e9646712a
  • Change various deparsing functions to return NULL for invalid input. Previously, some functions returned various fixed strings and others failed with a cache lookup error. Per discussion, standardize on returning NULL. Although user-exposed "cache lookup failed" error messages might normally qualify for bug-fix treatment, no back-patch; the risk of breaking user code which is accustomed to the current behavior seems too high. Michael Paquier http://git.postgresql.org/pg/commitdiff/976b24fb477464907737d28cdf18e202fa3b1a5b
  • Fix thinko in copyParamList. There's no point in consulting retval->paramMask; it's always NULL. Instead, we should consult from->paramMask. Reported by Andrew Gierth. http://git.postgresql.org/pg/commitdiff/b31875b1fe7131ac29f118efd81c9aba7255eced
  • Eliminate a few more user-visible "cache lookup failed" errors. Michael Paquier http://git.postgresql.org/pg/commitdiff/3153b1a52f8f2d1efe67306257aec15aaaf9e94c

Bruce Momjian pushed:

Stephen Frost pushed:

  • Correctly handle owned sequences with extensions. With the refactoring of pg_dump to handle components, getOwnedSeqs needs to be a bit more intelligent regarding which components to dump when. Specifically, we can't simply use the owning table's components as the set of components to dump as the table might only be including certain components while all components of the sequence should be dumped, for example, when the table is a member of an extension while the sequence is not. Handle this by combining the set of components to be dumped for the sequence explicitly and those to be dumped for the table when setting the components to be dumped for the sequence. Also add a number of regression tests around this to, hopefully, catch any future changes which break the expected behavior. Discovered by: Philippe BEAUDOIN Reviewed by: Michael Paquier http://git.postgresql.org/pg/commitdiff/f9e439b1ca81e3305b677d93c67299549625370c

Correctifs en attente

Thomas Munro sent in three more revisions of a patch to add LWLocks for DSM memory.

Michaël Paquier sent in another revision of a patch to add SCRAM-SHA-256 authentication over the SASL communication protocol.

Kyotaro HORIGUCHI sent in a patch to remove validation status condition from equalTupleDescs.

Amit Langote sent in a patch to make equalTupleDescs() parameterized on whether or not to performed an equality check on TupleConstr.

Kyotaro HORIGUCHI sent in a patch to split equalTupleDescs into two functions.

Tom Lane sent in a patch to create statement-level temporary memory contexts in plpgsql.

Heikki Linnakangas sent in a patch to optimize SUM().

Fujii Masao sent in a patch to add a 0 as a possible backup compression level for pg_basebackup.

Amit Langote sent in a patch to fix a comment on ATExecValidateConstraint.

Andrew Borodin sent in two more revisions of a patch to optimize GiST and BRIN memmoves.

Thomas Munro sent in a patch to clarify the meaning of NOT NULL constraints in light of implementation details.

Amit Kapila sent in a patch to fix some locking and pinning issues in the freeze map code.

John Harvey and Michaël Paquier traded patches to fix an issue with Perl on Windows.

Andrew Gierth sent in a patch to document the new index description functions which replace functionality taken out of the catalog.

Andres Freund sent in a PoC patch to add a new high-performance hashing function.

David Fetter sent in another revision of a patch to allow disallowing UPDATEs and DELETEs that lack a WHERE clause.

Etsuro Fujita sent in a patch to make the FDW infrastructure be more explicit about what foreign objects it is joining remotely.

Robert Haas sent in two revisions of a patch intended to fix an issue where old_snapshot_threshold allows heap:toast disagreement.

Tom Lane sent in a patch to fix an issue with target-column indirection in INSERT with multiple VALUES.

Aleksander Alekseev sent in two revisions of a patch to fix the RBtree iteration interface.

Fujii Masao sent in a patch to remove some unneeded arguments from the definition of pg_replication_origin_xact_reset().

Alexey Grishchenko sent in a patch to fix a slowness in PL/Python input array traversal.

Thomas Munro sent in a patch to remove a double invocation of InitPostmasterChild in bgworker with -DEXEC_BACKEND.

Michaël Paquier sent in two more revisions of a patch to fix wal level minimal.

Aleksander Alekseev sent in a patch to make a faster version of temporary tables by not making a catalog entry.

par N Bougain le lundi 1 août 2016 à 21h47

vendredi 20 mai 2016

Guillaume Lelarge

Quelques nouvelles sur les traductions du manuel

J'ai passé beaucoup de temps ces derniers temps sur la traduction du manuel de PostgreSQL.

  • mise à jour pour les dernières versions mineures
  • corrections permettant de générer les PDF déjà disponibles (9.1, 9.2, 9.3 et 9.4) mais aussi le PDF de la 9.5
  • merge pour la traduction de la 9.6

Elles sont toutes disponibles sur le site docs.postgresql.fr.

De ce fait, la traduction du manuel de la 9.6 peut commencer. Pour les intéressés, c'est par là : https://github.com/gleu/pgdocs_fr/wiki/Translation-9.6

N'hésitez pas à m'envoyer toute question si vous êtes intéressé pour participer.

par Guillaume Lelarge le vendredi 20 mai 2016 à 20h35

jeudi 21 avril 2016

Adrien Nayrat

Index BRIN – Performances

La version 9.5 de PostgreSQL sortie en Janvier 2016 propose un nouveau type d’index : les Index BRIN pour Bloc Range INdex. Ces derniers sont recommandés pour les tables volumineuses et corrélées avec leur emplacement. J’ai décidé de consacrer une série d’article sur ces index :

Pour information, je serai présent au PGDay France à Lille le mardi 31 mai pour présenter cet index. Il y aura également plein d’autres conférences intéressantes!

Cet article est la dernier de la série, il sera consacré aux performances (maintenance, lecture, insertion…)

Performances

Les articles précédents ont abordé le fonctionnement et les spécificités des index BRIN. Cet article sera plutôt consacré aux performances. Les exemples précédents portaient sur de petites volumétries. Maintenant nous allons voir ce que peuvent apporter ces index sur une volumétrie plus importante.

Les tests ont été effectués sur un PC portable, les journaux de transaction, la table et les index sont stockés sur un disque mécanique. Les résultats seront différents suivant la matériel utilisé. Ces chiffres sont purement indicatifs et servent surtout à se donner un ordre d’idée.

Exemple

Dans un premier temps il est nécessaire de créer une table avec une volumétrie importante.

Par exemple : un système de mesure avec 100 sondes et une mesure toutes les secondes. Nous allons donc obtenir 100*365*24*3600 mesures => un peu plus de 3 milliards de lignes.

-- Utilisation d'une fonction pour générer du texte aléatoire.
-- Trouvée ici : http://stackoverflow.com/questions/3970795/how-do-you-create-a-random-string-in-postgresql

CREATE OR REPLACE FUNCTION random_string(LENGTH INTEGER) RETURNS text AS 
$$
DECLARE
  chars text[] := '{0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}';
  RESULT text := '';
  i INTEGER := 0;
BEGIN
  IF LENGTH &lt; 0 THEN
    raise exception 'Given length cannot be less than 0';
  END IF;
  FOR i IN 1..LENGTH loop
    RESULT := RESULT || chars[1+random()*(array_length(chars, 1)-1)];
  END loop;
  RETURN RESULT;
END;
$$ LANGUAGE plpgsql;
-- Creation de la table contenant les sondes 
CREATE TABLE probe (id serial PRIMARY KEY, name text);
INSERT INTO probe (name ) SELECT random_string(5) FROM generate_series(1,100);

CREATE TABLE data AS
WITH generation AS (
SELECT '2015-01-01'::TIMESTAMP + i * INTERVAL '1 second' AS date_metric,sonde::text,random() AS metric
FROM generate_series(0, 3600*24*365) i,
LATERAL (SELECT name FROM probe) sonde)
SELECT * FROM generation;

La table obtenue fait un peu plus de 150Go, elle ne rentre donc pas dans  la RAM de la machine hébergeant l’instance et encore moins dans la mémoire partagée de PostgreSQL.

Maintenance

On crée plein d’index pour comparer leur taille :

CREATE INDEX metro_btree_idx ON data USING btree (date_metric);
CREATE INDEX metro_brin_idx_8 ON data USING brin (date_metric) WITH (pages_per_range = 8);
CREATE INDEX metro_brin_idx_16 ON data USING brin (date_metric) WITH (pages_per_range = 16);
CREATE INDEX metro_brin_idx_32 ON data USING brin (date_metric) WITH (pages_per_range = 32);
CREATE INDEX metro_brin_idx_64 ON data USING brin (date_metric) WITH (pages_per_range = 64);
CREATE INDEX metro_brin_idx_128 ON data USING brin (date_metric);
CREATE INDEX metro_brin_idx_256 ON data USING brin (date_metric) WITH (pages_per_range = 256);

Voici les résultats obtenus pour la durée de création des index et leurs tailles :

size-largesize-large

La création de l’index a été 4 fois plus rapide pour les index BRIN. Il est possible que leur création aurait été plus rapide avec un stockage plus performant.

La taille des index est également frappante,  l’index b-tree fait 66 Go alors que l’index BRIN avec le pages_per_range par défaut fait seulement 5 Mo.

On peut tout suite constater le gain sur l’espace utilisé et la rapidité de création des index. Les opérations de maintenances (REINDEX) seront grandement facilitées.

Performances en lecture

Nous allons effectuer plusieurs tests, l’idée est d’essayer de mettre en valeur les différences de comportements entre les index BRIN et b-tree.

La requête utilisée sera tout simple :

EXPLAIN (ANALYSE,BUFFERS,VERBOSE) SELECT date_metric,sonde,metric FROM DATA WHERE date_metric = 'xxx';

pour obtenir un résultat avec peu de lignes  :

WHERE date_metric = '2015-05-01 00:00:00'::TIMESTAMP

Pour obtenir plus de résultat on prendra un intervalle avec :

WHERE date_metric BETWEEN 'xxx'::TIMESTAMP AND 'xxx'::TIMESTAMP;

Voici les résultats obtenus :

lignes BRIN Btree Gain
Durée Blocs lus Durée Blocs lus Durée Volume données lues
100 24ms 697 0.06ms 7 Btree (x400) Btree (x100)
267 millions 170s 13Go 228s 18Go BRIN (x1.3) BRIN (x1.4)
777 millions 8min 38Go 11min 54Go BRIN (x1.37) BRIN (x1.4)
1.3 milliard 13min 63Go 32min (seqscan)
18min
153 Go (seqscan)
90 Go
BRIN (x2) vs seqscan
BRIN (1.4x) vs Btree
BRIN (x2.4) vs seqscan
BRIN (1.4x) vs Btree

Pour comparer le volume de données lues et la durée d’exécution nous pouvons désactiver les index dans une transaction :

BEGIN;
DROP index ...;
explain (analyse,verbose,buffers) SELECT ...
rollback;

Pour le 1er test, le moteur choisit l’index-btree. En supprimant l’index b-tree il choisit l’index BRIN.

Pour les tests 2 et 3, le moteur choisit l’index BRIN, en supprimant l’index BRIN il choisit l’index b-tree.

Pour le dernier test j’ai rajouté d’autres mesures. En effet, en supprimant l’index BRIN le moteur va effectuer un seqscan (parcours de toute la table). Pour obtenir les mêmes comparaisons que les résultats précédents j’ai donc supprimé l’index BRIN et désactivé les parcours séquentiels (set enable_seqscan to ‘off’;)

Globalement on peut constater un gain de 30-40% dans les cas où beaucoup de résultats sont demandés. Le moteur lit moins de blocs lorsqu’il utilise les index BRIN, l’index b-tree étant volumineux, ses lectures sont coûteuses.

En revanche l’index b-tree s’avère particulièrement performant lorsque la requête est très sélective et que peu de résultats sont retournés. En effet, en utilisant un index BRIN, le moteur commence par lire l’intégralité de l’index. Puis il va lire un ensemble de blocs qui contiennent la valeur recherchée, certains ne contenants aucun résultat. Ces lectures supplémentaires se ressentent sur la durée d’exécution de la requête.

Performances en insertion

Vu que les index BRIN sont plus petits et leur durée de création plus courte, on peut se demander ce qu’il advient du surcoût de cet index lors d’insertion de données. Pour cela on va créer une table et mesurer l’insertion de 10 millions de lignes en fonction des index déjà présents sur la table. Afin de cibler le surcoût dû à la mise à jour des index, la table est non-journalisée, ceci permet d’éviter les écritures dans les journaux de transaction. L’autovacuum est également désactivé.

CREATE UNLOGGED TABLE brin_demo_2 (c1 INT);
INSERT INTO brin_demo_2 SELECT * FROM generate_series(1,10000000);
TRUNCATE brin_demo_2;

CREATE INDEX brin_demo_2_brin_idx ON brin_demo_2 USING brin (c1);
INSERT INTO brin_demo_2 SELECT * FROM generate_series(1,10000000);
DROP INDEX brin_demo_2_brin_idx;
TRUNCATE brin_demo_2;
 
CREATE INDEX brin_demo_2_brin_idx ON brin_demo_2 USING brin (c1) WITH (pages_per_range = 256);
INSERT INTO brin_demo_2 SELECT * FROM generate_series(1,10000000);
DROP INDEX brin_demo_2_brin_idx;
TRUNCATE brin_demo_2;
...

Voici les résultats obtenus :

insertionComme pour les chiffres sur les performances en lecture, ces chiffres ne représentent que les durées d’insertion sur mon matériel. La table, les index et les journaux de transaction sont sur le même disque physique, ce qui ralenti les opérations.

Cependant on peut constater qu’il est moins coûteux d’insérer des données dans une table avec un index BRIN qu’avec un index b-tree. On constate également qu’il n’y a pas d’écart significatif entre les différents types d’index BRIN.

Conclusion

Cette série d’articles a permis de présenter les principes des index BRIN. puis leur fonctionnement à travers des exemples simples.

Ensuite nous avons vu l’importance de la corrélation pour exploiter pleinement ces index. Enfin, nous avons essayé de mesurer le gain que pouvait apporter cet index sur de multiples aspect (maintenance, performance en lecture et insertion).

Décrire le fonctionnement d’un index en simplifiant sa représentation est un exercice compliqué. On peut vite sacrifier le fond à la forme. Présenter des chiffres est également délicat tellement ils peuvent dépendre du contexte. J’ai fait l’effort de détailler comment je les ai obtenu afin que chacun puisse reproduire ses propres tests. L’idée est de donner un aperçu des cas d’utilisation de ce type d’index.

Globalement il faut retenir que les index BRIN sont utiles pour les tables volumineuses et où la corrélation avec l’emplacement des données est importante. Ils seront plus lents que les index b-tree lorsque la recherche nécessite de parcourir peu de blocs. Ils seront un peu plus rapide que les index b-tree dans les situations où le moteur doit lire beaucoup de blocs (moins de blocs à lire dans l’index).

L’étude de cet index ouvre d’autres pistes de réflexion. Comme la prise en compte de la corrélation dans le calcul du coût. J’avais également pensé à la possibilité d’utiliser un index pour créer un autre index.

Dans l’exemple avec la table volumineuse (150Go). Si on souhaite créer un index partiel sur le mois précédent, le moteur va parcourir l’intégralité de la table pour créer d’index. On pourrait envisager créer l’index b-tree en utilisant l’index BRIN pour ne parcourir que les lignes correspondant au moins précédent.

FacebookTwitterGoogle+ViadeoPrintEmailGoogle GmailLinkedInPartager

par Adrien Nayrat le jeudi 21 avril 2016 à 20h58

dimanche 13 mars 2016

Guillaume Lelarge

Fin de la traduction du manuel de la 9.5

Beaucoup de retard pour cette fois, mais malgré tout, on a fini la traduction du manuel 9.5 de PostgreSQL. Évidemment, tous les manuels ont aussi été mis à jour avec les dernières versions mineures.

N'hésitez pas à me remonter tout problème sur la traduction.

De même, j'ai pratiquement terminé la traduction des applications. Elle devrait être disponible pour la version 9.5.2 (pas de date encore connue).

par Guillaume Lelarge le dimanche 13 mars 2016 à 10h41

mardi 8 mars 2016

Sébastien Lardière

Dates à retenir

Trois dates à retenir autour de PostgreSQL : 17 mars, à Nantes, un premier meetup, dans lequel j'évoquerai les nouveautés de PostgreSQL 9.5. 31 mars, à Paris, où j'essayerai de remonter le fil du temps de bases de données. 31 mai, à Lille, où je plongerai dans les structures du stockage de PostgreSQL. Ces trois dates sont l'occasion... Lire Dates à retenir

par Sébastien Lardière le mardi 8 mars 2016 à 08h30

samedi 6 février 2016

Guillaume Lelarge

Début de la traduction du manuel 9.5

J'ai enfin fini le merge du manuel de la version 9.5. Très peu de temps avant la 9.5, le peu de temps que j'avais étant consacré à mon livre. Mais là, c'est bon, on peut bosser. D'ailleurs, Flavie a déjà commencé et a traduit un paquet de nouveaux fichiers. Mais il reste du boulot. Pour les intéressés, c'est par là : https://github.com/gleu/pgdocs_fr/wiki/Translation-9.5

N'hésitez pas à m'envoyer toute question si vous êtes intéressé pour participer.

par Guillaume Lelarge le samedi 6 février 2016 à 12h18

jeudi 4 février 2016

Rodolphe Quiédeville

Indexer pour rechercher des chaines courtes dans PostgreSQL

Les champs de recherche dans les applications web permettent de se voir propooser des résultats à chaque caractère saisies dans le formulaire, pour éviter de trop solliciter les systèmes de stockage de données, les modules standards permettent de définir une limite basse, la recherche n'étant effective qu'à partir du troisième caractères entrés. Cette limite de 3 caractères s'explique par la possibilité de simplement définir des index trigram dans les bases de données, pour PostgreSQL cela se fait avec l'extension standard pg_trgm, (pour une étude détaillé des trigrams je conseille la lecture de cet article).

Si cette technique a apporté beaucoup de confort dans l'utilisation des formulaires de recherche elle pose néanmoins le problème lorsque que l'on doit rechercher une chaîne de deux caractères, innoportun, contre-productif me direz-vous (je partage assez cet avis) mais imaginons le cas de madame ou monsieur Ba qui sont présent dans la base de données et dont on a oublié de saisir le prénom ou qui n'ont pas de prénom, ils ne pourront jamais remonter dans ces formulaires de recherche, c'est assez fâcheux pour eux.

Nous allons voir dans cet article comment résoudre ce problème, commençons par créer une table avec 50000 lignes de données text aléatoire :

CREATE TABLE blog AS SELECT s, md5(random()::text) as d 
   FROM generate_series(1,50000) s;
~# SELECT * from blog LIMIT 4;
 s |                 d                
---+----------------------------------
 1 | 8fa4044e22df3bb0672b4fe540dec997
 2 | 5be79f21e03e025f00dea9129dc96afa
 3 | 6b1ffca1425326bef7782865ad4a5c5e
 4 | 2bb3d7093dc0fffd5cebacd07581eef0
(4 rows)

Admettons que l'on soit un fan de musique des années 80 et que l'on recherche si il existe dans notre table du texte contenant la chaîne fff.

~# EXPLAIN ANALYZE SELECT * FROM blog WHERE d like '%fff%';

                                             QUERY PLAN                                              
-----------------------------------------------------------------------------------------------------
 Seq Scan on blog  (cost=0.00..1042.00 rows=5 width=37) (actual time=0.473..24.130 rows=328 loops=1)
   Filter: (d ~~ '%fff%'::text)
   Rows Removed by Filter: 49672
 Planning time: 0.197 ms
 Execution time: 24.251 ms
(5 rows)

Sans index on s'en doute cela se traduit pas une lecture séquentielle de la table, ajoutons un index. Pour indexer cette colonne avec un index GIN nous allons utiliser l'opérateur gin_trgm_ops disponible dans l'extension pg_trgm.

~# CREATE EXTENSION pg_trgm;
CREATE EXTENSION
~# CREATE INDEX blog_trgm_idx ON blog USING GIN(d gin_trgm_ops);
CREATE INDEX
~# EXPLAIN ANALYZE SELECT * FROM blog WHERE d like '%fff%';
                                                       QUERY PLAN                                                        
-------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on blog  (cost=16.04..34.46 rows=5 width=37) (actual time=0.321..1.336 rows=328 loops=1)
   Recheck Cond: (d ~~ '%fff%'::text)
   Heap Blocks: exact=222
   ->  Bitmap Index Scan on blog_trgm_idx  (cost=0.00..16.04 rows=5 width=0) (actual time=0.176..0.176 rows=328 loops=1)
         Index Cond: (d ~~ '%fff%'::text)
 Planning time: 0.218 ms
 Execution time: 1.451 ms

Cette fois l'index a pu être utilisé, on note au passage que le temps de requête est réduit d'un facteur 20, mais si l'on souhaite désormais rechercher une chaîne de seulement 2 caractères de nouveau une lecture séquentielle a lieu, notre index trigram devient inefficace pour cette nouvelle recherche.

~# EXPLAIN ANALYZE SELECT * FROM blog WHERE d like '%ff%';
                                               QUERY PLAN                                                
---------------------------------------------------------------------------------------------------------
 Seq Scan on blog  (cost=0.00..1042.00 rows=3030 width=37) (actual time=0.016..11.712 rows=5401 loops=1)
   Filter: (d ~~ '%ff%'::text)
   Rows Removed by Filter: 44599
 Planning time: 0.165 ms
 Execution time: 11.968 ms

C'est ici que vont intervenir les index bigram, qui comme leur nom l'index travaille sur des couples et non plus des triplets. En premier nous allons tester pgroonga, packagé pour Debian, Ubuntu, CentOS et d'autres systèmes exotiques vous trouverez toutes les explications pour le mettre en place sur la page d'install du projet.

Les versions packagées de la version 1.0.0 ne supportent actuellement que les versions 9.3 et 9.4, mais les sources viennent d'être taguées 1.0.1 avec le support de la 9.5.

CREATE EXTENSION pgroonga;

La création de l'index se fait ensuite en utilisant

~# CREATE INDEX blog_pgroonga_idx ON blog USING pgroonga(d);
CREATE INDEX
~# EXPLAIN ANALYZE SELECT * FROM blog WHERE d like '%ff%';
                                                           QUERY PLAN                                                            
---------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on blog  (cost=27.63..482.51 rows=3030 width=37) (actual time=3.721..5.874 rows=2378 loops=1)
   Recheck Cond: (d ~~ '%ff%'::text)
   Heap Blocks: exact=416
   ->  Bitmap Index Scan on blog_pgroonga_idx  (cost=0.00..26.88 rows=3030 width=0) (actual time=3.604..3.604 rows=2378 loops=1)
         Index Cond: (d ~~ '%ff%'::text)
 Planning time: 0.280 ms
 Execution time: 6.230 ms

On retrouve une utilisation de l'index, avec comme attendu un gain de performance.

Autre solution : pg_bigm qui est dédié plus précisément aux index bigram, l'installation se fait soit à partie de paquets RPM, soit directement depuis les sources avec une explication sur le site, claire et détaillée. pg_bigm supporte toutes les versions depuis la 9.1 jusqu'à 9.5.

~# CREATE INDEX blog_bigm_idx ON blog USING GIN(d gin_bigm_ops);
CREATE INDEX
~# EXPLAIN ANALYZE SELECT * FROM blog WHERE d like '%ff%';
                                                         QUERY PLAN                                                          
-----------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on blog  (cost=35.48..490.36 rows=3030 width=37) (actual time=2.121..5.347 rows=5401 loops=1)
   Recheck Cond: (d ~~ '%ff%'::text)
   Heap Blocks: exact=417
   ->  Bitmap Index Scan on blog_bigm_idx  (cost=0.00..34.73 rows=3030 width=0) (actual time=1.975..1.975 rows=5401 loops=1)
         Index Cond: (d ~~ '%ff%'::text)
 Planning time: 4.406 ms
 Execution time: 6.052 ms

Sur une table de 500k tuples la création de l'index prend 6,5 secondes pour bigm contre 4,8 pour pgroonga ; en terme de lecture je n'ai pas trouvé de pattern avec de réelle différence, bien pgroonga s'annonce plus rapide que pg_bigm, ce premier étant plus récent que le second on peut s'attendre à ce qu'il ait profité de l'expérience du second.

Coté liberté les deux projets sont publiés sous licence PostgreSQL license.

La réelle différence entre les deux projets est que Pgroonga est une sous partie du projet global Groonga qui est dédié à la recherche fulltext, il existe par exemple Mgroonga dont vous devinerez aisément la cible, pg_bigm est lui un projet autonome qui n'implémente que les bigram dans PostgreSQL.

Vous avez désormais deux méthodes pour indexer des 2-gram, prenez garde toutefois de ne pas en abuser.

La version 9.4.5 de PostgreSQL a été utilisée pour la rédaction de cet article.

par Rodolphe Quiédeville le jeudi 4 février 2016 à 08h38

lundi 1 février 2016

Guillaume Lelarge

Parution de mon livre : "PostgreSQL, architecture et notions avancées"

Après pratiquement deux ans de travail, mon livre est enfin paru. Pour être franc, c'est assez étonnant de l'avoir entre les mains : un vrai livre, avec une vraie reliure et une vraie couverture, écrit par soi. C'est à la fois beaucoup de fierté et pas mal de questionnements sur la façon dont il va être reçu.

Ceci étant dit, sans savoir si le livre sera un succès en soi, c'est déjà pour moi un succès personnel. Le challenge était de pouvoir écrire un livre de 300 pages sur PostgreSQL, le livre que j'aurais aimé avoir entre les mains quand j'ai commencé à utiliser ce SGBD il y a maintenant plus de 15 ans sous l'impulsion de mon ancien patron.

Le résultat est à la hauteur de mes espérances et les premiers retours sont très positifs. Ce livre apporte beaucoup d'explications sur le fonctionnement et le comportement de PostgreSQL qui, de ce fait, n'est plus cette espèce de boîte noire à exécuter des requêtes. La critique rédigée par Jean-Michel Armand dans le GNU/Linux Magazine France numéro 190 est vraiment très intéressante. Je suis d'accord avec son auteur sur le fait que le début est assez ardu : on plonge directement dans la technique, sans trop montrer comment c'est utilisé derrière, en production. Cette partie-là n'est abordée qu'après. C'est une question que je m'étais posée lors de la rédaction, mais cette question est l'éternel problème de l'oeuf et de la poule ... Il faut commencer par quelque chose : soit on explique la base technique (ce qui est un peu rude), puis on finit par montrer l'application de cette base, soit on fait l'inverse. Il n'y a certainement pas une solution meilleure que l'autre. Le choix que j'avais fait me semble toujours le bon, même maintenant. Mais en effet, on peut avoir deux façons de lire le livre : en commençant par le début ou en allant directement dans les chapitres thématiques.

Je suis déjà prêt à reprendre le travail pour proposer une deuxième édition encore meilleure. Cette nouvelle édition pourrait se baser sur la prochaine version majeure de PostgreSQL, actuellement numérotée 9.6, qui comprend déjà des nouveautés très excitantes. Mais cette édition ne sera réellement intéressante qu'avec la prise en compte du retour des lecteurs de la première édition, pour corriger et améliorer ce qui doit l'être. N'hésitez donc pas à m'envoyer tout commentaire sur le livre, ce sera très apprécié.

par Guillaume Lelarge le lundi 1 février 2016 à 17h57

mercredi 13 janvier 2016

Sébastien Lardière

Version 9.5 de PostgreSQL - 3

Une nouvelle version majeure de PostgreSQL est disponible depuis le 7 janvier. Chacune des versions de PostgreSQL ajoute son lot de fonctionnalités, à la fois pour le développeur et l'administrateur. Cette version apporte de nombreuses fonctionnalités visant à améliorer les performances lors du requêtage de gros volumes de données.

Cette présentation en trois billets introduit trois types de fonctionnalités :

Ce dernier billet de la série liste quelques paramètres de configuration qui font leur apparition dans cette nouvelle version.Suivi de l'horodatage des COMMITLe paramètre track_commit_timestamp permet de marquer dans les journaux de transactions ("WAL") chaque validation ("COMMIT") avec la date et l'heure du serveur. Ce paramètre est un booléen, et... Lire Version 9.5 de PostgreSQL - 3

par Sébastien Lardière le mercredi 13 janvier 2016 à 15h12

Rodolphe Quiédeville

Index multi colonnes GIN, GIST

Ce billet intéressera tous les utilisateurs de colonnes de type hstore ou json avec PostgreSQL. Bien que celui-ci prenne pour exemple hstore il s'applique également aux colonnes json ou jsonb.

Commençons par créer une table et remplissons là avec 100 000 lignes de données aléatoires. Notre exemple représente des articles qui sont associés à un identifiant de langue (lang_id) et des tags catégorisés (tags), ici chaque article peut être associé à un pays qui sera la Turquie ou l'Islande.

~# CREATE TABLE article (id int4, lang_id int4, tags hstore);
CREATE TABLE
~# INSERT INTO article 
SELECT generate_series(1,10e4::int4), cast(random()*20 as int),
CASE WHEN random() > 0.5 
THEN 'country=>Turquie'::hstore 
WHEN random() > 0.8 THEN 'country=>Islande' ELSE NULL END AS x;
INSERT 0 100000

Pour une recherche efficace des articles dans une langue donnée nous ajountons un index de type B-tree sur la colonne lang_id et un index de type GIN sur la colonne tags.

~# CREATE INDEX ON article(lang_id);
CREATE INDEX
~# CREATE INDEX ON article USING GIN (tags);
CREATE INDEX

Nous avons maintenant nos données et nos index, nous pouvons commencer les recherches. Recherchons tous les articles écrit en français (on considère que l'id du français est le 17), qui sont associés à un pays (ils ont un tag country), et analysons le plan d'exécution.

~# EXPLAIN ANALYZE SELECT * FROM article WHERE lang_id=17 AND tags ? 'country';
                                                                QUERY PLAN                                                                
------------------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on article  (cost=122.42..141.21 rows=5 width=35) (actual time=12.348..13.912 rows=3018 loops=1)
   Recheck Cond: ((tags ? 'country'::text) AND (lang_id = 17))
   Heap Blocks: exact=663
   ->  BitmapAnd  (cost=122.42..122.42 rows=5 width=0) (actual time=12.168..12.168 rows=0 loops=1)
         ->  Bitmap Index Scan on article_tags_idx  (cost=0.00..12.75 rows=100 width=0) (actual time=11.218..11.218 rows=60051 loops=1)
               Index Cond: (tags ? 'country'::text)
         ->  Bitmap Index Scan on article_lang_id_idx  (cost=0.00..109.42 rows=4950 width=0) (actual time=0.847..0.847 rows=5016 loops=1)
               Index Cond: (lang_id = 17)
 Planning time: 0.150 ms
 Execution time: 14.111 ms
(10 rows)

On a logiquement 2 parcours d'index, suivis d'une opération de combinaison pour obtenir le résultat final. Pour gagner un peu en performance on penserait naturellement à créer un index multi colonnes qui contienne lang_id et tags, mais si vous avez déjà essayé de le faire vous avez eu ce message d'erreur :

~# CREATE INDEX ON article USING GIN (lang_id, tags);
ERROR:  42704: data type integer has no default operator class for access method "gin"
HINT:  You must specify an operator class for the index or define a default operator class for the data type.
LOCATION:  GetIndexOpClass, indexcmds.c:1246

Le HINT donnne une piste intéressante, en effet les index de type GIN ne peuvent pas s'appliquer sur les colonnes de type int4 (et bien d'autres).

La solution réside dans l'utilisation d'une extension standard, qui combine les opérateurs GIN et B-tree, btree-gin, précisons tout de suite qu'il existe l'équivalent btree-gist.

Comme toute extension elle s'installe aussi simplement que :

~# CREATE EXTENSION btree_gin;
CREATE EXTENSION

Désormais nous allons pouvoir créer notre index multi-colonne et rejouer notre requête pour voir la différence.

~# CREATE INDEX ON article USING GIN (lang_id, tags);
CREATE INDEX
~# EXPLAIN ANALYZE SELECT * FROM article WHERE lang_id=17 AND tags ? 'country';
                                                             QUERY PLAN                                                              
-------------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on article  (cost=24.05..42.84 rows=5 width=35) (actual time=1.983..3.777 rows=3018 loops=1)
   Recheck Cond: ((lang_id = 17) AND (tags ? 'country'::text))
   Heap Blocks: exact=663
   ->  Bitmap Index Scan on article_lang_id_tags_idx  (cost=0.00..24.05 rows=5 width=0) (actual time=1.875..1.875 rows=3018 loops=1)
         Index Cond: ((lang_id = 17) AND (tags ? 'country'::text))
 Planning time: 0.211 ms
 Execution time: 3.968 ms
(7 rows)

A la lecture de ce deuxième explain le gain est explicite, même avec un petit jeu de données le coût estimé est divisé par 3, l'on gagne une lecture d'index et une opération de composition. Maintenant nous pouvons supprimer les 2 autres index pour ne conserver que celui-ci.

par Rodolphe Quiédeville le mercredi 13 janvier 2016 à 14h11

mardi 12 janvier 2016

Sébastien Lardière

Version 9.5 de PostgreSQL - 2

Une nouvelle version majeure de PostgreSQL est disponible depuis le 7 janvier. Chacune des versions de PostgreSQL ajoute son lot de fonctionnalités, à la fois pour le développeur et l'administrateur. Cette version apporte de nombreuses fonctionnalités visant à améliorer les performances lors du requêtage de gros volumes de données.

Cette présentation en trois billets introduit trois types de fonctionnalités :

Ce deuxième billet évoque donc les nouvelles méthodes internes du moteur, c'est-à-dire de nouveaux outils dont PostgreSQL dispose pour traiter les données.Index BRINIl s'agit d'un nouveau type d'index, créé pour résoudre des problèmes d'accès à de très gros volumes de données ; le cas d'usage est une table de log, dans laquelle les données sont... Lire Version 9.5 de PostgreSQL - 2

par Sébastien Lardière le mardi 12 janvier 2016 à 10h45

vendredi 8 janvier 2016

Sébastien Lardière

Version 9.5 de PostgreSQL

Une nouvelle version majeure de PostgreSQL est disponible depuis le 7 janvier. Chacune des versions de PostgreSQL ajoute son lot de fonctionnalités, à la fois pour le développeur et l'administrateur. Cette version apporte de nombreuses fonctionnalités visant à améliorer les performances lors du requêtage de gros volumes de données.

Cette présentation en trois billets introduit trois types de fonctionnalités :

Ce premier billet revient donc sur les ajouts au langage SQL de la version 9.5 de PostgreSQL.Ces ajouts portent sur de nombreux champs de fonctionnalités, de la sécurité aux gestions de performances en passant par la gestion des transactions.UPSERTCe mot clé désigne en réalité la possibilité d'intercepter une erreur de clé primaire sur un ordre... Lire Version 9.5 de PostgreSQL

par Sébastien Lardière le vendredi 8 janvier 2016 à 10h49

mercredi 30 décembre 2015

Nicolas Thauvin

Réflexions sur l'archivage des fichiers WAL

PostgreSQL faisant son bonhomme de chemin, on se retrouve désormais avec des configurations où il faut archiver plusieurs fois les WAL parce qu'on a de la sauvegarde PITR et de la réplication.

Le plus gros piège lorsqu'on a du PITR et de la réplication, ou plusieurs serveurs standby, c'est d'oublier que chaque élément de l'architecture qui consomme du WAL doit avoir son propre répertoire de WAL archivés, car chacun purge les fichiers différemment.

On tombe dans le piège facilement, en se disant, "pas de problème pour la purge des vieux fichiers, c'est le PITR ou le slave le plus éloigné qui purgera". Si la purge du PITR passe alors que le standby était déconnecté du maitre, cette purge peut casser la réplication.

La solution est d'archiver plusieurs fois le même fichier WAL. Pour optimiser, le plus efficace à l'usage est d'utiliser des hardlinks. En gros, on archive une fois le fichier WAL et on crée autant de lien hard qu'il faut pour les autres consommateurs de WAL archivés. Rappelons, que la donnée n'est supprimée que lorsqu'il n'existe plus aucun lien et que plus aucun processus n'a le fichier ouvert, à ne pas confondre avec un lien symbolique.

Pour archiver vite, il vaut mieux éviter de compresser et stocker les archives soit en local, soit sur un partage NFS, l'archivage par SSH restant le plus lent. Tout est compromis entre chiffrage des communications sur le réseau et espace disque disponible, les liens hard restant rapides à créer et avec une consommation d'espace disque supplémentaire négligeable.

Enfin, PostgreSQL exécute la commande d'archivage avec l'appel system() qui fork un shell : toutes les possibilités du shell sont alors disponibles, par exemple :

archive_command = 'rsync -a %p slave1:/archived_xlog/slave1/%f && ssh slave1 "for h in slave2 pitr; do ln /archived_xlog/slave1/%f /archived_xlog/$h/%f; done"'

Oui, une boucle et une seule copie avec rsync par SSH pour 3 utilisations. On préfèrera surement faire un script pour rendre les choses plus lisibles. Ça marche aussi pour restore_command et archive_cleanup_command dans recovery.conf :

restore_command = 'cp /archived_xlog/$(hostname)/%f %p'
archive_cleanup_command = 'pg_archivecleanup /archived_xlog/$(hostname) %r'

mercredi 30 décembre 2015 à 21h01

jeudi 19 novembre 2015

Guillaume Lelarge

Version finale du livre

Elle n'est pas encore sortie. Elle est pratiquement terminée, on attend d'avoir le livre en version imprimée.

Néanmoins, je peux déjà dire les nouveautés par rapport à la beta 0.4 :

  • Global
    • mise à jour du texte pour la 9.5
    • ajout du chapitre sur la sécurité
    • ajout du chapitre sur la planification
    • mise à jour des exemples avec PostgreSQL 9.5 beta 1
  • Fichiers
    • Ajout d'un schéma sur les relations entre tables, FSM et VM
    • Ajout de la description des répertoires pg_dynshmem et pg_logical
  • Contenu des fichiers
    • Ajout d'informations sur le stockage des données, colonne par colonne
    • Ajout d'un schéma sur la structure logique et physique d'un index B-tree
    • Ajout de la description des index GIN
    • Ajout de la description des index GiST
    • Ajout de la description des index SP-GiST
  • Architecture mémoire
    • calcul du work_mem pour un tri
    • calcul du maintenance_work_mem pour un VACUUM
  • Gestion des transactions
    • Gestion des verrous et des accès concurrents
  • Maintenance
    • Description de la sortie d'un VACUUM VERBOSE

J'avoue que j'ai hâte d'avoir la version finale entre mes mains :-) Bah, oui, c'est quand même 1 an et demi de boulot acharné !

par Guillaume Lelarge le jeudi 19 novembre 2015 à 22h36