diff options
| author | Tom Lane <tgl@sss.pgh.pa.us> | 2022-05-21 13:13:41 -0400 |
|---|---|---|
| committer | Tom Lane <tgl@sss.pgh.pa.us> | 2022-05-21 13:13:44 -0400 |
| commit | a916cb9d5a89804998dd4e7fd7bbb27cb5a7abc8 (patch) | |
| tree | 95fff229b2b404be908ad28ff2f9406aa1e3f497 /src/backend/executor | |
| parent | ac1ae477f85c6aeb3119071c1c00eb042b4afa4d (diff) | |
| download | postgresql-a916cb9d5a89804998dd4e7fd7bbb27cb5a7abc8.tar.gz postgresql-a916cb9d5a89804998dd4e7fd7bbb27cb5a7abc8.zip | |
Avoid overflow hazard when clamping group counts to "long int".
Several places in the planner tried to clamp a double value to fit
in a "long" by doing
(long) Min(x, (double) LONG_MAX);
This is subtly incorrect, because it casts LONG_MAX to double and
potentially back again. If long is 64 bits then the double value
is inexact, and the platform might round it up to LONG_MAX+1
resulting in an overflow and an undesirably negative output.
While it's not hard to rewrite the expression into a safe form,
let's put it into a common function to reduce the risk of someone
doing it wrong in future.
In principle this is a bug fix, but since the problem could only
manifest with group count estimates exceeding 2^63, it seems unlikely
that anyone has actually hit this or will do so anytime soon. We're
fixing it mainly to satisfy fuzzer-type tools. That being the case,
a HEAD-only fix seems sufficient.
Andrey Lepikhov
Discussion: https://postgr.es/m/ebbc2efb-7ef9-bf2f-1ada-d6ec48f70e58@postgrespro.ru
Diffstat (limited to 'src/backend/executor')
| -rw-r--r-- | src/backend/executor/nodeSubplan.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/src/backend/executor/nodeSubplan.c b/src/backend/executor/nodeSubplan.c index 60d22900304..43b36dcd3b4 100644 --- a/src/backend/executor/nodeSubplan.c +++ b/src/backend/executor/nodeSubplan.c @@ -26,7 +26,6 @@ */ #include "postgres.h" -#include <limits.h> #include <math.h> #include "access/htup_details.h" @@ -35,6 +34,7 @@ #include "miscadmin.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" +#include "optimizer/optimizer.h" #include "utils/array.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -498,7 +498,7 @@ buildSubPlanHash(SubPlanState *node, ExprContext *econtext) node->havehashrows = false; node->havenullrows = false; - nbuckets = (long) Min(planstate->plan->plan_rows, (double) LONG_MAX); + nbuckets = clamp_cardinality_to_long(planstate->plan->plan_rows); if (nbuckets < 1) nbuckets = 1; |