]> git.kaiwu.me - nginx.git/log
nginx.git
5 years agoCancel keepalive and lingering close on EOF better (ticket #2145).
Sergey Kandaurov [Wed, 24 Mar 2021 11:03:33 +0000 (14:03 +0300)]
Cancel keepalive and lingering close on EOF better (ticket #2145).

Unlike in 75e908236701, which added the logic to ngx_http_finalize_request(),
this change moves it to a more generic routine ngx_http_finalize_connection()
to cover cases when a request is finalized with NGX_DONE.

In particular, this fixes unwanted connection transition into the keepalive
state after receiving EOF while discarding request body.  With edge-triggered
event methods that means the connection will last for extra seconds as set in
the keepalive_timeout directive.

5 years agogRPC: fixed handling of padding on DATA frames.
Maxim Dounin [Tue, 23 Mar 2021 13:52:23 +0000 (16:52 +0300)]
gRPC: fixed handling of padding on DATA frames.

The response size check introduced in 39501ce97e29 did not take into
account possible padding on DATA frames, resulting in incorrect
"upstream sent response body larger than indicated content length" errors
if upstream server used padding in responses with known length.

Fix is to check the actual size of response buffers produced by the code,
similarly to how it is done in other protocols, instead of checking
the size of DATA frames.

Reported at:
http://mailman.nginx.org/pipermail/nginx-devel/2021-March/013907.html

5 years agoRemoved "ch" argument from ngx_pass_open_channel().
Ruslan Ermilov [Thu, 11 Mar 2021 06:58:45 +0000 (09:58 +0300)]
Removed "ch" argument from ngx_pass_open_channel().

5 years agoMail: fixed build without SSL.
Maxim Dounin [Thu, 11 Mar 2021 01:46:26 +0000 (04:46 +0300)]
Mail: fixed build without SSL.

Broken by d84f13618277 and 12ea1de7d87c (1.19.8).

Reported by Sergey Osokin.

5 years agoVersion bump.
Maxim Dounin [Thu, 11 Mar 2021 01:46:22 +0000 (04:46 +0300)]
Version bump.

5 years agorelease-1.19.8 tag
Maxim Dounin [Tue, 9 Mar 2021 15:27:51 +0000 (18:27 +0300)]
release-1.19.8 tag

5 years agonginx-1.19.8-RELEASE release-1.19.8
Maxim Dounin [Tue, 9 Mar 2021 15:27:50 +0000 (18:27 +0300)]
nginx-1.19.8-RELEASE

5 years agoUpdated OpenSSL used for win32 builds.
Maxim Dounin [Tue, 9 Mar 2021 13:38:55 +0000 (16:38 +0300)]
Updated OpenSSL used for win32 builds.

5 years agoMail: sending of the PROXY protocol to backends.
Maxim Dounin [Fri, 5 Mar 2021 14:16:32 +0000 (17:16 +0300)]
Mail: sending of the PROXY protocol to backends.

Activated with the "proxy_protocol" directive.  Can be combined with
"listen ... proxy_protocol;" and "set_real_ip_from ...;" to pass
client address provided to nginx in the PROXY protocol header.

5 years agoMail: realip module.
Maxim Dounin [Fri, 5 Mar 2021 14:16:29 +0000 (17:16 +0300)]
Mail: realip module.

When configured with the "set_real_ip_from", it can set client's IP
address as visible in logs to the one obtained via the PROXY protocol.

5 years agoMail: parsing of the PROXY protocol from clients.
Maxim Dounin [Fri, 5 Mar 2021 14:16:24 +0000 (17:16 +0300)]
Mail: parsing of the PROXY protocol from clients.

Activated with the "proxy_protocol" parameter of the "listen" directive.
Obtained information is passed to the auth_http script in Proxy-Protocol-Addr,
Proxy-Protocol-Port, Proxy-Protocol-Server-Addr, and Proxy-Protocol-Server-Port
headers.

5 years agoMail: made auth http creating request easier to extend.
Maxim Dounin [Fri, 5 Mar 2021 14:16:23 +0000 (17:16 +0300)]
Mail: made auth http creating request easier to extend.

5 years agoMail: fixed log action after SSL handshake.
Maxim Dounin [Fri, 5 Mar 2021 14:16:20 +0000 (17:16 +0300)]
Mail: fixed log action after SSL handshake.

5 years agoMail: postponed session initialization under accept mutex.
Maxim Dounin [Fri, 5 Mar 2021 14:16:19 +0000 (17:16 +0300)]
Mail: postponed session initialization under accept mutex.

Similarly to 40e8ce405859 in the stream module, this reduces the time
accept mutex is held.  This also simplifies following changes to
introduce PROXY protocol support.

5 years agoMail: added missing event handling after reading data.
Maxim Dounin [Fri, 5 Mar 2021 14:16:17 +0000 (17:16 +0300)]
Mail: added missing event handling after reading data.

If we need to be notified about further events, ngx_handle_read_event()
needs to be called after a read event is processed.  Without this,
an event can be removed from the kernel and won't be reported again,
notably when using oneshot event methods, such as eventport on Solaris.

For consistency, existing ngx_handle_read_event() call removed from
ngx_mail_read_command(), as this call only covers one of the code paths
where ngx_mail_read_command() returns NGX_AGAIN.  Instead, appropriate
processing added to the callers, covering all code paths where NGX_AGAIN
is returned.

5 years agoMail: added missing event handling after blocking events.
Maxim Dounin [Fri, 5 Mar 2021 14:16:16 +0000 (17:16 +0300)]
Mail: added missing event handling after blocking events.

As long as a read event is blocked (ignored), ngx_handle_read_event()
needs to be called to make sure no further notifications will be
triggered when using level-triggered event methods, such as select() or
poll().

5 years agoEvents: fixed eventport handling in ngx_handle_read_event().
Maxim Dounin [Fri, 5 Mar 2021 14:16:15 +0000 (17:16 +0300)]
Events: fixed eventport handling in ngx_handle_read_event().

The "!rev->ready" test seems to be a typo, introduced in the original
commit (719:f30b1a75fd3b).  The ngx_handle_write_event() code properly
tests for "rev->ready" instead.

Due to this typo, read events might be unexpectedly removed during
proxying after an event on the other part of the proxied connection.
Catched by mail proxying tests.

5 years agoSSL: fixed build by Sun C with old OpenSSL versions.
Maxim Dounin [Fri, 5 Mar 2021 14:16:13 +0000 (17:16 +0300)]
SSL: fixed build by Sun C with old OpenSSL versions.

Sun C complains about "statement not reached" if a "return" is followed
by additional statements.

5 years agoProxy: variables support in "proxy_cookie_flags" flags.
Ruslan Ermilov [Mon, 1 Mar 2021 21:58:24 +0000 (00:58 +0300)]
Proxy: variables support in "proxy_cookie_flags" flags.

5 years agoIntroduced strerrordesc_np() support.
Maxim Dounin [Mon, 1 Mar 2021 17:00:45 +0000 (20:00 +0300)]
Introduced strerrordesc_np() support.

The strerrordesc_np() function, introduced in glibc 2.32, provides an
async-signal-safe way to obtain error messages.  This makes it possible
to avoid copying error messages.

5 years agoImproved maximum errno detection.
Maxim Dounin [Mon, 1 Mar 2021 17:00:43 +0000 (20:00 +0300)]
Improved maximum errno detection.

Previously, systems without sys_nerr (or _sys_nerr) were handled with an
assumption that errors start at 0 and continuous.  This is, however, not
something POSIX requires, and not true on some platforms.

Notably, on Linux, where sys_nerr is no longer available for newly linked
binaries starting with glibc 2.32, there are gaps in error list, which
used to stop us from properly detecting maximum errno.  Further, on
GNU/Hurd errors start at 0x40000001.

With this change, maximum errno detection is moved to the runtime code,
now able to ignore gaps, and also detects the first error if needed.
This fixes observed "Unknown error" messages as seen on Linux with
glibc 2.32 and on GNU/Hurd.

5 years agoHTTP/2: client_header_timeout before first request (ticket #2142).
Maxim Dounin [Mon, 1 Mar 2021 14:31:28 +0000 (17:31 +0300)]
HTTP/2: client_header_timeout before first request (ticket #2142).

With this change, behaviour of HTTP/2 becomes even closer to HTTP/1.x,
and client_header_timeout instead of keepalive_timeout is used before
the first request is received.

This fixes HTTP/2 connections being closed even before the first request
if "keepalive_timeout 0;" was used in the configuration; the problem
appeared in f790816a0e87 (1.19.7).

5 years agoContrib: vim syntax, default highlighting (ticket #2141).
Maxim Dounin [Thu, 25 Feb 2021 20:42:25 +0000 (23:42 +0300)]
Contrib: vim syntax, default highlighting (ticket #2141).

Using default highlighting makes it possible to easily overrule
highlighting specified in the syntax file, see ":highlight-default"
in vim help for details.

5 years agoSSL: added check for debugging.
Maxim Dounin [Sat, 20 Feb 2021 15:03:04 +0000 (18:03 +0300)]
SSL: added check for debugging.

If debugging is not enabled, there is no need to do extra work in
ngx_ssl_verify_callback() and ngx_ssl_handshake_log().

5 years agoSSL: added missed error reporting during variables evaluation.
Maxim Dounin [Sat, 20 Feb 2021 15:02:54 +0000 (18:02 +0300)]
SSL: added missed error reporting during variables evaluation.

5 years agoSSL: X509_NAME_oneline() error handling.
Maxim Dounin [Sat, 20 Feb 2021 15:02:49 +0000 (18:02 +0300)]
SSL: X509_NAME_oneline() error handling.

5 years agoConfigure: marked top-level make targets as phony.
Ruslan Ermilov [Sat, 20 Feb 2021 09:44:26 +0000 (12:44 +0300)]
Configure: marked top-level make targets as phony.

Reported by Thibault NĂ©lis.

5 years agoVersion bump.
Ruslan Ermilov [Sat, 20 Feb 2021 09:44:07 +0000 (12:44 +0300)]
Version bump.

5 years agorelease-1.19.7 tag
Maxim Dounin [Tue, 16 Feb 2021 15:57:18 +0000 (18:57 +0300)]
release-1.19.7 tag

5 years agonginx-1.19.7-RELEASE release-1.19.7
Maxim Dounin [Tue, 16 Feb 2021 15:57:18 +0000 (18:57 +0300)]
nginx-1.19.7-RELEASE

5 years agoHTTP/2: removed http2_max_field_size and http2_max_header_size.
Maxim Dounin [Thu, 11 Feb 2021 18:52:26 +0000 (21:52 +0300)]
HTTP/2: removed http2_max_field_size and http2_max_header_size.

Instead, size of one large_client_header_buffers buffer and all large
client header buffers are used.

5 years agoHTTP/2: keepalive_timeout now armed once between requests.
Maxim Dounin [Thu, 11 Feb 2021 18:52:24 +0000 (21:52 +0300)]
HTTP/2: keepalive_timeout now armed once between requests.

Previously, PINGs and other frames extended possible keepalive time,
making it possible to keep an open HTTP/2 connection for a long time.
Now the connection is always closed as long as keepalive_timeout expires,
similarly to how it happens in HTTP/1.x.

Note that as a part of this change, incomplete frames are no longer
trigger a separate timeout, so http2_recv_timeout (replaced by
client_header_timeout in previous patches) is essentially cancelled.
The client_header_timeout is, however, used for SSL handshake and
while reading HEADERS frames.

5 years agoHTTP/2: removed http2_idle_timeout and http2_max_requests.
Maxim Dounin [Thu, 11 Feb 2021 18:52:23 +0000 (21:52 +0300)]
HTTP/2: removed http2_idle_timeout and http2_max_requests.

Instead, keepalive_timeout and keepalive_requests are now used.  This
is expected to simplify HTTP/2 code and usage.  This also matches
directives used by upstream module for all protocols.

In case of default settings, this effectively changes maximum number
of requests per connection from 1000 to 100.  This looks acceptable,
especially given that HTTP/2 code now properly supports lingering close.

Further, this changes default keepalive timeout in HTTP/2 from 300 seconds
to 75 seconds.  This also looks acceptable, and larger than PING interval
used by Firefox (network.http.spdy.ping-threshold defaults to 58s),
the only browser to use PINGs.

5 years agoHTTP/2: removed http2_recv_timeout.
Maxim Dounin [Thu, 11 Feb 2021 18:52:20 +0000 (21:52 +0300)]
HTTP/2: removed http2_recv_timeout.

Instead, the client_header_timeout is now used for HTTP/2 reading.
Further, the timeout is changed to be set once till no further data
left to read, similarly to how client_header_timeout is used in other
places.

5 years agoHTTP/2: removed SPDY directives handling.
Maxim Dounin [Thu, 11 Feb 2021 18:52:19 +0000 (21:52 +0300)]
HTTP/2: removed SPDY directives handling.

The spdy_* directives are not available since introduction of HTTP/2 module
in nginx 1.9.5 more than five years ago.

5 years agoHTTP/2: fixed reusing connections with active requests.
Maxim Dounin [Thu, 11 Feb 2021 18:52:17 +0000 (21:52 +0300)]
HTTP/2: fixed reusing connections with active requests.

New connections are marked reusable by ngx_http_init_connection() if there
are no data available for reading.  As a result, if SSL is not used,
ngx_http_v2_init() might be called when the connection is marked reusable.
If a HEADERS frame is immediately available for reading, this resulted
in connection being preserved in reusable state with an active request,
and possibly closed later as if during worker shutdown (that is, after
all active requests were finalized).

Fix is to explicitly mark connections non-reusable in ngx_http_v2_init()
instead of (incorrectly) assuming they are already non-reusable.

Found by Sergey Kandaurov.

5 years agoHTTP/2: reuse of connections with incomplete frames.
Maxim Dounin [Thu, 11 Feb 2021 18:52:12 +0000 (21:52 +0300)]
HTTP/2: reuse of connections with incomplete frames.

Prodded by Taewoo Kim.

5 years agoAdditional connections reuse.
Maxim Dounin [Thu, 11 Feb 2021 18:52:11 +0000 (21:52 +0300)]
Additional connections reuse.

If ngx_drain_connections() fails to immediately reuse any connections
and there are no free connections, it now additionally tries to reuse
a connection again.  This helps to provide at least one free connection
in case of HTTP/2 with lingering close, where merely trying to reuse
a connection once does not free it, but makes it reusable again,
waiting for lingering close.

5 years agoReuse of connections in lingering close.
Maxim Dounin [Thu, 11 Feb 2021 18:52:09 +0000 (21:52 +0300)]
Reuse of connections in lingering close.

This is particularly important in HTTP/2, where keepalive connections
are closed with lingering.  Before the patch, reusing a keepalive HTTP/2
connection resulted in the connection waiting for lingering close to
remain in the reusable connections queue, preventing ngx_drain_connections()
from closing additional connections.

The patch fixes it by marking the connection reusable again, and so
moving it in the reusable connections queue.  Further, it makes actually
possible to reuse such connections if needed.

5 years agoHTTP/2: lingering close changed to handle NGX_AGAIN.
Ruslan Ermilov [Mon, 1 Feb 2021 13:42:50 +0000 (16:42 +0300)]
HTTP/2: lingering close changed to handle NGX_AGAIN.

This part somehow slipped away from c5840ca2063d.

While it is not expected to be needed in case of lingering close,
it is good to keep it for correctness (see 2b5528023f6b).

5 years agoClean up trailers in ngx_http_clean_header() as well.
Sergey Kandaurov [Tue, 26 Jan 2021 09:39:28 +0000 (12:39 +0300)]
Clean up trailers in ngx_http_clean_header() as well.

The function has not been updated with introduction of trailers support
in 7034:1b068a4e82d8 (1.13.2).

5 years agoYear 2021.
Maxim Dounin [Tue, 19 Jan 2021 17:35:17 +0000 (20:35 +0300)]
Year 2021.

5 years agoCore: removed post_accept_timeout.
Maxim Dounin [Tue, 19 Jan 2021 17:32:00 +0000 (20:32 +0300)]
Core: removed post_accept_timeout.

Keeping post_accept_timeout in ngx_listening_t is no longer needed since
we've switched to 1 second timeout for deferred accept in 5541:fdb67cfc957d.

Further, using it in HTTP code can result in client_header_timeout being
used from an incorrect server block, notably if address-specific virtual
servers are used along with a wildcard listening socket, or if we've switched
to a different server block based on SNI in SSL handshake.

5 years agoRemoved incorrect optimization of HEAD requests.
Maxim Dounin [Tue, 19 Jan 2021 17:21:12 +0000 (20:21 +0300)]
Removed incorrect optimization of HEAD requests.

The stub status module and ngx_http_send_response() (used by the empty gif
module and the "return" directive) incorrectly assumed that responding
to HEAD requests always results in r->header_only being set.  This is not
true, and results in incorrect behaviour, for example, in the following
configuration:

   location / {
       image_filter size;
       return 200 test;
   }

Fix is to remove this incorrect micro-optimization from both stub status
module and ngx_http_send_response().

Reported by Chris Newton.

5 years agoUpstream: fixed zero size buf alerts on extra data (ticket #2117).
Maxim Dounin [Tue, 12 Jan 2021 13:59:31 +0000 (16:59 +0300)]
Upstream: fixed zero size buf alerts on extra data (ticket #2117).

After 7675:9afa45068b8f and 7678:bffcc5af1d72 (1.19.1), during non-buffered
simple proxying, responses with extra data might result in zero size buffers
being generated and "zero size buf" alerts in writer.  This bug is similar
to the one with FastCGI proxying fixed in 7689:da8d758aabeb.

In non-buffered mode, normally the filter function is not called if
u->length is already 0, since u->length is checked after each call of
the filter function.  There is a case when this can happen though: if
the response length is 0, and there are pre-read response body data left
after reading response headers.  As such, a check for u->length is needed
at the start of non-buffered filter functions, similar to the one
for p->length present in buffered filter functions.

Appropriate checks added to the existing non-buffered copy filters
in the upstream (used by scgi and uwsgi proxying) and proxy modules.

5 years agoContrib: vim syntax, update core and 3rd party module directives.
Gena Makhomed [Tue, 29 Dec 2020 11:13:57 +0000 (13:13 +0200)]
Contrib: vim syntax, update core and 3rd party module directives.

5 years agoVersion bump.
Maxim Dounin [Mon, 11 Jan 2021 19:06:27 +0000 (22:06 +0300)]
Version bump.

5 years agorelease-1.19.6 tag
Maxim Dounin [Tue, 15 Dec 2020 14:41:39 +0000 (17:41 +0300)]
release-1.19.6 tag

5 years agonginx-1.19.6-RELEASE release-1.19.6
Maxim Dounin [Tue, 15 Dec 2020 14:41:39 +0000 (17:41 +0300)]
nginx-1.19.6-RELEASE

5 years agoUpdated OpenSSL used for win32 builds.
Maxim Dounin [Tue, 15 Dec 2020 13:49:24 +0000 (16:49 +0300)]
Updated OpenSSL used for win32 builds.

5 years agoFixed double close of non-regular files in flv and mp4.
Maxim Dounin [Fri, 11 Dec 2020 10:42:07 +0000 (13:42 +0300)]
Fixed double close of non-regular files in flv and mp4.

With introduction of open_file_cache in 1454:f497ed7682a7, opening a file
with ngx_open_cached_file() automatically adds a cleanup handler to close
the file.  As such, calling ngx_close_file() directly for non-regular files
is no longer needed and will result in duplicate close() call.

In 1454:f497ed7682a7 ngx_close_file() call for non-regular files was removed
in the static module, but wasn't in the flv module.  And the resulting
incorrect code was later copied to the mp4 module.  Fix is to remove the
ngx_close_file() call from both modules.

Reported by Chris Newton.

5 years agoRemoved extra allocation for r->uri.
Maxim Dounin [Thu, 10 Dec 2020 17:09:39 +0000 (20:09 +0300)]
Removed extra allocation for r->uri.

The ngx_http_parse_complex_uri() function cannot make URI longer and does
not null-terminate URI, so there is no need to allocate an extra byte.  This
allocation appears to be a leftover from changes in 461:a88a3e4e158f (0.1.5),
where null-termination of r->uri and many other strings was removed.

5 years agoFixed parsing of absolute URIs with empty path (ticket #2079).
Maxim Dounin [Thu, 10 Dec 2020 17:09:30 +0000 (20:09 +0300)]
Fixed parsing of absolute URIs with empty path (ticket #2079).

When the request line contains request-target in the absolute-URI form,
it can contain path-empty instead of a single slash (see RFC 7230, RFC 3986).
Previously, the ngx_http_parse_request_line() function only accepted empty
path when there was no query string.

With this change, non-empty query is also correctly handled.  That is,
request line "GET http://example.com?foo HTTP/1.1" is accepted and results
in $uri "/" and $args "foo".

Note that $request_uri remains "?foo", similarly to how spaces in URIs
are handled.  Providing "/?foo", similarly to how "/" is provided for
"GET http://example.com HTTP/1.1", requires allocation.

5 years agoSSL: fixed SSL shutdown on lingering close.
Ruslan Ermilov [Mon, 7 Dec 2020 22:43:36 +0000 (01:43 +0300)]
SSL: fixed SSL shutdown on lingering close.

Ensure c->recv is properly reset to ngx_recv if SSL_shutdown()
blocks on writing.

The bug had appeared in 554c6ae25ffc.

5 years agoUpstream: excluded down servers from the next_upstream tries.
Ruslan Ermilov [Thu, 26 Nov 2020 21:01:20 +0000 (00:01 +0300)]
Upstream: excluded down servers from the next_upstream tries.

Previously, the number of next_upstream tries included servers marked
as "down", resulting in "no live upstreams" with the code 502 instead
of the code derived from an attempt to connect to the last tried "up"
server (ticket #2096).

5 years agoVersion bump.
Ruslan Ermilov [Thu, 26 Nov 2020 20:46:59 +0000 (23:46 +0300)]
Version bump.

5 years agorelease-1.19.5 tag
Maxim Dounin [Tue, 24 Nov 2020 15:06:34 +0000 (18:06 +0300)]
release-1.19.5 tag

5 years agonginx-1.19.5-RELEASE release-1.19.5
Maxim Dounin [Tue, 24 Nov 2020 15:06:34 +0000 (18:06 +0300)]
nginx-1.19.5-RELEASE

5 years agogRPC: RST_STREAM(NO_ERROR) handling after "trailer only" responses.
Pavel Pautov [Thu, 19 Nov 2020 02:41:16 +0000 (18:41 -0800)]
gRPC: RST_STREAM(NO_ERROR) handling after "trailer only" responses.

Similarly to the problem fixed in 2096b21fcd10 (ticket #1792),
when a "trailer only" gRPC response (that is, a response with the
END_STREAM flag in the HEADERS frame) was immediately followed by
RST_STREAM(NO_ERROR) in the data preread along with the response
header, RST_STREAM wasn't properly skipped and caused "upstream
rejected request with error 0" errors.
Observed with "unknown service" gRPC errors returned by grpc-go.

Fix is to set ctx->done if we are going to parse additional data,
so the RST_STREAM(NO_ERROR) is properly skipped.  Additionally, now
ngx_http_grpc_filter() will complain about frames sent for closed
stream if there are any.

5 years agoUse .Mt to mark up email addresses.
Sergey Kandaurov [Thu, 19 Nov 2020 17:15:22 +0000 (17:15 +0000)]
Use .Mt to mark up email addresses.

5 years agoCore: "-e" command line option.
Igor Ippolitov [Thu, 19 Nov 2020 16:59:00 +0000 (16:59 +0000)]
Core: "-e" command line option.

When installing or running from a non-root user it is sometimes required to
override default, compiled in error log path. There was no way to do this
without rebuilding the binary (ticket #147).

This patch introduced "-e" command line option which allows one to override
compiled in error log path.

5 years agoConfigure: shared sources for addon modules.
Maxim Dounin [Tue, 10 Nov 2020 14:13:20 +0000 (17:13 +0300)]
Configure: shared sources for addon modules.

Addon modules, both dynamic and static, can now use shared source files.
Shared sources result in only one make rule even if specified several
times in different modules.

5 years agoConfigure: initialization of NGX_ADDON_SRCS.
Maxim Dounin [Tue, 10 Nov 2020 14:13:17 +0000 (17:13 +0300)]
Configure: initialization of NGX_ADDON_SRCS.

5 years agoConfigure: style.
Maxim Dounin [Tue, 10 Nov 2020 14:13:14 +0000 (17:13 +0300)]
Configure: style.

5 years agoRequest body: removed error assumption (ticket #2058).
Maxim Dounin [Mon, 9 Nov 2020 19:41:54 +0000 (22:41 +0300)]
Request body: removed error assumption (ticket #2058).

Before introduction of request body filter in 42d9beeb22db, the only
possible return code from the ngx_http_request_body_filter() call
without actual buffers was NGX_HTTP_INTERNAL_SERVER_ERROR, and
the code in ngx_http_read_client_request_body() hardcoded the only
possible error to simplify the code of initial call to set rb->rest.

This is no longer true after introduction of request body filters though,
as a request body filter might need to return other errors, such as 403.
Fix is to preserve the error code actually returned by the call
instead of assuming 500.

5 years agoRequest body: improved logging.
Maxim Dounin [Mon, 9 Nov 2020 19:40:53 +0000 (22:40 +0300)]
Request body: improved logging.

Added logging before returning NGX_HTTP_INTERNAL_SERVER_ERROR if there
are busy buffers after a request body flush.  This should never happen
with current code, though bugs can be introduced by 3rd party modules.
Make sure debugging will be easy enough.

5 years agoSSL: fixed non-working SSL shutdown on lingering close.
Ruslan Ermilov [Fri, 6 Nov 2020 20:44:54 +0000 (23:44 +0300)]
SSL: fixed non-working SSL shutdown on lingering close.

When doing lingering close, the socket was first shut down for writing,
so SSL shutdown initiated after lingering close was not able to send
the close_notify alerts (ticket #2056).

The fix is to call ngx_ssl_shutdown() before shutting down the socket.

5 years agoRemoved dead code from ngx_http_set_keepalive().
Ruslan Ermilov [Fri, 6 Nov 2020 20:44:47 +0000 (23:44 +0300)]
Removed dead code from ngx_http_set_keepalive().

The code removed became dead after 98f03cd8d6cc (0.8.14),
circa when the request reference counting was introduced.

5 years agoCore: added format specifiers to output binary data as hex.
Vladimir Homutov [Wed, 28 Oct 2020 07:56:11 +0000 (10:56 +0300)]
Core: added format specifiers to output binary data as hex.

Now "s", "V", and "v" format specifiers may be prefixed with "x" (lowercase)
or "X" (uppercase) to output corresponding data in hexadecimal format.

In collaboration with Maxim Dounin.

5 years agoVersion bump.
Vladimir Homutov [Thu, 5 Nov 2020 19:37:27 +0000 (22:37 +0300)]
Version bump.

5 years agorelease-1.19.4 tag
Maxim Dounin [Tue, 27 Oct 2020 15:09:20 +0000 (18:09 +0300)]
release-1.19.4 tag

5 years agonginx-1.19.4-RELEASE release-1.19.4
Maxim Dounin [Tue, 27 Oct 2020 15:09:20 +0000 (18:09 +0300)]
nginx-1.19.4-RELEASE

5 years agoSSL: ssl_reject_handshake directive (ticket #195).
Maxim Dounin [Thu, 22 Oct 2020 15:02:28 +0000 (18:02 +0300)]
SSL: ssl_reject_handshake directive (ticket #195).

In some cases it might be needed to reject SSL handshake based on SNI
server name provided, for example, to make sure an invalid certificate
is not returned to clients trying to contact a name-based virtual server
without SSL configured.  Previously, a "ssl_ciphers aNULL;" was used for
this.  This workaround, however, is not compatible with TLSv1.3, in
particular, when using BoringSSL, where it is not possible to configure
TLSv1.3 ciphers at all.

With this change, the ssl_reject_handshake directive is introduced,
which instructs nginx to reject SSL handshakes with an "unrecognized_name"
alert in a particular server block.

For example, to reject handshake with names other than example.com,
one can use the following configuration:

    server {
        listen 443 ssl;
        ssl_reject_handshake on;
    }

    server {
        listen 443 ssl;
        server_name example.com;
        ssl_certificate example.com.crt;
        ssl_certificate_key example.com.key;
    }

The following configuration can be used to reject all SSL handshakes
without SNI server name provided:

    server {
        listen 443 ssl;
        ssl_reject_handshake on;
    }

    server {
        listen 443 ssl;
        server_name ~^;
        ssl_certificate example.crt;
        ssl_certificate_key example.key;
    }

Additionally, the ssl_reject_handshake directive makes configuring
certificates for the default server block optional.  If no certificates
are configured in the default server for a given listening socket,
certificates must be defined in all non-default server blocks with
the listening socket in question.

5 years agoStream: proxy_ssl_conf_command directive.
Maxim Dounin [Thu, 22 Oct 2020 15:00:27 +0000 (18:00 +0300)]
Stream: proxy_ssl_conf_command directive.

Similarly to ssl_conf_command, proxy_ssl_conf_command can be used to
set arbitrary OpenSSL configuration parameters as long as nginx is
compiled with OpenSSL 1.0.2 or later, when connecting to upstream
servers with SSL.  Full list of available configuration commands
can be found in the SSL_CONF_cmd manual page
(https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html).

5 years agoUpstream: proxy_ssl_conf_command and friends.
Maxim Dounin [Thu, 22 Oct 2020 15:00:23 +0000 (18:00 +0300)]
Upstream: proxy_ssl_conf_command and friends.

Similarly to ssl_conf_command, proxy_ssl_conf_command (grpc_ssl_conf_command,
uwsgi_ssl_conf_command) can be used to set arbitrary OpenSSL configuration
parameters as long as nginx is compiled with OpenSSL 1.0.2 or later,
when connecting to upstream servers with SSL.  Full list of available
configuration commands can be found in the SSL_CONF_cmd manual page
(https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html).

5 years agoSSL: ssl_conf_command directive.
Maxim Dounin [Thu, 22 Oct 2020 15:00:22 +0000 (18:00 +0300)]
SSL: ssl_conf_command directive.

With the ssl_conf_command directive it is now possible to set
arbitrary OpenSSL configuration parameters as long as nginx is compiled
with OpenSSL 1.0.2 or later.  Full list of available configuration
commands can be found in the SSL_CONF_cmd manual page
(https://www.openssl.org/docs/man1.1.1/man3/SSL_CONF_cmd.html).

In particular, this allows configuring PrioritizeChaCha option
(ticket #1445):

    ssl_conf_command Options PrioritizeChaCha;

It can be also used to configure TLSv1.3 ciphers in OpenSSL,
which fails to configure them via the SSL_CTX_set_cipher_list()
interface (ticket #1529):

    ssl_conf_command Ciphersuites TLS_CHACHA20_POLY1305_SHA256;

Configuration commands are applied after nginx own configuration
for SSL, so they can be used to override anything set by nginx.
Note though that configuring OpenSSL directly with ssl_conf_command
might result in a behaviour nginx does not expect, and should be
done with care.

5 years agoCore: ngx_conf_set_keyval_slot() now accepts NGX_CONF_UNSET_PTR.
Maxim Dounin [Thu, 22 Oct 2020 15:00:20 +0000 (18:00 +0300)]
Core: ngx_conf_set_keyval_slot() now accepts NGX_CONF_UNSET_PTR.

With this change, it is now possible to use ngx_conf_merge_ptr_value()
to merge keyval arrays.  This change actually follows much earlier
changes in ngx_conf_merge_ptr_value() and ngx_conf_set_str_array_slot()
in 1452:cd586e963db0 (0.6.10) and 1701:40d004d95d88 (0.6.22).

To preserve compatibility with existing 3rd party modules, both NULL
and NGX_CONF_UNSET_PTR are accepted for now.

5 years agoCpp test: added stream.
Ruslan Ermilov [Tue, 13 Oct 2020 04:44:09 +0000 (07:44 +0300)]
Cpp test: added stream.

5 years agoLimit req: unlocking of nodes on complex value errors.
Maxim Dounin [Thu, 8 Oct 2020 14:44:34 +0000 (17:44 +0300)]
Limit req: unlocking of nodes on complex value errors.

Previously, if there were multiple limits configured, errors in
ngx_http_complex_value() during processing of a non-first limit
resulted in reference count leak in shared memory nodes of already
processed limits.  Fix is to explicity unlock relevant nodes, much
like we do when rejecting requests.

5 years agoMail: proxy_smtp_auth directive.
Maxim Dounin [Sat, 3 Oct 2020 18:04:57 +0000 (21:04 +0300)]
Mail: proxy_smtp_auth directive.

The proxy_smtp_auth directive instructs nginx to authenticate users
on backend via the AUTH command (using the PLAIN SASL mechanism),
similar to what is normally done for IMAP and POP3.

If xclient is enabled along with proxy_smtp_auth, the XCLIENT command
won't try to send the LOGIN parameter.

5 years agoVersion bump.
Maxim Dounin [Sat, 3 Oct 2020 18:01:12 +0000 (21:01 +0300)]
Version bump.

5 years agorelease-1.19.3 tag
Maxim Dounin [Tue, 29 Sep 2020 14:32:10 +0000 (17:32 +0300)]
release-1.19.3 tag

5 years agonginx-1.19.3-RELEASE release-1.19.3
Maxim Dounin [Tue, 29 Sep 2020 14:32:10 +0000 (17:32 +0300)]
nginx-1.19.3-RELEASE

5 years agoUpdated OpenSSL used for win32 builds.
Maxim Dounin [Tue, 29 Sep 2020 12:56:16 +0000 (15:56 +0300)]
Updated OpenSSL used for win32 builds.

5 years agoProxy: error checking for array init, missed in 7716:d6a5e14aa3e4.
Maxim Dounin [Tue, 29 Sep 2020 12:54:09 +0000 (15:54 +0300)]
Proxy: error checking for array init, missed in 7716:d6a5e14aa3e4.

Found by Coverity (CID 1467637).

5 years agoUserid: userid_flags fixup.
Maxim Dounin [Tue, 29 Sep 2020 12:52:18 +0000 (15:52 +0300)]
Userid: userid_flags fixup.

In 7717:e3e8b8234f05, the 1st bit was incorrectly used.  It shouldn't
be used for bitmask values, as it is used by NGX_CONF_BITMASK_SET.

Additionally, special value "off" added to make it possible to clear
inherited userid_flags value.

5 years agoResolver: improved error messages (ticket #2024).
Maxim Dounin [Mon, 28 Sep 2020 14:41:22 +0000 (17:41 +0300)]
Resolver: improved error messages (ticket #2024).

5 years agoUserid: userid_flags directive to set cookie flags.
Maxim Dounin [Mon, 28 Sep 2020 14:07:48 +0000 (17:07 +0300)]
Userid: userid_flags directive to set cookie flags.

5 years agoProxy: added the "proxy_cookie_flags" directive.
Ruslan Ermilov [Sun, 27 Sep 2020 20:21:11 +0000 (23:21 +0300)]
Proxy: added the "proxy_cookie_flags" directive.

5 years agoProxy: changed interface of some internal functions.
Ruslan Ermilov [Sun, 27 Sep 2020 20:21:10 +0000 (23:21 +0300)]
Proxy: changed interface of some internal functions.

This is in preparation for the next change.

Also, moved optimization from ngx_http_proxy_rewrite_regex_handler()
to ngx_http_proxy_rewrite().

5 years agoProxy: strengthen syntax checking for some directives.
Ruslan Ermilov [Sun, 27 Sep 2020 20:21:09 +0000 (23:21 +0300)]
Proxy: strengthen syntax checking for some directives.

The "false" parameter of the proxy_redirect directive is deprecated.
Warning has been emitted since c2230102df6f (0.7.54).

The "off" parameter of the proxy_redirect, proxy_cookie_domain, and
proxy_cookie_path directives tells nginx not to inherit the
configuration from the previous configuration level.

Previously, after specifying the directive with the "off" parameter,
any other directives were ignored, and syntax checking was disabled.

The syntax was enforced to allow either one directive with the "off"
parameter, or several directives with other parameters.

Also, specifying "proxy_redirect default foo" no longer works like
"proxy_redirect default".

5 years agoStream: set module.
Pavel Pautov [Fri, 28 Aug 2020 21:10:54 +0000 (14:10 -0700)]
Stream: set module.

Adds 'set' directive to the stream server context.

5 years agoSSL: abort handshake on SSL_set_SSL_CTX() errors.
Sergey Kandaurov [Thu, 24 Sep 2020 12:51:29 +0000 (13:51 +0100)]
SSL: abort handshake on SSL_set_SSL_CTX() errors.

In rare cases, such as memory allocation failure, SSL_set_SSL_CTX() returns
NULL, which could mean that a different SSL configuration has not been set.
Note that this new behaviour seemingly originated in OpenSSL-1.1.0 release.

5 years agoHTTP/2: run posted requests after reading body.
Maxim Dounin [Wed, 23 Sep 2020 16:52:31 +0000 (19:52 +0300)]
HTTP/2: run posted requests after reading body.

HTTP/2 code failed to run posted requests after calling the request body
handler, and this resulted in connection hang if a subrequest was created
in the body handler and no other actions were made.

5 years agoHTTP/2: fixed segfault on DATA frames after 400 errors.
Maxim Dounin [Wed, 23 Sep 2020 16:50:49 +0000 (19:50 +0300)]
HTTP/2: fixed segfault on DATA frames after 400 errors.

If 400 errors were redirected to an upstream server using the error_page
directive, DATA frames from the client might cause segmentation fault
due to null pointer dereference.  The bug had appeared in 6989:2c4dbcd6f2e4
(1.13.0).

Fix is to skip such frames in ngx_http_v2_state_read_data() (similarly
to 7561:9f1f9d6e056a).  With the fix, behaviour of 400 errors in HTTP/2
is now similar to one in HTTP/1.x, that is, nginx doesn't try to read the
request body.

Note that proxying 400 errors, as well as other early stage errors, to
upstream servers might not be a good idea anyway.  These errors imply
that reading and processing of the request (and the request headers)
wasn't complete, and proxying of such incomplete request might lead to
various errors.

Reported by Chenglong Zhang.

5 years agoSSL: disabled shutdown when there are buffered data.
Maxim Dounin [Wed, 16 Sep 2020 15:26:25 +0000 (18:26 +0300)]
SSL: disabled shutdown when there are buffered data.

This fixes "SSL_shutdown() failed (SSL: ... bad write retry)" errors
as observed on the second SSL_shutdown() call after SSL shutdown fixes in
09fb2135a589 (1.19.2), notably when HTTP/2 connections are closed due
to read timeouts while there are incomplete writes.

5 years agoSSL: disabled shutdown after connection errors.
Maxim Dounin [Wed, 16 Sep 2020 15:26:24 +0000 (18:26 +0300)]
SSL: disabled shutdown after connection errors.

This fixes "SSL_shutdown() failed (SSL: ... bad write retry)" errors
as observed on the second SSL_shutdown() call after SSL shutdown fixes in
09fb2135a589 (1.19.2), notably when sending fails in ngx_http_test_expect(),
similarly to ticket #1194.

Note that there are some places where c->error is misused to prevent
further output, such as ngx_http_v2_finalize_connection() if there
are pending streams, or in filter finalization.  These places seem
to be extreme enough to don't care about missing shutdown though.
For example, filter finalization currently prevents keepalive from
being used.

5 years agoSSL: fixed event handling during shutdown.
Maxim Dounin [Wed, 16 Sep 2020 15:26:23 +0000 (18:26 +0300)]
SSL: fixed event handling during shutdown.

The c->read->ready and c->write->ready flags need to be cleared to ensure
that appropriate read or write events will be reported by kernel.  Without
this, SSL shutdown might wait till the timeout after blocking on writing
or reading even if there is a socket activity.

5 years agoSSL: workaround for incorrect SSL_write() errors in OpenSSL 1.1.1.
Maxim Dounin [Wed, 16 Sep 2020 15:26:22 +0000 (18:26 +0300)]
SSL: workaround for incorrect SSL_write() errors in OpenSSL 1.1.1.

OpenSSL 1.1.1 fails to return SSL_ERROR_SYSCALL if an error happens
during SSL_write() after close_notify alert from the peer, and returns
SSL_ERROR_ZERO_RETURN instead.  Broken by this commit, which removes
the "i == 0" check around the SSL_RECEIVED_SHUTDOWN one:

https://git.openssl.org/?p=openssl.git;a=commitdiff;h=8051ab2

In particular, if a client closed the connection without reading
the response but with properly sent close_notify alert, this resulted in
unexpected "SSL_write() failed while ..." critical log message instead
of correct "SSL_write() failed (32: Broken pipe)" at the info level.

Since SSL_ERROR_ZERO_RETURN cannot be legitimately returned after
SSL_write(), the fix is to convert all SSL_ERROR_ZERO_RETURN errors
after SSL_write() to SSL_ERROR_SYSCALL.

5 years agoCache: keep c->body_start when Vary changes (ticket #2029).
Sergey Kandaurov [Wed, 9 Sep 2020 16:26:27 +0000 (19:26 +0300)]
Cache: keep c->body_start when Vary changes (ticket #2029).

If the variant hash doesn't match one we used as a secondary cache key,
we switch back to the original key.  In this case, c->body_start was kept
updated from an existing cache node overwriting the new response value.
After file cache update, it led to discrepancy between a cache node and
cache file seen as critical errors "file cache .. has too long header".