| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
| |
The SF_NOCACHE flag, introduced in FreeBSD 11 along with the new non-blocking
sendfile() implementation by glebius@, makes it possible to use sendfile()
along with the "directio" directive.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Starting with FreeBSD 11, there is no need to use AIO operations to preload
data into cache for sendfile(SF_NODISKIO) to work. Instead, sendfile()
handles non-blocking loading data from disk by itself. It still can, however,
return EBUSY if a page is already being loaded (for example, by a different
process). If this happens, we now post an event for the next event loop
iteration, so sendfile() is retried "after a short period", as manpage
recommends.
The limit of the number of EBUSY tolerated without any progress is preserved,
but now it does not result in an alert, since on an idle system event loop
iteration might be very short and EBUSY can happen many times in a row.
Instead, SF_NODISKIO is simply disabled for one call once the limit is
reached.
With this change, sendfile(SF_NODISKIO) is now used automatically as long as
sendfile() is enabled, and no longer requires "aio on;".
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With sendfile in threads, "task already active" alerts might appear in logs
if a write event happens on the main HTTP/2 connection, triggering a sendfile
in threads while another thread operation is already running. Observed
with "aio threads; aio_write on; sendfile on;" and with thread event handlers
modified to post a write event to the main HTTP/2 connection (though can
happen without any modifications).
Similarly, sendfile() with AIO preloading on FreeBSD can trigger duplicate
aio operation, resulting in "second aio post" alerts. This is, however,
harder to reproduce, especially on modern FreeBSD systems, since sendfile()
usually does not return EBUSY.
Fix is to avoid starting a sendfile operation if other thread operation
is active by checking r->aio in the thread handler (and, similarly, in
aio preload handler). The added check also makes duplicate calls protection
redundant, so it is removed.
|
| |
|
|
| |
No functional changes.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The ngx_chain_coalesce_file() function may produce more bytes to send then
requested in the limit passed, as it aligns the last file position
to send to memory page boundary. As a result, (limit - send) may become
negative. This resulted in big positive number when converted to size_t
while calling ngx_output_chain_to_iovec().
Another part of the problem is in ngx_chain_coalesce_file(): it changes cl
to the next chain link even if the current buffer is only partially sent
due to limit.
Therefore, if a file buffer was not expected to be fully sent due to limit,
and was followed by a memory buffer, nginx called sendfile() with a part
of the file buffer, and the memory buffer in trailer. If there were enough
room in the socket buffer, this resulted in a part of the file buffer being
skipped, and corresponding part of the memory buffer sent instead.
The bug was introduced in 8e903522c17a (1.7.8). Configurations affected
are ones using limits, that is, limit_rate and/or sendfile_max_chunk, and
memory buffers after file ones (may happen when using subrequests or
with proxying with disk buffering).
Fix is to explicitly check if (send < limit) before constructing trailer
with ngx_output_chain_to_iovec(). Additionally, ngx_chain_coalesce_file()
was modified to preserve unfinished file buffers in cl.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If sendfile in threads is used, it is possible that multiple
subrequests will trigger multiple ngx_linux_sendfile_thread() calls,
as operations are only serialized in output chain based on r->aio,
that is, on subrequest level.
This resulted in "task #N already active" alerts, in particular, when
running proxy_store.t with "aio threads; sendfile on;".
Fix is to tolerate duplicate calls, with an additional safety check
that the file is the same as previously used.
The same problem also affects "aio on; sendfile on;" on FreeBSD
(previously known as "aio sendfile;"), where aio->preload_handler()
could be called multiple times due to similar reasons, resulting in
"second aio post" alerts. Fix is the same as well.
It is also believed that similar problems can arise if a filter
calls the next body filter multiple times for some reason. These are
mostly theoretical though.
|
| | |
|
| |
|
|
|
|
|
|
| |
This reduces layering violation and simplifies the logic of AIO preread, since
it's now triggered by the send chain function itself without falling back to
the copy filter. The context of AIO operation is now stored per file buffer,
which makes it possible to properly handle cases when multiple buffers come
from different locations, each with its own configuration.
|
| |
|
|
|
| |
This reduces code duplication and unifies debug logging of the writev() syscall
among various send chain functions.
|
| |
|
|
| |
No functional changes.
|
| | |
|
| |
|
|
|
| |
It deduplicates code of the send chain functions and uses only preallocated
memory, which completely solves the problem mentioned in d1bde5c3c5d2.
|
| |
|
|
| |
No functional changes.
|
| |
|
|
|
|
| |
Previously, a value of the "send" variable wasn't properly adjusted
in a rare case when syscall was interrupted by a signal. As a result,
these functions could send less data than the limit allows.
|
| |
|
|
| |
No functional changes.
|
| |
|
|
|
| |
It was made redundant by the previous change, since the "sent" variable
is no longer modified.
|
| | |
|
| | |
|
| |
|
|
| |
Signed-off-by: Piotr Sikora <piotr@cloudflare.com>
|
| |
|
|
|
|
|
|
| |
In ngx_*_sendfile_chain() when calculating pointer to a first
non-zero sized buf, use "in" as iterator. This fixes processing
of zero sized buf(s) after EINTR. Otherwise function can return
zero sized buf to caller, and later ngx_http_write_filter()
logs warning.
|
| |
|
|
|
|
|
|
| |
We now stop on IOV_MAX iovec entries only if we are going to add new one,
i.e. next buffer can't be coalesced into last iovec.
This also fixes incorrect checks for trailer creation on FreeBSD and
Mac OS X, header.nelts was checked instead of trailer.nelts.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
FreeBSD kernel checks headers/trailers pointer against NULL, not
corresponding count. Passing NULL if there are no headers/trailers
helps to avoid unneeded work in kernel, as well as unexpected 0 bytes
GIO in traces.
|
| |
|
|
| |
Fixed nearby grammar errors.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
for 2G+ files
|
| |
|
|
|
|
|
| |
*) Bugfix: in SSI handling.
*) Bugfix: the ngx_http_memcached_module did not support the keys in
the "/usr?args" form.
|
| |
|
|
|
|
|
|
|
|
| |
*) Feature: the new 444 code of the "return" directive to close
connection.
*) Feature: the "so_keepalive" directive in IMAP/POP3 proxy.
*) Bugfix: if there are unclosed connection nginx now calls abort()
only on gracefull quit and active "debug_points" directive.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Security: if nginx was built with the ngx_http_realip_module and the
"satisfy_any on" directive was used, then access and authorization
directives did not work. The ngx_http_realip_module was not built
and is not built by default.
*) Change: the "$time_gmt" variable name was changed to "$time_local".
*) Change: the "proxy_header_buffer_size" and
"fastcgi_header_buffer_size" directives was renamed to the
"proxy_buffer_size" and "fastcgi_buffer_size" directives.
*) Feature: the ngx_http_memcached_module.
*) Feature: the "proxy_buffering" directive.
*) Bugfix: the changes in accept mutex handling when the "rtsig" method
was used; the bug had appeared in 0.3.0.
*) Bugfix: if the client sent the "Transfer-Encoding: chunked" header
line, then nginx returns the 411 error.
*) Bugfix: if the "auth_basic" directive was inherited from the http
level, then the realm in the "WWW-Authenticate" header line was
without the "Basic realm" text.
*) Bugfix: if the "combined" format was explicitly specified in the
"access_log" directive, then the empty lines was written to the log;
the bug had appeared in 0.3.8.
*) Bugfix: nginx did not run on the sparc platform under any OS except
Solaris.
*) Bugfix: now it is not necessary to place space between the quoted
string and closing bracket in the "if" directive.
|
| |
|
|
|
|
|
|
|
|
|
| |
*) Feature: the IMAP/POP3 proxy supports SSL.
*) Feature: the "proxy_timeout" directive of the ngx_imap_proxy_module.
*) Feature: the "userid_mark" directive.
*) Feature: the $remote_user variable value is determined independently
of authorization use.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Feature: the listen(2) backlog in the "listen" directive can be
changed using the -HUP signal.
*) Feature: the geo2nginx.pl script was added to contrib.
*) Change: the FastCGI parameters with the empty values now are passed
to a server.
*) Bugfix: the segmentation fault occurred or the worker process may
got caught in an endless loop if the proxied or FastCGI server sent
the "Cache-Control" header line and the "expires" directive was
used; in the proxied mode the the bug had appeared in 0.1.29.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Bugfix: nginx did run on Linux parisc.
*) Feature: nginx now does not start under FreeBSD if the sysctl
kern.ipc.somaxconn value is too big.
*) Bugfix: if a request was internally redirected by the
ngx_http_index_module module to the ngx_http_proxy_module or
ngx_http_fastcgi_module modules, then the index file was not closed
after request completion.
*) Feature: the "proxy_pass" can be used in location with regular
expression.
*) Feature: the ngx_http_rewrite_filter_module module supports the
condition like "if ($HTTP_USER_AGENT ~ MSIE)".
*) Bugfix: nginx started too slow if the large number of addresses and
text values were used in the "geo" directive.
*) Change: a variable name must be declared as "$name" in the "geo"
directive. The previous variant without "$" is still supported, but
will be removed soon.
*) Feature: the "%{VARIABLE}v" logging parameter.
*) Feature: the "set $name value" directive.
*) Bugfix: gcc 4.0 compatibility.
*) Feature: the --with-openssl-opt=OPTIONS autoconfiguration directive.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Bugfix: if the response were transferred by chunks, then on the HEAD
request the final chunk was issued.
*) Bugfix: the "Connection: keep-alive" header were issued, even if the
keepalive_timeout directive forbade the keep-alive use.
*) Bugfix: the errors in the ngx_http_fastcgi_module caused the
segmentation faults.
*) Bugfix: the compressed response encrypted by SSL may not transferred
complete.
*) Bugfix: the TCP-specific TCP_NODELAY, TCP_NOPSUH, and TCP_CORK
options, are not used for the unix domain sockets.
*) Feature: the rewrite directive supports the arguments rewriting.
*) Bugfix: the response code 400 was returned for the POST request with
the "Content-Length: 0" header; the bug had appeared in 0.1.14.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Feature: the autoconfiguration directives:
--http-client-body-temp-path=PATH, --http-proxy-temp-path=PATH, and
--http-fastcgi-temp-path=PATH
*) Change: the directory name for the temporary files with the client
request body is specified by directive client_body_temp_path, by
default it is <prefix>/client_body_temp.
*) Feature: the ngx_http_fastcgi_module and the directives:
fastcgi_pass, fastcgi_root, fastcgi_index, fastcgi_params,
fastcgi_connect_timeout, fastcgi_send_timeout, fastcgi_read_timeout,
fastcgi_send_lowat, fastcgi_header_buffer_size, fastcgi_buffers,
fastcgi_busy_buffers_size, fastcgi_temp_path,
fastcgi_max_temp_file_size, fastcgi_temp_file_write_size,
fastcgi_next_upstream, and fastcgi_x_powered_by.
*) Bugfix: the "[alert] zero size buf" error; the bug had appeared in
0.1.3.
*) Change: the URI must be specified after the host name in the
proxy_pass directive.
*) Change: the %3F symbol in the URI was considered as the argument
string start.
*) Feature: the unix domain sockets support in the
ngx_http_proxy_module.
*) Feature: the ssl_engine and ssl_ciphers directives.
Thanks to Sergey Skvortsov for SSL-accelerator.
|
| |
|
|
|
|
|
|
|
|
|
| |
*) Feature: the server_names_hash and server_names_hash_threshold
directives.
*) Bugfix: the *.domain.tld names in the "server_name" directive did
not work.
*) Bugfix: the %request_length log parameter logged the incorrect
length.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Feature: the worker_priority directive.
*) Change: both tcp_nopush and tcp_nodelay directives affect the
transferred response.
*) Bugfix: nginx did not call initgroups().
Thanks to Andrew Sitnikov and Andrei Nigmatulin.
*) Change: now the ngx_http_autoindex_module shows the file size in the
bytes.
*) Bugfix: the ngx_http_autoindex_module returned the 500 error if the
broken symlink was in a directory.
*) Bugfix: the files bigger than 4G could not be transferred using
sendfile.
*) Bugfix: if the backend was resolved to several backends and there
was an error while the response waiting then process may got caught
in an endless loop.
*) Bugfix: the worker process may exit with the "unknown cycle" message
when the /dev/poll method was used.
*) Bugfix: "close() channel failed" errors.
*) Bugfix: the autodetection of the "nobody" and "nogroup" groups.
*) Bugfix: the send_lowat directive did not work on Linux.
*) Bugfix: the segmentation fault occurred if there was no events
section in configuration.
*) Bugfix: nginx could not be built on OpenBSD.
*) Bugfix: the double slashes in "://" in the URI were converted to
":/".
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Bugfix: the proxied request was sent without arguments if the
request contains "//", "/./", "/../" or "%XX".
*) Bugfix: the large compressed responses may be transferred not
completely.
*) Bugfix: the files bigger than 2G was not transferred on Linux that
does not support sendfile64().
*) Bugfix: while the build configuration on Linux the
--with-poll_module parameter was required; the bug had appeared in
0.1.8.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Bugfix: on Solaris and Linux there may be too many "recvmsg()
returned not enough data" alerts.
*) Bugfix: there were the "writev() failed (22: Invalid argument)"
errors on Solaris in proxy mode without sendfile. On other platforms
that do not support sendfile at all the process got caught in an
endless loop.
*) Bugfix: segmentation fault on Solaris in proxy mode and using
sendfile.
*) Bugfix: segmentation fault on Solaris.
*) Bugfix: on-line upgrade did not work on Linux.
*) Bugfix: the ngx_http_autoindex_module module did not escape the
spaces, the quotes, and the percent signs in the directory listing.
*) Change: the decrease of the copy operations.
*) Feature: the userid_p3p directive.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*) Feature: the --user=USER, --group=GROUP, and --with-ld-opt=OPTIONS
options in configure.
*) Feature: the server_name directive supports *.domain.tld.
*) Bugfix: the portability improvements.
*) Bugfix: if configuration file was set in command line, the
reconfiguration was impossible; the bug had appeared in 0.1.1.
*) Bugfix: proxy module may get caught in an endless loop when sendfile
is not used.
*) Bugfix: with sendfile the response was not recoded according to the
charset module directives; the bug had appeared in 0.1.1.
*) Bugfix: very seldom bug in the kqueue processing.
*) Bugfix: the gzip module compressed the proxied responses that was
already compressed.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
