While ngx_get_full_name() might have a bit more descriptive arguments,
the ngx_conf_full_name() is generally easier to use when parsing
configuration and limits exposure of cycle->prefix / cycle->conf_prefix
details.
SPDY: alert about activated fake events instead of deleting them.
They refer to the same socket descriptor as our real connection, and
deleting them will stop processing of the connection.
Events of fake connections must not be activated, and if it happened there
is nothing we can do. The whole processing should be terminated as soon as
possible, but it is not obvious how to do this safely.
SPDY: do not reject headers with empty value (ticket #396).
A quote from SPDY draft 2 specification: "The length of each name and
value must be greater than zero. A receiver of a zero-length name or
value must send a RST_STREAM with code PROTOCOL error."
But it appears that Chrome browser allows sending requests over SPDY/2
connection using JavaScript that contain headers with empty values.
For better compatibility across SPDY clients and to be compliant with
HTTP, such headers are no longer rejected.
Also, it is worth noting that in SPDY draft 3 the statement has been
changed so that it permits empty values for headers.
Unbreak building on Linux without sendfile64() support.
It was broken in 8e446a2daf48 when the NGX_SENDFILE_LIMIT constant was added
to ngx_linux_sendfile_chain.c having the same name as already defined one in
ngx_linux_config.h.
The newer is needed to overcome a bug in old Linux kernels by limiting the
number of bytes to send per sendfile() syscall. The older is used with
sendfile() on ancient kernels that works with 32-bit offsets only.
Sergey Kandaurov [Tue, 13 Aug 2013 13:47:04 +0000 (17:47 +0400)]
Referer module: fixed regex matching against HTTPS referers.
When matching a compiled regex against value in the "Referer" header field,
the length was calculated incorrectly for strings that start from "https://".
This might cause matching to fail for regexes with end-of-line anchors.
Gleb Smirnoff [Thu, 8 Aug 2013 11:06:39 +0000 (15:06 +0400)]
Don't lose pointer to first nonempty buf in ngx_*_sendfile_chain().
In ngx_*_sendfile_chain() when calculating pointer to a first
non-zero sized buf, use "in" as iterator. This fixes processing
of zero sized buf(s) after EINTR. Otherwise function can return
zero sized buf to caller, and later ngx_http_write_filter()
logs warning.
a) ssl as listen parameter is preferable.
b) ssl_protocols defaults are better because they do not forbid TLS versions
1.1 and 1.2.
c) ssl_session_timeout has sense only with SSL cache.
Fixed memory leaks in the root and auth_basic_user_file directives.
If a relative path is set by variables, then the ngx_conf_full_name()
function was called while processing requests, which causes allocations
from the cycle pool.
A new function that takes pool as an argument was introduced.
Ruslan Ermilov [Mon, 5 Aug 2013 06:55:59 +0000 (10:55 +0400)]
Core: only resolve address families configured on the local system.
This is done by passing AI_ADDRCONFIG to getaddrinfo().
On Linux, setting net.ipv6.conf.all.disable_ipv6 to 1 will now be
respected.
On FreeBSD, AI_ADDRCONFIG filtering is currently implemented by
attempting to create a datagram socket for the corresponding family,
which succeeds even if the system doesn't in fact have any addresses
of that family configured. That is, if the system with IPv6 support
in the kernel doesn't have IPv6 addresses configured, AI_ADDRCONFIG
will filter out IPv6 only inside a jail without IPv6 addresses or
with IPv6 disabled.
Though there are several MIME types commonly used for JavaScript nowadays,
the most common being "text/javascript", "application/javascript", and
currently used by nginx "application/x-javascript", RFC 4329 prefers
"application/javascript".
The "charset_types" directive's default value was adjusted accordingly.
Previously, after sending a header we always sent a last buffer and
finalized a request with code 0, even in case of errors. In some cases
this resulted in a loss of ability to detect the response wasn't complete
(e.g. if Content-Length was removed from a response by gzip filter).
This change tries to propogate to a client information that a response
isn't complete in such cases. In particular, with this change we no longer
pretend a returned response is complete if we wasn't able to create
a temporary file.
If an error code suggests the error wasn't fatal, we flush buffered data
and disable keepalive, then finalize request normally. This allows to to
propogate information about a problem to a client, while still sending all
the data we've got from an upstream.
No semantic changes expected, though some checks are done differently.
In particular, the r->cached flag is no longer explicitly checked. Instead,
we relay on u->header_sent not being set if a response is sent from
a cache.
Upstream: NGX_HTTP_CLIENT_CLOSED_REQUEST no longer reset to 0.
The NGX_HTTP_CLIENT_CLOSED_REQUEST code is allowed to happen after we
started sending a response (much like NGX_HTTP_REQUEST_TIME_OUT), so there
is no need to reset response code to 0 in this case.
Checks were added to both buffered and unbuffered code paths to detect
and complain if a response is incomplete. Appropriate error codes are
now passed to ngx_http_upstream_finalize_request().
With this change in unbuffered mode we now use u->length set to -1 as an
indicator that EOF is allowed per protocol and used to indicate response
end (much like its with p->length in buffered mode). Proxy module was
changed to set u->length to 1 (instead of previously used -1) in case of
chunked transfer encoding used to comply with the above.
Upstream: u->length now defaults to -1 (API change).
That is, by default we assume that response end is signalled by
a connection close. This seems to be better default, and in line
with u->pipe->length behaviour.
Upstream: fixed store/cache of unfinished responses.
In case of upstream eof, only responses with u->pipe->length == -1
are now cached/stored. This ensures that unfinished chunked responses
are not cached.
Note well - previously used checks for u->headers_in.content_length_n are
preserved. This provides an additional level of protection if protol data
disagree with Content-Length header provided (e.g., a FastCGI response
is sent with wrong Content-Length, or an incomple SCGI or uwsgi response),
as well as protects from storing of responses to HEAD requests. This should
be reconsidered if we'll consider caching of responses to HEAD requests.
Upstream: NGX_HTTP_GATEWAY_TIME_OUT after upstream timeouts.
There is no real difference from previously used 0 as NGX_HTTP_* will
become 0 in ngx_http_upstream_finalize_request(), but the change
preserves information about a timeout a bit longer. Previous use of
ETIMEDOUT in one place was just wrong.
Note well that with cacheable responses there will be a difference
(code in ngx_http_upstream_finalize_request() will store the error
in cache), though this change doesn't touch cacheable case.
Upstream: ngx_http_upstream_finalize_request(NGX_ERROR) on errors.
Previously, ngx_http_upstream_finalize_request(0) was used in most
cases after errors. While with current code there is no difference,
use of NGX_ERROR allows to pass a bit more information into
ngx_http_upstream_finalize_request().
Upstream: consistent error handling after u->input_filter_init().
In all cases ngx_http_upstream_finalize_request() with NGX_ERROR now used.
Previously used NGX_HTTP_INTERNAL_SERVER_ERROR in the subrequest in memory
case don't cause any harm, but inconsistent with other uses.
Event pipe: fixed writing cache header to a temp file.
With previous code the p->temp_file->offset wasn't adjusted if a temp
file was written by the code in ngx_event_pipe_write_to_downstream()
after an EOF, resulting in cache not being used with empty scgi and uwsgi
responses with Content-Length set to 0.
Fix it to call ngx_event_pipe_write_chain_to_temp_file() there instead
of calling ngx_write_chain_to_temp_file() directly.
Maxim Dounin [Fri, 14 Jun 2013 16:56:07 +0000 (20:56 +0400)]
Fixed ngx_http_test_reading() to finalize request properly.
Previous code called ngx_http_finalize_request() with rc = 0. This is
ok if a response status was already set, but resulted in "000" being
logged if it wasn't. In particular this happened with limit_req
if a connection was prematurely closed during limit_req delay.
After a failed partial match we now check if there is another partial
match in previously matched substring to fix cases like "aab" in "aaab".
The ctx->saved string is now always sent if it's present on return
from the ngx_http_sub_parse() function (and reset accordingly). This
allows to release parts of previously matched data.
Sub filter: fixed incomplete last buffer on partial match.
If a pattern was partially matched at a response end, partially matched
string wasn't send. E.g., a response "fo" was truncated to an empty response
if partially mathed by a pattern "foo".
SPDY: fixed segfault with "client_body_in_file_only" enabled.
It is possible to send FLAG_FIN in additional empty data frame, even if it is
known from the content-length header that request body is empty. And Firefox
actually behaves like this (see ticket #357).
To simplify code we sacrificed our microoptimization that did not work right
due to missing check in the ngx_http_spdy_state_data() function for rb->buf
set to NULL.
Xslt: exsltRegisterAll() moved to preconfiguration.
The exsltRegisterAll() needs to be called before XSLT stylesheets
are compiled, else stylesheet compilation hooks will not work. This
change fixes EXSLT Functions extension.
Events: honor NGX_USE_GREEDY_EVENT when kqueue support is enabled.
Currently this flag is needed for epoll and rtsig, and though these methods
usually present on different platforms than kqueue, nginx can be compiled to
support all of them.
The $NGX_AUTO_CONFIG_H added to perl module Makefile dependencies to
make sure it's always rebuild after a configure. It is needed as we
expand various variables used for Makefile generation during configure
(in particular, nginx version).
Vladimir Homutov [Thu, 11 Jul 2013 15:50:19 +0000 (19:50 +0400)]
Core: fixed possible use of an uninitialized variable.
The call to ngx_sock_ntop() in ngx_connection_local_sockaddr() might be
performed with the uninitialized "len" variable. The fix is to initialize
variable to the size of corresponding socket address type.
Vladimir Homutov [Thu, 11 Jul 2013 12:07:25 +0000 (16:07 +0400)]
Core: extended ngx_sock_ntop() with socklen parameter.
On Linux, sockaddr length is required to process unix socket addresses properly
due to unnamed sockets (which don't have sun_path set at all) and abstract
namespace sockets.
Minimal data length we expect for further calls was calculated incorrectly
if parsing stopped right after parsing chunk size. This might in theory
affect clients and/or backends using LF instead of CRLF.
Maxim Dounin [Wed, 26 Jun 2013 11:47:27 +0000 (15:47 +0400)]
Fixed build on Linux with x32 ABI.
On Linux x32 inclusion of sys/sysctl.h produces an error. As sysctl() is
only used by rtsig event method code, which is legacy and not compiled
in by default on modern linuxes, the sys/sysctl.h file now only included
if rtsig support is enabled.
Vladimir Homutov [Thu, 20 Jun 2013 16:47:39 +0000 (20:47 +0400)]
Core: support several "error_log" directives.
When several "error_log" directives are specified in the same configuration
block, logs are written to all files with a matching log level.
All logs are stored in the singly-linked list that is sorted by log level in
the descending order.
Specific debug levels (NGX_LOG_DEBUG_HTTP,EVENT, etc.) are not supported
if several "error_log" directives are specified. In this case all logs
will use debug level that has largest absolute value.
Vladimir Homutov [Thu, 13 Jun 2013 10:50:10 +0000 (14:50 +0400)]
Core: moved initialization of log level.
The cycle->new_log->log_level should only be initialized by ngx_init_cycle()
if no error logs were found in the configuration. This move allows to get rid
of extra initialization in ngx_error_log().
Maxim Dounin [Wed, 5 Jun 2013 15:44:22 +0000 (19:44 +0400)]
Valgrind: supressed complaints about uninitialized bytes.
Valgrind complains if we pass uninitialized memory to a syscall:
==36492== Syscall param sendmsg(msg.msg_iov[0]) points to uninitialised byte(s)
==36492== at 0x6B5E6A: sendmsg (in /usr/lib/system/libsystem_kernel.dylib)
==36492== by 0x10004288E: ngx_signal_worker_processes (ngx_process_cycle.c:527)
==36492== by 0x1000417A7: ngx_master_process_cycle (ngx_process_cycle.c:203)
==36492== by 0x100001F10: main (nginx.c:410)
==36492== Address 0x7fff5fbff71c is on thread 1's stack
Even initialization of all members of the structure passed isn't enough, as
there is padding which still remains uninitialized and results in Valgrind
complaint. Note there is no real problem here as data from uninitialized
memory isn't used.
Maxim Dounin [Wed, 5 Jun 2013 15:44:20 +0000 (19:44 +0400)]
Valgrind: sigaction() failure now ignored.
Valgrind intercepts SIGUSR2 in some cases, and nginx might not be able to
start due to sigaction() failure. If compiled with NGX_VALGRIND defined,
we now ignore the failure of sigaction().
If "stderr" was specified in one of the "error_log" directives,
stderr is not redirected to the first error_log on startup,
configuration reload, and reopening log files.
Core: fixed stderr redirection on win32 in ngx_reopen_files().
On win32 stderr was not redirected into a file specified by "error_log"
while reopening files. Fix is to use platform-independent functions to
work with stderr, as already used by ngx_init_cycle() and main() since
rev. d8316f307b6a.
Maxim Dounin [Fri, 31 May 2013 10:59:26 +0000 (14:59 +0400)]
Win32: accept_mutex now always disabled (ticket #362).
Use of accept mutex on win32 may result in a deadlock if there are multiple
worker_processes configured and the mutex is grabbed by a process which
can't accept connections.
Maxim Dounin [Mon, 27 May 2013 12:54:09 +0000 (16:54 +0400)]
Upstream: http_403 support in proxy_next_upstream (and friends).
The parameter is mostly identical to http_404, and is expected to
be used in similar situations. The 403 code might be returned by
a backend instead of 404 on initial sync of new directories with rsync.
See here for feature request and additional details:
http://mailman.nginx.org/pipermail/nginx-ru/2013-April/050920.html
Maxim Dounin [Fri, 24 May 2013 18:27:23 +0000 (22:27 +0400)]
SSI: ssi_last_modified directive.
The "ssi_last_modified" directive allows to preserve Last-Modified header
in SSI responses. The directive is similar to SSILastModified one available
in Apache:
Ruslan Ermilov [Thu, 23 May 2013 12:26:10 +0000 (16:26 +0400)]
Memcached: stricten header validation.
An invalid memcached reply that started with '\n' could cause
segmentation fault.
An invalid memcached reply "VALUE / 0 2\r?ok\r\nEND\r\n" was
considered as a valid response.
In addition, if memcached reports that the key was not found,
set u->headers_in.content_length_n to 0. This ensures that
ngx_http_memcached_filter() will not be called while previous
code relied on always intercepting 404. Initialization of
ctx->rest was moved to where it belongs.
Maxim Dounin [Tue, 21 May 2013 17:47:50 +0000 (21:47 +0400)]
Upstream: fixed fail_timeout and max_fails > 1.
Due to peer->checked always set since rev. c90801720a0c (1.3.0)
by round-robin and least_conn balancers (ip_hash not affected),
the code in ngx_http_upstream_free_round_robin_peer() function
incorrectly reset peer->fails too often.
Reported by Dmitry Popov,
http://mailman.nginx.org/pipermail/nginx-devel/2013-May/003720.html