--------------------------------------------------------------------
ChangeLog for Privoxy
--------------------------------------------------------------------
+*** Version 3.0.18 Stable ***
+
+- Bug fixes:
+ - Fix a logic bug that could cause Privoxy to reuse a tainted
+ server socket.
+ It could happen for server sockets that got tainted by a
+ server-header-tagger-induced block, in which case Privoxy
+ doesn't necessarily read the whole server response.
+ If keep-alive was enabled and the request following the
+ blocked one was to the same host and using the same
+ forwarding settings, Privoxy would send it on the tainted
+ server socket.
+ While the server would simply treat it as a pipelined request,
+ Privoxy would later on fail to properly parse the server's
+ response as it would try to parse the unread data from the
+ first response as server headers for the second one.
+ Regression introduced in 3.0.17.
+ - When implying keep-alive in client_connection(), remember that the client didn't
+ Fixes a regression introduced in 3.0.13 that would cause
+ Privoxy to wait for additional client requests after
+ receiving a HTTP/1.1 request with "Connection: close" set
+ and connection sharing enabled.
+ With clients like curl which terminates the client connection
+ after detecting that the whole body has been received it doesn't
+ really matter, but with clients like FreeBSD's fetch the client
+ connection would be kept open until it timed out.
+
+ - Fix a subtle race condition between prepare_csp_for_next_request() and sweep()
+ A thread preparing itself for the next client request
+ could briefly appear to be inactive.
+ If all other threads were already using more recent files,
+ the thread could get its files swept away under its feet.
+ I've only seen it while stress testing in valgrind while
+ touching action files in a loop. It's unlikely to have
+ caused any actual problems in the real world.
+
+- General improvements:
+ - Privoxy can (re)compress buffered content before delivering
+ it to the client. Disabled by default as most users wouldn't
+ benefit from it.
+ - The +fast-redirects{check-decoded-url} action checks URL
+ segments separately.
+ If there are other parameters behind the redirect URL,
+ this makes it unnecessary to cut them of by additionally
+ using a +redirect{} pcrs command.
+ Initial patch submitted by Jamie Zawinski in #3429848.
+ - Properly deal with FEATURE_TOGGLE being disabled
+ - Adjust url_code_map[] so spaces are replaced with %20 instead of '+'
+ While '+' can be used by client's submitting form data, this is not
+ actually what Privoxy is using the lookups for. This is more of a
+ cosmetic issue and doesn't fix any actual problems I'm aware of.
+ - When compiled without FEATURE_FAST_REDIRECTS, do not silently
+ ignore +fast-redirect{} directives
+ - Added a workaround for GNU libc's strptime() reporting negative
+ year values when the parsed year is only specified with two digits.
+ On affected systems cookies with such a date would not be turned
+ into session cookies by the +session-cookies-only action.
+ Reported by Vaeinoe in #3403560
+ - When loading action sections, verify that the referenced filters exist
+ Currently missing filters only result in an error message,
+ but eventually the severity will be upgraded to fatal.
+ - Allow to bind to multiple separate addresses.
+ Patch set submitted by Petr Pisar in #3354485.
+ - Set socket_error to errno if connecting fails in rfc2553_connect_to()
+ Previously rejected direct connections could be incorrectly reported as DNS issues.
+ - Fixed bind failures with certain GNU libc versions if no non-loopback
+ IP address has been configured on the system. This is mainly an issue
+ if the system is using DHCP and Privoxy is started before the network
+ is completely configured.
+ Reported by Raphael Marichez in #3349356.
+ Additional insight from Petr Pisar.
+ - Disable filters if SDCH compression is used unless filtering is forced.
+ If SDCH was combined with a supported compression algorithm,
+ we'd previously try to decompress it, when successful apply
+ the enabled filters and ditch the Content-Encoding header
+ even though the SDCH compression wasn't removed.
+ Reported by zebul666 in #3225863.
+ - Privoxy log messages now use the ISO 8601 date format %Y-%m-%d.
+ It's only slightly longer than the old format, but contains
+ the full date including the year and allows sorting by date
+ (when grepping in multiple log files) without hassle.
+ - Make a copy of the --user value and only mess with that when splitting user and group.
+ On some operating systems modifying the value directly
+ is reflected in the output of ps and friends and can
+ be misleading.
+ Reported by zepard in #3292710.
+ - If forwarded-connect-retries is set, only retry if the we are actually
+ forwarding the request. Previously direct connections would be retried
+ as well.
+ - Fixed a small memory leak when retrying connection
+ - Remove an incorrect assertion in compile_dynamic_pcrs_job_list()
+ It could be triggered by a pcrs job with an invalid pcre
+ pattern (for example one that contains a lone quantifier).
+
+- Action file improvements:
+
+ - Moved the site-specific block pattern section below the one for the
+ generic patterns so for requests that are matched in both, the block
+ reason for the domain is shown which is usually more useful than showing
+ the one for the generic pattern.
+ - Add a (disabled) section to block various Facebook tracking URLs
+ Reported by Dan Stahlke in #3421764.
+ - Add a (disabled) section to rewrite and redirect click-tracking URLs used on news.google.com
+ Reported by Dan Stahlke in #3421755.
+ - Unblock linuxcounter.net/
+ Reported by Dan Stahlke in #3422612.
+ - Block 'www91.intel.com/' which is used by Omniture.
+ Reported by Adam Piggott in #3167370.
+ - Disable the handle-as-empty-doc-returns-ok option and mark it as deprecated.
+ Reminded by tceverling in #2790091.
+ - Add ".ivwbox.de/" to the "Cross-site user tracking" section.
+ Reported by Nettozahler in #3172525.
+ - Unblock and fast-redirect ".awin1.com/.*=http://"
+ Reported by Adam Piggott in #3170921.
+ - Block "b.collective-media.net/".
+ - Widen the Debian popcon exception to "qa.debian.org/popcon".
+ Seen in Debian's 05_default_action.dpatch by Roland Rosenfeld.
+ - Block ".gemius.pl/" which only seems to be used for user tracking.
+ Reported by johnd16 in #3002731. Additional input from Lee and movax.
+ - Disable banners-by-size filters for '.thinkgeek.com/'
+ The filter only seems to catch pictures of the inventory.
+ - Block requests for 'go.idmnet.bbelements.com/please/showit/'
+ Reported by kacperdominik in #3372959.
+ - Unblock adainitiative.org/
+ - Add a fast-redirects exception for '.googleusercontent.com/.*=cache'
+ - Add a fast-redirects exception for webcache.googleusercontent.com/
+ - Remove -prevent-compression from the fragile alias
+ It's no longer used anywhere by default and isn't
+ known to break stuff anyway.
+ - Unblock http://adassier.wordpress.com/ and http://adassier.files.wordpress.com/
+
+- Filter file improvements:
+ - Let the yahoo filter hide '.ads'
+ - Let the msn filter hide overlay ads for Facebook 'likes' in search results.
+ - Let the msn filter hide elements with the id 's_notf_div'.
+ They only seem to be used to advertise site 'enhancements'.
+ - Let the js-events filter additionally disarm setInterval()
+ Suggested by dg1727 in #3423775.
+
+- Documentation improvements:
+ - Clarify the effect of compiling Privoxy with zlib support
+ Suggested by dg1727 in #3423782.
+ - Point out that the SourceForge messaging system works
+ like a blackhole and should thus not be used
+ - Mention some of the problems one can experience when not
+ explicitly configuring an IP addresses as listen address.
+ - Explicitly mention that hostnames can be used instead of
+ IP addresses for the listen-address, that only the first
+ address returned will be used and what happens if the
+ address is invalid.
+ Requested by Calestyo in #3302213.
+
+- Log message improvements:
+ - If only the server connection is kept alive, do not pretent to wait for a new client request.
+ - Remove a superfluos log message in forget_connection()
+ - In chat(), properly report missing server responses as such instead of calling them empty
+ - In forwarded_connect(), fix a log message nobody should ever see
+ - Fix a log message in socks5_connect(), a failed write operation was logged as failed read operation
+ - Let load_one_actions_file() properly complain about a missing '{' at the beginning of the file
+ Simply stating that a line is invalid isn't particularly helpful.
+ - Do not claim to listen on a socket until we actually do.
+ Patch submitted by Petr Pisar #3354485
+ - Prevent a duplicated LOG_LEVEL_CLF message when sending out the "no-server-data" response
+ - Also log the client socket when dropping a connection.
+ - Include the destination host in the
+ 'Request ... marked for blocking. limit-connect{...} doesn't allow CONNECT ...' message
+ Patch submitted by Saperski in #3296250.
+ - Prevent a duplicated log message if none of the resolved IP
+ addresses were reachable
+ - In connect_to(), do not pretend to retry if forwarded-connect-retries is zero or unset.
+ - When a specified user or group can't be found, put the name in single-quotes when logging it.
+ - In rfc2553_connect_to(), explain getnameinfo() errors differently.
+ - Remove a useless log message in chat()
+ - When retrying to connect, also log the maximum number of connection attempts
+ - Rephrase a log message in compile_dynamic_pcrs_job_list()
+ Divide the error code and its meaning with a colon.
+ Call the pcrs job dynamic and not the filter. Filters may
+ contain dynamic and non-dynamic pcrs jobs at the same time.
+ Only mention the name of the filter or tagger, but don't
+ claim it's a filter when it could be a tagger.
+ - In a fatal error message in load_one_actions_file(), cover both URL and TAG patterns
+ - In pcrs_strerror(), properly report unknown positive error code values as unknown.
+ Previously they were handled like 0 (no error).
+ - In compile_dynamic_pcrs_job_list(), also log the actual error code as
+ pcrs_strerror() doesn't handle all errors reported by pcre
+ - Don't bother trying to continue chatting if the client didn't ask for it.
+ Reduces log noise a bit.
+ - Make two fatal error message in load_one_actions_file() more descriptive
+ - In cgi_send_user_manual(), log when rejecting a file name due to '/' or '..'
+ - In load_file(), log a message if opening a file failed
+ The CGI error message alone isn't too helpful.
+ - In connection_destination_matches(), improve two log messages to
+ help understand why the destinations don't match
+ - Rephrase a log message in serve(). Client request arrival
+ should be differentiated from closed client connections now.
+ - In serve(), log if a client connection isn't reused due to a
+ configuration file change.
+ - Let mark_server_socket_tainted() always mark the server socket tainted,
+ just don't talk about it in cases where it has no effect.
+ It doesn't change Privoxy's behaviour, but makes understanding
+ the log file easier.
+
+
+- Miscellaneous Privoxy improvements:
+ - In get_last_url(), do not bother trying to decode URLs that do
+ not contain at least one '%' sign. It reduces the log noise and
+ a number of unnecessary memory allocations.
+ - If the --user argument user[.group] contains a dot,
+ always bail out if no group has been specified.
+ Previously the intended, but undocumented (and apparently
+ untested), behaviour was to try interpreting the whole
+ argument as user name, but the detection was flawed and
+ checked for '0' isntead of '\0', thus merely preventing
+ group names beginning with a zero.
+ - Simplify the signal setup in main()
+ - Streamline socks5_connect() slightly
+ - In case of SOCKS5 failures, dump the socks response
+ - In socks5_connect(), require a complete socks response from the server
+ Previously we didn't care how much data the server response
+ contained as long as the first two bytes contained the expected
+ values.
+ While at it, shrink the buffer size so we can't read more
+ than a whole socks response. This is required to support
+ Tor's optimistic data extension.
+ - In chat(), do not bother to generate a client request in case of direct CONNECT requests
+ - Reduce server_last_modified()'s stack size
+ - Shorten get_http_time() by using strftime()
+ - Constify the known_http_methods pointers in unknown_method()
+ - Constify the time_formats pointers in parse_header_time()
+ - Constify the formerly_valid_actions pointers in action_used_to_be_valid()
+ - In html_code_map[], use a numeric character reference instead of '
+ which wasn't standardized before XHTML 1.0
+ - Introduce a MAN_PAGE variable that defaults to privoxy.1.
+ The Debian package uses section 8 for the man page and this should simplify the patch.
+ - Deduplicate the INADDR_NONE definition for Solaris by moving it to jbsockets.h
+ - In block_url(), ditch the obsolete workaround for ancient Netscape versions
+ that supposedly couldn't properly deal with status code 403.
+ - Remove a useless NULL pointer check in load_trustfile()
+ - Remove two useless NULL pointer checks in load_one_re_filterfile().
+ - Change url_code_map[] from an array of pointers to an array of arrays
+ It removes an unnecessary layer of indirection and on
+ 64bit system reduces the size of the binary a bit.
+ - Fix various typos.
+ Fixes taken from Debian's 29_typos.dpatch by Roland Rosenfeld.
+ - Add a dok-tidy GNUMakefile target to clean up the messy HTML
+ generated by the other dok targets.
+ - GNUisms in the GNUMakefile have been removed.
+ - Change the HTTP version in static responses to 1.1
+ - Synced config.sub and config.guess with upstream
+ 2011-11-11/386c7218162c145f5f9e1ff7f558a3fbb66c37c5.
+ - Add a dedicated function to parse the values of toggles
+ Reduces duplicated code in load_config() and provides
+ better error handling. Invalid or missing toggle values
+ are now a fatal error instead of being silently ignored.
+ - Terminate HTML lines in static error messages with \n instead of \r\n.
+ - Simplify cgi_error_unknown() a bit.
+ - In LogPutString(), don't bother looking at pszText when not actually logging anything
+ - Change ssplit()'s fourth parameter from int to size_t.
+ Fixes a clang complaint.
+ - Add a warning that the statistics currently can't be trusted.
+ Mention Privoxy-Log-Parser's --statistics option as
+ an alternative for the time being.
+ - In rfc2553_connect_to(), start setting cgi->error_message on error
+ - Change the expected status code returned for http://p.p/die depending
+ on whether or not FEATURE_GRACEFUL_TERMINATION is available.
+ - In cgi_die(), mark the client connection for closing.
+ If the client will fetch the style sheet through another connection
+ it gets the main thread out of the accept() state and should thus
+ trigger the actual shutdown.
+ - Add a proper CGI message for cgi_die().
+ - Fix an invalid free when compiled with FEATURE_GRACEFUL_TERMINATION
+ and shut down through http://config.privoxy.org/die
+ - Don't enforce a logical line length limit in read_config_line()
+ - Slightly refactor server_last_modified() to remove useless gmtime*() calls
+ - In get_content_type(), also recognize '.jpeg' as JPEG extension
+ - Add '.png' to the list of recognized file extenstions in get_content_type()
+ - In block_url(), consistently use the block reason "Request blocked by Privoxy"
+ In two places the reason was "Request for blocked URL" which
+ hides the fact that the request got blocked by Privoxy and
+ isn't necessarly correct as the block may be due to tags.
+ - In get_actions(), fix the "temporary" backwards compatibility hack
+ to accept block actions without reason.
+ It also covered other actions that should be rejected as invalid.
+ Reported by Billy Crook.
+ - In listen_loop(), reload the configuration files after accepting
+ a new connection instead of before.
+ Previously the first connection that arrived after a configuration
+ change would still be handled with the old configuration.
+ - In chat()'s receive-data loop, skip a client socket check if
+ the socket will be written to right away anyway. This can
+ increase the transfer speed for unfiltered content on fast
+ network connections.
+ - The socket timeout is used for SOCKS negotiation as well.
+ - Don't keep the client connection alive if any configuration file
+ changed since the time the connection came in.
+ This is closer to Privoxy's behaviour before keep-alive support
+ for client connection has been added and also less confusing in
+ general.
+ - Treat all Content-Type header values containing the pattern
+ 'script' as a sign of text. Reported by pribog in #3134970.
+
+
+- configure:
+ - Added a --disable-ipv6-support switch for platforms where support
+ is detected but doesn't actually work.
+ - Do not check for the existence of strerror() and memmove() twice
+ - Remove a useless test for setpgrp(2). Privoxy doesn't
+ need it and it can cause problems when cross-compiling
+ - Rename the --disable-acl-files switch to --disable-acl-support
+ Since about 2001, ACL directives are specified in the standard config file.
+ - Update the URL of the 'Removing outdated PCRE version after the
+ next stable release' posting.
+ The old URL stopped working after one of SF's recent layout pessimizations.
+ Reported by Han Liu.
+
+
+- Privoxy-Regression-Test:
+ - Added --shuffle-tests option to increase the chances of detection race conditions
+ - Added a --local-test-file option that allows to use Privoxy-Regression-Test without Privoxy
+ - Added tests for missing socks4 and socks4a forwarders
+ - The --privoxy-address option now works with IPv6 addresses
+ containing brackets, too
+ - Perform limited sanity checks for parameters that are supposed
+ to have numerical values.
+ - Added a --sleep-time option to specify a number of seconds to
+ sleep between tests, defaults to 0.
+ - Disable the range-requests tagger for tests that break if it's enabled
+ - Log messages use the ISO 8601 date format %Y-%m-%d.
+ - Fix spelling in two error messages.
+ - In the --help output, include a list of supported tests and their default levels.
+
+
+- Privoxy-Log-Parser:
+ - Perform limited sanity checks for parameters that are supposed
+ to have numerical values.
+ - Implement a --unbreak-lines-only option to try to revert MUA breakage.
+ - Accept and highlight: Added header: Content-Encoding: deflate
+ - Accept and highlight: Compressed content from 29258 to 8630 bytes.
+ - Accept and highlight: Client request arrived in time on socket 21.
+ - Highlight: Didn't receive data in time: a.fsdn.com:443
+ - Accept log messages with ISO 8601 time stamps, too
+
+
+- uagen:
+ - Bump generated Firefox version to 9.0
+ - Only randomize the release date if the new --randomize-release-date option is enabled.
+ Firefox versions after 4 use a fixed date string without meaning.
+
+
*** Version 3.0.17 Stable ***
- Fixed last-chunk-detection for responses where the body was small