1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
2 "http://www.w3.org/TR/html4/loose.dtd">
5 <title>Appendix</title>
6 <meta name="GENERATOR" content=
7 "Modular DocBook HTML Stylesheet Version 1.79">
8 <link rel="HOME" title="Privoxy 3.0.27 User Manual" href="index.html">
9 <link rel="PREVIOUS" title="See Also" href="seealso.html">
10 <link rel="STYLESHEET" type="text/css" href="../p_doc.css">
11 <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
12 <link rel="STYLESHEET" type="text/css" href="p_doc.css">
14 <body class="SECT1" bgcolor="#EEEEEE" text="#000000" link="#0000FF" vlink=
15 "#840084" alink="#0000FF">
16 <div class="NAVHEADER">
17 <table summary="Header navigation table" width="100%" border="0"
18 cellpadding="0" cellspacing="0">
20 <th colspan="3" align="center">Privoxy 3.0.27 User Manual</th>
23 <td width="10%" align="left" valign="bottom"><a href="seealso.html"
24 accesskey="P">Prev</a></td>
25 <td width="80%" align="center" valign="bottom"></td>
26 <td width="10%" align="right" valign="bottom"> </td>
29 <hr align="left" width="100%">
32 <h1 class="SECT1"><a name="APPENDIX" id="APPENDIX">14. Appendix</a></h1>
34 <h2 class="SECT2"><a name="REGEX" id="REGEX">14.1. Regular
36 <p><span class="APPLICATION">Privoxy</span> uses Perl-style
37 <span class="QUOTE">"regular expressions"</span> in its <a href=
38 "actions-file.html">actions files</a> and <a href=
39 "filter-file.html">filter file</a>, through the <a href=
40 "http://www.pcre.org/" target="_top">PCRE</a> and <span class=
41 "APPLICATION">PCRS</span> libraries.</p>
42 <p>If you are reading this, you probably don't understand what
43 <span class="QUOTE">"regular expressions"</span> are, or what they can
44 do. So this will be a very brief introduction only. A full explanation
45 would require a <a href="http://www.oreilly.com/catalog/regex/" target=
46 "_top">book</a> ;-)</p>
47 <p>Regular expressions provide a language to describe patterns that can
48 be run against strings of characters (letter, numbers, etc), to see if
49 they match the string or not. The patterns are themselves (sometimes
50 complex) strings of literal characters, combined with wild-cards, and
51 other special characters, called meta-characters. The <span class=
52 "QUOTE">"meta-characters"</span> have special meanings and are used to
53 build complex patterns to be matched against. Perl Compatible Regular
54 Expressions are an especially convenient <span class=
55 "QUOTE">"dialect"</span> of the regular expression language.</p>
56 <p>To make a simple analogy, we do something similar when we use
57 wild-card characters when listing files with the <b class=
58 "COMMAND">dir</b> command in DOS. <tt class="LITERAL">*.*</tt> matches
59 all filenames. The <span class="QUOTE">"special"</span> character here
60 is the asterisk which matches any and all characters. We can be more
61 specific and use <tt class="LITERAL">?</tt> to match just individual
62 characters. So <span class="QUOTE">"dir file?.text"</span> would match
63 <span class="QUOTE">"file1.txt"</span>, <span class=
64 "QUOTE">"file2.txt"</span>, etc. We are pattern matching, using a
65 similar technique to <span class="QUOTE">"regular
66 expressions"</span>!</p>
67 <p>Regular expressions do essentially the same thing, but are much,
68 much more powerful. There are many more <span class="QUOTE">"special
69 characters"</span> and ways of building complex patterns however. Let's
70 look at a few of the common ones, and then some examples:</p>
74 <td><span class="emphasis"><i class="EMPHASIS">.</i></span> -
75 Matches any single character, e.g. <span class=
76 "QUOTE">"a"</span>, <span class="QUOTE">"A"</span>, <span class=
77 "QUOTE">"4"</span>, <span class="QUOTE">":"</span>, or
78 <span class="QUOTE">"@"</span>.</td>
85 <td><span class="emphasis"><i class="EMPHASIS">?</i></span> - The
86 preceding character or expression is matched ZERO or ONE times.
94 <td><span class="emphasis"><i class="EMPHASIS">+</i></span> - The
95 preceding character or expression is matched ONE or MORE
103 <td><span class="emphasis"><i class="EMPHASIS">*</i></span> - The
104 preceding character or expression is matched ZERO or MORE
112 <td><span class="emphasis"><i class="EMPHASIS">\</i></span> - The
113 <span class="QUOTE">"escape"</span> character denotes that the
114 following character should be taken literally. This is used where
115 one of the special characters (e.g. <span class=
116 "QUOTE">"."</span>) needs to be taken literally and not as a
117 special meta-character. Example: <span class=
118 "QUOTE">"example\.com"</span>, makes sure the period is
119 recognized only as a period (and not expanded to its
120 meta-character meaning of any single character).</td>
127 <td><span class="emphasis"><i class="EMPHASIS">[ ]</i></span> -
128 Characters enclosed in brackets will be matched if any of the
129 enclosed characters are encountered. For instance, <span class=
130 "QUOTE">"[0-9]"</span> matches any numeric digit (zero through
131 nine). As an example, we can combine this with <span class=
132 "QUOTE">"+"</span> to match any digit one of more times:
133 <span class="QUOTE">"[0-9]+"</span>.</td>
140 <td><span class="emphasis"><i class="EMPHASIS">( )</i></span> -
141 parentheses are used to group a sub-expression, or multiple
142 sub-expressions.</td>
149 <td><span class="emphasis"><i class="EMPHASIS">|</i></span> - The
150 <span class="QUOTE">"bar"</span> character works like an
151 <span class="QUOTE">"or"</span> conditional statement. A match is
152 successful if the sub-expression on either side of <span class=
153 "QUOTE">"|"</span> matches. As an example: <span class=
154 "QUOTE">"/(this|that) example/"</span> uses grouping and the bar
155 character and would match either <span class="QUOTE">"this
156 example"</span> or <span class="QUOTE">"that example"</span>, and
161 <p>These are just some of the ones you are likely to use when matching
162 URLs with <span class="APPLICATION">Privoxy</span>, and is a long way
163 from a definitive list. This is enough to get us started with a few
164 simple examples which may be more illuminating:</p>
165 <p><span class="emphasis"><i class="EMPHASIS"><tt class=
166 "LITERAL">/.*/banners/.*</tt></i></span> - A simple example that uses
167 the common combination of <span class="QUOTE">"."</span> and
168 <span class="QUOTE">"*"</span> to denote any character, zero or more
169 times. In other words, any string at all. So we start with a literal
170 forward slash, then our regular expression pattern (<span class=
171 "QUOTE">".*"</span>) another literal forward slash, the string
172 <span class="QUOTE">"banners"</span>, another forward slash, and lastly
173 another <span class="QUOTE">".*"</span>. We are building a directory
174 path here. This will match any file with the path that has a directory
175 named <span class="QUOTE">"banners"</span> in it. The <span class=
176 "QUOTE">".*"</span> matches any characters, and this could conceivably
177 be more forward slashes, so it might expand into a much longer looking
178 path. For example, this could match: <span class=
179 "QUOTE">"/eye/hate/spammers/banners/annoy_me_please.gif"</span>, or
180 just <span class="QUOTE">"/banners/annoying.html"</span>, or almost an
181 infinite number of other possible combinations, just so it has
182 <span class="QUOTE">"banners"</span> in the path somewhere.</p>
183 <p>And now something a little more complex:</p>
184 <p><span class="emphasis"><i class="EMPHASIS"><tt class=
185 "LITERAL">/.*/adv((er)?ts?|ertis(ing|ements?))?/</tt></i></span> - We
186 have several literal forward slashes again (<span class=
187 "QUOTE">"/"</span>), so we are building another expression that is a
188 file path statement. We have another <span class="QUOTE">".*"</span>,
189 so we are matching against any conceivable sub-path, just so it matches
190 our expression. The only true literal that <span class=
191 "emphasis"><i class="EMPHASIS">must match</i></span> our pattern is
192 <span class="APPLICATION">adv</span>, together with the forward
193 slashes. What comes after the <span class="QUOTE">"adv"</span> string
194 is the interesting part.</p>
195 <p>Remember the <span class="QUOTE">"?"</span> means the preceding
196 expression (either a literal character or anything grouped with
197 <span class="QUOTE">"(...)"</span> in this case) can exist or not,
198 since this means either zero or one match. So <span class=
199 "QUOTE">"((er)?ts?|ertis(ing|ements?))"</span> is optional, as are the
200 individual sub-expressions: <span class="QUOTE">"(er)"</span>,
201 <span class="QUOTE">"(ing|ements?)"</span>, and the <span class=
202 "QUOTE">"s"</span>. The <span class="QUOTE">"|"</span> means
203 <span class="QUOTE">"or"</span>. We have two of those. For instance,
204 <span class="QUOTE">"(ing|ements?)"</span>, can expand to match either
205 <span class="QUOTE">"ing"</span> <span class="emphasis"><i class=
206 "EMPHASIS">OR</i></span> <span class="QUOTE">"ements?"</span>. What is
207 being done here, is an attempt at matching as many variations of
208 <span class="QUOTE">"advertisement"</span>, and similar, as possible.
209 So this would expand to match just <span class="QUOTE">"adv"</span>, or
210 <span class="QUOTE">"advert"</span>, or <span class=
211 "QUOTE">"adverts"</span>, or <span class="QUOTE">"advertising"</span>,
212 or <span class="QUOTE">"advertisement"</span>, or <span class=
213 "QUOTE">"advertisements"</span>. You get the idea. But it would not
214 match <span class="QUOTE">"advertizements"</span> (with a <span class=
215 "QUOTE">"z"</span>). We could fix that by changing our regular
216 expression to: <span class=
217 "QUOTE">"/.*/adv((er)?ts?|erti(s|z)(ing|ements?))?/"</span>, which
218 would then match either spelling.</p>
219 <p><span class="emphasis"><i class="EMPHASIS"><tt class=
220 "LITERAL">/.*/advert[0-9]+\.(gif|jpe?g)</tt></i></span> - Again another
221 path statement with forward slashes. Anything in the square brackets
222 <span class="QUOTE">"[ ]"</span> can be matched. This is using
223 <span class="QUOTE">"0-9"</span> as a shorthand expression to mean any
224 digit one through nine. It is the same as saying <span class=
225 "QUOTE">"0123456789"</span>. So any digit matches. The <span class=
226 "QUOTE">"+"</span> means one or more of the preceding expression must
227 be included. The preceding expression here is what is in the square
228 brackets -- in this case, any digit one through nine. Then, at the end,
229 we have a grouping: <span class="QUOTE">"(gif|jpe?g)"</span>. This
230 includes a <span class="QUOTE">"|"</span>, so this needs to match the
231 expression on either side of that bar character also. A simple
232 <span class="QUOTE">"gif"</span> on one side, and the other side will
233 in turn match either <span class="QUOTE">"jpeg"</span> or <span class=
234 "QUOTE">"jpg"</span>, since the <span class="QUOTE">"?"</span> means
235 the letter <span class="QUOTE">"e"</span> is optional and can be
236 matched once or not at all. So we are building an expression here to
237 match image GIF or JPEG type image file. It must include the literal
238 string <span class="QUOTE">"advert"</span>, then one or more digits,
239 and a <span class="QUOTE">"."</span> (which is now a literal, and not a
240 special character, since it is escaped with <span class=
241 "QUOTE">"\"</span>), and lastly either <span class=
242 "QUOTE">"gif"</span>, or <span class="QUOTE">"jpeg"</span>, or
243 <span class="QUOTE">"jpg"</span>. Some possible matches would include:
244 <span class="QUOTE">"//advert1.jpg"</span>, <span class=
245 "QUOTE">"/nasty/ads/advert1234.gif"</span>, <span class=
246 "QUOTE">"/banners/from/hell/advert99.jpg"</span>. It would not match
247 <span class="QUOTE">"advert1.gif"</span> (no leading slash), or
248 <span class="QUOTE">"/adverts232.jpg"</span> (the expression does not
249 include an <span class="QUOTE">"s"</span>), or <span class=
250 "QUOTE">"/advert1.jsp"</span> (<span class="QUOTE">"jsp"</span> is not
251 in the expression anywhere).</p>
252 <p>We are barely scratching the surface of regular expressions here so
253 that you can understand the default <span class=
254 "APPLICATION">Privoxy</span> configuration files, and maybe use this
255 knowledge to customize your own installation. There is much, much more
256 that can be done with regular expressions. Now that you know enough to
257 get started, you can learn more on your own :/</p>
258 <p>More reading on Perl Compatible Regular expressions: <a href=
259 "http://perldoc.perl.org/perlre.html" target=
260 "_top">http://perldoc.perl.org/perlre.html</a></p>
261 <p>For information on regular expression based substitutions and their
262 applications in filters, please see the <a href=
263 "filter-file.html">filter file tutorial</a> in this manual.</p>
266 <h2 class="SECT2"><a name="INTERNAL-PAGES" id="INTERNAL-PAGES">14.2.
267 Privoxy's Internal Pages</a></h2>
268 <p>Since <span class="APPLICATION">Privoxy</span> proxies each
269 requested web page, it is easy for <span class=
270 "APPLICATION">Privoxy</span> to trap certain special URLs. In this way,
271 we can talk directly to <span class="APPLICATION">Privoxy</span>, and
272 see how it is configured, see how our rules are being applied, change
273 these rules and other configuration options, and even turn <span class=
274 "APPLICATION">Privoxy's</span> filtering off, all with a web
276 <p>The URLs listed below are the special ones that allow direct access
277 to <span class="APPLICATION">Privoxy</span>. Of course, <span class=
278 "APPLICATION">Privoxy</span> must be running to access these. If not,
279 you will get a friendly error message. Internet access is not necessary
283 <p>Privoxy main page:</p><a name="AEN6034" id="AEN6034"></a>
284 <blockquote class="BLOCKQUOTE">
285 <p><a href="http://config.privoxy.org/" target=
286 "_top">http://config.privoxy.org/</a></p>
288 <p>There is a shortcut: <a href="http://p.p/" target=
289 "_top">http://p.p/</a> (But it doesn't provide a fall-back to a
290 real page, in case the request is not sent through <span class=
291 "APPLICATION">Privoxy</span>)</p>
294 <p>Show information about the current configuration, including
295 viewing and editing of actions files:</p><a name="AEN6042" id=
297 <blockquote class="BLOCKQUOTE">
298 <p><a href="http://config.privoxy.org/show-status" target=
299 "_top">http://config.privoxy.org/show-status</a></p>
303 <p>Show the source code version numbers:</p><a name="AEN6047" id=
305 <blockquote class="BLOCKQUOTE">
306 <p><a href="http://config.privoxy.org/show-version" target=
307 "_top">http://config.privoxy.org/show-version</a></p>
311 <p>Show the browser's request headers:</p><a name="AEN6052" id=
313 <blockquote class="BLOCKQUOTE">
314 <p><a href="http://config.privoxy.org/show-request" target=
315 "_top">http://config.privoxy.org/show-request</a></p>
319 <p>Show which actions apply to a URL and why:</p><a name="AEN6057"
321 <blockquote class="BLOCKQUOTE">
322 <p><a href="http://config.privoxy.org/show-url-info" target=
323 "_top">http://config.privoxy.org/show-url-info</a></p>
327 <p>Toggle Privoxy on or off. This feature can be turned off/on in
328 the main <tt class="FILENAME">config</tt> file. When toggled
329 <span class="QUOTE">"off"</span>, <span class=
330 "QUOTE">"Privoxy"</span> continues to run, but only as a
331 pass-through proxy, with no actions taking place:</p><a name=
332 "AEN6065" id="AEN6065"></a>
333 <blockquote class="BLOCKQUOTE">
334 <p><a href="http://config.privoxy.org/toggle" target=
335 "_top">http://config.privoxy.org/toggle</a></p>
337 <p>Short cuts. Turn off, then on:</p><a name="AEN6069" id=
339 <blockquote class="BLOCKQUOTE">
340 <p><a href="http://config.privoxy.org/toggle?set=disable" target=
341 "_top">http://config.privoxy.org/toggle?set=disable</a></p>
342 </blockquote><a name="AEN6072" id="AEN6072"></a>
343 <blockquote class="BLOCKQUOTE">
344 <p><a href="http://config.privoxy.org/toggle?set=enable" target=
345 "_top">http://config.privoxy.org/toggle?set=enable</a></p>
351 <h2 class="SECT2"><a name="CHAIN" id="CHAIN">14.3. Chain of
353 <p>Let's take a quick look at how some of <span class=
354 "APPLICATION">Privoxy's</span> core features are triggered, and the
355 ensuing sequence of events when a web page is requested by your
359 <p>First, your web browser requests a web page. The browser knows
360 to send the request to <span class="APPLICATION">Privoxy</span>,
361 which will in turn, relay the request to the remote web server
362 after passing the following tests:</p>
365 <p><span class="APPLICATION">Privoxy</span> traps any request for
366 its own internal CGI pages (e.g <a href="http://p.p/" target=
367 "_top">http://p.p/</a>) and sends the CGI page back to the
371 <p>Next, <span class="APPLICATION">Privoxy</span> checks to see if
372 the URL matches any <a href="actions-file.html#BLOCK"><span class=
373 "QUOTE">"+block"</span></a> patterns. If so, the URL is then
374 blocked, and the remote web server will not be contacted. <a href=
375 "actions-file.html#HANDLE-AS-IMAGE"><span class=
376 "QUOTE">"+handle-as-image"</span></a> and <a href=
377 "actions-file.html#HANDLE-AS-EMPTY-DOCUMENT"><span class=
378 "QUOTE">"+handle-as-empty-document"</span></a> are then checked,
379 and if there is no match, an HTML <span class=
380 "QUOTE">"BLOCKED"</span> page is sent back to the browser.
381 Otherwise, if it does match, an image is returned for the former,
382 and an empty text document for the latter. The type of image would
383 depend on the setting of <a href=
384 "actions-file.html#SET-IMAGE-BLOCKER"><span class=
385 "QUOTE">"+set-image-blocker"</span></a> (blank, checkerboard
386 pattern, or an HTTP redirect to an image elsewhere).</p>
389 <p>Untrusted URLs are blocked. If URLs are being added to the
390 <tt class="FILENAME">trust</tt> file, then that is done.</p>
393 <p>If the URL pattern matches the <a href=
394 "actions-file.html#FAST-REDIRECTS"><span class=
395 "QUOTE">"+fast-redirects"</span></a> action, it is then processed.
396 Unwanted parts of the requested URL are stripped.</p>
399 <p>Now the rest of the client browser's request headers are
400 processed. If any of these match any of the relevant actions (e.g.
401 <a href="actions-file.html#HIDE-USER-AGENT"><span class=
402 "QUOTE">"+hide-user-agent"</span></a>, etc.), headers are
403 suppressed or forged as determined by these actions and their
407 <p>Now the web server starts sending its response back (i.e.
408 typically a web page).</p>
411 <p>First, the server headers are read and processed to determine,
412 among other things, the MIME type (document type) and encoding. The
413 headers are then filtered as determined by the <a href=
414 "actions-file.html#CRUNCH-INCOMING-COOKIES"><span class=
415 "QUOTE">"+crunch-incoming-cookies"</span></a>, <a href=
416 "actions-file.html#SESSION-COOKIES-ONLY"><span class=
417 "QUOTE">"+session-cookies-only"</span></a>, and <a href=
418 "actions-file.html#DOWNGRADE-HTTP-VERSION"><span class=
419 "QUOTE">"+downgrade-http-version"</span></a> actions.</p>
422 <p>If any <a href="actions-file.html#FILTER"><span class=
423 "QUOTE">"+filter"</span></a> action or <a href=
424 "actions-file.html#DEANIMATE-GIFS"><span class=
425 "QUOTE">"+deanimate-gifs"</span></a> action applies (and the
426 document type fits the action), the rest of the page is read into
427 memory (up to a configurable limit). Then the filter rules (from
428 <tt class="FILENAME">default.filter</tt> and any other filter
429 files) are processed against the buffered content. Filters are
430 applied in the order they are specified in one of the filter files.
431 Animated GIFs, if present, are reduced to either the first or last
432 frame, depending on the action setting.The entire page, which is
433 now filtered, is then sent by <span class=
434 "APPLICATION">Privoxy</span> back to your browser.</p>
435 <p>If neither a <a href="actions-file.html#FILTER"><span class=
436 "QUOTE">"+filter"</span></a> action or <a href=
437 "actions-file.html#DEANIMATE-GIFS"><span class=
438 "QUOTE">"+deanimate-gifs"</span></a> matches, then <span class=
439 "APPLICATION">Privoxy</span> passes the raw data through to the
440 client browser as it becomes available.</p>
443 <p>As the browser receives the now (possibly filtered) page
444 content, it reads and then requests any URLs that may be embedded
445 within the page source, e.g. ad images, stylesheets, JavaScript,
446 other HTML documents (e.g. frames), sounds, etc. For each of these
447 objects, the browser issues a separate request (this is easily
448 viewable in <span class="APPLICATION">Privoxy's</span> logs). And
449 each such request is in turn processed just as above. Note that a
450 complex web page will have many, many such embedded URLs. If these
451 secondary requests are to a different server, then quite possibly a
452 very differing set of actions is triggered.</p>
455 <p>NOTE: This is somewhat of a simplistic overview of what happens with
456 each URL request. For the sake of brevity and simplicity, we have
457 focused on <span class="APPLICATION">Privoxy's</span> core features
461 <h2 class="SECT2"><a name="ACTIONSANAT" id="ACTIONSANAT">14.4.
462 Troubleshooting: Anatomy of an Action</a></h2>
463 <p>The way <span class="APPLICATION">Privoxy</span> applies <a href=
464 "actions-file.html#ACTIONS">actions</a> and <a href=
465 "actions-file.html#FILTER">filters</a> to any given URL can be complex,
466 and not always so easy to understand what is happening. And sometimes
467 we need to be able to <span class="emphasis"><i class=
468 "EMPHASIS">see</i></span> just what <span class=
469 "APPLICATION">Privoxy</span> is doing. Especially, if something
470 <span class="APPLICATION">Privoxy</span> is doing is causing us a
471 problem inadvertently. It can be a little daunting to look at the
472 actions and filters files themselves, since they tend to be filled with
473 <a href="appendix.html#REGEX">regular expressions</a> whose
474 consequences are not always so obvious.</p>
475 <p>One quick test to see if <span class="APPLICATION">Privoxy</span> is
476 causing a problem or not, is to disable it temporarily. This should be
477 the first troubleshooting step (be sure to flush caches afterward!).
478 Looking at the logs is a good idea too. (Note that both the toggle
479 feature and logging are enabled via <tt class="FILENAME">config</tt>
480 file settings, and may need to be turned <span class=
481 "QUOTE">"on"</span>.)</p>
482 <p>Another easy troubleshooting step to try is if you have done any
483 customization of your installation, revert back to the installed
484 defaults and see if that helps. There are times the developers get
485 complaints about one thing or another, and the problem is more related
486 to a customized configuration issue.</p>
487 <p><span class="APPLICATION">Privoxy</span> also provides the <a href=
488 "http://config.privoxy.org/show-url-info" target=
489 "_top">http://config.privoxy.org/show-url-info</a> page that can show
490 us very specifically how <span class="APPLICATION">actions</span> are
491 being applied to any given URL. This is a big help for
493 <p>First, enter one URL (or partial URL) at the prompt, and then
494 <span class="APPLICATION">Privoxy</span> will tell us how the current
495 configuration will handle it. This will not help with filtering effects
496 (i.e. the <a href="actions-file.html#FILTER"><span class=
497 "QUOTE">"+filter"</span></a> action) from one of the filter files since
498 this is handled very differently and not so easy to trap! It also will
499 not tell you about any other URLs that may be embedded within the URL
500 you are testing. For instance, images such as ads are expressed as URLs
501 within the raw page source of HTML pages. So you will only get info for
502 the actual URL that is pasted into the prompt area -- not any sub-URLs.
503 If you want to know about embedded URLs like ads, you will have to dig
504 those out of the HTML source. Use your browser's <span class=
505 "QUOTE">"View Page Source"</span> option for this. Or right click on
506 the ad, and grab the URL.</p>
507 <p>Let's try an example, <a href="http://google.com" target=
508 "_top">google.com</a>, and look at it one section at a time in a sample
509 configuration (your real configuration may vary):</p>
510 <table border="0" bgcolor="#E0E0E0" width="100%">
513 <pre class="SCREEN"> Matches for http://www.google.com:
515 In file: default.action <span class="GUIBUTTON">[ View ]</span> <span class=
516 "GUIBUTTON">[ Edit ]</span>
518 {+change-x-forwarded-for{block}
519 +deanimate-gifs {last}
520 +fast-redirects {check-decoded-url}
521 +filter {refresh-tags}
522 +filter {img-reorder}
523 +filter {banners-by-size}
525 +filter {jumping-windows}
526 +filter {ie-exploits}
527 +hide-from-header {block}
528 +hide-referrer {forge}
529 +session-cookies-only
530 +set-image-blocker {pattern}
533 { -session-cookies-only }
539 In file: user.action <span class="GUIBUTTON">[ View ]</span> <span class=
540 "GUIBUTTON">[ Edit ]</span>
541 (no matches in this file)</pre>
545 <p>This is telling us how we have defined our <a href=
546 "actions-file.html#ACTIONS"><span class="QUOTE">"actions"</span></a>,
547 and which ones match for our test case, <span class=
548 "QUOTE">"google.com"</span>. Displayed is all the actions that are
549 available to us. Remember, the <tt class="LITERAL">+</tt> sign denotes
550 <span class="QUOTE">"on"</span>. <tt class="LITERAL">-</tt> denotes
551 <span class="QUOTE">"off"</span>. So some are <span class=
552 "QUOTE">"on"</span> here, but many are <span class=
553 "QUOTE">"off"</span>. Each example we try may provide a slightly
554 different end result, depending on our configuration directives.</p>
555 <p>The first listing is for our <tt class=
556 "FILENAME">default.action</tt> file. The large, multi-line listing, is
557 how the actions are set to match for all URLs, i.e. our default
558 settings. If you look at your <span class="QUOTE">"actions"</span>
559 file, this would be the section just below the <span class=
560 "QUOTE">"aliases"</span> section near the top. This will apply to all
561 URLs as signified by the single forward slash at the end of the listing
562 -- <span class="QUOTE">" / "</span>.</p>
563 <p>But we have defined additional actions that would be exceptions to
564 these general rules, and then we list specific URLs (or patterns) that
565 these exceptions would apply to. Last match wins. Just below this then
566 are two explicit matches for <span class="QUOTE">".google.com"</span>.
567 The first is negating our previous cookie setting, which was for
568 <a href="actions-file.html#SESSION-COOKIES-ONLY"><span class=
569 "QUOTE">"+session-cookies-only"</span></a> (i.e. not persistent). So we
570 will allow persistent cookies for google, at least that is how it is in
571 this example. The second turns <span class="emphasis"><i class=
572 "EMPHASIS">off</i></span> any <a href=
573 "actions-file.html#FAST-REDIRECTS"><span class=
574 "QUOTE">"+fast-redirects"</span></a> action, allowing this to take
575 place unmolested. Note that there is a leading dot here -- <span class=
576 "QUOTE">".google.com"</span>. This will match any hosts and
577 sub-domains, in the google.com domain also, such as <span class=
578 "QUOTE">"www.google.com"</span> or <span class=
579 "QUOTE">"mail.google.com"</span>. But it would not match <span class=
580 "QUOTE">"www.google.de"</span>! So, apparently, we have these two
581 actions defined as exceptions to the general rules at the top somewhere
582 in the lower part of our <tt class="FILENAME">default.action</tt> file,
583 and <span class="QUOTE">"google.com"</span> is referenced somewhere in
584 these latter sections.</p>
585 <p>Then, for our <tt class="FILENAME">user.action</tt> file, we again
586 have no hits. So there is nothing google-specific that we might have
587 added to our own, local configuration. If there was, those actions
588 would over-rule any actions from previously processed files, such as
589 <tt class="FILENAME">default.action</tt>. <tt class=
590 "FILENAME">user.action</tt> typically has the last word. This is the
591 best place to put hard and fast exceptions,</p>
592 <p>And finally we pull it all together in the bottom section and
593 summarize how <span class="APPLICATION">Privoxy</span> is applying all
594 its <span class="QUOTE">"actions"</span> to <span class=
595 "QUOTE">"google.com"</span>:</p>
596 <table border="0" bgcolor="#E0E0E0" width="100%">
604 +change-x-forwarded-for{block}
605 -client-header-filter{hide-tor-exit-notation}
606 -content-type-overwrite
607 -crunch-client-header
608 -crunch-if-none-match
609 -crunch-incoming-cookies
610 -crunch-outgoing-cookies
611 -crunch-server-header
612 +deanimate-gifs {last}
613 -downgrade-http-version
616 -filter {content-cookies}
618 -filter {banners-by-link}
619 -filter {tiny-textforms}
620 -filter {frameset-borders}
621 -filter {demoronizer}
622 -filter {shockwave-flash}
623 -filter {quicktime-kioskmode}
625 -filter {crude-parental}
626 -filter {site-specifics}
627 -filter {js-annoyances}
628 -filter {html-annoyances}
629 +filter {refresh-tags}
630 -filter {unsolicited-popups}
631 +filter {img-reorder}
632 +filter {banners-by-size}
634 +filter {jumping-windows}
635 +filter {ie-exploits}
642 -handle-as-empty-document
644 -hide-accept-language
645 -hide-content-disposition
646 +hide-from-header {block}
647 -hide-if-modified-since
648 +hide-referrer {forge}
651 -overwrite-last-modified
654 -server-header-filter{xml-to-html}
655 -server-header-filter{html-to-xml}
656 -session-cookies-only
657 +set-image-blocker {pattern} </pre>
661 <p>Notice the only difference here to the previous listing, is to
662 <span class="QUOTE">"fast-redirects"</span> and <span class=
663 "QUOTE">"session-cookies-only"</span>, which are activated specifically
664 for this site in our configuration, and thus show in the <span class=
665 "QUOTE">"Final Results"</span>.</p>
666 <p>Now another example, <span class=
667 "QUOTE">"ad.doubleclick.net"</span>:</p>
668 <table border="0" bgcolor="#E0E0E0" width="100%">
672 { +block{Domains starts with "ad"} }
675 { +block{Domain contains "ad"} }
678 { +block{Doubleclick banner server} +handle-as-image }
679 .[a-vx-z]*.doubleclick.net</pre>
683 <p>We'll just show the interesting part here - the explicit matches. It
684 is matched three different times. Two <span class=
685 "QUOTE">"+block{}"</span> sections, and a <span class="QUOTE">"+block{}
686 +handle-as-image"</span>, which is the expanded form of one of our
687 aliases that had been defined as: <span class=
688 "QUOTE">"+block-as-image"</span>. (<a href=
689 "actions-file.html#ALIASES"><span class="QUOTE">"Aliases"</span></a>
690 are defined in the first section of the actions file and typically used
691 to combine more than one action.)</p>
692 <p>Any one of these would have done the trick and blocked this as an
693 unwanted image. This is unnecessarily redundant since the last case
694 effectively would also cover the first. No point in taking chances with
695 these guys though ;-) Note that if you want an ad or obnoxious URL to
696 be invisible, it should be defined as <span class=
697 "QUOTE">"ad.doubleclick.net"</span> is done here -- as both a <a href=
698 "actions-file.html#BLOCK"><span class="QUOTE">"+block{}"</span></a>
699 <span class="emphasis"><i class="EMPHASIS">and</i></span> an <a href=
700 "actions-file.html#HANDLE-AS-IMAGE"><span class=
701 "QUOTE">"+handle-as-image"</span></a>. The custom alias <span class=
702 "QUOTE">"<tt class="LITERAL">+block-as-image</tt>"</span> just
703 simplifies the process and make it more readable.</p>
704 <p>One last example. Let's try <span class=
705 "QUOTE">"http://www.example.net/adsl/HOWTO/"</span>. This one is giving
706 us problems. We are getting a blank page. Hmmm ...</p>
707 <table border="0" bgcolor="#E0E0E0" width="100%">
712 Matches for http://www.example.net/adsl/HOWTO/:
714 In file: default.action <span class="GUIBUTTON">[ View ]</span> <span class=
715 "GUIBUTTON">[ Edit ]</span>
719 +change-x-forwarded-for{block}
720 -client-header-filter{hide-tor-exit-notation}
721 -content-type-overwrite
722 -crunch-client-header
723 -crunch-if-none-match
724 -crunch-incoming-cookies
725 -crunch-outgoing-cookies
726 -crunch-server-header
728 -downgrade-http-version
729 +fast-redirects {check-decoded-url}
731 -filter {content-cookies}
733 -filter {banners-by-link}
734 -filter {tiny-textforms}
735 -filter {frameset-borders}
736 -filter {demoronizer}
737 -filter {shockwave-flash}
738 -filter {quicktime-kioskmode}
740 -filter {crude-parental}
741 -filter {site-specifics}
742 -filter {js-annoyances}
743 -filter {html-annoyances}
744 +filter {refresh-tags}
745 -filter {unsolicited-popups}
746 +filter {img-reorder}
747 +filter {banners-by-size}
749 +filter {jumping-windows}
750 +filter {ie-exploits}
757 -handle-as-empty-document
759 -hide-accept-language
760 -hide-content-disposition
761 +hide-from-header{block}
764 -overwrite-last-modified
767 -server-header-filter{xml-to-html}
768 -server-header-filter{html-to-xml}
769 +session-cookies-only
770 +set-image-blocker{blank} }
773 { +block{Path contains "ads".} +handle-as-image }
778 <p>Ooops, the <span class="QUOTE">"/adsl/"</span> is matching
779 <span class="QUOTE">"/ads"</span> in our configuration! But we did not
780 want this at all! Now we see why we get the blank page. It is actually
781 triggering two different actions here, and the effects are aggregated
782 so that the URL is blocked, and <span class=
783 "APPLICATION">Privoxy</span> is told to treat the block as if it were
784 an image. But this is, of course, all wrong. We could now add a new
785 action below this (or better in our own <tt class=
786 "FILENAME">user.action</tt> file) that explicitly <span class=
787 "emphasis"><i class="EMPHASIS">un</i></span> blocks ( <a href=
788 "actions-file.html#BLOCK"><span class="QUOTE">"{-block}"</span></a>)
789 paths with <span class="QUOTE">"adsl"</span> in them (remember, last
790 match in the configuration wins). There are various ways to handle such
791 exceptions. Example:</p>
792 <table border="0" bgcolor="#E0E0E0" width="100%">
801 <p>Now the page displays ;-) Remember to flush your browser's caches
802 when making these kinds of changes to your configuration to insure that
803 you get a freshly delivered page! Or, try using <tt class=
804 "LITERAL">Shift+Reload</tt>.</p>
805 <p>But now what about a situation where we get no explicit matches like
807 <table border="0" bgcolor="#E0E0E0" width="100%">
812 { +block{Path starts with "ads".} +handle-as-image }
817 <p>That actually was very helpful and pointed us quickly to where the
818 problem was. If you don't get this kind of match, then it means one of
819 the default rules in the first section of <tt class=
820 "FILENAME">default.action</tt> is causing the problem. This would
821 require some guesswork, and maybe a little trial and error to isolate
822 the offending rule. One likely cause would be one of the <a href=
823 "actions-file.html#FILTER"><span class="QUOTE">"+filter"</span></a>
824 actions. These tend to be harder to troubleshoot. Try adding the URL
825 for the site to one of aliases that turn off <a href=
826 "actions-file.html#FILTER"><span class=
827 "QUOTE">"+filter"</span></a>:</p>
828 <table border="0" bgcolor="#E0E0E0" width="100%">
834 .worldpay.com # for quietpc.com
841 <p><span class="QUOTE">"<tt class="LITERAL">{ shop }</tt>"</span> is an
842 <span class="QUOTE">"alias"</span> that expands to <span class=
843 "QUOTE">"<tt class="LITERAL">{ -filter -session-cookies-only
844 }</tt>"</span>. Or you could do your own exception to negate
846 <table border="0" bgcolor="#E0E0E0" width="100%">
851 # Disable ALL filter actions for sites in this section
858 <p>This would turn off all filtering for these sites. This is best put
859 in <tt class="FILENAME">user.action</tt>, for local site exceptions.
860 Note that when a simple domain pattern is used by itself (without the
861 subsequent path portion), all sub-pages within that domain are included
862 automatically in the scope of the action.</p>
863 <p>Images that are inexplicably being blocked, may well be hitting the
864 <a href="actions-file.html#FILTER-BANNERS-BY-SIZE"><span class=
865 "QUOTE">"+filter{banners-by-size}"</span></a> rule, which assumes that
866 images of certain sizes are ad banners (works well <span class=
867 "emphasis"><i class="EMPHASIS">most of the time</i></span> since these
868 tend to be standardized).</p>
869 <p><span class="QUOTE">"<tt class="LITERAL">{ fragile }</tt>"</span> is
870 an alias that disables most actions that are the most likely to cause
871 trouble. This can be used as a last resort for problem sites.</p>
872 <table border="0" bgcolor="#E0E0E0" width="100%">
877 # Handle with care: easy to break
879 mybank.example.com</pre>
883 <p><span class="emphasis"><i class="EMPHASIS">Remember to flush
884 caches!</i></span> Note that the <tt class="LITERAL">mail.google</tt>
885 reference lacks the TLD portion (e.g. <span class=
886 "QUOTE">".com"</span>). This will effectively match any TLD with
887 <tt class="LITERAL">google</tt> in it, such as <tt class=
888 "LITERAL">mail.google.de.</tt>, just as an example.</p>
889 <p>If this still does not work, you will have to go through the
890 remaining actions one by one to find which one(s) is causing the
894 <div class="NAVFOOTER">
895 <hr align="left" width="100%">
896 <table summary="Footer navigation table" width="100%" border="0"
897 cellpadding="0" cellspacing="0">
899 <td width="33%" align="left" valign="top"><a href="seealso.html"
900 accesskey="P">Prev</a></td>
901 <td width="34%" align="center" valign="top"><a href="index.html"
902 accesskey="H">Home</a></td>
903 <td width="33%" align="right" valign="top"> </td>
906 <td width="33%" align="left" valign="top">See Also</td>
907 <td width="34%" align="center" valign="top"> </td>
908 <td width="33%" align="right" valign="top"> </td>