Adam Innes · Blog

Put Varnish in Front of Apache

· 4 min · varnish, apache, wordpress, drupal, performance

A WordPress or Drupal site on Apache with mod_php does a lot of repeated work. Every anonymous visitor who hits the front page gets a fresh run through PHP and a pile of MySQL queries, and they all get the same HTML. When a post takes off, Apache children pile up, each holding a chunk of memory, and the database starts to crawl. Most of those requests don’t need PHP at all. That’s the job Varnish does well: it sits in front of Apache, keeps rendered pages in memory, and only bothers the backend when it has to.

The current release is Varnish 3.0.2, tagged on October 26. If your distro still ships 2.1, be aware that VCL changed in 3.0: what 2.1 called purge is now ban (3.0 reuses the word purge for removing a single object, which comes up below), and return (pass) in vcl_fetch became return (hit_for_pass). Everything below is 3.0 syntax.

Swap the ports

Varnish takes port 80 and Apache moves to a port only Varnish talks to. On Apache 2.2 that’s a change to Listen, NameVirtualHost and each <VirtualHost>. Replace the existing Listen 80 and NameVirtualHost *:80 lines (on Debian and Ubuntu they live in ports.conf) rather than adding to them, or Varnish won’t be able to bind port 80:

Listen 127.0.0.1:8080
NameVirtualHost 127.0.0.1:8080

<VirtualHost 127.0.0.1:8080>
    ServerName www.example.com
    DocumentRoot /var/www/example
</VirtualHost>

Then point Varnish’s backend at it and start varnishd on port 80:

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}
varnishd -a :80 -f /etc/varnish/default.vcl -s malloc,1G

Apache will now see every request coming from 127.0.0.1. The default VCL adds an X-Forwarded-For header with the real client address, but only when the built-in vcl_recv actually runs, which matters in a minute. Log %{X-Forwarded-For}i in your LogFormat instead of %h, and in Drupal 7 turn on $conf['reverse_proxy'] with Varnish’s address in reverse_proxy_addresses so it trusts that header.

Cookies are what kill your hit rate

Install Varnish with no custom VCL and you’ll probably see very few cache hits. The reason is in the default VCL for 3.0.2: any request carrying a Cookie or Authorization header is passed straight to the backend, and any response with a Set-Cookie header isn’t cached. If you use Google Analytics, its JavaScript sets cookies too, so nearly every request carries a Cookie header even though your PHP never reads those cookies.

The fix is to decide which cookies actually matter and throw the rest away for anonymous visitors. On WordPress the one that matters on the front end is wordpress_logged_in_ plus a hash, which WordPress 3.2.1 sets for the whole site when you log in. The other auth cookies are scoped to the admin and plugin paths. So:

sub vcl_recv {
    if (req.url ~ "^/wp-(admin|login)" || req.http.Cookie ~ "wordpress_logged_in_|wp-postpass_") {
        set req.http.X-Forwarded-For = client.ip;
        return (pass);
    }
    unset req.http.Cookie;
}

sub vcl_fetch {
    if (beresp.status == 200 && !req.http.Cookie && !(req.url ~ "^/wp-(admin|login)")) {
        unset beresp.http.Set-Cookie;
        set beresp.ttl = 5m;
    }
}

Your own vcl_recv and vcl_fetch run first, and if they don’t return, Varnish’s built-in defaults run afterward, so you still get the sane handling of POSTs and other methods. The catch is that a return (pass) skips the built-in code, including the part that sets X-Forwarded-For, so the pass branch sets it itself; otherwise every logged in user shows up in your logs as 127.0.0.1. The status check keeps Varnish from holding on to a PHP error or a 404 for five minutes. The TTL line matters because WordPress core doesn’t send caching headers on ordinary pages for logged out visitors, and without them Varnish falls back to its default_ttl of 120 seconds. There are trade-offs. The comment_author_ cookies never reach PHP for anonymous visitors, so “remember my name” and the awaiting moderation notice stop working, and the pattern above passes the wp-postpass_ cookie through so password protected posts still open.

Drupal 7 is friendlier. Its session code doesn’t give anonymous users a session cookie unless something is actually stored in the session, specifically so HTTP proxies can cache anonymous page views. The session cookie name starts with SESS (or SSESS when secure session cookies are on), so pass when req.http.Cookie ~ "SESS" (setting X-Forwarded-For in that branch too, or Drupal will think every logged in user is 127.0.0.1) and strip everything else. Also turn on “Cache pages for anonymous users” and set “Expiration of cached pages” on the Performance page. That makes Drupal send Cache-Control: public, max-age=... to visitors without a session, which Varnish uses as the TTL. Leave it off, or leave the expiration at zero, and Drupal sends no-cache headers with an Expires date in 1978, and Varnish won’t cache the page.

Purge when content changes

A longer TTL means more hits and more stale pages, so tell Varnish when something changes. The 3.0.2 purging tutorial handles a PURGE method from an ACL of trusted addresses:

acl purge {
    "localhost";
}

sub vcl_recv {
    if (req.request == "PURGE") {
        if (!client.ip ~ purge) {
            error 405 "Not allowed.";
        }
        return (lookup);
    }
}

sub vcl_hit {
    if (req.request == "PURGE") { purge; error 200 "Purged."; }
}

sub vcl_miss {
    if (req.request == "PURGE") { purge; error 200 "Purged."; }
}

In 3.0 it’s fine to define vcl_recv more than once; Varnish joins them in the order they appear, so put the PURGE block above the cookie handling. A purge removes one URL on one host, along with its variants. From the server, curl -X PURGE -H "Host: www.example.com" http://127.0.0.1/some-post/ clears a page. Hook that into WordPress’s save_post action or Drupal 7’s hook_node_update, and purge the front page and feeds too, since those change with every new post. New comments don’t fire save_post, so hook comment_post as well if comments matter to you. When you need something broader, like everything under a path, the same tutorial covers bans, which match cached objects against an expression instead of a single URL.

My advice is to start conservative: short TTLs, cookie handling, and varnishstat open in a terminal so you can watch the hit rate climb. Once it looks right, raise the TTLs and let purging keep things fresh.

← all posts