/usr/bin/pwn

Like I mentioned previously, this site is now being served up by lighttpd; it took a while to get clean URLs working right, but what right now is making my server tick is a very carefully laid-out filesystem and set of bash scripts and lighty config files, wrapped around a Drupal multisite installation to make installing modules and upgrading the entire system as intuitive and painless as possible. Upgrading Drupal (from 6.2 to the brand-new 6.3 for example) takes no more than running a single command. Here's an overview of how my own site is set up, generalized to make it applicable to almost any other system.

As a general disclaimer, my server is running Ubuntu Server 8.04 so your filesystem layout may vary slightly, and I'm assuming you have a general knowledge of Linux terminology and bash commands, and that lighttpd with PHP and FastCGI are already set up.

To start off, here's how the web server file system root (/var/www for me) looks:

/var
  |- www
       |- drupal -> drupal-6.3
       |- drupal-6.3
            | ...
            |- includes
                 | ...
                 |- image.imagemagick.inc -> /var/www/drupal-sites/all/modules/image/image.imagemagick.inc
                 | ...
            | ...
            |- sites -> /var/www/drupal-sites
            | ...
       |- drupal-sites
            |- all
                |- modules
                |- themes
            |- default
            |- example1.com -> /var/www/example1.com/drupal
            |- example2.com -> /var/www/example2.com/drupal
       |- example1.com
            |- drupal
            |- files
            |- htdocs -> /var/www/drupal
            |- sub
                |- htdocs
       |- example2.com
            |- drupal
            |- files
            |- htdocs -> /var/www/drupal

To explain briefly in words:

• The symlink drupal points to the folder of the current version of Drupal, so that the only thing that has to change when upgrading to a new version of Drupal is the one symlink.
• The drupal/sites directory points to the drupal-sites directory, which contains the all (and all/modules, where you should install modules for all sites to access them, and all/themes, the same as with modules except for themes) and default directories (which you'll never put anything in or do anything with, but need to keep around; copy it over from a freshly-downloaded Drupal).
• example1.com and example2.com have their own directories in /var/www; each one has three folders:
  • drupal, which has a symbolic link (ln -s) pointing to it from /var/www/drupal-sites as indicated above;
  • files, which is used as the upload directory with the private download method (if you use the public method, you can just leave the files directory in the site's drupal directory alone and omit this one);
  • htdocs, which is the document root for the site pointed to by the site's lighttpd configuration file (explained later), and which points back to the Drupal directory symlink for Drupal's multisite handling to work.
• If you install Image.module and want to use the ImageMagick library, make a symlink in the drupal/includes directory pointing to the image.imagemagick.inc file in drupal-sites/all/modules/image (the Image.module directory) so that upgrading Image.module also upgrades the ImageMagick include.

Next up is configuring lighttpd. Only a couple changes need to be made in /etc/lighttpd/lighttpd.conf (the global lighttpd configuration file):

1. Uncomment the lines with "mod_rewrite", "mod_redirect", and "mod_evhost" to enable those modules.
2. Further down, look for a commented line beginning with #evhost.path-pattern. Uncomment it or start a new line below it, and write the following:

   evhost.path-pattern = "/var/www/%0/htdocs/"

   Or if you're like me, and want (a) to forcibly remove "www." from the beginning of URLs and (b) handle subdomains for each site:

   $HTTP["host"] =~ "(^|^www\.)[^.]+\.[^.]+$" {
       evhost.path-pattern = "/var/www/%0/htdocs/"
   }
   $HTTP["host"] !~ "(^|^www\.)[^.]+\.[^.]+$" { 
       evhost.path-pattern = "/var/www/%0/%3/htdocs/"
   }

   and for each subdomain sub.example1.com you want to have, make the directory /var/www/example1.com/sub/htdocs.

To handle Drupal's clean URLs we'll use lighttpd's mod_magnet, which lets you use Lua scripts to handle requests in lighttpd. Put the following in /etc/lighttpd/drupal.lua (or save the attached drupal.lua file to /etc/lighttpd):

-- /etc/lighttpd/drupal.lua
-- Based on <http://www.morphir.com/Lighttpd-Install-and-configuration-for-Drupal-with-clean-url>
-- Taken from <http://pub.jbhannah.net/scripts/drupal.lua>
-- little helper function
function file_exists(path)
  local attr = lighty.stat(path)
  if (attr) then
      return true
  else
      return false
  end
end
function removePrefix(str, prefix)
  return str:sub(1,#prefix+1) == prefix.."/" and str:sub(#prefix+2)
end
 
-- prefix without the trailing slash
local prefix = ''
 
-- the magic ;)
if (not file_exists(lighty.env["physical.path"])) then
    -- file still missing. pass it to the fastcgi backend
    request_uri = removePrefix(lighty.env["uri.path"], prefix)
    if request_uri then
      lighty.env["uri.path"]          = prefix .. "/index.php"
      local uriquery = lighty.env["uri.query"] or ""
      lighty.env["uri.query"] = uriquery .. (uriquery ~= "" and "&amp;" or "") .. "q=" .. request_uri
      lighty.env["physical.rel-path"] = lighty.env["uri.path"]
      lighty.env["request.orig-uri"]  = lighty.env["request.uri"]
      lighty.env["physical.path"]     = lighty.env["physical.doc-root"] .. lighty.env["physical.rel-path"]
    end
end
-- fallthrough will put it back into the lighty request loop
-- that means we get the 304 handling for free. ;)

Ubuntu's lighttpd comes with a neat system for handling the configuration for separate modules: additional configuration files can be placed in /etc/lighttpd/conf-available with the naming convention ##-NAME.conf (the lower the two-digit number ##, the earlier it gets loaded), then enabled (or, symlinked in the /etc/lighttpd/conf-enabled directory) with the command (as root or sudo)

~# lighty-enable-mod NAME

replacing NAME with the NAME part of the filename of the configuration file. After that you have to make lighttpd reload its configuration and see the new symlink in the conf-enabled directory:

~# /etc/init.d/lighttpd force-reload

For example, do the following to install mod_magnet on Ubuntu:

~# apt-get install lighttpd-mod-magnet
~# lighty-enable-mod magnet
~# /etc/init.d/lighttpd force-reload

Installing mod_magnet creates the file /etc/lighttpd/conf-available/10-magnet.conf, and the lighty-enable-mod command creates a symlink to that file in the conf-enabled directory; then lighttpd picks up the newly enabled configuration file upon reloading.

Another use for this configuration layout is creating and keeping separate configuration files for each domain. Drupal will install and work up to the point before installing mod_magnet and creating drupal.lua, but clean URLs won't work because lighttpd can't handle directory-specific configuration files (like Apache's .htaccess files, which Drupal is set up to handle out of the box). Instead, we'll create a file in /etc/lighttpd/conf-available for each domain, and in there tell lighttpd to use drupal.lua when handling requests for that domain. For example, the file /etc/lighttpd/conf-available/20-example1-com.conf might look like:

# The following redirects all non-existing subdomains and the "www." prefix to
# the top domain. Change 'sub' below to a vertical-bar (|)-delimited list of
# subdomains.
$HTTP["host"] !~ "^((sub)\.)?example1\.com" {
	$HTTP["host"] =~ "^(.+\.)example1\.com" {
		url.redirect = ( "^/(.*)" => "http://example1.com/$1" )
	}
}
 
# If you don't have any subdomains you want to be web-accessible, comment out
# the above and uncomment the following:
#$HTTP["host"] =~ "^(.+\.)example1\.com" {
#	url.redirect = ( "^/(.*)" => "http://example1.com/$1" )
#}
 
$HTTP["host"] == "example1.com" {
	index-file.names = ( "index.php" )
 
	url.rewrite += ( "^/frontpage$" => "/" )
#	A couple examples of rewrite rules
#	url.rewrite += ( "^/story/[0-9]{4}/[0-9]{2}/[0-9]{2}/(.*)$" => "/blog/$1" )
#	url.rewrite += ( "^/archive/([0-9]{4})([0-9]{2})$" => "/archive/$1/$2" )
 
#	Change /etc/lighttpd to the path of the drupal.lua file
	magnet.attract-physical-path-to = ( "/etc/lighttpd/drupal.lua" )
}

The naming convention of the file, 20-example1-com.conf, may need a little explaining. Actual module configurations have a priority of 10; you want your site's configuration to load after mod_magnet has loaded, so giving it a priority of 20 will have it load after all of the default modules you have enabled (but most importantly mod_magnet). Also, periods (.) are not allowed in the filenames of configuration files, so use a dash instead between the domain name and top level domain. Next, enable the module and reload lighttpd:

~# lighty-enabled-mod example1-com
~# /etc/init.d/lighttpd reload

And there you have it: a Drupal multisite installation served up by lighttpd, with clean URLs fully functional (remember to enable them from the Drupal administration page). Install modules in /var/www/drupal-sites/all/modules; or put the following shell script in /usr/local/bin and make it executable (chmod a+x; this and the next script are available for download at the bottom of this post):

#! /bin/sh
 
# /usr/local/bin/drupal-install-mod
# Downloads and extracts a Drupal module to the Drupal sites/all/modules
# directory.
 
# Copyright (C)2008 Jesse B. Hannah <jesse@jbhannah.net>
# Available under the GNU General Public License
# <http://www.gnu.org/licenses/gpl.html>. Taken from
# <http://pub.jbhannah.net/scripts/drupal-install-mod>.
 
# Must run as root or sudo!
if [ "$(id -u)" != "0" ]; then
  echo "Must be run as root!"
  exit 1
fi
 
 
if [ "$1" == "" ]; then
  echo "Usage: drupal-install-mod NAME-N.n-VERSION"
  exit 1
fi
 
cd /var/www/drupal-sites/all/modules
wget http://ftp.drupal.org/files/projects/$1.tar.gz
tar xzf *.gz
rm *.gz

and then, to install (for example) version 6.x-1.0-alpha2 of Image.module (Drupal project name "image"; run as root or sudo):

~# drupal-install-mod image-6.x-1.0-alpha2

then go to the Drupal modules administration page and enable the newly-downloaded module.

Now, say you've had this site running nicely for a while, then a new version of Drupal comes out. Luckily, thanks to the filesystem layout that keeps all of your site-specific configuration out of the Drupal codebase directory, you don't have to manually back everything up elsewhere; simply download the new version of Drupal, give it the symlinks to the sites directory and ImageMagick include, update the /var/www/drupal symlink to point to the new version, and delete the old version. Or, save the following script as /usr/local/bin/upgrade-drupal and make it executable:

#! /bin/sh
 
# /usr/local/bin/upgrade-drupal
# Upgrades from one Drupal version to a newer one.
# Copyright (C)2008 Jesse B. Hannah <jesse@jbhannah.net>
# Available under the GNU General Public License
# <http://www.gnu.org/licenses/gpl.html>. Taken from
# <http://pub.jbhannah.net/scripts/upgrade-drupal>.
 
# Must run as root or sudo!
if [ "$(id -u)" != "0" ]; then
	echo "Must be run as root!"
	exit 1
fi
 
# SAFETY! BACK UP /var/www BEFORE BEGINNING!
cd /var
tar cjf www-`date +%Y%m%d`-`date +%H%M`.tar.bz2 www
 
# First argument: old version; second argument: new version (both X.x)
OLD=$1
NEW=$2
 
# Get and unpackage the new version and get rid of the package
cd /var/www
wget http://ftp.osuosl.org/pub/drupal/files/projects/drupal-${NEW}.tar.gz
tar xzf drupal-${NEW}.tar.gz
rm drupal-${NEW}.tar.gz
 
# symlink to sites directory
cd drupal-${NEW}
rm -rf sites
ln -s /var/www/drupal-sites sites
 
# symlink to ImageMagick
cd includes
ln -s /var/www/drupal-sites/all/modules/image/image.imagemagick.inc
 
# Update /var/www/drupal symlink and remove old folder
cd /var/www
rm drupal
ln -s drupal-${NEW} drupal
rm -rf drupal-${OLD}

Modify the above for any differences in your setup (comment out the ImageMagick section, for example), then to upgrade (for example) from 6.2 to 6.3:

~# upgrade-drupal 6.2 6.3

then visit update.php for each domain using Drupal in your web browser to update the database schema. If everything borks spectacularly after the upgrade, delete the /var/www directory and decompress the bzipped tar file created by the upgrade script in /var to restore your file tree to its prior state.

And there you have it. That's pretty much a full rundown of how my server is set up; leave a comment if these instructions work for you, or if you try it and encounter any issues, or if you get it working on another platform besides Ubuntu and want to share what you did differently. drupal.lua script and basic clean URL setup instructions taken from http://www.morphir.com/Lighttpd-Install-and-configuration-for-Drupal-wit.... All scripts indicated as such are copyright ©2008 Jesse B. Hannah, and are available with no warranty or guarantee for fitness for a particular purpose under the terms of the GNU General Public License version 3 or later; also available for download at http://pub.jbhannah.net/scripts.