Thursday, May 14, 2009

Jails (VM's) in FreeBSD 7

==Building your jail for the first time==

===Creating an appropriate make.conf===
You'll need to run make world (or make installworld) to create your jail.
If you don't want to install the whole kitchen sink you can use the make.conf
below. You can put it in your jail for future use and it'll be used by future
port builds inside your jail. One thing I've noticed is that make installworld
doesn't seem to respect and MAKE_CONF or __MAKE_CONF variables passed to it so
we'll just put it in /etc/make.conf for now.

Lets first back our current make.conf up:
cp /etc/make.conf /etc/make.conf.bak

And new one in there. Keep in mind, depending on what you want to use
this jail for you may want to modify this make.conf. For me this has worked
on building a variety of services from ports (inside the jail).
I like to name the below file make.conf.jail and copy it to make.conf,
then copy make.conf.bak back to make.conf when I'm done building the jail.

NO_ACPI= true # do not build acpiconf(8) and related programs
NO_BOOT= true # do not build boot blocks and loader
NO_BLUETOOTH= true # do not build Bluetooth related stuff
NO_FORTRAN= true # do not build g77 and related libraries
NO_GDB= true # do not build GDB
NO_GPIB= true # do not build GPIB support
NO_I4B= true # do not build isdn4bsd package
NO_IPFILTER= true # do not build IP Filter package
NO_PF= true # do not build PF firewall package
NO_AUTHPF= true # do not build and install authpf (setuid/gid)
NO_KERBEROS= true # do not build and install Kerberos 5 (KTH Heimdal)
NO_LPR= true # do not build lpr and related programs
NO_MAILWRAPPER=true # do not build the mailwrapper(8) MTA selector
NO_MODULES= true # do not build modules with the kernel
NO_NETCAT= true # do not build netcat
NO_NIS= true # do not build NIS support and related programs
NO_SENDMAIL= true # do not build sendmail and related programs
NO_SHAREDOCS= true # do not build the 4.4BSD legacy docs
NO_USB= true # do not build usbd(8) and related programs
NO_VINUM= true # do not build Vinum utilities
NO_ATM= true # do not build ATM related programs and libraries
NO_CRYPT= true # do not build any crypto code
NO_GAMES= true # do not build games (games/ subdir)
NO_INFO= true # do not make or install info files
NO_MAN= true # do not build manual pages
NO_PROFILE= true # Avoid compiling profiled libraries

# BIND OPTIONS
NO_BIND= true # Do not build any part of BIND
NO_BIND_DNSSEC= true # Do not build dnssec-keygen, dnssec-signzone
NO_BIND_ETC= true # Do not install files to /etc/namedb
NO_BIND_LIBS_LWRES= true # Do not install the lwres library
NO_BIND_MTREE= true # Do not run mtree to create chroot directories
NO_BIND_NAMED= true # Do not build named, rndc, lwresd, etc.

===Building the Jail===
Now for actually building your jail...

I'm defining JAILDIR here because I'm going to use it in a shellscript style example throughout the rest of this howto.

# Let's first make some directories
JAILDIR=/home/jail
mkdir -p $JAILDIR/dev
mkdir -p $JAILDIR/etc
mkdir -p $JAILDIR/usr/tmp
chmod 777 $JAILDIR/usr/tmp

cd /usr/src/

# You can replace the below with make installworld if you've built your
# world previously
make buildworld
make installworld DESTDIR=$JAILDIR
cd /usr/src/etc
cp /etc/resolv.conf $JAILDIR

make distribution DESTDIR=$JAILDIR NO_OPENSSH=YES NO_OPENSSL=YES
cd $JAILDIR

# At this point we'll mount devfs, and then hide the unneeded devs
mount_devfs devfs $JAILDIR/dev
devfs -m $JAILDIR/dev rule -s 4 applyset

# Create a null kernel
ln -s dev/null kernel

# Quell warnings about fstab
touch $JAILDIR/etc/fstab

# Use our existing resolv.conf
cp /etc/resolv.conf $JAILDIR/etc/resolv.conf

# Copy our settings for ssl
mkdir -p $JAILDIR/etc/ssl
mkdir -p $JAILDIR/usr/local/openssl
cp /etc/ssl/openssl.cnf $JAILDIR/etc/ssl
cd $JAILDIR/usr/local/openssl/
ln -s ../../../etc/ssl/openssl.cnf openssl.cnf

Make a decent rc.conf:

hostname="jail.example.com" # Set this!
ifconfig_em0="inet 10.0.0.20 netmask 255.255.255.255"
defaultrouter="10.0.0.1" # Set to default gateway (or NO).
clear_tmp_enable="YES" # Clear /tmp at startup.
# Once you set your jail up you may want to consider adding a good securelevel:
# Same as sysctl -w kern.securelevel=3
kern_securelevel_enable="YES" # kernel security level (see init(8)),
kern_securelevel="3"


You'll also want to make an alias on your interface for the ip above so we'll
do something like:

ifconfig em0 10.0.0.20 netmask 255.255.255.255 alias

Now you'll want to have devfs inside your jail, so to get it working for
the first time do this:
mount_devfs devfs $JAILDIR/devfs

And finally, copy your original make.conf back.
cp /etc/make.conf.bak /etc/make.conf

==Starting the jail for the first time==
OPTIONAL (but probably necessary):
You'll want to mount /usr/ports and /usr/src so you can install ports inside
your jail, unless you have another way you want to do this (such as downloading
packages).
mount_nullfs /usr/ports $JAILDIR
mount_nullfs /usr/src $JAILDIR

Now we can start our jail
jail $JAILDIR jail.example.com 10.0.0.20 /bin/sh

Once inside the jail you'll want to start services:
/bin/sh /etc/rc

While you're here you'll want to edit your password file since if someone
breaks into your jail, and starts cracking it you won't want them to have
the same passwords as your root system has. Also remove all users you
don't need in the jail:
vipw
passwd root

From here, assuming all went well you can do something like:
cd /usr/ports/security/openssh
make install clean

And build your port(s) inside your jail. Once you're finished be sure to
unmount the directories so a compromised jail can't build more ports.

I've verifed that truss works correctly in a jail
so between ldd and truss you should be set.

Also note that if you try to start your jail with just:
jail $JAILDIR jail.example.com 10.0.0.20 /bin/sh /etc/rc
but you have no services/daemons/programs set to run, the jail will simply start and then exit since there's nothing running inside.

==Getting it to start automatically==
You'll now need to put your settings in /etc/rc.conf
First put the alias you jail has in there:

ifconfig_em0_alias0="inet 10.0.0.20 netmask 0xffffffff"

===Editing the rc.conf===
For those of you that are looking to make your own rc script, I don't recommend it.
I've found issues getting devfs rules to be applied with the a script, and really this way is much easier. It's also the standard way and you can attach to jails later on quite easily
without using screen (read below).

Here's the standard rc.conf way of getting your jail to run at startup:

jail_enable="YES" # Set to NO to disable starting of any jails
jail_list="pk" # Space separated list of names of jails
jail_set_hostname_allow="NO" # Allow root user in a jail to change its hostname
jail_socket_unixiproute_only="YES" # Route only TCP/IP within a jail

jail_pk_rootdir="/usr/home/prison/pk"
jail_pk_hostname="pk.example.com"
jail_pk_ip="10.0.0.20"
jail_pk_exec_start="/bin/sh /etc/rc"
jail_pk_devfs_enable="YES"
jail_pk_devfs_ruleset="devfsrules_jail"

==Jail maintenance==
Of course from time to time you may have to upgrade ports in your jail, or the world in the jail itself. This isn't a big deal either. Instead of using jail (which makes its own IP address and everything) we can use chroot instead which is similar since all we're using is a simple shell
and then we'll be done with it.

First mount the dirs so they're accessable in the chroot:
mount_nullfs /usr/ports $JAILDIR
mount_nullfs /usr/src $JAILDIR

Connect to your jail:
find the jail id of the jail you are running with jls:
#jls
JID IP Address Hostname Path
1 10.0.0.20 pk.example.com /usr/home/prison/pk

Now connect to it using the JID:
jexec 1 /bin/sh

To upgrade your world:
cd /usr/src
make buildworld
make installworld
NOTE: If you've just done make buildworld previously you can do make installworld and install all the newly compiled binaries again.

To build a port:
cd /usr/ports/sysutils/example
make install clean
NOTE: You may also want to install portupgrade to make port management easier.

When you're done just exit:
exit

No comments:

Post a Comment