Index of /pub/NetBSD/NetBSD-current/src/share/examples/rump/virtual_ip_router/

	$NetBSD: README.txt,v 1.2 2010/03/29 02:11:14 pooka Exp $

Using rump it is possible to build a router test setup consisting
of thousands of NetBSD IP stacks within a single host OS, one
networking stack per application process.  Each IP stack instance
has its own set of interfaces, addresses and routing tables.  These
instances may or may not share the same code, i.e. it is possible
to do compatibility testing of new features.  The advantage over
using full-fledged virtual OS setups (qemu, Xen, etc.) is scalability:
the rump IP router base runtime takes less than 500kB of memory
per instance.

The code is _ONLY AN EXAMPLE_ as opposed a fully featured test kit.
Some code tweaking is probably required to make this do what you
want.  Usage examples follow.

To use one single rump networking stack instance with access to
two real networks, you need tap and bridge on the host system (yes,
this involves some memory copies.  the resulting router setup can
still saturate a GigE, though.  it should not be difficult to bring
performance to be ~the same as an in-kernel stack, but haven't
managed to implement that yet).

Anyway, the following can be done with the current code:

/*
 * Usage:
 *
 * # ifconfig yourrealif0 up
 * # ifconfig tap0 create
 * # ifconfig tap0 up
 * # ifconfig bridge0 create
 * # brconfig bridge0 add tap0 add yourrealif0
 * # brconfig bridge0 up
 * #
 * # ifconfig yourrealif1 up
 * # ifconfig tap1 create
 * # ifconfig tap1 up
 * # ifconfig bridge1 create
 * # brconfig bridge1 add tap1 add yourrealif1
 * # brconfig bridge1 up
 * #
 * # ./router virt0 192.168.1.1 255.255.255.0 192.168.1.255 \
 * #          virt1 192.168.2.1 255.255.255.0 192.168.2.255
 *
 * This will bind virtN to tapN and act as a router.
 */

As brilliant ascii art, it would look something like this:

           network                                 network
              ^                                       ^
              |                                       |
         /----v-------------\            /------------v----\
 kernel  | realif0 <-> tap0 |            | tap1 -> realif1 |
         \---------------^--/            \---^-------------/
-------------------------|-------------------|--------------------
                    /----v-------------------v----\
   user             | virt0 <-> rump IP <-> virt1 |
		    \-----------------------------/

(ok, no more drawing)

The addresses configured to the rump virt0 and virt1 interfaces
will be visible on the physical network, and their traffic can be
examined with e.g. wireshark.   You can also use wireshark on
tap0/tap1.

The alternate approach is to use purely internal simulation.  The
shmif rump driver uses a memory-mapped file as an ethernet "bus"
between multiple rump networking stack instances.  Just use
rump_pub_shmif_create() in the code.  This can also of course be
combined with the tap setup, and you can have setups where border
nodes talk to an internal mesh of shmif's.  Semi-drawn, it looks
like this:

net1 <-> virt0, shm0 <-> shm1, shm2 <-> .... <-> shmN, virt1 <-> net1
           (rump0)         (rump1)      ....      (rumpN)

Linear setups (where router n talks to exactly router n-1 and n+1)
can be easily autogenerated.  Here's a snippet of executed commands
I used to start a few hundred routers (NOTE! the usage of the
example code is different!):

./a.out 10.0.0.1 10.0.0.255 /tmp/rumpshm_0 0 10.0.1.2 10.0.1.255 /tmp/rumpshm_1 10.0.1.1
./a.out 10.0.1.1 10.0.1.255 /tmp/rumpshm_1 10.0.1.2 10.0.2.2 10.0.2.255 /tmp/rumpshm_2 10.0.2.1
./a.out 10.0.2.1 10.0.2.255 /tmp/rumpshm_2 10.0.2.2 10.0.3.2 10.0.3.255 /tmp/rumpshm_3 10.0.3.1
./a.out 10.0.3.1 10.0.3.255 /tmp/rumpshm_3 10.0.3.2 10.0.4.2 10.0.4.255 /tmp/rumpshm_4 10.0.4.1
....
./a.out 10.0.252.1 10.0.252.255 /tmp/rumpshm_252 10.0.252.2 10.0.253.2 10.0.253.
255 /tmp/rumpshm_253 10.0.253.1
./a.out 10.0.253.1 10.0.253.255 /tmp/rumpshm_253 10.0.253.2 10.0.255.1 10.0.255.
255 /tmp/rumpshm_255 0

(see startrouters.sh for a script to produce that output)

Easy but slightly more interesting setups, such as a M^N matrix
(hyper-matrix?) are also possible, but left as an exercise to the
reader.

Compiling the router depends a little on what networking domain
and what interface you want to use for testing.  The very basic
setup with IP+virtif will get you quite far:

cc rumprouter.c -lrumpnet_virtif -lrumpnet_netinet -lrumpnet_net -lrumpnet \
    -lrump -lrumpuser -lpthread