Log in

No account? Create an account


I have several entries to write: one flaming about Openswan, one about eventual conclusions for clothes matching and one on general state of life/moving into apartment. The Openswan entrie requires the least new thought and I'm getting tired so I'll write it now.

So, at new apartment, I need a tunnel so I can have static IPs. I have a colo box and it has a /29 for my use.

Ideally the goal is to set up an IPsec tunnel to that box and then to route the tunneled address block over that set of addresses. However we get charged somewhat for colo bandwith so it is mildly desirable to minimize usage of it. So I'd like to NAT outgoing HTTP, http, https and ssh. In truth, natting ssh is more about robustness in case of network failure than bandwith. So I need an IPsec solution that will allow some packets to skip IPsec and then go through the NAT rules and other packets to be encapsulated. In addition I'd like to eventually support a second tunnel with net 18 (MIT) addresses because sometimes those are useful. Ideally you could do complex things like say "Access to websites that MIT has a subscription to use an MIT address; other websites are natted through the cable modem and everything else goes over IPsec to the colo."

Linux IPsec support is very much in the free puppy stage. Apparently the Linux IPsec puppy owners don't believe in neutering or spaying; lots of options all poorly cared for. There seem to be two main choices to make: choose an IPsec engine and a key exchange implementation. The IPsec engine does the per-packet processing and the key exchange implementation probably implements something sort of like IKE.

There are a few combined implementations that provide both parts of the solution, usually in userspace instead of as a kernel module. One is called vpnc and is designed to talk to some Cisco VPN products including the one MIT uses. I'm not sure why you can't use a general purpose IPsec implementation for the Cisco products or whether vpnc can be used to talk to a general purpose security gateway. There's something in Debian called pipsecd; no idea what it is other than that it seems to be purely userspace. I discarded the combined implementations because they seemed to be incomplete or special purpose.

There are two IPsec engines. The first is Clipse. It was originally written for Freeswan. It has some fairly horrible design choices. For example for every interface you want to send IPsec traffic over, you will associate an ipsec interface. You route the traffic to the associated ipsec interface and it will magically generate encrypted packets on the underlying interface. If your routing changes, you lose. If there are multiple routes to a destination, you lose. I don't think Clipse supports policy specification on a finer granularity than host. So if you want port 80 to go one place and everything else encrypted, you lose. Clipse is notoriously bad at making sure that incoming traffic is validated; if traffic that your policy requires be encrypted comes in over a clear channel it will be accepted. The other implementation is a port of the Came IPsec from BSD found in the 2.6 kernel. AWarlord was one of the people who worked on this. It has some fairly horrible design choices. IPsec policy
specification is fairly rigid and is enforced early in the packet processing, particularly before things like route decisions, most packet mangling etc—fails to play all that well with iproute2. The policy specification language is an incomplete subset of both of the IPsec protocol specifications; under the old syntax you should get per-interface policy and under the new syntax you get more flexible selectors. This implementation gives you neither.

There's only one key exchange implementation that only works with Clipse. That's Freeswan. You don't want to use it: their funding ran out and the code is not under active development. It has one of the worst build systems I've ever had the displeasure not to get working. Freeswan has evolved into Openswan. Openswan seems to have been designed with some use cases in mind—rare in the IPsec community. Even more impressive, the use case seems to have something to do with what real users want to do. The configuration file is kind of special. It describes connections in terms of left side and right side instead of local and remote. The idea is that you can cut and paste configurations without swapping when you move from one side to another. I find I get tripped up by the Openswan configuration syntax a fair bit; it does not match my intuition. It is well documented and is about as much work to get working as Racoon which I discuss below. The real problem with Openswan is that the documentation
all assumes you are using Clipse. You can do that if you really like. However I find that I like the native IPsec engine better. Openswan works well with that; the active development seems to be done using the native stuff. However the documentation is mostly not updated and talks about things like binding ipsec interfaces to ethernets which only happens on Clipse. You get tripped up by differences in handling of MTU and other things. Also the debugging tools mostly don't work without Clipse. If the debugging tools were any good this might be a significant issue. The other problem with Openswan is that its policy language is as weak as Clipse even when using native tools. No port-specific packet rules for your connections.

Before I move on I should actually dismiss the other key exchange implementations. There is isakmpd, the OpenBSD IKE daemon. don't know much about it. There is Racoon, the Came IKE daemon. Racoon seems to be a fairly faithful IKE implementation. That is, it directly exposes aspects of the RFC in ways that probably make interop events easier, but make your life as a user horrible. You didn't actually care about the difference between a phase 1 and phase 2 SA, did you? Did you really want to have to specify policy both in the SPD and in a Racoon configuration file and keep them in sync? Sure, it makes it easier to have the separation between the kernel and IKE daemon, but is that really the user's primary concern? Racoon does have the nice feature of working and of mostly delivering on what IPsec promises to deliver. O, yeah, lots of debugging output.

Anyway, I ended up choosing Openswan. For those of you behind ted-williams you can stop getting your hopes up: ted-williams doesn't have a X.509 certificate so getting it working is more complex than what I did. It is possible; I know two people who have done it. I set up what looked like it should work and tried it. Once I remembered to enable IP forwarding, it even worked sort of. Machines behind the security gateway could see out to the Internet. Their traffic was encrypted. All is well until you try to talk to the security gateway. You see, the policy on the security gateway requires packets addressed to its inner address actually be encrypted. Yep, encrypted even if they are coming in on the inside interface.

Fixing the machines behind tunnel talking to tunnel server problem should have been easy. It was not. Openswan has this idea of a passthrough connection which allows packets to be sent cleartext. But it is a connection; it needs to have a left and right side. The left and right side cannot be the same nor they can be any of the standard Openswan wildcards. Also, you need to make sure Openswan never tries to authenticate the connection. Once you overcome all this, you realize that Openswan's Clipse heritage hates you. There is this complex mess of scripts that gets run everything time you do anything. They want to manipulate the routing table to make sure things go through the magic Clipse interfaces. They have mostly been hacked to not do too much damage in the native case. One case was missed. The script always removes the route to the destination because a new IPsec route will be installed later. So, you bring up the passthrough connection and presto! Your route to the local network goes
away. You didn't need that anyway. You can replace the goo of scripts with a call to /bin/true using the "leftupdown" configuration directive and all is mostly well.

You will have to do your own policy manipulation if you want some things natted. In general you will need to add an "out none" rule in the forward direction and a "fwd none" rule in the reverse direction as well as manipulating the POSTROUTING chain on the nat table.

Linux is sadly not really that far behind in the IPsec world. The only thing that is Linux-specific is that there are a lot of options to choose from. As far as I can tell all the operating systems I've tried to get IPsec working on—AIX, Cisco IOS, NetBSD—are about as complicated. My conclusion is that I'll keep using IPsec since I've got it working. However my needs would have been more easily met by an IPIP tunnel and some mechanism to deal with mobility. No, for my next project I will *not* get Mobile IP working.



I used IP advanced routing in my linux kernel and have for a couple years now to do port and host based natting (ssh/imap/telnet to cmu uses my tunnel; those to anywhere else do not). The iptables rules use marks to decide how to route.
This is exactly what I wanted to do, but the IPsec implementations make this more or less impossible. I currently have one rule in the nat POSTROUTING chain that basically says masquerade everything that gets that far. Then I have a bunch of SPD rules that keep some traffic from getting IPsec protection. There is an SPD rule that says treat everything with a particular fwmark specially. However I cannot get setkey to add a rule of that type. Also it would play poorly with the rules Openswan adds.


use routing for routing and firewalls for nat'ing, don't mix.

Your SPD rules are meant to just apply to packets that have a src/dst address that is from/to your /29. You add a default iptable rule to SNAT into your /29 and then add clauses before (use a chain) that rule to SNAT instead to your DSL WAN IP for 'bulk' egress traffic.

In tunnel mode you should *not* be thinking or using IPsec at anything other than Layer 3, if you are using it in transport mode then you can start being fancy.

To be honest it sounds like you are actually after mobile IP for ingress traffic and if your internal machine is initiating an outbound connection to be NATed to your DSP WAN IP. If this is the case focus around RELATED/ESTABLISHED/NEW rather than Layer-4 hooks.

Either way, leave the routing to the routing sub-system and the NATing to your firewall. From your co-location end, the routing table would have a 'catch all' /29 route pointing down the tunnel towards your DSL line; anything more specific would be caught at the co-location box end.


You've gotten this working? Rock! I need to get the config files from you at some point. I tried a couple of times, and gave up in frustration....