Home > Articles > Security > Network Security

Linux VPN Fundamentals

  • Print
  • + Share This
This chapter is from the book

VPN and Firewall Interaction

VPNs enable you to set up secure communications between endpoints and are just one weapon in your security arsenal. Firewalls, an older and more established security technology, are common in almost every environment. A VPN should be integrated into your security policy and that means making sure your VPN and firewall play nicely with each other.

Types of Firewalls

Three main types of firewalls are in common use today: packet filters, application gateways, and stateful inspection firewalls. They are described in the following sections.

Packet Filters

A packet filter is the simplest form of firewall. A packet filter firewall will compare any IP packet that attempts to traverse the firewall against its access control list (ACL). If the packet is allowed, it is sent through. If not, the packet filter can either silently drop the packet (DENY in ipchains speak) or send back an ICMP error response (REJECT).

Packet filters only look at five things: the source and destination IP addresses, the source and destination ports, and the protocol (UDP, TCP, and so on). These tests are very fast because each packet contains all the data (in the packet headers) necessary to make its determination. Due to its simplicity and speed, a packet filter can be enabled on your routers, eliminating the need for a dedicated firewall.

One problem with packet filters is that they generally do not look deeply enough into the packet to have any idea what is actually being sent in the packet. Though you might have configured a packet filter to allow inbound access to port 25, the Simple Mail Transfer Protocol (SMTP) port, a packet filter would never know if some other protocol was used on that port. For example, a user on one system might run his Secure Shell (SSH) daemon on that port, knowing that the traffic would be allowed by the packet filter, and be able to SSH through the firewall against policy.

Another problem with packet filters is that they are not effectively able to handle protocols that rely on multiple dynamic connections. The FTP protocol, for example, opens a command channel on which the various commands such as USER, RECV, and LIST are sent. Whenever data is transferred between the hosts, such as files or the LIST output, a separate connection is established. You would need to have an ACL that would allow these data connections through for FTP to work. However, packet filters do not read the FTP command channel to know when such an ACL should be permitted.

Application Gateways

An application gateway goes one step beyond a packet filter. Instead of simply checking the IP parameters, it actually looks at the application layer data. Individual application gateways are often called proxies, such as an SMTP proxy that understands the SMTP protocol. These proxies inspect the data that is being sent and verify that the specified protocol is being used correctly.

Let's say we were creating an SMTP application gateway. It would need to keep track of the state of the connection: Has the client sent a HELO/ELHO request? Has it sent a MAIL FROM before attempting to send a DATA request? As long as the protocol is obeyed, the proxy will shuttle the commands from the client to the server.

The application gateway must understand the protocol and process both sides of the conversation. As such, it is a much more CPU-intensive process than a simple packet filter. However, this also lends it a greater element of security. You will not be able to run the previously described SSH-over-port-25 trick when an application gateway is in the way because it will realize that SMTP is not in use.

Additionally, because an application gateway understands the protocols in use, it is able to support tricky protocols such as FTP that create random data channels for each file transfer. As it reads the FTP command channel, it will see (and rewrite, if necessary) the data channel declaration and allow the specified port to traverse the firewall only until the data transfer is complete.

Often there is a protocol that is not directly understood by your application gateway but that must be allowed to traverse the firewall. SSH and HTTPS are two simple examples. Because they are encrypted end to end, an application gateway cannot read the traffic actually being sent.

In these cases, there is usually a way to configure your firewall to allow the appropriate packets to be sent without interference by the firewall. You might hear this called a plug, which comes from the plug-gw, a part of the Firewall Toolkit (FWTK) that was used to connect a client and server directly when the protocol was not supported.

It can be difficult to integrate application gateways into your standard routing hardware due to the processing overhead. Some newer high-end routers are able to function as application gateways, but you'll need a lot of CPU power for acceptable performance.


Even application gateways can be fooled if you are crafty enough. For example, you could tunnel any arbitrary protocol over SMTP: The client could send data as the DATA portion of the transaction, and the server could respond in the resulting error/success code message. The nature and ubiquity of the HTTP protocol makes it even easier; SOAP and .NET are just two "accepted" examples of tunneling other protocols across HTTP. Httptunnel, available at www.httptunnel.com, is a good freeware tool capable of tunneling any protocol across HTTP.

Stateful Inspection Firewalls

Stateful inspection firewalls are a middle ground between application gateways and packet filters. Rather than truly reading the whole dialog between client and server, a stateful inspection firewall will read only the amount necessary to determine how it should behave.

Take the SMTP DATA command, for example. When this command is sent, the client will send the data (the text of the email) ending with a line containing a single ".". The server then responds with a success or error code. An application gateway will need to be reading all the data that is sent and looking for the "." and error code. A stateful inspection firewall, however, will realize that the client is sending data until the server responds. Thus, it will simply forward the client's packets without inspection until the server responds. Simply put, a stateful inspection firewall understands the manner in which stateful protocols must conduct themselves and manages that traffic accordingly within the confines of its rulebase.

By not reading all the data sent, a stateful inspection firewall achieves a significant performance gain over an application gateway while maintaining the higher level of security and protocol support. Our VPN traffic, however, will be encrypted end to end. As such, there will be very little that a stateful inspection firewall will need to look at in our VPN datastream—it can't inspect the actual data anyway. Because of this, there is no functional difference between a stateful inspection firewall and an application gateway for our VPN traffic. There is, however, a solid performance boost from using a stateful inspection firewall. Because our VPN is already introducing latency due to the overhead of encryption, the more performance you can get with your firewall, the better.

Stateful Inspection Concerns

Stateful inspection firewalls have suffered from occasional faults. For example, to support the FTP protocol, stateful inspection firewalls would look at the beginning of a packet to see whether a data channel was requested. An attacker could trick the server into sending a large error message containing a data channel request that aligned perfectly at the beginning of a second packet. The firewall would then open a channel dictated by the attacker. A program called ftpd-ozone (available at http://www.monkey.org/~dugsong/) can exploit this vulnerability. This particular problem has been fixed by firewall vendors, but there might be more lurking around.

Common Linux Firewalls

Several firewalls can run on Linux. We will briefly describe them here, but it is beyond the scope of this book to cover them in depth. See New Riders' Linux Firewalls, Second Edition by Robert Ziegler (2002) for more information on firewalls.

  • The Firewall Toolkit (FWTK). This was the first publicly available application gateway suite and was the basis of the commercial firewall Gauntlet. It is a set of userland applications that support various protocols such as STMP, HTTP, telnet, and X11. It is still available at http://www.fwtk.org, though it has not been officially supported in years.

  • IPF. A Linux packet filter for 2.0 kernels. If you are still using a 2.0 kernel, you should really upgrade.

  • IPChains. The Linux packet filter for 2.2 kernels. Though it is a simple packet filter, you can support some protocols via kernel modules. The ip_masq_ftp module enables you to support the FTP protocol, for example. The problem with IPChains is that the kernel packet filters are handled before the modules can see packets, meaning you must allow inbound access to ports that potentially could be required by the kernel modules.

  • IPTables. The Linux firewall software for 2.4 kernels, also known as Netfilter. IPTables supports both packet filtering and application gateway support together. Taking the FTP protocol as our example, the data channels that are used are accepted as RELATED packets. This makes your firewall much cleaner because you do not need to leave holes in your firewall for packets that might or might not be used by modules.

  • IPFilter. Created by Darren Reed, IPFilter is the default kernel packet filter of NetBSD and FreeBSD, it and runs on many other UNIX-like systems. IPFilter only runs on the very old Linux 2.0 kernels, so if you want to use IPFilter, we suggest that you use a BSD machine instead. It is available at http://coombs.anu.edu.au/~avalon/.

  • Note

    Darren Reed modified the IPFilter license (or clarified it, depending on your position) to deny modifications to the source, rendering it not completely open source. Because of this, it was removed from OpenBSD. Reed later allowed modifications when used as part of NetBSD and FreeBSD. OpenBSD is creating a new packet filter, PF, from scratch to replace IPFilter. Yes, now and then there doth be strife in the open-source community ...

  • Dante. Though normally included as part of a larger commercial firewall system, Dante is available as open source under a BSD-style license. It is a circuit-level firewall/proxy that is largely transparent to users. It is available at http://www.inet.no/dante/.

  • T.REX. Open sourced in April 2000, T.REX is a sophisticated application gateway firewall that includes intrusion-detection features, strong authentication support, and extensive logging. Version 2 of T.REX was not yet available at the time this was written; though precompiled versions are available on CD. You can obtain T.REX at http://www.opensourcefirewall.com.

Placing Your Firewall

We strongly suggest that you use a firewall as part of your security infrastructure. Using a VPN in conjunction with your firewall, however, requires careful planning and configuration. Several different configurations are available when using both a firewall and a VPN server. Each has its pros and cons, and we'll do our best to help you pick the option that makes sense for you.

VPN Server on a Firewall

The solution that feels most natural is to install your VPN software on your firewall itself. Many commercial firewalls include VPN components as an extra option. If you are working in a mixed commercial and freeware VPN environment, you will likely end up supporting this configuration.

As seen in Figure 2.4, we have a single point of entry into our network, which serves three purposes:

  • The firewall allows outbound access to the Internet.

  • The firewall prevents inbound access from the Internet.

  • The VPN service encrypts traffic to remote clients or networks.

Figure 2.4 The VPN server on a firewall.

The pros of putting your VPN on your firewall are as follows:

  • You have one place controlling all your security, meaning fewer machines to manage.

  • You can create firewall rules that apply to your VPN traffic using the same tools you already use to manage your firewall.

The cons of putting your VPN on your firewall are as follows:

  • You have one place controlling all your security. You'd better make sure this machine is extremely secure.

  • You must configure your routes carefully to make sure the traffic goes through the appropriate interfaces (eth0 vs. vpn1, for example.)

  • Improper configuration in your firewall rules could allow traffic from the Internet to get through to the inside by slipping through using VPN addresses.

  • Your Internet and VPN traffic will compete for resources on the machine, so a larger machine is likely necessary.

VPN Server Parallel to Firewall

Another topology that seems logical is to use your VPN parallel to your firewall. Your internal machines will still point to the firewall as their default route. The firewall will have a route to the VPN-accessible networks via the VPN server and will inform clients to send packets to the VPN machine when appropriate. (The "Routing" section later in this chapter discusses how this works.) You can see this topology in Figure 2.5.

Figure 2.5 The VPN server parallel to a firewall.

If you prefer, you can place a router between the internal network and the VPN and firewall machines. You would configure the router to know the networks accessible through the VPN and would set up your routing rules there instead of on the firewall.

The pros of putting your VPN server parallel to your firewall are as follows:

  • Your VPN traffic is not going through the firewall at all, so you do not need to modify your firewall configuration to support the VPN packets. Because some VPN protocols are not supported by all firewalls, this might be your only option.

  • You can scale much easier. If you find that a VPN server is under too much load, you can add machines and distribute the VPNs that can be established between them. If you are connecting multiple remote networks, you can have one VPN machine per network.

  • If you are supporting roaming users in a host-network configuration, you can simply add VPN servers and use round-robin DNS to distribute the load between the VPN servers.

The cons of putting your VPN server parallel to your firewall are as follows:

  • Your VPN server is directly attached to the Internet. You had better be very sure that it is well secured; otherwise, an attacker could break in and have direct access to your internal network.

  • You now have two machines that communicate with the Internet, and you must make sure that both are correctly configured to only pass legitimate traffic, thereby increasing the support workload and associated costs.

  • Cost, of course, increases incrementally with the addition of new servers and related support staff.

VPN Server Behind Your Firewall

Another location for your VPN server is to put it behind your firewall completely, attached to the internal network. As shown in Figure 2.6, our VPN server is not directly reachable from the Internet at all, and all packets must reach it through our dedicated firewall.

As with the previous topology, you will need to add a route to your firewall that redirects VPN traffic from internal machines to the VPN server. You will also need to configure your firewall to pass the encrypted VPN traffic directly to your VPN server.

Figure 2.6 The VPN server behind a firewall.

The pros of putting your VPN server behind your firewall are as follows:

  • The VPN is completely protected from the Internet by the firewall.

  • There is only one machine controlling all access to and from the Internet.

  • Network restrictions for VPN traffic are located only on the VPN server, which can make writing rulesets easier.

The cons of putting your VPN server behind your firewall are as follows:

  • All the VPN traffic must travel through the firewall as well, adding a degree of latency.

  • The firewall will need to shuttle VPN traffic from the Internet to the VPN directly. Because it cannot inspect this traffic (it is encrypted, after all), it will require a simple packet filter ACL or application plug proxy.

  • Getting your firewall to pass the encrypted VPN traffic to the VPN server is sometimes tricky. Some firewalls do not know what to do with IP protocols other than ICMP, TCP, and UDP. This means it might be harder or impossible to support VPNs that use different IP protocols, such as ESP packets for IPSec VPNs or GRE packets for PPTP VPNs.

  • All the VPN traffic travels through the same LAN wire twice, once from client to VPN server in the clear and once from the VPN to the firewall encrypted. This can degrade LAN performance.

One option to decrease latency would be to connect a second network card (eth1) on the VPN server directly to a network card on the firewall (eth2) with a crossover cable. You could use a hub or switch and create an actual network segment if you prefer, but because you'd only have the two hosts here, a crossover cable would work as well and would be faster because no extra equipment is involved. This enables you to route the encrypted VPN traffic directly over this wire and through the firewall rather than backtracking over the existing LAN segment. This means the VPN traffic won't be competing for LAN resources.

The direct VPN-to-firewall link will need to either be a point-to-point link or be given its own network segment. If you prefer the latter, we suggest a small network such as a /252, which can hold only two hosts, plenty for our purposes. We could allocate to the firewall and to the "external" VPN interface, and the network itself would be

Configuring Your VPN with Its Own Firewall

In any of the configurations previously described, it is possible to restrict what traffic can traverse the VPN connection. This might be desired when the networks or hosts you are connecting are not of the same trust level.

For the case in which your Internet firewall and VPN server are the same machine, this can be achieved simply by adding new rules to the firewall using your existing firewall software.

For cases when you have a separate VPN server, you might either install a separate firewall machine in front of your VPN server or simply rely on Linux kernel packet filters on the VPN server itself. For example, if you want to allow only mail-related traffic to traverse your VPN, you might implement kernel restrictions on your VPN server, such as the following:

# Allow only SMTP, POP, IMAP and POPs/IMAPs through our VPN: for port in
25 109 110 143 993 995
 ipchains -A output —destination-port $port -i vpn1 -j
 ipchains -A input   —source-port $port -i vpn1 -j ACCEPT
ipchains -A output -i vpn1 -j DENY
ipchains -A input -i vpn1 -j DENY

These rules assume that your VPN traffic goes through the virtual interface vpn1. This is a very simple example. You can create ACLs tailored for your VPN using whatever tools (IPchains, IPtables, IPfilter, and so on) you are comfortable with.

  • + Share This
  • 🔖 Save To Your Account

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information

To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.


Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.


If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information

Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.


This site is not directed to children under the age of 13.


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information

If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information

Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents

California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure

Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact

Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice

We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020