Rudimentary IP Dump

Tuesday, 20. July 2010

Would you like to dump an IP interraction without much fuss? If you have netcat, it’s remarkably easy.

cat |tee -a smtpdump | nc mail.linuxcommando.com 25 | tee -a smtpdump

your exchange, in addition to taking place on your terminal, will also be appended to ./smtpdump, assuming your version of tee matches mine (GNU Coreutils 8.5). If you can speak SMTP, you can talk to the mail server and deliver me a message.

Exchange the cat for another netcat, listening on a local port:

nc -l localhost -p 2500 |tee -a smtpdump | nc mail.linuxcommando.com 25 | tee -a smtpdump

You can then telnet into localhost:2500 and record the entire conversation in smtpdump. With a little imagination, you can come up with all sorts of fun ways to use it.

Use df and GNU sort to investigate hard drive usage

Friday, 16. July 2010

The standard UNIX utility df and and GNU’s sort implementation can be used in conjunction as a handy way to explore hard drive usage.

du -hs * | sort -h

This command will sort all files in the current working directory by human-readable size. It’s very convenient if you happen to be wondering where the bulk of your hard drive space is going. The trick here is the use of -h to both df and sort; BSD implementations of df faithfully accept -h, but BSD sort dosen’t accept -h. On BSD, you’ll have to du -s * | sort -n instead and look at big numbers.

The Danger in Requiring Excessive User Input

Saturday, 26. June 2010

Web 2.0 and modern Internet technologies have capitalized on the value of user input. Content driven by the huge number of users, rather than the small number of writers and maintainers, provides a number of substantial benefits. Firstly, the volume of user generated content is typically far greater than the volume of maintainer-generated content. The sheer number of users makes them a valuable source of content as well as moderation. Furthermore, only the users themselves (in the collective sense) are capable of providing metrics for trends and public opinion, as well as any information about users.

Unfortunately, the undeniable value of user supplied information often spurs executives, marketers, and site owners to take the concept too far. Perhaps the most glaring and dangerous case of excessive user input is requiring such input from all users when its accuracy is irrelevant to the user and unmeasurable by the site owner or maintainer.

Take as a typical example a web contact form that requires users to enter a mailing address and phone number, as well as name and email. Users have gradually become comfortable with the concept of providing an email address to websites; and when they are communicating online, can easily appreciate the value of this information. However, they are likely to be cautious of providing their address or phone number online. The volume of unsolicited sales calls and junk mail may dissuade them from providing such information, or perhaps they are simply sensitive to the privacy and security of their personal information. For whatever reason, they have decide not to provide their phone number and mailing address.

The site owners have told their design and development team to require this information, because they see the value of geographically locating prospective customers and being able to follow up with a sales call. It is easy to see the benefit of having such information available. Unfortunately, content owners may be less sensitive to the concerns of consumers who don’t care to share such information. Furthermore, they haven’t realized that the value of this information is nullified if the information is wrong. They can’t of course verify the users’ mailing address; if they had some way to do so, they wouldn’t need to collect it. And the site users are unlikely to see the benefit for them in providing personal data that the website doesn’t need to perform whatever service they need, a large number of customers are likely to provide inaccurate data, typing in a falsified mailing address and phone number. They are also likely to be irritated by the attempt to collect their own data. In the modern age of identity theft and massive marketing communications, most Internet users have become wary of providing more data than they have to.

By unintentionally requiring unverifiable user data, the site owners have made their objectives virtually unachievable. Since they can’t verify the user provided data, they can’t trust it. Since they have required all form submissions to include the data, but the users don’t benefit from providing that data accurately, they’ve effectively invited their users to falsify the information. And so, just as they wanted, they’ve collected far more information than they would have otherwise obtained; but they have no reason to think that information is accurate. They may as well have just chosen random mailing addresses. Some of the form’s data might be accurate, but without a mechanism to verify which submissions are accurate and which are falsified, this data is worth nothing.

Requiring excessive user data is dangerous. It devalues the data that’s being collected and alienates many Internet users.

On the other hand, optional data provision is likely to result in much more accurate data, albeit in lower quantities. By making, say, a phone number optional, you give the users who don’t care to provide it an option other than falsification. Therefore the vast majority of phone number people choose to provide are likely to be accurate. Some might not realize that it’s not required, and providing incorrect information might appeal to others, but these groups are likely to comprise only a minority of internet users. (Who enjoys filling out forms inaccurately to the extent that they’d seek them out just to fill them out with misinformation?) Furthermore, making non-required data optional shows a level of respect for internet users who are sensitive to privacy concerns, or just don’t like filling out forms. In exchange, the site owners benefit from a high degree of confidence in the user supplied data and a better relationship with their users.

Another good example can be made of the observation that inspired me to write this post, my first addressing my own web design philosophies. A client’s website required users to input a rating for the site’s content before accessing more content. Ratings, of course are more or less impossible to verify. The client is forcing any users who don’t have a strong opinion about the content to make up a rating. The site owners may be pleased by the level of feedback they obtain, but only because they can’t be convinced that it’s essentially meaningless; they may as well randomly choose a rating whenever the content is viewed, because in that case at least they wouldn’t be alienating their users.

User information is valuable, but only insofar as it is accurate. That is exactly why required fields are appropriate only for data that must be collected. All other data submission should be optional.

A Review of Delta In Flight WiFi

Wednesday, 2. June 2010

I’m writing this post from an airplane.

I’ve taken advantage the opportunity to test the Delta In-Flight WiFi.  32,000 feet above the ground, my signal is carried by AirCell LLC from the plane all the way down to Earth.  AirCell’s website explains that the connection is uses a special form of 3G to span the few miles from Earh to the plane on exclusive frequencies purchased at auction from the FCC in 2006. The ‘last mile’ so to speak is of course covered by an in-plane access point. There appear to be two such access points in the plane, as well as 13 other APs that, I can only guess, are used by the plane itself for something. Aircell seems to be using an AT&T line to the Internet on it’s side.

Prices for the service are, as you would expect, somewhat exorbitant; I paid $10 for the privilege of being online during my 3 hour flight (though, to be fair, the price would have covered a longer trip as well, from what I can tell; it may be a 24 hour pass.

I’m happy to announce that the service is acceptably good. The standard internet speed test at speedtest.net provided a ping latency of about 200ms, download speeds of 500 to 1000 Kbps, and uploads of about 330 Kbps, with otherwise small deviations between a few test sites.

Furthermore, AirCell has been very kind about port access. I’m happy to report that the connection does not firewall outgoing ports; SSH, FTP, HTTP[S], DNS, IMAP[S] and even the default SMTP port, 25, have been left open. There doesn’t appear to be any sort of firewall in place whatsoever; my SSH and VPN connections are functioning normally and I’ve been able to successfully use a mail client. All systems are go!

Once very nice feature of the AirCell system is real time flight tracking. Simply enter the airline identifier and flight number and you can track your plane’s progress accross the country on a Google map. This feature is intriguing, no doubt, to anyone who, like me, is constantly looking down through holes in the clouds and wondering what part of the country he’s seeing. Trip details like scheduled and estimated arrival time, and airplane positional data including distance to and from the destination and origin respectively, is also available. The google map is a bit small, and appears to be implemented with the older version of their API (remember Hybrid view?), but is a very nice feature. Although it’s difficult to pinpoint specific landmarks like lakes and roads, the map does an excellent job of giving passengers a general idea of where their plane is and what larger metropolitan and geographic landmarks might be seen from the window.

All in all, I’d say that, for those who cannot afford or stand to be disconnected for the duration of their flight, the in flight wifi is a reasonable proposition. The price is high, but, given that the service does seem to be exclusive to AirCell, could be higher. $10 for a new technology – or at least a new service – such as this does not seem unreasonable. The connection speeds aren’t great, but they aren’t intolerable either. I haven’t had any connection problems yet. Good job, AirCell!

Linux 2.6.34 Released

Wednesday, 19. May 2010

Kernel 2.6.34 was released Monday! Here’s an overview of some of the changes from the changelog.

  • Improvements for ARM, MIPS, Microblaze, powerpc, x86, amd, k8, x86_64, m68k, sparc64, and more
  • Lots of Virtualization and Virtualization-related fixups
  • Additional ALSA hardware support
  • Synaptics, HID fixes
  • Ceph added to kernel
  • Fixes for OCFS2, CIFS, Coda,XFS,  NFS3/4, btfrs, squashfs, reiserfs, ext2/3/4  and others
  • i915, radeon graphics driver fixes
  • MMC improvements
  • SCSI, Firewire, Infiniband  &  PCMCIA bugfixes
  • Support for various Realtek USB wireless variants
  • Addressing recent Database performance regressions (Yay!)
  • Merge of many branches, including a big V4L merge
  • Fixes for many recent regressions
  • Improvement to wireless stack
  • Fixes for e1000, rtl8169,tg3, iwlwifi, forcedeth, tun, ath9k, ath5k, b43, bonding, bridge,  and other network drivers
  • Manufacturer and device IDs have been added to many drivers (more hardware support!)
  • Fixes for MD, Raid
  • ACPI, PM, & Suspend improvements
  • Network stack bugfixes across the board
  • i2c, hwmon fixes
  • More and better support for input devices
  • Documentation updates
  • Much needed attention to Staging drivers
  • Many USB improvements and device additions

Additionally, the release addresses many more obscure, unintelligible and uncommon bus that I didn’t mention. I only tried to overview some changes that are likely to affect the masses.  The changelog for this release is 6.4 MB, so you’ll have to forgive me if I missed anything noteworthy.

It’s too early to say, but so far, this kernel seems to be doing better than ever.  My ath9k chip appears happier than ever before, and maybe it’s just my imagination, but it seems as if the recent increase in suspend time has been addressed.  Suspend has always been a little unstable on my netbook, so I’ll have to wait and see if thing seem to do better now.

All in all, 2.6.34 looks like, and feels like,  a good step forward.  Why not try it today?

Set Date with SSH

Wednesday, 12. May 2010

If ntpdate isn’t feasible, and you need only a few seconds worth of accuracy, this one-liner can set the date over the network with ssh.

ssh root@$HOST "date `date +%m%d%H%M%y.%S`"

Load all modules at a location

Tuesday, 4. May 2010

If you’d like to load all modules under a particular path, you may use the following command.

loc="/lib/modules/`uname -r`/kernel/net/netfilter";
for i in `find $loc -iname '*ko'`; do
mod=`basename $i|sed s/\.ko//`;
echo -n $mod...;
sudo modprobe $mod;
echo loaded;
done

This command includes a sudo, which you’d want to remove if you don’t use it. (Clearly, you’d have to run it as root in that case.)

Linux CPU Frequency at Command Line

Monday, 3. May 2010

If you’d like to determine your CPUs’ frequencies at the command line, simply run this (sudo included):

sudo bash -c 'for i in `find /sys/devices/system/cpu/ -regextype posix-extended -iname "cpu[0-9]*"`; do echo -n `basename $i` ": "; cat $i/cpufreq/cpuinfo_cur_freq; done;'

If instead you’d like a continuous display, try this:


sudo bash -c '(while [[ -f /tmp/adf ]]; do clear; for i in `find /sys/devices/system/cpu/ -regextype posix-extended -iname "cpu[0-9]*"`; do echo -n `basename $i` ": "; cat $i/cpufreq/cpuinfo_cur_freq; done; sleep 1; done)'

The above code shows an example of controlling a while loop with the existence of a file. For the while loop to run at all, the file must be created (touch /tmp/adf); to stop the loop, the file need simply be removed (rm /tmp/adf).

Protect your Network with a Stateful Firewall

Sunday, 2. May 2010

Introduction

You can increase security in your network by implementing a stateful firewall in iptables. A stateful firewall can track the state of connections so that established connections can be differentiated from new connections. This facility allows iptables rules to deny connections on all ports by default, and allow traffic on higher ports only if that traffic originated on one of the ports that are explicitly opened.

Such a firewall protects from connection hijacking (as the traffic from the hijacker’s IP would be categorized as NEW and dropped), and since ports can be closed by default, the system administrator can turn on any services they desire, secure in their knowledge that the outside world can only access the services they specifically allow.

Preparation

There’s just a few things we need to do before we’re ready to go.

Assumptions

This configuration should work as-is for almost everybody, but I must assume that you:

  • Are configuring iptables manually. I am not familiar with Shorewall or other iptables configuration utilities; but I think you’ll find iptables straightforward and extra utilities an unnecessary complication, if you’re familiar with IP networking.
  • Have only one network interface on an untrusted, external connection. I call this your external interface, $EXTIF. Typically this would be the interface with a public IP address.

    If all of your IP addresses are RFC 1918 private addresses, this would be the interface that corresponds to the default route (in this case you are almost guaranteed to be firewalled anyway, since your private addressesmust go through a Network Adress Translating router, but you can still follow the direction here to firewall this connection if it’s untrusted).

    If you have more than one untrusted connection, you’ll have to duplicate the rules for the trusted connection (since negating $EXTIF won’t define an exclusive list of trusted networks), but, since you’ve already become familiar with the process of configuring multiple public connections, you won’t have a problem with this modification.

  • Trust all traffic on all interfaces other than $EXTIF. This allows us to make one rule for all such trusted connections. If this isn’t true, you’ll have to duplicate some of the rules for each trusted network because they cannot be defined as ‘all interfaces that do not match $EXTIF, but this is not difficult.
  • Use bash. A few of the commands won’t work on other shells – you’ll have to write individual rules to take the place of the bash for loops, but this is trivial.

Before you Begin

To get the most out of this howto, you’ll need to know

  • The external interface you’ll be firewalling
  • The ports you’d like to leave open to the world
  • The addresses of servers behind the firewall, to which you’d like to forward ports (if any)
  • Whether you’d like to enable IP Masquerading so that other computers can connect to the internet through your firewall.

Kernel Configuration

Your kernel must be configured to support ipTables, connection tracking, and masquerading.

[*] Networking support  --->
  Networking options  --->
    [*] Network packet filtering framework (Netfilter)  --->
      Core Netfilter Configuration  --->
        < M > Netfilter connection tracking support
          < M >   FTP protocol support [ if masquerading ]
            [ Enable other protocols you use, if masquerading ]
          {M} Netfilter Xtables support (required for ip_tables)
            < M >   "state" match support
    < M > IPv4 connection tracking support (... [ if masquerading ]
    < M > IP tables support (required for filtering/masq/NAT)
      < M >   Packet filtering
        < M >     REJECT target support
        < M >   Full NAT [ if masquerading ]
          < M >     MASQUERADE target support
          < M >     REDIRECT target support

As you can see, some of those are only required for masquerading; if there are computers behind your firewall that will connect to the Internet or provide services through your firewall, you’ll need to enable at least most of the [ masquerading ] options. The configuration shown here uses modules, so no additional memory
will be wasted if you configure options you aren’t using.

IPTables Starting Point

We will flush all tables and set their default policies to ACCEPT. We do this so we have a fresh starting point.

iptables -P INPUT ACCEPT
iptables -F INPUT
iptables -P FORWARD ACCEPT
iptables -F FORWARD
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT
# for masquerading/DNAT
iptables -t nat -P PREROUTING ACCEPT
iptables -t nat -F PREROUTING
iptables -t nat -P POSTROUTING ACCEPT
iptables -t nat -F POSTOUTING
iptables -t nat -P OUTPUT ACCEPT
iptables -t nat -F OUTPUT

Now any traffic will be accepted and we’ve cleared out all iptables rules. We’re ready to start the configuration.

Configuring iptables

Defining Your Settings

Here’s where you’ll be using your knowledge of your system(s) to personalize your configuration. To accomplish this, I’ll provide a few environment variables whose values you should alter to fit your configuration.

# set this to whatever interface is external (see above).
EXTIF="eth0"
# set this to the list of ports you'd like to accept traffic on.
# this default list allows SSH and HTTP.  You might want to add others
# to this list.
PORTS="22 80"
# an example that also opens HTTPS, DNS, IMAP/IMAPS
# PORTS="22 80 443 53 443 143 993"

Execute these lines in your shell to set environment variables specific to your configuration; then you can copy and paste the lines below verbatim.

INPUT

The input table is used for any incoming IP traffic, including traffic that will eventually be forwarded through to other servers behind the firewall. The majority of settings will be put in this table, as it’s the first line of defense in disallowing undesirable traffic.

Allow all traffic on any but the external interface

We accept any traffic that doesn’t come in on the external interface, opening all ports by default on all interfaces that don’t match $EXTIF.

iptables -A INPUT -i ! $EXTIF -j ACCEPT
Allow Acceptable Traffic

First we allow traffic destined for the ports we’d like to open.

for p in $PORTS; do iptables -A INPUT -p tcp --dport $p -j ACCEPT; done;

Established connections between public hosts and computers behind the firewall will use high level ports to communicate. Some services might also move to a high level port to allow the port associated with the service to be kept open. In both these cases, we’d like to allow incoming traffic only if it is related to an established connection. This is the iptables rule that makes this configuration stateful.

iptables -A INPUT -i $EXTIF -m state --state ESTABLISHED,RELATED -j ACCEPT

Finally, you may allow ICMP traffic if you like. Dropping all ICMP traffic is the safest option, but many servers (including Google’s) reply to ICMP traffic, at least to PINGs. To do the same, execute the following command.

iptables -A INPUT -p icmp -j ACCEPT
Drop all other traffic

We now set a default policy to DROP all traffic that doesn’t match the above rules. This disallows traffic going to ports we didn’t explicitly open as well as traffic that speaks to a port that was opened for another connection (eg a hijack attempt).

If you are logged in via SSH, you would be wise to ensure that you haven’t forgotten to allow your own connection. I generally run the following command first. If my ssh session doesn’t lock up for 10 seconds, I can enable the policy permanently. If you’re at a local console, and not logged in over the network, this failsafe does nothing for you.

iptables -P INPUT DROP; sleep 10; iptables -P INPUT ACCEPT;

If your terminal doesn’t lock up for 10 seconds, it means that your connection wasn’t interrupted by the DROP policy and you can enable it permanently.

iptables -P INPUT DROP;

OUTPUT and FORWARD tables

At this point we’re already dropping incoming connections we don’t want. This generally ensures that any traffic that gets through the INPUT is meant to be allowed. Therefore, you may leave the FORWARD and OUTPUT tables empty with a default policy of ACCEPT, if you’d like to.

However, you might want to make extra sure that you won’t forward anything through to internal networks. This is an appropriate configuration if you have a number of servers on external IPs, that also use a private, internal network on the back end for high speed communications between servers. In this case you might want to make sure that you aren’t functioning as a router for these other servers; you don’t want to FORWARD packets through to internal networks.

# drop packets from the public network to internal networks.
# do NOT do this if you want to configure a router / gateway!
iptables -A FORWARD -i $EXTIF -o ! $EXTIF -j DROP

You could use the OUTPUT table to disallow packets based on the port to which they’re being directed. This might be desirable in some situations, but typically you’ll just want to leave OUTPUT empty with a default policy of ACCEPT.

NAT Tables

If you’d like to redirect packets to other servers behind the firewall (DNAT), or allow internal computers to connect the the Internet through your firewall (Masquerading), you’ll also want some rules in your NAT table. (if neither applies to you, you may leave all the NAT tables empty with a default policy of ACCEPT).

Allow Computers to Connect to the Internet through this Firewall

If you wish internal computers to connect to the Internet through this firewall, you may enable IP Masquerading. In this case, all traffic originating on the internal network will be altered by the firewall to appear to originate from the firewall itself. When response traffic is returned, the firewall performs reverse alterations and forwards the packet to the real origin. In this way, the firewall performs NAT, making all public computers unaware that they are communicating with a system behind the firewall, rather than the firewall itself.

# masquerade as the origin of all traffic leaving on the external interface
iptables -t nat -I POSTROUTING -o $EXTIF  -j MASQUERADE
Clone One Open Port on Another

In some situations, you may want to redirect traffic coming in on port A to port B. Consider an example; you run a mail server on port 25, but sometimes have to submit mail to the server from systems behind firewalls where port 25 connections are blocked. Blocking port 25 outgoing is common for public networks – internet cafes and the like – as well as many residential ISPs. In this case, you might want to open a diffrerent port – say, 26 – that you’d like to be redirected to port 25 so you can submit mail on either 25 or 26. You can generally configure a mail server to listen on more than one port, but an iptables rule is easier.

Case I: You’d like to redirect traffic to another port on the firewall itself

In this case you’d like to redirect the traffic to the firewall. Hint: this is also what you want if you’d like to force outgoing traffic through a proxy, but you’ll have to modify the rule!

# we modify the traffic pre-routing, effectively routing to ourselves instead
iptables -t nat -I PREROUTING  -p tcp --dport 26 -j REDIRECT --to-ports 25
Case II: You would like to pass incoming connections through to servers behind the firewall

Perhaps your servers are behind the firewall, but you’d still like them to server public connections. To do so, you’ll have to add a slightly different rule. For the purposes of our example, we’ll redirect traffic on port 221 to an internal host, 192.168.125.22 port 22, enabling the admins to log into the machine over SSH from the outside world. You might find this convenient for many of your firewalled hosts!

iptables -t nat -I PREROUTING -i $EXTIF -p tcp --dport 221 -j DNAT --to-destination 192.168.125.22:22

Traffic coming in on $EXTIF on tcp port 221 to 192.168.125.22:22.

Hint:, if the incoming port (221, here) and the real destination port (22, here) are the same, you need only specify the IP address as a value to --to-destination. The default destination port is the same port as specified for --dport.

Allow Forwarding

To make use of this rule, your firewall must be allowed to forward IP traffic between interfaces. This is typically disabled by default since it can expose networks that were meant to be isolated. You may enable it with sysctl:

sysctl net.ipv4.ip_forward=1

or with a simple echo, through /proc:

echo 1 > /proc/sys/net/ipv4/ip_forward

In either case, you’ll probably want to save this configuration in /etc/sysctl.conf so that it is re-enabled each time the system boots.

# the following line could be added to /etc/sysctl.conf
net.ipv4.ip_forward = 1

Wrapping Up

Your firewall is now configured. However, you’ll want to ensure that the iptables rules you’ve just configured will persist between reboots. Each distribution has a different way of doing this; if you like, you can simply put your commands in a local runscript. In Gentoo, my distro of choice, you would execute the following.

/etc/init.d/iptables save

Gentoo’s init system will restore your rules when it reboots, as long as you make sure the iptables initscript is in your boot or default runlevel.

On other distributions, you’ll have to find your own way of accomplishing this.

Conclusion

Now you’ve set up your very own stateful firewall. Your traffic will be sanitized and you can start services with reckless abandon, knowing that you’ll have to open ports before those services are externally accessible. And now that you’ve been introduced to iptables, hopefully you’ll be able to take advantage of the many excellent applications of this excellent firewall solution.

Synaptics Tap-To-Click in Xorg 1.6

Tuesday, 27. April 2010

In an archive of a poorly cited discussion on the debian bugs mailing list, are listed the magic words to enable tap-to-click on synaptics touchpads under newer versions of Xorg.

Try out synclient TapButton1=n, where
n can be:

  1. Left Mouse Button
  2. Middle Mouse Button
  3. Right Mouse Button

Typically, you would want to use synclient TapButton=1 so that a tap registers as a left mouse button.