SPF and DKIM

Wow, has it been THAT long since my last post ? Been moving house and now I'm a certified Telecommuter, 100% of my Job done from my lovely home office at the beach.

Anyways, I had to do a large mailshot to about 1.4 million customers, and in the runup, to increase delivery rates, I instituted SPF and DKIM.

A quick explanation of each:

SPF - Sender Policy Framework
DKIM - Domain Keys Identified Mail

Both systems are a way of making sure that mail sent by a particular server is from the correct domain in question. Both are heavily used in the enterprise, and having them on your mailserver will increase hit rates.

SPF is easy, you just need to add a TXT DNS record to your domain, like so:

v=spf1 ip4:11.11.11.11 ip4:22.22.22.22 mx ptr mx:somedomain.com include:somedomain.com ~all

lets break it down:

v=spf1 Version (currently only v1 exists).
ip4:11.11.11.11 ip4:22.22.22.22 If source mail server is IP 11.11.11.11 or 22.22.22.22 then it is authorised.
mx ptr if source IP has reverse lookup to this domain then it is authorised.
mx:somedomain.com include:somedomain.com source domain authorised to send mails on behalf of this domain.

DKIM is a bit harder, first you need an MTA that supports it, I've used Merak 9 (which has it built in) and postfix (with the dkim-milter plugin).

The Logic is that you generate a keypair, the private part of which is integrated into your mailsystem, and the public part in a TXT DNS record on your domain.

The receiving MTA looks up the TXT record, finds the public key, and checks with the sending server through keypair handshake to confirm that it IS in fact the correct sender. If it is, the mail is accepted.

The source MTA adds headers like this:

X-DKIM: Sendmail DKIM Filter v2.8.2 sender1.yourdomain.com 75866730012
DKIM-Signature: v=1; a=rsa-sha1; c=simple/simple; d=yourdomain.com;
s=default; t=1239981026; bh=+NNkD6jOlYKtY2AIGNRToH2tkm0=;
h=Date:List-Help:List-Subscribe:List-Unsubscribe:List-Owner:
List-Post:From:Reply-To:To:Subject:MIME-Version:Content-Type:
Message-Id;
b=MrjXBShjNexWy62fC4Uu7xS3Hxav+cHtqIBzwMlcufadsffLtW9KmF5sO58+yHjyy
I3SiX0TNyEbvXtSHvRKm9z630zDiN0dxVXGqhgEfdklaj4jlkfhR6GrsRgzW2YOW6/9
sKFnz214AkhAPrFBD30hNmZfRfY75v5q94FnGDUo=

and the domain TXT record is setup thus:

v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GWETBNiQKBgQC5KT1eN2lqCRQGDX+20I4liM2mktrtjWkV6mW9WX7q46cZAYgNrus53vgfl2z1Y/95mBv6Bx9WOS56OAVBQw62+ksXPT5cRUAUN9GkENPdOoPdpvrU1KdAMW5c3zmGOvEOa4jAlB4/wYTV5RkLq/1XLxXfTKNy58v+CKETLQS/eQIDAQAB


Where:

v=DKIM1 Version 1 of DKIM
k=rsa Keytype DSA or RSA
p=xxxxxx The actual public key

These two techniques, along with actually having proper forward and reverse lookups on your mailserver will increase delivery rates and decrease spam scores, making sure your mail ends up in the inbox, and not junkmail.

Tool of the Day: TCPMSSD

I've had issue with an ADSL multilink implementation over a FreeBSD firewall, that just wont seem to work with a particular PC's MTU. I've of course changed the pc MTU with TCPOptimizer, but now I've gone a level up and am actually mangling the packets with a tcp MTU/MSS Clamp.

The Clamp works by making sure that all packets are lower than or equal to a particular MTU, therefore making sure traffic flows correctly.

I installed TCPMSSD from FreeBSD ports /usr/ports/net/tcpmssd, and made sure the daemon starts from rc.local, with the following command line:

/usr/local/bin/tcpmssd -b -p 7777 -m 1300

-b = mangle both SYN and ACK packets
-p 7777 = run the Daemon on port 7777
-m 1300 = MTU size of 1300 Bytes

Then, you need to pass traffic to the daemon through IPFW:

add 00042 divert 7777 ip from any to 10.0.0.1 (mangle traffic to source)
add 00043 skipto 00047 ip from 10.0.0.1 to 10.0.0.0/20 (skip internal traffic)
add 00043 skipto 00047 ip from not 10.0.0.1 to any (skip anything else other than this host)
add 00044 divert 7777 ip from any to any (mangle traffic from source)
add 00045 divert 8670 ip from any to any (NATD traffic from source)
add 00046 fwd 192.168.0.1 ip from any to any (FWD and end traffic from source)

this is tested, working and in production on my live FreeBSD firewalls, a great little tool...

Tool of the Day: TCP-Z Windows Vista Half-Open Connection Patch

To speed up Vista internet network speeds, you can now run a simplified tool, that modifies kernel values on the fly, without changing any settings permanently.

You can download it here: Softpedia

If you WOULD like to change the settings, follow microsofts article on how to HERE

Either way, look forward to increased network performance in Vista/Windows 7

Increasing performance of Static IP (L2TP) ADSL Lines

With most of the South African ISP's offering static IP ADSL, I Think there is some confusion or just plain not knowing how these offerings work. The machines connected to these Routers benefit from having correct MTU Size set, both to reduce packet fragmentation, and also allow packets to traverse if they have DF Bit set (DF = Do Not Fragment).

Standard Ethernet MTU is 1500 bytes, and ADSL PPPoE MTU is 1492 Bytes. What ISP's do is run an L2TP Tunnel from the ISP to the ADSL Router, presenting the user with static IP's, but reducing MTU size to 1472 Bytes (Standard L2TP Data segment).

To optimize your Linux/BSD machines, the easiest option is to just adjust the mtu thus:

ifconfig em0 mtu 1472

That will make sure all packets originating from that interface are the correct size. The issue arises if you use windows machines, as MTU Discovery does not always work correctly. This is tried and tested on a windows box I have here:

Ping Google with a 1492 Byte ICMP Packet:

C:\Users\kim.attree>ping -l 1492 www.google.com

Pinging www.l.google.com [74.125.45.103] with 1492 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 74.125.45.103:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

And Pinging Google with a 1472 Byte ICMP Packet:

C:\Users\kim.attree>ping -l 1472 www.google.com

Pinging www.l.google.com [74.125.45.103] with 1472 bytes of data:
Reply from 74.125.45.103: bytes=1472 time=354ms TTL=49
Reply from 74.125.45.103: bytes=1472 time=353ms TTL=49
Reply from 74.125.45.103: bytes=1472 time=354ms TTL=49
Reply from 74.125.45.103: bytes=1472 time=354ms TTL=49

Ping statistics for 74.125.45.103:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 353ms, Maximum = 354ms, Average = 353ms

The easiest way to fix this is by using a TCP Stack modifier, and I suggest using the freeware tool TCP Optimizer, which you can download here http://www.speedguide.net/files/TCPOptimizer.exe

Choose "Custom Options" and set your MTU to 1472 Bytes, and reap the rewards of improved performance and throughput.

Tool of the Day: www.oldports.org

I had a problem installing Asterisk today, and had to rollback the zaptel port from 1.4.11 to 1.4.6 - the easiest way to do this was to get the port directory misc/zaptel from my old friend http://www.oldports.org

This site has years and years of port iterations, meaning it's painless and easy to get an older version of a port for yourself.

SSH host based authentication and security

I use SSH as my main form of connection, even with connecting to other boxes (I just create TCP Tunnels to get to RDP/VNC services). The problem with password based authentication is that social engineering and brute force CAN break through.

I prefer host based authentication - meaning that the server I am connecting to will only allow the server I'm connecting from to login, using DSA or RSA keys for the authentication process.

For clarity, Source means the server/workstation you are connecting FROM, Destination means the server you are connecting TO.

To setup the whole thing, you need to generate a private/public keypair on your Source server, do this with ssh-keygen:

ssh-keygen -t dsa

Follow the prompts and you DONT have to fill in a password if you dont want to, but it adds a level of security. The following files will be created in $HOME/.ssh :

id_dsa
id_dsa.pub

Cat the id_dsa.pub file, and copy the output, this is going to need to be placed on the Destination server. Login to your destination server as the user you normally connect as. Make sure there is a $HOME/.ssh directory and then create a file called "authorized_keys" (Case and spelling sensitive). Paste the previous output of the id_dsa.pub file into this file and save it.

Now you might need to edit your /etc/ssh/sshd_config file to allow host-based key authentication, make sure the following parameters are set:

PermitRootLogin yes (ONLY if you NEED root login, rather su to root)
RSAAuthentication yes (Allow RSA as well as DSA Keys)
PubkeyAuthentication yes (Allow Public key authentication)
AuthorizedKeysFile .ssh/authorized_keys (Location of the Server authorized keys)

Restart sshd, or HUP it (pkill -HUP sshd).

For added security, I disable PAM authentication in the /etc/ssh/sshd_config file:

ChallengeResponseAuthentication no

This prevents ANY type of password authentication, meaning bruteforce attacks are impossible to conduct against your SSH Server.

Should you wish to allow tunnelling through your SSH Server, set these parameters in your /etc/ssh/sshd_config:

AllowTcpForwarding yes

Again, restart SSHD.

Thats it - a pretty secure SSH system, which is usually the first point of attack for UNIX Hackers.

Tool of the Day: Running Multiple versions of Internet Explorer

For testing websites/software on older versions of IE can be a pain in the posterier, but with this tool, you can run multiple versions from 3.0 to 6.0. This is for Windows XP ONLY !!! But its helped me with some dodgy client complaints in the past.

Link is here: http://tredosoft.com/Multiple_IE

Load Balanced SSL based Internet Casino Client System

My personally designed SSL-offloaded load-balanced casino system went live today across 49 Internet Casinos, it has increased performance immensely and is able to add multiple servers on the fly to alleviate stress on existing servers.

The SSL Offload does SSL encryption calculations on the Layer4 switch, using onboard floating-point processors, meaning NO SSL encode/decode on the servers themselves. The switches can handle 115,000 concurrent connections, so plenty space to grow.

I designed and implemented the entire system, barring the existing MSSQL implementation. Casino is now handling 200,000 spins per day.

Just cant say "which" casinos....

See my System Diagram here:

Squid Proxy server, Ads blocking

I added an ad-blocking component to squid, not a third-party product, just a few configs of my own. The way I've implemented it, you don't see any "AD BLOCKED" crap on your webpages, you just have a clear block where the ad was, with no error message, increasing the "look/feel" experience for your users.

First, in the ACL section, create the ACL for ads:

acl ads_block_list dstdomain -i "/usr/local/etc/squid/blocks/ads_block.list"

Above all your normal http_access/http_deny rules, place this one:

deny_info ERR_BLOCKED_ADS ads_block_list
http_access deny ads_block_list

Now, create the file /usr/local/etc/squid/blocks/ads_block.list and populate it (I've just shown a head from my file):

[root@nas /usr/local/etc/squid/blocks]# head ads_block.list
101com.com
101order.com
103bees.com
1100i.com
123banners.com
123found.com
123pagerank.com
180hits.de
180searchassistant.com
180solutions.com

You can get anti-ads lists such as mine from various locations, use google to search.

Now edit the custom error message for the ads_block_list ACL, which is: ERR_BLOCKED_ADS

[root@nas /usr/local/etc/squid/errors/English]# cat ERR_BLOCKED_ADS



That last line, you'll notice the ! character which means don't display the standard squid error message.

once all is complete, reload your configs with "squid - k reconfigure" and try access pages now. Ads are blocked, and all you see is the page background. This way you can save large amounts of bandwidth on your internet lines, without creating errors or graphical problems on user viewed pages.

Tool of the Day: pkg-get for Solaris 10

Package management in Solaris 10 is less than stellar, with no automated system such as apt-get, ports, portage or yum like linux distro's, but I found a FreeBSD-like portinstall package called pkg-get. It allows installation of packages from a web repository and is compatible with the Solaris pkgadd system.

To install pkg-get use the folowing command.

"pkgadd -d http://www.opencsw.org/pkg_get.pkg".

edit the mirror url using.

"vi /opt/csw/etc/pkg-get.conf" and changing the default site url to.

url=ftp://ftp.heanet.ie/pub/opencsw/stable (This is Ireland's repository - closest one for SA)

run "pkg-get -U" to update catalog.

pkg-get is now ready.

"pkg-get -a |grep package" to find the desired packages, then:

"pkg-get -i packagename" to install it.

Very simple and powerful, even allows upgrading of the complete CSW subsystem in Solaris.

Tool of the Day: DGTeam firmware for Netgear DG834 series routers

keywords: netgear, dg834, firmware, dgteam

Found a great tool which increased stability on my router (DG834PN), as well as including about 100 features not seen on the standard firmware - even SNR increase/reduction to affect speed/stability.

You can grab the firmware for most of the 834 series routers here: http://dgteam.ilbello.com

Update:

I did this on a DG834 straight router, connection speeds (synchronised ADSL) went from 2908Kbps to 3824Kbps - On the SAME LINE !! No connection drops - 34 hours on the same PPPoE session.

Highly Recommended

Hacked my XBOX 360

Microsoft say their new Lite-On DVD Rom is unhackable due to firmguard protection.

Well I hacked mine to the latest iXtreme firmware v1.6 - I'm now able to play the 65 XBOX backups I have. Quite a convaluted procedure involving drive door positioning, soldering the PCB to unlock the developer mode on the drive, then extracting the DVD keys and re-inserting them into the hacked firmware, erasing the drive (scary) and then flashing the hacked firmware back.

Been enjoying "Battlestations Pacific" over the weekend :)

Update

Been off having my Jaw broken to have some Wisdom teeth extracted, so I'll be back to posting articles in no time !

For personal note, i'll be firmware updating my XBOX360 tonight, as well as doing the technical drawings for my "PC in and XBOX 360" project...More on that later.

Seems I'm missed at work, 8 days off and systems crashing in flames always ensures job security when you're away :D

Tool of the Day: Lambda Probe

keywords: tomcat5, memory usage, thread usage

This great little java app runs on Apache Tomcat 5 and allows memory/thread/process monitoring for all installed applications/contexts. Very small meaning low system impact. It's been really helpful tracing down Java code errors in one of my deployed systems.

Download link: http://www.lambdaprobe.org/d/download.htm

Setting up a multi-gateway FreeBSD firewall/Router

Keywords: Multi-homed, source based routing, FreeBSD, ADSL gateways.

The reason for this article is that due to strange and sometimes entertaining DSL solutions in South Africa, we must use a bit of technical ability to provide proper services for small to medium enterprises, where the prohibitive cost of true leased lines would sink such a small company.

Basic Premise: 2 x 1Mbps Uncapped Fibre ADSL Lines from ISP1, plus a bonded ADSL solution (bonding 3 x 4mbps ADSL Lines into a 12Mbps line). Problem is that on a standard Linux router, you get ONE gateway, meaning that all traffic coming in on interface1 does not return to sender through interface1, but rather interface2 if it is the default gateway.

The solution here will allow Incoming traffic to be routed correctly back through the orginating link, making multi-homing a reality.

Refer to the Network Diagram:

em0 - 1mbps Uncapped through ISP1
em1 - 1mbps Uncapped through ISP1
em2 - 12Mbps Capped through ISP2
fxp0 - interface to local LAN



First, we'll need to add some kernel compilation options (I'll do a followup article on creating custom FreeBSD Kernels).

In your kernel config, add:

# Firewalling and NAT
options IPFIREWALL (IPFW firewalling, better than IPTables IMHO)
options IPDIVERT (The Divert class in IPFW)
options IPFIREWALL_FORWARD (The Forward class in IPFW)
options IPFIREWALL_DEFAULT_TO_ACCEPT (create a default allow all rule)
options IPFIREWALL_VERBOSE (Verbose logging to stdout)

you'll need to compile your kernel, install it. Add the following line to /etc/sysctl.conf:

net.inet.ip.sourceroute=1

This will activate source based routing. Now reboot the box. IP information for the system is as follows:

em0: 196.1.0.2/29 em0 gateway: 196.1.0.1 (default gateway for server)
em1: 196.2.0.2/29 em1 gateway: 196.2.0.1
em2: 196.3.0.2/29 em2 gateway: 196.3.0.1
fxp0: 10.0.0.1/24 fxp0 gateway: none (self)

NAT Deamons are setup as follows:

em0: port 8668
em1: port 8669
em2: port 8670

You need to activate the source based forward rules in your firewall, so add the following lines to IPFW:

ipfw add 00059 divert 8668 ip from any to any recv em0
ipfw add 00060 fwd 196.1.0.1 ip from 196.1.0.2 to not 196.1.0.2 out recv em0

Rule 59 does not END traffic, but rule 60 does, so further processing of packets is not needed.

ipfw add 00061 divert 8669 ip from any to any recv em1
ipfw add 00062 fwd 196.2.0.1 ip from 196.2.0.2 to not 196.2.0.2 out recv em1

ipfw add 00063 divert 8670 ip from any to any recv em2
ipfw add 00064 fwd 196.3.0.1 ip from 196.3.0.2 to not 196.3.0.2 out recv em2

This will activate source based routing, so if you have a NAT rule on em1 saying redirect all port 25 traffic to internal IP 10.0.0.2, then it will return traffic back to the source over em1, instead of em0 (the default gateway).

Thus ends the horrible congestion of one line, as well as allowing incoming static NAT for multiple interfaces, really expanding possibilities for your business.