The best way I found was to store parsed MAC to vendor information in a database table which was easy to do since I was already using SQLite for the WIP Kismet parsing code.  I used this schema:

CREATE TABLE manuf (
mac TEXT PRIMARY KEY NOT NULL,
manuf TEXT NOT NULL);

and the following parsing code:

<?php
try
{
  $dbh = new PDO("sqlite:/path/to/database.sqlite");
}
catch(PDOException $e)
{
  echo $e->getMessage();
}

$ouifile = file("http://standards.ieee.org/regauth/oui/oui.txt");

$dbh->exec("BEGIN;");
$delete = $dbh->prepare("DELETE FROM manuf;");

$insert = $dbh->prepare("INSERT INTO manuf (mac, manuf) VALUES (?, ?);");
$insert->bindParam(1, $mac);
$insert->bindParam(2, $manuf);

$vacuum = $dbh->prepare("VACUUM;");

$delete->execute();
$vacuum->execute();

foreach ($ouifile as $line)
{
  if (!substr_count($line, "(hex)"))
  {
    continue;
  }
  $mac = str_replace("-", ":", substr($line, 0, 8));
  $manuf = trim(substr($line, strpos($line, "(hex)") + 5));
  $manuf = preg_replace('/[\',\.\(\)]/', '', $manuf);
  $manuf = preg_replace(
    '/(the|inc|incorporated|plc|s\/a|a\/s|ab|ag|kg|gmbh|limited|ltd|spa|llc)/i',
    '', $manuf);
  $manuf = ucwords(strtolower($manuf));
  $manuf = preg_replace('/\s\s+/', ' ', $manuf);
  $manuf = trim($manuf);
  $insert->execute();
}

$dbh->exec("END;");
$dbh = null;
echo "Done inserting";
?>

Basically, it pulls the oui.txt from the IEEE, parses it and removes some unneeded characters/info, and puts it in the database.  This is then easy to query against using standard SQL and it is now easily run and updated and takes milliseconds instead of seconds to run.

About a month or so ago, the CPU fan in my old server, which had been getting louder and louder for a while, finally got to the point where it was so loud that I couldn’t stand it any longer (most likely dust buildup in the motor). I ripped it out of the server which fixed the noise problem, but since I didn’t have any spare fans it meant that now the server was running on passive CPU cooling–something I was not fond of and could lead to total hardware failure.

So, I began investigating my options, and eventually found someone selling 2 Dell PowerEdge 2450 servers on Craigslist along with a bunch of server parts. I drove to the guy’s place, checked out the servers, and bought them along with a bunch of server parts and drives.

The only problem was that I didn’t really have room for rackmount servers in my room, and even if I did, they were much too loud to be in the open. So I designed and built a rack for the two servers and fit them into my closet where they couldn’t be heard. My clothes muffle the sound even more, and with some experimenting with airflow they only cause the closet to be 5-10 degrees hotter than normal. It did take some clever wire management to run power and networking into my closet though!

So now, I have the main server running 24/7 with the second server as a parts machine. It runs, but has a fault I need to investigate (and in fairness, it was sold to me as only a parts machine with the benefit that it actually does run). Its specs (and benefits over the old server) are:

Dual PIII 733MHz CPUs (over 2x increase)
1.5 GB RAM (currently, might step up to 2GB) (4x increase)
4x 147GB 10K RPM SCSI drives in RAID 10 (7x the space, ~2x the speed)
1x 10/100 LAN (built in)
3x 10/100/1000 LAN (PCI) (Gigabit LAN is really nice when transferring lots of files since I upgraded my room network to gigabit recently)
2x Hot Swap Power Supplies

It runs the new Ubuntu Server 10.04 with my standard Apache, PHP, MySQL, SQLite, SSH, FTP, TeamSpeak, etc. setup. Pages should load faster now (depending on network speeds) and I have a lot more room to expand. Another hit like my Portal 2 SSTV discovery should be less hard on my server now (over 1000 hits in about 15 minutes on a 400KB image).

Enjoy the newness!

CREATE TABLE networks (
id INTEGER PRIMARY KEY AUTOINCREMENT,
bssid TEXT DEFAULT NULL,
essid TEXT DEFAULT NULL,
cloaked BOOLEAN DEFAULT NULL,
channel INTEGER DEFAULT NULL,
encryption TEXT DEFAULT NULL,
manuf TEXT DEFAULT NULL,
ipaddress TEXT DEFAULT NULL,
iptype TEXT DEFAULT NULL,
maxrate INTEGER DEFAULT NULL,
maxseenrate INTEGER DEFAULT NULL,
beaconrate INTEGER DEFAULT NULL,
llcpackets INTEGER DEFAULT NULL,
datapackets INTEGER DEFAULT NULL,
cryptpackets INTEGER DEFAULT NULL,
totalpackets INTEGER DEFAULT NULL,
datasize INTEGER DEFAULT NULL,
firsttime TEXT DEFAULT NULL,
lasttime TEXT DEFAULT NULL,
gpsminlat NUMERIC DEFAULT NULL,
gpsminlon NUMERIC DEFAULT NULL,
gpsminalt NUMERIC DEFAULT NULL,
gpsmaxlat NUMERIC DEFAULT NULL,
gpsmaxlon NUMERIC DEFAULT NULL,
gpsmaxalt NUMERIC DEFAULT NULL,
gpspeaklat NUMERIC DEFAULT NULL,
gpspeaklon NUMERIC DEFAULT NULL,
gpspeakalt NUMERIC DEFAULT NULL)

CREATE TABLE clients (
id INTEGER PRIMARY KEY AUTOINCREMENT,
netid INTEGER DEFAULT NULL,
macaddress TEXT DEFAULT NULL,
channel INTEGER DEFAULT NULL,
manuf TEXT DEFAULT NULL,
ipaddress TEXT DEFAULT NULL,
iptype TEXT DEFAULT NULL,
maxseenrate INTEGER DEFAULT NULL,
llcpackets INTEGER DEFAULT NULL,
datapackets INTEGER DEFAULT NULL,
cryptpackets INTEGER DEFAULT NULL,
totalpackets INTEGER DEFAULT NULL,
datasize INTEGER DEFAULT NULL)

Its a two table design–the networks table holds the info about the APs while the clients table holds info about seen clients. If you don’t recognize the SQL variant, its SQLite, a choice I’m thinking of making so that the database can easily be replicated/edited/utilized without running a server.

And yes, the netid in the clients table should be a foreign key to id in the networks table, but as SQLite just added support for foreign keys, I’m not sure I want to include that yet.

The AP code work I started last year is about complete for the old gpsdrive database from 2005/2006, but I am currently researching and planning on moving the AP code to a much better database that holds more AP information; the only problem is that all the old data cannot be used.  But as Videlais and I geocache, the data will slowly build.  When I am more pleased with the quality of the code I will release it.

My “stuff” pages are almost complete; books are 100%, movies are 99%, and games are around 80%.  Those pages will be pubic eventually…maybe.

Also, the site is now open to search bots, and I’ve been watching them trickle in through my access logs.  Sadly, I lose almost all referrer info because of the frame mess I must do to get around port 80.  But its interesting seeing hits–and at least one a day is a vulnerability scan.

I have hidden all my old Nucleus and Blogger posts because I didn’t want all that info public to the search bots and whoever finds their way here.

Lastly, I have re-enabled comment posting but all comments are moderated to eliminate spam.  I don’t want to open up user registration due to spam bots and the havoc they can cause, but if someone really wants one for whatever reason, I can make it happen.

So, whats the plan?  None really, this site continues to be mainly a learning platform and a place to mess around with.  I can host things for people and it makes a nice development platform I can SSH into from anywhere.  But as far as posts go, well, I’ll update when I have something to say.  I have a couple planned out (like instructions for cloning a Backtrack4 USB persistent install) but I need to spend time writing them up.

I first started with the OUI file published by the IEEE: http://standards.ieee.org/regauth/oui/oui.txt  This file gives the vendor information for the leading 3 bytes of the 6 byte MAC address.  Each leading 3 bytes is uniquely assigned to one vendor, which can hold 2^24 or 16.8 million unique MAC addresses.

To begin, I ran

grep "(hex)" oui.txt>ouitab.txt

to get just the lines with the leading half of the mac in the form XX-XX-XX and the vendor’s name.  I then replaced the spaces, (hex), and one tab to get all lines in the form XX-XX-XX(tab)Vendor and saved this as oui.txt.

Instead of creating a new field in the geoinfo database from GPSDrive, I instead chose to use the unused comment field to hold the vendor information.  The rest was simple: write a php script to search the revised oui.txt file for the leading 3 bytes of the MAC for each AP in the database and then update the database by writing the Vendor to the comment field.  I came up with this:

<?php
$oui = file('oui.txt', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$dbconnection = mysql_connect('localhost', 'root', 'Insert Password Here');
$dbselected = mysql_select_db('geoinfo', $dbconnection);
$result = mysql_query("SELECT * FROM waypoints");
while ($row = mysql_fetch_assoc($result)) {
  $mac = $row['macaddr'];
  $macorig = $row['macaddr'];
  $mac = substr($mac, 0, 8);
  $mac = str_replace(":", "-", $mac);
  $mac = strtoupper($mac);
  $search = preg_grep("/{$mac}/i", $oui);
  $search = array_values($search);
  $exploded = explode("\t", $search[0]);
  if ($exploded[1] == "")
    $update = "UPDATE waypoints SET comment = 'Unknown' WHERE macaddr = '$macorig'";
  else
    $update = "UPDATE waypoints SET comment = '$exploded[1]' WHERE macaddr = '$macorig'";
  $updateresult = mysql_query($update);
}
mysql_close($dbconnection);
echo "Complete";
?>

This set the comment field with the found vendor name if it could be found or put Unknown if the MAC could not be found.

Note that using preg_grep in this way is VERY slow as it linearly searches oui.txt for a match for EACH AP.  For ~3700 APs it took about 45 seconds to run, which was fine for a quick and dirty script but for more APs would of course take longer.

More code updates to come as I work more on the AP code!