One of the new projects I had this summer was a project proposed by the Democrat & Chronicle. The project involved access to a selection of emails sent to the Greece school district in the wake of the YouTube video involving several of their students. The idea was that we would get a dump of email bodies and try to glean some information out of them.

The first hurdle, unfortunately, was getting at the information. Shortly after we were approached with the request, I received an Access file containing around 5000 email bodies to go through. Being what we are, most of the resources in the FOSSBox are oriented around Linux, and no one about had a copy of Access installed to get the data into a more friendly format.

In theory, there are ODBC drivers for access databases, just like there are for any other database system. In practice, however, they seem to only exist for Windows machines which, while not surprising, was disappointing. This led me to dig out an old VirtualBox VM with Windows XP on it, install the Access drivers from Microsoft, and throw LibreOffice on it, too. There's other ways I could have gotten this done instead of LibreOffice, but I was still hoping this could be a simple action at the moment.

LibreOffice Base eventually got into the Access file, but then the troubles started again. It initially prompted me to save a LibreOffice database file, which sounded great to me... it could export it immediately, then I could copy it over and finish the task in Linux. Unfortunately, all this file did was create a small wrapper around the Access file, telling LibreOffice where the file was located, and what was needed to open it. So now I was back to trying to export the data. LibreOffice, though, was not willing to play along. I admit I am less than familiar with the Base component of LibreOffice, however some exploration and more searching online led me to believe I could not do the simple translation of data from one format to another from within Base.

Instead, I needed to select the table I was interested in (the only table in the database), tell LibreOffice to copy the table, then open a new spreadsheet in LibreOffice Calc and save the data that way. While this makes some sense to me (Base being simply for basic interaction with databases, Calc for manipulating raw data), I was dismayed that I could not find some way to export a single table to a common data format, like CSV, instead of having to go through yet another step. In any case, once I dumped the data into Calc, I could easily save it to CSV, drop that into my real computer, stop the VM, and get to work for real.

The end result is this. I'm not sure it will ever be of any particular use to anyone other than myself to remind me how to use the Python NLTK module (whose documentation seems to be geared more towards researchers than those already familiar with Python), and is hardcoded to certain facets of the data I was given, but it does manage to do a few things, and at each step it dumps the state of the data to a file so I can inspect the process and consider possible improvements.