Project Report: PTOS2
May 04, 2010 12:41

A few days ago I wrapped up four months of development and deployed PTOS2, a rewrite of PTOS: The Price Ticket Ordering System, for The Beer Store.

People often ask me what I actually do for a living - most of my projects are private business applications and can't be shown to the general public, so it's sometimes hard to explain. PTOS is a good example of work I've been doing for the last decade or so, so I thought it would be useful to describe this project.

The Situation

Back in the early 2000s, The Beer Store redesigned all of the interior signs and pricing displays for their 400+ stores. The company that did the redesign was Get Signage. I've been building software for Get Signage and its sister company, Signs of Change since the mid-1990s.

Beer Store Beer Wall Price Display

The problem with pricing displays, especially for a product like beer, is that the prices change quite often. The Beer Store has a variety of store types with different kinds of price displays, including special "Ice Cold Express" and "Single Can Cooler" displays for popular brands. Each display type uses different quantities of slightly different kinds of price tickets, and different brands are stocked at different locations.

There is a short window of time between when the price changes are announced and when the newly-printed tickets need to be in the stores, and The Beer Store and Get Signage have to manage this carefully to make sure everything runs smoothly.

Like most business processes, this was initially managed through Excel spreadsheets, email, and a lot of copy-and-pasting. This was slow and error-prone, especially since these spreadsheets had grown to more than 2 megabytes each. So Get Signage asked me to figure out a way to automate this.

I sometimes describe the work I do as "replacing emailed spreadsheets". Excel is the blue-green algae of business software: simple but ubiquitous. Anyone can build a spreadsheet to help with the work that they need to do - but once a process grows to a certain complexity and involves lots of different people, it's better to turn it into a full application.

PTOS 1

After a clunky start trying to extend Get Signage's existing Lotus Notes system with custom Java agents, I moved to a standalone Java/JSP/Struts application, hosted on Tomcat and with a Postgresql back-end. This went live in early 2005, and was the first large Java application that I had built entirely on my own. It used Struts 1.1 taglibs for page layout and I built my own rudimentary object-relational mapping for it, based on some PHP work I had done earlier.

Because the original process involved spreadsheets, I designed the interface to have a lot of large tables that mimicked the original spreadsheets.

It wasn't the world's most elegant or gorgeous web application, either on the surface or underneath, but it did its job well for over five years and several hundred orders. In many ways it has been the most successful of the dozens of applications I've built in my career.

Changes in the business enviroment forced a number of changes to PTOS. While it was solid and reliable at what it normally did, PTOS wasn't very flexible, and had trouble processing extremely large orders. I work almost exclusively in Ruby on Rails now, and PTOS was the only remaining Java/Struts application I worked with - so maintenance was a big headache.

PTOS2

PTOS2 Job Screen

So we decided that instead of trying to graft new changes onto the old Java application, I would build a new version of PTOS in Ruby on Rails. This would make the application easier to maintain and customize in future, and I could use new(er) technologies like AJAX and DHTML to clean up the interface.

To ensure a smooth transition, we kept the same server machine, running Tomcat and Postgresql. I used JRuby and Warbler to turn the Rails application into a drop-in .war Java Web app, having PTOS2 running happily alongside the original PTOS.

How Much Better Was Ruby?

As I discovered in the ClearFit/Careerious Project, Rails can make for dramatically cleaner and clearer code. Over 200k of Java database logic simply disappeared in the transition to ActiveRecord. The switch from Struts Taglib-laden JSPs (there's no 'else' tag!) to HAML was equally dramatic. Pages full of awkward verbose tags were replaced by a simple haiku of identifiers and helper calls.

PTOS JSP Code
PTOS1 JSP / Struts 1.1 Taglibs
PTOS2 HAML Code
PTOS2 Rails and HAML

One of the concerns I've heard about JRuby is how much of a performance hit there is in running interpreted Ruby on top of the JVM. In this case the opposite happened: the JRuby application ended up running faster than the Java application. Any performance hit from running in Ruby was more than countered by the fact that the Ruby code was so much easier to read and modify that I was able to clear out many performance bottlenecks that had been in the application for years.

It's hard to do direct comparison like I did with C#/ASP and Ruby/Rails on the Careerious project, since much of the application had changed, and I cleaned out a lot of extra cruft. However, most of the files were at least half the number of lines and even less raw text than their Java equivalents.

iText PDFs

One of the new requirements for PTOS2 was to have it generate its own PDFs of the various price tickets. With PTOS, every price ticket had its own InDesign file that had to be manually updated and printed. Now, PTOS2 puts all of the price changes directly into PDF forms, locks them, and merges them into single-download files. This removed a major source of errors and sped up the production process by at least a day, which makes a big difference on this tight a timetable.

JRuby was a great help, since I was able to use the powerful iText Java library to fill out the PDF forms. It felt magical to click on a button and have an entire set of perfectly laid-out price tickets download automatically.

Dynamic Listings

PTOS involves several large (400+ elements) lists of jobs, brands, and store locations. As a developer, I spend a lot of time building and managing listings like these - but I've always felt they could be more flexible and easier to use. For this project, I built my own plugin that uses JavaScript to build sortable, searchable listings out of JSON data. It made a big impression in demos and has quickly become an irreplacable part of the application interface. The DynamicListingsHelper plugin can be found on Github. Feel free to play with it. Note that this has been set up to be useful for my own projects, and hasn't been cleaned up and scrubbed for general use - some of the code is ugly and it might not work very well in all cases. However, it's certainly easier than building your own library from scratch. I'll probably write a blog post on this later.

Headaches

Compared to the initial Java development for PTOS, building PTOS2 was a breeze. The technical details of getting the PDF tickets working properly took about a third of the development time, and a lot of close work with the designer (the talented and meticulous Carol Clapham, who also paints nice watercolours)to get all of the pieces fitting together properly. Text field contents have a habit of jumping up the page by several dozen pixels when filled in and locked down, so we had to do a lot of messy rearranging of the template files.

Early on, I decided that PTOS2 would hold all of the information that was already in PTOS. PTOS2 had its own new database, but I was able to bring over all of the data from the original application. This data migration was pretty complex, and involved a number of big migration scripts that updated and cleaned up the schema and data to fit with ActiveRecord's naming standards and support the new model.

Having an existing application with existing live data made it much easier to check that everything was running properly, but it also meant that quirky legacy logic had to be tolerated in the new code.

The biggest technical headache involved database drivers. A JRuby project will need Ruby database drivers, Java database drivers, and JDBC drivers, as well as connectors between all of these. These problems were compounded by having an older version of Postgresql on the live server (that couldn't be easily upgraded, for various complicated reasons). Even now, PTOS2 works fine in Tomcat, but I can't connect to the database if I use rake or script/console. I ended up putting some of my migration calls behind buttons in the application's admin interface.

JRuby and Warbler advertise that they make deployment of Rails apps to Java servers a simple matter of "dropping in the .war file" - as usual with Rails (or any other major framework) it wasn't that easy. There were the usual configuration, gem and dependency issues and a fair bit of hunting obscure error messages. Everything works well now: build the war with Warbler, upload the war, copy it into Tomcat's "webapps" directory. Occasionally I have to restart the server so it frees up some memory, but otherwise it's pretty clean.

At the last minute, days before going live, I discovered some very strange CSS behaviour that only showed up in Internet Explorer: when the page first loaded, none of the background colours would show up in any of the divs on the page. Resizing the window, overlapping it with another window, or otherwise triggering an OS-level redraw would make the backgrounds show up. This happened reliably in IE7 and IE8 (the browsers that most of PTOS2's corporate users will be using) but I couldn't find any mention of it anywhere online. I finally noticed that I hadn't specified a 'DOCTYPE' at the top of my main layout template page - once I put that in, IE behaved just fine.

Next Steps

It's great to see a project go live - especially a project that's an improvement over a previous one that has already proven its worth. Now that the application is on a clean and solid foundation, we already have many plans for new features and updates to make it even more useful and streamline the business process even more.

It also means that I don't have any more legacy Java applications to worry about anymore, and that's a good feeling as well.

Previous:
A Quick Note on Encryption
Mar 02, 2010 09:54