Congratulations! Welcome to Your Nightmare!
Mar 11, 2011 09:30

(This is part of an ongoing series of posts about my experiences in building, releasing, and promoting my first iOS app Remembary: The Connected Diary for Your iPad. Several posts are expanded from a presentation I gave in January. There's an overview here.)

Maker Neurosis

One of the things I discovered in building and promoting Remembary is that having my own product has helped me understand the mind-set of many of my startup-founder clients - especially their worries and neuroses, since I started getting them myself.

One of the common worries I had heard a lot was: "What if the app gets really popular, but then it turns out there's a terrible error in it? We'll look bad in front of the whole world!" This worry manages to combine two delusions at the same time: wish fulfillment and paranoia. It's a close cousin to the endless worries that web-app founders have about how things will "scale".

I started getting this worry myself every time a Remembary release got closer to going live. I countered it by 1) having a thorough QA checklist that I run through at least twice (once in the simulator and once on the device) before submitting to the store, and 2) reminding myself that this is just a neurotic worry that's doing more harm than good.

Big News - Big Problems

So, one day in January, Remembary got a great review on a leading App Review site and the sales skyrocketed. No longer was Remembary stuck at the far end of the Long Tail - it was actually getting an audience.

Unfortunately, this audience wasn't completely happy. I started getting emails from the website's support form, the review article started getting angry comments, and, worst of all, Remembary started getting 1-star reviews in the App Store. People were saying that Remembary was doing the absolute worst thing that a diary app can do: it was deleting their entries!

What's Going On?

As more problem reports came in, I felt like I was in a nightmare of the 'standing-in-front-of-your-high-school-class-with-no-pants-on' variety. The thing is, I had spent a lot of time and energy making sure that Remembary's data layer was rock-solid. Remembary may have been my first iOS app, but I've built dozens of database-backed web apps over the last decade or so, using several of the major relational database engines. In fact, I purposefully skipped Apple's Core Data data abstraction layer so I could work more directly with the database and craft the queries myself - the database was the one place in the app where I really knew how everything worked.

What made this even more like a bad dream was that I had been using Remembary myself almost daily since the time that it was just a text field and some buttons on a grey screen, and I had never experienced this problem. I checked with some of the other users who had contacted me before, and they reported that everything was working fine - so the problem was only happening to some people, but not everyone, and - most maddening of all - not me.

If Remembary had been on another kind of platform, like a Windows PC or Android, I could have explained this away as some kind of strange hardware incompatibility - but at this time there was only one kind of iPad. Everyone was using basically the same device as I was, but some of them were having trouble.

So, in light of this, there were two things that I had to do:

  1. Figure out what on earth was going on in the program and fix it.

  2. Do some emergency PR to let people know what's going on and to salvage Remembary's reputation.

The PR part is worth an entire post on its own, which is coming up next - but here's how I found and fixed the problem:

The Investigation

First, I had to figure out exactly what was happening. After some emailing with people who had submitted messages on the support form, I found out the basics of the problem: when people left the application and then came back in, they sometimes found the current diary entry blank. The other entries would be fine, and the application otherwise worked normally. For some people, this happened all the time, but only occasionally for others, and never for yet other people.

This led me to believe that the problem wasn't in the data layer, but through some kind of interface activity. One of the things that make iOS apps feel so natural and intuitive is that they don't have "Save" buttons - content should just be persisted automatically without the user thinking about it. This silent saving is one of those seemingly simple things that actually take a lot of work: I had to figure out all of the cases where someone might change their context, and make sure that the current entry was properly saved in all of them. My guess was that there must have been some rare permutation of user actions that wasn't causing the save command to be called properly. So I advised people to try going to another date and then back before leaving the app - thus properly triggering the call to save the data.

Some people reported that this solved the problem for them, but other people were still losing data. I was completely flummoxed. I was using Remembary myself every day, and was trying everything I could think of to make it delete things, but I just couldn't reproduce the problem.

Remembary was now showing a score of 1 1/2 stars in the US store and had comments that said things like "I bought this based on AppAdvice's recommendation. Not so sure I trust them anymore" and "Reinstalled but not very confident... Hard to trust this at the moment." Sales had collapsed again back down almost to the level they had been at before the review. And I still couldn't reproduce or figure out the problem.

Then, one night I had been doing a lot of work on my iPad and had Pages, Keynote, Numbers, Twitter, and World of Goo open in the background and Remembary in the foreground. I switched over to The Early Edition RSS reader to check a reference from some article I had read yesterday, and it started automatically downloading from the 60 feed sources I have - I closed out of Early Edition and back to Remembary and discovered an empty page.

I had never, ever, been so happy to lose data in my entire life.

Gotcha!

A bit more investigation on my iPad and in the XCode simulator confirmed it: if Remembary was asleep in the background when the iPad had a low memory situation, it would wake up knowing the currently displayed date, but only showing a blank page with empty fields. When you moved to a different date or left the application again, it would automatically save the page contents - a bunch of empty fields - to the current date's entry.

Now I knew what was causing the problem - and why I had missed it. The current version of Remembary had come out right around the time that iOS 4.2 had arrived, with its support for multitasking. I had of course updated Remembary to support going to sleep and waking up in this multitasking environment, and it had worked fine in all of my testing and in my day-to-day usage. Remembary isn't a particularly memory-intensive application - it's almost entirely text, and text is cheap. For example, I had done some testing with all ten years worth of Samuel Pepys' diary, and it had taken up only slightly more than 2 megabytes. I had triggered low memory situations in the simulator and the application had behaved perfectly fine. But I had never considered triggering a low memory situation while Remembary was asleep, and this particular combination was causing the problem.

This explained why I didn't experience the problem, but many of the new users did: I don't usually use that many iPad apps over the course of a normal day, so I rarely trigger low memory situations. The kinds of people who buy apps right after reading a review in AppAdvice are more likely to be running lots of apps, and thus more likely to run low on memory - and so more likely to see this problem.

Fixing The Problem

Unfortunately, I still didn't know how to fix it. I'm lucky to share an office space with the great iOS developers at MindSea, the folks behind the Hall-of-Fame app PocketBooth, and I asked for some time to help me figure this out.

It took them all of five minutes to find the problem and fix it.

Here's what it was: It turns out that when iOS apps run low on memory, they unload any views that aren't in the foreground. Then, when these views come back to the foreground, they're reloaded from the template files, but they aren't necessarily populated fully. One of the things I had added in the latest version of Remembary was a nice startup animation that presented a leather-bound book cover opening to reveal the diary (actually a 'default.png' and a modal view and some clever slight-of-hand). To ensure consistency and security (if the journal was password-protected), this modal view was brought up automatically whenever the app was put to sleep, so that it would already be showing when Remembary was woken up again. This meant that if a low memory situation occurred, the main diary page was now in the background and so was unloaded to preserve memory. That made it come up blank when the application was restored - and then the blank values got auto-saved to the database, deleting whatever diary entry was already there.

All that was needed to fix the problem was an extra few lines of view configuration in the viewDidAppear method and everything now worked. I spent a day running the app through the regular QA checklist (this time with extra checks for low memory situations while backgrounded) and, less than a week after first hearing about the problem, I submitted Remembary 1.3, complete with fixes, to the store.

The Happy Ending

The spike in sales from the AppAdvice review got the attention of the App Store curatorial staff, and, only a few days later, Remembary 1.3 was approved and was featured in the "New and Notable" and in the Lifestyle "What's Hot" sections of the Canadian and 'Worldwide' (all the 'smaller' markets like Portugal, Russia, Thailand, Argentina, etc. together) stores. Thankfully, this version actually worked properly and didn't delete anybody's diaries. As of this writing, Remembary 1.3.5, with new themes and improved functionality, has been submitted to the store and will (review team willing) be coming out soon.

Lessons

  • You won't be able to write 100% perfect software. Problems will come up. While it is important to release reliable, tested software, it is equally important to be able to respond to and fix problems quickly.

  • Make sure you've tested what happens in your iOS app if it gets low memory situations while backgrounded.

  • A lot of iOS' functionality is in interface interactions, and these are a lot harder to test automatically than, say, a server-based Rails web app. Make sure you have a QA checklist to go through manually before submitting your app. Make sure it's thorough and covers as many odd interface permutations as possible. Run this checklist in the simulator and on as many devices as possible.

  • Make sure your users have a way of reaching you, either post an email address or have an easy-to-find support/feedback form on the app site and linked from the App Store page.

  • Don't discount user problems as a "wrong hardware / platform" issue - it probably isn't. It's more likely to be people using your application in ways that you didn't expect.

  • In other words: Just because it a problem doesn't happen for you doesn't mean it isn't happening for other people.

  • It's always good to know other programmers - especially ones who know a lot more than you do.

  • Don't let the fear of things going wrong get in the way of you doing anything. Ignore the scared little voice that's using fear to keep you from achieving things.

Previous:
How My iPad App Remembary Took Off
Mar 07, 2011 10:07
Next:
Remembary Video
Mar 19, 2011 15:31