The Diary Diaries: Fixing Remembary's Facebook Connection
Sep 18, 2016 16:05

I recently did a major update to my iOS "Connected Diary" app Remembary and had a lot of trouble getting properly connected with Facebook. I wasn't able to find much help online for my specific issues, so I'm writing down what I figured out in case it helps other people out there.

Background

Remembary is a diary app that helps you remember what you did every day by pulling in various timestamped things and showing them in popovers linked to each day's entry. This includes iOS calendar events and photographs, as well as RSS feeds, Twitter tweets, and Facebook status updates. When I first added Facebook functionality back in 2011 or so, the process was pretty straightforward: add the Facebook SDK to the project, set up an app profile on Facebook with the right permissions, run a query against at the Graph API, and parse the resulting JSON.

The Facebook connection is just one small feature of Remembary, so once I got it working acceptably I moved on to the many other things in the app and didn't spend a lot of time or energy keeping up with what Facebook has been doing.

When I finally did a long-overdue overhaul of Remembary in the summer of 2016, I discovered that the version of the API I had been using was going to expire in early August. I looked into the API docs a bit more and discovered that the Graph API call I had been using reliably all these years, "/me/statuses" was going to be retired early next year. This ended up being the start of a whole chain of problems.

Facebook API Versions

Facebook is constantly changing their APIs, just like they regularly change their front-end UI. They used to do this fairly arbitrarily and without much warning, and I remember having to do some frantic scrambling in the early days to respond to things suddenly breaking. Now they have versioned APIs: the query that you send to Facebook includes the version of the API that it's working against, and Facebook says that each version of the API will work for at least two years. This is a big improvement over the arbitrary changes from before, but you still need to keep track of what's changing.

I discovered that the "/statuses" call I used was being retired soon, and I had difficulty finding the replacement. Eventually I discovered that the call I wanted was "/me/posts". So I put that in, made sure it worked on my various test devices, and sent the update to the App Store.

And that's when I started getting the emails from users saying they couldn't get into Facebook anymore.

Facebook Permissions and Users

This was the nightmare situation of every developer: users complaining about app behaviours that just aren't showing up on any of your own devices or even in the simulators. "Hey it works for me!" may be a cliche of unhelpful IT people but it's also sometimes the painful truth.

It turned out that I had a mismatch between the permissions that Remembary was requesting from Facebook and the settings in the app's Facebook profile. If these don't match up, not only will the requests fail, but the access permissions prompt will show up every time the app sends a request. At some point I had added a call to fetch new statuses in the "AppDidBecomeActive" method, which is called every time a permission prompt is dismissed - and this led to an endless loop of permission requests until the poor user tapped on "Don't Allow".

Why didn't I see any of this on my devices? It turns out that Facebook automatically grants all access permissions to the Facebook user who manages the app profile, as well as for any test users that they create. While this is handy for testing new functionality, it's clearly not very good for making sure your app is requesting the proper access!

Facebook App Approval

So now I needed to add "user_posts" access to the app's Facebook profile. This used to be a simple matter of updating the app profile, but now every access change has to go through an approval process, just like the App Store's app review. This involves not only writing an explanation for what your app is doing with the access, but you also have to submit a video of the app requesting and using the access and even a working binary of the app!

So I made a video from the simulator, recorded and edited with ScreenFlow, and uploaded a binary - and ... got rejected.

At least it only took a day, rather than the "5 working days" that they threatened. I hadn't realized at this point that I already had access to all permissions, so in the demo video Remembary was still using the "status_updates" request, which didn't match what I was requesting, so they failed me on that. Then they also said that the binary I had sent them didn't work.

At first, I had no idea why I had been rejected, and fell into a week of despair as users continued to complain to me. I'm always surprised by how emotional I get about Remembary - it's a very personal thing for me, and I'm putting it and myself out in public every day. I also don't do a lot of iOS work outside of Remembary, so I often feel out of my depth in a way that I don't with things I use every day like Ruby on Rails.

Once I figured out I had access to all permissions, I was able to make a new video with the proper requests, and I also made a binary that worked properly this time. The key was to follow Facebook's instructions precisely and then also confirm that the resulting file worked.

The magic command was:

ditto -ck --sequesterRsrc --keepParent `ls -1 -d -t ~/Library/Developer/Xcode/DerivedData/*/Build/Products/*-iphonesimulator/*.app | head -n 1` path/to/YourApp.zip

The first time around I think I had left out a flag or something as I had typed it in manually to figure out what it was doing (it's actually rather clever). The second time I just copied the whole thing over from the documentation. I then made sure to confirm that the binary worked by using ios-sim (which I installed using Homebrew, after having to update my homebrew install as well - sigh).

In writing this post, I've discovered some better documentation from Facebook, including examples, that would have really helped me if I had been able to find them: https://developers.facebook.com/docs/facebook-login/review/examples#instructions)

The second approval submission worked, and now Remembary's app profile on Facebook was set up properly.

TestFlight

Building and maintaining an app in the App Store, one learns as much about customer support as about programming. Of the people who had complained about their Facebook access, I was able to corral about half to join my testing team. This is a great way to keep happy customers - it lets them know that you're working hard on a problem, and it also lets them become part of the solution. Some people might find this annoying, but those who want to help are more likely to become strong supporters.

This gave me a long-overdue chance to work with Apple's integrated TestFlight service. I had used TestFlight when it was an independent company and while it was extremely useful for getting test code out into the world, it had a confusing interface. Thankfully Apple had cleaned things up quite a bit, and it's now one of the less confusing parts of the whole App Store experience.

I was able to send multiple builds out to my testers, and even though the App settings were correct, they were still getting the approval prompts all the time. After some more checking, I discovered that some leftover old code in Remembary was asking for extra permissions that it didn't need and for which it hadn't been configured. It's always good to remember to check both sides of a permission request!

I fixed this up and finally everything was working smoothly. The app has now been submitted to the App Store for review and should hopefully be live soon.

Facebook SDK vs iOS Accounts Framework

One of the worries I had with the Facebook app review was that I wasn't using Facebook's own iOS SDK, and was instead using Apple's Accounts framework. Since Remembary also fetches from Twitter, it's convenient to use the Accounts framework to do both - but I figured if that was the problem with the approval, I would try the Facebook SDK instead. I had used earlier versions of it for earlier versions of Remembary, so how hard could it be?

Well, after I added the latest version of the Facebook SDK, my app simply refused to compile. It had some kind of obscure compiler trouble with the "Bolts" framework. Stack Overflow noted that CocoaPods would help resolve these kinds of problems, so I gave that a try. All I got with CocoaPods was weird errors from the automated build shell scripts. Hunting these down took me to obscure posts about weird file permissions on mounted drive images and such, but no matches to my exact problem.

There's a special feeling when your error messages don't show up in any web searches - like you're the first botanist to discover a new species of fungus.

So I gave up, killed that Git branch, and went back to using the iOS Accounts framework - which turned out not to be a problem in getting Facebook approvals after all.

Facebook Paging Queries Quirks

I had been wrestling with Facebook for a while, and now that everything seemed to be working, I decided to try adding a feature that had long been requested: the ability to download your entire Facebook post history. When asked for a list of posts, Facebook only returns a small subset, but also helpfully adds "previous" and "next" links that can be used to walk through all of them. I set up a function that ran in a loop to continue getting the "next" query and running it until it ran out of posts - but frustratingly this only seemed to want to run once. If I ran the query manually I could copy-and-paste the "next" value into the Facebook Graph API explorer and it would run perfectly fine, so this really had me scratching my head.

Looking more closely I noticed that the queries that failed all finished with the "until" parameter:

https://graph.facebook.com/v2.7/732421557/posts?limit=25&format=json&__paging_token=enc_...pkQ1&access_token=EAA...ZD&until=1469368561"

... and also that the error I was getting back from Facebook was something about invalid timestamps. I tried rearranging the parameters in the query, putting "until" earlier - and it worked! So something was going on in the round-trip from Facebook to the app and back to Facebook again, adding some kind of extra hidden character (extra newline maybe?) to the end of the query that caused Facebook's timestamp reader to choke. I didn't want to have to parse and re-build the queries every time, so I simply added the very Generation-X-coder-is-tired-now "&dummy=whatever" to the end of every query, and this worked fine.

Okay - For Now

So I endured several weeks of headaches simply to keep a small piece of Remembary's functionality mostly the same as it had already been for several years. Some new features have been added and enhanced, but that was only after a huge effort just to regain what had previously been fine.

This is the world of the modern app developer. If you meet one, be nice to them and offer to buy them a drink - they probably need it!