That time novlr made the words invisible

Slap, bang, we were so close! So tantalising close to launching offline. I mean we did launch it. But there was a problem.

Last night at 00.30am GMT, we launched offline editing. We always release new functionality at least 12 hours before announcing it in case any issues crop up. Well they did. We woke up to two people rightly very worried that their words seemed to have disappeared. We reverted the code and within an hour they could see them again.

But what had happened?
Well, it’s a strange one. When you look at Novlr, you are probably looking at the most recent code, but browsers also “cache” code to stop it reloading it every time. The code can tell the browser if it is modified and that will tell it load the newest stuff. Well our code had a bug that meant that it was telling the browser that it wasn’t modified and so users weren’t getting the latest copy of the code. (If you are technically minded, details of the bug are here.) One of the things that was affected was the stylesheet which defines what everything looks like and, as a consequence, the colour of the font. Meaning that all of the words were there but were ‘invisible’! Yeah, I know. Not something we’ve come across before either!

The solution
Reverting the code, then asking the affected users to refresh brought the old code back. We then corrected the code that was telling the browser that it was always unmodified. We have also added extra caching instruction to make sure that you always have a full copy of the latest code when you need it. We launched all that today. As for the offline stuff which we had to take down, we are giving the caching 24 hours as it is new functionality, before then launching the offline work again.

Another lesson learnt and another improvement to the Novlr code and deploy process.

Oh and I don’t think I can say this enough…WE WILL NEVER LOSE YOUR WORDS. We triple back them up every five minutes to different locations, so even in the hugely unlikely event they really did go missing, we would have them.