On Friday May 19th, with May release’s rollout, customers experienced significant server slowness and some timeout-related outages.

The immediate action taken to restore smooth server operation was reverting all customers to the April release. This was completed by 1PM AEST, same day.

‌

The error turned out to be the result of some unperformant code in the latest version of a library that Uptick uses for its error monitoring. (the irony!)

Once the issue was identified and resolved, the May release was rolled out once more, across May 22nd-23rd.

‌

Moving forward, we’ll be including some stress testing, and more rigorous performance monitoring, to assist in early detection of unusually high CPU usage.

‌

To all users affected: we apologise for the disruption!