Workforce / Logbooks Outage
Incident Report for Uptick
Postmortem

On July 12th 2023, Uptick experienced two partial outages, between the times of 12:15PM-12:30PM and 13:10PM-13:40PM AEST.
Customers would've experienced degraded server performance through to a complete outage during these times.
The outages were due to an interaction with our Uptick Logbooks product, which was also down during this time 12:15PM-2PM AEST.

The cause of the incident was an inundation of load on our Logbooks product, which resulted in a temporary outage – our first in a while! Uptick customer servers had not been configured to handle this scenario gracefully, and ended up hanging as a result on trying to talk to Logbooks.

The situation has since been remedied by both beefing up Logbooks' resources, and limiting how much of a detrimental effect Logbooks can have on Uptick customer servers should such an incident reoccur in future.

Apologies for the disruption!

Posted Jul 12, 2023 - 18:21 AEST

Resolved
Uptick had two partial outages today, between the times of 12:15PM-12:30PM and 13:10PM-13:40PM AEST.
Customers would've experienced degraded server performance through to a complete outage during these times.
The outages were due to an interaction with our Uptick Logbooks product, which was also down during this time 12:15PM-2PM AEST.
Posted Jul 12, 2023 - 18:08 AEST