Partial Outage of Customer Solutions
Incident Report for Appfarm AS
Postmortem

On Tuesday, September 19 at 09:04 CEST, Appfarm Customer Solutions, using a shared Cloud Memorystore, experienced availability issues. Solutions returned to an operational state at 09:25.

We have identified the root cause and are taking steps to better manage such events in the future.

After investigating the incident, we have identified that the root cause was a hardware failure on the master node on Google Clouds Memorystore. When the new master was up and running, Appfarm Customer Solutions started to recover.

To minimize the impact of future similar events, we have identified some concrete measures that will be implemented.

  • We will change the components dependent on the Cloud Memory Store to degrade service gracefully rather than taking it down.
  • We will increase the distribution level of the Cloud Memory Store so that future failures of this service will impact a smaller share of the solutions.

We sincerely apologize for any inconvenience this may have caused.

Posted Sep 21, 2023 - 09:42 CEST

Resolved
This incident has been resolved.
Posted Sep 19, 2023 - 09:25 CEST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Sep 19, 2023 - 09:04 CEST
This incident affected: Customer Applications.