User Impact
Processing delay or print and scan jobs submitted to uniFLOW Online.
Scope of Impact
Europe (EU) Deployment
Incident Start Date and Time
August 22, 2024, 9:30 UTC
Incident End Date and Time
August 22, 2024, 12:35 UTC
Root Cause
There was performance issues identified in our malware scanner. This was validated and confirmed by our security vendor as a regional issue impacting all endpoints hosted off the EU deployment.
This was a performance issue and was not a security incident. At no time was the security posture impacted, and all processed files were still scanned.
How did we respond
9:30 UTC: Telemetry and alerting identified a growing number of unprocessed jobs. Actions were taken to mitigate this issue by provisioning resources to process the growing number of jobs.
9:45 UTC: The Operations team learnt of the wider outage by our security malware vendor. This validated the delays we were seeing and explained why the earlier resource provisioning was not helping as expected. Jobs were still being processed but there were growing delays impacting end users.
11:13 UTC: Operations saw a sudden recovery of the system with all resources returning to 100% efficiently. This was shortly after aligned to the recovery of our malware security vendor. There was a backlog of jobs, but they were processed through very quickly.
12:35 UTC: Full system recovery was confirmed.
Next Steps
We apologize for the impact on affected customers. We are continuously taking steps to improve the uniFLOW Online Platform and our processes to help ensure such incidents do not occur in the future. In this case, this includes (but is not limited to):
We have reviewed this incident across the global operations team and key management members.
Response and recovery efforts will be reviewed for future improvements to our response times and actions.