Device Login Issues
Incident Report for uniFLOW Online
Postmortem

User Impact

During the incident people would have experienced delays when logging into their devices or would not have been able to login.

Scope of Impact

This issue was isolated our EU (European Union) Deployment.

Incident Start Date and Time

Jun 27, 2022, 06:30 AM UTC

Incident End Date and Time

Jun 27. 2022, 08:15 AM UTC

Root Cause

Due to a sudden increase in load authentication requests at the Canon device were taking longer than normal to be processed which resulted in requests to be delayed and/or time out.

This was automatically resolved by scaling within the backend after which queued requests were processed, and services returned to normal. However, the scaling reaction time was not fast enough to address the peak load at the time.

Next Steps

We apologize for the impact to affected customers. We are continuously taking steps to improve the uniFLOW Online Platform and our processes to help ensure such incidents do not occur in the future. In this case, this includes (but is not limited to):

  • Improved the scaling process to automatically react better and faster to sudden load increases
  • Additional monitoring and alerting have been implemented to receive earlier notification as peak limits are approached.
Posted Jul 12, 2022 - 17:26 UTC

Resolved
Hello All

We are closing this issue as resolved.

Once we have completed our internal investigations a Post Mortem will be provided no later then 10 business days.

Kind Regards
uniFLOW Online Operations Team
Posted Jun 27, 2022 - 09:37 UTC
Update
We are continuing to monitor for any further issues.
Posted Jun 27, 2022 - 08:55 UTC
Monitoring
Hello Everyone,

We are getting positive feedback that the issues is resolved, our telemetry also backs this up. Early investigation points to resource utilization which specifically services requests from connected devices. We will continue to monitor and are investigating as to the root cause.

Sorry for any inconvenience caused.

Kind Regards
uniFLOW Online Operations Team
Posted Jun 27, 2022 - 08:36 UTC
Investigating
Hello Everyone,

It has been reported and replicated that device are failing to login which is impacting our (EU) European Union deployment.

We are investigating this within operations and will provide further details as they become available.

Please Note: We have had reports that restarting the device has resolved the issues but this is not confirmed by NT-ware at this time.
Posted Jun 27, 2022 - 08:06 UTC
This incident affected: EU Deployment (Identification, General Printing).