A small number of users who submitted mobile print jobs during this incident experienced delays in retrieving their files. Users submitted mobile print jobs are not believed to be lost with the exception of one (1) file that was corrupted and caused the conversion queue to fail.
This only affected the Japan deployment and was limited to a small number of users and approximately 20 print jobs.
Jun 16, 2020 - 05:39 UTC
Jun 16, 2020 - 06:14 UTC
It was found that a single Excel file had caused the mobile print conversion engine to become unresponsive. This same file was submitted multiple times blocking all conversion engines. This ultimately meant jobs from that point forward were now not being converted. The resolution was to clear the offending job from all queues and restart the services. The then saw the job processing return to normal.
As we can not ensure the integrity of a submitted file and nor should we. The development team has implemented safeguards to ensure this type of incident can not occur in the future. The processing queues are now monitored for stale jobs and if a timing threshold is reached we will remove the file first followed by a service restart. This fix will be incorporated in the next release and will be monitored closely to ensure its effectiveness.