At approximately 12:45 PM PT on December 13, 2023, a server which had been temporarily removed from service for maintenance was put back into rotation. A misconfiguration on that server prevented it from processing the requests it received, resulting in intermittent service of the Inkling reader. During this time - approximately 53 minutes - users of the platform may have noticed some pages responding slowly or returning errors. Refreshing the page may have shown ordinary performance.
To resolve this issue, engineers identified the incorrect configuration and restored it, which immediately brought the platform back to normal. The documentation concerning the maintenance procedure has been updated to include explicit instructions surrounding this issue to prevent any further interruptions. The incident generated a pattern in metrics which we have also set up additional monitoring around to improve response time for this and other potential issues which might present in the same way.
