On September 13, 2023, from 7:14 AM PT to 7:22 AM PT, performance issues in Inkling's central authentication service created failures throughout the rest of the platform. The problems related to a dependency between the authentication service and another internal application which created latency in the ordinary handling of authentication requests.
Engineering teams have responded to these issues in the following ways:
- Many of the known performance issues have been isolated and corrected.
- Hardware capacity has been added to this service to help support it during peak demand.
- Additional performance monitoring tools have been installed to help detect this type of issue going forward. This has already helped identify a number of places where performance of this critical service can be improved.
- Engineering work to identify and resolve this type of problem has been prioritized.
