On the morning of August 31, 2023, the servers that process media when projects are published came under abnormally heavy load due to a defect in this service. At 7:22 AM Pacific Time, these servers became unresponsive. This resulted in some images not loading in Habitat and a small number of failed builds. Inkling Engineering responded by restarting the relevant services. The issues were resolved by 7:34. Additional capacity was added to these servers to further mitigate these issues while Engineering diagnosed the problem. A fix was later deployed to correct this problem permanently. All builds which failed during the time of the incident were rerun shortly after service was restored. All of those builds completed successfully.
