Inkling - Habitat Performance Degradation - Investigating – Incident details

Habitat Performance Degradation - Investigating

Resolved
Partial outage
Started about 1 year agoLasted about 3 hours

Affected

Habitat

Partial outage from 2:24 PM to 5:52 PM

Updates
  • Resolved
    Update

    On the morning of August 31, 2023, the servers that process media when projects are published came under abnormally heavy load due to a defect in this service. At 7:22 AM Pacific Time, these servers became unresponsive. This resulted in some images not loading in Habitat and a small number of failed builds. Inkling Engineering responded by restarting the relevant services. The issues were resolved by 7:34. Additional capacity was added to these servers to further mitigate these issues while Engineering diagnosed the problem. A fix was later deployed to correct this problem permanently. All builds which failed during the time of the incident were rerun shortly after service was restored. All of those builds completed successfully.

  • Resolved
    Resolved

    This incident has been resolved.

  • Monitoring
    Monitoring

    We implemented a fix and are currently monitoring the result.

  • Identified
    Identified

    We are continuing to work on a fix for this incident.

  • Investigating
    Investigating

    Inkling has received reports of Habitat performance degradation and our team is currently investigating.