On Tuesday 23 July Gradwell engineers were alerted to new call setup failures that impacted inbound and outbound calls. However, this incident did not impact any inflight calls. Gradwell engineers were fully engaged in the incident which was given a priority one status.
The root cause of the issue was traced to a fault in a vendor's software, causing a cascade failure which affected the system's ability to create new calls. This type of failure occurs when an initial error triggers a series of additional failures in the system. Fortunately, Gradwell’s automated monitoring and response systems were already in place, allowing us to quickly detect and resolve the problem without manual intervention.
Gradwell have identified further enhancements which can be made to speed up the automated processes to resolve these types of failures in more quickly. We have also identified other improvements to better isolate our systems from the vendor. Please accept Gradwell’s apologies for the service disruption and the impact it has had on your business.