We are continuing to monitor for any further issues.
Posted Oct 20, 2021 - 10:10 EDT
Update
We are continuing to monitor for any further issues.
Posted Oct 07, 2021 - 09:45 EDT
Monitoring
Our engineering team has implemented the required fix for monitors that use the old stack, as described in our KB - Emulated and Selenium API. We have tested and deployed this fix to the following nodes while we continue full regression testing and will deploy it to all nodes after testing is completed
Ashburn, VA - AWS Cloud, Dublin, IE - AWS Cloud, Washington, US - Azure Cloud, London, UK - Azure Cloud, Arlington, VA - Tencent Cloud, Mumbai, IN - Tencent Cloud
Please be reminded, Catchpoint did not experience outages; however, it did detect that some servers were not properly sending certificates and being impacted
Posted Oct 02, 2021 - 16:42 EDT
Update
Certificate issue for Object and API - Javascript tests is now resolved, we have finished updating all of our nodes. We are still working on making similar updates for our Emulated and API - Selenium monitors.
Posted Sep 30, 2021 - 17:54 EDT
Update
Today's issue is caused by an expired root CA certificate, used to sign server certificates issued by Let's Encrypt. This certificate expired at 10AM EDT today, September 30th. There can be two classes of issues which can cause your customers to not be able to connect to your site due to this certificate's expiration. The first is because the client device or OS has not been updated to include updated CA certificates. The second is because the client device or OS relies on underlying libraries which fail as soon as they see the expired CA certificate from the server, regardless of the presence of non-expired alternatives.
Catchpoint has updated all our public nodes (Backbone, Cloud, Last Mile, Wireless) to resolve these certificate errors for Object and API-Javascript tests. We are still working on making similar updates for our Emulated and API-Selenium monitors, which are running in our older HTTP stack. If you are relying on these, you have several options described below. Importantly, make sure to resolve the issue on your server - fixing the certificate chains on the Catchpoint nodes will resolve the test errors, but your clients may still have issues accessing your site, API, or other TLS-based service.
1. Remove the DST Root CA X3 from your server. If you do this, your server will stop sending this as an option to clients, and clients won't attempt to validate it. 2. Enable ignore SSL errors in your Catchpoint tests. 3. Switch tests to API - Javascript or Object - these use newer Catchpoint monitors which rely on different underlying libraries and are already using the updated certificate chain.
Alternately, Catchpoint engineering is working on updating the monitors which rely on our older HTTP stack. We will update on their progress when we have more information.
We are done updating about 50% of the nodes, we will provide an update when we have completed all of the impacted nodes.
Posted Sep 30, 2021 - 13:24 EDT
Identified
We have identified a fix for this issue and are working on updating our nodes.
Posted Sep 30, 2021 - 11:00 EDT
Investigating
We’re investigating an SSL issue caused by another Let's Encyrpt root certificate that was set to expire at 10 AM ET on 9/30, causing Object, emulated and API tests to fail for some of our clients.
We will update here as more information becomes available.