Affected
Partial outage from 3:30 PM to 6:57 PM, Degraded performance from 6:57 PM to 7:50 PM, Operational from 7:50 PM to 3:24 PM, Degraded performance from 3:24 PM to 8:09 PM, Operational from 8:09 PM to 4:48 PM, Degraded performance from 4:48 PM to 5:34 PM
- ResolvedResolved
This incident has been resolved. We identified the exact issue that caused the problem that began on 12/30. Please reach out to us if you need any support or assistance with issuing refunds.
- UpdateUpdate
We are still seeing intermittent issues, and some payments may require manual processing. We are actively working to resolve this as quickly as possible. If you experience any issues, please reach out to us and we will work with you to diagnose and address them.
- MonitoringMonitoring
We implemented a fix and are currently monitoring the result. There are still some reports that we will be continue looking into.
- ResolvedResolved
We ran a script that reconciled most payments that required manual processing. However, some payments did not have a numberReference or pspReference and therefore still require manual intervention. Please reach out to support if you have any questions.
- UpdateUpdate
We wanted to share an update on the payment processing issue we have been investigating.
As part of our work, we aligned our staging and production environments to ensure consistency and reduce the risk of environment-specific issues. We also upgraded the Adyen components to their latest versions to benefit from recent fixes and improvements.
During this process, we identified an error that could cause some payments to require manual processing. This issue has now been fixed.
The updated changes have been successfully deployed to production. Customers should be able to complete payments as expected, and we have put measures in place to prevent the issue from recurring.
Thank you for your patience, and please let us know if you experience any further issues.
- UpdateUpdate
We are continuing to investigate this issue. Customers should be able to complete their payments, and we are running scripts to automatically reconcile them to avoid manual processing.
- MonitoringMonitoring
We have received reports that payments are going through and funds are being collected. Adyen shows the status as "SettleScheduled", while CTP remains "Pending". The funds have been collected, but the payment data is not appearing in DME even when it set to auto-post.
- ResolvedResolved
This incident has been resolved. We identified the issue and deployed a fix. We will begin contacting customers and re-enabling Click-to-Pay. I will follow up with an official breakdown of what happened and how we can avoid this in the future.
- MonitoringMonitoring
We have implemented a fix and are currently monitoring the results while performing additional testing. We have not yet begun re-enabling Click-to-Pay (CTP) for those who requested it.
- UpdateUpdate
We are starting to test the fix, and once testing is complete, we will push an update. We will re-enable all Click-to-Pay (CTP) accounts that were deactivated. The update will include additional retries to properly verify transaction status and prevent duplicate payments.
- UpdateUpdate
We expect to restore Click-to-Pay functionality by the end of the day today. The issue is occurring intermittently. We will upgrade all components to the latest versions, perform end-to-end testing, and then re-enable Click-to-Pay.
- UpdateUpdate
We are continuing to work on a fix for this incident. We have reached out to our payment provider for assistance and are working with them to resolve the issue.
- IdentifiedIdentified
When processing a payment via CTP’s Click-to-Pay link, an incorrect “Something went wrong” message may appear even though the transaction is successfully completed. As a result, customers may attempt to pay again, which can lead to duplicate payments.
If you’d prefer, you can reach out to us to temporarily disable Click-to-Pay. We’re actively working on a fix and expect to have it resolved by tomorrow.
