Update regarding the App Data Loading incident

We would like to provide an update in full, in regards to the network issue (DNS) that at the end of last week affected some users’ ability to access and use Plum. DNS servers are what translate domain names into IP addresses to lead people to the correct places.

What was the issue?
In essence, our DNS provider had to perform an internal migration to another registrar which included a domain contact update. Typically, such updates are done quickly and we had been told that no action would be necessary on our side. Plum was informed that a verification process had not been completed at our end, resulting in the Internet Corporation for Assigned Names and Numbers (ICANN) suspending the ‘withplum.com’ domain, i.e. a so-called “domain suspension”. At that point, occurring at 2022-03-17 09:29 UTC, Plum users gradually started reaching the “domain suspension” page instead of actual Plum endpoints

During the time of the suspension, the entire Plum infrastructure continued to operate normally without compromise or data loss. For users who were not yet reached by the DNS record changes, the Plum application continued to function.

What did we do to fix the issue?
We found out that there were issues reaching our infrastructure at 2022-03-17 10:08 UTC. We immediately figured out that it was a DNS issue and verified the domain at 2022-03-17 10:15 UTC. In order to resolve the issue, we requested a DNS cache flush on all public DNS servers that support that (e.g. Cloudflare & Google). Then we started estimating how many users were affected at that moment and kept monitoring the situation as DNS changes were still propagating.

Is this still ongoing?
Yes. This issue is still affecting a very small percentage of our users. It is ongoing due to certain Internet Service Providers (ISPs) not respecting the Time-To-Live (TTL) property of DNS records. Such providers cache DNS records for an arbitrary amount of time which may be way higher than the TTL of the record. Therefore, our now fixed DNS records can still not reach their users.

We must assure you once again that while access to your account may have been temporarily interrupted, the safety or security of your money was never compromised in any way. We truly apologise for the inconvenience and are working hard to ensure the issue is fully resolved.

1 Like