Impacted by your broadband provider’s outage?!

On October 14th 2018, one of Ireland’s largest broadband providers, Eir, had an outage that affect all of their customers and potentially their wholesale customers also.

What was the impact?

Eir worked hard to get the issue sorted and they did have it turned around in just over 12 hours. However, 12 hours without broadband access can be very painful for business customers. It could mean no orders coming through, payments delayed, services unreachable, users not being able to work, etc. The impact on business is quite significant!

What happened?

As it turns out, this outage was wholly avoidable for customers, with a little bit of configuration change on their side. The broadband was not down but, in fact, it was Eir’s domain name system (DNS) that resolves hostnames/domain names to IP addresses that was causing the problem. The impact on Eir’s customers was that their browsers/systems could not convert the hostnames to IP address, which all happens in the background, meaning they effectively could not connect to the internet as usual.

What to do to mitigate against it?

This isn’t the first time this has happened either. It can happen far too frequently for an ISP or broadband provider to have problems with their DNS services.  ISPs and Broadband providers run their own DNS servers with caching generally turned on, to give their customers a local service for name resolution. The benefit of using your ISP’s DNS over a third party DNS is to limit the distance your traffic has to travel to resolve a name. The shorter the distance the quicker the response, right?

Well, that is the case in theory but in practice it’s a lot different.  Some ISPs do not invest in a robust DNS infrastructure, while DNS is a critical service, it’s generally secondary to their objectives as an organisation. This means the DNS servers are usually under powered and slow to respond to new or even cached resolution requests.

The solution is to take advantage of a third party DNS, built from the ground up for speed, reliability and availability.

Other DNS providers?

There are many third party DNS providers out there, but here are the three most commonly known players in the market:

  • Cloudflare  –  Primary: 1.1.1.1 , Secondary: 1.0.0.1
  • OpenDNS (now Cisco Umbrella) – Primary: 208.67.222.222 , Secondary: 208.67.220.220
  • Google Public DNS – Primary: 8.8.8.8 , Secondary: 8.8.4.4

Lifewire has a good list of the free and public DNS servers if you wish to see what other options are out there.

DNS Response Times
(provided by Cloudflare)

These third party DNS services run on a geo-based network which sends your requests to the server closest to your IP source. For example, if you use Google’s DNS, and your are based in Ireland, your DNS requests are not being sent to California, they are in fact being routed to a Google service in a data center in Ireland.  Fast response from these services is the key to why they are a better bet that your ISPs DNS service!

Moreover, some of these services offer content controls too, like parental controls for all your internet traffic. They also offer benefits of privacy that your ISP DNS doesn’t. Speed, reliability and privacy are the main reasons for switching to a third party DNS (see the chart for response time comparisons).

How to switch your DNS?

There are a number of ways to switch to using a third party DNS, the easiest of which is to change the DNS settings on your broadband router to use the IPs of the third party DNS provider.  This will then mean that all devices (laptops, phones, desktops, etc) on your home or office network or WiFi will be using the third party DNS.  This will make your web traffic faster but most important, more reliable! Next time your ISP has a DNS outage, you wont even notice!

Each of the third party DNS listed above have their own setup guides, doesn’t matter which guide you follow, just replace the primary and secondary IPs with the ones of the provider you want to use.

More Security?

DNS is actually a very complex area that until recent years was largely neglected by most ISPs. More and more servers are now implementing DNSSEC (DNS Security Extensions) to help prevent attacks such as man-in-the-middle or cache poisoning. Newer protocols are also being developed/available that encrypt all the DNS requests between the client and the server to protect privacy, such as DNS over HTTPs (DoH).  You just need to find the service that best fits your needs.

Why Support Renewals?

You can use Support Renewals to track ISP contracts so you know when you are out-of-contract and ready to perhaps find a new supplier or re-negotiate for the next period. Find out how Support Renewals can help your company keep track of all your IT support renewals, contracts and licenses here.  Then, sign up for your account today!

References

https://www.rte.ie/news/2018/1013/1002966-eir-outage/

http://www.thejournal.ie/eir-broadband-down-4285098-Oct2018/

https://downdetector.ie/problems/eircom/news/227299-problems-at-eir-2

https://www.lifewire.com/free-and-public-dns-servers-2626062

https://en.wikipedia.org/wiki/Domain_Name_System_Security_Extensions

https://en.wikipedia.org/wiki/DNS_over_HTTPS

Share this with others...