Building our next generation network

A post about the upgrades and changes being made to provide a next generation game server network....

3 months ago

Latest Post 23 things you didn’t know about Minecraft by CreeperHost Team

Starting in June 2019, we began building out our new network.

This started with beginning to use our Autonomous System number once again. It has been 5 years since we last attempted this.

Juniper MX960 and Infinera DWDM


In June, we purchased our first Juniper router, a Juniper MX960. The MX960 was then configured at our Grantham data centre, and three uplinks ordered;

We currently use a Riello Multi Sentry inline UPS in our Grantham location

Riello Multi Sentry UPS

And a Stulz HVAC

Stulz HVAC

We became members of the London Internet Exchange to allow low latency traffic to several major broadband providers and content providers. This means there are no third parties between ourselves and these networks so that any issues can be resolved directly with providers.

We took bandwidth from Voxility because of their presence in the Amsterdam Internet Exchange, Deutscher Commercial Internet Exchange and that they peer with Comcast in the United States, this reduced the regular multi-carrier routes to many European and US customers to our Grantham location. Meaning easier troubleshooting and lower latencies.

We've just placed an order for an additional 1gbps of transit from British Telecom, this will act as a backup to our main transit link to iomart in case of outages, and will be fitted to complete our requirement for diverse fibre routes. We will utilise this link also to provide better latencies to British Telecom's direct peers.

Our average traffic in our Grantham, UK location at this time is 450mbps, with peaks above 3gbps. However, this means our current bandwidth availability is well in excess of requirements.
Hardware was purchased, and our existing customers from Maidenhead and Gosport were transferred to Grantham in July, we saw a peak in customer retention at this point due to the improved network quality.

As of last week, we have committed to rolling out similar changes across our US network.

We've partnered with three providers in the United States, and we are placing our Juniper routers and equipment in Coresite LA2, ColoCrossing Buf1 and Cologix Dallas.

Our routers are still arriving at the time of writing this, our one in Coresite LA2 is currently on-site and racked, the one to Cologix Dallas arrived yesterday and is awaiting remote technicians to rack.

We have settled on our new specification for our compute nodes to host virtual servers (containers) and have placed orders for this hardware.

Currently, there has been over USD 115,000 invested on new compute nodes; this excludes the cost of switchgear, routers and procuring IPv4 space to facilitate building our new network and costs of technicians for building and connecting devices.

While hopefully, to the end-user, the only change you'll see is your server starts faster, and you have lower latency, this represents a significant investment and a massive undertaking. Very few other server hosts run their own networks from the ground up and have the kind of monitoring and metrics we do, for example full line-rate DPI using FPGA network cards, which we've used in the recent months in Grantham to diagnose lag on someone's server due to a flood of movement packets being sent to their client, saturating their home internet connection.

We will not name competitors, but we keep a close eye on the hardware and direction they take, and the most significant expansion of which we've seen recently involves them leasing hardware that came into service in 2013.

Our new hardware spec will be based on the latest generation Intel Xeon E-2288G and Intel Xeon E-2286G processors, using the latest DDR4 memory and clock speeds as high as 5ghz without overclocking.

This specification, we believe, will give unrivalled performance and stability at the hardware level.

For our United States networks, we're using Juniper MX-series routers a combination of on-site/in-line DDoS mitigation configured to offload to Voxility should the attack approach our in-line mitigation capacity.

This should provide high uptime due to a total mitigation capacity over 1tbps with the lowest latency possible for the situation.

So while others may be utilising their money to purchase advertising and gain customers, we remain as always committed to building the best product on the market, from the ground up.

Paul Taylor

Published 3 months ago

Comments?

Leave us your opinion.