Connect All The Things!

The ZeroTier Blog

ZeroTier 2018 Road Map

 

In March of 2018 ZeroTier will turn three years old as a company and five as a project. Today I'd like to revisit and re-articulate our vision and talk a bit about what we hope to accomplish in 2018 and beyond.

... read more

ZeroTier's mission statement is to directly connect the world's devices.

At first this might seem a puzzling reason for a venture to exist. It's 2017 and Internet connections are available almost everywhere. The key word in our statement is directly. Take a few arbitrary devices -- a laptop, a server in a closet, and a cloud VM -- and try to connect them directly so that each device can easily communicate with its peers. Keep in mind that the server is behind an edge router and the laptop is mobile. Now throw in a phone, a cloud VM at a different data center, and a few desktops at your other branch office.

It's 2017. We think this should be easy. It's not. That's why we exist.

A long series of largely unplanned and uncoordinated engineering decisions have given us networks that are remarkably inflexible. The wires go everywhere, but outside of local area networks and vertically integrated clouds it's difficult to do anything more sophisticated than basic client-server requests to fixed public IPs.

People have become so accustomed to these limitations they're largely oblivious to just how much time, money, and mental bandwidth they waste working around them. I could spend quite a bit of time listing technologies, design patterns, and even entire publicly traded companies that exist largely for no other reason than to work around the inconvenience of just connecting things.

In the past 3-5 years we've come a long way toward realizing this vision. We've deployed a software-based secure "planetary smart switch" that allows peer-to-peer virtual networks to be created and joined with ease. We've started describing ZeroTier virtual networks as as "chat rooms for machines." It's a bit like Amazon Virtual Private Cloud networking for the entire Internet from the core and the cloud all the way out to the edge.

While we've experienced some hiccups in our SaaS hosted secondary services (we'll talk about this more later) our core virtual switch has remained remarkably stable and has recently passed three years of uninterrupted global uptime. Years of 11-14% monthly growth in connected device count has caused no scalability problems. We consider this to be excellent real world validation of our core architecture and its ability to scale.

We've also managed to ship ZeroTier for six operating systems and many third party products like NAS devices and routers. We've been included in projects like FreeNAS and opnsense. Last but not least we've shipped a rules engine in ZeroTier that is at least as powerful as what's found in most OpenFlow and other SDN/NFV products from major vendors. Being distributed and edge-centric it works differently but it offers similar capabilities.

So what now? A lot, it turns out. We're nowhere near "done," if that's even a meaningful word in software.

2018 Road Map

In 2018 our focus will be on reliability, advanced enterprise features, improvements to user and developer experience, and offering products like the ZeroTier Edge and others to extend connectivity to things that can't directly run ZeroTier.

Here's a quick TL;DR:


  • Virtual Network Reliability: Improve path selection and fault recovery algorithms.
  • SaaS Reliability: Multi-cloud, performance improvements, better orchestration.
  • SD-WAN Features: Multipath with trunking and hot standby, QoS awareness, plugin architecture for protocol classification and integration with rules engine.
  • UI / UX / DX: ZeroTier Central GUI improvements, new network configuration methods that allow networks to be managed and deployed as code, integration of controllers/Central with client UI for Mac and Windows.

The ZeroTier Edge and SD-WAN Capabilities


In recent years we've seen a proliferation of products under the banner of SD-WAN or software defined wide area networking. These are advanced managed VPNs that support sophisticated multi-path logic to achieve leased line levels of reliablity over redundant bundles of much cheaper commodity Internet connections. They also generally support rules, protocol fingerprinting, and advanced remote diagnostics features. SD-WAN is typically delivered in the form of a physical device and its primary market niche is in displacing older much more costly and inflexible leased line technologies like MPLS.

ZeroTier's users have independently deployed it in simple SD-WAN roles for quite some time, but setting up a SD-WAN box with ZeroTier takes time and Linux system administration expertise. To make this easier for more people we decided to create the ZeroTier Edge. As of today we are pre-selling it at a discount and plan to start shipping the device in early 2018. Once shipments begin we will have the device on sale at full price via our site and probably venues like Amazon.

The Edge is a pre-configured network bridge, effectively giving you physical ports into virtual networks. To deliver a full SD-WAN solution we are also using the Edge project as an excuse to add SD-WAN type features to the ZeroTier core network virtualization engine.

In the longer term we have more ideas for the Edge that we plan to distribute to it via software updates. It will be a growing platform with a growing feature set.

Remote Management and Diagnostics

Right now if you have issues with ZeroTier and need to debug you generally have to access the remote system with ssh, RDP, etc. There are no built-in facilities for doing remote diagnostics on ZeroTier devices through the ZeroTier protocol itself.

We will soon be changing that. In the current development tree networks can set a network-level ZeroTier address to receive remote trace information about important events happening on the node that pertain to a given network. Nodes can also set a global remote trace recipient to receive all events. (This must be done locally on the node since it would be a security risk to allow one network to configure tracing for all networks.)

The second phase will be introduction of remote management capabilities. Right now these don't really exist. This will also require node opt-in and will allow ZeroTier devices to control other ZeroTier devices via the ZeroTier protocol itself.

These features will make it much easier to administrate and debug ZeroTier networks from one place without ssh'ing and RDP'ing all over your network.

Developer-Friendly Network Management

ZeroTier Central (our only real non-OSS product) gives you a pretty UI and an API to control networks and network controllers. It's not mandatory to use ZeroTier but it makes things easier.

Right now Central is GUI-centric and doesn't have good facilities for rapidly cloning or otherwise manipulating networks the way developers can manipulate git repositories or virtual machines on higher-end VM and cloud hosts.

We're working on a more advanced design that allows networks to be viewed, edited, and managed as either code or UI elements with seamless two-way movement between these two paradigms. This will allow networks to be manipulated like git repositories (and maybe eventually with git) while at the same time offering a convenient GUI. GUIs are better for quick changes and viewing the state of things at a glance while code is better for more in-depth and large-scale configuration of systems. We'll be offering both worlds.

We're also working to make it easier to run your own ZeroTier Central locally or in your own cloud. We already have license management and a Docker image for this and we'll be adding instructions for provisioning it without Docker and auto-updates to the mix.

SaaS Infrastructure Improvements

The recent OVH mega-outage took my.zerotier.com offline for several hours by disrupting two of the three data centers where it is hosted.

"Three data centers going down at once," we thought, "that can't happen!" :)

Lots of people wrote in to say "go to AWS," with a few going so far as to argue that AWS is a requirement to be a "serious" company. We disagree. All providers have issues. Earlier this year for example S3 went down for all of Northern Virginia. AWS is very good at what they do but we think it's better to put more eggs in more baskets.

To prevent another outage similar to this one we will soon be going multi-cloud. We'll be keeping our current OVH footprint but adding capacity at data centers owned by at least one more provider.

Our most important infrastructure, namely our VL1 root servers, have always been multi-cloud and have recently passed three years of un-interrupted global uptime. Five different cloud providers on seven continents would have to fail for Alice and Bob to be taken offline.

Blowing Up the Moon (Easier Federation)

ZeroTier VL1 is an anchored cryptographically-addressed peer to peer network. The "anchors" are called root servers as they serve a role very closely analogous to DNS root name servers. Right now we run them, but a while back we added an easier (no source hacking) way to run your own and add them alongside. There are many reasons people want this with independence, robustness, performance, and air-gapped operation being the most popular.

While "moons" a.k.a. "federated roots" were a usability improvement over source hacking, they're not really what we want. They're another thing you have to deploy and manage and adding them is another step that has to be done at every node. We want something more transparent and something that makes air-gapped operation easier and more elegant.

Right now we have at least one major customer with a strong interest in this, so vastly improving federated and detached operation is probably going to land somewhere in 2018.

Network Containers, Take Two

Nobody understood network containers. The idea is simple but it's so foreign to how networking is typically done with containers and thin VMs that most people had trouble wrapping their minds around it. We still think it's a good idea and we have not given up on it.

The idea behind network containers is to allow containers and other modular jobs (Docker, LXC, Rkt, Nomad, etc.) to be deployed and migrated without any need to consider networking beyond the mere presence of connectivity. Each container holds its own unprivileged network stack that possesses a cryptographically defined static IP via ZeroTier and can travel with it.

The code behind network containers is libzt. Right now it's used by application developers to provide zero-privilege virtual network connectivity to apps. This is really the same use case but in a different deployment scenario, namely that of an app on a phone or a desktop computer rather than a container or app deployed on a server.

In 2018 we plan to explore how the network containers concept could be used to deliver modular plug and play distributed services on top of popular container and orchestration engines. When coupled with our planned developer friendliness improvements to ZeroTier Central this could allow entire infrastructures including their network topologies to be deployed onto any cloud or private data center without modification or customization. As it stands the complexity and variation of networking is a major barrier preventing the equivalent of "git clone ; make ; run" for entire infrastructure snapshots.

Conclusion

What we're working toward is a networking system that encompasses and renders obsolete the functionality of a huge number of acronyms: VPN, DVPN, SD-WAN, SDN, NFV, and so forth. Our goal is to create a global network where our users can simply connect stuff.

We're not just doing this to make personal and business networking easier. ZeroTier began life as an Internet decentralization project. We believe that the difficulty and inflexibility of networking is a major factor driving more and more of the Internet's functionality into vertically integrated silos managed by a small number of large companies. We hope that by making networking agile, secure, and easy we will help to empower a new generation of endpoint and edge centric applications and services that place control of compute and data back into the hands of the Internet's users.

If you like what we're doing here and want to support us there are three simple things you can do: tell other people about ZeroTier, upgrade to a paid account at our hosted ZeroTier Central, and pre-order the ZeroTier Edge. If $150 is a bit steep for you we also have a $40 support category that includes our famous "connect ALL the things!" mug (with thanks to Allie Brosh for the inspiration).

Sincerely,

Adam Ierymenko
Founder and CEO
ZeroTier, Inc.
November 16th, 2017


Connect

Company