Open Play – the background

During CDN World Summit, Jet-Stream founder Stef van der Ziel introduced Open Play CDN federation in his Keynote speech. This topic was also one of the CDN Academy courses presented during the event.

(Important: read the article About Federation first) and download the slides here (PDF 1.3MB). This article by Stef describes the background of how Jet-Stream conceptualized, materialized and developed full CDN integration:

CDN interconnection has been on our agenda for many years. We built the first federated CDN in 2000 and had taken part of many discussions in the early years of CDN around the concept of federation on all levels: business, strategy, technology, standardization, business logic. However CDN interconnection is one of the hardest problems to solve. It is so complex, because you need to oversee and consider the entire spectrum of the CDN space. The pitfalls are easy, you can completely lose yourself in theoretical business models and scenarios which will never materialize, but you can also very quickly develop a blinding focus on too narrow scenarios. You can completely lose yourselves in feature hell for years, but you can also make too easy architecture decisions that will have dramatic impact in the future, rendering a solution useless when the industry evolves. The CDN interconnection feature was put on our roadmap many years ago but we felt the ideas were not materialized enough to start development. Sometimes it just takes time to execute things right.

We knew that a locked in Jet-Stream-to-Jet-Stream solution would be a lock-in and unacceptable for the industry. We knew that it would take too much time for standards to become usable and that standards would be limiting us and our customers. We knew that developing a separate translation layer between CDNs was not a good option, too many elements. We also knew that overlaying was too simple, we needed actual integration with CDNs. We knew that a solution should be usable both as an upstream and a downstream CDN.

We also knew that we wanted to treat other CDNs as a logical server node, just as if it was an actual edge server or edge PoP. That was an important decision. Which we actually could do, since we had designed our architecture quite abstracted. We had done quite some core innovations before which helped us implement a true holistic CDN integration environment. Many prior Jet-Stream inventions helped us to implement a true holistic integrated multi-CDN solution:

  • Software only CDN invention: delivery and management in software enables the ability to run on top of commoditized servers: physical, virtual, cloud, and any mix of these
  • Parallelization invention: the ability to support and run multiple applications in parallel on a single node to prevent performance and stability issues when stacking of virtualization on top of virtualization
  • Separation of the data plane and the control plane invention (which is now coined as Software Defined Networking)
  • Reduction of complexity in the data plane (dumb boxes) and an intelligent control plane (active core CDN management system)
  • Abstraction of delivery technologies through APIs invention: the ability to integrate with various delivery technologies on the delivery plane, enabling a homogenous management plane and simplified communication between abstracted layers
  • Intelligent management plane invention, enabling instant intelligence about status and functionality within the underlying data plane
  • Software assigned delivery node roles invention: intake(origin), core, overflow, edge roles
  • Tiered distribution and delivery layers invention: the ability to dynamically send users to multiple layers inside the CDN
  • Active request routing invention: the ability to dynamically send users to delivery nodes per individual request instead of passive low-granular DNS or basic request routing
  • Request analysis invention: active analysis of requests using multiple internal and external parameters to select the most optimal delivery node and layer per individual request
  • Business logic invention: the ability to set dynamic rules and receive external information to optimize the selection of the most optimal delivery node per object request
  • Dynamic caching management invention: the ability to rewrite requests dynamically to support multiple origin services enables easy integration with simplified data planes

Before we started implementing the additional required technologies, I started to analyze business scenarios. Jet-Stream is not a regular vendor that innovates technology, we always start from strategic and business analysis and then R&D technologies that can be optimally used to support these business opportunities. This process took a long time. I’ve always known we were thinking in the right direction but the ideas were not clear enough to start development.

The ideas materialized beginning this year. Our own StreamZilla CDN service was only focussing on Europe: our customers typically have a European audience and our performance to the US was ok. However a number of customers asked for improved performance in the USA. So we looked at our options:

  1. Deploy a physical edge node. Too expensive for the volumes required
  2. Rent virtual machines through an online hosting provider with US presence. Low cost, fast deployment, pay as you grow.
  3. Offload into Amazon Cloudfront. Low cost, fast deployment, pay as you grow. Required simple integration.
  4. Offload into generic CDNs. Low cost, fast deployment, pay as you grow. Required medium integration.

We decided to kill option one, go for option two in the short term and investigate options 3 and 4 as well. We litterally ordered and paid for the virtual machines through an online store, got access within four hours and two hours later the edges were configured, tested and added to our pool of resources. That is the power of cloud. That is the power of software CDNs.

At around the same time we had meetings with various telecom operators who were sharing their ideas about Network Function Virtualization, and that fully aligned with our ideas to run CDNs on clouds. Jet-Stream demonstrated the ability to run both the control and data planes fully on Amazon years ago. After some technical and business sessions both Jet-Stream and the operators concluded that the need for the complex, expensive and limiting CDN federation standards was rendered useless by the ease of use of commoditized cloud infrastructures. The appliance is dead, CDN federation unnecessary and software CDNs were the future. A future Jet-Stream invented over ten years ago :)

When we started to investigate the capabilities of other CDNs for integration purposes it was actually the first time we encountered the features and APIs of other CDNs. And we had mixed feelings. On one side it was a shock to see how poor the feature set of regular CDNs typically is. You can set some parameters like C-naming and TTL and hopefully you can get some logs out of these systems, but that is about it. If you compare that to our advanced media workflow management environment and rich APIs it’s really a shocker. On the other hand we managed to make integrated offloading work with a number of CDNs in a few days time, with some hacks in an isolated development environment: our management system didn’t know any better than that it was talking to a physical edge node, while in reality it was an account on a third party CDN. And live streams flowed through. Score!

So we reached out to a number of CDNs and asked them to implement some additional features, or to explain why basic features such as anti-deep-linking token management didn’t work according to their documentation, or to ask adjustments to logs, et cetera.

It was interesting to hear that these CDNs were very open to cooperate. We had always seen other CDNs as competition for StreamZilla. However, we started to see other CDNs as commoditized infrastructures which we could use to offload (overflow) to. And they were happy to become a resource supplier they have overcapacity, need volume and liked us adding unique value to their systems.

It also aligned with my vision with StreamZilla: do not invest in commodities, invest in innovation instead. Regular CDNs invest millions in PoPs, operational staff, sales staff. StreamZilla’s strategy has always been to outsource commodities and automate everything via software to reduce costs.

And that was where all ideas came together: the ability to really integrate multiple CDNs in a rich CDN management system. Suddenly we could support a whole range of new business options. One of the benefits of running your own CDN is that you pragmatically encounter content owner requests and operational challenges on a daily basis which you immediately can translate into innovations. Not that many vendors who can do that.

The actual development of integrated 3rd party CDN support in our technology took less than three months to implement, including the framework, the enhancements of existing technologies and basic support of a limited number of CDNs:

  • Upgrade of our multi-granular topo/geo system to support multiple granularities (for instance countries and ISPs) at the same time, so we can prioritize small edge nodes within ISP networks over edge definded CDNs that define a larger area (like a country)
  • Rewriting of our caching back-end system to be able to dynamically rewrite inbound requests from 3rd party CDNs to be able to dynamically send their requests to the right origins and the right services (multiple web and caching services) within the Jet-Stream CDN
  • Upgrade from three distribution tiers to four (from intake->core->overflow/edge to intake->core->overflow->edge/3rdpartyCDN
  • Upgrade from three request routing tiers to four (from core->overflow->edge to core->overflow->3rdpartyCDN->edge
  • Added the ability to define accounts on third party CDNs as delivery nodes within the Jet-Stream CDN management system
  • Added support for tokenized access control to third party CDNs to prevent anti-deep-linking
  • Added support for 3rd party access log processing
  • Minor enhancements

We found out that the access logs and the access management tokens were the hardest nut to crack, and they still are with several CDNs. Many CDNs have quite exotic and limited implementations which in some cases makes access control and log processing impossible. We also encountered CDNs who claim to support HTTP adaptive streaming but in reality they have quite some caching issues and also struggle with the Thundering Herd problem for HTTP live streaming. But in general my (amazing!) development team was able to make basic integration and basic functionality work with some CDNs and got things working. Compared to having to implement proposed CDN interconnectivity implementation was a breeze on Jet-Stream side an almost nonexistent on the third party CDNs sides while Jet-Stream can offer much higher standards in terms of features, flexibility and performance.

The most important is that the core foundation for integrated third party CDN support as overflow or edge nodes within the Jet-Stream CDN is fully implemented. No additional screens, no separate tools. Between our list of delivery nodes you will find third party CDNs. Just as if they were always there.

This means that it is now relatively simple to add support for new CDNs and also simple to enhance the feature set with pre-integrated CDNs. And it is very easy to keep adding additional business logic to further optimize costs and performance, and offer more USPs compared to relatively simple overlay systems. Oh and we don’t force anyone to run code within clients. CDNs cannot enforce this, remember?

(slides 10 and 11) In less than three months time, our system had become a meta-CDN management system, that allows you to mix all kinds of resources within one CDN: mix physical servers, virtual servers, cloud based servers and 3rd party CDNs as if they are logical building blocks. All our existing logic applies to all delivery nodes. It is not a bolted-on feature.

All premium features apply, and the CDN management system is intelligent so it knows which services and features are and are not supported per logical delivery resource. It automatically geo load balances requests dynamically over any mix of nodes, including 3rd party CDNs, based upon business logic, popularity, performance and many other static and dynamic values.

(slides 13 and the rest) Some interesting new business scenarios thanks to Open Play:

  • Load balance requests on-net and off-net to internal and external resources right within one integrated CDN management environment, without the need for separate overlay services or technologies. Operators can offer global capacity to their content provider customers and extremely optimize costs by keeping traffic on-net and sending traffic off-net using smart business rules.
  • Build a massive virtual CDN: host our software on 2-8 cloud servers, use multiple CDNs (pay as you grow) for delivery. No CAPEX, no OPEX. (Let us) deploy (and host and manage) your own competitive feature-rich CDN in weeks at the lowest possible investments.
  • Content providers: load balance over multiple CDNs without the need for negotiating contracts and integrating APIs with multiple CDNs and also with an additional overlay service provider. Just one contract partner with a much richer feature stack compared to simple overlay systems.
  • Become a broker: sell traffic volumes to content providers, then purchase volumes from the best/cheapest CDN
  • Start a marketplace: sell traffic volumes to content providers, let third party CDNs bid for the volume at the lowest price.

Oh and of course virtually all 3rd party CDNs, clouds and hosted services can be used as a remote origin for the Jet-Stream CDN. So it can also be used as a downstream CDN.

Open Play is the first real CDN interconnectivity technology and it is executed in a beautiful integrated way. Contact us for information and a demo! The new StreamZilla CDN will be the first CDN to operationally implement it. The technology includes many unique Jet-Stream innovations and these are protected. Note that all described technologies are unique Jet-Stream IPR, we explicitly do not give you the right to mimick or replicate any of the described mechanisms or technologies in any way. Jet-Stream is opening up our code through licensing to anyone. Stealing is intellectual poverty. License and let’s share innovation!