This is part 4 of a six part series based on a talk I gave in Trento, Italy. To start from the beginning go here.
We don’t believe that any of our software, not a single line of code, provides us with a long-term advantage. We could, today, open source every single line of code at Cloudflare and we don’t believe we’d be hurt by it.
How we think about Open Source
Why don’t we? We actually do open source a lot of code, but we try to be thoughtful about it. Firstly, a lot of our code is so Cloudflare-specific, full of logic about how our service works, that it’s not generic enough for someone else to pick up and use for their service. So, for example, open sourcing the code that runs our web front end would be largely useless.
But other bits of software are generic. There’s currently a debate going on internally about a piece of software called Quicksilver. I mentioned before that Cloudflare used a distributed key-value store to send configuration to machines across the world. We used to use an open source project called Kyoto Tycoon. It was pretty cool.
But it ended up not scaling to our size. It was great when we had a small number of locations worldwide, but we ran into operational problems with 100s of locations. And it wasn’t, by default, secure and so we had to add security to it. Once we did, we open sourced that change, but at some point when using open source software you have to make a “modify or rewrite” decision.
We’d done that in the past with PowerDNS. Originally our DNS service was based on PowerDNS. And it was great. But as we scaled, we ran into problems fitting it into our system. Not because there’s something wrong with PowerDNS but because we have a lot of DNS-related logic and we were shoehorning things into PowerDNS, and it was getting less and less maintainable for us. This was not PowerDNS' fault; we'd built such a large edifice of business logic around it that PowerDNS was being crushed by the sheer weight of that logic: it made sense to start over and integrate logic and DNS into a single code base.
Eventually we wrote our own server, RRDNS, in Go, that is now the code behind the largest and fastest authoritative DNS service on the planet. That’s another piece of software we haven’t open sourced. That one because it’s riddled with business logic and handling of special conditions (like the unique challenges of working inside China).
But back to Quicksilver. It’s based on LMDB and does all data and code sync. across our global network. Typically, a change (you click a button in our UI or you upload code for our edge compute product) and it’s distributed globally in 5s. That’s cool.
And Quicksilver is generic. It doesn’t contain lots of Cloudflare-specific logic and it’s likely useful for others. The internal debate is about whether we have time to nurture and handle the community that would grow up around Quicksilver. You may recently have seen the creator of Ruby saying on Twitter “We are mere mortals” pointing out that the people behind popular open source projects only have so much time. And we take a lesson from the creators of Kyoto Tycoon who have now largely abandoned it to do other things.
Perhaps Quicksilver will get open sourced this year, we’ll see. But our rule for open sourcing is: “Is this something others can use and is this something we have time to maintain in public?”. Of course, where we modify existing open source software, we upstream everything we can. Inevitably, some projects don’t accept our PRs and so we have to maintain internal forks.
How we think about Patents
While we’re on the topic of intellectual property let’s talk about patents. Cloudflare has a lot of patents. Although it might be nice to live in a world where there were no software patents it’s a little like nuclear weapons. It’s very hard for one country to disarm unilaterally because a power imbalance is left behind. If Cloudflare didn’t patent aspects of our software, then others would and would then use them against us.
So, we patent for defensive reasons. To stop others from using the patent system against us.
Working With Governments
And speaking of patents let’s talk about governments. Lots of technology companies think they are too cool for school. They don’t need to think about governments because technology moves faster than them and what do those old, boring lawmakers know about anything anyway?
Wrong. Dead wrong.
Yes, governments move slowly. You actually want them to. Imagine if governments changed policies as fast as chat apps get launched. It would be a nightmare. But just because they are slow, they can’t be ignored.
Put simply governments have tanks and you don’t. Eventually lawmakers will make laws that affect you and unless you’ve spent time explaining to them what it is you do you may have a nasty surprise.
Cloudflare decided very early on to engage with lawmakers in the US and Europe. We did this by helping them understand what is happening in the Internet, what challenges we foresee, and helping them with the technical arcana that we all deal with.
If there’s any chance that your business ends up being regulated by a government then you should engage early. Cloudflare thinks a lot about things like copyright law, the fight against online extremism, and privacy. We have to because our network is used by 13 million web sites and services and all manner of things pass through it.
Lots of times people get mad at us because they don’t like a particular customer on our network. This is tough for us because oftentimes we don’t like them either. But here’s the tricky thing: do you really want me, or Matthew, deciding what’s online? Because many times that’s what angry mobs are asking.
“Shut this down”, “Throw this off your service”. It’s odd that people are asking that corporations be gatekeepers when corporations answer to shareholders and not the people. The right answer is that if you see something you don’t like online: engage in the political process in your country.
The transparency of democratic institutions and, in particular, the judiciary is vital to the long-term survival of countries. It’s through those institutions that people need to express their desire for what’s allowed and not allowed.
How do you engage with governments? Every single government has committees and advisory bodies that are dying to have people from industry help out. Go find the bodies that are doing work that overlaps with your company, don’t be put off by how old-fashioned they seem, and get involved.