I’ve been really actively involved with building things on Microsoft’s Azure cloud for probably about 4 or 5 years now. Many of you will know already that Have I been pwned (HIBP) was built from the ground up on Azure (in fact, one of the reasons I built the service was to play with Azure “in anger”!), what less people know is the work I’d been doing before that. In my previous life looking after Pfizer’s software architecture in this corner of the world, I was pushing hard to move apps we were building into Azure, in particular the PaaS constructs they have available. Time and time again, the discussion would go like this:
Vendor: (Pfizer outsourced all their dev work to vendors) We’re building this web app for you so we need a virtual machine and a SQL database.
Me: Is there a reason we can’t run it in the Azure App Service under a PaaS model?
Vendor: Uh, we want control!
Me: Control of what?
Vendor: We need to be able to remote into the machine.
because that’s the way we’ve always done it. No – we mean to troubleshoot stuff and access logs!
Me: So everything you’ve just described to me (and it’s usually a lot more than just the above point!) can be done with the App Service and the information surfaced in the portal, PowerShell and APIs.
Vendor: Well, we still want a SQL database!
Vendor: Because we want to store data – everyone knows that whenever you want to store any data of any kind the only possible way to do it is with a full-blown SQL Server database!
Me: What about other storage constructs such as blobs, tables and queues? They can be significantly cheaper and there are use cases where they’re faster too. Or a combination of these storage constructs and an RDBMS, how about that?
Vendor: Uh, but then we’d have to learn new things…
Which brings me to this new course because yes, if you want to learn how to do things more efficiently you do need to learn new things! That was the most common barrier I faced back then: I struggled to move things in the right direction because people were happy doing what they’d always done and consequently, there was a kludge of virtual machines all over the place managed by all sorts of people and connecting to expensive SQL Server databases that were massively underutilised. But I digress, let’s get back to Azure.
I’ve written extensively in the past about how I’ve built HIBP on Azure. In fact, there’s a great “thousand foot” view of it here in an infographic I did a couple of years ago (it’s still mostly accurate as of today, the addition of Cloudflare is the biggest change):
There you’ll see a liberal use of PaaS constructs for the web and database as well as the use of tables, blobs, queues and web jobs, alongside the aforementioned RDBMS. I love this model because it’s a hybrid of technologies which applies each in a way that leverages their strengths including performance, scalability and cost. Table storage is probably the epitome of that: I store about 2.7 billion (yes, with a “b”) records in there and it takes as little as a single-digit millisecond to look up a record. It scales in a totally linear fashion too so imagine one line shooting for the sky and that reflects traffic numbers (into 7 digit unique visitors in a day at times) and another line that’s just perfectly horizontal which is the table storage performance. And the cost for such performance? My last table storage bill was $50 in our Aussie money for the month which is about $38 in the US – and that’s for over 400GB of data performing at those levels too.
The point of all this is that there are better ways of building applications on the cloud than just taking yesterday’s patterns and repeating them in this new paradigm. When I was in London for the NDC event in January, I got together with Aaron Powell and we recorded the Play by Play course in the title of this blog post – Azure Beyond Websites – to talk about many of the things I touched on above. More importantly though, we show these in action. This is a very demo-centric Play by Play so there’s a lot of this sort of stuff going one:
In the one-and-a-half hours the course runs for, we end up building out a heap of features from the ground up including creating the website, provisioning the Azure resources, committing to GitHub, configuring auto-deployment, connecting to table storage, using queues, creating a WebJob, using Functions and eventually, tying it all together to watch as data flows between everything. I’m really happy with the way this all came together, particularly the way the workflow right at the end shows how easy it is to use this technology to do awesome things.