Let me begin by quoting Stefan during the livestream: "Turns out having tons of data integrity is expensive". Yeah, and working with tons of data in a fashion that's both fast and cost effective is bloody painful. I'm reminded of the old "fast, good and cheap - pick 2" saying, but there's a lot more nuance to it than that, of course. I mean Table Storage was all 3 of those, just so long as we never needed to restore at all, let alone to a point in time. Or geo-replicate. Or do ad hoc queries and do on and so forth. Mind you, I think that with a combination of Azure SQL in Hyperscale mode, some better index optimisation, and a willingness to scale up more aggressively when processing large breaches, we might be able to find a happy balance. Literally as I'm writing this, we're upgrading to Hyperscale so hopefully when I do next week's video from Tokyo, there'll be a happy story to tell (or I'll be drowning my sorrows in sake).
References
- Sponsored by: Kolide ensures that if a device isn't secure, it can't access your apps. It's Device Trust for Okta. Watch the demo today!
- The German government has become the 35th national gov to be granted access to all their gov domains in HIBP (and one more to come next week)
- WoTLabs got very pwned (site defacement on top of leaked data is never a good look)
- The Онлайн Трейд (Online Trade) breach was an oldie, but it's helping us tune the import process as part of the RDBMS rollover (which is... painful)
- Speaking of RDBMS rollover, most of the ideas I had during this video have proven to be completely useless, so we're now rolling to Hyperscale as well (it's actually only very slightly more expensive)
- We're still contributing to the HIBP UX rebuild repo (consider it a "soft launch" for now, I'll blog about it in more detail after I get back from Japan)