I was going to write about how much I've enjoyed "tinkering" with the HIBP API, but somehow, that term doesn't really seem appropriate any more for a service of this scale. On the contrary, we're putting in huge amounts of effort to get this thing fast, stable, and sustainable. We could do the first two very easily just by throwing money at the cloud, but that makes the last one a bit hard. Besides, both Stefán and I do enjoy the challenge of optimising an increasingly large system to run on a s...

I've spent more than a decade now writing about how to make Have I Been Pwned (HIBP) fast. Really fast. Fast to the extent that sometimes, it was even too fast: The response from each search was coming back so quickly that the user wasn’t sure if it was legitimately checking subsequent addresses they entered or if there was a glitch. Over the years, the service has evolved to use emerging new techniques to not just make things fast, but make them scale more under load, increase availability an...

I have absolutely no problem at all talking about the code I've screwed up. Perhaps that's partly because after 3 decades of writing software (and doing some meaningful stuff along the way), I'm not particularly concerned about showing my weaknesses. And this week, I screwed up a bunch of stuff; database queries that weren't resilient to SQL database scale changes, partially completed breach notifications I didn't notice until it was too late to easily fix, and some queries that performed so bad...

Apparently, before a child reaches the age of 13, advertisers will have gathered more 72 million data points on them. I knew I'd seen a metric about this sometime recently, so I went looking for "7,000", which perfectly illustrates how unaware we are of the extent of data collection on all of us. I started Have I Been Pwned (HIBP) in the first place because I was surprised at where my data had turned up in breaches. 11 years and 14 billion breached records later, I'm still surprised! Jason (not...

This was a much longer than usual update, largely due to the amount of time spent discussing the Earth 2 incident. As I said in the video (many times!), the amount of attention this has garnered from both Earth 2 users and the company itself is incommensurate with the impact of the incident itself. It's a nothing-burger. Email addresses and usernames, that's it, and of course, their association with the service, which may lead to some very targeted spam or phishing attempts. It's still a breach...

I have really clear memories of listening to the Stack Overflow podcast in the late 2000's and hearing Jeff and Joel talk about the various challenges they were facing and the things they did to overcome them. I just suddenly thought of that when realising how long this week's video went for with no real plan other than to talk about our HIBP backlog. People seem to love this in the same way I loved listening to the guys a decade and a half ago. I'll do one of these with Stefan as well over the...

Firstly, my apologies for the minute and a bit of echo at the start of this video, OBS had somehow magically decided to start recording both the primary mic and the one built into my camera. Easy fix, moving on... During the livestream, I was perplexed as to why the HIBP DB was suddenly maxing out. Turns out that this aligned with dropping a constraint on the table of domains which appears to have caused the table to reindex and massively slow down the queries for breached email addresses. Furt...

Apparently, Stefan and I trying to work stuff out in real time about how to build more efficient features in HIBP is entertaining watching! If I was to guess, I think it's just seeing people work through the logic of how things work and how we might be able to approach things differently, and doing it in real time very candidly. I'm totally happy doing that, and the comments from the audience did give us more good food for thought too. I'll try and line up a session just like that before the end...

It wasn't easy talking about the Muah.AI data breach. It's not just the rampant child sexual abuse material throughout the system (or at least requests for the AI to generate images of it), it's the reactions of people to it. The tweets justifying it on the basis of there being noo "actual" abuse, the characterisation of this being akin to "merely thoughts in someone's head", and following my recording of this video, the backlash from their users about any attempts to curb creating sexual image...

Ok, the scenery here is amazing, but the real story is data breach victim notification. Charlotte and I wanted to do this one together today and chat about some of the things we'd been hearing from government and law enforcement on our travels, and the victim notification angle featured heavily. She reminded me of the trouble even the police have when reaching out to organisations about security issues, often being confronted by lawyers or other company representatives worried about legal repris...