Back in 2013, I was beginning to get the sense that data breaches were becoming a big thing. The prevalence of them seemed to be really ramping up as was the impact they were having on those of us that found ourselves in them, myself included. Increasingly, I was writing about what I thought was a pretty fascinating segment of the infosec industry; password reuse across Gawker and Twitter resulting in a breach of the former sending Acai berry spam via the latter. Sony Pictures passwords being, well, precisely the kind of terrible passwords we expect people to use but hey, actually seeing them for yourself is still shocking. And while I'm on Sony, the prevalence with which their users applied the same password to their Yahoo! accounts (59% of common email addresses had exactly the same password).
Around this time the Adobe data breach happened and that got me really interested in this segment of the industry, not least because I was in there. Twice. Most significantly though, it contained 153M other people which was a massive incident, even by today’s standards. All of these things combined – the prevalence of breaches, the analysis I was doing and the scale of Adobe – got me thinking: I wonder how many people know? Do they realise they were breached? Do they realise how many times they were breached? And perhaps most importantly, have they changed their password (yes, almost always singular) across the other services they use? And so Have I Been Pwned was born.
It's alive! "Have I been pwned?" by @troyhunt is now up and running. Search for your account across multiple breaches http://t.co/U0QyHZxP6k
— Have I Been Pwned (@haveibeenpwned) December 4, 2013
I’ll save the history lesson for the years between then and today because there are presently 106 blog posts with the HIBP tag you can go and read if you’re interested, let me just talk briefly about where the service is at today. It has almost 8B breached records, there are nearly 3M people subscribed to notifications, I’ve emailed those folks about a breach 7M times, there are 120k people monitoring domains they’ve done 230k searches for and I’ve emailed them another 1.1M times. There are 150k unique visitors to the site on a normal day, 10M on an abnormal day, another couple of million API hits to the breach API and then 10M a day to Pwned Passwords. Except even that number is getting smashed these days:
Pwned Passwords in @haveibeenpwned is going from strength to strength - 16M requests in the last 24 hous with a cache hit ratio of 99.4% ? /cc @IcyApril pic.twitter.com/ZLi61e8qbY
— Troy Hunt (@troyhunt) May 24, 2019
Oh – and as I’ve written before, commercial subscribers that depend on HIBP to do everything from alert members of identity theft programs to enable infosec companies to provide services to their customers to protecting large online assets from credential stuffing attacks to preventing fraudulent financial transactions and on and on. And there are the governments around the world using it to protect their departments, the law enforcement agencies leveraging it for their investigations and all sorts of other use cases I never, ever saw coming (my legitimisation of HIBP post from last year has a heap of other examples). And to date, every line of code, every configuration and every breached record has been handled by me alone. There is no “HIBP team”, there’s one guy keeping the whole thing afloat.
@haveibeenpwned
— FlashdriveGordon (@FlashdriveGord1) April 5, 2019
Ho. Ly. Shit.
3,505 results for my org.
I have work to do, thank you guys.
@haveibeenpwned You guys are awesome, thank you so much for your services! @disqus 2012 breach never disclosed, my old creds still worked! <just deleted account> Would never have known if not for your eagle eyes and #totallyawesome service. +10
— ɯɐıllıʍ (@WilliamCaraher) October 10, 2017
@haveibeenpwned you guys are legends.
— CentristAgnostic (@BruvPeace) July 28, 2018
When I wanted an infographic to explain the architecture, I sat there and built the whole thing myself by hand. I manually sourced every single logo of a pwned company, cropping it, resizing it and optimising it. Each and every disclosure to an organisation that didn't even know their data was out there fell to me (and trust me, that's massively time-consuming and has proven to be the single biggest bottleneck to loading new data). Every media interview, every support request and frankly, pretty much every single thing you could possibly conceive of was done by just one person in their spare time. This isn't just a workload issue either; I was becoming increasingly conscious of the fact that I was the single point of failure. And that needs to change.
It's Time to Grow Up
That was a long intro but I wanted to set the scene before I got to the point of this blog post: it’s time for HIBP to grow up. It’s time to go from that one guy doing what he can in his available time to a better-resourced and better-funded structure that's able to do way more than what I ever could on my own. To better understand why I’m writing this now, let me share an image from Google Analytics:
That graph is the 12 months to Jan 18 this year and the spike corresponds with the loading of the Collection #1 credential stuffing list. It also corresponds with the day I headed off to Europe for a couple of weeks of “business as usual” conferences, preceded by several days of hanging out with my 9-year old son and good friends in a log cabin in the Norwegian snow. I was being simultaneously bombarded by an unprecedented level of emails, tweets, phone calls and every other imaginable channel due to the huge attention HIBP was getting around the world, and also turning things off, sitting by a little fireplace in the snow and enjoying good drinks and good conversation. At that moment, I realised I was getting very close to burn-out. I was pretty confident I wasn’t actually burned out yet, but I also became aware I could see that point in the not too distant future if I didn’t make some important changes in my life. (I’d love to talk more about that in the future as there are some pretty significant lessons in there, but for now, I just want to set the context as to the timing and talk about what happens next.) All of this was going on at the same time as me travelling the world, speaking at events, running workshops and doing a gazillion other things just to keep life ticking along.
To be completely honest, it's been an enormously stressful year dealing with it all. The extra attention HIBP started getting in Jan never returned to 2018 levels, it just kept growing and growing. I made various changes to adjust to the workload, perhaps one of the most publicly obvious being a massive decline in engagement over social media, especially Twitter:
Up until (and including) December last year in that graph, I was tweeting an average of 1,141 times per month (for some reason, Twitter's export feature didn't include May and June 2017 and only half of July so I've dropped those months from the graph). From Feb to May this year, that number has dropped to 315 so I've backed off social to the tune of 72% since January. That may seem like a frivolous fact to focus on, but it's a quantifiable number that's directly attributable to the impact the growth of HIBP was having on my life. Same again if you look at my blog post cadence; I've religiously maintained my weekly update videos but have had to cut way back on all the other technical posts I've otherwise so loved writing over the last decade.
After I got home from that trip, I started having some casual conversations with a couple of organisations I thought might be interested in acquiring HIBP. These were chats with people I already knew in places I respected so it was a low-friction “put out the feelers” sort of situation. It’s not the first time I’d had discussions like this – I’d done this several times before in response to organisations reaching out and asking what my appetite for acquisition was like – but it was the first time since the overhead of managing the service had gone off the charts. There was genuine enthusiasm which is great, but I quickly realised that when it comes to discussions of this nature, I was in well over my head. Sure, I can handle billions of breached records and single-handedly run a massive online data breach services that’s been used by hundreds of millions of people, but this was a whole different ballgame. It was time to get help.
Project Svalbard
Back in April during a regular catchup with the folks at KPMG about some otherwise mundane financial stuff (I've met with advisers regularly as my own financial state became more complex), they suggested I have a chat with their Mergers and Acquisition (M&A) practice about finding a new home for HIBP. I was comfy doing that; we have a long relationship and they understand not just HIBP, but the broader spectrum of the cyber things I do day to day. It wasn't a hard decision to make - I needed help and they had the right experience and the right expertise.
In meeting with the M&A folks, it quickly became apparent how much support I really needed. The most significant thing that comes to mind is that I'd never really taken the time just to step back and look at what HIBP actually does. That might sound odd, but as it's grown organically over the years and I've built it out in response to a combination of what I think it should do and where the demand is, I've not taken the time to step back and look at the whole thing holistically. Nor have I taken enough time to look at what it could do; I'm going to talk more about that later in this post, but there's so much potential to do so much more and I really needed the support of people that specialise in finding the value in a business to help me see that.
One of the first tasks was to come up with a project name for the acquisition because apparently, that's what you do with these things. There were many horribly kitschy options and many others that leaned on overused infosec buzzwords, and then I had a thought: what's that massive repository of seeds up in the Arctic Circle? I'd seen references to it before and the idea of a huge vault stockpiling something valuable for the betterment of humanity started to really resonate. Turns out the place is called Svalbard and it looks like this:
Also turns out the place is part of Norway and all these things combined started to make it sound like a befitting name, beginning with the obvious analogy of storing a massive quantity of "units". There's a neat video from a few years ago which talks about the capacity being about a billion seeds; not quite as many records as are in HIBP, but you get the idea. Then there's the name: it's a bit weird and hard to pronounce for those not familiar with it (although this video helps), kinda like... pwned. And finally, Norway has a lot of significance for me being the first international talk I did almost 5 years ago to the day. I spoke in front of an overflowing room and as the audience exited, every single one of them dropped a green rating card into the box.
The feedback after @troyhunt's talk #ndcoslo pic.twitter.com/tNx48Pe6sE
— Erlend Oftedal (@webtonull) June 6, 2014
That was an absolute turning point in my career. It was also in Norway this January that HIBP went nuts as you saw in the earlier graph. It was there in that little log cabin in the snow that I realised it was time for HIBP to grow up. And by pure coincidence, I'm posting this today from Norway, back again for my 6th year in a row of NDC Oslo. So as you can see, Svalbard feels like a fitting name ?
My Commitments for the Future of HIBP
So what does it mean if HIBP is acquired by another company? In all honesty, I don't know precisely what that will look like so let me just candidly share my thoughts on it as they stand today and there are a few really important points I want to emphasise:
- Freely available consumer searches should remain freely available. The service became this successful because I made sure there were no barriers in the way for people searching their data and I absolutely, positively want that to remain the status quo. That's number 1 on the list here for a reason.
- I'll remain a part of HIBP. I fully intend to be part of the acquisition, that is some company gets me along with the project. HIBP's brand is intrinsically tied to mine and at present, it needs me to go along with it.
- I want to build out much, much more capabilities wise. There's a heap of things I want to do with HIBP which I simply couldn't do on my own. This is a project with enormous potential beyond what it's already achieved and I want to be the guy driving that forward.
- I want to reach a much larger audience than I do at present. The numbers are massive as they are, but it's still only a tiny slice of the online community that's learning of their exposure in data breaches.
- There's much more that can be done to change consumer behaviour. Credential stuffing, for example, is a massive problem right now and it only exists due to password reuse. I want HIBP to play a much bigger role in changing the behaviour of how people manage their online accounts.
- Organisations can benefit much more from HIBP. Following on from the previous point, the services people are using can do a much better job of protecting their customers from this form of attack and data from HIBP can (and for some organisations, already does) play a significant role in that.
- There should be more disclosure - and more data. I mentioned earlier how responsible disclosure was massively burdensome and Svalbard gives me the chance to fix that. There's a whole heap of organisations out there that don't know they've been breached simply because I haven't had the bandwidth to deal with it all.
In considering which organisations are best positioned to help me achieve this, there's a solid selection that are at the front of my mind. There's also a bunch that I have enormous respect for but are less well-equipped to help me achieve this. As the process plays out, I'll be working with KPMG to more clearly identify which organisations fit into the first category. As I'm sure you can imagine, there are some very serious discussions to be had: where HIBP would fit into the organisation, how they'd help me achieve those bullet-pointed objectives above and frankly, whether it's the right place for such a valuable service to go. There are also some major personal considerations for me including who I'd feel comfortable working with, the impact on travel and family and, of course, the financial side of the whole thing. I'll be honest - it's equal parts daunting and exciting.
Last week I began contacting each stakeholder that would have an interest in the outcome of Project Svalbard before making it public in this blog post. I explained the drivers behind it and the intention for this exercise to make HIBP not just more sustainable, but also for it to make a much bigger impact on the data breach landscape. This has already led to some really productive discussions with organisations that could help HIBP make a much more positive impact on the industry. There's been a lot of enthusiasm and support for this process which is reassuring.
One question I expect I'll get is "why don't I turn it into a more formal, commercially-centric structure and just hire people?" I've certainly had that opportunity for some time either by funding it myself or via the various VCs that have come knocking over the years. The main reason I decided not to go down that path is that it massively increases my responsibilities at a time where I really need to reduce the burden on me. As of today, I can't just switch off for a week and frankly, if I tried even for a day I'd be worried about missing something important. In time, building up a company myself might allow me to do that but only after investing a substantial amount of time (and money) which is just not something I want to do at this point.
Summary
I'm enormously excited about the potential of Project Svalbard. In those early discussions with other organisations, I'm already starting to see a pattern emerge around better managing the entire data breach ecosystem. Imagine a future where I'm able to source and process much more data, proactively reach out to impacted organisations, guide them through the process of handling the incident, ensure impacted individuals like you and me better understand our exposure (and what to do about it) and ultimately, reduce the impact of data breaches on organisations and consumers alike. And it goes much further than that too because there's a lot more that can be done post-breach, especially to tackle attacks such as the huge rate of credential stuffing we're seeing these days. I'm really happy with what HIBP has been able to do to date, but I've only scratched the surface of potential with it so far.
I've made this decision at a time where I have complete control of the process. I'm not under any duress (not beyond the high workload, that is) and I've got time to let the acquisition search play out organically and allow it to find the best possible match for the project. And as I've always done with HIBP, I'm proceeding with complete transparency by detailing that process here. I'm really conscious of the trust that people have put in me with this service and every single day I'm reminded of the responsibility that brings with it.
@troyhunt I just wanted to say I think you're doing God's work with @haveibeenpwned. I've used it with every company I've worked for so far.
— iloveinfosec (@iloveinfosec) June 2, 2019
HIBP may only be less than 6 years old, but it’s the culmination of a life’s work. I still have these vivid memories stretching back to the mid-90's when I first started building software for the web and had a dream of creating something big; “Isn’t it amazing that I can sit here at home and write code that could have a real impact on the world one day”. I had a few false starts along the way and it took a combination of data breaches, cloud and an independent career that allowed me the opportunity to make HIBP what it is today, but it's finally what I'd always hoped I'd be able to do. Project Svalbard is the realisation of that dream and I'm enormously excited about the opportunities that will come as a result.