Mastodon

The Internet of Things is a Complete Mess (and how to Fix it)

13 July 2021

I've spent more time IoT'ing my house over the last year than any sane person ever should. But hey, it's been strange times for all of us and it's kept me entertained whilst no longer travelling. Plus, it's definitely added to our lives in terms of the things it enables us to do; see them in part 5 of my IoT unravelled blog series. But it also remains a complete mess and I want to demonstrate why based on some recent experiences:

Let's say you want an IoT light. You want it connected because you want it to come on at a certain time of the day therefore it requires scheduling. You also want to be able to change the colour because hey, that's kinda cool. The light sits on your desk and plugs into mains power and connects wirelessly to your network. An app provided by the device manufacturer controls the schedule, the colour and other features such as the brightness. Now you also want one in your ceiling, but it's different because it's slotting into a standard 92mm cut out. It's still connecting to a mains socket in the roof and it's still connecting to your wifi, but it's a different brand. With a different app. And you have to schedule it differently.

But we're only just getting started here because the next problem is the dependencies you're creating on "the cloud". The manufacturer ships these lights with tiny microprocessors and wifi chips in them at bargain basement prices as a "turnkey" solution; your average consumer picks one up off the shelf, connects it to their wifi, sets up an account and its job done. That account is on "the cloud" and when they're using the custom app, they're communicating with said cloud which is then communicating with the devices. Not all IoT works this way, but a lot does, for example the massively popular Tuya lights I've been putting all over my house.

And this is where the problems begin.

Ideally, you want local control and local orchestration of all your IoT things and there are several very good reasons for this:

  1. Privacy: You may not want to share the state of your connected things with a third party
  2. Autonomy: You may not want to be dependent on either the cloud service being up or your internet connection being available
  3. Latency: The round trip to the cloud can produce a noticeable delay, especially evident when performing multiple tasks in sequence

The vast majority of consumers will be fine with this because they never expected anything more and frankly, they're still just marvelling at being able to turn on their lights from their phone. But they're experiencing a very watered-down version of IoT that is merely one little self-contained unit. Start pushing the boundaries a bit and things become... a complete and utter mess. Let me explain:

I have 86 Tuya lights in my house (they're not branded as Tuya, but they're all compatible and are controlled by the Tuya app) and they're all surfaced into Home Assistant (HA) via the Tuya integration:

With a bit of mucking around (manually setting the supported features), I can control on / off, colour, warmth and brightness all from within the HA user interface:

I can also control it via YAML which I can use to define scenes and automations:

And finally, I can also control the digital light via a physical switch on the wall as it has a Shelly behind it which sends a message via MQTT to toggle the light.

In short, it's the best of all worlds. Almost - there's still that cloud dependency and there's really only 2 ways around that:

  1. Control the existing device locally with the original firmware
  2. Flash the device with 3rd party firmware that supports local control

Let's explore these more starting with the first option because it feels like the most low-friction path. There's a dedicated integration for local control of Tuya devices and it's literally just called Local Tuya. In order for it to work, you need to extract both the internal ID of the device and a local key, which is where all the problems begin. You used to be able to pull this out of Tuya's IoT portal but recent changes they've made have completely killed that option (see "!!! UPDATE   !!!" in the description of that video). So that's out, let's instead fire up BlueStacks, install the Smart Life APK, fire up BSTweaker and then pull the keys out of there. I had a false start here because I pulled current versions of everything when what I really needed was an old version of BlueStacks and an old version of the Smart Life APK. Which presented another problem:

I don't know why there are 2 Tuya apps nor why one is called "Smart Life" and one "TuyaSmart". But fuck it (yes, I'm swearing a lot by now), let's just create a brand new account on Smart Life, pair a new bulb to it then pull out the keys. It'd be hard work to do that for every single bulb, so I'll just do a test one first:

Hey, I've got keys! Which don't work in Local Tuya:

"See log for details." Ok then:

And even if it did work, I really don't like the idea of unpairing then re-pairing 86 lights one by one.

Let me pause here and reflect on the problem I'll come back to later on: All of us following one of these paths whether it be local control or firmware flashing are attempting to make devices behave in a fundamentally different fashion to how the manufacturer intended them to be used. We're tinkerers trying to solve problems, but we're working against the basic design intent of the devices. It's the same as the problems late last year when TP-Link killed local control of smart plugs:

The manufacturer is under no obligation to support us tinkerers. I will come back to this because I don't believe it should be that way, but I want to emphasise this point because we should know what we're getting ourselves into here.

So, local control of the out of the box device is out, let's move onto updating the firmware and one approach is to literally solder wires to smart lights as part of the firmware flashing process. That's... messy. And time consuming. And if you screw it up then your smart light isn't just dumb, it's dead. And whilst you might be able to do that with a downlight which has an exposed circuit board, how do you do that with a light globe? So. let's explore the non-soldering route which brings us to flashing ESPHome with tuya-convert, a process which requires the following:

In order to run tuya-convert, you'll need a Linux computer with wifi, plus a second wifi device (such as a smartphone)

Let's pause here a second: If we think about IoT as being something that should be consumable by the masses, just how far down the rabbit hole have we gone once you need a Linux machine and a secondary device just to make your lights behave. This is nuts. But let's do it anyway 🙂

I go buy another Raspberry Pi, set it all up, run tuya-convert and then...

By this stage, my fiancé has rushed upstairs to ask what all the swearing is about. I've also well and truly passed the ROI of trying to gain local control of my lights, so I put that effort to bed and wrote this blog post instead.

Let's move on and instead talk about fixing it:

Devices should enable local control via a baseline set of APIs that remain stable. Lights, for example, should provide dimming, colour and warmth controls in a fashion that is not broken by later updates. Some lights will need to extend on this; the Nanonleaf is able to control light shows across panels so requires functionality beyond what's needed in a single bulb. All of this requires manufacturer buy-in. It's not necessarily hard (have a look at Shelly's fantastic API documentation), but it requires commitment and for that, there needs to be incentive. I'd like to think that the incentive is both to play an active part in contributing to a more interoperable IoT world and financial. The former drives adoption and the latter is realised when the barriers to that adoption are lowered. Which is why I was especially happy to see this recently:

This is precisely what we want to see: the vendor themselves playing nice with other platforms and working towards local control. At least that's their indication, but then TP-Link also made indications last year and is yet to deliver so I'll hold off giving Tuya too many kudos for right now. What we need to actively move away from is polyfill solutions like Home Assistant's existing Tuya integration. Don't get me wrong - this is an admiral effort in lieu of Tuya themselves providing APIs - but having Tuya themselves provide APIs (and ideally the Home Assistant integration as well), is precisely where the industry needs to move to.

Lastly, this might seem painful right now (and frankly, it is), but it's also a very exciting time in IoT. It feels like the very early days of the web where everything was a bit of a kludge we hacked together but we made things work and it turned into something amazing. That's where I think we are now with IoT and as infuriating as it often is, it's an exciting time to be a part of it and well and truly worth a few lighting problems here and there 😊

IoT
Tweet Post Update Email RSS

Hi, I'm Troy Hunt, I write this blog, create courses for Pluralsight and am a Microsoft Regional Director and MVP who travels the world speaking at events and training technology professionals

Troy Hunt

Hi, I'm Troy Hunt, I write this blog, run "Have I Been Pwned" and am a Microsoft Regional Director and MVP who travels the world speaking at events and training technology professionals

Upcoming Events

I often run private workshops around these, here's upcoming events I'll be at:

Must Read

Don't have Pluralsight already? How about a 10 day free trial? That'll get you access to thousands of courses amongst which are dozens of my own including:

  1. OWASP Top 10 Web Application Security Risks for ASP.NET
  2. What Every Developer Must Know About HTTPS
  3. Hack Yourself First: How to go on the Cyber-Offense
  4. The Information Security Big Picture
  5. Ethical Hacking: Social Engineering
  6. Modernizing Your Websites with Azure Platform as a Service
  7. Introduction to Browser Security Headers
  8. Ethical Hacking: SQL Injection
  9. Web Security and the OWASP Top 10: The Big Picture
  10. Ethical Hacking: Hacking Web Applications