Developing a Thermostat API

In a previous post, I built a basic network-connected thermostat out of an ESP8266, a temperature sensor, and a little screen. This thermostat has been working fine, but I always intended for it to have smart capabilities. In this post, I'll be documenting the addition of a web API that will allow other devices on the network to query the thermostat for stats and update the thermostat's settings.

Design

Some things I want to make a note of before writing this API:

JSON decoding sucks on microcontrollers of this caliber. It's very possible to do, but the only decent libraries for it require you to allocate a fixed size buffer for the resulting object. Since our input JSON is user-generated, the buffer has to be big enough to support a reasonably sized input, which means wasting a ton of memory in the vast majority of cases. Even with a huge buffer, a user can still supply an even larger input that the library will fail to parse. In light of this, I'm just going to use normal non-JSON POST parameters for the requests since the HTTP server library I'm using already handles those, and all the responses will be JSON.

Any API calls that allow the temperature to be changed should be authenticated. If I screw up my firewall at some point or somehow give this microcontroller a public IPv6 address, I don't mind if people can tell what the temperature in my home is, but I do care if they can change it.

There aren't a lot of settings or stats to be queried, so it's probably not worth it to make a full REST API with individual member resource endpoints. Just one endpoint for fetching an object of all the stats, and another endpoint for fetching an object of all the configuration should be sufficient.

Building

The only statistic currently available in the firmware is the current ambient temperature, which is being displayed on the screen. This seems like good information to present at the / endpoint. A hard-coded JSON string template and the temperature variable in the firmware are all that's needed to craft the first response of this API.

$ curl -s http://10.0.0.9/
{"temp":68.59}

But while I was poking around, I noticed that the library for the temperature sensor I'm using also supplies functions for accessing the humidity and calculating the heat index. The heat index is probably a better metric to use for a thermostat than the raw temperature, since the goal is to keep a space at the same perceived temperature for a human, and humans feel a little warmer when the humidity is higher, even if the temperature is the same. So the firmware needs to keep track the raw temperature, the humidity, the heat index as calculated from the raw temperature and humidity, and the displayed temperature which is a rolling average of the last few heat index samples, offset by a tuning parameter for calibrating the temperature sensor. On top of all that, I'm also going to throw in the HVAC status string as a stat in the API, which now looks like this:

$ curl -s http://10.0.0.9/ | jq
{
  "temp": 68.67,
  "temp_raw": 71.49,
  "humidity": 52.6,
  "heatindex": 71.53,
  "status": "Off"
}

That probably does it for the status endpoint. Next up is settings.

There are a handful of variables in the firmware that can be considered settings, and they might as well all be configurable. Most obviously, there's the target temperature, which is usually changed by the physical buttons on the thermostat, and there's also the WiFi network credentials. There are three less obvious variable that can be made into settings: the one for setting the maximum allowed deviation from the target temperature before the thermostat needs to kick on the HVAC system to start bringing the temperature back to the target, the one that sets how long the thermostat must wait before it ever tries switching directions (if the deviation is set too low, we can't have the thermostat rapidly switching back and forth between heating and cooling), and the temperature sensor offset for calibration.

$ curl -s http://10.0.0.9/config | jq
{
  "wifi_ssid": "myssid",
  "wifi_pass": "[hidden]",
  "target_temp": 68.0,
  "allowed_deviation": 0.7,
  "minimum_switch_seconds": 600,
  "therm_offset": -3.0
}

The WiFi password is not actually fetched and the constant string "[hidden]" is returned in its place for security reasons. There's not really any reason to ever let someone query the WiFi password from a thermostat. The only reason the key is there at all is to indicate that the value can be set through the API, which is the next thing that needs to be implemented.

The previous JSON object of configurables was returned in response to a GET request. To set those configurables, there should be a POST enpoint at the same path, that accepts any of those keys as a parameter to be set. The usage looks like this:

$ curl -s http://10.0.0.9/config | jq
{
  "wifi_ssid": "myssid",
  "wifi_pass": "[hidden]",
  "target_temp": 68.0,
  "allowed_deviation": 0.7,
  "minimum_switch_seconds": 600,
  "therm_offset": -3.0
}

$ curl -d "target_temp=67" -X POST http://10.0.0.9/config
{"rebooting":false}

$ curl -s http://10.0.0.9/config | jq
{
  "wifi_ssid": "myssid",
  "wifi_pass": "[hidden]",
  "target_temp": 67.0,
  "allowed_deviation": 0.7,
  "minimum_switch_seconds": 600,
  "therm_offset": -3.0
}

Where in a normal REST API, the POST request would probably be met with a response similar to that of the GET request but with the modified values changed, in this one we just get {"rebooting":false}. Most settings will immediately take effect, but changing the WiFi credentials will cause the microcontroller to reboot and go through its network initialization steps again in order to connect with the new credentials (or start an access point if the credentials are invalid). The response to this API call informs the user whether or not the device is about to reboot in order to apply their changes.

One problem at this point is that most of these values aren't actually saved through a reboot. As a holdover from the older firmware I based this off of that I originally wrote to control a fan over the network, the WiFi credential strings are written to specific spots in the EEPROM and read from there on boot. The rest of the mentioned variables are just simple variables in the firmware, and aren't persisted. To make all of this work, I needed to consolidate all of the configurable values into a single struct, and just write that entire struct to the EEPROM whenever a POST request is made (or a button is pressed), and read them all back on boot.

At this point, all the wanted functionality is present. However, one of the initial design goals does not appear to be met. There was a security concern about allowing unauthenticated API calls to change settings, but it seems like that's specifically not being prevented. It's fairly simple to add a quick check at the top of the POST handler to read some kind of authentication parameter and verify it, bailing early if it's not present or incorrect. I would have used HTTP Basic Auth, but the web server library for this microcontroller doesn't appear to implement support for it, and I'm not going to get or write a base64 decoder just for this. So now there's just always an extra value in the POST parameters, and the command to alter a setting looks like

$ curl -d "target_temp=67&auth=some_long_random_string" -X POST http://10.0.0.9/config

Of course this is easily snoopable over HTTP, so it's important that there be a TLS reverse proxy involved if this thing is ever going to be accessed over the internet.

With that all said and done, the thermostat API is finally complete. You can read the code for the entire firmware, including the API implementation, here