Tag: ESP8266

Domoticz, ESP & MQTT, startup

Domoticz, ESP & MQTT, startup

Of course domotica with Domoticz is fun but it’s more fun if you can combine your own ESPs with Domoticz. That is what I have been fiddling with and what I will describe here.

There are basically 2 ways to connect to Domoticz, through HTTP or MQTT. With HTTP you can only tell Domoticz what to do. With MQTT however, Domoticz shares it’s changes so you can also respond to changes. I choose to go the MQTT route because that way I can create sensors and switches with my ESPs.

To get things started, you need to have an MQTT broker. There have been many people describing this on their blogs, so I will not cover that here. Suffice it to say that I choose to put the MQTT broker on my NAS since that is always up anyway. Once you have the MQTT broker, you need to configure MQTT for Domoticz. This is described nicely on their Wiki. You will also find instructions on how to install Node-RED. I did install that but you don’t need it for these projects. Once you have installed MQTT for Domoticz, you need to tell it to publish everything on the flat version (called out in the drop down list).

We are getting there. Once you have installed and configured MQTT for Domoticz, use an MQTT viewer to view the messages it generates. I personally like mqtt-spy. Remember you have to subscribe to the topic domoticz/out. Look at some of the messages and how they are structured. As you can see it’s JSON formatted. The messages you post to Domoticz on the domoticz/in topic are also JSON formatted.

Ok, let’s configure a dummy switch in Domoticz. I called mine TESTMQTT but you can call it whatever you want. Make sure it is a switch. Once you did that, write down the id number.

Good, now let’s play. Turn it on and back off again. Now go to your viewer and look for the id of your switch. If you are using mqtt-spy, it is the search button on the right of the screen next to where all the messages are shown. Once you find your message, you will see that it looks something like this:

{
  "Battery" : 255,
  "RSSI" : 12,
  "description" : "",
  "dtype" : "Light/Switch",
  "id" : "00014648",
  "idx" : 1528,
  "name" : "TESTMQTT",
  "nvalue" : 0,
  "stype" : "Switch",
  "svalue1" : "0",
  "switchType" : "On/Off",
  "unit" : 1
}

I won’t go in detail now, just now that “idx” is your id and in the case of a switch, “nvalue” tells whether it was turned on (1) or off (0).

Next, let’s turn on the switch using MQTT. Make sure the switch is off and than go to your viewer and send this message (remember to use your id number, not mine)

topic: domoticz/in
message: { "command" : "switchlight" , "idx" : 1528 , "switchcmd" : "On" }

And click on publish. Your dummy switch should turn on.

Next up is coding an ESP to be a light. As we need JSON to listen to the switches, we need to install the ArduinoJson library which you install like any library. We also need to install the pubsubclient library which again installs like any library. However in this case we need to make a change after the installation. Go to your libraries directory and go to the pubsubclient-master/src directory. There is a file called PubSubClient.h. Open it to make a change. We need to increase the maximum packet size. Around line 26 there is a line that we need to change. It should read after you’re done:

#define MQTT_MAX_PACKET_SIZE 512

Next save the file and we are ready to try some code. We will start with something simple, the internal led being turned on and off through Domoticz. I will start by sharing the code (also available on github-mark-32px Github) and discuss it below the code:

#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <ArduinoJson.h>

const char* ssid = "MySSID";
const char* password = "MyPassword";
const char* mqtt_server = "mqtt.broker.com";
#define LED D4
#define MYIDX 1528

WiFiClient espClient;
PubSubClient client(espClient);

long lastMsg = 0;
char msg[50];
int value = 0;
char macAddr[18];
char mqttClient[25];

void setup_wifi() {
  delay(10);
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void actionOn () {
  digitalWrite(LED,LOW);
}

void actionOff () {
  digitalWrite(LED,HIGH);
}

void errorOccurred (String err) {
  Serial.println(err);
}

void callback(char* topic, byte* payload, unsigned int length) {
  DynamicJsonBuffer jsonBuffer(512);
  JsonObject& root = jsonBuffer.parseObject(payload);
  if (!root.success()) {
    errorOccurred("Parse json failed");
  } else {
    int idx = root["idx"];
    if (idx==MYIDX) {
      if (root["nvalue"]==0) {
        actionOff();
      } else {
        actionOn();
      }
    }
  }
}

void reconnect() {
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    sprintf(mqttClient,"ESP%s",macAddr);
    Serial.println(mqttClient);
    if (client.connect(mqttClient)) {
      Serial.println("connected");
      client.publish("outTopic","hello world");
      client.subscribe("domoticz/out");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  pinMode(LED,OUTPUT);
  digitalWrite(LED,LOW);
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server,1883);
  client.setCallback(callback);
  byte ar[6];
  WiFi.macAddress(ar);
  sprintf(macAddr,"%02X%02X%02X%02X%02X%02X",ar[0],ar[1],ar[2],ar[3],ar[4],ar[5]);
}

void loop() {
  if (!client.connected()) {
    reconnect();
  }
  client.loop();
}

Let’s start at the top. We include the needed libraries and define our constants for wifi and the MQTT broker. I use a Wemos D1 so my internal led is linked to D4 and my test Domoticz dummy switch has id 1528. Initialize the wifi client and the MQTT client. I keep the mac address which I use for creating the client id and the name of the MQTT client.

The function setup_wifi() does what it says. The functions actionOn() and actionOff() are called to turn on and turn off our switch. In this case it is just write to a port but you can put more here. The function errorOccurred gets called when an error happens. It now just outputs to serial but there might be a need to change that.

The function callback() is the interesting one. This one gets called every time we receive a message on a topic that we are subscribed to. We assume the data is in JSON format and decode it using the ArduinoJson library. Next we check if this was a success and if so, if the idx passed is the one we are looking for. If it is, check nvalue and determine what to do based on it’s value.

The function reconnect() is straight from the pubsubclient example and basically reconnects if the connection is dropped and subscribes to our domoticz/out topic.

Finally the function setup() sets everything up, including getting a unique MQTT client name and the function loop() continously checks for messages.

If you upload this to your ESP it should work. Remember that we do not set any initial values so, the light starts on but maybe in Domoticz it is still off which means you will have to click twice for any change.

This is a very simple example, hardly any error checking but for some of you this may be enough to get going. In the next example I will expand a bit and also publish to Domoticz.

 

Hacking the SONOFF RF

Hacking the SONOFF RF

Some time ago I bought two SONOFF RF’s, the power switch with an ESP8266 inside. I wasn’t planning on using them out of the box, mostly because I don’t want another app on my phone. I have domotica and I want to control all with the same app. My domotica can control devices based on an http call, so this should work.

While digging around on the internet, I found several sites which explain how to hack the normal SONOFF, without the RF. I liked this page but there are many others. Unfortunately, I couldn’t find any page on how to hack the SONOFF RF so I decided to figure it out myself.

I couldn’t get the ESP in flash mode, no matter how hard I tried. When you’re hacking the SONOFF apparently you can use the button which is connected to GPIO0 to put it in flash mode but that did not work on the SONOFF RF. Finally I decided to connect the GPIO0 to ground straight from the chip and see what happens. That worked!

Let’s look at the chip layout:

esp8266_extra_gpio

The LNA (pin 2) should be easy to find since that is the antenna. I looked at the bottom of the print and found it. Forgive the lousy quality but you can see what I found:

antenna and gpio0

You can see C27 connected to the 2nd pin from the top on the left side. That aligns with the schematic above and this means that the 2nd pin from the right on the bottom side should be GPIO0. It runs to a resistor at which point I soldered a wire to it.

The rest of the connections are the same as with the normal SONOFF:

sonoffrf1.jpg

I soldered a strip to the board, from the button to the top the pins are:

  • 3.3V
  • TX
  • RX
  • GND
  • GPIO14

You connect the 3.3V to your FTDI module on the 3.3V, TX on SONOFF RF to RX on FTDI, RX on SONOFF RF to TX on FTDI and GND to GND. What I did is put a break switch on the line to the power because that way I can quickly reboot the SONOFF RF without unplugging the FTDI (reboot to put it in flash or take it out of flash):
(forgive the sloppy soldering 🙂

SONOFF 2 FTDI

Almost there… Now I want to be able to put the SONOFF RF into flash mode whenever I want and apparently the button does not work. So I soldered a small push button (break if not pressed) between my wire on the bottom and GND but I made the button at the top so I can access it easily. I passed the wire to the front:

sonoffrf2

and connected it to the GND of the button of the SONOFF RF (which is the left side in the picture below) via a small push button:

sonoffrf3

Now if I want to put the ESP in flash mode, I push the button on the power cable (cutting the power to the ESP), push the small button (connecting GPIO0 to GND), release the power button (which boots the ESP) and after a short while release the small button because the ESP will now be in flash mode.

Now that I can program the ESP I can figure out why the regular push button does not work to connect GPIO0 to GND like in the normal SONOFFs.

In short, these are the connections:

  • The relay is connected to GPIO12
  • The led is connected to GPIO13
  • The button is connected to GPIO0

Even though the button is connected to GPIO0, pushing it however only brings GPIO0 to GND for a short while. Even if you keep the button pushed, it only briefly connects GPIO0 to ground which is why you cannot use the button to put the ESP in flash mode. The reason for that is that the RF module also does the same. Pushing the button on the remote actually causes GPIO0 to connect to GND which is probably why they did something different with the button than in the normal SONOFF.

Now that we know that we can build the program. We need to control the relay and the led and we need to watch the button/RF button through GPIO0.

I have created this simple program which should do the trick:

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WebServer.h>

// Replace with your network credentials
const char* ssid = "YourSSID";
const char* password = "YourPassword";

ESP8266WebServer server(80);

String pagePart1, pagePart2;

int sonoffLed = 13;
int sonoffRelay = 12;
int sonoffButton = 0;
bool buttonState = false;
bool powerState = false;

void powerOn (void) {
  digitalWrite(sonoffLed,LOW);
  digitalWrite(sonoffRelay,HIGH);
  powerState = true;
}

void powerOff (void) {
  digitalWrite(sonoffLed,HIGH);
  digitalWrite(sonoffRelay,LOW);
  powerState = false;
}

void switchPower (void) {
  if (powerState) {
    powerOff();
  } else {
    powerOn();
  }
}

void setup(void){
  pagePart1 = "<h1>SONOFF Switch</h1><br><br>Your switch is ";
  pagePart2 = "</font>.<br><br><a href='on'><button>ON</button></a> ";
  pagePart2 = pagePart2+<a href=\"off\"><button>OFF</button></a></p>";  
  // preparing GPIOs
  pinMode(sonoffLed,OUTPUT);
  pinMode(sonoffRelay,OUTPUT);
  powerOff();

  Serial.begin(9600); 
  Serial.println("");
  WiFi.begin(ssid,password);

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  server.on("/",[](){
    String tmp = pagePart1+"<font color=";
    if (powerState) {
      tmp = tmp+"green>on";
    } else {
      tmp = tmp+"red>off";
    }
    tmp = tmp+pagePart2;
    server.send(200,"text/html",tmp);
  });
  server.on("/on",[](){
    String tmp = pagePart1+"now switched <font color=green>on"+pagePart2;
    server.send(200,"text/html",tmp);
    powerOn();
    delay(1000);
  });
  server.on("/off",[](){
    String tmp = pagePart1+"now switched <font color=red>off"+pagePart2;
    server.send(200,"text/html",tmp);
    powerOff();
    delay(1000); 
  });
  server.begin();
  Serial.println("HTTP server started");
}
 
void loop(void){
  server.handleClient();
  bool currentState = digitalRead(sonoffButton);
  if (currentState!=buttonState) {
    buttonState = currentState;
    if (buttonState==LOW) {
      switchPower();
    }
  }
  delay(50);
}

The nice thing about the way the SONOFF RF is designed is that the RF part is completely separate from the ESP part so after reprogramming the ESP, the RF still works as originally designed. This means that you can still program the RF receiver as you would normally:

sonoff_rf_pair_with_433-1

Press the button on the SONOFF RF twice quickly and than press a button on the remote. This button will than be linked to the RF and can be used, with the above program, to turn the switch on and off.

Build your binary for NodeMCU

Build your binary for NodeMCU

Thanks to frightanic who replied to my post on my onewire problem, I learned about building your own binary with just the modules you need. Of course, I need that share that.

As you know, NodeMCU is constantly being improved. The version I used initially are, as it turns out, hopelessly outdated. On the building the firmware page on the NodeMCU documentation site, there are 3 options to get a new custom build for the NodeMCU. I might investigate the docker option later but for now, I’ve used the cloud build service to get 2 binaries, one with all the modules that I think I will need and one with just the following: file, gpio, net, node, tmr, wifi and pwm. It’s actually so straight forward that it feels silly to write about it.

Fill in your email address (this is where you will get the link to download the binaries once created) and again and select the modules you expect to use and than click on start your build and wait for the email. When you receive the email, it will contain 2 links, one to the float version and one to the integer version which you can than download. Be sure to click on Donate with PayPal if you use and appreciate this service.

ESP8266, recent lineup

ESP8266, recent lineup

When I started working with the ESP8266 series, the ESP-12E was already available and this was the first one I’ve used. I bought several as a NodeMCU v3 on AliExpress. These work well but the more I used them, the more I wanted to learn. There have been many ESP’s before the ESP-12E and from what I read, the ESP-01, 07, 08 and 12 were the most popular. While reading about this, I found out that in the meantime there are also the ESP-12F, the ESP-13 and the ESP-14. The ESP-12F seems to be a 12E but with a better antenna.

There are many different (development) boards which have the ESP installed on a board with better pin layout, usually also with some kind of TTL and a power conversion. The NodeMCU I initially started out with is one of them. I’ve just bought a couple of D1 Mini NodeMCU from Wemos through AliExpress together with some proboards which are the same size print which you can use to stack your leds, sensors etc on a D1. As soon as I have them and have worked on them, I will post an article on them. If you feel uncomfortable soldering wires straight to the chip, you can use a board which will also make the pin usage easier. The small boards are designed to work for the 07, 08, 12, 12E, 12F and 14.

If you look at the chip, the quick way to tell which chip you have in your hand:

  • Is the wifi antenna on the long side of the chip? YES => ESP-13, NO => keep going
  • Are there connections opposite the wifi antenna? NO => ESP-12, NO => keep going
  • Is the antenna small? YES => ESP-12E, NO => keep going
  • Is there STM#### inside on the chip? YES => ESP-14, NO => ESP-12F

If you are planning to use the LUA interpreter on your ESP, according to the description, it is written for the ESP-12E. I can confirm it also works on the ESP-12F and the ESP-14, both of which I now use with the LUA interpretor.

Below you will find the front and back (and pin layout if I found it) of the different models:

ESP-12
ESP-12ESP-12 BackESP-12 Layout

ESP-12E
ESP-12EESP-12E BackESP-12E Layout

ESP-12F
ESP-12F

ESP-13
ESP-13ESP-13 Back

ESP-14
ESP-14ESP-14 Back

Meet ESPöka

Meet ESPöka

This is Spöka, an IKEA led night light:
Spoka
and this is ESPöka:
Spoka
the ESP8266 enabled Spöka.

Why? Because it’s fun. ESPöka runs an improved version of my WebLights app on an ESP-12E.

So, let’s build this.

First, you take apart your Spöka. You start by removing the skin. This requires some force and you should be able to get it off without tearing the rubberlike skin. This leaves you with:
Spoka-1
Next, you remove the white bottom. I have taken apart two. One of the was slightly glued around the edges while the other was more or less clipped on. You should have something like this:
Spoka-3
Next you remove the board with the battery and the button:
Spoka-4
We will be using the button and you can still use the battery as well even though I have not tried the runtime with the ESP inside. In my case I wanted the ESPöka to be full RGB so I’ve removed all leds from the board as they will be replaced by RGB leds. I have also removed both ends of the button from the board.
Spoka-5

Now with the inside cleaned up, let’s get started on our remodelling.

What you will need (links to AliExpress):

First, you solder the ESP-12E to the board. I did this by first putting some solder on every connection point, then putting the ESP-12E on it, making sure it is aligned, hold it down and solder one side. Wait for it to cool a bit (so the ESP-12E does not move) and then turn around and do the other side. Use a meter to make sure that all connections are soldered probably. Next I soldered the pins to the board but next time I won’t do that so you might not want to do that now. You might also decide not to use the board at all either. (I will probably also use this 3.3v regulator for my next ESPöka because the pins get in the way a bit). You should now have this:
Spoka-6
Next we connect the wires. This is the layout of the ESP-12E chip:
ESP-12E Layout
We will need 3 ouput wires (I’ve used D2, D3 and D4) and one input (I’ve used D1).
Connect the 3.3v regulator input side to the points on the IKEA board to where the battery is connected. Please note that you have to do this from the top of the board because there is no room in the led light itself to run wires from the bottom to the top. The output side of the 3.3v regulator go to the VCC and GND pins. Also connect VCC to the EN pin. The button on top of the led light will be the acknowledge button. Connect one side to GND and the other to D1. In order to program the ESP-12E you will need to connect TXD and RXD. In this version I have an internal connector for TXD, RXD, GND and VCC. In the next version I plan to integrate a USB TTL adapter and replace the normal 5 volt plug with a mini USB connector so I can (re)program the ESPöka without opening it. Finally, connect D2, D3 and D4 to the RGB led pins (the long one is the common one) and connect to common anode to the resistor and the other end of the resistor to VCC (or if you have a common cathode, connect that to GND).

In the end it should look like this (without the RGB leds):
Spoka-7
and another angle, with the RGB leds:
Spoka-8

In a (very poorly drawn) schematic, your setup will look like this:
ESPoka layout

Now all that remains is programming the chip with my WebLights, which I will describe in my next post and than getting it back in the Spöka.

After installing the software, test it. Turn on the power and see if it connects. I initially tested with the TTL power, next without the TTL and with the original power adapter. Finally I tested when I put everything in the Spöka, right before I closed it.

Spoka-9
Spoka-13Spoka-11Spoka-12

 

What I did was put the ESP in first, to the back and than move the main board back (you have to wiggle a bit to get the power connector aligned again). The button and the Spöka board are up next. Then I put the 6 RGB leds in (I wanted these as close to the bottom as possible and finally squeeze in the resistor, regulator and TTL connector. Next close the Spöka making sure you again align the board and no wires get stuck between the many pins. Than put the white bottom on. Last step, put the skin back (again tricky because it’s quite tight and you need to align so the hole in the skin matches the power hole). That’s it, you’re done!

 

NodeMCU & KY-031 Hit sensor

NodeMCU & KY-031 Hit sensor

The KY-031 Hit sensor, from the 37 sensors kit, is, as the name implies, a sensor which triggers if it is hit (or the board it is on).

There are 3 pins, +, – and S (for Signal). The – and S are left and right (or vice versa depending on how you view the sensor) and are marked on the print. The + is in the middle. Simply connect – to a ground, + to a 3.3v connection and the S to a digital signal (in the example I have used D3).

To use the sensor in the NodeMCU, you can use the following example to understand it’s workings:

gpio.mode(3,gpio.INPUT)

function showhit (level)
  print(level)
end

gpio.trig(3,"both",showhit)

A 1 means it’s not being hit and a 0 means it is hit. The sensor is actually so fast that even though I set the trigger for both it is only triggered when it is hit.

What I did to try it is pick up the breadboard and shake it or drop it.