Month: March 2016

Build your binary for NodeMCU

Build your binary for NodeMCU

Thanks to frightanic who replied to my post on my onewire problem, I learned about building your own binary with just the modules you need. Of course, I need that share that.

As you know, NodeMCU is constantly being improved. The version I used initially are, as it turns out, hopelessly outdated. On the building the firmware page on the NodeMCU documentation site, there are 3 options to get a new custom build for the NodeMCU. I might investigate the docker option later but for now, I’ve used the cloud build service to get 2 binaries, one with all the modules that I think I will need and one with just the following: file, gpio, net, node, tmr, wifi and pwm. It’s actually so straight forward that it feels silly to write about it.

Fill in your email address (this is where you will get the link to download the binaries once created) and again and select the modules you expect to use and than click on start your build and wait for the email. When you receive the email, it will contain 2 links, one to the float version and one to the integer version which you can than download. Be sure to click on Donate with PayPal if you use and appreciate this service.

Advertisements
ESP8266 and Homewizard

ESP8266 and Homewizard

As you may have seen in my meet ESPöcka article, I have created a led light with an ESP8266 inside. I’ve setup the ESP with a webserver that allows me to turn on the lights (in different colors) through an API call. I plan to build more of these (with normal led strips) and wouldn’t it be great if these can be controlled via normal switches.

In my house I also have a homewizard which is basically a gateway to my different COCO switches, temperature and other sensors. It comes with it’s own smartphone app but more importantly, it run a webserver with an API and it allows you to control IP switches which is what my ESPöcka is.

Setting up an IP switch (more on the API in a different article) is easy. Through the interface on the app, you select the menu (top left) and click on Manage Devices.
HWSetup01
Than select Add Device, followed by Switches & lights
HWSetup02
followed by IP switch.
HWSetup03
Give it a name
HWSetup04
and save it after which you will be prompted with an On URL and an Off URL.
HWSetup06

The URLs you give can be local (to the homewizard) URLs because your homewizard device does the call so there is no need to allow internet access to your lights.
HWSetup07
As you can see, there are many options, HTTP GET, HTTP POST, a UDP or a TCP call and even Wake on LAN. For my ESPöcka, I have used a HTTP GET call with parameters to turn it on in red or turn it off. It works.

ESP8266, recent lineup

ESP8266, recent lineup

When I started working with the ESP8266 series, the ESP-12E was already available and this was the first one I’ve used. I bought several as a NodeMCU v3 on AliExpress. These work well but the more I used them, the more I wanted to learn. There have been many ESP’s before the ESP-12E and from what I read, the ESP-01, 07, 08 and 12 were the most popular. While reading about this, I found out that in the meantime there are also the ESP-12F, the ESP-13 and the ESP-14. The ESP-12F seems to be a 12E but with a better antenna.

There are many different (development) boards which have the ESP installed on a board with better pin layout, usually also with some kind of TTL and a power conversion. The NodeMCU I initially started out with is one of them. I’ve just bought a couple of D1 Mini NodeMCU from Wemos through AliExpress together with some proboards which are the same size print which you can use to stack your leds, sensors etc on a D1. As soon as I have them and have worked on them, I will post an article on them. If you feel uncomfortable soldering wires straight to the chip, you can use a board which will also make the pin usage easier. The small boards are designed to work for the 07, 08, 12, 12E, 12F and 14.

If you look at the chip, the quick way to tell which chip you have in your hand:

  • Is the wifi antenna on the long side of the chip? YES => ESP-13, NO => keep going
  • Are there connections opposite the wifi antenna? NO => ESP-12, NO => keep going
  • Is the antenna small? YES => ESP-12E, NO => keep going
  • Is there STM#### inside on the chip? YES => ESP-14, NO => ESP-12F

If you are planning to use the LUA interpreter on your ESP, according to the description, it is written for the ESP-12E. I can confirm it also works on the ESP-12F and the ESP-14, both of which I now use with the LUA interpretor.

Below you will find the front and back (and pin layout if I found it) of the different models:

ESP-12
ESP-12ESP-12 BackESP-12 Layout

ESP-12E
ESP-12EESP-12E BackESP-12E Layout

ESP-12F
ESP-12F

ESP-13
ESP-13ESP-13 Back

ESP-14
ESP-14ESP-14 Back

Meet ESPöka

Meet ESPöka

This is Spöka, an IKEA led night light:
Spoka
and this is ESPöka:
Spoka
the ESP8266 enabled Spöka.

Why? Because it’s fun. ESPöka runs an improved version of my WebLights app on an ESP-12E.

So, let’s build this.

First, you take apart your Spöka. You start by removing the skin. This requires some force and you should be able to get it off without tearing the rubberlike skin. This leaves you with:
Spoka-1
Next, you remove the white bottom. I have taken apart two. One of the was slightly glued around the edges while the other was more or less clipped on. You should have something like this:
Spoka-3
Next you remove the board with the battery and the button:
Spoka-4
We will be using the button and you can still use the battery as well even though I have not tried the runtime with the ESP inside. In my case I wanted the ESPöka to be full RGB so I’ve removed all leds from the board as they will be replaced by RGB leds. I have also removed both ends of the button from the board.
Spoka-5

Now with the inside cleaned up, let’s get started on our remodelling.

What you will need (links to AliExpress):

First, you solder the ESP-12E to the board. I did this by first putting some solder on every connection point, then putting the ESP-12E on it, making sure it is aligned, hold it down and solder one side. Wait for it to cool a bit (so the ESP-12E does not move) and then turn around and do the other side. Use a meter to make sure that all connections are soldered probably. Next I soldered the pins to the board but next time I won’t do that so you might not want to do that now. You might also decide not to use the board at all either. (I will probably also use this 3.3v regulator for my next ESPöka because the pins get in the way a bit). You should now have this:
Spoka-6
Next we connect the wires. This is the layout of the ESP-12E chip:
ESP-12E Layout
We will need 3 ouput wires (I’ve used D2, D3 and D4) and one input (I’ve used D1).
Connect the 3.3v regulator input side to the points on the IKEA board to where the battery is connected. Please note that you have to do this from the top of the board because there is no room in the led light itself to run wires from the bottom to the top. The output side of the 3.3v regulator go to the VCC and GND pins. Also connect VCC to the EN pin. The button on top of the led light will be the acknowledge button. Connect one side to GND and the other to D1. In order to program the ESP-12E you will need to connect TXD and RXD. In this version I have an internal connector for TXD, RXD, GND and VCC. In the next version I plan to integrate a USB TTL adapter and replace the normal 5 volt plug with a mini USB connector so I can (re)program the ESPöka without opening it. Finally, connect D2, D3 and D4 to the RGB led pins (the long one is the common one) and connect to common anode to the resistor and the other end of the resistor to VCC (or if you have a common cathode, connect that to GND).

In the end it should look like this (without the RGB leds):
Spoka-7
and another angle, with the RGB leds:
Spoka-8

In a (very poorly drawn) schematic, your setup will look like this:
ESPoka layout

Now all that remains is programming the chip with my WebLights, which I will describe in my next post and than getting it back in the Spöka.

After installing the software, test it. Turn on the power and see if it connects. I initially tested with the TTL power, next without the TTL and with the original power adapter. Finally I tested when I put everything in the Spöka, right before I closed it.

Spoka-9
Spoka-13Spoka-11Spoka-12

 

What I did was put the ESP in first, to the back and than move the main board back (you have to wiggle a bit to get the power connector aligned again). The button and the Spöka board are up next. Then I put the 6 RGB leds in (I wanted these as close to the bottom as possible and finally squeeze in the resistor, regulator and TTL connector. Next close the Spöka making sure you again align the board and no wires get stuck between the many pins. Than put the white bottom on. Last step, put the skin back (again tricky because it’s quite tight and you need to align so the hole in the skin matches the power hole). That’s it, you’re done!

 

NodeMCU Web Lights

NodeMCU Web Lights

After experimenting a lot with the NodeMCU, I’ve soldered the first one to a board together with some leds, microswitches and a button. What this contribution does is what I call Web Lights. Basically it is a webserver which controls the set of 3 color leds. You can do an http call to set the lights to a color or give it a range of colors and it will loop through them.

The microswitches are connected to pins D1 and D2 which also have a pull up resistor (next to it in the picture). These are generally off and are only on, to prevent the main program to start or to start the config server (see my post on my init.lua). The button is connected to D6. The four 3 color leds are connected to pins D3, D4 and D5. The connector you see at the top is for easy connection of my USB to UART device.

Apart from the init.lua and accompanying files described in my earlier post on my init.lua and the one for managing the 3 color led this project requires 2 more files, one for the functions to allow for color switching and one for the http server.

The functions for color switching:

cl = {}
clpos = 0
clc = ""
clrpt = 0
cltmr = 0
mcltmr = 0

function setclrpos ()
  if clrpt>0 then
    cltmr = cltmr+1
    if cltmr>=mcltmr then
      cltmr = 0
      clpos = clpos+1
      if clpos>#clc then
        clrpt = clrpt-1
        if clrpt<=0 then
          resetcolor()
        else
          clpos = 1
        end
      end
      if clpos<=#clc then
        rgbcolor(cl[tonumber(clc:sub(clpos,clpos))])
      end
    end
  end
end

function multicol (xcllist,xclscheme,xclrpt,xcltmr)
  clrpt = 0
  cl = {}
  local k
  clc = ""
  if (xcllist~=nil) then
    for k in string.gmatch(xcllist,"(%w+),*") do
      cl[#cl+1] = k
      clc = clc..tostring(#cl)
    end
  end
  mcltmr = 2
  if xcltmr then
    mcltmr = tonumber(xcltmr)
    if cltmr<1 then
      mcltmr = 1
    elseif mcltmr>10 then
      mcltmr = 10
    end
  end
  if xclscheme then
    clc = xclscheme
  end
  cltmr = 0
  clpos = 0
  if xclrpt then
    clrpt = tonumber(xclrpt)
  else
    clrpt = 1
  end
end

function resetcolor (level)
  clrpt = 0
  rgbcolor("off")
end

tmr.alarm(2,250,1,setclrpos)

I know, quite a lot of code to dump on you but it’s not as bad as it looks. The mechanics behind this are as follows:

The function multicol accepts four parameters:

  • A list of comma seperated colors
  • A list of numbers in which order the colors provided in the first parameter should appear
  • A number value how many times the sequence should repeat
  • A timer value

Each of these, except the first one, has a default value but let’s first look at an example:

multicol("red,blue,green","1213",9999,200)

This call will give you the following colors (changed every 200 ms) red,blue,red,green and repeats until you set a different color scheme or call resetcolor (or if you wait 9999 cycles).

The function’s default values are such that if you just supply a range of colors, that range is cycled through once with a timer value of 500 milliseconds.

The function sets the global variables:

  • cl, a numbered array of colors, based on the first parameter
  • clc, the list of numbers referencing the color array cl to cycle through
  • clrpt, the number of times the cycle should repeat
  • clpos, the position of the character in clc where we are at

and uses one local variable:

  • cltmr, the timer value

After the variables are set, the timer is set with the function setclrpos.

The function setclrpos moves to the next character in the variable clc by increasing clpos. If clpos has moved beyond the end of the clc string, move it back to the start if you still need to do a cycle (checked by reducing clrpt). If there is no cycle repeat left, turn off the leds and stop the timer. The last set of lines in the function actually set the color.

The logistics behind this is a continuous timer, which triggers every 250 milliseconds, and runs setclrpos. If there is nothing to do, than the function will simple just stop.

 

My init.lua

My init.lua

The file init.lua is the one that gets executed automatically everytime your NodeMCU boots. What I wanted to do is have a generic init.lua that does the basics like connecting to wifi but also allow for an abort or a config option.

Please note that this is my development init.lua. If I get around to calling some product final, I might consider updating this. The reason is that it is my development init.lua is because it allows for an abort of all program execution (by connecting D1 to ground) or start a config http server (by connecting D2 to ground) instead of the main program. In the final product I might need the extra pins and the software should be stable enough that I do not need an abort or a config option.

There are 3 files which stay the same, 1 config file and finally the main program file. The program flow is as follows:

init.lua starts
check if input D1 is low. if so, abort the rest of the program and goto end
init_functions.lc is executed
init_vars.lua is executed
if an array of ssids is defined, check which one of the available ssids matches one
  in the list. If you find a match, connect to it and goto wait for ip
if just one ssid is defined, connect to it
wait for an ip
if within a certain time, you do not have an ip, print a message and goto end
check if input D2 is low. if so, start the config server by executing
  httpserver_cfg.lc, send an ifttt trigger and goto end
check if main.lc exists. if not, print a message and goto end
send an ifttt trigger
start main.lc
end

That’s the basics. You will see that I have used compiled LUA code to save on heap (*.lc instead of *.lua) except for init_vars.lua which can be changed by the config server.

Let’s go through the code, starting with the easiest one, init_vars.lua

this_node = "Example"
ssids = {}
ssids["ssid1"] = "pass1"
ssids["ssid2"] = "pass2"
ssids["ssid3"] = "pass3"
srvtype = "HTTP"

this_node is the name of the device which is used in IFTTT triggers.
ssids is a list of possible ssids. Convenient if your device travels and will have multiple possible wifi networks.
srvtype is the type of device it is. It is only used in IFTTT triggers.

The next up is init_functions.lua:

function geturl (h, u)
  local conn = net.createConnection(net.TCP, false)
  conn:on("connection", function(cn, answer)
    cn:send("GET "..u.." HTTP/1.1\r\n"
    .."Host: "..h.."\r\n"
    .."Connection: keep-alive\r\n"
    .."Accept: */*\r\n\r\n")
  end)
  conn:connect(80,h)
end  

function ifttt (ev, v1, v2, v3)
  geturl("maker.ifttt.com",
"/trigger/"..ev.."/with/key/---yourkeyhere----?value1="..v1.."
&value2="..v2.."&value3="..v3)
end

function unescape(s)
  local rt, i, len = "", 1, #s
  s = s:gsub('+', ' ')
  local j, xx = s:match('()%%(%x%x)', i)
  while j do
    rt = rt .. s:sub(i, j-1) .. string.char(tonumber(xx,16))
    i = j+3
    j, xx = s:match('()%%(%x%x)', i)
  end
  return rt .. s:sub(i)
end

Three functions, geturl, which does an api call to host h with path u. Notice that this function does not return any page info. It just does the call which is sufficient for IFTTT and a lot of other API calls. The function unescape is exactly what the name suggest, it unescapes url request, turning %3A into : etc.

The following file to look at is httpserver_cfg.lua. This file starts a config server on port 81 which allows you to modify the init_vars.lua file without hooking up the device to a computer.

if srvcfg~=nil then
  srvcfg:close()
end
srvcfg = net.createServer(net.TCP)
srvcfg:listen(81,function(conn)
  conn:on("receive",function(cn,request)
--
-- extract the request uri
--
    local cmd = ""
    local method,path,vars
    _,_,method,path,vars = string.find(request,"([A-Z]+) (.+)?(.+) HTTP")
    if(method==nil) then
      _, _, method, path = string.find(request,"([A-Z]+) (.+) HTTP")
    end
    print(request)
    local httpget = {}
--
-- convert the request uri into httpget variables
--
    local k,v
    if (vars~=nil) then
      for k,v in string.gmatch(vars,"(%w+)=([%w%s%%%.%,%-%+]+)&*") do
        httpget[k] = unescape(v)
      end
    end
--
-- reboot if requested
--
    if httpget["cmd"]=="reboot" then
      node.restart()
--
-- rewrite the init_vars.lua file
--
    elseif httpget["change"]=="Change" then
      file.open("init_vars.lua","w")
      if httpget["node"] then
        this_node = httpget["node"] 
      else
        this_node = "New"
      end
      file.writeline("this_node = \""..this_node.."\"")
      file.writeline("")
      if httpget["srvt"] then
        srvtype = httpget["srvt"] 
      else
        srvtype = "New"
      end
      file.writeline("srvtype = \""..srvtype.."\"")
      file.writeline("")
      local c
      if httpget["ssid2"] then
        file.writeline("ssids = {}")
        ssids = {}
        c = 1
        while c<=5 do
          if httpget["ssid"..c] and httpget["ssid"..c]~="" then
            if httpget["pass"..c] then
              k = httpget["ssid"..c] 
              v = httpget["pass"..c] 
            else
              k = httpget["ssid"..c] 
              v = ""
            end
            file.writeline("ssids[\""..k.."\"] = \""..v.."\"")
            ssids[k] = v
          else
            c = 100
            break
          end
          c = c+1
        end
      else
        ssids = nil
        if httpget["ssid1"] then
          ssid = httpget["ssid1"] 
        else
          ssid = "None"
        end
        if httpget["pass1"] then
          pass = httpget["pass1"] 
        else
          pass = ""
        end
        file.writeline("ssid = \""..ssid.."\"")
        file.writeline("pass = \""..pass.."\"")
      end
      file.close()
    end
--
-- create the webpage
--
    local buf = "HTTP/1.1 200 OK\r\nContent-type: text/html\r\nConnection: close\r\n\r\n"
    buf = buf.."Config NodeMCU "..this_node..""
    buf = buf.."

Config NodeMCU: “..this_node..”


    buf = buf..”

This node


    buf = buf..”Reboot this node


    buf = buf..”

Node number/name:


    buf = buf..”Server type:


    buf = buf..”

Wifi networks


    buf = buf..”Enter list of wifi SSIDs and passwords:

    if ssids then
      c = 1
      for k,v in pairs(ssids) do
        buf = buf..”Wifi “..c..”, SSID: ”
        buf = buf..”password:

        c = c+1
      end
      while c<=5 do
        buf = buf..”Wifi “..c..”, SSID: ”
        buf = buf..”password:

        c = c+1
      end
    else
      buf = buf..”Wifi 1, SSID: password:

      c = 2
      while c<=5 do
        buf = buf..”Wifi “..c..”, SSID: ”
        buf = buf..”password:

        c = c+1
      end
    end
    buf = buf..”


    buf = buf..”

Available wifi networks


    if listaps then
      buf = buf..listaps
    end
    buf = buf..”
“..node.heap()
    buf = buf..”

"
    cn:send(buf)
    buf = ""
    cn:close()
    collectgarbage()
  end)
end)

Maybe a bit too big to just publish here, but the logic behind it is quite simple. The first part of the webfunction checks for input and creates a new init_vars.lua file if it receives a change request. The second part creates the webpage and send it to the requestor.

The file main.lua is where you code your actual program or where you load the different modules. This will be different per application.

This leaves the final file, init.lua.

local pin_error = 0
local pin_abort = 1
local pin_startconfig = 2
gpio.mode(pin_error,gpio.OUTPUT)
gpio.mode(pin_abort,gpio.INPUT)
gpio.mode(pin_startconfig,gpio.INPUT)
gpio.write(pin_error,gpio.HIGH)
--
-- check if the abort pin is high
-- (this means no abort is requested)
--
if gpio.read(pin_abort)==1 then
--
-- load the functions and variables
-- and print the mac address
--
  wifi.setmode(wifi.STATION)
  dofile("init_vars.lua")
  dofile("init_functions.lc")
  print("\n\nNodeMCU:   "..this_node)
  print("\nMAC address: "..wifi.sta.getmac())
  if not this_node then
    this_node = "New"
  end
  listaps = ""
--
-- if multiple ssids have been defined
-- loop through the available APs and choose
-- the first one for which we have credentials
--
  if ssids then
    ssid = ""
    pass = ""
    wifi.sta.getap(function(t)
      if t then 
        local k,v
        for k,v in pairs(t) do
          listaps = listaps..k.."
"
          if ssid=="" and ssids then
            local k2,v2
            for k2,v2 in pairs(ssids) do
              if k2..""==k.."" then
                ssid = k2
                pass = ssids[k2]..""
                print("Connecting to:"..ssid)
              end
            end
          end
        end
--
-- if we did not find an ssid broadcasted that
-- we know, the first ssid is selected
-- (so this could be a hidden ssid)
--
        if ssid=="" and ssids then
          for k,v in pairs(ssids) do
            ssid = k
            pass = v
            break
          end
        end
        wifi.sta.config(ssid,pass)
      end
    end)
  else
--
-- there is either one or no ssid defined
-- connect to the defined ssid or to None
--
    if not ssid then
      ssid = "None"
    end
    if not pass then
      pass = ""
    end
    wifi.sta.getap(function(t)
      if t then
        local k,v
        for k,v in pairs(t) do
          listaps = listaps..k.."
"
        end
        wifi.sta.config(ssid,pass)
      end
    end)
    wifi.sta.config(ssid,pass)
  end
--
-- define a timer to wait for an IP
-- it tries this at most "cnt" times
-- in the timer, you check whether an IP is assigned
-- which means succes connecting to an AP
--
  local cnt = 15
  tmr.alarm(0,500,1,function()
    if wifi.sta.getip()==nil then
      cnt = cnt-1
      if cnt<=0 then
        tmr.stop(0)
        gpio.write(pin_error,gpio.LOW)
        print("Not connected to wifi")
      end
    else
--
-- We have a connection to an AP
-- Get our IP address and check whether we need
-- to start the main program or the config server
-- (determined whether port 2 is low (config)
-- or high (main program))
--
      tmr.stop(0)
      ip, __, __ = wifi.sta.getip()
      print("My IP:"..ip)
      if gpio.read(pin_startconfig)==1 then
        if srvtype then
          ifttt("NodeMCU"..srvtype,this_node,ip,"")
          iot("&ip="..ip.."&ssid="..ssid.."&type="..srvtype)
        else
          ifttt("NodeMCUBoot",this_node,ip,"")
          iot("&ip="..ip.."&ssid="..ssid.."&type=Boot")
        end
        if file.open("main.lc","r") then
          file.close()
          print("\nStarting main")
          dofile("main.lc")
        else
          print("Could not find main.lc")
        end
      else
        print("Starting config server")
        ifttt("NodeMCUConfig",this_node,ip,"")
        iot("&ip="..ip.."&ssid="..ssid.."&type=Config")
        dofile("httpserver_cfg.lc")
      end
    end
  end)
else
  gpio.write(pin_error,gpio.LOW)
  print("Abort button pressed. Stopping execution of init.lua")
end

I have put comments in the text to show which section does what. Most should be self explanatory based on the high level description in the beginning. The most tricky part is probably the waiting for IP. This is based on a solution I found somewhere else. What it does is define a trigger which triggers every 500 milliseconds. The function it triggers is defined in-line. Every time the function is triggered, it checks whether an IP is assigned. If it is, the timer is stopped and, depending on input 2 either the main program or the config server is started. If an IP has not yet been assigned, the counter cnt is decreased. If it reaches 0 it means that we assume we will not get an IP and the program execution is stopped.

The whole idea behind waiting for an IP is the assumption that you need an IP for this device to be effective. If you do not need that you can of course skip those steps and immediately start the main program (it would also not make sense to start the config server since that needs an IP as well).

I hope that my development init.lua makes sense together with my previous articles on LUA. If not, please let me know in the comments.

NodeMCU LUA & the heap

NodeMCU LUA & the heap

Once you have gone past the simple programs, reading a sensor, turning on a led etc you will notice that there is something that you will run out of, heap memory.

The ESP8266 does not come with a lot of memory and unfortunately, the LUA interpreter for it comes with not a whole lot of heap and heap is what you need while executing programs, storing variables etc. And if you run out of it, the NodeMCU reboots…..

While there is plenty written on how to use little memory, I did want to give you two tips that I look at:

use local variables where possible

If you do not declare something local in LUA, it is automatically a global variable which means it stays in memory (and uses heap) after the program or function stops. If you execute the same program/function again, the variable (and it’s value) are still there. To conserve heap, you should use local variables if possible. An example of this:

function countsteps()
  local i
  for i = 1,10 do
    print("Step "..tostring(i))
  end
end

In the function above, if you do not include the line local i, the variable i will be available after the function countsteps finishes. You can assign a value to a variable while declaring it local:

local s = "A string value"

compile your programs

The LUA interpreter on the NodeMCU comes with a compiler. You can compile your lua file into a lc (lua compiled) file which when executed uses less heap.

To compile you use compile(“myprogram.lua”) which creates myprogram.lc and to use it instead of executing dofile(“myprogram.lua”), you use dofile(“myprogram.lc”). That’s it.