xAP Plugboard (V2 script engine)

You must be using a HAH build >=280 otherwise you should use the v1 release

Whilst developing the HAH project I noticed there was an architectural component that would simplify the interactions between xAP compliant devices and daemons. I call this the plug-board.

Consider the following use cases:

  • Send a twitter message when the power being monitored with Current Cost goes above 2kW
  • If the temperature drops below 10C turn on a heater turning it off again when it reaches 12C
  • When a Current Cost/temperature event is seen, log it to pachube
  • If Input 1 goes high turn on Relay 1
  • If RF 1 is turned on schedule a Google calendar event to turn it off in 1 hour
  • Log a Google calendar event whenever Input 2 changes state
  • If no change is detected on Input 3 during the last 2 hours, send an SMS message
  • If a change is detected on Input 4 AND a named Google calendar event is still active, send an email
  • If 'uptime' is > a given number of hours, perform a scheduled reboot

It's possible to cascade rules in the plug-board. It's also possible to code circular dependency which the scripter must be careful to avoid.

In this model we want to decouple the producers from the consumers and use a scripting intermediary, the plugboard, to perform an action. With N producers and N consumers we have N*N ways of plugging these together making a very flexible and expandable system. Consumers can also be thought of as SERVICE providers as they accept an xAP message and perform something useful.

Example of how the script engine is used to interpret a tweet command to turn on a relay.

  • User tweets
  • This message is picked up by the xap-twitter components and broadcast as an xAP alias command
  • The script engine interprets this alias and re-transmits a 'RELAY on' command
  • 'Relay on' is seen by the adapter and is processed as a control for the HAH hardware, thus turning the relay on

Examples of daemons that currently conform to this model are:

  • xap-twitter
    • PRODUCER: monitor a twitter feed and produces a CMD when an action is found
    • CONSUMER: for sending a tweet.
  • xap-googlecal
    • PRODUCER: monitors Google calendar and pushes whatever command has been registered.
    • CONSUMER: used as a service to create Google calendar events.
  • xap-sms
    • PRODUCER: emits an xAP message when an SMS is received
    • CONSUMER: accept an xAP message to send an SMS.
  • xap-currentcost
    • PRODUCER: sends a messages every minute to report on temperature or power levels.
  • xap-livebox:
    • PRODUCER: generates xAP events on INPUT, 1WIRE or I2C changes
    • CONSUMER: accept messages for the RF, RELAYS and I2C sub systems
  • xap-pachube:
    • CONSUMER: accepts data logging messages.
      For simplistic messages, it can can also directly watch for events, manipulate the message, and send these to pachube without the plugboards involvement. This is configurable through the web interface.

If Pachube is used in the CONSUMER model the following would occur:

  • xap-currentcost - xmit an xAPBSC.event telling of a power or temperature level.
  • xap-plugboard - scripted to watch for any message where the source is CURRENTCOST parse and transform the event to a command and send out a PACHUBE logging message.
  • xap-pachube - accepts logging messages and pushes to the pachube web service.

Standard libraries

The HAH is supplied with some standard Lua library offerings.

  • luasocket - C - Sockets + modules for SMTP, FTP, HTTP, URL, MIME
  • lrexlib - C - POSIX Regular expressions
  • penlight - Lua - General purpose library of common patterns - I RECOMMEND YOU REFERENCE THIS
  • lfs - C - Lua File System

For more about each of these libraries see their associated reference pages.

The Lua xAP library extension

Lua xAP is an extension library that provides support for the construction of xAP compliant applications. The functions exposed are those used by the xaplib2 library and are extensively used by the 'built in' HAH applications.

The first thing to note is that using the plugboard generally takes a little personal effort. There are sample scripts available to help get you going, but it is assumed that you can prepare a script, get it onto the HAH and run it. Being familiar with tools such as FTP & vi, together with a basic knowledge of some Unix commands will make your work with the plugboard a lot easier. Familiarity of how xAP works will help too. The extension library has been designed to do much of the 'hard work' for you. Best of all, once you get the basic knowledge under your belt, there is a whole world of possibility for rule based processing and actioning.

There are only two calls that are needed to build an xAP Lua application.

Return Type Method Description
void xap.init(source,uid)
deprecated
Initialise an xAP application, bind with a hub if found and send a heartbeat every minute
source - a fully qualified 'vendorid.deviceid.instance' address.
void xap.init{vendorid=,deviceid=,instance=,uid=}
Initialize an xAP application, bind with a hub if found and send a heartbeat every minute.
vendorid - optional, default 'dbzoo'
deviceid - optional, default to hostname or setting from /etc/xap.d/system.ini
instance - mandatory, last component of 'vendorid.deviceid.instance' xAP address specification.
uid - mandatory, a unique identifier
void xap.process() Enter the xAP processing loop
void xap.send(msg) Send RAW data on the well known xAP UDP port 3639
String xap.expandShortMsg(msg) Inject/Replace xap-header information in a message
void xap.sendShort(msg) A shortcut for send(expandShortMsg(msg))
String xap.buildXapAdddress{vendorid=,deviceid=,instance=} Build an xAP address target string. vendorid and deviceid will default following the same rules as xap.init{}
String xap.getDeviceID() Returns the resolved deviceid, either hostname or /etc/xap.d/system.ini override

In its most simplistic form a basic application would look like this; It wouldn't do much except send a heartbeat every minute but it's functional.

require("xap")
xap.init{instance="simple",uid="FF00DD00"}
xap.process()

When building plugboard applets these calls won't need to be made as the plugboard engine will take care of it. They will be needed if writing standalone Lua applications something you'll do whilst in development/testing mode. More on writing Applets later.

The Frame class provides a container for holding and processing a RAW xAP message. A Filter callback will pass a frame object as the first parameter.

Return Type Method/Attribute Description
Frame xap.Frame(msg) Constructor: msg is an xAP message
int <obj>:getType() xap message type - xap.MSG_ORDINARY, xap.MSG_HBEAT or xap.MSG_UNKNOWN
string <obj>:getValue(section,key) Extract a value from a message by section/key - you may also use the short form <obj>[section].key ie frame[“xap-header”].class
boolean <obj>:isValue(section,key,value) Test if a section/key is a value
string tostring(f) Reconstruct the xAP message from the supplied Frame (f)

The following constants can be used when dealing with Frames whether they are global or local.

  • xap.FILTER_ANY
  • xap.FILTER_ABSENT

These are special keys that can be supplied as the key parameter to getValue() and isValue() to perform wild-carding and negation logic.

Sample usage

A Frame class is constructed like this.

msg = [[
xap-header
{
target=dbzoo.livebox.Controller:relay.1
class=xAPBSC.cmd
}
output.state.1
{
id=*
state=off
}]]
f = xap.Frame(msg)
print(tostring(f))
print(f) -- Implicitly calls the tostring() method as above.
 
-- These assertions are all TRUE, that is the message is *not* displayed.
 
assert(f:getType() == xap.MSG_ORDINARY, "getType failure")
assert(f:getValue("xap-header","class") == "xAPBSC.cmd")
assert(f:isValue("output.state.1","state","off"), "state is not off")
assert(f:isValue("output.state.1","state", xap.FILTER_ANY), "state is absent")
assert(f:isValue("output.state.1","missing", xap.FILTER_ABSENT), "missing is present")
 
print(f["xap-header"].class)

A quick note about empty sections: in this message the query section is empty.

xap-header
{
stuff...
}
query
{
}

And the follow tests will all assert to TRUE

assert(f:getValue("query","x") == nil)
assert(f:getValue("query",nil) == xap.FILTER_ANY)
assert(f:isValue("query",nil, xap.FILTER_ANY))

It's a bit weird that we overload xap.FILTER_ANY for this purpose perhaps we should introduce a new special return value called xap.EMPTY_SECTION

A filter allows an inbound xAP message to be acted upon based on the set of filter conditions applied. This is CORE to how a function is called to perform some work when the correct message is seen.

Return Type Method Description
Filter xap.Filter(filter) Constructor
void <obj>:destroy() Destructor
void <obj>:add(section,key,value) A condition matching an inbound xAP message, as many filters as needed can be added to the Filter object for matching.
void <obj>:delete(section,key,value) Removing a matching condition from a filter
void <obj>:callback(function, userdata) When all the filter conditions are met invoke the function, the function has a single parameter this is the FRAME matched by the filters. The function passed will receive as parameters (frame, userdata)

Wild-card pattern matching The source and class keys of an xap-header may be used in wild-card filter matching using the add method. Read the xAP protocol definition to find out more about wild-card addressing.

For example:

f:add("xap-header", "source" "dbzoo.livebox.controller:relay.*")
f:add("xap-header", "source" "dbzoo.livebox.*:relay.1")
f:add("xap-header", "source" "dbzoo.livebox.controller:>")
f:add("xap-header", "class" "xapbsc.*")

Wildcarding of addresses cannot be applied to a xap-header/target filter pair. WHY? Because an inbound message is allowed to have wildcarding. So trying to pattern match an inbound wildcard address against a wildcard filter address is meaningless.

f:add("xap-header", "target" "dbzoo.livebox.controller:relay.*")

NOT ALLOWED

The constructor can also accept a set of filter patterns directly; these are equivalent.

  filter = xap.Filter()
  filter:add("xap-header","class","xapbsc.event")
  filter:add("xap-header","source","dbzoo.livebox.>")
  filter:add("output.state","state","off")
 
  filter = xap.Filter {["xap-header"]={
                               class="xapbsc.event",
                               source="dbzoo.livebox.>"
                              },
                        ["output.state"]={
                               state="off"
                              }
                      }

Sample usage

Consider the following inbound xAP message for a heartbeat

xap-hbeat
{
v=12
hop=1
uid=FFAABB00
class=xap-hbeat.alive
source=acme.meteor.home.line1
interval=60
}

We can build a simple standalone heartbeat snooper in Lua with the following code.

require("xap")
 
function snoop(frame, pat)
   print(string.format(pat, 
                       frame["xap-hbeat"].source,
                       frame["xap-hbeat"].class))
end
 
xap.init{instance="test",uid="FF00CC00"}
f = xap.Filter()
f:add("xap-hbeat","source",xap.FILTER_ANY)
f:callback(snoop, "Heartbeat - Source %s Class %s")
xap.process()

A timer object allows a function to be called at a later date. Once created, it will count down from the interval supplied to the constructor until it reaches ZERO. At this point the function will be called and the timer will be reset. It will continue to call the function after every interval, until stopped.

Return Type Method/Attribute Description
Timer xap.Timer(function, interval, userdata) Constructor: Callback function and number of interval seconds before it's called. The function, when called, is passed the Timer object.
Timer <obj>:start(when) Start the timer. when=true fire immediately, when=false fire on 1st expiry, default: false (can be omitted)
Timer <obj>:reset() Reset the time to firing to the start interval
Timer <obj>:stop() Stop a timer
int <obj>.interval Attribute that is the interval of the timer, as this is mutable it's possible to dynamically adjust when the timer next fires.
int <obj>.ttl How long before the timer fires
int <obj>.userdata User Datum
void <obj>:delete() Stop the timer and free the Timer resources

Sample usage

This demonstration will fire the tick() function every 2 seconds until 10 seconds have elapsed at which point the timer will stop itself.

require "xap"
elapsed = 0
 
function tick(self, userdata)
  elapsed = elapsed + self.interval
  print("Tick "..elapsed)
  if elapsed > 10 then
    print(userdata)
    self:stop()
  end  
end
 
xap.Timer(tick, 2, "user data!"):start()
xap.process()

The select object allows a program to monitor multiple file descriptors, waiting until one of more of the file descriptors become “ready” for some class of I/O operation (e.g. input possible). A file descriptor is considered ready if it is possible to perform the corresponding I/O operation (e.g. read) without blocking.

Return Type Method Description
Select xap.Select(function, fd) Constructor: Function to callback when the file descriptor is ready.
void <obj>:delete() Stop listening and clean-up

Sample usage

As this is a complex class, although simple enough to use, the sample will be a little larger then normal to show fully how this works. The code is split into two pieces; the server and the client. The assert() wrapper just makes sure that the call worked … if not, the code stops at that point.

The SERVER

-- UDP to xAP gateway
--
-- Demonstrates listening on a socket and
-- rebroadcasting the incoming data as an xAP message
 
require("xap")
require("socket")
 
function handleSocket()
   dgram, ip, port = udp:receivefrom()   -- Read data from the UDP socket
   print("Received '"..dgram .. "' from "..ip..":"..port)
 
-- Retransmit as an xAP payload.
   msg = string.format([[
xap-header
{
class=udp.data
}
body
{
ip=%s
port=%s
data=%s
}]], ip, port, dgram)
   xap.sendShort(msg)
end
 
-- MAIN --
host = host or "localhost"
port = port or 6666
if arg then
    host = arg[1] or host
    port = arg[2] or port
end
 
xap.init{instance="socket",uid="FF00CC00"}
 
print("Binding to host '" ..host.. "' and port " ..port.. "...")
host = socket.dns.toip(host)          -- Convert host name to an IP
udp = assert(socket.udp())            -- Create a UDP socket
assert(udp:setsockname(host, port))   -- on IP:PORT
 
-- Listen on the SOCKET file descriptor and call handleSocket when something is 'ready'
xap.Select(handleSocket, udp)
 
ip, port = udp:getsockname()  -- Get IP:PORT from the UDP object
print("Waiting for packets on " .. ip .. ":" .. port .. "...")
 
xap.process()   -- Enter the processing LOOP

the CLIENT

-- UDP to xAP client
-- Send data to our gateway
 
local socket = require("socket")
host = host or "localhost"
port = port or 6666
if arg then
    host = arg[1] or host
    port = arg[2] or port
end
 
host = socket.dns.toip(host)         -- host name to IP
udp = assert(socket.udp())           -- Create a UDP socket
assert(udp:setpeername(host, port))  -- On IP:PORT
print("Using remote host '" ..host.. "' and port " .. port .. "...")
print("Enter text to sent to the UDP/xAP gateway, blank line will exit.")
while 1 do
        line = io.read()
        if not line or line == "" then os.exit() end
        assert(udp:send(line))       -- Send keyboard input to UDP://host:port
end

This class allows the easy construction of BSC endpoints. By default the following functionality is provided:

  • responding to a xAPBSC.query with an xAPBSC.info message
  • sending a xAPBSC.info message every 2 mins
  • handling a xAPBSC.cmd and sending a xAPBSC.event
Return Type Method Description
Endpoint Endpoint(table) Constructor: a container to hold endpoints
nil <obj>:destroy() Destructor: remove the endpoint from existence
nil <obj>:sendEvent() Send xAPBSC.event for this endpoint
nil <obj>:sendInfo() Send xAPBSC.info for this endpoint
nil <obj>:setText(string) Populate the “text=” field and mark the “state=on”
nil <obj>:setDisplayText(string) Populate the “displaytext=” field and mark the “state=on”
nil <obj>:setState(arg) Populate the “state=” field. arg is a BSC state constant
state decodeState(string) helper function to decode a string into a BSC state constant, the following strings are correctly handled: 1,0,on,off,yes,no,true,false,toggle

The following constants can be used when dealing with the state:

  • bsc.STATE_ON
  • bsc.STATE_OFF
  • bsc.STATE_TOGGLE

More about the Endpoint(table) constructor. The key for the table may be one of the following:

name : Append the endpoint name. Completes the triple; vendorid.deviceid.instance(:name)

instance: xAP address for the endpoint. Completes the triple; vendorid.deviceid.(instance)

direction: Has two possible values bsc.INPUT and bsc.OUTPUT. An OUTPUT endpoint will respond to xAPBSC.cmd events, INPUTS will not.

type: BSC supports 3 endpoint types: bsc.STREAM, bsc.LEVEL and bsc.BINARY

cmdCB(Endpoint): This function allows the program to take some action when the endpoint received a directed command. It takes one argument - the endpoint that just fired

infoEventCB(Endpoint, string): This function is called whenever a xAPBSC.info or xAPBSC.event is to be sent out. If you want the action to happen, return true otherwise return false. This allows programmatic control for setting up the displayText or to suppress an event if the change isn't out of range yet. It takes two arguments: the endpoint and the class of the event, being either: bsc.INFO_CLASS which has the value “xAPBSC.info” or bsc.EVENT_CLASS assigned to “xAPBSC.event”.

timeout Allow overriding of the default 120 second timeout for sending an xapBSC.info message.

ttl Time To Live. If a xapBSC.event event is not seen in this many seconds the current value is invalidated and future xapBSC.info message will report a question mark (?). A value of ZERO will disable this functionality. Default 0.

uid Override the UID (last 2 digits) for the created Enpoint. If this is not supplied an internal uid counter will be used. Any time this UID key is supplied it resets the internal counter so subsequent Endpoints that are created automatically start incrementing from this value.

Endpoint attributes that accessible from the Callbacks (all of the above plus)

state The BSC state of the endpoint nominally: on, off or ?

text The current TEXT value: applicable to STREAM and LEVEL types

displaytext An additional parameter that may be include in an EVENT of INFO message, configurable with a infoEventCB callback function.

uid UID key. If nothing is provide one will be automatically assigned based on the order of BSC endpoints created so far.

A standalone example save to a file an run with # lua <filename>

require "xap"
require "xap.bsc"
 
function lcdCmd(endpoint)
   print("LCD action: "..endpoint.text)
end
 
function relayCmd(endpoint)
   print("Relay action: " .. endpoint.name)
end
 
function relayInfoEvent(endpoint, clazz)
   endpoint.displaytext = "Relay is " .. endpoint.state
   return true -- Return TRUE if the INFO/EVENT should be emitted / FALSE otherwise
end
 
xap.init{instance="test",uid="FF0CC000"}
 
-- Form 1 : Constructed using the xAP address from xap.init()
-- creates -> dbzoo.livebox.test:lcd, dbzoo.livebox.test:relay.1 etc..
bsc.Endpoint{name="lcd", direction=bsc.OUTPUT, type=bsc.STREAM, cmdCB=lcdCmd}
bsc.Endpoint{name="relay.1", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd, infoEventCB=relayInfoEvent}
bsc.Endpoint{name="relay.2", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd}
 
-- Form 2 : vendorid and deviceid default, remaining from instance is appended
-- creates -> dbzoo.livebox.my:relay.3
bsc.Endpoint{instance="my:relay.3", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd}
 
-- Form 3 : vendorid defaults, override deviceid.
-- creates -> dbzoo.test.my:relay.4
bsc.Endpoint{deviceid="test", instance="my:relay.4", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd}
 
-- Form 4 : full override all keys
-- creates -> hello.test.my:relay.4
bsc.Endpoint{vendorid="hello", deviceid="test", instance="my:relay.4", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd}
 
-- Form 5 : Shorter way of doing form 4.
-- creates -> helloworld.test.my:relay.5
bsc.Endpoint{source="hithere.test.my:relay.5", direction=bsc.OUTPUT, type=bsc.BINARY, cmdCB=relayCmd}
 
xap.process()

lcdCMD() and relayCmd() perform some work to generate a message on the LCD or to turn on a Relay - These are the CONTROL function that interface the xAP endpoint to a REAL action. RelayInfoEvent() builds up any additional information that needed to be sent.

A working example. This Applet creates an endpoint that shows how much memory Lua is currently using.

--[[
  LUA Garbage memory usage monitor
--]]
 
module(...,package.seeall)
 
require("xap")
require("xap.bsc")
 
info={
   version="1.0", description="LUA GC Usage"
}
 
function update(t,e)
  e:setText( collectgarbage("count")*1024 )
  e:sendEvent()
end
 
function init()
  -- Creates:  dbzoo.livebox.Plugboard:gc
  local e = bsc.Endpoint{name="gc", direction=bsc.INPUT, type=bsc.STREAM}
  xap.Timer(update, 60, e):start()
end

For a working example see the JeeNode integration code on your HAH box in /usr/share/lua/5.1/xap

BSC Command helper functions

The following functions are used to send an xAPBSC.cmd message.

  • bsc.sendState(target, state)
  • bsc.sendText(target,text, state) – state is optional, defaults to 'on'
  • bsc.sendLevel(target, level, state) – state is optional, defaults to 'on'
  • bsc.send{} – takes table of elements. Target will always go into the xap-header, all others will be placed into the body of output.state.1

It's a RAW low level control function that is utilized by the higher level functions: sendText, sendState, sendLevel

STATE can be any of the following: on, off, true, false, 1, 0, toggle (Nominally you would use on and off)

Why are these new API here? Consider this fragment of code to send a BSC state control message.

xap.sendShort(string.format([[xap-header
{
target=dbzoo.livebox.Controller:ppe.%s
class=xAPBSC.cmd
}
output.state.1
{
id=*
state=%s
}]], ppeTarget, state))
end

This can now be written in a nice compact form.

   bsc.sendState("dbzoo.livebox.Controller:ppe."..ppeTarget, state)

Standalone sample demonstrating their usage.

require("xap")
require("xap.bsc")
 
xap.init{instance="test",uid="FF00AA00"}
 
bsc.sendState("dbzoo.livebox.Controller:relay.1", "on")
bsc.sendState("dbzoo.livebox.Controller:relay.1", "toggle")
bsc.sendState("dbzoo.livebox.Controller:relay.1", "off")
bsc.send{target="dbzoo.livebox.Controller:relay.1",state="on"}
 
bsc.sendText("dbzoo.livebox.Controller:relay.1","hello")
bsc.send{target="dbzoo.livebox.Controller:relay.1",text="hello", state="on"}
bsc.send{target="dbzoo.livebox.Controller:relay.1",text="hello", displaytext="hello world", state="on"}

Writing an Applet

Applets are small Lua scripts that are automatically invoked by the plugboard launcher. Their basic structure is

require("xap")
module (...,package.seeall)
 
info = {version="1.0", description="What this does"}
 
function init()
end

The init() function will be invoked when the applet is loaded by the plugboard.lua script.

To be automatically loaded they must be placed in the /etc/plugboard directory and have the suffix Applet.lua. Example file names:

  • hbeatWatchdogApplet.lua
  • bindRelaysApplet.lua

As an applet we can run the plugboard engine manually. Running it this way will also display other information not normally seen when running it as a service and is useful for debugging.

# xap-plugboard
Loading /etc/plugboard/bindRelaysApplet.lua     [ Binding relays 2,3,4 ]
Loading /etc/plugboard/hbeatWatchdogApplet.lua  [ Process Watchdog ]
Running...

Writing xAP Lua applications

When developing a new Applet it may be convenient to start our writing a self contained Lua xAP application using the following structure which is only a small step away from its Applet form.

require("xap")
 
info = {version="1.0", description="What this does"}
 
function init()
end
 
xap.init{instance="example",uid="FF00DD00"}
init()
xap.process()

This allows you to work in another directory without constantly stopping and starting applets already in place when developing something new.

How do you run these code fragments? From the command prompt it's as easy as:

# lua standAloneProgram.lua

More samples

Whilst there is flash space available, some samples will be included with the firmware

cd /etc_ro_fs/plugboard/samples

There is a section of the forum, http://www.homeautomationhub.com/forum dedicated to example scripts, and another area for discussing scripting.