Merge pull request #93 from sidoh/v1.4.0

v1.4.0 -- passive listening, MQTT updates

README.md

@@ -11,6 +11,7 @@ This is a replacement for a Milight/LimitlessLED remote/gateway hosted on an ESP
 2. This project exposes a nice REST API to control your bulbs.
 3. You can secure the ESP8266 with a username/password, which is more than you can say for the Milight gateway! (The 2.4 GHz protocol is still totally insecure, so this doesn't accomplish much :).
 4. Official hubs connect to remote servers to enable WAN access, and this behavior is not disableable.
+5. This project is capable of passively listening for Milight packets sent from other devices (like remotes). It can publish data from intercepted packets to MQTT. This could, for example, allow the use of Milight remotes while keeping your home automation platform's state in sync. See the MQTT section for more detail.
 
 ## Supported bulbs
 
@@ -89,7 +90,7 @@ The HTTP endpoints (shown below) will be fully functional at this point. You sho
 1. `GET /settings`. Gets current settings as JSON.
 1. `PUT /settings`. Patches settings (e.g., doesn't overwrite keys that aren't present). Accepts a JSON blob in the body.
 1. `GET /radio_configs`. Get a list of supported radio configs (aka `device_type`s).
-1. `GET /gateway_traffic/:device_type`. Starts an HTTP long poll. Returns any Milight traffic it hears. Useful if you need to know what your Milight gateway/remote ID is. Since protocols for RGBW/CCT are different, specify one of `rgbw`, `cct`, or `rgb_cct` as `:device_type.
+1. `GET /gateway_traffic(/:device_type)?`. Starts an HTTP long poll. Returns any Milight traffic it hears. Useful if you need to know what your Milight gateway/remote ID is. Since protocols for RGBW/CCT are different, specify one of `rgbw`, `cct`, or `rgb_cct` as `:device_type.  The path `/gateway_traffic` without a `:device_type` will sniff for all protocols simultaneously.
 1. `PUT /gateways/:device_id/:device_type/:group_id`. Controls or sends commands to `:group_id` from `:device_id`. Accepts a JSON blob. The schema is documented below in the _Bulb commands_ section.
 1. `POST /raw_commands/:device_type`. Sends a raw RF packet with radio configs associated with `:device_type`. Example body:
     ```
@@ -174,6 +175,28 @@ irb(main):004:0> client.publish('milight/0x118D/rgb_cct/1', '{"status":"ON","col
 
 This will instruct the ESP to send messages to RGB+CCT bulbs with device ID `0x118D` in group 1 to turn on, set color to RGB(255,200,255), and brightness to 100.
 
+#### Updates
+
+To enable passive listening, make sure that `listen_repeats` is set to something larger than 0 (the default value of 3 is a good choice).
+
+To publish data from intercepted packets to an MQTT topic, configure MQTT server settings, and set the `mqtt_update_topic_pattern` to something of your choice. As with `mqtt_topic_pattern`, the tokens `:device_id`, `:device_type`, and `:group_id` will be substituted with the values from the relevant packet.
+
+The published message is a JSON blob containing the following keys:
+
+* `device_id`
+* `device_type` (rgb_cct, rgbw, etc.)
+* `group_id`
+* Any number of: `status`, `level`, `hue`, `saturation`, `kelvin`
+
+As an example, if `mqtt_update_topic_pattern` is set to `milight/updates/:device_id/:device_type/:group_id`, and the group 1 on button of a Milight remote is pressed, the following update will be dispatched:
+
+```ruby
+irb(main):005:0> client.subscribe('milight/updates/+/+/+')
+=> 27
+irb(main):006:0> puts client.get.inspect
+["lights/updates/0x1C8E/rgb_cct/1", "{\"device_id\":7310,\"group_id\":1,\"device_type\":\"rgb_cct\",\"status\":\"on\"}"]
+```
+
 ## UDP Gateways
 
 You can add an arbitrary number of UDP gateways through the REST API or through the web UI. Each gateway server listens on a port and responds to the standard set of commands supported by the Milight protocol. This should allow you to use one of these with standard Milight integrations (SmartThings, Home Assistant, OpenHAB, etc.).

data/web/index.html

@@ -31,11 +31,22 @@
     .error-info:before { content: '('; }
     .error-info:after { content: ')'; }
     .header-btn { margin: 20px; }
+    #sniffed-traffic { max-height: 50em; overflow-y: auto; }
     .btn-secondary {
       background-color: #fff;
       border: 1px solid #ccc;
     }
     .inline { display: inline-block; }
+    .white-temp-picker {
+      height: 2em;
+      background: linear-gradient(to right,
+        rgb(166, 209, 255) 0%,
+        rgb(255, 255, 255) 50%,
+        rgb(255, 160, 0) 100%
+      );
+      display: inline-block;
+      padding: 3px 0;
+    }
     .hue-picker {
       height: 2em;
       width: 100%;
@@ -124,8 +135,9 @@
   <script lang="text/javascript">
     var FORM_SETTINGS = [
       "admin_username", "admin_password", "ce_pin", "csn_pin", "reset_pin","packet_repeats",
-      "http_repeat_factor", "auto_restart_period", "discovery_port", "mqtt_server", 
-      "mqtt_topic_pattern", "mqtt_username", "mqtt_password", "radio_interface_type"
+      "http_repeat_factor", "auto_restart_period", "discovery_port", "mqtt_server",
+      "mqtt_topic_pattern", "mqtt_update_topic_pattern", "mqtt_username", "mqtt_password",
+      "radio_interface_type", "listen_repeats"
     ];
 
     var FORM_SETTINGS_HELP = {
@@ -141,9 +153,14 @@
       mqtt_server : "Domain or IP address of MQTT broker. Optionally specify a port " +
         "with (example) mymqqtbroker.com:1884.",
       mqtt_topic_pattern : "Pattern for MQTT topics to listen on. Example: " +
-        "lights/:device_id/:type/:group. See README for further details.",
+        "lights/:device_id/:device_type/:group_id. See README for further details.",
+      mqtt_update_topic_pattern : "Pattern to publish MQTT updates. Packets that " +
+        "are received from other devices, and packets that are sent from this device will " +
+        "result in updates being sent.",
       discovery_port : "UDP port to listen for discovery packets on. Defaults to " +
-        "the same port used by MiLight devices, 48899. Use 0 to disable."
+        "the same port used by MiLight devices, 48899. Use 0 to disable.",
+      listen_repeats : "Increasing this increases the amount of time spent listening for " +
+        "packets. Set to 0 to disable listening. Default is 3."
     }
 
     var UDP_PROTOCOL_VERSIONS = [ 5, 6 ];
@@ -207,10 +224,8 @@
     var sniffRequest;
     var sniffing = false;
     var getTraffic = function() {
-      var sniffType = $('#sniff-type input:checked').data('value');
-
-      sniffRequest = $.get('/gateway_traffic/' + sniffType, function(data) {
-        $('#sniffed-traffic').html(data + $('#sniffed-traffic').html());
+      sniffRequest = $.get('/gateway_traffic', function(data) {
+        $('#sniffed-traffic').prepend('<pre>' + data + '</pre>');
         getTraffic();
       });
     };
@@ -855,11 +870,13 @@ <h5>Color Temperature</h5>
       </div>
       <div class="row">
         <div class="col-sm-6">
-          <input class="slider raw-update" name="temperature"
-              data-slider-min="0"
-              data-slider-max="100"
-              data-slider-value="100"
-          />
+          <div class="white-temp-picker">
+            <input class="slider raw-update" name="temperature"
+                data-slider-min="0"
+                data-slider-max="100"
+                data-slider-value="100"
+            />
+          </div>
         </div>
       </div>
     </div>
@@ -1006,24 +1023,9 @@ <h1>Sniff Traffic</h1>
       <div class="col-sm-12">
         <button type="button" id="sniff" class="btn btn-primary">Start Sniffing</button>
 
-        <div class="btn-group" id="sniff-type" data-toggle="buttons">
-          <label class="btn btn-secondary active">
-            <input type="radio" name="options" autocomplete="off" data-value="rgbw" checked> RGBW
-          </label>
-          <label class="btn btn-secondary">
-            <input type="radio" name="options" autocomplete="off" data-value="cct"> CCT
-          </label>
-          <label class="btn btn-secondary">
-            <input type="radio" name="options" autocomplete="off" data-value="rgb_cct"> RGB+CCT
-          </label>
-          <label class="btn btn-secondary">
-            <input type="radio" name="options" autocomplete="off" data-value="rgb"> RGB
-          </label>
-        </div>
-
         <div> &nbsp; </div>
 
-        <pre id="sniffed-traffic"></pre>
+        <div id="sniffed-traffic"></div>
       </div>
     </div>
 

lib/Helpers/Units.h

@@ -0,0 +1,39 @@
+#include <Arduino.h>
+#include <inttypes.h>
+
+#ifndef _UNITS_H
+#define _UNITS_H
+
+// MiLight CCT bulbs range from 2700K-6500K, or ~370.3-153.8 mireds.
+#define COLOR_TEMP_MAX_MIREDS 370
+#define COLOR_TEMP_MIN_MIREDS 153
+
+class Units {
+public:
+  template <typename T, typename V>
+  static T rescale(T value, V newMax, float oldMax = 255.0) {
+    return round(value * (newMax / oldMax));
+  }
+
+  static uint8_t miredsToWhiteVal(uint16_t mireds, uint8_t maxValue = 255) {
+      uint32_t tempMireds = constrain(mireds, COLOR_TEMP_MIN_MIREDS, COLOR_TEMP_MAX_MIREDS);
+
+      uint8_t scaledTemp = round(
+        maxValue*
+        (tempMireds - COLOR_TEMP_MIN_MIREDS)
+          /
+        static_cast<double>(COLOR_TEMP_MAX_MIREDS - COLOR_TEMP_MIN_MIREDS)
+      );
+
+      return scaledTemp;
+  }
+
+  static uint16_t whiteValToMireds(uint8_t value, uint8_t maxValue = 255) {
+    uint8_t reverseValue = maxValue - value;
+    uint16_t scaled = rescale<uint16_t, uint16_t>(reverseValue, (COLOR_TEMP_MAX_MIREDS - COLOR_TEMP_MIN_MIREDS), maxValue);
+
+    return COLOR_TEMP_MIN_MIREDS + scaled;
+  }
+};
+
+#endif

lib/MQTT/MqttClient.cpp

@@ -85,6 +85,27 @@ void MqttClient::handleClient() {
   mqttClient->loop();
 }
 
+void MqttClient::sendUpdate(MiLightRadioType type, uint16_t deviceId, uint16_t groupId, const char* update) {
+  String topic = settings.mqttUpdateTopicPattern;
+
+  if (topic.length() == 0) {
+    return;
+  }
+
+  String deviceIdStr = String(deviceId, 16);
+  deviceIdStr.toUpperCase();
+
+  topic.replace(":device_id", String("0x") + deviceIdStr);
+  topic.replace(":group_id", String(groupId));
+  topic.replace(":device_type", MiLightRadioConfig::fromType(type)->name);
+
+#ifdef MQTT_DEBUG
+  printf_P(PSTR("MqttClient - publishing update to %s: %s\n"), topic.c_str(), update);
+#endif
+
+  mqttClient->publish(topic.c_str(), update);
+}
+
 void MqttClient::subscribe() {
   String topic = settings.mqttTopicPattern;
 

lib/MQTT/MqttClient.h

@@ -18,6 +18,7 @@ class MqttClient {
   void begin();
   void handleClient();
   void reconnect();
+  void sendUpdate(MiLightRadioType type, uint16_t deviceId, uint16_t groupId, const char* update);
 
 private:
   WiFiClient tcpClient;

lib/MiLight/CctPacketFormatter.cpp

@@ -107,6 +107,62 @@ uint8_t CctPacketFormatter::getCctStatusButton(uint8_t groupId, MiLightStatus st
   return button;
 }
 
+uint8_t CctPacketFormatter::cctCommandIdToGroup(uint8_t command) {
+  switch (command & 0xF) {
+    case CCT_GROUP_1_ON:
+    case CCT_GROUP_1_OFF:
+      return 1;
+    case CCT_GROUP_2_ON:
+    case CCT_GROUP_2_OFF:
+      return 2;
+    case CCT_GROUP_3_ON:
+    case CCT_GROUP_3_OFF:
+      return 3;
+    case CCT_GROUP_4_ON:
+    case CCT_GROUP_4_OFF:
+      return 4;
+    case CCT_ALL_ON:
+    case CCT_ALL_OFF:
+      return 0;
+  }
+
+  return 255;
+}
+
+MiLightStatus CctPacketFormatter::cctCommandToStatus(uint8_t command) {
+  switch (command & 0xF) {
+    case CCT_GROUP_1_ON:
+    case CCT_GROUP_2_ON:
+    case CCT_GROUP_3_ON:
+    case CCT_GROUP_4_ON:
+    case CCT_ALL_ON:
+      return ON;
+    case CCT_GROUP_1_OFF:
+    case CCT_GROUP_2_OFF:
+    case CCT_GROUP_3_OFF:
+    case CCT_GROUP_4_OFF:
+    case CCT_ALL_OFF:
+      return OFF;
+  }
+}
+
+void CctPacketFormatter::parsePacket(const uint8_t* packet, JsonObject& result) {
+  uint8_t command = packet[CCT_COMMAND_INDEX] & 0x7F;
+
+  result["device_id"] = (packet[1] << 8) | packet[2];
+  result["device_type"] = "cct";
+  result["group_id"] = packet[3];
+
+  uint8_t onOffGroupId = cctCommandIdToGroup(command);
+  if (onOffGroupId < 255) {
+    result["state"] = cctCommandToStatus(command) == ON ? "ON" : "OFF";
+  }
+
+  if (! result.containsKey("state")) {
+    result["state"] = "ON";
+  }
+}
+
 void CctPacketFormatter::format(uint8_t const* packet, char* buffer) {
   PacketFormatter::formatV1Packet(packet, buffer);
 }

lib/MiLight/CctPacketFormatter.h

@@ -43,8 +43,11 @@ class CctPacketFormatter : public PacketFormatter {
 
   virtual void format(uint8_t const* packet, char* buffer);
   virtual void initializePacket(uint8_t* packet);
+  virtual void parsePacket(const uint8_t* packet, JsonObject& result);
 
   static uint8_t getCctStatusButton(uint8_t groupId, MiLightStatus status);
+  static uint8_t cctCommandIdToGroup(uint8_t command);
+  static MiLightStatus cctCommandToStatus(uint8_t command);
 };
 
 #endif

lib/MiLight/MiLightClient.cpp

@@ -2,14 +2,13 @@
 #include <MiLightRadioConfig.h>
 #include <Arduino.h>
 #include <RGBConverter.h>
-
-#define COLOR_TEMP_MAX_MIREDS 370
-#define COLOR_TEMP_MIN_MIREDS 153
+#include <Units.h>
 
 MiLightClient::MiLightClient(MiLightRadioFactory* radioFactory)
   : resendCount(MILIGHT_DEFAULT_RESEND_COUNT),
     currentRadio(NULL),
-    numRadios(MiLightRadioConfig::NUM_CONFIGS)
+    numRadios(MiLightRadioConfig::NUM_CONFIGS),
+    packetSentHandler(NULL)
 {
   radios = new MiLightRadio*[numRadios];
 
@@ -63,7 +62,14 @@ void MiLightClient::prepare(MiLightRadioConfig& config,
   const uint16_t deviceId,
   const uint8_t groupId) {
 
-  switchRadio(config.type);
+  prepare(config.type, deviceId, groupId);
+}
+
+void MiLightClient::prepare(MiLightRadioType type,
+  const uint16_t deviceId,
+  const uint8_t groupId) {
+
+  switchRadio(type);
 
   if (deviceId >= 0 && groupId >= 0) {
     formatter->prepare(deviceId, groupId);
@@ -110,6 +116,10 @@ void MiLightClient::write(uint8_t packet[]) {
     currentRadio->write(packet, currentRadio->config().getPacketLength());
   }
 
+  if (this->packetSentHandler) {
+    this->packetSentHandler(packet, currentRadio->config());
+  }
+
 #ifdef DEBUG_PRINTF
   int iElapsed = millis() - iStart;
   Serial.print("Elapsed: ");
@@ -274,7 +284,7 @@ void MiLightClient::update(const JsonObject& request) {
   }
   // HomeAssistant
   if (request.containsKey("brightness")) {
-    uint8_t scaledBrightness = round(request.get<uint8_t>("brightness") * (100/255.0));
+    uint8_t scaledBrightness = Units::rescale(request.get<uint8_t>("brightness"), 100, 255);
     this->updateBrightness(scaledBrightness);
   }
 
@@ -283,20 +293,9 @@ void MiLightClient::update(const JsonObject& request) {
   }
   // HomeAssistant
   if (request.containsKey("color_temp")) {
-    // MiLight CCT bulbs range from 2700K-6500K, or ~370.3-153.8 mireds. Note
-    // that mireds are inversely correlated with color temperature.
-    uint32_t tempMireds = request["color_temp"];
-    tempMireds = tempMireds > COLOR_TEMP_MAX_MIREDS ? COLOR_TEMP_MAX_MIREDS : tempMireds;
-    tempMireds = tempMireds < COLOR_TEMP_MIN_MIREDS ? COLOR_TEMP_MIN_MIREDS : tempMireds;
-
-    uint8_t scaledTemp = round(
-      100*
-      (tempMireds - COLOR_TEMP_MIN_MIREDS)
-        /
-      static_cast<double>(COLOR_TEMP_MAX_MIREDS - COLOR_TEMP_MIN_MIREDS)
+    this->updateTemperature(
+      Units::miredsToWhiteVal(request["color_temp"], 100)
     );
-
-    this->updateTemperature(100 - scaledTemp);
   }
 
   if (request.containsKey("mode")) {
@@ -375,3 +374,7 @@ void MiLightClient::flushPacket() {
   setResendCount(prevNumRepeats);
   formatter->reset();
 }
+
+void MiLightClient::onPacketSent(PacketSentHandler handler) {
+  this->packetSentHandler = handler;
+}