Data types, write quota, better stats

Author: budulinek

README.md

@@ -54,14 +54,13 @@ The controller has a built-in web interface. You can use this web interface to c
 # Hardware
 Get the hardware and connect together:
 
-* **Arduino Uno**.
-
-* **Ethernet shield (with W5100, W5200 or W5500 chip)**. The ubiquitous W5100 shield for Uno is sufficient. !!! ENC28J60 will not work !!!
-* **Custom P1P2 Uno adapter**. You can [solder your own adapter](https://github.com/Arnold-n/P1P2Serial/tree/main/circuits#p1p2-adapter-as-arduino-uno-hat) or buy one from Arnold-n (his e-mail address can be found on line 3 of his library [P1P2Serial.cpp](https://github.com/Arnold-n/P1P2Serial/blob/main/P1P2Serial.cpp)).
+* **Arduino Uno**.<br>Cheap clones are sufficient.
+* **Ethernet shield with WIZnet chip (W5100, W5200 or W5500)**.<br>The ubiquitous W5100 shield for Uno/Mega is sufficient. If available, I recommend W5500 Ethernet Shield. You can also use combo board MCU + ethernet (such as ATmega328 + W5500 board from Keyestudio).<br>ATTENTION: Ethernet shields with ENC28J60 chip will not work !!!
+* **Custom P1P2 Uno adapter**.<br>You can [solder your own adapter](https://github.com/Arnold-n/P1P2Serial/tree/main/circuits#p1p2-adapter-as-arduino-uno-hat) or buy one from Arnold-n (his e-mail address can be found on line 3 of his library [P1P2Serial.cpp](https://github.com/Arnold-n/P1P2Serial/blob/main/P1P2Serial.cpp)).
 
 Here is my HW setup (cheap Arduino Uno clone + W5500 Ethernet shield from Keyestudio + custom P1P2 Uno adapter):
 
-<img src="pics/HW.jpg" alt="01" style="zoom:100%;" />
+<img src="pics/HW.jpg" alt="HW" style="zoom:100%;" />
 
 # Firmware
 
@@ -80,19 +79,19 @@ This controller has a built-in webserver which allows you to configure the contr
 
 ## System Info
 
-<img src="pics/daikin1.png" alt="01" style="zoom:100%;" />
+<img src="pics/daikin1.png" alt="daikin1" style="zoom:100%;" />
 
 **Load Default Settings**. Loads default settings (see DEFAULT_CONFIG in advanced settings). MAC address is retained.
 
 **Reboot**.
 
 **EEPROM Health**. Keeps track of EEPROM write cycles (this counter is persistent, never cleared during factory resets). Replace your Arduino once you reach 100 000 write cycles (with 6 hours EEPROM_INTERVAL you have more than 50 years lifespan).
 
-**Generate New MAC**. Generate new MAC address. First 3 bytes are fixed 90:A2:DA, remaining 3 bytes are true random.
+**MAC Address**. First 3 bytes are fixed 90:A2:DA, remaining 3 bytes are random. You can also set manual MAC in IP Settings.
 
 ## P1P2 Status
 
-<img src="pics/daikin2.png" alt="02" style="zoom:100%;" />
+<img src="pics/daikin2.png" alt="daikin2" style="zoom:100%;" />
 
 **Controller**. Shows you the status of the P1/P2 connection and allows you to manually connect (or disconnect) the controller. The status can be:
 * **Disabled**. Controller mode has been disabled in **P1P2 Settings**. The controller can not write to the P1/P2 bus but it still passively monitors the P1/P2 bus and sends (most) data from the heat pump via UDP messages.
@@ -123,8 +122,14 @@ This controller has a built-in webserver which allows you to configure the contr
 * **Value**. Parameter value, the number of bytes differs for various packet types. See PACKET_PARAM_VAL_SIZE in advanced settings for the correct number of bytes. Value is also **<ins>in little endian format</ins>**!
 
 **P1P2 Packets**. Counters for packets read from the P1/P2 bus or written to the P1/P2 bus, counters for various read and write errors. If any of the counters rolls over the unsigned long maximum (4,294,967,295), all counters will reset to 0.
-* **Read OK**. Number of packets read from the P1/P2 bus, without errors. Not all of them are sent via UDP (see the **Packet Filter** settings). Packets are read from the P1/P2 bus (and sent via UDP) even if the controller is not connected to the P1/P2 bus.
-* **Write OK**. Number of packets written to the P1/P2 bus. Includes both packets written automatically by the controller (requests for the counters packets), write commands from the web interface (**Write Packet**) and write commands received via UDP. Writing to the P1/P2 bus is only possible if the controller is connected to the P1/P2 bus (to the main Daikin controller).
+* **Bus Read OK**. Number of packets read from the P1/P2 bus, without errors. Not all of them are sent via UDP (see the **Packet Filter** settings). Packets are read from the P1/P2 bus (and sent via UDP) even if the controller is not connected to the P1/P2 bus.
+* **Bus Write OK**. Number of packets written to the P1/P2 bus. Includes both packets written automatically by the controller (requests for the counters packets), write commands from the web interface and write commands received via UDP. Writing to the P1/P2 bus is only possible if the controller is connected to the P1/P2 bus (to the main Daikin controller).
+* **EEPROM Write Quota Reached**. Daily EEPROM Write Quota (configured in **P1P2 Settings**) was reached. The command (received via UDP or from the web interface) was dropped.
+* **Write Queue Full**. Internal queue (circular buffer) for commands is full. The command (received via UDP or from the web interface) was dropped.
+* **Write Command Invalid**. Command received via UDP or from the web interface was invalid, it was dropped. Possible reasons:
+  - Packet type (first byte) is not supported (PACKET_PARAM_VAL_SIZE in advanced settings is set to zero).
+  - Incorrect packet length. Command should have 1 byte for type, 2 bytes for parameter number and the correct numer of bytes for the parameter value (see PACKET_PARAM_VAL_SIZE in advanced settings).
+
 * **Parity Read Error**.
 * **Too Long Read Error**. Packet received is longer than the read buffer.
 * **Start Bit Write Error**. Start bit error during write.
@@ -134,15 +139,13 @@ This controller has a built-in webserver which allows you to configure the contr
 
 **UDP Messages**.\*\*
 * **Sent to UDP**. Counts packets (messages) read from the P1/P2 bus and sent via UDP. Not all packets read from the P1/P2 bus are sent via UDP (see the **Packet Filter** settings).
-* **Received from UDP**. Counts valid messages received via UDP. Messages are validated:
-  - Packet type (first byte) is supported (PACKET_PARAM_VAL_SIZE in advanced settings is not zero).
-  - Correct packet length. 1 byte for type, 2 bytes for parameter number and the correct numer of param vallue bytes (see PACKET_PARAM_VAL_SIZE in advanced settings).
-  - The internal queue (circular buffer) for commands is not full.
-
+* **Received from UDP**. Counts all messages received via UDP from a valid remote IP.
 
 ## IP Settings
 
-<img src="pics/daikin3.png" alt="02" style="zoom:100%;" />
+<img src="pics/daikin3.png" alt="daikin3" style="zoom:100%;" />
+
+**MAC Address**.\*\* Change MAC address. "Randomize" button will generate new random MAC (first 3 bytes fixed 90:A2:DA, last 3 bytes will be random).
 
 **Auto IP**.\* Once enabled, Arduino will receive IP, gateway, subnet and DNS from the DHCP server.
 
@@ -156,7 +159,7 @@ This controller has a built-in webserver which allows you to configure the contr
 
 ## TCP/UDP Settings
 
-<img src="pics/daikin4.png" alt="02" style="zoom:100%;" />
+<img src="pics/daikin4.png" alt="daikin4" style="zoom:100%;" />
 
 **Remote IP**. IP address of your home automation system which listens for UDP messages and sends UDP commands.
 
@@ -170,7 +173,7 @@ This controller has a built-in webserver which allows you to configure the contr
 
 ## P1P2 Settings
 
-<img src="pics/daikin5.png" alt="05" style="zoom:100%;" />
+<img src="pics/daikin5.png" alt="daikin5" style="zoom:100%;" />
 
 **Controller (Write to P1P2)**.
 * **Disabled** (safe). The controller is permanently disconnected from the P1/P2 bus, manual connection is not possible. The controller can not write to the P1/P2 bus but it still passively monitors the P1/P2 bus and sends (most) data from the heat pump via UDP. Disable the controller in case you experience persistent connection failures and/or write errors.
@@ -185,7 +188,7 @@ This controller has a built-in webserver which allows you to configure the contr
 
 ## Packet Filter
 
-<img src="pics/daikin6.png" alt="06" style="zoom:100%;" />
+<img src="pics/daikin6.png" alt="daikin6" style="zoom:100%;" />
 
 The **Packet Filter** page lists all packet types observed on the P1/P2 bus. Some of them are exchanged between the heat pump and the main Daikin controller, others are exchanged between our controller (or other external controllers) and the main Daikin controller. If you do not see any packet types, wait few seconds. If a new packet type is detected, it will be automatically added to the list. **Packet types enabled on this page are forwarded via UDP**. By default, only Counter Packet (0xB8) and Data Packets (usually 0x10 - 0x16) are sent via UDP. Enable additional packet types if you want to test or reverse-engineer the P1/P2 protocol.
 

arduino-altherma-controller/01-interfaces.ino

@@ -1,5 +1,5 @@
 /* *******************************************************************
-   Ethernet and serial interface functions
+   Ethernet interface functions
 
    startEthernet()
    - initiates ethernet interface
@@ -18,15 +18,15 @@
    maintainCounters(), rollover()
    - synchronizes roll-over of data counters to zero
 
-   resetStats()
-   - resets Modbus stats
+   resetStats(), resetEepromStats()
+   - resets P1P2 stats and Daikin EEPROM stats
 
    generateMac()
    - generate random MAC using pseudo random generator (faster and than build-in random())
 
    manageSockets()
    - closes sockets which are waiting to be closed or which refuse to close
-   - forwards sockets with data available (webserver or Modbus TCP) for further processing
+   - forwards sockets with data available for further processing by the webserver
    - disconnects (closes) sockets which are too old / idle for too long
    - opens new sockets if needed (and if available)
 
@@ -47,11 +47,7 @@ void startEthernet() {
     delay(25);
     digitalWrite(ETH_RESET_PIN, HIGH);
     delay(ETH_RESET_DELAY);
-    pinMode(ETH_RESET_PIN, INPUT);
   }
-  byte mac[6];
-  memcpy(mac, MAC_START, 3);               // set first 3 bytes
-  memcpy(mac + 3, localConfig.macEnd, 3);  // set last 3 bytes
 #ifdef ENABLE_DHCP
   if (localConfig.enableDhcp) {
     dhcpSuccess = Ethernet.begin(mac);
@@ -72,10 +68,12 @@ void startEthernet() {
 #endif
 }
 
+void (*resetFunc)(void) = 0;  //declare reset function at address 0
+
 #ifdef ENABLE_DHCP
 void maintainDhcp() {
   if (localConfig.enableDhcp && dhcpSuccess == true) {  // only call maintain if initial DHCP request by startEthernet was successfull
-    uint8_t maintainResult = Ethernet.maintain();
+    byte maintainResult = Ethernet.maintain();
     if (maintainResult == 1 || maintainResult == 3) {  // renew failed or rebind failed
       dhcpSuccess = false;
       startEthernet();  // another DHCP request, fallback to static IP
@@ -86,7 +84,7 @@ void maintainDhcp() {
 
 #ifdef ENABLE_EXTRA_DIAG
 void maintainUptime() {
-  unsigned long milliseconds = millis();
+  uint32_t milliseconds = millis();
   if (last_milliseconds > milliseconds) {
     //in case of millis() overflow, store existing passed seconds
     remaining_seconds = seconds;
@@ -100,7 +98,7 @@ void maintainUptime() {
 }
 #endif /* ENABLE_EXTRA_DIAG */
 
-const unsigned long ROLLOVER = 0xFFFFFF00;
+const uint32_t ROLLOVER = 0xFFFFFF00;
 bool rollover() {
   // synchronize roll-over of run time, data counters and modbus stats to zero, at 0xFFFFFF00
   for (byte i = 0; i < P1P2_LAST; i++) {
@@ -143,20 +141,23 @@ void generateMac() {
   seed1 = 36969L * (seed1 & 65535L) + (seed1 >> 16);
   seed2 = 18000L * (seed2 & 65535L) + (seed2 >> 16);
   uint32_t randomBuffer = (seed1 << 16) + seed2; /* 32-bit random */
+  memcpy(mac, MAC_START, 3);                     // set first 3 bytes
   for (byte i = 0; i < 3; i++) {
-    localConfig.macEnd[i] = randomBuffer & 0xFF;
+    mac[i + 3] = randomBuffer & 0xFF;  // random last 3 bytes
     randomBuffer >>= 8;
   }
 }
 
 void updateEeprom() {
   eepromTimer.sleep(EEPROM_INTERVAL * 60UL * 60UL * 1000UL);  // EEPROM_INTERVAL is in hours, sleep is in milliseconds!
   eepromCount.eepromWrites++;                                 // we assume that at least some bytes are written to EEPROM during EEPROM.update or EEPROM.put
-  int address = CONFIG_START;
+  byte address = CONFIG_START;
   EEPROM.put(address, eepromCount);
   address += sizeof(eepromCount);
   EEPROM.put(address, VERSION[0]);
   address += 1;
+  EEPROM.put(address, mac);
+  address += 6;
   EEPROM.put(address, localConfig);
   address += sizeof(localConfig);
   EEPROM.put(address, p1p2Count);
@@ -167,9 +168,9 @@ void updateEeprom() {
 }
 
 #if MAX_SOCK_NUM == 8
-unsigned long lastSocketUse[MAX_SOCK_NUM] = { 0, 0, 0, 0, 0, 0, 0, 0 };  // +rs 03Feb2019 - records last interaction involving each socket to enable detecting sockets unused for longest time period
+uint32_t lastSocketUse[MAX_SOCK_NUM] = { 0, 0, 0, 0, 0, 0, 0, 0 };
 #elif MAX_SOCK_NUM == 4
-unsigned long lastSocketUse[MAX_SOCK_NUM] = { 0, 0, 0, 0 };                          // +rs 03Feb2019 - records last interaction involving each socket to enable detecting sockets unused for longest time period
+uint32_t lastSocketUse[MAX_SOCK_NUM] = { 0, 0, 0, 0 };
 #endif
 
 // from https://github.com/SapientHetero/Ethernet/blob/master/src/socket.cpp
@@ -302,12 +303,14 @@ void manageController() {
         cmdQueue.push(0);
         indoorInQueue = true;
       }
+#ifdef ENABLE_EXTRA_DIAG
       if (daikinOutdoor[0] == '\0' && outdoorInQueue == false) {
         cmdQueue.push(2);
         cmdQueue.push(PACKET_TYPE_OUTDOOR_NAME);
         cmdQueue.push(0);
         outdoorInQueue = true;
       }
+#endif /* ENABLE_EXTRA_DIAG */
       break;
     default:
       break;

arduino-altherma-controller/02-UDP.ino

@@ -1,14 +1,14 @@
 /* *******************************************************************
-   Modbus TCP/UDP functions
+   UDP functions
 
    recvUdp()
    - receives P1P2 command via UDP
-   - calls checkRequest
+   - calls checkCommand
 
    checkCommand()
-   - checks P1P2 command (correct MBAP header, CRC in case of Modbus RTU over TCP/UDP)
+   - checks P1P2 command
    - checks availability of queue
-   - stores requests into queue or returns an error
+   - stores commands into queue or returns an error
 
    deleteCmd()
    - deletes command from queue
@@ -21,39 +21,42 @@
 
    ***************************************************************** */
 
-uint8_t masks[8] = { 1, 2, 4, 8, 16, 32, 64, 128 };
+byte masks[8] = { 1, 2, 4, 8, 16, 32, 64, 128 };
 
 void recvUdp() {
-  unsigned int udpLen = Udp.parsePacket();
+  uint16_t udpLen = Udp.parsePacket();
   if (udpLen) {
     byte command[1 + 2 + MAX_PARAM_SIZE];  // 1 byte packet type + 2 bytes param number + MAX_PARAM_SIZE bytes param value
     if (udpLen > sizeof(command) || (!localConfig.udpBroadcast && Udp.remoteIP() != IPAddress(localConfig.remoteIp))) {
       while (Udp.available()) Udp.read();
-      // TODO error
+      // TODO error: UDP too long or wrong remote IP
       return;
     }
     Udp.read(command, sizeof(command));
     checkCommand(command, byte(udpLen));
+#ifdef ENABLE_EXTRA_DIAG
+    udpCount[UDP_RECEIVED]++;
+#endif /* ENABLE_EXTRA_DIAG */
   }
 }
 
 void checkCommand(byte command[], byte cmdLen) {
-  if (cmdQueue.available() > cmdLen                                                             // check available space in queue
-      && PACKET_PARAM_VAL_SIZE[command[0] - PACKET_TYPE_CONTROL[FIRST]] != 0                    // check if param size is not zero (0 = write command not supported (yet))
-      && cmdLen - 3 == (PACKET_PARAM_VAL_SIZE[command[0] - PACKET_TYPE_CONTROL[FIRST]])         // check parameter value size
-      && (command[0] >= PACKET_TYPE_CONTROL[FIRST] && command[0] <= PACKET_TYPE_CONTROL[LAST])  // check packet type
-      && changed36Param(command) == true) {
-#ifdef ENABLE_EXTRA_DIAG
-    udpCount[UDP_RECEIVED]++;
-#endif /* ENABLE_EXTRA_DIAG */  // check hysteresis for packet type 0x36
-    // push to queue (incl. cmdLen)
-    cmdQueue.push(cmdLen);  // first byte in queue is cmdLen
-    for (byte i = 0; i < cmdLen; i++) {
-      cmdQueue.push(command[i]);
-      Serial.println(hex(command[i]));
+  if (cmdQueue.available() > cmdLen) {                                                               // check available space in queue
+    if (PACKET_PARAM_VAL_SIZE[command[0] - PACKET_TYPE_CONTROL[FIRST]] != 0                          // check if param size is not zero (0 = write command not supported (yet))
+        && cmdLen - 3 == (PACKET_PARAM_VAL_SIZE[command[0] - PACKET_TYPE_CONTROL[FIRST]])            // check parameter value size
+        && (command[0] >= PACKET_TYPE_CONTROL[FIRST] && command[0] <= PACKET_TYPE_CONTROL[LAST])) {  // check packet type
+      if (changed36Param(command) == true) {
+        // push to queue (incl. cmdLen)
+        cmdQueue.push(cmdLen);  // first byte in queue is cmdLen
+        for (byte i = 0; i < cmdLen; i++) {
+          cmdQueue.push(command[i]);
+        }
+      }
+    } else {
+      p1p2Count[P1P2_WRITE_INVALID]++;  // Write Command Invalid
     }
   } else {
-    // TODO error
+    p1p2Count[P1P2_WRITE_QUEUE]++;  // TODO error: Write Queue Full
   }
 }
 
@@ -65,11 +68,11 @@ void deleteCmd()  // delete command from queue
   }
 }
 
-bool getPacketStatus(const uint8_t packetType, const byte status) {
+bool getPacketStatus(const byte packetType, const byte status) {
   return (localConfig.packetStatus[status][packetType / 8] & masks[packetType & 7]) > 0;
 }
 
-bool setPacketStatus(const uint8_t packetType, byte status, const bool value) {
+bool setPacketStatus(const byte packetType, byte status, const bool value) {
   if (getPacketStatus(packetType, status) == value) return false;
   if (value == 0) {
     localConfig.packetStatus[status][packetType / 8] &= ~masks[packetType & 7];