Saturday, April 27, 2013

Enhanced Virtual Wire Interface

This post presents the final design of the Cosa Virtual Wire Interface. It now includes many of the features discussed in the previous posts. The main enhancement is reliable wireless communication on cheap RF433 transmitter and receiver modules with node addressing, message filtering, auto-retransmission and support for message dispatch. As before this Cosa sub-system may be used with ATtinyX5 to create ultra tiny and low power sensor devices.

The first section of the post describes the modifications to the Cosa VWI Receiver and Transmitter classes, adding addressing and message tags. The second section describes the new VWI::Transceiver class which provides message acknowledgement with auto-retransmission and reduces much of the complexity of designing and implementing wireless message protocols.

To achieve this the original Virtual Wire Interface (VWI) has been extended. A new overloaded send() method is introduced to allow gathering of multiple buffers (iovec_t). This makes implementation of communication protocols much easier and reduces message memory buffer copying.

To improve performance of retransmission a new resend() method is also introduced. This method will simple retransmit the message already encoded in the transmission buffer. There are also access methods for the next message sequence counter as this will be needed later for retransmission. Below is the updated VWI::Transmitter API.

 Fig.1: Enhanced VWI Transmitter Member Functions

The enhanced mode is enabled by providing a 32-bit node address when calling VWI::begin(). Below is the setup() from the example sketch CosaVWItempsensor.ino to give an idea how this works. 

// Connect RF433 transmitter to Arduino/D9
VirtualWireCodec codec;
VWI::Transmitter tx(Board::D9, &codec);

const uint16_t SPEED = 4000;
const uint32_t ADDR = 0xC05a0002;

...
void setup()
{
  ...
  // Start the Virtual Wire Interface/Transmitter
  VWI::begin(ADDR, SPEED);
  tx.begin();
  ...
}


The blue section is the only change needed to enable addressing from the sensor node, i.e. the new address parameter. The loop() of the temperature sensor will read two 1-Wire Thermometers and send the values together with a reading of the power supply voltage. The message is defined as a struct (sample_t) and given a message type tag, SAMPLE_CMD. This is used as an extra parameter to the send() method and provides the receiver with information about what type of message the payload contains. See the blue sections below.

// Message from the device; temperature and voltage reading
const uint8_t SAMPLE_CMD = 42;
struct sample_t {
  int16_t temperature[2];
  uint16_t voltage;
};

...
void loop()
{

  ...  // Initiate the message with measurements
  sample_t msg;  
  msg.temperature[0] = indoors.get_temperature();
  msg.temperature[1] = outdoors.get_temperature();
  msg.voltage = AnalogPin::bandgap(1100);
  // Enable wireless transmitter and send. Wait completion and disable
  VWI::enable();
  tx.send(&msg, sizeof(msg), SAMPLE_CMD);
  tx.await();
  VWI::disable();
  ...
}


The VWI::Receiver class is also enhanced with filtering of incoming messages. The API is not changed, i.e. the recv() method has the same prototype. Incoming messages are matched against the receiver node address using a simple sub-net mask. The mask is provided as an extra parameter to the begin() method for the receiver. The logic of the address filtering is:

(incoming-message-address & receive-node-sub-net-mask) == receive-node-address

This implies that receiving nodes only listens for messages from transmitters with the same sub-net address as the receiver. Below is the setup() snippet from the CosaVWItempmonitor.ino example sketch:

// Virtual Wire Interface Receiver connected to pin D8
VirtualWireCodec codec;
VWI::Receiver rx(Board::D8, &codec);

const uint16_t SPEED = 4000;
const uint32_t ADDR = 0xc05a0000UL;
const uint32_t MASK = 0xffffff00UL;

...
void setup()
{
  ...
  // Start virtual wire interface and receiver. Use eight bit sub-net mask
  // Transmitters must have the same 24 MSB address bits as the receiver.
  VWI::begin(ADDR, SPEED);
  rx.begin(MASK);
}


In the above snippet the sub-net MASK will allow for 255 transmitting sensor reporting to the monitoring node. The first sub-net address (0) is the receiver itself. With the <24:8> bit addressing scheme in the example sketch there could be 2**24 monitor nodes if in a flat structure. Additional sub-nets are possible, if for instance <16:8:8>, the monitoring node would also report to a sub-net with yet another layer of nodes. And parts of the address could be used to sensor type tagging.

The VWI::Receiver::recv() method will return the full message with the enhanced mode header and the application payload to the caller. Further filtering and dispatch of message will require access to the enhanced VWI header. Defining a struct (msg_t) with the header and a union of the application message data types, as in the snippet below, is a convenient way to describe the incoming messages. Also it is practical to collect all messages in a separate header file to keep the code base consistent. Below is the loop() of the example monitor with an example of message filtering based on the message type (command/cmd). 

// Message received from VWI in extended mode
struct msg_t {
  VWI::header_t header;
  union {
    sample_t sample;

    // Add additional message types here
  };
};

...
void loop()
{
  // Receive a message. Sanity check the message size
  msg_t msg;
  int8_t len = rx.recv(&msg, sizeof(msg));
  if (len <= 0) return;
  // Print message header; transmitter address and sequence number
  trace << hex << msg.header.addr << ':' << msg.header.nr << ':';
  // Check message type and print contents
  if (msg.header.cmd == SAMPLE_CMD) {
  ...
  }
  else {
    trace << msg.header.cmd << PSTR(":unknown message type") << endl;
  }
}


VWI::header_t contains the transmitter node address, message type tag and sequence number. The node address is provided as the parameter to VWI::begin(), the message type tag (cmd) as the additional parameter to VWI::Transmitter::send() and the sequence number is maintained internally to the Transmitter. Below is the implementation:

class VWI {
public:
  ...
  /** Message header for enhanced Virtual Wire Interface mode */
  struct header_t {
    uint32_t addr;         /**< Transmitter node address */
    uint8_t cmd;           /**< Command or message type */
    uint8_t nr;            /**< Message sequence number */
  };
  ...

};

So far the modifications are to the original Virtual Wire Interface and have provided addressing and message types. In the following section the new VWI::Transceiver class is presented. This class adds reliable communication with message acknowledgement and auto-retransmission.


Fig.2: VWI::Transceiver Public Member Functions and Attributes

The VWI::Transceiver interface is now reduced to basically send() and recv(). The Transceiver contains both a Transmitter and Receiver instance (rx and tx attributes above). The send() method will transmit an enhanced mode message and wait for an acknowledgement. The acknowledgement is transmitted by the receiving node. If the message or acknowledgement is lost an automatic retransmission will occur. The sender node will wait a TIMEOUT period before retransmission and perform RETRANS_MAX retransmissions. The send() method will return the number of transmissions or a negative error code(-1) if the maximum number of retransmissions was exceeded.

Note that the VWI::Transmitter::send() method will return directly after placing the message into the transmission buffer. This is not the case with VWI::Transceiver::send(). This method will wait until it receives an acknowledgement with possible retransmission.

Below is a snippet of the CosaVWIclient.ino example sketch. This sketch demonstrates how to use the VWI::Transceiver and multiple message types. The first message type (sample_t/SAMPLE_CMD) contains two analog readings and is sent every second from the wireless sensor device. The second message type (stat_t/STAT_CMD) contains power supply reading and transceiver statistics. This message is sent every 15th second.

// Network configuration
const uint32_t ADDR = 0xc05a0001UL;
const uint16_t SPEED = 4000;
...
// Virtual Wire Interface Transceiver
VWI::Transceiver trx(Board::D8, Board::D9, &codec);
...
// Analog pins to sample for values to send
AnalogPin luminance(Board::A2);
AnalogPin temperature(Board::A3);
...
// Message type to send (should be in an include file for client and server)
const uint8_t SAMPLE_CMD = 1;
struct sample_t {
  uint16_t luminance;
  uint16_t temperature;
  sample_t(uint16_t lum, uint16_t temp)
  {
    luminance = lum;
    temperature = temp;
  }
};
...
const uint8_t STAT_CMD = 2;
struct stat_t {
  uint16_t voltage;
  uint16_t sent;
  uint16_t resent;
  uint16_t received;
  uint16_t failed;
  void update(int8_t nr)
  {
    sent += 1;
    if (nr <= 0)
      failed += 1;
    else if (nr > 1)
      resent += (nr - 1);
  }
};
...
// Statistics
stat_t stat;
... 
void setup()
{
  // Start watchdog for delay
  Watchdog::begin();
  RTC::begin();
  ...
  // Start virtual wire interface in extended mode; transceiver
  VWI::begin(ADDR, SPEED);
  trx.begin();
}
...
void loop()
{
  // Send message with luminance and temperature
  sample_t sample(luminance.sample(), temperature.sample());
  int8_t nr = trx.send(&sample, sizeof(sample), SAMPLE_CMD);
  stat.update(nr);
  // Send message with battery voltage and statistics every 15 messages
  if (stat.sent % 15 == 0) {
    stat.voltage = AnalogPin::bandgap(1100);
    nr = trx.send(&stat, sizeof(stat), STAT_CMD);
    stat.update(nr);
  }

  // Take a nap
  SLEEP(1);
}


A server will receive the messages from the client (above), filter, acknowledge and print the messages. Note that the receiving node must filter received messages so that multiple messages with the same sequence number are only handled once (if necessary). Below is a snippet from the CosaVWIserver.ino example sketch:

// Virtual Wire Interface Transceiver
VWI::Transceiver trx(Board::D8, Board::D9, &codec);
...
// Network configuration
const uint32_t ADDR = 0xc05a0000UL;
const uint32_t MASK = 0xffffff00UL;
const uint16_t SPEED = 4000;
..
void setup()
{
  ...
  // Start virtual wire interface transceiver
  VWI::begin(ADDR, SPEED);
  trx.begin(MASK);
}
...
// Message types
const uint8_t SAMPLE_CMD = 1;
struct sample_t {
  uint16_t luminance;
  uint16_t temperature;
};
const uint8_t STAT_CMD = 2;
struct stat_t {
  uint16_t voltage;
  uint16_t sent;
  uint16_t resent;
  uint16_t received;
  uint16_t failed;
};
...
// Extended mode message with header
struct msg_t {
  VWI::header_t header;
  union {
    sample_t sample;
    stat_t stat;
  };
};

...
void loop()
{
  // Processed sequence number (should be one per client)
  static uint8_t nr = 0xff;
  // Wait for a message. Sanity check the length
  msg_t msg;
  int8_t len = trx.recv(&msg, sizeof(msg));
  if (len <= 0) return; 
  // Check that this is not a retransmission

  // Should be one counter (nr) per connection
  if (nr == msg.header.nr) return;
  nr = msg.header.nr;
  // Print header, type message type and print contents
  trace << hex << msg.header.addr << ':' << msg.header.nr << ':';
  switch (msg.header.cmd) {
  case SAMPLE_CMD:
    ...
    break;
  case STAT_CMD:
    ...
    break;
  }


The next development of Cosa VWI is to increase performance and adapt to the event driven state-machine framework. One goal is to abstracting the interface toward wireless connection over Ethernet, WiFi, RF433, NFR24L01P, etc, so that applications are written without concern to the wireless device. RF433 is an interesting challenge as the protocol stack must be built from the ground up. 

[Update 2013-11-17]
This extension of the Virtual Wire interface in Cosa has been deprecated and replaced by an abstract Wireless interface. This interface is implemented by RF433 modules (VWI), CC1101 and NRF24L01P. The above extended VWI interface is still possible to implement on top of the new Wireless interface. This will be added in later releases of Cosa. Please see the Cosa documentation for further details. 

23 comments:

  1. Hello,
    Love your work on the VirtualWire library !

    Does the enhanced version can be used without all the Cosa package ?

    It does not look that way because receiver and transmitter inherit from inputpin and outputpin which are including a lot of Cosa files !

    I am currently working on an Arduino-like board, Sparkfun pro micro (ATmega32U4), What would be the effort to port this board to Cosa ?

    Thanks,
    Regards,

    Nicolas BOUTIN

    ReplyDelete
    Replies
    1. Hi Nicolas! Good news! Cosa will now support boards based on ATmega32U4. Have fun! Mikael

      Delete
  2. Hi thanks for your interest in this project.

    I am pleased to learn that you liked the refactoring and enhancement of the VirtualWire library. There is an Arduino version of the refactoring to C++. Unfortunately I have not had time to port the enhanced version back to Arduino. If there is an interest of this I will put more attention on it. So far the feedback is limited.

    I have plans to include support for USB and ATmega32U4. Right now the focus is ATtiny and using them as slave units in clusters of processors and more advanced shields.

    It would be possible to do an intermediate port of Cosa to ATmega32U4 and reuse the Arduino USB suport without implementing a Cosa version (as I plan).

    The effort is a few hours study of the spec and a programming a few hours followed by testing. I did the Mighty port in a few hours. But the ATmega32U4 will require a bit more work.

    Thanks again, Mikael

    ReplyDelete
  3. Hi,

    thanks for the quick answer.

    Ok, for the moment I will focus my effort on using the Cosa library on an Arduino Uno R3.

    Keep that way.
    Regards

    ReplyDelete
  4. Awesome work on this project! My one problem (so far) is that I am using some HD44780 4-bit parallel LCDs but the Arduino data pins are fixed to use D11-D13. Can you create another Port4b constructor that takes more arguments for the data pins? Or let me know where I can modify the code to allow for different data pins.

    Thanks!
    John

    ReplyDelete
  5. Thanks for the feedback. The current Port4b is very much an optimized version for a LCD Shield with 6 buttons. There is no real problem adapting or writing a new port adapter for another pin range. The implementation assumes that the pins are in the same PORT.

    BW: D11-D13 is only 3 pins. Guess it is D10-D13 that your LCDs use?

    Cheers!

    ReplyDelete
    Replies
    1. Thanks for the quick reply!

      I see in HD44780_IO_Port4b.cpp the code for setting up the LCD, but I am not familiar with PORT coding, so I don't know how to change it to use other pins: pin 9 for lcd backlight, 10-13 for lcd data. Can you give some info on how to do that?

      (FYI, I am a Java developer during the day, and I have used C/C++ for many years but nothing embedded like this. I also am using Eclipse as my IDE, so if you have any suggestions about Eclipse I would also appreciate it).

      Thanks,
      John

      Delete
    2. Forgot to mention that I want to use analog pins for RS and Enable!

      Delete
  6. Hi John.

    Unfortunately that configuration is not supported "out-of-the-box". To do that you need to adjust the DDR and PORT to DDRB and PORTB. This is actually the setting for Mega and Mighty. Now the tricky part is that your LCD configuration is not on four bit alignment as you are using D10-D13. This implies that all the settings must also be adjusted. For instance setup() will have to initiate D10-D13 as output pins. That is DDR |= 0b00111100.

    Then there is the adjustment of data going to the LCD in member functions write4b() and write8b(). The data will need to be shifted and masked. PORT = (((data & 0x0f) << 2) | (PORT & 0b11000011)) for the LSB and PORT = (((data >> 4) & 0x0f) << 2) etc for the MSB. Here I am assuming that LCD:D4 is connected to Arduino:D10, D5 to D11 and so on. If not the bits will need to be reversed.

    Using analog pins for RS and EN is no problem but you will have to change or introduce a new constructor that takes Board::AnalogPin instead of Board::DigitalPin. You could also cast the binding for the LCD port instance but that is not very clean. See the Pins definition on how to do that.

    I count 2+6 lines to change ;-)

    Cheers!

    ReplyDelete
    Replies
    1. Hi John.

      Please find an update to the LCD four bit parallel port driver on github. You find the new version much easier to adapt to your needs.

      Cheers!

      Delete
  7. Thanks for the update. I have been experimenting with your library and it is awesome! Love that it is true C++. Unfortunately, due to limited time and experience with ATMega, I have not been able to modify the LCD module to use any Arduino pins. I will keep at it though.

    ReplyDelete
  8. Hi Mikael,

    I finally got enough time in front of your code and was able to do what I wanted to do.

    I subclassed your Port4b class and changed POS = 2 and the WRITE4B macro to:
    #define WRITE4B(data) PORT = ((( ((data & 0x08) >> 3) | ((data & 0x04) >> 1) | ((data & 0x02) << 1) | ((data & 0x01) << 3)) << POS) | (PORT & ~MASK))

    This allowed me to run the LCD D4-7 from Arduino D13-10.

    However, I also had to change the original Port4b class. I had to change the private section to protected in order to call "m_en.toggle()". I was hoping you could change that in your code because I don't want to have a separate fork. Or you could provide getter/setter methods for your private data members.

    Also, let me say again that I love developing with your library! The client code is clean and understandable!

    Thanks,
    John

    ReplyDelete
    Replies
    1. Hi John. Thanks for the suggestion. I have push an update with the change from private to protected. Great to see that you was able to change the Port4b adapter for your project.

      Cheers, Mikael

      Delete
  9. Hi Mikael, you mentioned an update to the Cosa will be available soon.
    could you please tell when this version will be available ? alternately, could you tell from where can I download the current/old version


    Regards,
    Udi

    ReplyDelete
    Replies
    1. Hi Udi. Thanks for your interest in this project. Please see the install post how to get started. http://cosa-arduino.blogspot.se/2013/02/installing-cosa.html Regards, Mikael

      Delete
  10. In cosa framework. Is it possible to use analog pin A0 as an output pin? The normal arduino allows this using:

    pinMode(A0, OUTPUT);
    digitalWrite(A0, HIGH);

    The problem is that the 3d printer controller RAMPS 1.4 has stepper motor controller connected to A0.

    Best Regards!

    ReplyDelete
  11. Yes that is possible. The pin is called Board::D14 (a digital pin) on a standard Arduino board and can be used as any other digital pin on Cosa.

    OutputPin cntrl_pin(Board::D14);
    cntrl_pin.set();

    Cheers!

    Please see the Cosa Boards definition files. Below is a link to Arduino Uno: https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/Board/Arduino/Uno.hh#L44

    ReplyDelete
    Replies
    1. Thanks for the quick response Mikael. I did not see that pin because I was looking at the mega.hh board which does not come with a pinout diagram. But it is good to know that I cant use it as an output pin and I can search for the rest of the mega2560 pins in this link: http://arduino.cc/en/Hacking/PinMapping2560

      Oh and by the way, your code is great, I love the class herarchies, the way the code is written, the way you do threading etc. You should write a book, this is by far the best OO microcontroller code I have seen. I even like it more than the code that appears in the book "Real-Time C++: Efficient Object-Oriented and Template Microcontroller Programming".

      Delete
    2. Hmm no still no luck. The A0 pin on arduino mega is physically connected to pin 97 (PF0/ADC0). I have not been able to use it as an output pin, can't find anything on file "../mega.hh"

      Delete
  12. Looks like that for the Mega. Might have considered that there are enough digital pins :). When I get the time I will add the pins in port F as digital pins. Cheers!

    ReplyDelete
  13. Wow thanks Mikael! The X motor on the RAMPS 1.4 board is moving! I tried to do the change myself but I could not find any relation between the pins in the diagram and the ones on file mega.hh. For example the led. The led is defined as:

    LED = D13

    And:

    D13 = 23, // PB7

    But the pin in the schematic (http://arduino.cc/en/uploads/Hacking/PinMap2560big.png) is pin 26! So I'm wondering where that 23 comes from?

    Regards!

    ReplyDelete
  14. Great. Small steps forward.

    The D13 = 23 is the value for bit 7 in PORT B (PB7). The Cosa Mega board description uses a mapping where the lower tre bits is the pin/bit number and the higher bits are the port according to the mapping functions SFR() and BIT() in the Board definition file; https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/Board/Arduino/Mega.hh

    Values 16 to 23 are mapped to PINB: PB0..PB7. It all a lot of mappings to allow symbolic definition of the pins.

    To confuse you further the pin 26 in the schematic you linked is the physical pin on the package. And that is a totally different mapping that the software knows nothing about :)

    Cheers!

    ReplyDelete