picockpit.com makes heavy use of MQTT. For the upcoming version 2.0 I have decided to switch it to MsgPack.

Why MsgPack?

MsgPack (actually called MessagePack – packages tend to use MsgPack) is a binary format, aimed at compatibility with JSON, but decreasing file size (which is good for saving bandwidth & server resources).

Another advantage: MessagePack allows you to embed binary data (there’s some exciting use cases planned for PiCockpit v2.1+).

Compared to other Binary serialization formats (e.g. BSON) MsgPack aims for more compatibility with JSON, is more compact, and can be decoded as a stream!

Retained Messages from the past

Due to retained messages still being in JSON (and also legacy clients still sending JSON messages), however, backwards compatibility needs to be provided, especially on the JavaScript side.

Failing to provide backwards compatibility will lead to the following problems with Paho:

onConnectFailure;MQTT connection failed (5): AMQJS0005E Internal error. Error Message: AMQJS0009E Malformed UTF data:ad -4f ., Stack trace: No Error Stack Available

onConnectFailure;MQTT connection failed (5): AMQJS0005E Internal error. Error Message: Extra 231 byte(s) found at buffer[1], Stack trace: No Error Stack Available

Coding for fallback to JSON

Paho does not give us access to the message data itself it has in it’s structures (see Paho code here).

Therefore we can fall back to exception catching, assuming that the message will most likely be in the MsgPack format, and if decoding that fails, will switch to JSON.

And if decoding JSON fails, we provide a {} fallback against fuzzing:

extractMessage(message){
     // 20.7.2020: switch to msgPack; has to be payloadBytes, not payloadString
     var decodedMessage = {};
     try {
         decodedMessage = decode(message.payloadBytes);
     }
     catch (e){
         console.log(“decode error caught “);
         console.log(e);
         try {
             decodedMessage = JSON.parse(message.payloadString);
         }
         catch (e){
             console.log(“malformed message (neither JSON nor MsgPack) – interpreting as empty object {}”);
         }
     }
     return decodedMessage;
     // return JSON.parse(message.payloadString);
}

• note that we use payloadBytes to access the message for msgPack, and payloadString for JSON decoding
• note that decode (and encode) are included like this: import { encode, decode } from “@msgpack/msgpack”; in my Webpack Workflow

References

• MessagePack (msgpack.org)
• msgpack JavaScript library (use this one instead of the one linked on msgpack.org, as it has more recent activity)
• Paho MQTT JS Source Code
• Paho MQTT JS Documentation for the Class Message