{"id":27450,"date":"2020-07-21T13:10:57","date_gmt":"2020-07-21T11:10:57","guid":{"rendered":"https:\/\/pi3g.com\/?p=27450"},"modified":"2020-07-21T13:10:57","modified_gmt":"2020-07-21T11:10:57","slug":"switching-from-json-to-msgpack-in-a-backwards-compatible-manner-with-paho-mqtt-javascript","status":"publish","type":"post","link":"https:\/\/pi3g.com\/de\/switching-from-json-to-msgpack-in-a-backwards-compatible-manner-with-paho-mqtt-javascript\/","title":{"rendered":"Umstellung von JSON auf MsgPack in einer r\u00fcckw\u00e4rtskompatiblen Weise mit Paho MQTT (JavaScript)"},"content":{"rendered":"

picockpit.com<\/a> makes heavy use of MQTT. For the upcoming version 2.0 I have decided to switch it to MsgPack.<\/p>\n

Why MsgPack?<\/h1>\n

MsgPack (actually called MessagePack<\/a> \u2013 packages tend to use MsgPack) is a binary format, aimed at compatibility with JSON, but decreasing file size (which is good for saving bandwidth & server resources).<\/p>\n

Another advantage: MessagePack allows you to embed binary data <\/strong>(there\u2019s some exciting use cases planned for PiCockpit v2.1+). <\/p>\n

Compared to other Binary serialization formats (e.g. BSON) MsgPack aims for more compatibility with JSON, is more compact, and can be decoded as a stream<\/strong>!<\/p>\n

Retained Messages from the past<\/h1>\n

Due to retained messages still being in JSON (and also legacy clients still sending JSON messages), however, backwards compatibility needs to be provided, especially on the JavaScript side.<\/p>\n

Failing to provide backwards compatibility will lead to the following problems with Paho:<\/p>\n

onConnectFailure;MQTT connection failed (5): AMQJS0005E Internal error. Error Message: AMQJS0009E Malformed UTF data:ad -4f ., Stack trace: No Error Stack Available<\/b><\/p>\n

onConnectFailure;MQTT connection failed (5): AMQJS0005E Internal error. Error Message: Extra 231 byte(s) found at buffer[1], Stack trace: No Error Stack Available<\/b><\/p>\n

Coding for fallback to JSON<\/h1>\n

Paho does not give us access to the message data itself it has in it\u2019s structures (see Paho code here<\/a>).<\/p>\n

Therefore we can fall back to exception catching, assuming that the message will most likely be in the MsgPack format, and if decoding that fails, will switch to JSON. <\/p>\n

And if decoding JSON fails, we provide a {} fallback against fuzzing:<\/p>\n

\n

extractMessage(message){
     \/\/ 20.7.2020: switch to msgPack; has to be payloadBytes, not payloadString
     var decodedMessage = {};
     try {
         decodedMessage = decode(message.payloadBytes);
     }
     catch (e){
         console.log(“decode error caught “);
         console.log(e);
         try {
             decodedMessage = JSON.parse(message.payloadString);
         }
         catch (e){
             console.log(“malformed message (neither JSON nor MsgPack) – interpreting as empty object {}”);
         }
     }
     return decodedMessage;
     \/\/ return JSON.parse(message.payloadString);
\n}<\/p>\n<\/blockquote>\n

\n\"image\"<\/a><\/p>\n