{"id":10550,"date":"2019-06-04T23:45:26","date_gmt":"2019-06-04T21:45:26","guid":{"rendered":"https:\/\/pi3g.com\/?p=10550"},"modified":"2019-06-04T23:45:26","modified_gmt":"2019-06-04T21:45:26","slug":"debugging-vernemq-connection-authentication-problems-for-dual-authentication-with-webhooks-and-vmq_diversity","status":"publish","type":"post","link":"https:\/\/pi3g.com\/de\/debugging-vernemq-connection-authentication-problems-for-dual-authentication-with-webhooks-and-vmq_diversity\/","title":{"rendered":"Debugging von VerneMQ-Verbindungs-\/Authentifizierungsproblemen bei dualer Authentifizierung mit Webhooks und vmq_diversity"},"content":{"rendered":"

TL;DR<\/h1>\n

https:\/\/github.com\/vernemq\/vernemq\/blob\/master\/apps\/vmq_diversity\/src\/vmq_diversity_plugin.erl<\/a><\/p>\n

includes a statement which will block authentication \/ authorization through any other channels if your Lua script returns \u201cfalse\u201d:<\/p>\n

\"image\"<\/a><\/p>\n

will lead to:<\/p>\n

\"image\"<\/a><\/p>\n

instead of trying other plugins, as you might have assumed!<\/p>\n

In order to avoid this problem, simply do not return anything<\/strong> from your Lua script, if you do not want to handle the message!<\/p>\n

\"image\"<\/a><\/p>\n

Debugging<\/h1>\n

VerneMQ has a built-in trace tool, which will help you to see which problems can occur. First, display the sessions, in order to find out which data your clients are trying to connect with:<\/p>\n

\n

vmq-admin session show<\/p>\n<\/blockquote>\n

\"image\"<\/a><\/p>\n

Then run a trace:<\/p>\n

\n

vmq-admin trace client client-id=000000007eaefc47 –mountpoint=5c61a59ba6b76f558d256f42<\/p>\n<\/blockquote>\n

Note that the mountpoint defaults to \u201c\u201d (the default mountpoint for Verne) \u2013 if you are not setting a custom mountpoint on the listeners ( as me), then you can omit this.<\/p>\n

\"image\"<\/a><\/p>\n

Initially vmq-admin trace will report that no sessions are found for this client \u2013 if your client is not currently connected.<\/p>\n

Connect your client, and there will be more output.<\/p>\n

It is also helpful to increase the verbosity level in the logger to debug (for vernemq.conf):<\/p>\n

\n

log.console.level = debug<\/p>\n<\/blockquote>\n

\"image\"<\/a><\/p>\n

econnrefused<\/h1>\n

In my particular case, my webhook endpoint was not running. After starting it up, I now get (multiple attempts with similar output):<\/p>\n

\"image\"<\/a><\/p>\n

As you can see, the client is now being authenticated, but not being authorized to publish. The client is then disconnected (as per MQTT 3.1.1 spec).<\/p>\n

And this is the log output:<\/p>\n

\"image\"<\/a><\/p>\n

what is happening here, is that the client is authenticated using a webhook, and then it tries to become authorized using a Lua script ( MongoDB authorization). <\/p>\n

I am not sure why auth_on_publish for the client is not called using a webhook as well. This client in any case is supposed to be authenticated fully using MongoDB.<\/p>\n

I also only see requests for auth-on register:<\/p>\n

\"image\"<\/a><\/p>\n

To show which webhooks are registered, you can run<\/p>\n

\n

vmq-admin webhooks show<\/p>\n<\/blockquote>\n

\"image\"<\/a><\/p>\n

see: https:\/\/docs.vernemq.com\/plugindevelopment\/webhookplugins<\/a><\/p>\n

All webhooks are called with the method POST, and need to be answered with the HTTP code 200 to be considered successful. <\/p>\n

A request for the webhook server<\/h2>\n

Here is how a request will look like for the Webhook server:<\/p>\n

\"image\"<\/a><\/p>\n

headers are passed, with the header vernemq-hook set to auth_on_register<\/strong> in this case.<\/p>\n

The post body will include different information on the client trying to connect.<\/p>\n

After using \u201cnext\u201d as a reply from the webhook, the publish works correctly \u2013 the client is authenticated against the MongoDB, and afterwards the publish is authorized, as desired.<\/p>\n

Here\u2019s how the answer from the webhook should look like to pass authentication on to the next plugin:<\/p>\n

\n

{“result”: “next”}<\/p>\n<\/blockquote>\n

And here\u2019s how the trace looks now:<\/p>\n

\"image\"<\/a><\/p>\n

At the bottom you can see the keepalives for the client (PINGREQ, PINGRESP).<\/p>\n

So, one client is working now. Onwards to the next client \u2013 which should be authenticated and authorized using webhooks!<\/p>\n

Webhook client<\/h1>\n

\"image\"<\/a><\/p>\n

As you can see, by the way, the mountpoint gets rewritten from \u201cpiclient\u201d to a different mountpoint by the MongoDB Lua script.<\/p>\n

Basically I am using the mountpoint on the listeners to indicate which authentication method should be used for this client.<\/p>\n

In version 2 of VerneMQ, we are promised, there will be a separation of Plugins per Listener \u2013 therefore we can avoid these checks which add overhead to each connection.<\/p>\n

\"image\"<\/a><\/p>\n

apparently only one trace can be run at a time. Trying again after shutting down the other trace:<\/p>\n

\n

vmq-admin trace client client-id=my_clientid –mountpoint=jsclient<\/p>\n<\/blockquote>\n

\"image\"<\/a><\/p>\n

\"image\"<\/a><\/p>\n

\"image\"<\/a><\/p>\n

Please note that the message auth_on_publish debug is printed by an additional statement I inserted in the Lua script:<\/p>\n

\"image\"<\/a><\/p>\n

in auth_commons.lua<\/p>\n

Understanding what to return from the Lua script<\/h2>\n

What we want is probably to return from the Lua script that the next hook should be called.<\/p>\n

\u201cIn case the plugin can’t validate the publish message it is best to return next<\/code> as this would allow subsequent plugins in the chain to validate the request.\u201d<\/p>\n

(see https:\/\/docs.vernemq.com\/plugindevelopment\/publishflow<\/a> )<\/p>\n

I assumed that returning false<\/strong> would imply that, but apparently not?<\/p>\n

Let\u2019s look in the Erlang source of vernemq<\/p>\n

https:\/\/github.com\/vernemq\/vernemq<\/a><\/p>\n

This is a test suite for the auth_cache for vmq_diversity (where the Lua scripts live):<\/p>\n

https:\/\/github.com\/vernemq\/vernemq\/blob\/c8b92f398e76d6ce4b8cca5e438e8ae1e717d71c\/apps\/vmq_diversity\/test\/vmq_diversity_auth_cache_SUITE.erl<\/a><\/p>\n

\"image\"<\/a><\/p>\n

\n

vmq-admin trace client client-id=my_clientid –mountpoint=jsclient<\/p>\n<\/blockquote>\n

\"image\"<\/a><\/p>\n

Adding some additional debugging for the message in Lua:<\/p>\n

\n

function auth_on_publish(pub)
     print(“***auth_on_publish debug*** (LUA)”)
     for k,v in pairs(pub) do
         print(k .. ” ” .. v)
     end
     print(pub)
     return false
\nend<\/p>\n<\/blockquote>\n

\"image\"<\/a><\/p>\n

Note that the code seems to be faulty (false apparently can\u2019t be concatenated using .. in Lua \u2013 OK.)<\/p>\n

In any case, in the Lua table, which is passed in as \u201cpub\u201d, we have the following parameters:<\/p>\n