{"id":7312,"date":"2019-01-31T19:00:12","date_gmt":"2019-01-31T18:00:12","guid":{"rendered":"https:\/\/pi3g.com\/?p=7312"},"modified":"2019-02-02T20:46:57","modified_gmt":"2019-02-02T19:46:57","slug":"using-nuitka-compiler-for-python3-on-alpine-linux-armhf-musl","status":"publish","type":"post","link":"https:\/\/pi3g.com\/de\/using-nuitka-compiler-for-python3-on-alpine-linux-armhf-musl\/","title":{"rendered":"Verwendung des nuitka-Compilers f\u00fcr python3 auf Alpine Linux ARMHF (musl)"},"content":{"rendered":"

Motivation<\/h1>\n

pidoctor<\/a> is written in Python (as there was no easy way to get Crystal to work on ARMHF \/ musl). <\/p>\n

This means a dependency on Python \u2013 which adds overhead. <\/p>\n

I suspect that this overhead is the reason that pidoctor will not run on 256 MB Raspberry Pi\u2019s \u2013 because the RAM is exhausted by all the packages which are required.<\/p>\n

Furthermore, it is desirable to have the distribution be as small a download as possible! This means Python has to be removed from the equation somehow, the package should produce a binary (as would be the way with Crystal).<\/p>\n

Luckily, there is nuitka.<\/p>\n

And \u2013 lucky you \u2013 I have worked out some kinks of using nuitka on Alpine Linux, ARMHF (maybe it will work for x86 too).<\/p>\n

Nuitka<\/h1>\n

nuitka<\/a> is a fantastic piece of software, which will take a Python script and compile it to an executable, thereby speeding it up dramatically. <\/p>\n

nuitka does all the heavy lifting, you don\u2019t even need to adjust anything in your source code (methinks). You can run it like this:<\/p>\n

\n

python3 -m nuitka –follow-imports –standalone pidoctor.py<\/p>\n<\/blockquote>\n

Note the –standalone flag, which will compile the necessary shared libraries into your <projectname>.dist folder \u2013 thereby the installation will be able to run without any dependency <\/strong>on Python!<\/p>\n

Please note: WordPress unfortunately fucks up the double dashes and makes them into single dashes. the \u2013m has a single dash, the \u2013follow-imports and the \u2013standalone have double dashes. <\/p>\n

Installation of Nuitka on Alpine Linux (specifically arhmf, might work on other targets)<\/h2>\n
\n

apk add python3-dev<\/p>\n

apk add chrpath<\/p>\n

pip3 install \u2013U nuitka<\/p>\n<\/blockquote>\n

optionally, if you are running in diskless mode (to add nuitka to the files which should be persisted):<\/p>\n

\n

lbu add \/usr\/bin\/nuitka3-run<\/p>\n

lbu add \/usr\/bin\/nuitka3<\/p>\n

lbu add \/usr\/lib\/python3.6\/site-packages\/nuitka<\/p>\n

lbu commit \u2013d<\/p>\n<\/blockquote>\n

Note: nuitka\u2019s source is in Python. <\/p>\n

Check that nuitka runs:<\/p>\n

\n

python3 \u2013m nuitka<\/p>\n<\/blockquote>\n

should execute without <\/strong>showing any syntax errors.<\/p>\n

Fixing nuitka to run with Alpine Linux armhf<\/h2>\n

Important: WordPress unfortunately badly mangles the syntax. Please refer to the GitHUB issue I opened<\/a>, where I also attached the patch files for a reference.<\/p>\n

nuitka calls several support binary utilities on your system, and expects them to return the results in a certain fashion. This is, where nuitka initially stumbles.<\/p>\n

First error<\/strong><\/p>\n

\n

pidoctor:\/opt\/pidoctor# python3 -m nuitka –follow-imports –standalone pidoctor.py<\/p>\n

Traceback (most recent call last):<\/p>\n

File “\/usr\/lib\/python3.6\/site-packages\/nuitka\/__main__.py”, line 188, in <module><\/p>\n

main()<\/p>\n

File “\/usr\/lib\/python3.6\/site-packages\/nuitka\/__main__.py”, line 182, in main<\/p>\n

MainControl.main()<\/p>\n

File “\/usr\/lib\/python3.6\/site-packages\/nuitka\/MainControl.py”, line 846, in main<\/p>\n

Plugins.considerExtraDlls(dist_dir, module)<\/p>\n

File “\/usr\/lib\/python3.6\/site-packages\/nuitka\/plugins\/Plugins.py”, line 102, in considerExtraDlls<\/p>\n

for extra_dll in plugin.considerExtraDlls(dist_dir, module):<\/p>\n

File “\/usr\/lib\/python3.6\/site-packages\/nuitka\/plugins\/standard\/ImplicitImports.py”, line 372, in considerExtraDlls<\/p>\n

uuid_dll_path = locateDLL(“uuid”)<\/b><\/p>\n

File “\/usr\/lib\/python3.6\/site-packages\/nuitka\/utils\/SharedLibraries.py”, line 71, in locateDLL<\/p>\n

return dll_map[dll_name]<\/p>\n

KeyError: ‘libuuid.so.1.3.0’<\/b><\/p>\n<\/blockquote>\n

Reason behind the error: nuitka calls ldconfig \u2013p and expects it to list a list of cached shared libraries, in a format like this:<\/p>\n

libICE.so.6 (libc6,x86-64) => \/usr\/lib\/x86_64-linux-gnu\/libICE.so.6<\/p>\n

on Alpine Linux, ldconfig \u2013p fails \u2013 as there is no such option. ldconfig on Alpine Linux is incapable of producing a list of the form which Nuitka wants.<\/p>\n

Fix:<\/strong><\/p>\n

in \/usr\/lib\/python3.6\/site-packages\/nuitka\/utils\/SharedLibraries.py<\/p>\n

add the following code immediately after the import statements at the top:<\/p>\n

\n

def find_alpine(name,paths):
     for path in paths:
         for root, dirs, files in os.walk(path):
             if name in files:
                 return os.path.join(root,name)<\/p>\n<\/blockquote>\n

and in locateDLL, before import subprocess, add the following code:<\/p>\n

\n

if os.path.isfile(‘\/etc\/alpine-release’):
     return find_alpine(dll_name,[“\/lib”,”\/usr\/lib”,”\/usr\/local\/lib”])
\nelse:
     import subprocess<\/p>\n

    (\u2026)<\/p>\n<\/blockquote>\n

The (\u2026) above indicates that the remainder of the code of locateDLL should be indented under the else.<\/p>\n

What this code does: if the file \/etc\/alpine-release exists \u2013 indicating that we are running on Alpine Linux, we search for the shared library in three predefined directories:<\/p>\n