[egenix-users] single file executables with pyrun (and a little help from makeself)

[Holger Joukl]

Hi,

I'm currently beginning to think pyrun is one of the best things since
sliced bread ;-)

Of course users on this list have probably known that for years (I first
learned
about pyrun at PyCon.DE 2013 (I think) and have gotten round to actually
trying
it just now. Can you believe that.)

Sample usecase: A simple sane webdav client using easywebdav as a small,
single file
Python script (right now not a "proper" Python package at all, i.e. no
setup.py and
module structure and whatnot).

Run from a virtualenv but alas I now want to use this somewhere else:
A machine where not even said virtualenv's base interpreter is available.
Think an old RHEL box where it's not even Python 2.7-time yet (not even
that exotic
in the enterprisey world).

So, for future reference:

Enter pyrun:

# Prerequisites:
# Download install-pyrun
# Download makeself: https://github.com/megastep/makeself (nice!)

# Opionally, set your env intricacies e.g.
#
# export PROXY=...      # in case you need a https proxy
# export CA_BUNDLE=...  # in case you need to provide a custom ssl
certififates bundle to avoid ssl errors
# export LIBPATH=...    # in case some mandatory libs are not installed in
system search paths
#
# (I'm leaving all this in for documentational purposes)

# install pyrun and create "pyrun-venv" named webdav
# (add --log to make this write <venv-name>/pyrun-installation.log)
(LD_LIBRARY_PATH=$LIBPATH CURL_CA_BUNDLE=$CA_BUNDLE ./install-pyrun --debug
--setuptools-version=latest --pip-version=latest --python-unicode=ucs4
webdav)

# copy some mandatory library dependencies to the pyrun-venv to make it
self-sufficient
# (these are not globally available on our target machines and/or installed
# at our non-standard location LIBPATH)
cp $LIBPATH/libcrypto.so.1.0.0 $LIBPATH/libssl.so.1.0.0 ./webdav/lib/

# These next 2 steps would be replaced by installing my script's proper
python package
# and its requirements, if I had set this up
# install the python dependencies for my script
./webdav/bin/pip install --proxy "$PROXY" --cert "$CA_BUNDLE" --timeout=120
-r packages.txt
# copy my script to the pyrun-venv
cp ~/pydev/tools/bin/webdav webdav/bin/

# modify shebang line to enable relative pyrun python interpreter
sed -i 's%#!.*python.*%#!/usr/bin/awk BEGIN{a=ARGV[1];b="";for(i=1;i<ARGC;i
++){b=b"\\t"ARGV[i];}sub(/[a-z_.\\-]+$/,"python",a);rc=system(a""b);exit
rc}%g' webdav/bin/webdav

# extract makeself to use it
./makeself-2.4.0.run

# create a single file executable from the script with makeself
# (i.e. a self extracting archive that runs the included script)
./makeself-2.4.0/makeself.sh webdav webdav.run "webdav command line
client" ./bin/webdav

# run our executable
./webdav.run -- -h    # makeself treats options after -- as options for the
script to run

# we now have a fully *relocatable* single file executable, bundling the
*complete pyrun interpreter*
# *plus* our script with all its Python dependencies *and* some shared
libraries (!):
du -ms ./webdav.run
19	./webdav.run

Some notes:
- makeself adds stdout output when running the executable that I haven't
been
able to suppress, like
Verifying archive integrity...  100%   All good.
Uncompressing webdav command line client  100%

- makeself adds its own options list to the command line so options to the
target script must come after --

- makeself extracts to a temporary directory and changes into that
directory
before running the target script. Of course, this doesn't play well with
e.g. relative
path arguments to a script. To care for this, makeself provides the env
variable USER_PWD
to the invoked target script, containing the current working dir before the
"cd $TMPDIR".
USER_PWD can then be respected by the script. Of course the script must
then be
"makeself-instrumented", which is subpar somehow.

All of these could be addressed in a custom makeself-like self-extractor or
maybe added as optional makeself features.

- Is there a nicer way to provide for the relative pyrun interpreter path
in the script shebang?

This is really quite powerful - deployment is now moving around a
relocatable
"binary", much like go :-)

Best,
Holger


Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart
HRA 4356, HRA 104 440
Amtsgericht Mannheim
HRA 40687
Amtsgericht Mainz

Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre
personenbezogenen Daten.
Informationen finden Sie unter https://www.lbbw.de/datenschutz.