[egenix-users] single file executables with pyrun (and a little help from makeself)

[M.-A. Lemburg]

Hi Holger,

glad you like PyRun :-)

On 29.05.2019 14:00, Holger Joukl wrote:
> 
> Hi,
> 
> I'm currently beginning to think pyrun is one of the best things since
> sliced bread ;-)
> 
> Of course users on this list have probably known that for years (I first
> learned
> about pyrun at PyCon.DE 2013 (I think) and have gotten round to actually
> trying
> it just now. Can you believe that.)
> 
> Sample usecase: A simple sane webdav client using easywebdav as a small,
> single file
> Python script (right now not a "proper" Python package at all, i.e. no
> setup.py and
> module structure and whatnot).
> 
> Run from a virtualenv but alas I now want to use this somewhere else:
> A machine where not even said virtualenv's base interpreter is available.
> Think an old RHEL box where it's not even Python 2.7-time yet (not even
> that exotic
> in the enterprisey world).
> 
> So, for future reference:
> 
> Enter pyrun:
> 
> # Prerequisites:
> # Download install-pyrun
> # Download makeself: https://github.com/megastep/makeself (nice!)
> 
> # Opionally, set your env intricacies e.g.
> #
> # export PROXY=...      # in case you need a https proxy
> # export CA_BUNDLE=...  # in case you need to provide a custom ssl
> certififates bundle to avoid ssl errors
> # export LIBPATH=...    # in case some mandatory libs are not installed in
> system search paths
> #
> # (I'm leaving all this in for documentational purposes)
> 
> # install pyrun and create "pyrun-venv" named webdav
> # (add --log to make this write <venv-name>/pyrun-installation.log)
> (LD_LIBRARY_PATH=$LIBPATH CURL_CA_BUNDLE=$CA_BUNDLE ./install-pyrun --debug
> --setuptools-version=latest --pip-version=latest --python-unicode=ucs4
> webdav)
> 
> # copy some mandatory library dependencies to the pyrun-venv to make it
> self-sufficient
> # (these are not globally available on our target machines and/or installed
> # at our non-standard location LIBPATH)
> cp $LIBPATH/libcrypto.so.1.0.0 $LIBPATH/libssl.so.1.0.0 ./webdav/lib/
> 
> # These next 2 steps would be replaced by installing my script's proper
> python package
> # and its requirements, if I had set this up
> # install the python dependencies for my script
> ./webdav/bin/pip install --proxy "$PROXY" --cert "$CA_BUNDLE" --timeout=120
> -r packages.txt
> # copy my script to the pyrun-venv
> cp ~/pydev/tools/bin/webdav webdav/bin/
> 
> # modify shebang line to enable relative pyrun python interpreter
> sed -i 's%#!.*python.*%#!/usr/bin/awk BEGIN{a=ARGV[1];b="";for(i=1;i<ARGC;i
> ++){b=b"\\t"ARGV[i];}sub(/[a-z_.\\-]+$/,"python",a);rc=system(a""b);exit
> rc}%g' webdav/bin/webdav
> 
> # extract makeself to use it
> ./makeself-2.4.0.run
> 
> # create a single file executable from the script with makeself
> # (i.e. a self extracting archive that runs the included script)
> ./makeself-2.4.0/makeself.sh webdav webdav.run "webdav command line
> client" ./bin/webdav
> 
> # run our executable
> ./webdav.run -- -h    # makeself treats options after -- as options for the
> script to run
> 
> # we now have a fully *relocatable* single file executable, bundling the
> *complete pyrun interpreter*
> # *plus* our script with all its Python dependencies *and* some shared
> libraries (!):
> du -ms ./webdav.run
> 19	./webdav.run

Nice, I didn't know about makeself before.

> Some notes:
> - makeself adds stdout output when running the executable that I haven't
> been
> able to suppress, like
> Verifying archive integrity...  100%   All good.
> Uncompressing webdav command line client  100%
> 
> - makeself adds its own options list to the command line so options to the
> target script must come after --
> 
> - makeself extracts to a temporary directory and changes into that
> directory
> before running the target script. Of course, this doesn't play well with
> e.g. relative
> path arguments to a script. To care for this, makeself provides the env
> variable USER_PWD
> to the invoked target script, containing the current working dir before the
> "cd $TMPDIR".
> USER_PWD can then be respected by the script. Of course the script must
> then be
> "makeself-instrumented", which is subpar somehow.
> 
> All of these could be addressed in a custom makeself-like self-extractor or
> maybe added as optional makeself features.
> 
> - Is there a nicer way to provide for the relative pyrun interpreter path
> in the script shebang?
> 
> This is really quite powerful - deployment is now moving around a
> relocatable
> "binary", much like go :-)

It'll get even better:

With the next release, we'll officially support adding ZIP files
to the PyRun executable.

This means that you can do things like these:

- create a Python app dir with some Python modules:
  a.py
  b.py

- add a __main__ to the top-level:
  __main__.py
  (with code to start the app using the other two modules)

- ZIP all files:
  zip myapp.zip a.py b.py __main__.py

- "build" your one file app:
  cat pyrun myapp.zip > myapp
  chmod 755 myapp

- run your app:
  ./myapp

PyRun will check for a ZIP file add-on in case it finds that
the binary does not start with "pyrun" or "python".

Cheers,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jun 05 2019)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/