From mal at egenix.com Wed Jun 5 21:05:16 2019 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Jun 5 20:05:24 2019 Subject: [egenix-users] single file executables with pyrun (and a little help from makeself) In-Reply-To: References: Message-ID: <99a2bb6d-bbef-f774-8cef-c13fb3a163d6@egenix.com> Hi Holger, glad you like PyRun :-) On 29.05.2019 14:00, Holger Joukl wrote: > > Hi, > > I'm currently beginning to think pyrun is one of the best things since > sliced bread ;-) > > Of course users on this list have probably known that for years (I first > learned > about pyrun at PyCon.DE 2013 (I think) and have gotten round to actually > trying > it just now. Can you believe that.) > > Sample usecase: A simple sane webdav client using easywebdav as a small, > single file > Python script (right now not a "proper" Python package at all, i.e. no > setup.py and > module structure and whatnot). > > Run from a virtualenv but alas I now want to use this somewhere else: > A machine where not even said virtualenv's base interpreter is available. > Think an old RHEL box where it's not even Python 2.7-time yet (not even > that exotic > in the enterprisey world). > > So, for future reference: > > Enter pyrun: > > # Prerequisites: > # Download install-pyrun > # Download makeself: https://github.com/megastep/makeself (nice!) > > # Opionally, set your env intricacies e.g. > # > # export PROXY=... # in case you need a https proxy > # export CA_BUNDLE=... # in case you need to provide a custom ssl > certififates bundle to avoid ssl errors > # export LIBPATH=... # in case some mandatory libs are not installed in > system search paths > # > # (I'm leaving all this in for documentational purposes) > > # install pyrun and create "pyrun-venv" named webdav > # (add --log to make this write /pyrun-installation.log) > (LD_LIBRARY_PATH=$LIBPATH CURL_CA_BUNDLE=$CA_BUNDLE ./install-pyrun --debug > --setuptools-version=latest --pip-version=latest --python-unicode=ucs4 > webdav) > > # copy some mandatory library dependencies to the pyrun-venv to make it > self-sufficient > # (these are not globally available on our target machines and/or installed > # at our non-standard location LIBPATH) > cp $LIBPATH/libcrypto.so.1.0.0 $LIBPATH/libssl.so.1.0.0 ./webdav/lib/ > > # These next 2 steps would be replaced by installing my script's proper > python package > # and its requirements, if I had set this up > # install the python dependencies for my script > ./webdav/bin/pip install --proxy "$PROXY" --cert "$CA_BUNDLE" --timeout=120 > -r packages.txt > # copy my script to the pyrun-venv > cp ~/pydev/tools/bin/webdav webdav/bin/ > > # modify shebang line to enable relative pyrun python interpreter > sed -i 's%#!.*python.*%#!/usr/bin/awk BEGIN{a=ARGV[1];b="";for(i=1;i ++){b=b"\\t"ARGV[i];}sub(/[a-z_.\\-]+$/,"python",a);rc=system(a""b);exit > rc}%g' webdav/bin/webdav > > # extract makeself to use it > ./makeself-2.4.0.run > > # create a single file executable from the script with makeself > # (i.e. a self extracting archive that runs the included script) > ./makeself-2.4.0/makeself.sh webdav webdav.run "webdav command line > client" ./bin/webdav > > # run our executable > ./webdav.run -- -h # makeself treats options after -- as options for the > script to run > > # we now have a fully *relocatable* single file executable, bundling the > *complete pyrun interpreter* > # *plus* our script with all its Python dependencies *and* some shared > libraries (!): > du -ms ./webdav.run > 19 ./webdav.run Nice, I didn't know about makeself before. > Some notes: > - makeself adds stdout output when running the executable that I haven't > been > able to suppress, like > Verifying archive integrity... 100% All good. > Uncompressing webdav command line client 100% > > - makeself adds its own options list to the command line so options to the > target script must come after -- > > - makeself extracts to a temporary directory and changes into that > directory > before running the target script. Of course, this doesn't play well with > e.g. relative > path arguments to a script. To care for this, makeself provides the env > variable USER_PWD > to the invoked target script, containing the current working dir before the > "cd $TMPDIR". > USER_PWD can then be respected by the script. Of course the script must > then be > "makeself-instrumented", which is subpar somehow. > > All of these could be addressed in a custom makeself-like self-extractor or > maybe added as optional makeself features. > > - Is there a nicer way to provide for the relative pyrun interpreter path > in the script shebang? > > This is really quite powerful - deployment is now moving around a > relocatable > "binary", much like go :-) It'll get even better: With the next release, we'll officially support adding ZIP files to the PyRun executable. This means that you can do things like these: - create a Python app dir with some Python modules: a.py b.py - add a __main__ to the top-level: __main__.py (with code to start the app using the other two modules) - ZIP all files: zip myapp.zip a.py b.py __main__.py - "build" your one file app: cat pyrun myapp.zip > myapp chmod 755 myapp - run your app: ./myapp PyRun will check for a ZIP file add-on in case it finds that the binary does not start with "pyrun" or "python". Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jun 05 2019) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From info at egenix.com Tue Jun 11 10:54:13 2019 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Tue Jun 11 09:54:54 2019 Subject: [egenix-users] =?utf-8?q?ANN=3A_Python_Meeting_D=C3=BCsseldorf_-_12=2E06=2E2019?= Message-ID: <4cd397f8-2552-8882-6afe-15338e844755@egenix.com> [This announcement is in German since it targets a local user group meeting in D?sseldorf, Germany] ________________________________________________________________________ ANK?NDIGUNG Python Meeting D?sseldorf http://pyddf.de/ Ein Treffen von Python Enthusiasten und Interessierten in ungezwungener Atmosph?re. Mittwoch, 12.06.2019, 18:00 Uhr Raum 1, 2.OG im B?rgerhaus Stadtteilzentrum Bilk D?sseldorfer Arcaden, Bachstr. 145, 40217 D?sseldorf Diese Nachricht ist auch online verf?gbar: https://www.egenix.com/company/news/Python-Meeting-Duesseldorf-2019-06-12 ________________________________________________________________________ NEUIGKEITEN * Bereits angemeldete Vortr?ge: Detlef Lannert "Data classes in Python 3.7" Marc-Andre Lemburg "Unfacify ? Gesichtserkennung mit Python" Charlie Clark "Do you love your database enough?" Christian Liguda "Using Python in an e-mail based reporting workflow" Weitere Vortr?ge k?nnen gerne noch angemeldet werden: info@pyddf.de * Startzeit und Ort: Wir treffen uns um 18:00 Uhr im B?rgerhaus in den D?sseldorfer Arcaden. Das B?rgerhaus teilt sich den Eingang mit dem Schwimmbad und befindet sich an der Seite der Tiefgarageneinfahrt der D?sseldorfer Arcaden. ?ber dem Eingang steht ein gro?es "Schwimm' in Bilk" Logo. Hinter der T?r direkt links zu den zwei Aufz?gen, dann in den 2. Stock hochfahren. Der Eingang zum Raum 1 liegt direkt links, wenn man aus dem Aufzug kommt. Google Street View: http://bit.ly/11sCfiw ________________________________________________________________________ EINLEITUNG Das Python Meeting D?sseldorf ist eine regelm??ige Veranstaltung in D?sseldorf, die sich an Python Begeisterte aus der Region wendet: * http://pyddf.de/ Einen guten ?berblick ?ber die Vortr?ge bietet unser YouTube-Kanal, auf dem wir die Vortr?ge nach den Meetings ver?ffentlichen: * http://www.youtube.com/pyddf/ Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, D?sseldorf: * http://www.egenix.com/ * http://www.clark-consulting.eu/ ________________________________________________________________________ PROGRAMM Das Python Meeting D?sseldorf nutzt eine Mischung aus (Lightning) Talks und offener Diskussion. Vortr?ge k?nnen vorher angemeldet werden, oder auch spontan w?hrend des Treffens eingebracht werden. Ein Beamer mit XGA Aufl?sung steht zur Verf?gung. (Lightning) Talk Anmeldung bitte formlos per EMail an info@pyddf.de ________________________________________________________________________ KOSTENBETEILIGUNG Das Python Meeting D?sseldorf wird von Python Nutzern f?r Python Nutzer veranstaltet. Um die Kosten zumindest teilweise zu refinanzieren, bitten wir die Teilnehmer um einen Beitrag in H?he von EUR 10,00 inkl. 19% Mwst, Sch?ler und Studenten zahlen EUR 5,00 inkl. 19% Mwst. Wir m?chten alle Teilnehmer bitten, den Betrag in bar mitzubringen. ________________________________________________________________________ ANMELDUNG Da wir nur f?r ca. 20 Personen Sitzpl?tze haben, m?chten wir bitten, sich per EMail anzumelden. Damit wird keine Verpflichtung eingegangen. Es erleichtert uns allerdings die Planung. Meeting Anmeldung bitte per Meetup https://www.meetup.com/Python-Meeting-Dusseldorf/ oder formlos per EMail an info@pyddf.de ________________________________________________________________________ WEITERE INFORMATIONEN Weitere Informationen finden Sie auf der Webseite des Meetings: http://pyddf.de/ Mit freundlichen Gr??en, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jun 11 2019) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From jamshid.afshar at caringo.com Tue Jun 18 01:53:26 2019 From: jamshid.afshar at caringo.com (Jam Afshar) Date: Tue Jun 18 07:53:44 2019 Subject: [egenix-users] pip 19.1 breaks install: mxDateTime.c: No such file or directory Message-ID: I think this package has a bug or incompatibility with recent versions of pip or with docker containers. It's easily reproducible. I reported it to https://github.com/pypa/pip/issues/6609 but they suggested filing a bug with you. Downgrading to pip==19.0.3 *and* using --no-cache-dir seems to workaround the bug but hopefully there's a better fix. The OS-install pip also seems to work fine. $ docker run -ti centos:7 bash # yum install -y gcc python-devel # curl -fLO https://bootstrap.pypa.io/get-pip.py && python ./get-pip.py # pip --version pip 19.1.1 from /usr/lib/python2.7/site-packages/pip (python 2.7) # pip install egenix-mx-base DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. Collecting egenix-mx-base Downloading https://files.pythonhosted.org/packages/66/e6/e0709aedeb4a5c92a1aeb8c47ab50e9506eafc865806801bd3f01d72b671/egenix-mx-base-3.2.9.zip (74kB) |################################| 81kB 711kB/s Building wheels for collected packages: egenix-mx-base Building wheel for egenix-mx-base (setup.py) ... error ERROR: Complete output from command /usr/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-L6c0SN/egenix-mx-base/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-Y3oD6J --python-tag cp27: ERROR: running bdist_wheel found usable build data file 'build/build-py2.7_ucs4.pck' prebuilt archive found: skipping the build process and loading the prebuilt archive setting platform to 'linux-x86_64' restoring build data from a previous build run loaded build data for platform 'linux-x86_64' installing to build/bdist.linux-x86_64-py2.7_ucs4/wheel running install running install_lib running build_ext building extension "mx.DateTime.mxDateTime.mxDateTime" (required) checking for optional libraries gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _configtest.c -o _configtest.o gcc -pthread _configtest.o -L/usr/lib64 -lm -o _configtest success! removing: _configtest.c _configtest.o _configtest found optional library 'm' -- defining HAVE_M_LIB gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c _configtest.c -o _configtest.o gcc -pthread _configtest.o -L/usr/lib64 -lrt -o _configtest success! removing: _configtest.c _configtest.o _configtest found optional library 'rt' -- defining HAVE_RT_LIB building 'mx.DateTime.mxDateTime.mxDateTime' extension creating build/temp.linux-x86_64-2.7_ucs4 creating build/temp.linux-x86_64-2.7_ucs4/mx-DateTime-mxDateTime-mxDateTime creating build/temp.linux-x86_64-2.7_ucs4/mx-DateTime-mxDateTime-mxDateTime/mx creating build/temp.linux-x86_64-2.7_ucs4/mx-DateTime-mxDateTime-mxDateTime/mx/DateTime creating build/temp.linux-x86_64-2.7_ucs4/mx-DateTime-mxDateTime-mxDateTime/mx/DateTime/mxDateTime gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DUSE_FAST_GETCURRENTTIME -DHAVE_M_LIB=1 -DHAVE_RT_LIB=1 -Imx/DateTime/mxDateTime -I/usr/include/python2.7 -c mx/DateTime/mxDateTime/mxDateTime.c -o build/temp.linux-x86_64-2.7_ucs4/mx-DateTime-mxDateTime-mxDateTime/mx/DateTime/mxDateTime/mxDateTime.o gcc: error: mx/DateTime/mxDateTime/mxDateTime.c: No such file or directory gcc: fatal error: no input files compilation terminated. error: command 'gcc' failed with exit status 4 ---------------------------------------- ERROR: Failed building wheel for egenix-mx-base Running setup.py clean for egenix-mx-base Failed to build egenix-mx-base Installing collected packages: egenix-mx-base Running setup.py install for egenix-mx-base ... error ERROR: Complete output from command /usr/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-L6c0SN/egenix-mx-base/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-GhFuqy/install-record.txt --single-version-externally-managed --compile: ERROR: running install no build data file 'build/build-py2.7_ucs4.pck' found error: build/build-py2.7_ucs4.pck: No such file or directory ---------------------------------------- ERROR: Command "/usr/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-L6c0SN/egenix-mx-base/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-GhFuqy/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-L6c0SN/egenix-mx-base/