From info at egenix.com Thu May 2 11:55:12 2019 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Thu May 2 10:56:17 2019 Subject: [egenix-users] ANN: PyDDF Python Spring Sprint 2019 Message-ID: <706de1bf-c209-78aa-33dd-e4d5fef2882b@egenix.com> [This announcement is in German since it targets a local user group meeting in D?sseldorf, Germany] ________________________________________________________________________ ANK?NDIGUNG PyDDF Python Spring Sprint 2019 in D?sseldorf Samstag, 04.05.2019, 10:00-18:00 Uhr Sonntag, 05.05.2019, 10:00-18:00 Uhr trivago N.V., Kesselstrasse 5-7, 40221 D?sseldorf Python Meeting D?sseldorf https://www.pyddf.de/sprint2019/ ________________________________________________________________________ INFORMATION Das Python Meeting D?sseldorf (PyDDF) veranstaltet mit freundlicher Unterst?tzung der *trivago N.V.* ein Python Sprint Wochenende. Der Sprint findet am Wochenende 4./5.5.2019 in der trivago Niederlassung im Medienhafen D?sseldorf statt (Achtung: Nicht mehr am Karl-Arnold-Platz). * Google Maps: https://goo.gl/maps/dGM6ThfkLiJ2 Folgende Themengebiete haben wir als Anregung angedacht: * Openpyxl - https://pythonhosted.org/openpyxl/ * eGenix PyRun - One file Python Runtime Portierung auf Python 3.7 * Einfache Buchf?hrungssoftware in Python Nat?rlich kann jeder Teilnehmer weitere Themen vorschlagen. Alles weitere und die Anmeldung findet Ihr auf der Sprint Seite: https://www.pyddf.de/sprint2019/ WICHTIG: Ohne Anmeldung k?nnen wir kein Badge f?r den Geb?udezugang bereitstellen lassen. Eine spontane Anmeldung am Sprint Tag wird daher vermutlich nicht funktionieren. Also bitte unbedingt mit vollen Namen bis sp?testens Freitag, 03.05., anmelden. Teilnehmer sollten sich zudem auf der PyDDF Liste anmelden, da wir uns dort koordinieren: https://www.egenix.com/mailman/listinfo/pyddf ________________________________________________________________________ ?BER UNS Das Python Meeting D?sseldorf (PyDDF) ist eine regelm??ige Veranstaltung in D?sseldorf, die sich an Python Begeisterte aus der Region wendet: * https://pyddf.de/ Einen guten ?berblick ?ber die Vortr?ge bietet unser YouTube-Kanal, auf dem wir die Vortr?ge nach den Meetings ver?ffentlichen: * http://www.youtube.com/pyddf/ Veranstaltet wird das Meeting von der eGenix.com GmbH, Langenfeld, in Zusammenarbeit mit Clark Consulting & Research, D?sseldorf: * http://www.egenix.com/ * http://www.clark-consulting.eu/ Mit freundlichen Gr??en, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 02 2019) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ From Holger.Joukl at LBBW.de Mon May 6 12:48:28 2019 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Mon May 6 11:48:38 2019 Subject: [egenix-users] install-pyrun not able to download setuptools + pip due to pypi changes (install-pyrun script version 1.4.2) Message-ID: Hi, first post here, I'm experimenting a little with the very nice pyrun. Seems like several changes on PyPI (e.g. as described here: https://status.python.org/incidents/1y1f44q6srh2) make install-pyrun fail because - redirects are needed to retrieve the package JSONs with the URLs used in install-pyrun - the JSON content seems to have changed in a way that makes the regular expression-based retrieval of setuptools + pip package .tar.gz files fail. Here's a very basic patch that made things work for me: $ diff install-pyrun.ORIG install-pyrun 14c14 < -l of --log --- > -l or --log 74c74 < tools are downloaded from pypi.python.org. --- > tools are downloaded from pypi.org. 475,476c475,476 < SETUPTOOLS_JSON_URL="https://pypi.python.org/pypi/setuptools/json" < SETUPTOOLS_RE="https://pypi.python.org/packages/.+/setuptools-[0-9\.]+ \.tar\.gz" --- > SETUPTOOLS_JSON_URL="https://pypi.org/pypi/setuptools/json" > SETUPTOOLS_RE='https://files.pythonhosted.org/packages/[^":]+/setuptools-[0-9 \.]+\.tar\.gz' 497,498c497,498 < PIP_JSON_URL="https://pypi.python.org/pypi/pip/json" < PIP_RE="https://pypi.python.org/packages/.+/pip-[0-9\.]+\.tar\.gz" --- > PIP_JSON_URL="https://pypi.org/pypi/pip/json" > PIP_RE='https://files.pythonhosted.org/packages/[^":]+/pip-[0-9\.]+\.tar \.gz' egrep .+ expression matches the longest match which caused the previous regexp to not work correctly any more. The URLs and maybe some other stuff (e.g. curl options) should essentially be configurable but then again maybe it's not worth to bother because they won't change any time soon again(?). If there's a place where I should make a pull request for this patch, please let me know. Best, Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart HRA 4356, HRA 104 440 Amtsgericht Mannheim HRA 40687 Amtsgericht Mainz Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre personenbezogenen Daten. Informationen finden Sie unter https://www.lbbw.de/datenschutz. From mal at egenix.com Mon May 6 12:55:47 2019 From: mal at egenix.com (M.-A. Lemburg) Date: Mon May 6 11:55:55 2019 Subject: [egenix-users] install-pyrun not able to download setuptools + pip due to pypi changes (install-pyrun script version 1.4.2) In-Reply-To: References: Message-ID: <9027ebdb-7d08-1e57-c248-5443672fc0cb@egenix.com> Hi Holger, thank you for your bug report. We are aware of the issues due to the changes on the PyPI servers. I have attached our current version of install-pyrun, which does work with the new PyPI setup, but hasn't been released yet. We will probably make another pyrun release in the next couple of days, which will also have Python 3.6 and 3.7 support. Unfortunately, this will only be a source code release because our build farm server has stopped working. But compiling pyrun on your own servers should be possible without too much hassle. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, May 06 2019) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/ On 06.05.2019 11:48, Holger Joukl wrote: > > Hi, > > first post here, I'm experimenting a little with the very nice pyrun. > > Seems like several changes on PyPI (e.g. as described here: > https://status.python.org/incidents/1y1f44q6srh2) > make install-pyrun fail because > - redirects are needed to retrieve the package JSONs with the URLs used in > install-pyrun > - the JSON content seems to have changed in a way that makes the regular > expression-based > retrieval of setuptools + pip package .tar.gz files fail. > > Here's a very basic patch that made things work for me: > > $ diff install-pyrun.ORIG install-pyrun > 14c14 > < -l of --log > --- >> -l or --log > 74c74 > < tools are downloaded from pypi.python.org. > --- >> tools are downloaded from pypi.org. > 475,476c475,476 > < SETUPTOOLS_JSON_URL="https://pypi.python.org/pypi/setuptools/json" > < SETUPTOOLS_RE="https://pypi.python.org/packages/.+/setuptools-[0-9\.]+ > \.tar\.gz" > --- >> SETUPTOOLS_JSON_URL="https://pypi.org/pypi/setuptools/json" >> > SETUPTOOLS_RE='https://files.pythonhosted.org/packages/[^":]+/setuptools-[0-9 > \.]+\.tar\.gz' > 497,498c497,498 > < PIP_JSON_URL="https://pypi.python.org/pypi/pip/json" > < PIP_RE="https://pypi.python.org/packages/.+/pip-[0-9\.]+\.tar\.gz" > --- >> PIP_JSON_URL="https://pypi.org/pypi/pip/json" >> PIP_RE='https://files.pythonhosted.org/packages/[^":]+/pip-[0-9\.]+\.tar > \.gz' > > egrep .+ expression matches the longest match which caused the previous > regexp to not > work correctly any more. > > The URLs and maybe some other stuff (e.g. curl options) should essentially > be configurable but then again > maybe it's not worth to bother because they won't change any time soon > again(?). > > If there's a place where I should make a pull request for this patch, > please let me know. > > Best, > Holger > > Landesbank Baden-Wuerttemberg > Anstalt des oeffentlichen Rechts > Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz > HRA 12704 > Amtsgericht Stuttgart > HRA 4356, HRA 104 440 > Amtsgericht Mannheim > HRA 40687 > Amtsgericht Mainz > > Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre > personenbezogenen Daten. > Informationen finden Sie unter https://www.lbbw.de/datenschutz. > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users > -------------- next part -------------- #!/bin/bash # # Install eGenix PyRun in a given target directory. # HELP=" Install eGenix PyRun in a given target directory. USAGE: install-pyrun [options] targetdir OPTIONS: -m or --minimal install eGenix PyRun only (no setuptools and pip) -l of --log log installation to targetdir/pyrun-installation.log -q or --quiet quiet installation -r or --requirements have pip install the given requirements (only works if pip is installed as part of the pyrun installation) --python=2.7 install PyRun for Python version 2.6, 2.7 (default), 3.4, 3.5 --python-unicode=ucs2 install PyRun for Python Unicode version ucs2 (default for Python 2) or ucs4 (default for Python 3) --pyrun=2.2.2 install PyRun version 2.2.2 (default) --platform=linux-i686 install PyRun for the given platform; this is usually auto-detected --platform-list list available platform strings --pyrun-distribution=pyrun.tgz use the given PyRun distribution file; this overrides all other distribution selection parameters --pyrun-executable=pyrun symlink to and use an alternative name for the PyRun executable --setuptools-distribution=setuptools.tgz use the given setuptools distribution file instead of downloading it from PyPI --setuptools-version= install setuptools ; use --setuptools-version=latest to automatically find the latest version on PyPI (default) --distribute-distribution=distribute.tgz alias for --setuptools-distribution --pip-distribution=pip.tgz use the given pip distribution file instead of downloading it from PyPI --pip-version= install the pip ; use --pip-version=latest to automatically find the latest version on PyPI (default) --help show this text --version show the script version --copyright show copyright --debug enable debug output --disable-certificate-checks disable certificate checks when downloading packages; this should normally not be needed Without options, the script installs eGenix PyRun, setuptools and pip in targetdir. If no local versions of setuptools or pip are found, the tools are downloaded from pypi.org. " COPYRIGHT=" Copyright (c) 2012-2018, eGenix.com Software GmbH; mailto:info@egenix.com All Rights Reserved. This software may be used under the conditions and terms of the eGenix.com Public License Agreement. You should have received a copy with this software (usually in the file LICENSE located in the package's main directory). Please write to licenses@egenix.com to obtain a copy in case you should not have received a copy. " # Script version VERSION=1.5.0 # Generate debug output ? DEBUG=0 # List of available platform binaries PLATFORM_LIST="\ linux-x86_64 linux-i686 linux-armv6l freebsd-10.3-RELEASE-amd64 freebsd-10.3-RELEASE-i386 macosx-10.4-fat macosx-10.5-x86_64 " ### Configuration # Select Python version (major.minor) PYTHON_VERSION=2.7 # Select Python Unicode version (ucs2 or ucs4 depending on # Python version) PYTHON_UNICODE_VERSION= # Select PyRun version (major.minor.patch) #PYRUN_VERSION=2.3.0 PYRUN_VERSION=2.2.3 # Platform string on the PyRun binary distribution (usually # auto-detected, see below) PLATFORM= # PyRun distribution file. This may contain path information and is # set from the above configuration parameters, if not given explicitly # via --pyrun-distribution= PYRUN_DISTRIBUTION= # Local PyRun distribution directory to search in LOCAL_PYRUN_DISTRIBUTION_DIR=/downloads/egenix # Local PyRun executable name to use PYRUN_EXECUTABLE=pyrun #PYRUN_EXECUTABLE=python # Local setuptools distribution to search for #SETUPTOOLS_VERSION=15.2 SETUPTOOLS_VERSION=latest LOCAL_SETUPTOOLS_DISTRIBUTION_DIR=/downloads/python/setuptools # Local pip distribution to search for #PIP_VERSION=1.5.6 PIP_VERSION=latest LOCAL_PIP_DISTRIBUTION_DIR=/downloads/python/pip # pip requirements to install PIP_REQUIREMENTS= ### Parse options INSTALL_SETUPTOOLS=1 INSTALL_PIP=1 LOG_INSTALLATION=0 RUN_SILENT=0 VERBOSITY=0 DISABLE_CERTIFICATE_CHECKS=0 args=("$@") nargs=${#args[@]} #echo "Found $nargs args" for (( i=0; i<$nargs; i++ )); do arg="${args[i]}" #echo "Parsing arg $i: '$arg'" while [ -n $arg ]; do case $arg in - ) break ;; # Short options (can be comined, e.g. -mq) -m* ) INSTALL_SETUPTOOLS=0 INSTALL_PIP=0 arg="-"${arg:2} ;; -q* ) RUN_SILENT=1 arg="-"${arg:2} ;; -l* ) LOG_INSTALLATION=1 arg="-"${arg:2} ;; -r* ) # Note: We use eval here to deal with tilde expansions # and the like i=$((i+1)) eval PIP_REQUIREMENTS="${args[i]}" arg="-"${arg:2} ;; -h* ) # Note: The quotes are important to make sure the # newlines are interpreted correctly echo "$HELP" exit 0 ;; # Long options --minimal ) INSTALL_SETUPTOOLS=0 INSTALL_PIP=0 break ;; --quiet ) RUN_SILENT=1 break ;; --log ) LOG_INSTALLATION=1 break ;; --requirements=* ) # Note: We use eval here to deal with tilde expansions # and the like eval PIP_REQUIREMENTS="${arg:15}" break ;; --help ) # Note: The quotes are important to make sure the # newlines are interpreted correctly echo "$HELP" exit 0 ;; # Long-only options --python=* ) PYTHON_VERSION=${arg:9} break ;; --python-unicode=* ) PYTHON_UNICODE_VERSION=${arg:17} break ;; --pyrun=* ) PYRUN_VERSION=${arg:8} break ;; --platform=* ) PLATFORM=${arg:11} break ;; --platform-list ) echo "Available platform strings:" echo "---------------------------" echo "$PLATFORM_LIST" exit 0 break ;; --pyrun-distribution=* ) # Note: We use eval here to deal with tilde expansions # and the like eval PYRUN_DISTRIBUTION="${arg:21}" break ;; --pyrun-executable=* ) # Note: We use eval here to deal with tilde expansions # and the like eval PYRUN_EXECUTABLE="${arg:19}" break ;; --setuptools-distribution=*|--distribute-distribution=* ) # Note: We use eval here to deal with tilde expansions # and the like eval SETUPTOOLS_DISTRIBUTION="${arg:26}" break ;; --setuptools-version=* ) SETUPTOOLS_VERSION=${arg:21} break ;; --pip-distribution=* ) # Note: We use eval here to deal with tilde expansions # and the like eval PIP_DISTRIBUTION="${arg:19}" break ;; --pip-version=* ) PIP_VERSION=${arg:14} break ;; --disable-certificate-checks ) DISABLE_CERTIFICATE_CHECKS=1 break ;; --version ) echo "install-pyrun $VERSION" exit 0 ;; --copyright ) echo "$COPYRIGHT" exit 0 ;; --debug ) DEBUG=1 break ;; # Unknown option -* ) echo "Unknown option: $arg" exit 1 ;; # First non-option argument * ) # Note: We use eval here to deal with tilde expansions # and the like eval INSTALLATION_DIR="$arg" break 2 ;; esac done done # Installation directory if [ -z "$INSTALLATION_DIR" ]; then echo "$HELP" exit 1 fi ### Helpers # Current work dir CWD=`pwd` # Tools TAR=tar UNZIP=unzip LN=ln MKDIR=mkdir CURL=`command -v curl` CURL_OPTIONS="-Ssf" CURL_FILE_OPTIONS="-O" CURL_STDOUT_OPTIONS="-o -" WGET=`command -v wget` WGET_OPTIONS= WGET_FILE_OPTIONS= WGET_STDOUT_OPTIONS="-O -" RM=rm ECHO=echo # Detect platform, if not given if [ -z "$PYTHON_UNICODE_VERSION" ]; then if [[ "$PYTHON_VERSION" < "3" ]]; then # Python 2 default PYTHON_UNICODE_VERSION="ucs2" else # Python 3 default PYTHON_UNICODE_VERSION="ucs4" fi fi # Certificate checks if (( $DISABLE_CERTIFICATE_CHECKS )); then CURL_OPTIONS="-k $CURL_OPTIONS" WGET_OPTIONS="--no-check-certificate $WGET_OPTIONS" fi # Convert to absolute path case "$INSTALLATION_DIR" in /*) ;; *) INSTALLATION_DIR=$CWD/$INSTALLATION_DIR esac # Log file if (( $LOG_INSTALLATION )); then LOG_FILE=$INSTALLATION_DIR/pyrun-installation.log else LOG_FILE=/dev/null fi if (( $RUN_SILENT )); then LOG_FILE=/dev/null ECHO=true fi # Fetch URL tool if [ -e "$CURL" ]; then FETCHURL="$CURL $CURL_OPTIONS $CURL_FILE_OPTIONS " READURL="$CURL $CURL_OPTIONS $CURL_STDOUT_OPTIONS " elif [ -e "$WGET" ]; then FETCHURL="$WGET $WGET_OPTIONS $WGET_FILE_OPTIONS " READURL="$WGET $WGET_OPTIONS $WGET_STDOUT_OPTIONS " else echo "Could not find curl or wget. Please consider installing one of those tools." FETCHURL= READURL= fi # Detect platform, if not given if [ -z "$PLATFORM" ]; then PLATFORM_SYSTEM=`uname -s` PLATFORM_PROCESSOR=`uname -p` if [[ "$PLATFORM_PROCESSOR" -eq "unknown" ]]; then PLATFORM_PROCESSOR=`uname -m` fi # When updating this list, please also update the PLATFORM_LIST # variable further up. case "$PLATFORM_SYSTEM $PLATFORM_PROCESSOR" in Linux\ x86_64 ) PLATFORM=linux-x86_64 ;; Linux\ i?86 ) PLATFORM=linux-i686 ;; Linux\ armv6l ) # Raspberry Pi PLATFORM=linux-armv6l ;; FreeBSD\ amd64 ) PLATFORM=freebsd-8.3-RELEASE-p3-amd64 ;; FreeBSD\ i386 ) PLATFORM=freebsd-8.3-RELEASE-p3-i386 ;; Darwin\ powerpc ) PLATFORM=macosx-10.4-fat ;; Darwin\ i386 ) OS_VERSION=`uname -r` if (( ${OS_VERSION%%.*} < 10 )); then # Leopard and earlier default to 32-bit applications PLATFORM=macosx-10.4-fat else # Snow Leopard and later can run 64-bit applications PLATFORM=macosx-10.5-x86_64 fi ;; Darwin\ x86_64 ) PLATFORM=macosx-10.5-x86_64 ;; * ) echo "Unknown platform \"$PLATFORM_SYSTEM $PLATFORM_PROCESSOR\". Please set manually using --platform=..." exit 1 ;; esac fi # Double check Python compatibility # # Python 2.5 are no longer needed for eGenix PyRun 2.0, since # we don't support it anymore for 2.0. Leaving this code in # to be able to install older pyrun versions as well. # if [ "$PYTHON_VERSION" == "2.5" ]; then # setuptools dropped Python 2.5 support in setuptools 2.0, so # force to use the last compatible setuptools version if [[ "$SETUPTOOLS_VERSION" > "1.4.2" ]]; then echo "WARNING: setuptools $SETUPTOOLS_VERSION is not compatible with Python 2.5; using setuptools 1.4.2" 2>&1 | tee -a $LOG_FILE SETUPTOOLS_VERSION="1.4.2" fi # pip dropped Python 2.5 support in pip 1.4, so force to # use the last compatible pip version if [[ "$PIP_VERSION" > "1.3.1" ]]; then echo "WARNING: pip $PIP_VERSION is not compatible with Python 2.5; using pip 1.3.1" 2>&1 | tee -a $LOG_FILE PIP_VERSION="1.3.1" fi fi # eGenix PyRun distribution to use if [ -z "$PYRUN_DISTRIBUTION" ]; then PYRUN_DISTRIBUTION=egenix-pyrun-$PYRUN_VERSION-py${PYTHON_VERSION}_$PYTHON_UNICODE_VERSION-$PLATFORM.tgz fi # Local distribution file name (use PYRUN_DISTRIBUTION if it exists, # fall back to LOCAL_PYRUN_DISTRIBUTION_DIR otherwise) if [ -e $PYRUN_DISTRIBUTION ]; then LOCAL_PYRUN_DISTRIBUTION=$PYRUN_DISTRIBUTION else LOCAL_PYRUN_DISTRIBUTION=$LOCAL_PYRUN_DISTRIBUTION_DIR/$PYRUN_DISTRIBUTION fi # Convert to absolute path case "$LOCAL_PYRUN_DISTRIBUTION" in /*) ;; *) LOCAL_PYRUN_DISTRIBUTION=$CWD/$LOCAL_PYRUN_DISTRIBUTION esac # Remote distribution URL (this is only used in case no local # distribution file can be found) REMOTE_PYRUN_DISTRIBUTION=https://downloads.egenix.com/python/$PYRUN_DISTRIBUTION # setuptools package on PyPI SETUPTOOLS_JSON_URL="https://pypi.org/pypi/setuptools/json" SETUPTOOLS_RE="https://files.pythonhosted.org/packages/[^\"]+/setuptools-[0-9\.]+(\.tar\.gz|\.zip)" # setuptools distribution to use if [ -z "$SETUPTOOLS_DISTRIBUTION" ]; then SETUPTOOLS_DISTRIBUTION=setuptools-$SETUPTOOLS_VERSION.tar.gz fi # Local distribution file name for setuptools if [ -e $SETUPTOOLS_DISTRIBUTION ]; then LOCAL_SETUPTOOLS_DISTRIBUTION=$SETUPTOOLS_DISTRIBUTION else LOCAL_SETUPTOOLS_DISTRIBUTION=$LOCAL_SETUPTOOLS_DISTRIBUTION_DIR/$SETUPTOOLS_DISTRIBUTION fi # Convert to absolute path case "$LOCAL_SETUPTOOLS_DISTRIBUTION" in /*) ;; *) LOCAL_SETUPTOOLS_DISTRIBUTION=$CWD/$LOCAL_SETUPTOOLS_DISTRIBUTION esac # pip package on PyPI PIP_JSON_URL="https://pypi.org/pypi/pip/json" PIP_RE="https://files.pythonhosted.org/packages/[^\"]+/pip-[0-9\.]+(\.tar\.gz|\.zip)" # pip distribution to use if [ -z "$PIP_DISTRIBUTION" ]; then PIP_DISTRIBUTION=pip-$PIP_VERSION.tar.gz fi # Local distribution file name for pip if [ -e $PIP_DISTRIBUTION ]; then LOCAL_PIP_DISTRIBUTION=$PIP_DISTRIBUTION else LOCAL_PIP_DISTRIBUTION=$LOCAL_PIP_DISTRIBUTION_DIR/$PIP_DISTRIBUTION fi # Convert to absolute path case "$LOCAL_PIP_DISTRIBUTION" in /*) ;; *) LOCAL_PIP_DISTRIBUTION=$CWD/$LOCAL_PIP_DISTRIBUTION esac # Convert PIP_REQUIREMENTS to absolute path, if given case "$PIP_REQUIREMENTS" in "") ;; /*) ;; *) PIP_REQUIREMENTS=$CWD/$PIP_REQUIREMENTS esac # Debug output if (( DEBUG )); then $ECHO "Using the following PyRun installation settings:" $ECHO " PYRUN_VERSION=${PYRUN_VERSION}" $ECHO " PYTHON_VERSION=${PYTHON_VERSION}" $ECHO " PYTHON_UNICODE_VERSION=${PYTHON_UNICODE_VERSION}" $ECHO " PYRUN_DISTRIBUTION=${PYRUN_DISTRIBUTION}" $ECHO " LOCAL_PYRUN_DISTRIBUTION=${LOCAL_PYRUN_DISTRIBUTION}" $ECHO " REMOTE_PYRUN_DISTRIBUTION=${REMOTE_PYRUN_DISTRIBUTION}" $ECHO " SETUPTOOLS_DISTRIBUTION=${SETUPTOOLS_DISTRIBUTION}" $ECHO " SETUPTOOLS_VERSION=${SETUPTOOLS_VERSION}" $ECHO " SETUPTOOLS_JSON_URL=${SETUPTOOLS_JSON_URL}" $ECHO " LOCAL_SETUPTOOLS_DISTRIBUTION=${LOCAL_SETUPTOOLS_DISTRIBUTION}" $ECHO " PIP_VERSION=${PIP_VERSION}" $ECHO " PIP_DISTRIBUTION=${PIP_DISTRIBUTION}" $ECHO " PIP_JSON_URL=${PIP_JSON_URL}" $ECHO " LOCAL_PIP_DISTRIBUTION=${LOCAL_PIP_DISTRIBUTION}" $ECHO " PIP_REQUIREMENTS=${PIP_REQUIREMENTS}" $ECHO " INSTALLATION_DIR=${INSTALLATION_DIR}" $ECHO " FETCHURL=${FETCHURL}" fi ### Installation # Run installation in the INSTALLATION_DIR $MKDIR -p $INSTALLATION_DIR cd $INSTALLATION_DIR touch $LOG_FILE # Install PyRun if [ ! -e $LOCAL_PYRUN_DISTRIBUTION ]; then $ECHO "Downloading eGenix PyRun ..." 2>&1 | tee -a $LOG_FILE $FETCHURL $REMOTE_PYRUN_DISTRIBUTION >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Failed to download $REMOTE_PYRUN_DISTRIBUTION" 2>&1 | tee -a $LOG_FILE exit $rc fi $ECHO "" >> $LOG_FILE 2>&1 $ECHO "Installing eGenix PyRun ..." 2>&1 | tee -a $LOG_FILE $TAR -x -v -z -f $PYRUN_DISTRIBUTION >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Failed to extract $PYRUN_DISTRIBUTION" 2>&1 | tee -a $LOG_FILE exit $rc fi $RM -f $PYRUN_DISTRIBUTION else $ECHO "Installing eGenix PyRun ..." 2>&1 | tee -a $LOG_FILE $ECHO "extracting files from $LOCAL_PYRUN_DISTRIBUTION" >> $LOG_FILE 2>&1 $TAR -x -v -z -f $LOCAL_PYRUN_DISTRIBUTION >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Could not extract $LOCAL_PYRUN_DISTRIBUTION" 2>&1 | tee -a $LOG_FILE exit $rc fi fi $ECHO "" >> $LOG_FILE 2>&1 # Add symlink to an alternative name if [ "$PYRUN_EXECUTABLE" != "pyrun" ]; then $ECHO "adding symlink to from bin/pyrun to bin/$PYRUN_EXECUTABLE" >> $LOG_FILE 2>&1 $LN -sf pyrun bin/$PYRUN_EXECUTABLE >> $LOG_FILE 2>&1 $ECHO "" >> $LOG_FILE 2>&1 fi # Install setuptools if (( $INSTALL_SETUPTOOLS )); then if [ ! -e $LOCAL_SETUPTOOLS_DISTRIBUTION ]; then # Get package URLs SETUPTOOLS_URLS=$( \ $READURL -q ${SETUPTOOLS_JSON_URL} | \ egrep -o ${SETUPTOOLS_RE} ) if [ -z "$SETUPTOOLS_URLS" ]; then echo "Could not find setuptools on PyPI" 2>&1 | tee -a $LOG_FILE exit $rc fi if (( DEBUG )); then $ECHO "Available setuptools packages:" $ECHO "${SETUPTOOLS_URLS}" SETUPTOOLS_VERSIONS=$( \ echo "$SETUPTOOLS_URLS" | \ egrep -o ${SETUPTOOLS_RE} | \ sed -r -e 's/http[^ ]+-//g' -e 's/\.tar.gz|\.zip//g' | \ sort -t. -k 1,1nr -k 2,2nr -k 3,3nr -k 4,4nr ) $ECHO "Available setuptools versions:" $ECHO "${SETUPTOOLS_VERSIONS}" fi if [ "$SETUPTOOLS_VERSION" == "latest" ]; then # Find the URL of the latest setuptools distribution file $ECHO "Installing latest setuptools from PyPI ..." 2>&1 | tee -a $LOG_FILE SETUPTOOLS_VERSION=$( \ echo "$SETUPTOOLS_URLS" | \ egrep -o ${SETUPTOOLS_RE} | \ sed -r -e 's/http[^ ]+-//g' -e 's/\.tar\.gz|\.zip//g' | \ sort -t. -k 1,1nr -k 2,2nr -k 3,3nr -k 4,4nr | \ head -n 1 ) $ECHO "Found latest version $SETUPTOOLS_VERSION" 2>&1 | tee -a $LOG_FILE else $ECHO "Installing setuptools $SETUPTOOLS_VERSION from PyPI ..." 2>&1 | tee -a $LOG_FILE fi FETCH_SETUPTOOLS_URL=$( \ echo "$SETUPTOOLS_URLS" | \ egrep -o "http.+setuptools-${SETUPTOOLS_VERSION}(\.tar\.gz|\.zip)" ) $ECHO "Downloading setuptools from $FETCH_SETUPTOOLS_URL ..." >> $LOG_FILE 2>&1 $FETCHURL $FETCH_SETUPTOOLS_URL >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Failed to download $FETCH_SETUPTOOLS_URL" 2>&1 | tee -a $LOG_FILE exit $rc fi $ECHO "Extracting and installing setuptools ..." >> $LOG_FILE 2>&1 SETUPTOOLS_FILE=${FETCH_SETUPTOOLS_URL##*/} if [[ "$SETUPTOOLS_FILE" =~ .+\.tar\.gz ]]; then $TAR -x -v -z -f $SETUPTOOLS_FILE >> $LOG_FILE 2>&1 SETUPTOOLS_DIR=${SETUPTOOLS_FILE%.tar.gz} elif [[ "$SETUPTOOLS_FILE" =~ .+\.zip ]]; then echo "$UNZIP -o $SETUPTOOLS_FILE >> $LOG_FILE 2>&1" $UNZIP -o $SETUPTOOLS_FILE >> $LOG_FILE 2>&1 SETUPTOOLS_DIR=${SETUPTOOLS_FILE%.zip} else echo "*** Unsupported file format: $SETUPTOOLS_FILE" exit 1 fi cd $SETUPTOOLS_DIR ../bin/$PYRUN_EXECUTABLE setup.py install >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Failed to install setuptools" 2>&1 | tee -a $LOG_FILE exit $rc fi cd .. $RM -rf setuptools* >> $LOG_FILE 2>&1 $ECHO "" >> $LOG_FILE 2>&1 else $ECHO "Installing local setuptools $SETUPTOOLS_VERSION ..." 2>&1 | tee -a $LOG_FILE $ECHO "extracting files from $LOCAL_SETUPTOOLS_DISTRIBUTION" >> $LOG_FILE 2>&1 $TAR -x -v -z -f $LOCAL_SETUPTOOLS_DISTRIBUTION >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Could not extract $LOCAL_SETUPTOOLS_DISTRIBUTION" 2>&1 | tee -a $LOG_FILE exit $rc fi cd setuptools-* ../bin/$PYRUN_EXECUTABLE setup.py install >> $LOG_FILE 2>&1 rc=$? cd .. if (( $rc )); then echo "Failed to install setuptools" 2>&1 | tee -a $LOG_FILE exit $rc fi $RM -rf setuptools-* >> $LOG_FILE 2>&1 $ECHO "" >> $LOG_FILE 2>&1 fi fi # Install pip if (( $INSTALL_PIP )); then if [ ! -e $LOCAL_PIP_DISTRIBUTION ]; then # Get package URLs PIP_URLS=$( \ $READURL -q ${PIP_JSON_URL} | \ egrep -o ${PIP_RE} ) if [ -z "$PIP_URLS" ]; then echo "Could not find pip on PyPI" 2>&1 | tee -a $LOG_FILE exit $rc fi if (( DEBUG )); then $ECHO "Available pip packages:" $ECHO "${PIP_URLS}" PIP_VERSIONS=$( \ echo "$PIP_URLS" | \ egrep -o ${PIP_RE} | \ sed -r -e 's/http[^ ]+-//g' -e 's/\.tar.gz|\.zip//g' | \ sort -t. -k 1,1nr -k 2,2nr -k 3,3nr -k 4,4nr ) $ECHO "Available pip versions:" $ECHO "${PIP_VERSIONS}" fi if [ "$PIP_VERSION" == "latest" ]; then # Find the URL of the latest pip distribution file $ECHO "Installing latest pip from PyPI ..." 2>&1 | tee -a $LOG_FILE PIP_VERSION=$( \ echo "$PIP_URLS" | \ egrep -o ${PIP_RE} | \ sed -r -e 's/http[^ ]+-//g' -e 's/\.tar\.gz|\.zip//g' | \ sort -t. -k 1,1nr -k 2,2nr -k 3,3nr -k 4,4nr | \ head -n 1 ) $ECHO "Found latest version $PIP_VERSION" 2>&1 | tee -a $LOG_FILE else $ECHO "Installing pip $PIP_VERSION from PyPI ..." 2>&1 | tee -a $LOG_FILE fi FETCH_PIP_URL=$( \ echo "$PIP_URLS" | \ egrep -o "http.+pip-${PIP_VERSION}(\.tar\.gz|\.zip)" ) $ECHO "Downloading pip from $FETCH_PIP_URL ..." >> $LOG_FILE 2>&1 $FETCHURL $FETCH_PIP_URL >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Failed to download $FETCH_PIP_URL" 2>&1 | tee -a $LOG_FILE exit $rc fi $ECHO "Extracting and installing pip ..." >> $LOG_FILE 2>&1 PIP_FILE=${FETCH_PIP_URL##*/} if [[ "$PIP_FILE" =~ .+\.tar\.gz ]]; then $TAR -x -v -z -f $PIP_FILE >> $LOG_FILE 2>&1 PIP_DIR=${PIP_FILE%.tar.gz} elif [[ "$PIP_FILE" =~ .+\.zip ]]; then echo "$UNZIP -o $PIP_FILE >> $LOG_FILE 2>&1" $UNZIP -o $PIP_FILE >> $LOG_FILE 2>&1 PIP_DIR=${PIP_FILE%.zip} else echo "*** Unsupported file format: $PIP_FILE" exit 1 fi cd $PIP_DIR ../bin/$PYRUN_EXECUTABLE setup.py install >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Failed to install pip" 2>&1 | tee -a $LOG_FILE exit $rc fi cd .. $RM -rf pip* >> $LOG_FILE 2>&1 $ECHO "" >> $LOG_FILE 2>&1 else $ECHO "Installing local pip $PIP_VERSION ..." 2>&1 | tee -a $LOG_FILE $ECHO "extracting files from $LOCAL_PIP_DISTRIBUTION" >> $LOG_FILE 2>&1 $TAR -x -v -z -f $LOCAL_PIP_DISTRIBUTION >> $LOG_FILE 2>&1 rc=$? if (( $rc )); then echo "Could not extract $LOCAL_PIP_DISTRIBUTION" 2>&1 | tee -a $LOG_FILE exit $rc fi cd pip-* ../bin/$PYRUN_EXECUTABLE setup.py install >> $LOG_FILE 2>&1 rc=$? cd .. if (( $rc )); then echo "Failed to install pip" 2>&1 | tee -a $LOG_FILE exit $rc fi $RM -rf pip-* >> $LOG_FILE 2>&1 $ECHO "" >> $LOG_FILE 2>&1 fi # Support for -r switch if [ ! -z "$PIP_REQUIREMENTS" ]; then $ECHO "Installing pip requirements $PIP_REQUIREMENTS ..." 2>&1 | tee -a $LOG_FILE bin/pip install -r $PIP_REQUIREMENTS >> $LOG_FILE 2>&1 fi fi # Finished cd .. $ECHO "" $ECHO "eGenix PyRun was installed in $INSTALLATION_DIR" $ECHO "" $ECHO "To run eGenix PyRun, use $INSTALLATION_DIR/bin/$PYRUN_EXECUTABLE" $ECHO "" From Holger.Joukl at LBBW.de Mon May 6 16:01:20 2019 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Mon May 6 15:01:48 2019 Subject: [egenix-users] install-pyrun not able to download setuptools + pip due to pypi changes (install-pyrun script version 1.4.2) In-Reply-To: <9027ebdb-7d08-1e57-c248-5443672fc0cb@egenix.com> References: <9027ebdb-7d08-1e57-c248-5443672fc0cb@egenix.com> Message-ID: Hi Marc-Andre, > I have attached our current version of install-pyrun, which does work > with the new PyPI setup, but hasn't been released yet. Thanks a lot! Best regards, Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart HRA 4356, HRA 104 440 Amtsgericht Mannheim HRA 40687 Amtsgericht Mainz Die LBBW verarbeitet gemaess Erfordernissen der DSGVO Ihre personenbezogenen Daten. Informationen finden Sie unter https://www.lbbw.de/datenschutz. From Holger.Joukl at LBBW.de Wed May 29 15:00:14 2019 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Wed May 29 14:00:19 2019 Subject: [egenix-users] single file executables with pyrun (and a little help from makeself) Message-ID: Hi, I'm currently beginning to think pyrun is one of the best things since sliced bread ;-) Of course users on this list have probably known that for years (I first learned about pyrun at PyCon.DE 2013 (I think) and have gotten round to actually trying it just now. Can you believe that.) Sample usecase: A simple sane webdav client using easywebdav as a small, single file Python script (right now not a "proper" Python package at all, i.e. no setup.py and module structure and whatnot). Run from a virtualenv but alas I now want to use this somewhere else: A machine where not even said virtualenv's base interpreter is available. Think an old RHEL box where it's not even Python 2.7-time yet (not even that exotic in the enterprisey world). So, for future reference: Enter pyrun: # Prerequisites: # Download install-pyrun # Download makeself: https://github.com/megastep/makeself (nice!) # Opionally, set your env intricacies e.g. # # export PROXY=... # in case you need a https proxy # export CA_BUNDLE=... # in case you need to provide a custom ssl certififates bundle to avoid ssl errors # export LIBPATH=... # in case some mandatory libs are not installed in system search paths # # (I'm leaving all this in for documentational purposes) # install pyrun and create "pyrun-venv" named webdav # (add --log to make this write /pyrun-installation.log) (LD_LIBRARY_PATH=$LIBPATH CURL_CA_BUNDLE=$CA_BUNDLE ./install-pyrun --debug --setuptools-version=latest --pip-version=latest --python-unicode=ucs4 webdav) # copy some mandatory library dependencies to the pyrun-venv to make it self-sufficient # (these are not globally available on our target machines and/or installed # at our non-standard location LIBPATH) cp $LIBPATH/libcrypto.so.1.0.0 $LIBPATH/libssl.so.1.0.0 ./webdav/lib/ # These next 2 steps would be replaced by installing my script's proper python package # and its requirements, if I had set this up # install the python dependencies for my script ./webdav/bin/pip install --proxy "$PROXY" --cert "$CA_BUNDLE" --timeout=120 -r packages.txt # copy my script to the pyrun-venv cp ~/pydev/tools/bin/webdav webdav/bin/ # modify shebang line to enable relative pyrun python interpreter sed -i 's%#!.*python.*%#!/usr/bin/awk BEGIN{a=ARGV[1];b="";for(i=1;i