From christian at marquardt.sc Mon Dec 12 00:17:03 2011 From: christian at marquardt.sc (Christian Marquardt) Date: Mon Dec 12 00:17:14 2011 Subject: [egenix-users] mx.DateTime 3.2.x and numpy In-Reply-To: <14409750.336.1323645409568.JavaMail.root@athene> Message-ID: <15145075.339.1323645423254.JavaMail.root@athene> Hello, I hope this is the right place to ask this question... We are in the process of upgrading our mx.Base installation from 3.1.3 to 3.2.1, and while testing, we ran into a problem with the following script (which is also attached):: import numpy as np import mx.DateTime as dt print "Numpy version: ", np.__version__ print "DateTime version:", dt.__version__ delta_t = np.arange(5) * dt.oneHour t0 = dt.now() t = t0 + delta_t print t For mx.DateTime v3.1.3 (was the version string ever updated in the release?), the output is as expected: TCLXS9 (mx) /homespace/marq/python/work/mx>python ../object_arrays/mx_problem.py Numpy version: 1.5.1 DateTime version: 3.1.2 [2011-12-11 22:39:38.60 2011-12-11 23:39:38.60 2011-12-12 00:39:38.60 2011-12-12 01:39:38.60 2011-12-12 02:39:38.60] i.e. the addition of a single mx.DateTime object and a numpy array of mx.DateTimeDelta objects just works. But in v3.2.0 and later, we get TCLXS9 (mx) /homespace/marq/python/work/mx>python ../object_arrays/mx_problem.py Numpy version: 1.5.1 DateTime version: 3.2.1 Traceback (most recent call last): File "../object_arrays/mx_problem.py", line 12, in t = t0 + delta_t TypeError: only length-1 arrays can be converted to Python scalars This is independent from the numpy version (it fails with the more recent numpy 1.6.1 as well), so I think the problem is in mx.DateTime. It also surfaces in other circumstances, for example when trying to compare as scalar mx.DateTime object with an array of such objects. A workaround is to wrap the mx.DateTime scalar in an np.asarray() - but I think it's a bug anyway - and we have a large number of uses cases like the two described, so I would prefer if there would be a way to fix it within mx.DateTime. Any ideas what introduced this bug in 3.2.0? Many thanks, Christian. -------------- next part -------------- A non-text attachment was scrubbed... Name: mx_problem.py Type: text/x-python Size: 264 bytes Desc: not available Url : /mailman-archives/egenix-users/attachments/20111212/739737d1/mx_problem.py From mal at egenix.com Mon Dec 12 19:53:16 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Dec 12 19:52:34 2011 Subject: [egenix-users] mx.DateTime 3.2.x and numpy In-Reply-To: <15145075.339.1323645423254.JavaMail.root@athene> References: <15145075.339.1323645423254.JavaMail.root@athene> Message-ID: <4EE64D9C.5080702@egenix.com> Christian Marquardt wrote: > Hello, > > I hope this is the right place to ask this question... > > We are in the process of upgrading our mx.Base installation from 3.1.3 to 3.2.1, and while testing, we ran into a problem with the following script (which is also attached):: > > import numpy as np > import mx.DateTime as dt > > print "Numpy version: ", np.__version__ > print "DateTime version:", dt.__version__ > > delta_t = np.arange(5) * dt.oneHour > t0 = dt.now() > > t = t0 + delta_t > > print t > > For mx.DateTime v3.1.3 (was the version string ever updated in the release?), the output is as expected: > > TCLXS9 (mx) /homespace/marq/python/work/mx>python ../object_arrays/mx_problem.py > Numpy version: 1.5.1 > DateTime version: 3.1.2 > [2011-12-11 22:39:38.60 2011-12-11 23:39:38.60 2011-12-12 00:39:38.60 > 2011-12-12 01:39:38.60 2011-12-12 02:39:38.60] > > i.e. the addition of a single mx.DateTime object and a numpy array of mx.DateTimeDelta objects just works. But in v3.2.0 and later, we get > > TCLXS9 (mx) /homespace/marq/python/work/mx>python ../object_arrays/mx_problem.py > Numpy version: 1.5.1 > DateTime version: 3.2.1 > Traceback (most recent call last): > File "../object_arrays/mx_problem.py", line 12, in > t = t0 + delta_t > TypeError: only length-1 arrays can be converted to Python scalars > > This is independent from the numpy version (it fails with the more recent numpy 1.6.1 as well), so I think the problem is in mx.DateTime. It also surfaces in other circumstances, for example when trying to compare as scalar mx.DateTime object with an array of such objects. > > A workaround is to wrap the mx.DateTime scalar in an np.asarray() - but I think it's a bug anyway - and we have a large number of uses cases like the two described, so I would prefer if there would be a way to fix it within mx.DateTime. > > Any ideas what introduced this bug in 3.2.0? I'm not sure it's a bug. In mxDateTime 3.2 we replaced the old coercion hack we'd been using in the days before our coercion patches got integrated into Python, with the new methods. Since the error is created by numpy, it looks like numpy is not aware of the change. If there's something we can do to help, let me know. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From christian at marquardt.sc Mon Dec 12 22:21:38 2011 From: christian at marquardt.sc (Christian Marquardt) Date: Mon Dec 12 22:21:42 2011 Subject: [egenix-users] mx.DateTime 3.2.x and numpy In-Reply-To: <4EE64D9C.5080702@egenix.com> Message-ID: <19832693.354.1323724898647.JavaMail.root@athene> Marc-Andre, thank you for the quick response! ----- "M.-A. Lemburg" wrote: > I'm not sure it's a bug. In mxDateTime 3.2 we replaced the old > coercion hack we'd been using in the days before our coercion > patches got integrated into Python, with the new methods. > > Since the error is created by numpy, it looks like numpy is not > aware of the change. If there's something we can do to help, > let me know. I don't know much about the python core, so please apologize if my question is stupid - I just try to obtain some basic understanding so that I can follow this up further (if need be, with the numpy people). When talking about your coercion patches, are you talking about PEP 208? Didn't that go in in Python 2.1? Besides, I believe numpy has its own coercion system... Many thanks, Christian. From antoine+egenix at gymglish.com Thu Dec 15 16:57:51 2011 From: antoine+egenix at gymglish.com (Antoine Gymglish) Date: Thu Dec 15 16:58:18 2011 Subject: [egenix-users] buffer overflow with help(mx.DateTime) on RHEL6 (x86_64) - egenix-mx-base-3.2.1 Message-ID: Hello, I think there is a bug in mx.DateTime for egenix-mx-base-3.2.1, that kills python when triggered: help(mx.DateTime) kills python on Centos6 (RHEL 6 clone): *** buffer overflow detected ***: python terminated How to reproduce from a clean installation (tested on multiple computers): wget http://downloads.egenix.com/python/egenix-mx-base-3.2.1.tar.gz tar xvfz egenix-mx-base-3.2.1.tar.gz cd egenix-mx-base-3.2.1 python setup.py install --home=/home/admin/ export PYTHONPATH=/home/admin/lib64/python python Python 2.6.6 (r266:84292, Jul 20 2011, 10:22:43) [GCC 4.4.5 20110214 (Red Hat 4.4.5-6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import mx.DateTime >>> help(mx.DateTime) *** buffer overflow detected ***: python terminated ======= Backtrace: ========= /lib64/libc.so.6(__fortify_fail+0x37)[0x7f5901be1397] /lib64/libc.so.6(+0xfd280)[0x7f5901bdf280] /lib64/libc.so.6(+0xfc6d9)[0x7f5901bde6d9] /lib64/libc.so.6(_IO_default_xsputn+0xc9)[0x7f5901b55919] /lib64/libc.so.6(_IO_vfprintf+0x5fb)[0x7f5901b2636b] /lib64/libc.so.6(__vsprintf_chk+0x9d)[0x7f5901bde77d] /lib64/libc.so.6(__sprintf_chk+0x7f)[0x7f5901bde6bf] /home/admin/lib64/python/mx/DateTime/mxDateTime/mxDateTime.so(+0x7e7b)[0x7f58fb3dee7b] /usr/lib64/libpython2.6.so.1.0(PyObject_Repr+0x45)[0x7f590279cf05] /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x55a0)[0x7f59027f69f0] /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7f59027f783f] ======= Memory map: ======== 00400000-00401000 r-xp 00000000 08:02 9044917 /usr/bin/python 00600000-00601000 rw-p 00000000 08:02 9044917 /usr/bin/python 0259a000-027f8000 rw-p 00000000 00:00 0 [heap] 7f58f7dad000-7f58f7dc3000 r-xp 00000000 08:02 4718904 /lib64/libgcc_s-4.4.5-20110214.so.1 7f58f7dc3000-7f58f7fc2000 ---p 00016000 08:02 4718904 /lib64/libgcc_s-4.4.5-20110214.so.1 7f58f7fc2000-7f58f7fc3000 rw-p 00015000 08:02 4718904 /lib64/libgcc_s-4.4.5-20110214.so.1 7f58f7fc3000-7f58f7fcf000 r-xp 00000000 08:02 4718751 /lib64/libnss_files-2.12.so 7f58f7fcf000-7f58f81cf000 ---p 0000c000 08:02 4718751 /lib64/libnss_files-2.12.so 7f58f81cf000-7f58f81d0000 r--p 0000c000 08:02 4718751 /lib64/libnss_files-2.12.so 7f58f81d0000-7f58f81d1000 rw-p 0000d000 08:02 4718751 /lib64/libnss_files-2.12.so 7f58f81d1000-7f58f81d5000 r-xp 00000000 08:02 9044851 /usr/lib64/python2.6/lib-dynload/selectmodule.so 7f58f81d5000-7f58f83d5000 ---p 00004000 08:02 9044851 /usr/lib64/python2.6/lib-dynload/selectmodule.so 7f58f83d5000-7f58f83d7000 rw-p 00004000 08:02 9044851 /usr/lib64/python2.6/lib-dynload/selectmodule.so 7f58f83d7000-7f58f83db000 r-xp 00000000 08:02 9064226 /usr/lib64/python2.6/lib-dynload/cStringIO.so 7f58f83db000-7f58f85da000 ---p 00004000 08:02 9064226 /usr/lib64/python2.6/lib-dynload/cStringIO.so 7f58f85da000-7f58f85dc000 rw-p 00003000 08:02 9064226 /usr/lib64/python2.6/lib-dynload/cStringIO.so 7f58f85dc000-7f58f85f9000 r-xp 00000000 08:02 4718919 /lib64/libselinux.so.1 7f58f85f9000-7f58f87f8000 ---p 0001d000 08:02 4718919 /lib64/libselinux.so.1 7f58f87f8000-7f58f87f9000 r--p 0001c000 08:02 4718919 /lib64/libselinux.so.1 7f58f87f9000-7f58f87fa000 rw-p 0001d000 08:02 4718919 /lib64/libselinux.so.1 7f58f87fa000-7f58f87fb000 rw-p 00000000 00:00 0 7f58f87fb000-7f58f8811000 r-xp 00000000 08:02 4718856 /lib64/libresolv-2.12.so 7f58f8811000-7f58f8a10000 ---p 00016000 08:02 4718856 /lib64/libresolv-2.12.so 7f58f8a10000-7f58f8a11000 r--p 00015000 08:02 4718856 /lib64/libresolv-2.12.so 7f58f8a11000-7f58f8a12000 rw-p 00016000 08:02 4718856 /lib64/libresolv-2.12.so 7f58f8a12000-7f58f8a14000 rw-p 00000000 00:00 0 7f58f8a14000-7f58f8a16000 r-xp 00000000 08:02 4718877 /lib64/libkeyutils.so.1.3 7f58f8a16000-7f58f8c15000 ---p 00002000 08:02 4718877 /lib64/libkeyutils.so.1.3 7f58f8c15000-7f58f8c16000 rw-p 00001000 08:02 4718877 /lib64/libkeyutils.so.1.3 7f58f8c16000-7f58f8c20000 r-xp 00000000 08:02 4718779 /lib64/libkrb5support.so.0.1 7f58f8c20000-7f58f8e1f000 ---p 0000a000 08:02 4718779 /lib64/libkrb5support.so.0.1 7f58f8e1f000-7f58f8e20000 rw-p 00009000 08:02 4718779 /lib64/libkrb5support.so.0.1 7f58f8e20000-7f58f8e35000 r-xp 00000000 08:02 4718869 /lib64/libz.so.1.2.3 7f58f8e35000-7f58f9034000 ---p 00015000 08:02 4718869 /lib64/libz.so.1.2.3 7f58f9034000-7f58f9035000 rw-p 00014000 08:02 4718869 /lib64/libz.so.1.2.3 7f58f9035000-7f58f905f000 r-xp 00000000 08:02 4718700 /lib64/libk5crypto.so.3.1 7f58f905f000-7f58f925f000 ---p 0002a000 08:02 4718700 /lib64/libk5crypto.so.3.1 7f58f925f000-7f58f9261000 rw-p 0002a000 08:02 4718700 /lib64/libk5crypto.so.3.1 7f58f9261000-7f58f9264000 r-xp 00000000 08:02 4718739 /lib64/libcom_err.so.2.1 7f58f9264000-7f58f9463000 ---p 00003000 08:02 4718739 /lib64/libcom_err.so.2.1 7f58f9463000-7f58f9464000 rw-p 00002000 08:02 4718739 /lib64/libcom_err.so.2.1 7f58f9464000-7f58f9538000 r-xp 00000000 08:02 4718874 /lib64/libkrb5.so.3.3 7f58f9538000-7f58f9738000 ---p 000d4000 08:02 4718874 /lib64/libkrb5.so.3.3 7f58f9738000-7f58f9743000 rw-p 000d4000 08:02 4718874 /lib64/libkrb5.so.3.3 7f58f9743000-7f58f9782000 r-xp 00000000 08:02 4718606 /lib64/libgssapi_krb5.so.2.2 7f58f9782000-7f58f9981000 ---p 0003f000 08:02 4718606 /lib64/libgssapi_krb5.so.2.2 7f58f9981000-7f58f9984000 rw-p 0003e000 08:02 4718606 /lib64/libgssapi_krb5.so.2.2 7f58f9984000-7f58f9af4000 r-xp 00000000 08:02 9046550 /usr/lib64/libcrypto.so.1.0.0 7f58f9af4000-7f58f9cf4000 ---p 00170000 08:02 9046550 /usr/lib64/libcrypto.so.1.0.0 7f58f9cf4000-7f58f9d16000 rw-p 00170000 08:02 9046550 /usr/lib64/libcrypto.so.1.0.0 7f58f9d16000-7f58f9d1a000 rw-p 00000000 00:00 0 7f58f9d1a000-7f58f9d6d000 r-xp 00000000 08:02 9046141 /usr/lib64/libssl.so.1.0.0 7f58f9d6d000-7f58f9f6c000 ---p 00053000 08:02 9046141 /usr/lib64/libssl.so.1.0.0 7f58f9f6c000-7f58f9f74000 rw-p 00052000 08:02 9046141 /usr/lib64/libssl.so.1.0.0 7f58f9f74000-7f58f9f7b000 r-xp 00000000 08:02 9064219 /usr/lib64/python2.6/lib-dynload/_ssl.so 7f58f9f7b000-7f58fa17a000 ---p 00007000 08:02 9064219 /usr/lib64/python2.6/lib-dynload/_ssl.so 7f58fa17a000-7f58fa17c000 rw-p 00006000 08:02 9064219 /usr/lib64/python2.6/lib-dynload/_ssl.so 7f58fa17c000-7f58fa187000 r-xp 00000000 08:02 9064217 /usr/lib64/python2.6/lib-dynload/_socketmodule.so 7f58fa187000-7f58fa387000 ---p 0000b000 08:02 9064217 /usr/lib64/python2.6/lib-dynload/_socketmodule.so 7f58fa387000-7f58fa38b000 rw-p 0000b000 08:02 9064217 /usr/lib64/python2.6/lib-dynload/_socketmodule.so 7f58fa38b000-7f58fa395000 r-xp 00000000 08:02 9044868 /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so 7f58fa395000-7f58fa594000 ---p 0000a000 08:02 9044868 /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so 7f58fa594000-7f58fa598000 rw-p 00009000 08:02 9044868 /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so 7f58fa598000-7f58fa59e000 r-xp 00000000 08:02 9044812 /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so 7f58fa59e000-7f58fa79d000 ---p 00006000 08:02 9044812 /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so 7f58fa79d000-7f58fa79f000 rw-p 00005000 08:02 9044812 /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so 7f58fa79f000-7f58fa7a7000 r-xp 00000000 08:02 9044873 /usr/lib64/python2.6/lib-dynload/operator.so 7f58fa7a7000-7f58fa9a7000 ---p 00008000 08:02 9044873 /usr/lib64/python2.6/lib-dynload/operator.so 7f58fa9a7000-7f58fa9a9000 rw-p 00008000 08:02 9044873 /usr/lib64/python2.6/lib-dynload/operator.so 7f58fa9a9000-7f58fa9ae000 r-xp 00000000 08:02 9044853 /usr/lib64/python2.6/lib-dynload/stropmodule.so 7f58fa9ae000-7f58fabad000 ---p 00005000 08:02 9044853 /usr/lib64/python2.6/lib-dynload/stropmodule.so 7f58fabad000-7f58fabaf000 rw-p 00004000 08:02 9044853 /usr/lib64/python2.6/lib-dynload/stropmodule.so 7f58fabaf000-7f58fabbf000 r-xp 00000000 08:02 9044816 /usr/lib64/python2.6/lib-dynload/datetime.so 7f58fabbf000-7f58fadbf000 ---p 00010000 08:02 9044816 /usr/lib64/python2.6/lib-dynload/datetime.so 7f58fadbf000-7f58fadc3000 rw-p 00010000 08:02 9044816 /usr/lib64/python2.6/lib-dynload/datetime.so 7f58fadc3000-7f58fadc8000 r-xp 00000000 08:02 9044838 /usr/lib64/python2.6/lib-dynload/mathmodule.so 7f58fadc8000-7f58fafc8000 ---p 00005000 08:02 9044838 /usr/lib64/python2.6/lib-dynload/mathmodule.so 7f58fafc8000-7f58fafca000 rw-p 00005000 08:02 9044838 /usr/lib64/python2.6/lib-dynload/mathmodule.so 7f58fafca000-7f58fafcd000 r-xp 00000000 08:02 9064237 /usr/lib64/python2.6/lib-dynload/timemodule.so 7f58fafcd000-7f58fb1cd000 ---p 00003000 08:02 9064237 /usr/lib64/python2.6/lib-dynload/timemodule.so 7f58fb1cd000-7f58fb1cf000 rw-p 00003000 08:02 9064237 /usr/lib64/python2.6/lib-dynload/timemodule.so 7f58fb1cf000-7f58fb1d6000 r-xp 00000000 08:02 4718610 /lib64/librt-2.12.so 7f58fb1d6000-7f58fb3d5000 ---p 00007000 08:02 4718610 /lib64/librt-2.12.so 7f58fb3d5000-7f58fb3d6000 r--p 00006000 08:02 4718610 /lib64/librt-2.12.so 7f58fb3d6000-7f58fb3d7000 rw-p 00007000 08:02 4718610 /lib64/librt-2.12.so 7f58fb3d7000-7f58fb3e6000 r-xp 00000000 08:02 5767905 /home/admin/lib64/python/mx/DateTime/mxDateTime/mxDateTime.so 7f58fb3e6000-7f58fb5e5000 ---p 0000f000 08:02 5767905 /home/admin/lib64/python/mx/DateTime/mxDateTime/mxDateTime.so 7f58fb5e5000-7f58fb5e7000 rw-p 0000e000 08:02 5767905 /home/admin/lib64/python/mx/DateTime/mxDateTime/mxDateTime.so 7f58fb5e7000-7f58fb604000 r-xp 00000000 08:02 4718703 /lib64/libtinfo.so.5.7 7f58fb604000-7f58fb804000 ---p 0001d000 08:02 4718703 /lib64/libtinfo.so.5.7 7f58fb804000-7f58fb808000 rw-p 0001d000 08:02 4718703 /lib64/libtinfo.so.5.7 7f58fb808000-7f58fb842000 r-xp 00000000 08:02 4718852 /lib64/libreadline.so.6.0 7f58fb842000-7f58fba42000 ---p 0003a000 08:02 4718852 /lib64/libreadline.so.6.0 7f58fba42000-7f58fba4a000 rw-p 0003a000 08:02 4718852 /lib64/libreadline.so.6.0 7f58fba4a000-7f58fba4b000 rw-p 00000000 00:00 0 7f58fba4b000-7f58fba50000 r-xp 00000000 08:02 9064233 /usr/lib64/python2.6/lib-dynload/readline.so 7f58fba50000-7f58fbc4f000 ---p 00005000 08:02 9064233 /usr/lib64/python2.6/lib-dynload/readline.so 7f58fbc4f000-7f58fbc51000 rw-p 00004000 08:02 9064233 /usr/lib64/python2.6/lib-dynload/readline.so 7f58fbc51000-7f5901ae2000 r--p 00000000 08:02 9064194 /usr/lib/locale/locale-archive 7f5901ae2000-7f5901c68000 r-xp 00000000 08:02 4718859 /lib64/libc-2.12.so 7f5901c68000-7f5901e67000 ---p 00186000 08:02 4718859 /lib64/libc-2.12.so 7f5901e67000-7f5901e6b000 r--p 00185000 08:02 4718859 /lib64/libc-2.12.so 7f5901e6b000-7f5901e6c000 rw-p 00189000 08:02 4718859 /lib64/libc-2.12.so 7f5901e6c000-7f5901e71000 rw-p 00000000 00:00 0 7f5901e71000-7f5901ef4000 r-xp 00000000 08:02 4718611 /lib64/libm-2.12.so 7f5901ef4000-7f59020f3000 ---p 00083000 08:02 4718611 /lib64/libm-2.12.so 7f59020f3000-7f59020f4000 r--p 00082000 08:02 4718611 /lib64/libm-2.12.so 7f59020f4000-7f59020f5000 rw-p 00083000 08:02 4718611 /lib64/libm-2.12.so 7f59020f5000-7f59020f7000 r-xp 00000000 08:02 4718609 /lib64/libutil-2.12.so 7f59020f7000-7f59022f6000 ---p 00002000 08:02 4718609 /lib64/libutil-2.12.so 7f59022f6000-7f59022f7000 r--p 00001000 08:02 4718609 /lib64/libutil-2.12.so 7f59022f7000-7f59022f8000 rw-p 00002000 08:02 4718609 /lib64/libutil-2.12.so 7f59022f8000-7f59022fa000 r-xp 00000000 08:02 4718892 /lib64/libdl-2.12.so 7f59022fa000-7f59024fa000 ---p 00002000 08:02 4718892 /lib64/libdl-2.12.so 7f59024fa000-7f59024fb000 r--p 00002000 08:02 4718892 /lib64/libdl-2.12.so 7f59024fb000-7f59024fc000 rw-p 00003000 08:02 4718892 /lib64/libdl-2.12.so 7f59024fc000-7f5902513000 r-xp 00000000 08:02 4718776 /lib64/libpthread-2.12.so 7f5902513000-7f5902712000 ---p 00017000 08:02 4718776 /lib64/libpthread-2.12.so 7f5902712000-7f5902713000 r--p 00016000 08:02 4718776 /lib64/libpthread-2.12.so 7f5902713000-7f5902714000 rw-p 00017000 08:02 4718776 /lib64/libpthread-2.12.so 7f5902714000-7f5902718000 rw-p 00000000 00:00 0 7f5902718000-7f5902888000 r-xp 00000000 08:02 9064204 /usr/lib64/libpython2.6.so.1.0 7f5902888000-7f5902a87000 ---p 00170000 08:02 9064204 /usr/lib64/libpython2.6.so.1.0 7f5902a87000-7f5902ac3000 rw-p 0016f000 08:02 9064204 /usr/lib64/libpython2.6.so.1.0 7f5902ac3000-7f5902ad1000 rw-p 00000000 00:00 0 7f5902ad1000-7f5902af1000 r-xp 00000000 08:02 4718864 /lib64/ld-2.12.so 7f5902b67000-7f5902be9000 rw-p 00000000 00:00 0 7f5902be9000-7f5902bf0000 r--s 00000000 08:02 9063862 /usr/lib64/gconv/gconv-modules.cache 7f5902bf0000-7f5902c62000 rw-p 00000000 00:00 0 7f5902c63000-7f5902cea000 rw-p 00000000 00:00 0 7f5902ced000-7f5902cf0000 rw-p 00000000 00:00 0 7f5902cf0000-7f5902cf1000 r--p 0001f000 08:02 4718864 /lib64/ld-2.12.so 7f5902cf1000-7f5902cf2000 rw-p 00020000 08:02 4718864 /lib64/ld-2.12.so 7f5902cf2000-7f5902cf3000 rw-p 00000000 00:00 0 7fffecb3a000-7fffecb54000 rw-p 00000000 00:00 0 [stack] 7fffecb59000-7fffecb5a000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Aborted Best regards, Antoine From mal at egenix.com Tue Dec 20 16:27:56 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Dec 20 16:28:02 2011 Subject: [egenix-users] mx.DateTime 3.2.x and numpy In-Reply-To: <19832693.354.1323724898647.JavaMail.root@athene> References: <19832693.354.1323724898647.JavaMail.root@athene> Message-ID: <4EF0A97C.40905@egenix.com> Christian Marquardt wrote: > Marc-Andre, > > thank you for the quick response! > > ----- "M.-A. Lemburg" wrote: >> I'm not sure it's a bug. In mxDateTime 3.2 we replaced the old >> coercion hack we'd been using in the days before our coercion >> patches got integrated into Python, with the new methods. >> >> Since the error is created by numpy, it looks like numpy is not >> aware of the change. If there's something we can do to help, >> let me know. > > I don't know much about the python core, so please apologize if my > question is stupid - I just try to obtain some basic understanding > so that I can follow this up further (if need be, with the numpy > people). When talking about your coercion patches, are you talking > about PEP 208? Didn't that go in in Python 2.1? Yes, PEP 208. > Besides, I believe numpy has its own coercion system... We'll have to install numpy and check the problem. We may well have to patch mxDateTime to make it work with numpy, if this uses its own coercion mechanism. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 20 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From vovansim at gmail.com Wed Dec 21 21:18:37 2011 From: vovansim at gmail.com (V) Date: Wed Dec 21 22:30:14 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> Message-ID: M.-A. Lemburg writes: > We'll have to install numpy and check the problem. We may well have > to patch mxDateTime to make it work with numpy, if this uses its > own coercion mechanism. > Marc-Andre, I ran into the very same problem the other day. I have also found the root of the problem. It lies in file mx/DateTime/mxDateTime/mxDateTime.c, function mxDateTime_RichCompare, specifically this little bit of code: else if (PyFloat_Compatible(right)) { /* DateTime op Number -- compare ticks */ double t1 = PyFloat_AsDouble(right); double t0 = mxDateTime_AsTicksWithOffset(self, 0, -1); if ((t0 == -1.0 || t1 == -1.0) && PyErr_Occurred()) goto onError; cmp = (t0 < t1) ? -1 : (t0 > t1) ? 1 : 0; } Note here that if the thing on the left side of a comparison is an mx DateTime, and the thing on the right side of the comparison is a numpy array, this if clause will trigger (because numpy.ndarray implements a __float__ method), and the first thing it will attempt to do is to cast the array to a float. Now, if this array contains more than one element, precisely the exception that Christian observes will occur. TypeError: only length-1 arrays can be converted to Python scalars I suppose it's somewhat of a numpy fault in that perhaps they shouldn't implement conversion from array to float, if it doesn't work for arrays with more than one element, but that behavior has been in numpy for ages, whereas I believe the PyFloat_Compatible check is a semi-new addition to mx DateTime. Hope this helps, and hope you guys can find a good resolution to this. For now, I have to patch my local version of mx to work around this issue. V From mal at egenix.com Wed Dec 21 22:39:02 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Dec 21 22:39:08 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy In-Reply-To: References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> Message-ID: <4EF251F6.1070001@egenix.com> V wrote: > M.-A. Lemburg writes: >> We'll have to install numpy and check the problem. We may well have >> to patch mxDateTime to make it work with numpy, if this uses its >> own coercion mechanism. >> > > Marc-Andre, > > I ran into the very same problem the other day. I have also found the root of > the problem. It lies in file mx/DateTime/mxDateTime/mxDateTime.c, function > mxDateTime_RichCompare, specifically this little bit of code: > > else if (PyFloat_Compatible(right)) { > /* DateTime op Number -- compare ticks */ > double t1 = PyFloat_AsDouble(right); > double t0 = mxDateTime_AsTicksWithOffset(self, 0, -1); > > if ((t0 == -1.0 || t1 == -1.0) && PyErr_Occurred()) > goto onError; > > cmp = (t0 < t1) ? -1 : (t0 > t1) ? 1 : 0; > } > > Note here that if the thing on the left side of a comparison is an mx DateTime, > and the thing on the right side of the comparison is a numpy array, this if > clause will trigger (because numpy.ndarray implements a __float__ method), and > the first thing it will attempt to do is to cast the array to a float. Now, if > this array contains more than one element, precisely the exception that > Christian observes will occur. > > TypeError: only length-1 arrays can be converted to Python scalars > > I suppose it's somewhat of a numpy fault in that perhaps they shouldn't > implement conversion from array to float, if it doesn't work for arrays with > more than one element, but that behavior has been in numpy for ages, whereas I > believe the PyFloat_Compatible check is a semi-new addition to mx DateTime. Not really, but it's possible that the new per operation coercion code does things in a different order. Do you get the same exception when swapping the two arguments ? ... t = delta_t + t0 instead of t = t0 + delta_t > Hope this helps, and hope you guys can find a good resolution to this. For now, > I have to patch my local version of mx to work around this issue. Thank you for the helpful insight. I guess we will either have to add a special case for numpy arrays or fallback to try other methods in case conversion to a float fails. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 21 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From vovansim at gmail.com Wed Dec 21 22:00:33 2011 From: vovansim at gmail.com (V) Date: Wed Dec 21 23:00:53 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> <4EF251F6.1070001@egenix.com> Message-ID: M.-A. Lemburg writes: > Not really, but it's possible that the new per operation > coercion code does things in a different order. That's possible. I'm in the process of upgrading from an older version of python/numpy/mx, and all those data types used to live together nicely. So must be some new behavior, somewhere... > Do you get the same exception when swapping the two arguments ? ... No, when I swap the order, then it actually works. I did some debugging, and it looks like when the order is switched, then the code never really hits the mx DateTime code with the array, but only with individual elements of the array. My guess is that this is so because it goes to the numpy code first, which does an element-by-element comparison for the scalar on the right. > I guess we will either have to add a special case for numpy arrays > or fallback to try other methods in case conversion to a float > fails. Yes, I was thinking about that. I think probably that both of those options are "not ideal" from a software engineering point of view. Perhaps, as a first step you guys could just make the error message a bit more robust. I had a hell of a time looking through C code to figure out what went wrong, but it would have been easier if the message said something like this: [Some Exception]: Could not compare DateTime with , which failed to convert to float with error: [TypeError: only length-1 arrays can be converted to Python scalars] I think there's enough information in that function to make an error message like this, and at least it would be immediately clear that the exception is not actually coming from mx code, but from something it tries to do to an object that says it's convertible to float, but actually throws an exception when attempting said conversion. From jim.vickroy at noaa.gov Wed Dec 21 15:34:50 2011 From: jim.vickroy at noaa.gov (Jim Vickroy) Date: Wed Dec 21 23:34:56 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy In-Reply-To: References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> Message-ID: On Wed, Dec 21, 2011 at 2:18 PM, V wrote: > M.-A. Lemburg writes: > > We'll have to install numpy and check the problem. We may well have > > to patch mxDateTime to make it work with numpy, if this uses its > > own coercion mechanism. > > > > Marc-Andre, > > I ran into the very same problem the other day. I have also found the root > of > the problem. It lies in file mx/DateTime/mxDateTime/mxDateTime.c, function > mxDateTime_RichCompare, specifically this little bit of code: > > else if (PyFloat_Compatible(right)) { > /* DateTime op Number -- compare ticks */ > double t1 = PyFloat_AsDouble(right); > double t0 = mxDateTime_AsTicksWithOffset(self, 0, -1); > > if ((t0 == -1.0 || t1 == -1.0) && PyErr_Occurred()) > goto onError; > > cmp = (t0 < t1) ? -1 : (t0 > t1) ? 1 : 0; > } > > Note here that if the thing on the left side of a comparison is an mx > DateTime, > and the thing on the right side of the comparison is a numpy array, this if > clause will trigger (because numpy.ndarray implements a __float__ method), > and > the first thing it will attempt to do is to cast the array to a float. > Now, if > this array contains more than one element, precisely the exception that > Christian observes will occur. > > TypeError: only length-1 arrays can be converted to Python scalars > > I suppose it's somewhat of a numpy fault in that perhaps they shouldn't > implement conversion from array to float, if it doesn't work for arrays > with > more than one element, but that behavior has been in numpy for ages, > whereas I > believe the PyFloat_Compatible check is a semi-new addition to mx DateTime. > > Hope this helps, and hope you guys can find a good resolution to this. For > now, > I have to patch my local version of mx to work around this issue. > > V > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users > For what it is worth, as a mere "lurker" on this list, that is an error in the numpy implementation, and should be reported. My search attempts to find information on a justification for this behavior are so far unsuccessful. >>> a = numpy.arange(5) >>> a array([0, 1, 2, 3, 4]) >>> float(a) Traceback (most recent call last): File "", line 1, in TypeError: only length-1 arrays can be converted to Python scalars >>> a = numpy.arange(1) >>> a array([0]) >>> float(a) 0.0 >>> a = numpy.arange(5) >>> a.astype('float') array([ 0., 1., 2., 3., 4.]) >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20111221/80163afd/attachment.htm From vovansim at gmail.com Thu Dec 22 17:57:27 2011 From: vovansim at gmail.com (V) Date: Thu Dec 22 18:57:45 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> Message-ID: Jim Vickroy writes: > > For what it is worth, as a mere "lurker" on this list, > that is an error in the numpy implementation, and should be reported. > My search attempts to find information on a justification for this > behavior are so far unsuccessful. Hmm, gmane seems to have swallowed my previous couple attempts to send a response... Perhaps it got labeled as spam due to inclusion of a link to numpy documentation. Aany way, Jim, I believe that this is not actually a bug in numpy, but rather desired behavior. Documentation for numpy.ndarray.__float__ includes the following snippet: " Conversion; the operations complex, int, long, float, oct, and hex. They work only on arrays that have one element in them and return the appropriate scalar. " From jim.vickroy at noaa.gov Thu Dec 22 13:52:04 2011 From: jim.vickroy at noaa.gov (Jim Vickroy) Date: Thu Dec 22 21:52:13 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy In-Reply-To: References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> Message-ID: On Thu, Dec 22, 2011 at 10:57 AM, V wrote: > Jim Vickroy writes: > > > > For what it is worth, as a mere "lurker" on this list, > > that is an error in the numpy implementation, and should be reported. > > My search attempts to find information on a justification for this > > behavior are so far unsuccessful. > > Hmm, gmane seems to have swallowed my previous couple attempts to send a > response... Perhaps it got labeled as spam due to inclusion of a link to > numpy > documentation. Aany way, > > Jim, I believe that this is not actually a bug in numpy, but rather desired > behavior. Documentation for numpy.ndarray.__float__ includes the following > snippet: > > " > Conversion; the operations complex, int, long, float, oct, and hex. > They work only on arrays that have one element in them and return > the appropriate scalar. > " > > > Thanks for this information -- and good detective work to diagnose the original error. I do think, however, this is an unfortunate numpy design decision. I would rather __float__ not be provided at all since it only behaves as expected for a trivial corner case. This is, to me, very unintuitive. -- jv > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20111222/0e5d2012/attachment.htm From vovansim at gmail.com Thu Dec 22 17:15:45 2011 From: vovansim at gmail.com (V) Date: Thu Dec 22 23:27:03 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> Message-ID: Jim Vickroy writes: > For what it is worth, as a mere "lurker" on this list, > that is an error in the numpy implementation, and should be reported. > My search attempts to find information on a justification for this > behavior are so far unsuccessful. Hi, Jim, According to numpy documentation, this is not in fact a bug, but desired behavior. Check out this link: http://docs.scipy.org/doc/numpy-1.6.0/reference/arrays.ndarray.html And specifically the relevant part of the documentation says: " Conversion; the operations complex, int, long, float, oct, and hex. They work only on arrays that have one element in them and return the appropriate scalar. " From vovansim at gmail.com Thu Dec 22 16:52:19 2011 From: vovansim at gmail.com (V) Date: Thu Dec 22 23:27:03 2011 Subject: [egenix-users] Re: mx.DateTime 3.2.x and numpy References: <19832693.354.1323724898647.JavaMail.root@athene> <4EF0A97C.40905@egenix.com> Message-ID: Jim Vickroy writes: > For what it is worth, as a mere "lurker" on this list, > that is an error in the numpy implementation, and should be reported. > My search attempts to find information on a justification for this > behavior are so far unsuccessful. Hi, Jim, According to numpy documentation, this is not in fact a bug, but desired behavior. Check out this link: http://docs.scipy.org/doc/numpy-1.6.0/reference/arrays.ndarray.html And specifically the relevant part of the documentation says: " Conversion; the operations complex, int, long, float, oct, and hex. They work only on arrays that have one element in them and return the appropriate scalar. "