From fmcingvale at gmail.com Thu Aug 2 11:08:21 2007 From: fmcingvale at gmail.com (Frank McIngvale) Date: Thu Aug 2 17:08:29 2007 Subject: [egenix-users] mx.TextTools - pickling bug with UnicodeTagTable? Message-ID: <3c55fdbb0708020808j13ea2dd1p6e7632a5f1b2f97b@mail.gmail.com> Hi, there seems to be a bug in pickling tag tables: This works fine: tags = ( (None, Is, 'a'), ) t = TagTable(tags) print type(t) s = pickle.dumps(t) print pickle.loads(s) But this crashes ... tags = ( (None, Is, u'\u03a3'), ) t = UnicodeTagTable(tags) print type(t) s = pickle.dumps(t) print pickle.loads(s) -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070802/39071e5d/attachment.htm From mal at egenix.com Thu Aug 2 18:38:27 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 2 17:38:32 2007 Subject: [egenix-users] mx.TextTools - pickling bug with UnicodeTagTable? In-Reply-To: <3c55fdbb0708020808j13ea2dd1p6e7632a5f1b2f97b@mail.gmail.com> References: <3c55fdbb0708020808j13ea2dd1p6e7632a5f1b2f97b@mail.gmail.com> Message-ID: <46B1FA73.5020201@egenix.com> On 2007-08-02 17:08, Frank McIngvale wrote: > Hi, there seems to be a bug in pickling tag tables: > > This works fine: > > tags = ( > (None, Is, 'a'), > ) > > t = TagTable(tags) > print type(t) > s = pickle.dumps(t) > print pickle.loads(s) > > But this crashes ... > > tags = ( > (None, Is, u'\u03a3'), > ) > > t = UnicodeTagTable(tags) > print type(t) > s = pickle.dumps(t) > print pickle.loads(s) On which platform do you get this ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From fmcingvale at gmail.com Thu Aug 2 11:52:09 2007 From: fmcingvale at gmail.com (Frank McIngvale) Date: Thu Aug 2 17:52:16 2007 Subject: [egenix-users] mx.TextTools - pickling bug with UnicodeTagTable? In-Reply-To: <46B1FA73.5020201@egenix.com> References: <3c55fdbb0708020808j13ea2dd1p6e7632a5f1b2f97b@mail.gmail.com> <46B1FA73.5020201@egenix.com> Message-ID: <3c55fdbb0708020852i2d93c07ep28a31e76446accd4@mail.gmail.com> On 8/2/07, M.-A. Lemburg wrote: > > On 2007-08-02 17:08, Frank McIngvale wrote: > > Hi, there seems to be a bug in pickling tag tables: > > > > This works fine: > > > > tags = ( > > (None, Is, 'a'), > > ) > > > > t = TagTable(tags) > > print type(t) > > s = pickle.dumps(t) > > print pickle.loads(s) > > > > But this crashes ... > > > > tags = ( > > (None, Is, u'\u03a3'), > > ) > > > > t = UnicodeTagTable(tags) > > print type(t) > > s = pickle.dumps(t) > > print pickle.loads(s) > > On which platform do you get this ? WinXP with: "Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32" mx.__version__ = '3.0' Unfortunately there is no message from Python, just a popup box that says the program crashed. frank -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Aug 02 2007) > >>> Python/Zope Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > > :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: > > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070802/f2453e5e/attachment.htm From mal at egenix.com Thu Aug 2 18:55:40 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 2 17:55:45 2007 Subject: [egenix-users] mx.TextTools - pickling bug with UnicodeTagTable? In-Reply-To: <3c55fdbb0708020852i2d93c07ep28a31e76446accd4@mail.gmail.com> References: <3c55fdbb0708020808j13ea2dd1p6e7632a5f1b2f97b@mail.gmail.com> <46B1FA73.5020201@egenix.com> <3c55fdbb0708020852i2d93c07ep28a31e76446accd4@mail.gmail.com> Message-ID: <46B1FE7C.10303@egenix.com> On 2007-08-02 17:52, Frank McIngvale wrote: > On 8/2/07, M.-A. Lemburg wrote: >> On 2007-08-02 17:08, Frank McIngvale wrote: >>> Hi, there seems to be a bug in pickling tag tables: >>> >>> This works fine: >>> >>> tags = ( >>> (None, Is, 'a'), >>> ) >>> >>> t = TagTable(tags) >>> print type(t) >>> s = pickle.dumps(t) >>> print pickle.loads(s) >>> >>> But this crashes ... >>> >>> tags = ( >>> (None, Is, u'\u03a3'), >>> ) >>> >>> t = UnicodeTagTable(tags) >>> print type(t) >>> s = pickle.dumps(t) >>> print pickle.loads(s) >> On which platform do you get this ? > > > WinXP with: > "Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit > (Intel)] on win32" > > mx.__version__ = '3.0' > > Unfortunately there is no message from Python, just a popup box that says > the program crashed. Thanks. I can replicate the problem on Linux x64 as well. The fix is easy: --- mxTextTools.c (revision 22704) +++ mxTextTools.c (working copy) @@ -2654,7 +2654,7 @@ onError: if (own_args) { - Py_DECREF(args); + Py_XDECREF(args); } return -1; } Which then lets the error message get through: Traceback (most recent call last): File "testPickleSegFault.py", line 24, in ? print pickle.loads(s) File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 1394, in loads return Unpickler(file).load() File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 872, in load dispatch[key](self) File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 1153, in load_reduce value = func(*args) File "/home/lemburg/projects/mx/TextTools/__init__.py", line 17, in _TT return TagTable(definition) TypeError: tag table entry 0: conversion from Unicode to string failed I've never tried to pickle tag tables before :-) What's the use case for this ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From fmcingvale at gmail.com Thu Aug 2 12:20:51 2007 From: fmcingvale at gmail.com (Frank McIngvale) Date: Thu Aug 2 18:20:58 2007 Subject: [egenix-users] mx.TextTools - pickling bug with UnicodeTagTable? In-Reply-To: <46B1FE7C.10303@egenix.com> References: <3c55fdbb0708020808j13ea2dd1p6e7632a5f1b2f97b@mail.gmail.com> <46B1FA73.5020201@egenix.com> <3c55fdbb0708020852i2d93c07ep28a31e76446accd4@mail.gmail.com> <46B1FE7C.10303@egenix.com> Message-ID: <3c55fdbb0708020920t2f55c04cn4475b65cea3f00f0@mail.gmail.com> On 8/2/07, M.-A. Lemburg wrote: > > On 2007-08-02 17:52, Frank McIngvale wrote: > > On 8/2/07, M.-A. Lemburg wrote: > >> On 2007-08-02 17:08, Frank McIngvale wrote: > >>> Hi, there seems to be a bug in pickling tag tables: > >>> > >>> This works fine: > >>> > >>> tags = ( > >>> (None, Is, 'a'), > >>> ) > >>> > >>> t = TagTable(tags) > >>> print type(t) > >>> s = pickle.dumps(t) > >>> print pickle.loads(s) > >>> > >>> But this crashes ... > >>> > >>> tags = ( > >>> (None, Is, u'\u03a3'), > >>> ) > >>> > >>> t = UnicodeTagTable(tags) > >>> print type(t) > >>> s = pickle.dumps(t) > >>> print pickle.loads(s) > >> On which platform do you get this ? > > > > > > WinXP with: > > "Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit > > (Intel)] on win32" > > > > mx.__version__ = '3.0' > > > > Unfortunately there is no message from Python, just a popup box that > says > > the program crashed. > > Thanks. I can replicate the problem on Linux x64 as well. The fix is > easy: > > Which then lets the error message get through: > > Traceback (most recent call last): > File "testPickleSegFault.py", line 24, in ? > print pickle.loads(s) > File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 1394, > in loads > return Unpickler(file).load() > File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 872, > in load > dispatch[key](self) > File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 1153, > in load_reduce > value = func(*args) > File "/home/lemburg/projects/mx/TextTools/__init__.py", line 17, in _TT > return TagTable(definition) > TypeError: tag table entry 0: conversion from Unicode to string failed > > I've never tried to pickle tag tables before :-) What's the use > case for this ? I'm adding/updating the support for mx.* types in xml.pickle (Gnosis_Utils) getting ready for a new release, and am including all types that have a copy_reg() in mx.*. But I suppose if no one has hit this before then I don't need to worry about it :-) thanks, frank -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Aug 02 2007) > >>> Python/Zope Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > > :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: > > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070802/a5fdb0f6/attachment.htm From mal at egenix.com Thu Aug 2 19:33:57 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 2 18:34:02 2007 Subject: [egenix-users] mx.TextTools - pickling bug with UnicodeTagTable? In-Reply-To: <3c55fdbb0708020920t2f55c04cn4475b65cea3f00f0@mail.gmail.com> References: <3c55fdbb0708020808j13ea2dd1p6e7632a5f1b2f97b@mail.gmail.com> <46B1FA73.5020201@egenix.com> <3c55fdbb0708020852i2d93c07ep28a31e76446accd4@mail.gmail.com> <46B1FE7C.10303@egenix.com> <3c55fdbb0708020920t2f55c04cn4475b65cea3f00f0@mail.gmail.com> Message-ID: <46B20775.4040605@egenix.com> On 2007-08-02 18:20, Frank McIngvale wrote: >> Traceback (most recent call last): >> File "testPickleSegFault.py", line 24, in ? >> print pickle.loads(s) >> File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 1394, >> in loads >> return Unpickler(file).load() >> File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 872, >> in load >> dispatch[key](self) >> File "/usr/local/python-2.4-ucs2/lib/python2.4/pickle.py", line 1153, >> in load_reduce >> value = func(*args) >> File "/home/lemburg/projects/mx/TextTools/__init__.py", line 17, in _TT >> return TagTable(definition) >> TypeError: tag table entry 0: conversion from Unicode to string failed >> >> I've never tried to pickle tag tables before :-) What's the use >> case for this ? > > > I'm adding/updating the support for mx.* types in xml.pickle (Gnosis_Utils) > getting ready for a new release, and am including all types that have a > copy_reg() in mx.*. But I suppose if no one has hit this before then I don't > need to worry about it :-) We'll add support for pickling UnicodeTagTables with the next release (and of course fix the seg fault). Unfortunately, the change is not as trivial as fixing the segfault. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From Cliff.Xuan at artsalliancemedia.com Tue Aug 7 15:26:51 2007 From: Cliff.Xuan at artsalliancemedia.com (Cliff Xuan) Date: Tue Aug 7 15:27:05 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). Message-ID: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> I had a problem when using unicode object in mxODBC Cursor's executedirect() method. These are the lines from my test code: u'\xe5' is the Unicode object for Norwegian vow ?, spFindUser is a MS SQLServer stored procedure which returns the columns which contains the character in the parameter. Method execute() returns the right result, but executedirect doesn't. con = mx.ODBC.Windows.DriverConnect(dsn) con.encoding='utf-8' selectsql = "{call spFindUser(?)}" cur.execute(selectsql, (u'\xe5')) cur.executedirect(selectsql, (u'\xe5')) Any idea why it's happening? This is the explanation of method executedirect() in the official documentation: .executedirect(operation[,parameters]) Just like .execute(), except that no prepare step is issued and the operation is not cached. This can result in better performance with some ODBC driver setups, but also implies that Python type binding mode is used to bind the parameters. operation may be a Unicode object in case the ODBC driver and/or database support this. Return values are not defined. One questions on this: What does 'prepare step' mean? Many thanks Cliff -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070807/05e354fa/attachment.htm From charlie at egenix.com Tue Aug 7 16:54:42 2007 From: charlie at egenix.com (Charlie Clark) Date: Tue Aug 7 15:54:53 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). In-Reply-To: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> Message-ID: Am 07.08.2007, 15:26 Uhr, schrieb Cliff Xuan : > > One questions on this: > What does 'prepare step' mean? Most database engines allow sql statements to be "prepared", ie. parsed, and the parameters passed in separately and handled by the database ODBC driver and not Python. For repetitive queries with different parameters this can mean a significant difference in performance. As the documentation says there may be cases where the database ODBC driver doesn't support this or isn't very efficient and executedirect() may, therefore, be preferable. I don't think you should need to use executedirect() with MS SQL. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From Cliff.Xuan at artsalliancemedia.com Tue Aug 7 16:37:51 2007 From: Cliff.Xuan at artsalliancemedia.com (Cliff Xuan) Date: Tue Aug 7 16:38:09 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). Message-ID: <49A0DBD9577C0A4096A969872C94980A024FF630@aam-ex-1.aam.local> Hi Charlie, Thanks a lot for your email. It makes a lot of sense to me, however, another problem occurs here. In the first email I had this stored procedure spFindUser. selectsql = "{call spFindUser(?)}" Now I expand that bit by inserting the sql query of it in the python source code, then it looks like this: selectsql2= """ select id from vwUserWide where caller_language_id in ('none','en') and ( username like ('%' + ? + '%') or (coalesce(firstname+' ','')+coalesce(lastname,'')) like ('%' + ? + '%') or email like ('%' + ? + '%') or venue_longname like ('%' + ? + '%') or venue_shortname like ('%' + ? + '%') ) """ cur.execute(selectsql2, [(u'\xe5', u'\xe5', u'\xe5', u'\xe5',u'\xe5',)]) cur.executedirect(selectsql2, [(u'\xe5', u'\xe5', u'\xe5', u'\xe5',u'\xe5',)]) This time executedirect() returns nothing as before. Execute() doesn't work any more, the error message reads "Unable to derive parameter information when parameter marker is a function argument". Hope you can enlighten me again. Many Thanks Cliff Cliff Xuan -----Original Message----- From: Charlie Clark [mailto:charlie@egenix.com] Sent: 07 August 2007 14:55 To: Cliff Xuan; egenix-users@egenix.com Subject: Re: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). Am 07.08.2007, 15:26 Uhr, schrieb Cliff Xuan : > > One questions on this: > What does 'prepare step' mean? Most database engines allow sql statements to be "prepared", ie. parsed, and the parameters passed in separately and handled by the database ODBC driver and not Python. For repetitive queries with different parameters this can mean a significant difference in performance. As the documentation says there may be cases where the database ODBC driver doesn't support this or isn't very efficient and executedirect() may, therefore, be preferable. I don't think you should need to use executedirect() with MS SQL. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From charlie at egenix.com Tue Aug 7 17:49:34 2007 From: charlie at egenix.com (Charlie Clark) Date: Tue Aug 7 16:49:44 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). In-Reply-To: <49A0DBD9577C0A4096A969872C94980A024FF630@aam-ex-1.aam.local> References: <49A0DBD9577C0A4096A969872C94980A024FF630@aam-ex-1.aam.local> Message-ID: Am 07.08.2007, 16:37 Uhr, schrieb Cliff Xuan : > selectsql2= """ > select id > from vwUserWide > where > caller_language_id in ('none','en') and ( > username like ('%' + ? + '%') > or (coalesce(firstname+' ','')+coalesce(lastname,'')) > like ('%' + ? + '%') > or email like ('%' + ? + '%') > or venue_longname like ('%' + ? + '%') > or venue_shortname like ('%' + ? + '%') ) > """ > cur.execute(selectsql2, [(u'\xe5', u'\xe5', u'\xe5', > u'\xe5',u'\xe5',)]) > cur.executedirect(selectsql2, [(u'\xe5', u'\xe5', u'\xe5', > u'\xe5',u'\xe5',)]) > This time executedirect() returns nothing as before. Execute() doesn't > work any more, the error message reads "Unable to derive parameter > information when parameter marker is a function argument". This is a bit of a convoluted query but it looks like there is a problem with the ('%' + ? + '%') syntax. Can you rewrite that using strings? ie., '%?%' and this should get recognised as a string for quoting purposes. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From Cliff.Xuan at artsalliancemedia.com Tue Aug 7 17:16:02 2007 From: Cliff.Xuan at artsalliancemedia.com (Cliff Xuan) Date: Tue Aug 7 17:16:15 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). Message-ID: <49A0DBD9577C0A4096A969872C94980A024FF680@aam-ex-1.aam.local> The query itself is fine as it works in MS SQL Query Browser. I found it might be a Microsoft problem http://support.microsoft.com/kb/821789 Weird though as it's a problem in SQL Server 2000, but my db is 2005. Never mind. You mentioned that I shouldn't need to use executedirect() with MS SQL. What's the reason behind that? Is it to do with the Python type binding? Many Thanks Cliff Xuan Arts Alliance Media Ltd Company Registration No. 04801432 Registered office: 9-11 North End Road, London W14 8ST Registered in England & Wales -----Original Message----- From: Charlie Clark [mailto:charlie@egenix.com] Sent: 07 August 2007 15:50 To: Cliff Xuan; egenix-users@egenix.com Subject: Re: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). Am 07.08.2007, 16:37 Uhr, schrieb Cliff Xuan : > selectsql2= """ > select id > from vwUserWide > where > caller_language_id in ('none','en') and ( > username like ('%' + ? + '%') > or (coalesce(firstname+' ','')+coalesce(lastname,'')) > like ('%' + ? + '%') > or email like ('%' + ? + '%') > or venue_longname like ('%' + ? + '%') > or venue_shortname like ('%' + ? + '%') ) > """ > cur.execute(selectsql2, [(u'\xe5', u'\xe5', u'\xe5', > u'\xe5',u'\xe5',)]) > cur.executedirect(selectsql2, [(u'\xe5', u'\xe5', u'\xe5', > u'\xe5',u'\xe5',)]) > This time executedirect() returns nothing as before. Execute() doesn't > work any more, the error message reads "Unable to derive parameter > information when parameter marker is a function argument". This is a bit of a convoluted query but it looks like there is a problem with the ('%' + ? + '%') syntax. Can you rewrite that using strings? ie., '%?%' and this should get recognised as a string for quoting purposes. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Mon Aug 13 10:38:16 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Mon Aug 13 12:17:48 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string Message-ID: We are currently using version 2 of mxODBC, but there are a couple of problems: 1) Empty string "" is converted to maximum length string (varchar(8000) or nvarchar(4000)) when data is stored into database. 2) Unicode string u"abc" is converted to u"abc ", so that there are equal number of extra spaces in the end, when data is stored into database. We are using SQL Server 2005 and Python 2.4.4 (and Windows XP). Both of these problems must be worked around somehow, but the solution is not easy. 1) We use string with 1 character (space) or None (NULL). The problem is because there is no such datatype as char(0). varchar(0) would work. 2) Workaround not yet found. Except not to use unicode. It is clear that mxODBC calculates first the number of bytes, and then uses that as number of characters, which is twice too much. I don't know if these problems are fixed in the latest release, mxODBC 3. Thanks, Harri Pesonen From mal at egenix.com Mon Aug 13 14:57:24 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Aug 13 13:57:29 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: References: Message-ID: <46C04724.1090708@egenix.com> On 2007-08-13 11:38, Harri Pesonen wrote: > We are currently using version 2 of mxODBC, but there are a couple of problems: > > 1) Empty string "" is converted to maximum length string (varchar(8000) or > nvarchar(4000)) when data is stored into database. mxODBC itself does not do any such conversion. It is possible, however, that the ODBC driver you are using applies such an conversion. Note that on SQL Server char columns are padded with spaces up to the size given in the column definition. This should happen with varchar columns. > 2) Unicode string u"abc" is converted to u"abc ", so that there are equal > number of extra spaces in the end, when data is stored into database. > > We are using SQL Server 2005 and Python 2.4.4 (and Windows XP). Both of these > problems must be worked around somehow, but the solution is not easy. > > 1) We use string with 1 character (space) or None (NULL). The problem is because > there is no such datatype as char(0). varchar(0) would work. > 2) Workaround not yet found. Except not to use unicode. It is clear that mxODBC > calculates first the number of bytes, and then uses that as number of > characters, which is twice too much. > > I don't know if these problems are fixed in the latest release, mxODBC 3. We'll send you an eval license so that you can check this. Please note that you should always try to use the latest available ODBC driver for SQL Server. Because mxODBC uses SQL type binding mode whenever possible to achieve best possible performance (the ODBC tells mxODBC which data type to use), the quality of the ODBC matters a lot. You can also force Python type binding mode by setting connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE The Python type then determines how mxODBC interfaces data to the ODBC driver. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Mon Aug 13 16:19:09 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Mon Aug 13 14:19:29 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: <46C04724.1090708@egenix.com> References: <46C04724.1090708@egenix.com> Message-ID: I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used char(0) datatype internally, which was converted to char(8000) by SQL Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty string, which works correctly. It does not add any extra spaces. 2) is still a problem in mxODBC3. It still uses char instead of varchar for non-empty strings, so that I get "@P5 nchar(40)" and data "N'CustomerAgentRedial1 '". If mxODBC would use nvarchar(40), then SQL Server ODBC driver would not add spaces to the end. Of course this bug could be fixed by using the correct character count in the first place. So both problems happened because char was used instead of varchar. Char requires that there will be spaces in the end, there have to be exactly as many characters in the string. With empty string, char(0) was converted to char(8000) by the SQL Server driver because there is no such data type as char(0). Using connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE does not seem to have any effect. Thanks, Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 13. elokuuta 2007 14:57 To: Harri Pesonen Cc: egenix-users@lists.egenix.com Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string On 2007-08-13 11:38, Harri Pesonen wrote: > We are currently using version 2 of mxODBC, but there are a couple of problems: > > 1) Empty string "" is converted to maximum length string (varchar(8000) or > nvarchar(4000)) when data is stored into database. mxODBC itself does not do any such conversion. It is possible, however, that the ODBC driver you are using applies such an conversion. Note that on SQL Server char columns are padded with spaces up to the size given in the column definition. This should happen with varchar columns. > 2) Unicode string u"abc" is converted to u"abc ", so that there are equal > number of extra spaces in the end, when data is stored into database. > > We are using SQL Server 2005 and Python 2.4.4 (and Windows XP). Both of these > problems must be worked around somehow, but the solution is not easy. > > 1) We use string with 1 character (space) or None (NULL). The problem is because > there is no such datatype as char(0). varchar(0) would work. > 2) Workaround not yet found. Except not to use unicode. It is clear that mxODBC > calculates first the number of bytes, and then uses that as number of > characters, which is twice too much. > > I don't know if these problems are fixed in the latest release, mxODBC 3. We'll send you an eval license so that you can check this. Please note that you should always try to use the latest available ODBC driver for SQL Server. Because mxODBC uses SQL type binding mode whenever possible to achieve best possible performance (the ODBC tells mxODBC which data type to use), the quality of the ODBC matters a lot. You can also force Python type binding mode by setting connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE The Python type then determines how mxODBC interfaces data to the ODBC driver. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Mon Aug 13 16:08:03 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Aug 13 15:08:08 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: References: <46C04724.1090708@egenix.com> Message-ID: <46C057B3.2000202@egenix.com> On 2007-08-13 14:19, Harri Pesonen wrote: > I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used > char(0) datatype internally, which was converted to char(8000) by SQL > Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty string, > which works correctly. It does not add any extra spaces. > > 2) is still a problem in mxODBC3. It still uses char instead of varchar > for non-empty strings, so that I get "@P5 nchar(40)" and data > "N'CustomerAgentRedial1 '". If mxODBC would use > nvarchar(40), then SQL Server ODBC driver would not add spaces to the > end. Of course this bug could be fixed by using the correct character > count in the first place. > > So both problems happened because char was used instead of varchar. Char > requires that there will be spaces in the end, there have to be exactly > as many characters in the string. With empty string, char(0) was > converted to char(8000) by the SQL Server driver because there is no > such data type as char(0). > > Using > connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE > does not seem to have any effect. Could you provide us with the following ? We will then try to recreate the problem here and see what we can do about it. * table schema definition * inserts / selects exhibiting the problem Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 > Thanks, Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 13. elokuuta 2007 14:57 > To: Harri Pesonen > Cc: egenix-users@lists.egenix.com > Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and > unicode string > > On 2007-08-13 11:38, Harri Pesonen wrote: >> We are currently using version 2 of mxODBC, but there are a couple of > problems: >> >> 1) Empty string "" is converted to maximum length string > (varchar(8000) or >> nvarchar(4000)) when data is stored into database. > > mxODBC itself does not do any such conversion. It is possible, however, > that the ODBC driver you are using applies such an conversion. > > Note that on SQL Server char columns are padded with spaces up to the > size given in the column definition. This should happen with varchar > columns. > >> 2) Unicode string u"abc" is converted to u"abc ", so that there are > equal >> number of extra spaces in the end, when data is stored into database. >> >> We are using SQL Server 2005 and Python 2.4.4 (and Windows XP). Both > of these >> problems must be worked around somehow, but the solution is not easy. >> >> 1) We use string with 1 character (space) or None (NULL). The problem > is because >> there is no such datatype as char(0). varchar(0) would work. >> 2) Workaround not yet found. Except not to use unicode. It is clear > that mxODBC >> calculates first the number of bytes, and then uses that as number of >> characters, which is twice too much. >> >> I don't know if these problems are fixed in the latest release, mxODBC > 3. > > We'll send you an eval license so that you can check this. > > Please note that you should always try to use the latest available ODBC > driver for SQL Server. Because mxODBC uses SQL type binding mode > whenever possible to achieve best possible performance (the ODBC tells > mxODBC which data type to use), the quality of the ODBC matters a lot. > > You can also force Python type binding mode by setting > > connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE > > The Python type then determines how mxODBC interfaces data to the > ODBC driver. > From harri.pesonen at wicom.com Mon Aug 13 17:37:19 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Mon Aug 13 15:37:41 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: <46C057B3.2000202@egenix.com> References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> Message-ID: Here is the script: import mx.ODBC.Windows dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;" connection = mx.ODBC.Windows.DriverConnect("%s;APP=%s" % (dsn, "test"), 0) connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT o = connection.cursor() o.execute("CREATE TABLE #test (A nvarchar(64))") s = "INSERT INTO #test (A) VALUES (?)" uc = u'\u0430' c = o.executedirect rs = c(s, ("",)) rs = c(s, (u"",)) rs = c(s, ("a",)) rs = c(s, (uc,)) o.execute("SELECT * FROM #test") rs = o.fetchall() for r in rs: print r o = None Here is the output: (u'',) (u' ',) (u'a',) (u'\u0430 ',) So there is bug #3, empty unicode string is converted to one space. With mxODBC 2 the first case fails as well, you get 64 spaces. -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 13. elokuuta 2007 16:08 To: Harri Pesonen Cc: egenix-users@lists.egenix.com Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string On 2007-08-13 14:19, Harri Pesonen wrote: > I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used > char(0) datatype internally, which was converted to char(8000) by SQL > Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty string, > which works correctly. It does not add any extra spaces. > > 2) is still a problem in mxODBC3. It still uses char instead of varchar > for non-empty strings, so that I get "@P5 nchar(40)" and data > "N'CustomerAgentRedial1 '". If mxODBC would use > nvarchar(40), then SQL Server ODBC driver would not add spaces to the > end. Of course this bug could be fixed by using the correct character > count in the first place. > > So both problems happened because char was used instead of varchar. Char > requires that there will be spaces in the end, there have to be exactly > as many characters in the string. With empty string, char(0) was > converted to char(8000) by the SQL Server driver because there is no > such data type as char(0). > > Using > connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE > does not seem to have any effect. Could you provide us with the following ? We will then try to recreate the problem here and see what we can do about it. * table schema definition * inserts / selects exhibiting the problem Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 > Thanks, Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 13. elokuuta 2007 14:57 > To: Harri Pesonen > Cc: egenix-users@lists.egenix.com > Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and > unicode string > > On 2007-08-13 11:38, Harri Pesonen wrote: >> We are currently using version 2 of mxODBC, but there are a couple of > problems: >> >> 1) Empty string "" is converted to maximum length string > (varchar(8000) or >> nvarchar(4000)) when data is stored into database. > > mxODBC itself does not do any such conversion. It is possible, however, > that the ODBC driver you are using applies such an conversion. > > Note that on SQL Server char columns are padded with spaces up to the > size given in the column definition. This should happen with varchar > columns. > >> 2) Unicode string u"abc" is converted to u"abc ", so that there are > equal >> number of extra spaces in the end, when data is stored into database. >> >> We are using SQL Server 2005 and Python 2.4.4 (and Windows XP). Both > of these >> problems must be worked around somehow, but the solution is not easy. >> >> 1) We use string with 1 character (space) or None (NULL). The problem > is because >> there is no such datatype as char(0). varchar(0) would work. >> 2) Workaround not yet found. Except not to use unicode. It is clear > that mxODBC >> calculates first the number of bytes, and then uses that as number of >> characters, which is twice too much. >> >> I don't know if these problems are fixed in the latest release, mxODBC > 3. > > We'll send you an eval license so that you can check this. > > Please note that you should always try to use the latest available ODBC > driver for SQL Server. Because mxODBC uses SQL type binding mode > whenever possible to achieve best possible performance (the ODBC tells > mxODBC which data type to use), the quality of the ODBC matters a lot. > > You can also force Python type binding mode by setting > > connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE > > The Python type then determines how mxODBC interfaces data to the > ODBC driver. > From mal at egenix.com Mon Aug 13 17:48:30 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Aug 13 16:48:34 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> Message-ID: <46C06F3E.100@egenix.com> On 2007-08-13 15:37, Harri Pesonen wrote: > Here is the script: > > import mx.ODBC.Windows > dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;" > connection = mx.ODBC.Windows.DriverConnect("%s;APP=%s" % (dsn, "test"), > 0) > connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT > o = connection.cursor() > o.execute("CREATE TABLE #test (A nvarchar(64))") > s = "INSERT INTO #test (A) VALUES (?)" > uc = u'\u0430' > c = o.executedirect > rs = c(s, ("",)) > rs = c(s, (u"",)) > rs = c(s, ("a",)) > rs = c(s, (uc,)) > o.execute("SELECT * FROM #test") > rs = o.fetchall() > for r in rs: > print r > o = None > > Here is the output: > > (u'',) > (u' ',) > (u'a',) > (u'\u0430 ',) > > So there is bug #3, empty unicode string is converted to one space. With > mxODBC 2 the first case fails as well, you get 64 spaces. Thanks. We'll run a few tests and then report back our findings. > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 13. elokuuta 2007 16:08 > To: Harri Pesonen > Cc: egenix-users@lists.egenix.com > Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and > unicode string > > On 2007-08-13 14:19, Harri Pesonen wrote: >> I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used >> char(0) datatype internally, which was converted to char(8000) by SQL >> Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty string, >> which works correctly. It does not add any extra spaces. >> >> 2) is still a problem in mxODBC3. It still uses char instead of > varchar >> for non-empty strings, so that I get "@P5 nchar(40)" and data >> "N'CustomerAgentRedial1 '". If mxODBC would use >> nvarchar(40), then SQL Server ODBC driver would not add spaces to the >> end. Of course this bug could be fixed by using the correct character >> count in the first place. >> >> So both problems happened because char was used instead of varchar. > Char >> requires that there will be spaces in the end, there have to be > exactly >> as many characters in the string. With empty string, char(0) was >> converted to char(8000) by the SQL Server driver because there is no >> such data type as char(0). >> >> Using >> connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE >> does not seem to have any effect. > > Could you provide us with the following ? We will then try to > recreate the problem here and see what we can do about it. > > * table schema definition > > * inserts / selects exhibiting the problem > > Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Mon Aug 13 18:38:57 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Aug 13 17:38:59 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: <46C06F3E.100@egenix.com> References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> <46C06F3E.100@egenix.com> Message-ID: <46C07B11.5050409@egenix.com> Here are some preliminary findings: * we can replicate the problem with the SQL Server ODBC driver version 03.85.1117 * the ODBC driver appears to add min(1, len(data)) spaces to all Unicode data inserted into the table - up to the length limit of the column The problem goes away if you use .execute() instead of .executedirect(). We have to do some further testing, but this suggests that the ODBC driver does not work correctly when using .executedirect(). On 2007-08-13 16:48, M.-A. Lemburg wrote: > On 2007-08-13 15:37, Harri Pesonen wrote: >> Here is the script: >> >> import mx.ODBC.Windows >> dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;" >> connection = mx.ODBC.Windows.DriverConnect("%s;APP=%s" % (dsn, "test"), >> 0) >> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >> o = connection.cursor() >> o.execute("CREATE TABLE #test (A nvarchar(64))") >> s = "INSERT INTO #test (A) VALUES (?)" >> uc = u'\u0430' >> c = o.executedirect >> rs = c(s, ("",)) >> rs = c(s, (u"",)) >> rs = c(s, ("a",)) >> rs = c(s, (uc,)) >> o.execute("SELECT * FROM #test") >> rs = o.fetchall() >> for r in rs: >> print r >> o = None >> >> Here is the output: >> >> (u'',) >> (u' ',) >> (u'a',) >> (u'\u0430 ',) >> >> So there is bug #3, empty unicode string is converted to one space. With >> mxODBC 2 the first case fails as well, you get 64 spaces. > > Thanks. > > We'll run a few tests and then report back our findings. > >> -- Harri >> >> -----Original Message----- >> From: M.-A. Lemburg [mailto:mal@egenix.com] >> Sent: 13. elokuuta 2007 16:08 >> To: Harri Pesonen >> Cc: egenix-users@lists.egenix.com >> Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and >> unicode string >> >> On 2007-08-13 14:19, Harri Pesonen wrote: >>> I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used >>> char(0) datatype internally, which was converted to char(8000) by SQL >>> Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty string, >>> which works correctly. It does not add any extra spaces. >>> >>> 2) is still a problem in mxODBC3. It still uses char instead of >> varchar >>> for non-empty strings, so that I get "@P5 nchar(40)" and data >>> "N'CustomerAgentRedial1 '". If mxODBC would use >>> nvarchar(40), then SQL Server ODBC driver would not add spaces to the >>> end. Of course this bug could be fixed by using the correct character >>> count in the first place. >>> >>> So both problems happened because char was used instead of varchar. >> Char >>> requires that there will be spaces in the end, there have to be >> exactly >>> as many characters in the string. With empty string, char(0) was >>> converted to char(8000) by the SQL Server driver because there is no >>> such data type as char(0). >>> >>> Using >>> connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE >>> does not seem to have any effect. >> Could you provide us with the following ? We will then try to >> recreate the problem here and see what we can do about it. >> >> * table schema definition >> >> * inserts / selects exhibiting the problem >> >> Thanks, > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Tue Aug 14 10:04:08 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Tue Aug 14 08:04:19 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: <46C07B11.5050409@egenix.com> References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> <46C06F3E.100@egenix.com> <46C07B11.5050409@egenix.com> Message-ID: I'm sure that you'll find out that you are using nchar(N) data type, where N is calculated from the number of bytes in the unicode string, not number of characters. This is Microsoft SQL Server ODBC driver, so this is the standard. :-) -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 13. elokuuta 2007 18:39 To: Harri Pesonen Cc: egenix-users@lists.egenix.com Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string Here are some preliminary findings: * we can replicate the problem with the SQL Server ODBC driver version 03.85.1117 * the ODBC driver appears to add min(1, len(data)) spaces to all Unicode data inserted into the table - up to the length limit of the column The problem goes away if you use .execute() instead of .executedirect(). We have to do some further testing, but this suggests that the ODBC driver does not work correctly when using .executedirect(). On 2007-08-13 16:48, M.-A. Lemburg wrote: > On 2007-08-13 15:37, Harri Pesonen wrote: >> Here is the script: >> >> import mx.ODBC.Windows >> dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;" >> connection = mx.ODBC.Windows.DriverConnect("%s;APP=%s" % (dsn, "test"), >> 0) >> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >> o = connection.cursor() >> o.execute("CREATE TABLE #test (A nvarchar(64))") >> s = "INSERT INTO #test (A) VALUES (?)" >> uc = u'\u0430' >> c = o.executedirect >> rs = c(s, ("",)) >> rs = c(s, (u"",)) >> rs = c(s, ("a",)) >> rs = c(s, (uc,)) >> o.execute("SELECT * FROM #test") >> rs = o.fetchall() >> for r in rs: >> print r >> o = None >> >> Here is the output: >> >> (u'',) >> (u' ',) >> (u'a',) >> (u'\u0430 ',) >> >> So there is bug #3, empty unicode string is converted to one space. With >> mxODBC 2 the first case fails as well, you get 64 spaces. > > Thanks. > > We'll run a few tests and then report back our findings. > >> -- Harri >> >> -----Original Message----- >> From: M.-A. Lemburg [mailto:mal@egenix.com] >> Sent: 13. elokuuta 2007 16:08 >> To: Harri Pesonen >> Cc: egenix-users@lists.egenix.com >> Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and >> unicode string >> >> On 2007-08-13 14:19, Harri Pesonen wrote: >>> I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used >>> char(0) datatype internally, which was converted to char(8000) by SQL >>> Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty string, >>> which works correctly. It does not add any extra spaces. >>> >>> 2) is still a problem in mxODBC3. It still uses char instead of >> varchar >>> for non-empty strings, so that I get "@P5 nchar(40)" and data >>> "N'CustomerAgentRedial1 '". If mxODBC would use >>> nvarchar(40), then SQL Server ODBC driver would not add spaces to the >>> end. Of course this bug could be fixed by using the correct character >>> count in the first place. >>> >>> So both problems happened because char was used instead of varchar. >> Char >>> requires that there will be spaces in the end, there have to be >> exactly >>> as many characters in the string. With empty string, char(0) was >>> converted to char(8000) by the SQL Server driver because there is no >>> such data type as char(0). >>> >>> Using >>> connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE >>> does not seem to have any effect. >> Could you provide us with the following ? We will then try to >> recreate the problem here and see what we can do about it. >> >> * table schema definition >> >> * inserts / selects exhibiting the problem >> >> Thanks, > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Tue Aug 14 10:11:36 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Tue Aug 14 08:11:42 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: <46C07B11.5050409@egenix.com> References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> <46C06F3E.100@egenix.com> <46C07B11.5050409@egenix.com> Message-ID: One more thing: You could easily fix this problem by using nvarchar instead of nchar data type. Nchar(N) requires that there are exactly N characters, and SQL Server ODBC driver will add spaces if needed. Nvarchar(N) would not ever add spaces. -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 13. elokuuta 2007 18:39 To: Harri Pesonen Cc: egenix-users@lists.egenix.com Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string Here are some preliminary findings: * we can replicate the problem with the SQL Server ODBC driver version 03.85.1117 * the ODBC driver appears to add min(1, len(data)) spaces to all Unicode data inserted into the table - up to the length limit of the column The problem goes away if you use .execute() instead of .executedirect(). We have to do some further testing, but this suggests that the ODBC driver does not work correctly when using .executedirect(). On 2007-08-13 16:48, M.-A. Lemburg wrote: > On 2007-08-13 15:37, Harri Pesonen wrote: >> Here is the script: >> >> import mx.ODBC.Windows >> dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;" >> connection = mx.ODBC.Windows.DriverConnect("%s;APP=%s" % (dsn, "test"), >> 0) >> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >> o = connection.cursor() >> o.execute("CREATE TABLE #test (A nvarchar(64))") >> s = "INSERT INTO #test (A) VALUES (?)" >> uc = u'\u0430' >> c = o.executedirect >> rs = c(s, ("",)) >> rs = c(s, (u"",)) >> rs = c(s, ("a",)) >> rs = c(s, (uc,)) >> o.execute("SELECT * FROM #test") >> rs = o.fetchall() >> for r in rs: >> print r >> o = None >> >> Here is the output: >> >> (u'',) >> (u' ',) >> (u'a',) >> (u'\u0430 ',) >> >> So there is bug #3, empty unicode string is converted to one space. With >> mxODBC 2 the first case fails as well, you get 64 spaces. > > Thanks. > > We'll run a few tests and then report back our findings. > >> -- Harri >> >> -----Original Message----- >> From: M.-A. Lemburg [mailto:mal@egenix.com] >> Sent: 13. elokuuta 2007 16:08 >> To: Harri Pesonen >> Cc: egenix-users@lists.egenix.com >> Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and >> unicode string >> >> On 2007-08-13 14:19, Harri Pesonen wrote: >>> I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used >>> char(0) datatype internally, which was converted to char(8000) by SQL >>> Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty string, >>> which works correctly. It does not add any extra spaces. >>> >>> 2) is still a problem in mxODBC3. It still uses char instead of >> varchar >>> for non-empty strings, so that I get "@P5 nchar(40)" and data >>> "N'CustomerAgentRedial1 '". If mxODBC would use >>> nvarchar(40), then SQL Server ODBC driver would not add spaces to the >>> end. Of course this bug could be fixed by using the correct character >>> count in the first place. >>> >>> So both problems happened because char was used instead of varchar. >> Char >>> requires that there will be spaces in the end, there have to be >> exactly >>> as many characters in the string. With empty string, char(0) was >>> converted to char(8000) by the SQL Server driver because there is no >>> such data type as char(0). >>> >>> Using >>> connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE >>> does not seem to have any effect. >> Could you provide us with the following ? We will then try to >> recreate the problem here and see what we can do about it. >> >> * table schema definition >> >> * inserts / selects exhibiting the problem >> >> Thanks, > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Tue Aug 14 12:08:08 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 14 11:08:16 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> <46C06F3E.100@egenix.com> <46C07B11.5050409@egenix.com> Message-ID: <46C170F8.5080508@egenix.com> Some more investigation showed that even though the type binding was set to SQL (the ODBC driver providing the type information), when using .executedirect() the driver does not provide the type information. mxODBC uses Python type binding in such a case. On 2007-08-14 08:04, Harri Pesonen wrote: > I'm sure that you'll find out that you are using nchar(N) data type, > where N is calculated from the number of bytes in the unicode string, > not number of characters. The length information to use in ODBC APIs is a weak spot in the ODBC standard documentation... in some places it refers to byte counts, in others to code point counts and they aren't clear at all about which to use, see e.g. http://msdn2.microsoft.com/en-gb/library/ms716246.aspx > This is Microsoft SQL Server ODBC driver, so this is the standard. :-) Well, I wouldn't be so sure about this. Especially with respect to Unicode handling, the SQL Server's ODBC driver tends to have a very special interpretation. Things get even more complicated on 64-bit platforms. > One more thing: You could easily fix this problem by using nvarchar > instead of nchar data type. Nchar(N) requires that there are exactly N > characters, and SQL Server ODBC driver will add spaces if needed. > Nvarchar(N) would not ever add spaces. That's what we've done now and it fixes the problem even when using .executedirect(). However, that's just the SQL Server ODBC driver. We'll have to test this with other ODBC drivers as well before releasing a patch. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 14 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 13. elokuuta 2007 18:39 > To: Harri Pesonen > Cc: egenix-users@lists.egenix.com > Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and > unicode string > > Here are some preliminary findings: > > * we can replicate the problem with the SQL Server ODBC driver > version 03.85.1117 > > * the ODBC driver appears to add min(1, len(data)) spaces to > all Unicode data inserted into the table - up to the length > limit of the column > > The problem goes away if you use .execute() instead of > .executedirect(). > > We have to do some further testing, but this suggests that the > ODBC driver does not work correctly when using .executedirect(). > > > On 2007-08-13 16:48, M.-A. Lemburg wrote: >> On 2007-08-13 15:37, Harri Pesonen wrote: >>> Here is the script: >>> >>> import mx.ODBC.Windows >>> dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;" >>> connection = mx.ODBC.Windows.DriverConnect("%s;APP=%s" % (dsn, > "test"), >>> 0) >>> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >>> o = connection.cursor() >>> o.execute("CREATE TABLE #test (A nvarchar(64))") >>> s = "INSERT INTO #test (A) VALUES (?)" >>> uc = u'\u0430' >>> c = o.executedirect >>> rs = c(s, ("",)) >>> rs = c(s, (u"",)) >>> rs = c(s, ("a",)) >>> rs = c(s, (uc,)) >>> o.execute("SELECT * FROM #test") >>> rs = o.fetchall() >>> for r in rs: >>> print r >>> o = None >>> >>> Here is the output: >>> >>> (u'',) >>> (u' ',) >>> (u'a',) >>> (u'\u0430 ',) >>> >>> So there is bug #3, empty unicode string is converted to one space. > With >>> mxODBC 2 the first case fails as well, you get 64 spaces. >> Thanks. >> >> We'll run a few tests and then report back our findings. >> >>> -- Harri >>> >>> -----Original Message----- >>> From: M.-A. Lemburg [mailto:mal@egenix.com] >>> Sent: 13. elokuuta 2007 16:08 >>> To: Harri Pesonen >>> Cc: egenix-users@lists.egenix.com >>> Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string > and >>> unicode string >>> >>> On 2007-08-13 14:19, Harri Pesonen wrote: >>>> I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used >>>> char(0) datatype internally, which was converted to char(8000) by > SQL >>>> Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty > string, >>>> which works correctly. It does not add any extra spaces. >>>> >>>> 2) is still a problem in mxODBC3. It still uses char instead of >>> varchar >>>> for non-empty strings, so that I get "@P5 nchar(40)" and data >>>> "N'CustomerAgentRedial1 '". If mxODBC would use >>>> nvarchar(40), then SQL Server ODBC driver would not add spaces to > the >>>> end. Of course this bug could be fixed by using the correct > character >>>> count in the first place. >>>> >>>> So both problems happened because char was used instead of varchar. >>> Char >>>> requires that there will be spaces in the end, there have to be >>> exactly >>>> as many characters in the string. With empty string, char(0) was >>>> converted to char(8000) by the SQL Server driver because there is no >>>> such data type as char(0). >>>> >>>> Using >>>> connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE >>>> does not seem to have any effect. >>> Could you provide us with the following ? We will then try to >>> recreate the problem here and see what we can do about it. >>> >>> * table schema definition >>> >>> * inserts / selects exhibiting the problem >>> >>> Thanks, > From harri.pesonen at wicom.com Tue Aug 14 13:28:30 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Tue Aug 14 11:28:50 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: <46C170F8.5080508@egenix.com> References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> <46C06F3E.100@egenix.com> <46C07B11.5050409@egenix.com> <46C170F8.5080508@egenix.com> Message-ID: Great! How about mxODBC 2, is it supported at all, I can't find the download anymore. I guess that we need upgrade: "If you'd like to upgrade your mxODBC 2.0 licenses to version 3.0, please contact our sales team." -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 14. elokuuta 2007 12:08 To: Harri Pesonen Cc: egenix-users@lists.egenix.com Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string Some more investigation showed that even though the type binding was set to SQL (the ODBC driver providing the type information), when using .executedirect() the driver does not provide the type information. mxODBC uses Python type binding in such a case. On 2007-08-14 08:04, Harri Pesonen wrote: > I'm sure that you'll find out that you are using nchar(N) data type, > where N is calculated from the number of bytes in the unicode string, > not number of characters. The length information to use in ODBC APIs is a weak spot in the ODBC standard documentation... in some places it refers to byte counts, in others to code point counts and they aren't clear at all about which to use, see e.g. http://msdn2.microsoft.com/en-gb/library/ms716246.aspx > This is Microsoft SQL Server ODBC driver, so this is the standard. :-) Well, I wouldn't be so sure about this. Especially with respect to Unicode handling, the SQL Server's ODBC driver tends to have a very special interpretation. Things get even more complicated on 64-bit platforms. > One more thing: You could easily fix this problem by using nvarchar > instead of nchar data type. Nchar(N) requires that there are exactly N > characters, and SQL Server ODBC driver will add spaces if needed. > Nvarchar(N) would not ever add spaces. That's what we've done now and it fixes the problem even when using .executedirect(). However, that's just the SQL Server ODBC driver. We'll have to test this with other ODBC drivers as well before releasing a patch. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 14 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 13. elokuuta 2007 18:39 > To: Harri Pesonen > Cc: egenix-users@lists.egenix.com > Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and > unicode string > > Here are some preliminary findings: > > * we can replicate the problem with the SQL Server ODBC driver > version 03.85.1117 > > * the ODBC driver appears to add min(1, len(data)) spaces to > all Unicode data inserted into the table - up to the length > limit of the column > > The problem goes away if you use .execute() instead of > .executedirect(). > > We have to do some further testing, but this suggests that the > ODBC driver does not work correctly when using .executedirect(). > > > On 2007-08-13 16:48, M.-A. Lemburg wrote: >> On 2007-08-13 15:37, Harri Pesonen wrote: >>> Here is the script: >>> >>> import mx.ODBC.Windows >>> dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;" >>> connection = mx.ODBC.Windows.DriverConnect("%s;APP=%s" % (dsn, > "test"), >>> 0) >>> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >>> o = connection.cursor() >>> o.execute("CREATE TABLE #test (A nvarchar(64))") >>> s = "INSERT INTO #test (A) VALUES (?)" >>> uc = u'\u0430' >>> c = o.executedirect >>> rs = c(s, ("",)) >>> rs = c(s, (u"",)) >>> rs = c(s, ("a",)) >>> rs = c(s, (uc,)) >>> o.execute("SELECT * FROM #test") >>> rs = o.fetchall() >>> for r in rs: >>> print r >>> o = None >>> >>> Here is the output: >>> >>> (u'',) >>> (u' ',) >>> (u'a',) >>> (u'\u0430 ',) >>> >>> So there is bug #3, empty unicode string is converted to one space. > With >>> mxODBC 2 the first case fails as well, you get 64 spaces. >> Thanks. >> >> We'll run a few tests and then report back our findings. >> >>> -- Harri >>> >>> -----Original Message----- >>> From: M.-A. Lemburg [mailto:mal@egenix.com] >>> Sent: 13. elokuuta 2007 16:08 >>> To: Harri Pesonen >>> Cc: egenix-users@lists.egenix.com >>> Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string > and >>> unicode string >>> >>> On 2007-08-13 14:19, Harri Pesonen wrote: >>>> I tested with mxODBC 3, and 1) has been fixed. In mxODBC 2, it used >>>> char(0) datatype internally, which was converted to char(8000) by > SQL >>>> Server ODBC driver. In mxODBC 3, it uses varchar(1) with empty > string, >>>> which works correctly. It does not add any extra spaces. >>>> >>>> 2) is still a problem in mxODBC3. It still uses char instead of >>> varchar >>>> for non-empty strings, so that I get "@P5 nchar(40)" and data >>>> "N'CustomerAgentRedial1 '". If mxODBC would use >>>> nvarchar(40), then SQL Server ODBC driver would not add spaces to > the >>>> end. Of course this bug could be fixed by using the correct > character >>>> count in the first place. >>>> >>>> So both problems happened because char was used instead of varchar. >>> Char >>>> requires that there will be spaces in the end, there have to be >>> exactly >>>> as many characters in the string. With empty string, char(0) was >>>> converted to char(8000) by the SQL Server driver because there is no >>>> such data type as char(0). >>>> >>>> Using >>>> connection.bindmethod = mx.ODBC.Windows.BIND_USING_PYTHONTYPE >>>> does not seem to have any effect. >>> Could you provide us with the following ? We will then try to >>> recreate the problem here and see what we can do about it. >>> >>> * table schema definition >>> >>> * inserts / selects exhibiting the problem >>> >>> Thanks, > From mal at egenix.com Tue Aug 14 14:18:38 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Aug 14 13:18:31 2007 Subject: [egenix-users] Two bugs in mxODBC: storing empty string and unicode string In-Reply-To: References: <46C04724.1090708@egenix.com> <46C057B3.2000202@egenix.com> <46C06F3E.100@egenix.com> <46C07B11.5050409@egenix.com> <46C170F8.5080508@egenix.com> Message-ID: <46C18F8E.60103@egenix.com> On 2007-08-14 11:28, Harri Pesonen wrote: > Great! How about mxODBC 2, is it supported at all, I can't find the > download anymore. mxODBC 2.0 is no longer being actively maintained, however we still provide individual support based on support tickets for existing customers. > I guess that we need upgrade: "If you'd like to > upgrade your mxODBC 2.0 licenses to version 3.0, please contact our > sales team." I'll forward your request. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 14 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 14. elokuuta 2007 12:08 > To: Harri Pesonen > Cc: egenix-users@lists.egenix.com > Subject: Re: [egenix-users] Two bugs in mxODBC: storing empty string and > unicode string > > Some more investigation showed that even though the type binding > was set to SQL (the ODBC driver providing the type information), > when using .executedirect() the driver does not provide the > type information. mxODBC uses Python type binding in such a case. > > On 2007-08-14 08:04, Harri Pesonen wrote: >> I'm sure that you'll find out that you are using nchar(N) data type, >> where N is calculated from the number of bytes in the unicode string, >> not number of characters. > > The length information to use in ODBC APIs is a weak spot in the ODBC > standard documentation... in some places it refers to byte counts, > in others to code point counts and they aren't clear at all about > which to use, see e.g. > http://msdn2.microsoft.com/en-gb/library/ms716246.aspx > >> This is Microsoft SQL Server ODBC driver, so this is the standard. :-) > > Well, I wouldn't be so sure about this. Especially with respect to > Unicode handling, the SQL Server's ODBC driver tends to have a > very special interpretation. Things get even more complicated on > 64-bit platforms. > >> One more thing: You could easily fix this problem by using nvarchar >> instead of nchar data type. Nchar(N) requires that there are exactly N >> characters, and SQL Server ODBC driver will add spaces if needed. >> Nvarchar(N) would not ever add spaces. > > That's what we've done now and it fixes the problem even when > using .executedirect(). > > However, that's just the SQL Server ODBC driver. We'll have to > test this with other ODBC drivers as well before releasing a > patch. > > Thanks, From mal at egenix.com Wed Aug 15 17:21:29 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 15 16:21:22 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). In-Reply-To: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> Message-ID: <46C30BE9.6020007@egenix.com> Cliff, your problem seems to be related to the problem reported by Harri Pesonen. We will be releasing mxODBC 3.0.1 in the next few days which includes a work-around for the SQL Server problem with .executedirect(). This should also fix the problem you are seeing. Note that .executedirect() is normally only useful for one-shot queries that don't require parameters. Depending on the ODBC driver implementation it bypasses the normal processing done in the ODBC driver and sends off the query directly to the server. This can result in better performance, however, it also means that the ODBC driver doesn't have any type information available which usually results in less data type conversions and thus reduces memory foot-print and avoids extensive copying. For queries which take parameters it is therefore often better to use the standard .execute() method. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 On 2007-08-07 15:26, Cliff Xuan wrote: > I had a problem when using unicode object in mxODBC Cursor's executedirect() method. > > These are the lines from my test code: > > u'\xe5' is the Unicode object for Norwegian vow ?, spFindUser is a MS SQLServer stored procedure which returns the columns which contains the character in the parameter. > > Method execute() returns the right result, but executedirect doesn't. > > > > con = mx.ODBC.Windows.DriverConnect(dsn) > > con.encoding='utf-8' > > selectsql = "{call spFindUser(?)}" > > cur.execute(selectsql, (u'\xe5')) > > cur.executedirect(selectsql, (u'\xe5')) > > > > Any idea why it's happening? > > This is the explanation of method executedirect() in the official documentation: > > > > .executedirect(operation[,parameters]) > > Just like .execute(), except that no prepare step is issued and the operation is not cached. This can result in better performance with some ODBC driver setups, but also implies that Python type binding mode is used to bind the parameters. > > operation may be a Unicode object in case the ODBC driver and/or database support this. > > Return values are not defined. > > > > One questions on this: > > What does 'prepare step' mean? > > > > Many thanks > > Cliff > > > > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users From Cliff.Xuan at artsalliancemedia.com Wed Aug 15 16:28:22 2007 From: Cliff.Xuan at artsalliancemedia.com (Cliff Xuan) Date: Wed Aug 15 16:28:31 2007 Subject: [egenix-users] Using Cursor object method executedirect() Message-ID: <49A0DBD9577C0A4096A969872C94980A0267285C@aam-ex-1.aam.local> Skipped content of type multipart/related-------------- next part -------------- A non-text attachment was scrubbed... Name: executedirect.py Type: application/octet-stream Size: 3749 bytes Desc: executedirect.py Url : /mailman-archives/egenix-users/attachments/20070815/e7b8faa7/executedirect.obj From Cliff.Xuan at artsalliancemedia.com Wed Aug 15 16:47:42 2007 From: Cliff.Xuan at artsalliancemedia.com (Cliff Xuan) Date: Wed Aug 15 16:47:50 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). Message-ID: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> Hi Marc-Andre, Thanks very much for your answer. Dose that mean I should avoid using executedirect() if I do have parameters? I'm currently working on an I18N project and found a lot of db processing code doesn't work any more when applying to non English dataset. It's really worrying. Sorry to trouble you with more questions. I sent this email to the mailing list a few minutes ago but failed. I guess it might be the format problem? It was in html. Hi You might have seen a previous question I had on executedirect() dealing with Unicode parameter if you are subscribed to the mailing list. Unfortunately my problem is still unsolved and quite worrying about this. Would really appreciate if you can give me some clue. I have done the maximum simplification of my code and it looks like the attachment. The core part is this: 1 >>> con.encoding='utf-8' 2 >>> con.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT 3 >>> cur = con.cursor() 4 >>> uniqueID = str(pythoncom.CreateGuid())[1:-1] 5 >>> insertsql = "INSERT INTO simpletable (id, firstname) VALUES ('%s',?)" %uniqueID 6 >>> UNICODE = 1 7 >>> if UNICODE: 8 >>> cur.executedirect(insertsql, [(u'\xe6\xf8\xe5',)]) 9 >>> else: 10>>> cur.executedirect(insertsql, [('\xc3\xa6\xc3\xb8\xc3\xa5',)]) 11>>> con.commit() 12>>> selectsql = "exec spFindName ?" 13>>> # search for the record with the Norwegian character ? in the firstname column. 14>>> if UNICODE: 15>>> cur.executedirect(selectsql, [(u'\xe6',)]) 16>>> else: 17>>> cur.executedirect(selectsql, [('\xc3\xa6',)]) Explanations: 1. db table simpletable columns: id(uniqueidentifier) firstname(nchar(10)) 2. stored procedure spFindName is basically searching for a string in the firstname field: set ANSI_NULLS ON set QUOTED_IDENTIFIER ON go create procedure [dbo].[spFindName] ( @find nvarchar(64) ) as set nocount on select * from simpletable where firstname like ('%' + @find + '%') 3. | char | utf-8 byte string | Unicode object | | ? | '\xc3\xa6' | u'\xe6' | | ? | 'xc3\xb8' | u'\xf8' | | ? | '\xc3\xa5' | u' \xe5' | Questions: 1. When running this line python.exe costs 50% of CPU resource and never finishes. Why? >>> cur.executedirect(insertsql, [('\xc3\xa6\xc3\xb8\xc3\xa5',)]) 2. This line doesn't fetch any result >>> cur.executedirect(selectsql, [(u'\xe6',)]) But changing executedirect() to execute() will get the right result. Why? The official documentation explains that the difference between the two methods is that executedirect() is using Python binding. Is that the cause of the problem? If I change executedirect() to execute(), what's the cost on performance? 3. However, changing that line to the one below will get results. >>> cur.executedirect(selectsql, [(u'\xe6\xf8\xe5',)]) The difference between these two is one is the full of the field the other is just part of it. Why one is working but not the other? Could this be an error on my stored procedure? If so, how can I improve it? Or it's a misuse of executedirect() ? I know my use of the method looks silly, but without examples to follow it's really difficult. Answer to any of my questions is much appreciated. Many thanks Cliff Xuan Cliff Xuan Arts Alliance Media Ltd Company Registration No. 04801432 Registered office: 9-11 North End Road, London W14 8ST Registered in England & Wales -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 15 August 2007 15:21 To: Cliff Xuan Cc: egenix-users@egenix.com Subject: Re: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). Cliff, your problem seems to be related to the problem reported by Harri Pesonen. We will be releasing mxODBC 3.0.1 in the next few days which includes a work-around for the SQL Server problem with .executedirect(). This should also fix the problem you are seeing. Note that .executedirect() is normally only useful for one-shot queries that don't require parameters. Depending on the ODBC driver implementation it bypasses the normal processing done in the ODBC driver and sends off the query directly to the server. This can result in better performance, however, it also means that the ODBC driver doesn't have any type information available which usually results in less data type conversions and thus reduces memory foot-print and avoids extensive copying. For queries which take parameters it is therefore often better to use the standard .execute() method. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 On 2007-08-07 15:26, Cliff Xuan wrote: > I had a problem when using unicode object in mxODBC Cursor's executedirect() method. > > These are the lines from my test code: > > u'\xe5' is the Unicode object for Norwegian vow ?, spFindUser is a MS SQLServer stored procedure which returns the columns which contains the character in the parameter. > > Method execute() returns the right result, but executedirect doesn't. > > > > con = mx.ODBC.Windows.DriverConnect(dsn) > > con.encoding='utf-8' > > selectsql = "{call spFindUser(?)}" > > cur.execute(selectsql, (u'\xe5')) > > cur.executedirect(selectsql, (u'\xe5')) > > > > Any idea why it's happening? > > This is the explanation of method executedirect() in the official documentation: > > > > .executedirect(operation[,parameters]) > > Just like .execute(), except that no prepare step is issued and the operation is not cached. This can result in better performance with some ODBC driver setups, but also implies that Python type binding mode is used to bind the parameters. > > operation may be a Unicode object in case the ODBC driver and/or database support this. > > Return values are not defined. > > > > One questions on this: > > What does 'prepare step' mean? > > > > Many thanks > > Cliff > > > > > > > ------------------------------------------------------------------------ > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > https://www.egenix.com/mailman/listinfo/egenix-users -------------- next part -------------- A non-text attachment was scrubbed... Name: executedirect.py Type: application/octet-stream Size: 3749 bytes Desc: executedirect.py Url : /mailman-archives/egenix-users/attachments/20070815/ebe51b53/executedirect.obj From mal at egenix.com Wed Aug 15 17:57:21 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Aug 15 16:57:15 2007 Subject: [egenix-users] mxODBC Cursor Unicode issue with method executedirect(). In-Reply-To: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> References: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> Message-ID: <46C31451.8070607@egenix.com> On 2007-08-15 16:47, Cliff Xuan wrote: > Hi Marc-Andre, > > Thanks very much for your answer. > Dose that mean I should avoid using executedirect() if I do have parameters? You should always prefer .execute() over .executedirect(). If you do want to try .executedirect(), then please do timings checks and data transfer checks (like the ones you are apparently doing), since using .executedirect() may very well be slower for what you want to do, use more memory and result in unwanted recoding of data. Please note that .executedirect() results in mxODBC using Python type binding instead of the standard SQL type binding used for .execute(), so the way the communication between Python and the ODBC driver works can be somewhat different. > I'm currently working on an I18N project and found a lot of db processing code doesn't work any more when applying to non English dataset. It's really worrying. > > Sorry to trouble you with more questions. > > I sent this email to the mailing list a few minutes ago but failed. I guess it might be the format problem? It was in html. It's likely that it ended up in our spam system. HTML emails are rather uncommon on this list. > Hi > > You might have seen a previous question I had on executedirect() dealing with Unicode parameter if you are subscribed to the mailing list. Unfortunately my problem is still unsolved and quite worrying about this. > > Would really appreciate if you can give me some clue. > > I have done the maximum simplification of my code and it looks like the attachment. > > The core part is this: > > 1 >>> con.encoding='utf-8' > 2 >>> con.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT > 3 >>> cur = con.cursor() > 4 >>> uniqueID = str(pythoncom.CreateGuid())[1:-1] > 5 >>> insertsql = "INSERT INTO simpletable (id, firstname) VALUES ('%s',?)" %uniqueID > 6 >>> UNICODE = 1 > 7 >>> if UNICODE: > 8 >>> cur.executedirect(insertsql, [(u'\xe6\xf8\xe5',)]) > 9 >>> else: > 10>>> cur.executedirect(insertsql, [('\xc3\xa6\xc3\xb8\xc3\xa5',)]) > 11>>> con.commit() > 12>>> selectsql = "exec spFindName ?" > 13>>> # search for the record with the Norwegian character ? in the firstname column. > 14>>> if UNICODE: > 15>>> cur.executedirect(selectsql, [(u'\xe6',)]) > 16>>> else: > 17>>> cur.executedirect(selectsql, [('\xc3\xa6',)]) > > Explanations: > > 1. db table simpletable columns: > id(uniqueidentifier) > firstname(nchar(10)) > > 2. stored procedure spFindName is basically searching for a string in the firstname field: > set ANSI_NULLS ON > set QUOTED_IDENTIFIER ON > go > create procedure [dbo].[spFindName] > ( @find nvarchar(64) ) > as > set nocount on > select * from simpletable > where firstname like ('%' + @find + '%') > > 3. > | char | utf-8 byte string | Unicode object | > | ? | '\xc3\xa6' | u'\xe6' | > | ? | 'xc3\xb8' | u'\xf8' | > | ? | '\xc3\xa5' | u' \xe5' | > > Questions: > > 1. When running this line python.exe costs 50% of CPU resource and never finishes. Why? >>>> cur.executedirect(insertsql, [('\xc3\xa6\xc3\xb8\xc3\xa5',)]) This is a bug in the SQL Server ODBC driver. We have put a work-around for it in mxODBC 3.0.1. > 2. This line doesn't fetch any result >>>> cur.executedirect(selectsql, [(u'\xe6',)]) > > But changing executedirect() to execute() will get the right result. Why? The official documentation explains that the difference between the two methods is that executedirect() is using Python binding. Is that the cause of the problem? If I change executedirect() to execute(), what's the cost on performance? > > 3. However, changing that line to the one below will get results. >>>> cur.executedirect(selectsql, [(u'\xe6\xf8\xe5',)]) The above two should be fixed in 3.0.1 as well. > The difference between these two is one is the full of the field the other is just part of it. Why one is working but not the other? Could this be an error on my stored procedure? If so, how can I improve it? Or it's a misuse of executedirect() ? > > I know my use of the method looks silly, but without examples to follow it's really difficult. > > Answer to any of my questions is much appreciated. > > Many thanks Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 15 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Fri Aug 17 00:18:21 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 16 23:18:33 2007 Subject: [egenix-users] mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: <46C30BE9.6020007@egenix.com> References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> Message-ID: <46C4BF1D.7060000@egenix.com> Hello, in order to address the issues discussed in the last few days with using .executedirect() on connections to SQL Server, we are making a release candidate available. The final release of mxODBC 3.0.1 is planned for Monday, provided that the workarounds we've added do fix all the problems you've encountered. Please give the rc1 a try and let us know whether there are any outstanding issues. http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.exe http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.msi Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 16 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Fri Aug 17 13:21:26 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Fri Aug 17 11:21:41 2007 Subject: [egenix-users] RE: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: <46C4BF1D.7060000@egenix.com> References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> <46C4BF1D.7060000@egenix.com> Message-ID: Thanks, everything seems to work, we continue testing. -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 17. elokuuta 2007 0:18 To: egenix-users@egenix.com Cc: Cliff Xuan; Harri Pesonen Subject: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server Hello, in order to address the issues discussed in the last few days with using .executedirect() on connections to SQL Server, we are making a release candidate available. The final release of mxODBC 3.0.1 is planned for Monday, provided that the workarounds we've added do fix all the problems you've encountered. Please give the rc1 a try and let us know whether there are any outstanding issues. http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.e xe http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.m si Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 16 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Fri Aug 17 11:07:01 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Fri Aug 17 12:07:30 2007 Subject: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). References: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> <46C31451.8070607@egenix.com> Message-ID: M.-A. Lemburg writes: > > On 2007-08-15 16:47, Cliff Xuan wrote: > > Hi Marc-Andre, > > > > Thanks very much for your answer. > > Dose that mean I should avoid using executedirect() if I do have parameters? > > You should always prefer .execute() over .executedirect(). > > If you do want to try .executedirect(), then please do timings checks > and data transfer checks (like the ones you are apparently doing), > since using .executedirect() may very well be slower for what > you want to do, use more memory and result in unwanted recoding > of data. Hi, about executedirect() vs execute(). Generally executedirect() is faster (ODBC documentation says "SQLExecDirect is the fastest way to submit an SQL statement for one-time execution"). Execute() is faster only if the statement is prepared first, and then executed several times. Usually it is faster and easier to use executedirect(), because prepare+execute(N times) is harder to implement. I guess that mxODBC executemany() does exactly the same thing as prepare+execute(N times). Well we have experience only with SQL Server, so different databases may behave differently. -- Harri From mal at egenix.com Fri Aug 17 13:19:45 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 17 12:19:22 2007 Subject: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). In-Reply-To: References: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> <46C31451.8070607@egenix.com> Message-ID: <46C57641.20301@egenix.com> On 2007-08-17 12:07, Harri Pesonen wrote: > M.-A. Lemburg writes: > >> On 2007-08-15 16:47, Cliff Xuan wrote: >>> Hi Marc-Andre, >>> >>> Thanks very much for your answer. >>> Dose that mean I should avoid using executedirect() if I do have parameters? >> You should always prefer .execute() over .executedirect(). >> >> If you do want to try .executedirect(), then please do timings checks >> and data transfer checks (like the ones you are apparently doing), >> since using .executedirect() may very well be slower for what >> you want to do, use more memory and result in unwanted recoding >> of data. > > Hi, about executedirect() vs execute(). Generally executedirect() is faster > (ODBC documentation says "SQLExecDirect is the fastest way to submit an SQL > statement for one-time execution"). Execute() is faster only if the statement is > prepared first, and then executed several times. Usually it is faster and easier > to use executedirect(), because prepare+execute(N times) is harder to implement. There's an optimization in mxODBC which does this for you: If you pass in the same SQL command multiple times (and in sequence) to the .execute() command, with varying parameters tuples, it will only prepare the command once and then reuse the already prepared command for all subsequent invocations of .execute(). mxODBC 3.0 also features a .prepare() method which can be used to e.g. keep a set of prepared cursors in a pool for fast execution of common queries. .prepare() allows to build query plans without actually executing the statement in question. > I guess that mxODBC executemany() does exactly the same thing as > prepare+execute(N times). Right. The problem with .executemany() is that error detection is a bit harder, ie. say the insert of row 10 fails in a set of 100 rows. mxODBC will include the row information in the error message, but only textually. Using a loop with .execute() will allow you to catch and process such errors on a row-by-row basis. > Well we have experience only with SQL Server, so different databases may behave > differently. Thanks for the feedback. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Fri Aug 17 14:35:09 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Fri Aug 17 12:35:19 2007 Subject: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). In-Reply-To: <46C57641.20301@egenix.com> References: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> <46C31451.8070607@egenix.com> <46C57641.20301@egenix.com> Message-ID: Good to know, I was not aware of .execute() doing automatic caching of the prepared statement. So it may be faster than .executedirect(), depending on how the application is written. We have a multi-threaded application and it is not clear, in which order the executes are made, it may change from day to day, so it is hard to tell which one is better. Obviously we could have different cursors for the different cases, but it would be harder to implement. Hmm we already have a different database connection (and cursor) for some cases... So it could benefit of using execute instead of executedirect... -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 17. elokuuta 2007 13:20 To: Harri Pesonen Cc: egenix-users@lists.egenix.com Subject: Re: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). On 2007-08-17 12:07, Harri Pesonen wrote: > M.-A. Lemburg writes: > >> On 2007-08-15 16:47, Cliff Xuan wrote: >>> Hi Marc-Andre, >>> >>> Thanks very much for your answer. >>> Dose that mean I should avoid using executedirect() if I do have parameters? >> You should always prefer .execute() over .executedirect(). >> >> If you do want to try .executedirect(), then please do timings checks >> and data transfer checks (like the ones you are apparently doing), >> since using .executedirect() may very well be slower for what >> you want to do, use more memory and result in unwanted recoding >> of data. > > Hi, about executedirect() vs execute(). Generally executedirect() is faster > (ODBC documentation says "SQLExecDirect is the fastest way to submit an SQL > statement for one-time execution"). Execute() is faster only if the statement is > prepared first, and then executed several times. Usually it is faster and easier > to use executedirect(), because prepare+execute(N times) is harder to implement. There's an optimization in mxODBC which does this for you: If you pass in the same SQL command multiple times (and in sequence) to the .execute() command, with varying parameters tuples, it will only prepare the command once and then reuse the already prepared command for all subsequent invocations of .execute(). mxODBC 3.0 also features a .prepare() method which can be used to e.g. keep a set of prepared cursors in a pool for fast execution of common queries. .prepare() allows to build query plans without actually executing the statement in question. > I guess that mxODBC executemany() does exactly the same thing as > prepare+execute(N times). Right. The problem with .executemany() is that error detection is a bit harder, ie. say the insert of row 10 fails in a set of 100 rows. mxODBC will include the row information in the error message, but only textually. Using a loop with .execute() will allow you to catch and process such errors on a row-by-row basis. > Well we have experience only with SQL Server, so different databases may behave > differently. Thanks for the feedback. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From charlie at egenix.com Fri Aug 17 14:24:56 2007 From: charlie at egenix.com (Charlie Clark) Date: Fri Aug 17 13:25:35 2007 Subject: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). In-Reply-To: References: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> <46C31451.8070607@egenix.com> <46C57641.20301@egenix.com> Message-ID: Am 17.08.2007, 12:35 Uhr, schrieb Harri Pesonen : > Good to know, I was not aware of .execute() doing automatic caching of > the prepared statement. So it may be faster than .executedirect(), > depending on how the application is written. We have a multi-threaded > application and it is not clear, in which order the executes are made, > it may change from day to day, so it is hard to tell which one is > better. Obviously we could have different cursors for the different > cases, but it would be harder to implement. Hmm we already have a > different database connection (and cursor) for some cases... So it could > benefit of using execute instead of executedirect... In general you should use .execute() and .executedirect() only if you have a use case where it is demonstrably better, which will usually be if the database's ODBC driver is sloppy (and there are lots of them). As this recent issue has illustrated: .executedirect() is harder to use and support if there are problems and is provided to be used only when necessary. I think this is the more Pythonic approach. .executemany() is extremely elegant when passed an iterator (useful when copying from one database to another) but packing things in a Python list is hardly clumsy. My experience with PostgreSQL is that .executemany() is faster for batch inserts but the limiting factor for high volume is usually the management overhead with the database checking constraints. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Fri Aug 17 15:34:29 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Fri Aug 17 13:34:37 2007 Subject: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). In-Reply-To: References: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> <46C31451.8070607@egenix.com> <46C57641.20301@egenix.com> Message-ID: We first used .execute() but then changed to .executedirect() when we noticed that .execute() does two calls to SQL Server, SQLPrepare and SQLExecute. .executedirect() does only 1 call, SQLExecDirect. So .execute() may be two times slower if the SQL statement is always different. -- Harri -----Original Message----- From: Charlie Clark [mailto:charlie@egenix.com] Sent: 17. elokuuta 2007 14:25 To: Harri Pesonen Cc: egenix-users@lists.egenix.com Subject: Re: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). Am 17.08.2007, 12:35 Uhr, schrieb Harri Pesonen : > Good to know, I was not aware of .execute() doing automatic caching of > the prepared statement. So it may be faster than .executedirect(), > depending on how the application is written. We have a multi-threaded > application and it is not clear, in which order the executes are made, > it may change from day to day, so it is hard to tell which one is > better. Obviously we could have different cursors for the different > cases, but it would be harder to implement. Hmm we already have a > different database connection (and cursor) for some cases... So it could > benefit of using execute instead of executedirect... In general you should use .execute() and .executedirect() only if you have a use case where it is demonstrably better, which will usually be if the database's ODBC driver is sloppy (and there are lots of them). As this recent issue has illustrated: .executedirect() is harder to use and support if there are problems and is provided to be used only when necessary. I think this is the more Pythonic approach. .executemany() is extremely elegant when passed an iterator (useful when copying from one database to another) but packing things in a Python list is hardly clumsy. My experience with PostgreSQL is that .executemany() is faster for batch inserts but the limiting factor for high volume is usually the management overhead with the database checking constraints. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From charlie at egenix.com Fri Aug 17 14:42:29 2007 From: charlie at egenix.com (Charlie Clark) Date: Fri Aug 17 13:43:13 2007 Subject: [egenix-users] Re: mxODBC Cursor Unicode issuewith method executedirect(). In-Reply-To: References: <49A0DBD9577C0A4096A969872C94980A02672882@aam-ex-1.aam.local> <46C31451.8070607@egenix.com> <46C57641.20301@egenix.com> Message-ID: Am 17.08.2007, 13:34 Uhr, schrieb Harri Pesonen : > We first used .execute() but then changed to .executedirect() when we > noticed that .execute() does two calls to SQL Server, SQLPrepare and > SQLExecute. .executedirect() does only 1 call, SQLExecDirect. So > .execute() may be two times slower if the SQL statement is always > different. That may well be the case. I was just highlighting the case for not optimising too early for others on the list as, in general, SQL statements should not change as often as the parameters do so preparing statements will generally be more efficient. Furthermore, it is not always clear how the database itself implements calls. From a programmer's point of view, particularly for beginners, .execute() is a Godsend. Charlie -- Charlie Clark eGenix.com Professional Python Services directly from the Source >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Fri Aug 17 17:33:45 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Fri Aug 17 15:33:54 2007 Subject: [egenix-users] RE: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: <46C4BF1D.7060000@egenix.com> References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> <46C4BF1D.7060000@egenix.com> Message-ID: Hi, we found one problem in version mxODBC 3. The following works in mxODBC 2. import mx.ODBC.Windows dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;APP=test" connection = mx.ODBC.Windows.DriverConnect(dsn, 0) connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT o = connection.cursor() o.execute("""CREATE PROC #proc @p as nvarchar(200) AS BEGIN CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) INSERT INTO #test2 (A,B) VALUES ('A','B') SELECT * FROM #test2 END""") o.execute("EXEC #proc 'huhuu'") rs = o.fetchall() for r in rs: print r o = None In mxODBC 3 it fails with error: ProgrammingError: missing result set If I remove the INSERT statement, then it works! (No error but empty result set). Odd. It must be a bug somewhere. -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 17. elokuuta 2007 0:18 To: egenix-users@egenix.com Cc: Cliff Xuan; Harri Pesonen Subject: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server Hello, in order to address the issues discussed in the last few days with using .executedirect() on connections to SQL Server, we are making a release candidate available. The final release of mxODBC 3.0.1 is planned for Monday, provided that the workarounds we've added do fix all the problems you've encountered. Please give the rc1 a try and let us know whether there are any outstanding issues. http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.e xe http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.m si Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 16 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Fri Aug 17 16:42:38 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 17 15:42:06 2007 Subject: [egenix-users] Re: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> <46C4BF1D.7060000@egenix.com> Message-ID: <46C5A5CE.2040704@egenix.com> On 2007-08-17 15:33, Harri Pesonen wrote: > Hi, we found one problem in version mxODBC 3. The following works in > mxODBC 2. That's because mxODBC 3.0 supports multiple result sets. > import mx.ODBC.Windows > dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;APP=test" > connection = mx.ODBC.Windows.DriverConnect(dsn, 0) > connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT > o = connection.cursor() > o.execute("""CREATE PROC #proc > @p as nvarchar(200) > AS > BEGIN > CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) > INSERT INTO #test2 (A,B) VALUES ('A','B') > SELECT * FROM #test2 > END""") > o.execute("EXEC #proc 'huhuu'") > rs = o.fetchall() > for r in rs: > print r > o = None > > In mxODBC 3 it fails with error: > > ProgrammingError: missing result set > > If I remove the INSERT statement, then it works! (No error but empty > result set). Odd. It must be a bug somewhere. The INSERT creates an empty result set with no columns (this results in the "missing result set" message - a bit misleading perhaps). To get to the next result set, you have to use .nextset(): o.execute("EXEC #proc 'huhuu'") o.nextset() rs = o.fetchall() for r in rs: print r o = None > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 17. elokuuta 2007 0:18 > To: egenix-users@egenix.com > Cc: Cliff Xuan; Harri Pesonen > Subject: mxODBC 3.0.1 RC1 to address problems with .executedirect() and > SQL Server > > Hello, > > in order to address the issues discussed in the last few days > with using .executedirect() on connections to SQL Server, we > are making a release candidate available. > > The final release of mxODBC 3.0.1 is planned for Monday, provided > that the workarounds we've added do fix all the problems you've > encountered. > > Please give the rc1 a try and let us know whether there are > any outstanding issues. > > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.e > xe > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.m > si > > Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From harri.pesonen at wicom.com Fri Aug 17 17:59:54 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Fri Aug 17 16:00:03 2007 Subject: [egenix-users] RE: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: <46C5A5CE.2040704@egenix.com> References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> <46C4BF1D.7060000@egenix.com> <46C5A5CE.2040704@egenix.com> Message-ID: Yes, and the problem can be solved in the procedure by adding SET NOCOUNT ON as a first statement: o.execute("""CREATE PROC #proc @p as nvarchar(200) AS BEGIN SET NOCOUNT ON CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) INSERT INTO #test2 (A,B) VALUES ('A','B') SELECT * FROM #test2 END""") I don't know, perhaps mxODBC should check if the result set is a real result set, or just information about the number of inserted rows. Perhaps it is not easy to detect it. -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 17. elokuuta 2007 16:43 To: Harri Pesonen Cc: egenix-users@egenix.com Subject: Re: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server On 2007-08-17 15:33, Harri Pesonen wrote: > Hi, we found one problem in version mxODBC 3. The following works in > mxODBC 2. That's because mxODBC 3.0 supports multiple result sets. > import mx.ODBC.Windows > dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;APP=test" > connection = mx.ODBC.Windows.DriverConnect(dsn, 0) > connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT > o = connection.cursor() > o.execute("""CREATE PROC #proc > @p as nvarchar(200) > AS > BEGIN > CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) > INSERT INTO #test2 (A,B) VALUES ('A','B') > SELECT * FROM #test2 > END""") > o.execute("EXEC #proc 'huhuu'") > rs = o.fetchall() > for r in rs: > print r > o = None > > In mxODBC 3 it fails with error: > > ProgrammingError: missing result set > > If I remove the INSERT statement, then it works! (No error but empty > result set). Odd. It must be a bug somewhere. The INSERT creates an empty result set with no columns (this results in the "missing result set" message - a bit misleading perhaps). To get to the next result set, you have to use .nextset(): o.execute("EXEC #proc 'huhuu'") o.nextset() rs = o.fetchall() for r in rs: print r o = None > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 17. elokuuta 2007 0:18 > To: egenix-users@egenix.com > Cc: Cliff Xuan; Harri Pesonen > Subject: mxODBC 3.0.1 RC1 to address problems with .executedirect() and > SQL Server > > Hello, > > in order to address the issues discussed in the last few days > with using .executedirect() on connections to SQL Server, we > are making a release candidate available. > > The final release of mxODBC 3.0.1 is planned for Monday, provided > that the workarounds we've added do fix all the problems you've > encountered. > > Please give the rc1 a try and let us know whether there are > any outstanding issues. > > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.e > xe > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.m > si > > Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Fri Aug 17 17:32:43 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Aug 17 16:32:26 2007 Subject: [egenix-users] Re: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> <46C4BF1D.7060000@egenix.com> <46C5A5CE.2040704@egenix.com> Message-ID: <46C5B18B.4030206@egenix.com> On 2007-08-17 15:59, Harri Pesonen wrote: > Yes, and the problem can be solved in the procedure by adding > > SET NOCOUNT ON > > as a first statement: > > o.execute("""CREATE PROC #proc > @p as nvarchar(200) > AS > BEGIN > SET NOCOUNT ON > CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) > INSERT INTO #test2 (A,B) VALUES ('A','B') > SELECT * FROM #test2 > END""") > > I don't know, perhaps mxODBC should check if the result set is a real > result set, or just information about the number of inserted rows. > Perhaps it is not easy to detect it. The information mxODBC is looking for is the number of result set columns. If that number is zero, it regards the result set as not existing, since in most cases, that's the correct interpretation and avoids strange errors you'd otherwise get from the ODBC fetch calls. BTW, you can access the number of rows that were inserted by querying c.rowcount just before the .nextset() call. > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 17. elokuuta 2007 16:43 > To: Harri Pesonen > Cc: egenix-users@egenix.com > Subject: Re: mxODBC 3.0.1 RC1 to address problems with .executedirect() > and SQL Server > > On 2007-08-17 15:33, Harri Pesonen wrote: >> Hi, we found one problem in version mxODBC 3. The following works in >> mxODBC 2. > > That's because mxODBC 3.0 supports multiple result sets. > >> import mx.ODBC.Windows >> dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;APP=test" >> connection = mx.ODBC.Windows.DriverConnect(dsn, 0) >> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >> o = connection.cursor() >> o.execute("""CREATE PROC #proc >> @p as nvarchar(200) >> AS >> BEGIN >> CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) >> INSERT INTO #test2 (A,B) VALUES ('A','B') >> SELECT * FROM #test2 >> END""") >> o.execute("EXEC #proc 'huhuu'") >> rs = o.fetchall() >> for r in rs: >> print r >> o = None >> >> In mxODBC 3 it fails with error: >> >> ProgrammingError: missing result set >> >> If I remove the INSERT statement, then it works! (No error but empty >> result set). Odd. It must be a bug somewhere. > > The INSERT creates an empty result set with no columns (this results > in the "missing result set" message - a bit misleading perhaps). > > To get to the next result set, you have to use .nextset(): > > o.execute("EXEC #proc 'huhuu'") > o.nextset() > rs = o.fetchall() > for r in rs: > print r > o = None > >> -- Harri >> >> -----Original Message----- >> From: M.-A. Lemburg [mailto:mal@egenix.com] >> Sent: 17. elokuuta 2007 0:18 >> To: egenix-users@egenix.com >> Cc: Cliff Xuan; Harri Pesonen >> Subject: mxODBC 3.0.1 RC1 to address problems with .executedirect() > and >> SQL Server >> >> Hello, >> >> in order to address the issues discussed in the last few days >> with using .executedirect() on connections to SQL Server, we >> are making a release candidate available. >> >> The final release of mxODBC 3.0.1 is planned for Monday, provided >> that the workarounds we've added do fix all the problems you've >> encountered. >> >> Please give the rc1 a try and let us know whether there are >> any outstanding issues. >> >> > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.e >> xe >> > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.m >> si >> >> Thanks, > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From rdmoores at gmail.com Sun Aug 19 20:59:30 2007 From: rdmoores at gmail.com (Dick Moores) Date: Mon Aug 20 04:59:36 2007 Subject: [egenix-users] How to install mxDateTime on Windows XP Message-ID: Win XP Pro SP2, Python 2.5 I'm at a loss as to how to install mxDateTime. I have an unzipped folder named "egenix-mx-base-3.0.0.solaris-2.9-sun4u-py2.5_ucs4.prebuilt" Did I download the correct thing? Or what? Help, please! Dick Moores -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070819/796ee65f/attachment.htm From rdmoores at gmail.com Sun Aug 19 21:38:00 2007 From: rdmoores at gmail.com (Dick Moores) Date: Mon Aug 20 05:38:10 2007 Subject: [egenix-users] Re: How to install mxDateTime on Windows XP In-Reply-To: References: Message-ID: On 8/19/07, Dick Moores wrote: > > Win XP Pro SP2, Python 2.5 > > I'm at a loss as to how to install mxDateTime. I have an unzipped folder > named "egenix-mx-base-3.0.0.solaris-2.9-sun4u-py2.5_ucs4.prebuilt" > > Did I download the correct thing? Or what? > > Help, please! > > Dick Moores Yes, I did download the wrong file. Just clicked on the wrong one by mistake. :-( So false alarm. Sorry. I appear to have the correct stuff installed correctly. Dick Moores -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070819/804f7af6/attachment.htm From harri.pesonen at wicom.com Mon Aug 20 10:34:43 2007 From: harri.pesonen at wicom.com (Harri Pesonen) Date: Mon Aug 20 08:35:17 2007 Subject: [egenix-users] RE: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: <46C5B18B.4030206@egenix.com> References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> <46C4BF1D.7060000@egenix.com> <46C5A5CE.2040704@egenix.com> <46C5B18B.4030206@egenix.com> Message-ID: But if .colcount is zero, and you get the error "ProgrammingError: missing result set" when you try to use it, wouldn't it be better if mxODBC would skip that result set automatically? It would be more compatible with mxODBC 2. Now I think that every .fetchall() should be prepared with while o.colcount == 0: if not o.nextset(): break You would lose o.rowcount in this case, though. Perhaps it should be an option (on by default), so that all non-existing result sets should be skipped automatically. -- Harri -----Original Message----- From: M.-A. Lemburg [mailto:mal@egenix.com] Sent: 17. elokuuta 2007 17:33 To: Harri Pesonen Cc: egenix-users@egenix.com Subject: Re: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server On 2007-08-17 15:59, Harri Pesonen wrote: > Yes, and the problem can be solved in the procedure by adding > > SET NOCOUNT ON > > as a first statement: > > o.execute("""CREATE PROC #proc > @p as nvarchar(200) > AS > BEGIN > SET NOCOUNT ON > CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) > INSERT INTO #test2 (A,B) VALUES ('A','B') > SELECT * FROM #test2 > END""") > > I don't know, perhaps mxODBC should check if the result set is a real > result set, or just information about the number of inserted rows. > Perhaps it is not easy to detect it. The information mxODBC is looking for is the number of result set columns. If that number is zero, it regards the result set as not existing, since in most cases, that's the correct interpretation and avoids strange errors you'd otherwise get from the ODBC fetch calls. BTW, you can access the number of rows that were inserted by querying c.rowcount just before the .nextset() call. > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 17. elokuuta 2007 16:43 > To: Harri Pesonen > Cc: egenix-users@egenix.com > Subject: Re: mxODBC 3.0.1 RC1 to address problems with .executedirect() > and SQL Server > > On 2007-08-17 15:33, Harri Pesonen wrote: >> Hi, we found one problem in version mxODBC 3. The following works in >> mxODBC 2. > > That's because mxODBC 3.0 supports multiple result sets. > >> import mx.ODBC.Windows >> dsn = "Driver={SQL SERVER};Server=dskpesonha;Database=master;APP=test" >> connection = mx.ODBC.Windows.DriverConnect(dsn, 0) >> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >> o = connection.cursor() >> o.execute("""CREATE PROC #proc >> @p as nvarchar(200) >> AS >> BEGIN >> CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) >> INSERT INTO #test2 (A,B) VALUES ('A','B') >> SELECT * FROM #test2 >> END""") >> o.execute("EXEC #proc 'huhuu'") >> rs = o.fetchall() >> for r in rs: >> print r >> o = None >> >> In mxODBC 3 it fails with error: >> >> ProgrammingError: missing result set >> >> If I remove the INSERT statement, then it works! (No error but empty >> result set). Odd. It must be a bug somewhere. > > The INSERT creates an empty result set with no columns (this results > in the "missing result set" message - a bit misleading perhaps). > > To get to the next result set, you have to use .nextset(): > > o.execute("EXEC #proc 'huhuu'") > o.nextset() > rs = o.fetchall() > for r in rs: > print r > o = None > >> -- Harri >> >> -----Original Message----- >> From: M.-A. Lemburg [mailto:mal@egenix.com] >> Sent: 17. elokuuta 2007 0:18 >> To: egenix-users@egenix.com >> Cc: Cliff Xuan; Harri Pesonen >> Subject: mxODBC 3.0.1 RC1 to address problems with .executedirect() > and >> SQL Server >> >> Hello, >> >> in order to address the issues discussed in the last few days >> with using .executedirect() on connections to SQL Server, we >> are making a release candidate available. >> >> The final release of mxODBC 3.0.1 is planned for Monday, provided >> that the workarounds we've added do fix all the problems you've >> encountered. >> >> Please give the rc1 a try and let us know whether there are >> any outstanding issues. >> >> > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.e >> xe >> > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.m >> si >> >> Thanks, > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 17 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From phil at xfr.co.uk Mon Aug 20 12:09:49 2007 From: phil at xfr.co.uk (Philip Kilner) Date: Mon Aug 20 12:10:28 2007 Subject: [egenix-users] How to install mxDateTime on Windows XP In-Reply-To: References: Message-ID: <46C9686D.8050405@xfr.co.uk> Hi Dick, Dick Moores wrote: > Win XP Pro SP2, Python 2.5 > > I'm at a loss as to how to install mxDateTime. I have an unzipped folder > named "egenix-mx-base-3.0.0.solaris-2.9-sun4u-py2.5_ucs4.prebuilt" > > Did I download the correct thing? Or what? > Looks like you have a Solaris version - you'll need the Windows version. > Help, please! > Try here: - http://www.egenix.com/products/python/mxBase/ ...and look for the Windows options - in the drop-down, you should see "Python 2.5 .msi installer". HTH! -- Regards, PhilK history doesn't often repeat itself, but it often rhymes - john a. nagl (after mark twain) From info at egenix.com Wed Aug 22 15:58:44 2007 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Wed Aug 22 14:59:00 2007 Subject: [egenix-users] ANN: eGenix mxODBC Distribution 3.0.1 (mxODBC Database Interface) Message-ID: <46CC3304.6020800@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix.com mxODBC Database Interface Version 3.0.1 Our commercially supported Python extension providing ODBC database connectivity to Python applications on Windows and Unix platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Distribution-3.0.1-GA.html ________________________________________________________________________ INTRODUCTION The mxODBC Database Interface allows users to easily connect Python applications to just about any database on the market today - on both Windows and Unix platforms in a highly portable and convenient way. This makes mxODBC the ideal basis for writing cross-platform database programs and utilities in Python. mxODBC is included in the eGenix.com mxODBC Distribution for Python, which is part of the eGenix.com mx Extension Series - a collection of professional quality software tools aimed at enhancing Python's usability in many important areas such as database connectivity, fast text processing, date/time processing and web site programming. The package has proven its stability and usefulness in many mission critical applications and various commercial settings all around the world. It's been used in production for almost 10 years now. * About Python: Python is an object-oriented Open Source programming language which runs on all modern platforms (http://www.python.org/). By integrating ease-of-use, clarity in coding, enterprise application connectivity and rapid application design, Python establishes an ideal programming platform for todays IT challenges. * About eGenix: eGenix is a consulting and software product company focused on providing professional quality services and products to Python users and developers (http://www.egenix.com/). ________________________________________________________________________ NEWS mxODBC 3.0.1 is a patch-level release and includes the following updates: Enhanced SQL Server ODBC driver support: * work-around for using Unicode parameters together with cursor.executedirect() * bug-fix for cursor.stringformat setting when used with cursor.executedirect() * improved compatibility by using non-padding character binding codes Documentation: * added more documentation and an example of how to use the new connection and cursor error handlers. Error handlers were introduced in mxODBC 3.0.0 and allow for much greater control over how low-level errors in the interface are to be dealt with. For the full set of changes please check the mxODBC change log. ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/mxODBC/ IMPORTANT: In order to use the eGenix mxODBC package you will first need to install the eGenix mx Base package: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ UPGRADING You are encouraged to upgrade to this latest mxODBC release, especially if you are using MS SQL Server as database server. Customers who have purchased mxODBC 3.0 licenses can download and install this patch-level release on top of their existing installations. The licenses will continue to work with version 3.0.1. Users of mxODBC 2.0 will have to purchase new licenses from our online shop in order to upgrade to mxODBC 3.0.1. You can request 30-day evaluation licenses by writing to sales@egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need. We will then issue you licenses and send them to you by email. Please make sure that you can receive ZIP file attachments on the email you specify in the request, since the license files are send out as ZIP attachments. _______________________________________________________________________ SUPPORT Commercial support for these packages is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 22 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Mon Aug 27 13:06:50 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Aug 27 12:04:22 2007 Subject: [egenix-users] RE: mxODBC 3.0.1 RC1 to address problems with .executedirect() and SQL Server In-Reply-To: References: <49A0DBD9577C0A4096A969872C94980A024FF58A@aam-ex-1.aam.local> <46C30BE9.6020007@egenix.com> <46C4BF1D.7060000@egenix.com> <46C5A5CE.2040704@egenix.com> <46C5B18B.4030206@egenix.com> Message-ID: <46D2A23A.9010005@egenix.com> On 2007-08-20 08:34, Harri Pesonen wrote: > But if .colcount is zero, and you get the error "ProgrammingError: > missing result set" when you try to use it, wouldn't it be better if > mxODBC would skip that result set automatically? It would be more > compatible with mxODBC 2. > > Now I think that every .fetchall() should be prepared with > > while o.colcount == 0: > if not o.nextset(): > break > > You would lose o.rowcount in this case, though. Perhaps it should be an > option (on by default), so that all non-existing result sets should be > skipped automatically. Thanks for the suggestion. We'll put this on our TODO list. Please note that management of the result sets is up to the ODBC driver (or the backend). mxODBC 3.0 will make all results sets available to the user, to let the user decide what is best for the application. However, I agree that in some cases, skipping virtual result sets (like the one SQL Server creates in order to report the rowcount) and always skipping to the last result set is better handled in mxODBC than in the application. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 27 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 > -- Harri > > -----Original Message----- > From: M.-A. Lemburg [mailto:mal@egenix.com] > Sent: 17. elokuuta 2007 17:33 > To: Harri Pesonen > Cc: egenix-users@egenix.com > Subject: Re: mxODBC 3.0.1 RC1 to address problems with .executedirect() > and SQL Server > > On 2007-08-17 15:59, Harri Pesonen wrote: >> Yes, and the problem can be solved in the procedure by adding >> >> SET NOCOUNT ON >> >> as a first statement: >> >> o.execute("""CREATE PROC #proc >> @p as nvarchar(200) >> AS >> BEGIN >> SET NOCOUNT ON >> CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) >> INSERT INTO #test2 (A,B) VALUES ('A','B') >> SELECT * FROM #test2 >> END""") >> >> I don't know, perhaps mxODBC should check if the result set is a real >> result set, or just information about the number of inserted rows. >> Perhaps it is not easy to detect it. > > The information mxODBC is looking for is the number of result set > columns. > > If that number is zero, it regards the result set as not existing, > since in most cases, that's the correct interpretation and > avoids strange errors you'd otherwise get from the ODBC fetch > calls. > > BTW, you can access the number of rows that were inserted > by querying c.rowcount just before the .nextset() call. > >> -- Harri >> >> -----Original Message----- >> From: M.-A. Lemburg [mailto:mal@egenix.com] >> Sent: 17. elokuuta 2007 16:43 >> To: Harri Pesonen >> Cc: egenix-users@egenix.com >> Subject: Re: mxODBC 3.0.1 RC1 to address problems with > .executedirect() >> and SQL Server >> >> On 2007-08-17 15:33, Harri Pesonen wrote: >>> Hi, we found one problem in version mxODBC 3. The following works in >>> mxODBC 2. >> That's because mxODBC 3.0 supports multiple result sets. >> >>> import mx.ODBC.Windows >>> dsn = "Driver={SQL > SERVER};Server=dskpesonha;Database=master;APP=test" >>> connection = mx.ODBC.Windows.DriverConnect(dsn, 0) >>> connection.stringformat = mx.ODBC.Windows.MIXED_STRINGFORMAT >>> o = connection.cursor() >>> o.execute("""CREATE PROC #proc >>> @p as nvarchar(200) >>> AS >>> BEGIN >>> CREATE TABLE #test2 (A nvarchar(64), B varchar(64)) >>> INSERT INTO #test2 (A,B) VALUES ('A','B') >>> SELECT * FROM #test2 >>> END""") >>> o.execute("EXEC #proc 'huhuu'") >>> rs = o.fetchall() >>> for r in rs: >>> print r >>> o = None >>> >>> In mxODBC 3 it fails with error: >>> >>> ProgrammingError: missing result set >>> >>> If I remove the INSERT statement, then it works! (No error but empty >>> result set). Odd. It must be a bug somewhere. >> The INSERT creates an empty result set with no columns (this results >> in the "missing result set" message - a bit misleading perhaps). >> >> To get to the next result set, you have to use .nextset(): >> >> o.execute("EXEC #proc 'huhuu'") >> o.nextset() >> rs = o.fetchall() >> for r in rs: >> print r >> o = None >> >>> -- Harri >>> >>> -----Original Message----- >>> From: M.-A. Lemburg [mailto:mal@egenix.com] >>> Sent: 17. elokuuta 2007 0:18 >>> To: egenix-users@egenix.com >>> Cc: Cliff Xuan; Harri Pesonen >>> Subject: mxODBC 3.0.1 RC1 to address problems with .executedirect() >> and >>> SQL Server >>> >>> Hello, >>> >>> in order to address the issues discussed in the last few days >>> with using .executedirect() on connections to SQL Server, we >>> are making a release candidate available. >>> >>> The final release of mxODBC 3.0.1 is planned for Monday, provided >>> that the workarounds we've added do fix all the problems you've >>> encountered. >>> >>> Please give the rc1 a try and let us know whether there are >>> any outstanding issues. >>> >>> > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.4.e >>> xe >>> > http://downloads.egenix.com/python/egenix-mxodbc-3.0.1_rc1.win32-py2.5.m >>> si >>> >>> Thanks, > From indy77 at ameritech.net Tue Aug 28 15:31:01 2007 From: indy77 at ameritech.net (Indy Very) Date: Tue Aug 28 23:31:51 2007 Subject: [egenix-users] ANN: eGenix mxODBC Distribution 3.0.1 (mxODBC Database Interface) In-Reply-To: <46CC3304.6020800@egenix.com> Message-ID: <714135.71624.qm@web81508.mail.mud.yahoo.com> Is there a 64-bit FreeBSD download (for ZopeDA product) available yet? I believe that 64-bit and UCS4 is the default for FreeBSD 6 and has been for some time. If a download can be made available, it would be appreciated very much! Thanks! "eGenix Team: M.-A. Lemburg" wrote: ________________________________________________________________________ ANNOUNCING eGenix.com mxODBC Database Interface Version 3.0.1 Our commercially supported Python extension providing ODBC database connectivity to Python applications on Windows and Unix platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Distribution-3.0.1-GA.html ________________________________________________________________________ INTRODUCTION The mxODBC Database Interface allows users to easily connect Python applications to just about any database on the market today - on both Windows and Unix platforms in a highly portable and convenient way. This makes mxODBC the ideal basis for writing cross-platform database programs and utilities in Python. mxODBC is included in the eGenix.com mxODBC Distribution for Python, which is part of the eGenix.com mx Extension Series - a collection of professional quality software tools aimed at enhancing Python's usability in many important areas such as database connectivity, fast text processing, date/time processing and web site programming. The package has proven its stability and usefulness in many mission critical applications and various commercial settings all around the world. It's been used in production for almost 10 years now. * About Python: Python is an object-oriented Open Source programming language which runs on all modern platforms (http://www.python.org/). By integrating ease-of-use, clarity in coding, enterprise application connectivity and rapid application design, Python establishes an ideal programming platform for todays IT challenges. * About eGenix: eGenix is a consulting and software product company focused on providing professional quality services and products to Python users and developers (http://www.egenix.com/). ________________________________________________________________________ NEWS mxODBC 3.0.1 is a patch-level release and includes the following updates: Enhanced SQL Server ODBC driver support: * work-around for using Unicode parameters together with cursor.executedirect() * bug-fix for cursor.stringformat setting when used with cursor.executedirect() * improved compatibility by using non-padding character binding codes Documentation: * added more documentation and an example of how to use the new connection and cursor error handlers. Error handlers were introduced in mxODBC 3.0.0 and allow for much greater control over how low-level errors in the interface are to be dealt with. For the full set of changes please check the mxODBC change log. ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/mxODBC/ IMPORTANT: In order to use the eGenix mxODBC package you will first need to install the eGenix mx Base package: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ UPGRADING You are encouraged to upgrade to this latest mxODBC release, especially if you are using MS SQL Server as database server. Customers who have purchased mxODBC 3.0 licenses can download and install this patch-level release on top of their existing installations. The licenses will continue to work with version 3.0.1. Users of mxODBC 2.0 will have to purchase new licenses from our online shop in order to upgrade to mxODBC 3.0.1. You can request 30-day evaluation licenses by writing to sales@egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need. We will then issue you licenses and send them to you by email. Please make sure that you can receive ZIP file attachments on the email you specify in the request, since the license files are send out as ZIP attachments. _______________________________________________________________________ SUPPORT Commercial support for these packages is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 22 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 _______________________________________________________________________ eGenix.com User Mailing List http://www.egenix.com/ https://www.egenix.com/mailman/listinfo/egenix-users -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070828/6781dab2/attachment.htm From seandavi at gmail.com Thu Aug 30 16:28:09 2007 From: seandavi at gmail.com (Sean Davis) Date: Thu Aug 30 21:28:16 2007 Subject: [egenix-users] mxodbc and DataDirect driver manager Message-ID: <264855a00708301228m7f40484wdba2876fd6bb12c9@mail.gmail.com> I have a third-party database provider that provides a driver and DataDirect driver manager. I have tried using their driver with the unixODBC driver manager with poor results (got junk output, lack of output from queries). With other clients (perl and RODBC), I have been able to simply point the lib and include directories to the DataDirect directories and recompile. Is this possible to do with mxODBC? For what it is worth, the other clients seem to recognize the DataDirect driver manager as "unixODBC-like". Any suggestions? Thanks, Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: /mailman-archives/egenix-users/attachments/20070830/cf988efb/attachment.htm From mal at egenix.com Fri Aug 31 00:17:18 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 30 23:17:12 2007 Subject: [egenix-users] mxodbc and DataDirect driver manager In-Reply-To: <264855a00708301228m7f40484wdba2876fd6bb12c9@mail.gmail.com> References: <264855a00708301228m7f40484wdba2876fd6bb12c9@mail.gmail.com> Message-ID: <46D733DE.7050103@egenix.com> On 2007-08-30 21:28, Sean Davis wrote: > I have a third-party database provider that provides a driver and DataDirect > driver manager. I have tried using their driver with the unixODBC driver > manager with poor results (got junk output, lack of output from queries). > With other clients (perl and RODBC), I have been able to simply point the > lib and include directories to the DataDirect directories and recompile. Is > this possible to do with mxODBC? For what it is worth, the other clients > seem to recognize the DataDirect driver manager as "unixODBC-like". Any > suggestions? This is possible, but since we don't ship the source code, we'll have to do this for you. Which database driver and database are you referring to ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 30 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Fri Aug 31 00:59:18 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Aug 30 23:59:03 2007 Subject: [egenix-users] mxodbc and DataDirect driver manager In-Reply-To: <46D733DE.7050103@egenix.com> References: <264855a00708301228m7f40484wdba2876fd6bb12c9@mail.gmail.com> <46D733DE.7050103@egenix.com> Message-ID: <46D73DB6.1000504@egenix.com> On 2007-08-30 23:17, M.-A. Lemburg wrote: > On 2007-08-30 21:28, Sean Davis wrote: >> I have a third-party database provider that provides a driver and DataDirect >> driver manager. I have tried using their driver with the unixODBC driver >> manager with poor results (got junk output, lack of output from queries). >> With other clients (perl and RODBC), I have been able to simply point the >> lib and include directories to the DataDirect directories and recompile. Is >> this possible to do with mxODBC? For what it is worth, the other clients >> seem to recognize the DataDirect driver manager as "unixODBC-like". Any >> suggestions? > > This is possible, but since we don't ship the source code, we'll > have to do this for you. > > Which database driver and database are you referring to ? One thing you should try is pointing the mx.ODBC.unixODBC package at the DataDirect ODBC manager: mxODBC.so links against the libodbc.so.1 lib provided by unixODBC. According to this page: http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.dc00190_0500/html/edi50ig/CHDDJCJA.htm the DataDirect ODBC driver provides a libodbc.so file. Simply adding a link from libodbc.so to libodbc.so.1 may be all you need to do. cd /lib ln -sf libodbc.so libodbc.so.1 ldconfig You can verify the correct setup by doing a: ldconfig -p | fgrep odbc Then give mx.ODBC.unixODBC a try and see whether this works. If not, we can port mxODBC to the DataDirect manager as suggested in private email. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 30 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611