[egenix-users] Problems moving to mxTextTools 3.0

[M.-A. Lemburg]

Peter wrote:
> Thanks for your quick reply Marc-Andre,
> 
>>> Also could you clarify what is meant by "Removed support for
>>> buffer-compatible input objects"? Does this mean we can't use Python's
>>> StringIO handles?
>>
>> Yes, we had to do this as a result of the restructuring of the
>> underlying code which no longer works on a char* pointer, but
>> instead uses the object type information to see whether it needs
>> to compile a Unicode tag table or a string one.
> 
> Oh :(
> 
> I was afraid you might say that.
> 
> Do you have any suggested workarounds for using mxTextTools to parse
> data held in a string (rather than read from a handle to an opened file)?

I think I lost you there :-)

mxTextTools *does* work on Python strings and Unicode. It no longer works
on objects that just expose the buffer API. We'll likely add support for
that at some later stage, but for now, the Unicode support was more
important to get right.

You can easily convert a StringIO instance to a Python string using
.getvalue() method.

For larger amounts of data, it's also a good idea to process the data
in chunks. mxTextTools allows for this by returning the index of where
it stopped parsing the input.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 13 2007)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611