Intermittent Pickle Problems in Jython
The previous post mentioned "integrating" Jython and CPython by transmitting a stream of pickles between the two. I encountered one intermittent problem with this approach, and I'm unsure of its cause. (Hm, and I should probably post this to a Jython mailing list...)
Problem
In Jython I'd pickled the str() of a java.io.StringWriter, into which I'd just written the SD representation of a CDK molecule. Jython could create the pickle alright. But when I tried to unpickle it in CPython, sometimes, for some molecules, I got a traceback:
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/pickle.py", line 970, in load_string raise ValueError, "insecure string pickle" ValueError: insecure string pickle
The error occurred consistently in my application code, always on the same input structure. But I couldn't derive a simple test script to demonstrate the problem.
Investigation
Examination of the problematic pickle data showed that a Python unicode string literal marker had somehow been inserted, and the type code for the item was somehow S
(for string) rather than V
(for unicode):
... sS'sdf' p3 Su'ZINC00000181\n CDK... ^ What the... ?
Workaround
Google turned up a usable workaround: encode the offending string as utf-8 before trying to pickle it.
import codecs enc = codecs.getencoder('utf8') ... sdf = enc(sdf) ...