lxml + mod_python: cannot unmarshal code objects in restricted execution mode

D

Dmitri Fedoruk

Hello everyone,

I'm developing a mod_python application that is based on XML\XSLT
transforming.

I used 4Suite libraries for that, but as the speed was unacceptable
for me, I switched to lxml. Everything became much easier and 10 times
faster, but I've encountered the subject problem.

In brief - all my data and xslt are stored and transferred in UTF-8.
With 4Suite everything was fine all the time. With lxml it works fine
from the console, but inside mod_python it occasionaly dies, ~ one
time out of three. Strange - the same code with the same data works or
dies by its own means.

As far as I have found, there was a similar problem with PyXML and
encodings module, but there was no clear solution.

So, my configuration is the following:
Python 2.5.1
Server version: Apache/2.2.4 (FreeBSD)
mod_python-3.3.1

And the relevant parts of my code are these:

def extApplyXslt(xslt, data, logger ):
try:
strXslt = urllib2.urlopen(xslt).read()
# i have to read the xslt url to the python string
except urllib2.HTTPError, e:
.......
except urllib2.URLError, e:
.............
try:
xslt_parser = etree.XMLParser()
xslt_parser.resolvers.add( PrefixResolver("XSLT") )

# and now I have to use the string; a more elegant solution,
anyone?
f = StringIO(strXslt)
xslt_doc = etree.parse(f, xslt_parser)

# and here where the problem comes
transform = etree.XSLT(xslt_doc)

except Exception, exc:
logger.log(logging.CRITICAL, exc.__str__() )

try:
result_tree = transform(data)
return etree.tostring(result_tree, 'utf-8')
except Exception, exc:
print "xslt processing error!", exc.__str__()
return ""

It dies with the message 'cannot unmarshal code objects in restricted
execution mode'. By profiling I detected the point where problem
occurs:
transform = etree.XSLT(xslt_doc)

So, I would be grateful for any suggestions how to get rid of this.
I'd really like to use lxml. Maybe I should initialize the xslt
processor in somehow other way?

Thanks in advance,
Dmitri
 
S

Stefan Behnel

Dmitri said:
def extApplyXslt(xslt, data, logger ):
try:
strXslt = urllib2.urlopen(xslt).read()
# i have to read the xslt url to the python string
except urllib2.HTTPError, e:
.......
except urllib2.URLError, e:
.............
try:
xslt_parser = etree.XMLParser()
xslt_parser.resolvers.add( PrefixResolver("XSLT") )

# and now I have to use the string; a more elegant solution,
anyone?

Sure, lxml.etree can parse from file-like objects. Just hand in the result of
urlopen().

Apart from that, I saw that you found your way to the lxml mailing list, I'll
respond over there.

Stefan
 
D

Dmitri Fedoruk

So, my configuration is the following:
Python 2.5.1
Server version: Apache/2.2.4 (FreeBSD)
mod_python-3.3.1
update:
lxml-1.3.4
libxslt-1.1.21
libxml2-2.6.29
 
G

Graham Dumpleton

Hello everyone,

I'm developing a mod_python application that is based on XML\XSLT
transforming.

I used 4Suite libraries for that, but as the speed was unacceptable
for me, I switched to lxml. Everything became much easier and 10 times
faster, but I've encountered the subject problem.

In brief - all my data and xslt are stored and transferred in UTF-8.
With 4Suite everything was fine all the time. With lxml it works fine
from the console, but inside mod_python it occasionaly dies, ~ one
time out of three. Strange - the same code with the same data works or
dies by its own means.

As far as I have found, there was a similar problem with PyXML and
encodings module, but there was no clear solution.

So, my configuration is the following:
Python 2.5.1
Server version: Apache/2.2.4 (FreeBSD)
mod_python-3.3.1

And the relevant parts of my code are these:

def extApplyXslt(xslt, data, logger ):
try:
strXslt = urllib2.urlopen(xslt).read()
# i have to read the xslt url to the python string
except urllib2.HTTPError, e:
.......
except urllib2.URLError, e:
.............
try:
xslt_parser = etree.XMLParser()
xslt_parser.resolvers.add( PrefixResolver("XSLT") )

# and now I have to use the string; a more elegant solution,
anyone?
f = StringIO(strXslt)
xslt_doc = etree.parse(f, xslt_parser)

# and here where the problem comes
transform = etree.XSLT(xslt_doc)

except Exception, exc:
logger.log(logging.CRITICAL, exc.__str__() )

try:
result_tree = transform(data)
return etree.tostring(result_tree, 'utf-8')
except Exception, exc:
print "xslt processing error!", exc.__str__()
return ""

It dies with the message 'cannot unmarshal code objects in restricted
execution mode'. By profiling I detected the point where problem
occurs:
transform = etree.XSLT(xslt_doc)

So, I would be grateful for any suggestions how to get rid of this.
I'd really like to use lxml. Maybe I should initialize the xslt
processor in somehow other way?

Thanks in advance,
Dmitri

Try forcing mod_python to run your code in the first interpreter
instance created by Python.

PythonInterpreter main_interpreter

If that still doesn't work then you are hitting problems in mod_python
related to how it does GIL management or how the third party package
is implemented and whether it is compatible with use in secondary
Python interpreters. For details see:

If it still doesn't work and your application is a WSGI application
and you aren't using mod_python specific features, only choice at this
point would be to use mod_fastcgi or mod_wsgi instead. If using
mod_wsgi, you would still need to tell it to run your code in first
interpreter instance created by Python.

WSGIApplicationGroup %{GLOBAL}

For some details on issues related to this problem, see sections
'Python Simplified GIL State API' and 'Multiple Python Sub
Interpreters' in mod_wsgi document:

http://code.google.com/p/modwsgi/wiki/ApplicationIssues

Graham
 

Members online

No members online now.

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,009
Latest member
GidgetGamb

Latest Threads

Top