How to get time.strptime()?

[Gnarlodious]

Error: AttributeError: 'module' object has no attribute '_strptime'

This problem is driving me crazy. It only happens in Python 3.3.0, while on my server running 3.1.3 it behaves as expected. When I try to access time.strptime() it errors with

AttributeError: 'module' object has no attribute '_strptime'.

This error only occurs under mod_wsgi, when running as a one-shot webapp it behaves normally. All other functionalities of the time module are normal.

If anyone could explain why it thinks I want an underscored name maybe it would help.

Thanks.

-- Gnarlie

 

[Chris Rebert]

Error: AttributeError: 'module' object has no attribute '_strptime'

Please include the full Traceback, not just the final error message.

This problem is driving me crazy. It only happens in Python 3.3.0, while on

my server running 3.1.3 it behaves as expected. When I try to access
time.strptime() it errors with

AttributeError: 'module' object has no attribute '_strptime'.

Might be the wrong `time` module. What's `print time.__file__` output?
Did you perchance create a time.py file in your project?

This error only occurs under mod_wsgi, when running as a one-shot webapp it

 

[Steven D'Aprano]

Error: AttributeError: 'module' object has no attribute '_strptime'

This problem is driving me crazy. It only happens in Python 3.3.0, while
on my server running 3.1.3 it behaves as expected. When I try to access
time.strptime() it errors with

AttributeError: 'module' object has no attribute '_strptime'. [...]
If anyone could explain why it thinks I want an underscored name maybe
it would help.

Perhaps if you were to read the ENTIRE traceback, not just the last line,
you will see something that hints at an explanation. Here is a similar
error:

py> time.strptime(None)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.3/_strptime.py", line 494, in
_strptime_time
tt = _strptime(data_string, format)[0]
File "/usr/local/lib/python3.3/_strptime.py", line 306, in _strptime
raise TypeError(msg.format(index, type(arg)))
TypeError: strptime() argument 0 must be str, not <class 'NoneType'>


After reading the full traceback, I conclude that the time.strptime
function calls _strptime._strptime. After importing _strptime and looking
at the __file__ attribute, I see that it is a pure-python module. So, to
answer your immediate question:

"explain why it [the time.strptime function] thinks I want an underscored
name"

the answer is, it thinks you want an underscored name because you *do*
need an underscored name. It is part of the implementation of
time.strptime that it delegates some of the work to a private module
_strptime.

As for why that is not available under mod_wsgi module, but is available
when running under the normal Python environment, I have looked at the
_strptime.py source code and nothing stands out to explain why the
_strptime function might not be defined. So that seems like a question
about mod_wsgi. I suggest you start here:

http://code.google.com/p/modwsgi/wiki/WhereToGetHelp

If you do get an answer, please post it here as well, so others in the
future who stumble across this thread will see the answer.

Good luck!

 

[Gnarlodious]

Thanks for the help.
This error only occurs on my devbox which is running Py 3.3.0:

print(time.__file__, file=sys.stderr)
/usr/local/python-3.3.0/frameworks/Python.framework/Versions/3.3/lib/python3.3/lib-dynload/time.so

which looks normal to me.
The server box, which is running Py 3.1.3, says this:
/usr/local/python-3.1.3/frameworks/Python.framework/Versions/3.1/lib/python3.1/lib-dynload/time.so

Also of possible relevance is that my server is running mod_wsgi 3.3, whilemy devbox is running mod_wsgi 3.4. However I had a discussion with Dumpleton who decided it is a Python problem:
https://groups.google.com/forum/#!topic/modwsgi/9MF2ixebvfg

I don't know how to get any sort of traceback out of mod_wsgi. The script runs normally as a single-page CGI script but the error only occurs when imported as a module.

Exactly what I am saying is this:
import time
print(time.strptime("00", "%y"), file=sys.stderr)
--> No module named 'time.strptime'

But this works normally:
print(time.time(), file=sys.stderr)
--> 1356549188.368575

I can say this and see 'strptime' in the list of functions:
print(dir(time), file=sys.stderr)
--> ['_STRUCT_TM_ITEMS', '__doc__', '__file__', '__loader__', '__name__', '__package__', 'altzone', 'asctime', 'clock', 'ctime', 'daylight', 'get_clock_info', 'gmtime', 'localtime', 'mktime', 'monotonic', 'perf_counter', 'process_time', 'sleep', 'strftime', 'strptime', 'struct_time', 'time', 'timezone', 'tzname', 'tzset']

So I am stumped.

-- Gnarlie

 

[Nick Cash]

Error: AttributeError: 'module' object has no attribute '_strptime'

This problem is driving me crazy. It only happens in Python 3.3.0,
while on my server running 3.1.3 it behaves as expected. When I try to
access time.strptime() it errors with

AttributeError: 'module' object has no attribute '_strptime'.

This error only occurs under mod_wsgi, when running as a one-shot
webapp it behaves normally. All other functionalities of the time
module are normal.

I've run into this issue in the past with Python 2.6 (or maybe 2.7) and mod_python. My problem was that datetime.strptime imported _strptime at runtime, and multiple threads trying to import it at the same time broke an internal import lock.

I was able to work around this by simply importing _strptime myself at server startup time. This may or may not work for you.

See also: http://code.google.com/p/modwsgi/issues/detail?id=177 and http://www.mail-archive.com/[email protected]/msg248664.html

 

[Gnarlodious]

I was able to work around this by simply importing _strptime myself at server startup time.

THANK YOU! That fixed it, I simply put

import _strptime

in my *.wsgi script. It feels like a kludgy solution, but it works.

I should also mention that I had a similar problem with the "threading" module that I fixed in the same way. This problem occurred inside of sqlite3 because evidently threadlocking has gone standard as of Py 3.3.x

This is problem that has unduly vexed me. When you start learning Python they don't tell you about these sharp edges. Someone needs to explain.

-- Gnarlie

 

[Gnarlodious]

I was able to work around this by simply importing _strptime myself at server startup time.

THANK YOU! That fixed it, I simply put

import _strptime

in my *.wsgi script. It feels like a kludgy solution, but it works.

I should also mention that I had a similar problem with the "threading" module that I fixed in the same way. This problem occurred inside of sqlite3 because evidently threadlocking has gone standard as of Py 3.3.x

This is problem that has unduly vexed me. When you start learning Python they don't tell you about these sharp edges. Someone needs to explain.

-- Gnarlie

 

[Steven D'Aprano]

This is problem that has unduly vexed me. When you start learning Python
they don't tell you about these sharp edges.

"Hi, I've just started learning Python today, and I have a question. How
do I print a list?"

"Before I answer that question, let me explain to you how to fix a
problem with time.strptime that only occurs when running under mod_wsgi.
Now, I know that you're still struggling with print, haven't even got to
importing modules yet, have no idea what strptime is, and that 99.99% of
Python programmers will never use mod_wsgi in their life, but if we don't
tell you about this right now, in five years time you will complain that
nobody told you about the sharp edges when you started..."

*wink*

A serious question: when should somebody have told you about this sharp
edge, and who should have been responsible to do so?

 

[Roy Smith]

This is problem that has unduly vexed me. When you start learning Python they
don't tell you about these sharp edges. Someone needs to explain.

The same thing happens when you're born. You spend your entire life
figuring out the sharp edges. And then you die.

 

[Mitya Sirenef]

This is problem that has unduly vexed me. When you start learning
Python they don't tell you about these sharp edges. Someone needs to
explain. -- Gnarlie

In fact, if there were no bugs, there would be no need to explain nor to
pick the
right time / place for explanations. Why doesn't Python warn that it's not
100% perfect? Are people just supposed to "know" this, magically?

 

[Chris Angelico]

In fact, if there were no bugs, there would be no need to explain nor to
pick the
right time / place for explanations. Why doesn't Python warn that it's not
100% perfect? Are people just supposed to "know" this, magically?

People are supposed to know that Python is not 100% perfect because
there's no such thing as a 100% perfect language. The problem here is
that Python doesn't have any magical way to deal with messy imports in
multiple threads; the solution is to sort out the import before the
multiple threads get going (meaning that when each thread attempts the
import, it's resolved trivially by simply binding the existing module
to a new name).

Threads are fairly messy in the best of times. The only thing I could
imagine Python doing here would be a brutal global lock - as soon as
one thread starts any import, all other threads get locked out of
execution (or at least of other imports). And I'm not sure that'd
solve everything - plus it brings the risk that one thread can stall
the entire system with a failed or faulty import. If you can think of
of any way to improve this, I'm sure python-ideas will happily discuss
it.

ChrisA

 

[Gnarlodious]

The problem here is that Python
doesn't have any magical way to deal with messy imports in
multiple threads

But couldn't Py 3.3.1 at least raise an error mentioning threading as a possible cause? Because "No module named _strptime" is pretty cryptic.

-- Gnarlie

 

[Gnarlodious]

The problem here is that Python
doesn't have any magical way to deal with messy imports in
multiple threads

But couldn't Py 3.3.1 at least raise an error mentioning threading as a possible cause? Because "No module named _strptime" is pretty cryptic.

-- Gnarlie

 

[Gnarlodious]

Graham Dumpleton has informed me that the the relevant bug reports are:

http://bugs.python.org/issue8098
http://bugs.python.org/issue9260

To quote:

All the problems derive from a stupid function in Python internals called PyImport_ImportModuleNoBlock(). It was designed to avoid lockups of the import machinery but just causes other problems because a module import can fail if the import lock is held by a different thread. This faulty magic fairydust was sprinkled on the time.strptime() function which internally loads _strptime module. If you access time.strptime() when another thread holds the import lock, the call will fail because of a failed import.

So there. As more people update to Python 3.3.0 hopefully this magic fairy dust workaround will see some daylight.

-- Gnarlie

 

[Ian Kelly]

Graham Dumpleton has informed me that the the relevant bug reports are:

http://bugs.python.org/issue8098
http://bugs.python.org/issue9260

To quote:

All the problems derive from a stupid function in Python internals calledPyImport_ImportModuleNoBlock(). It was designed to avoid lockups of the import machinery but just causes other problems because a module import can fail if the import lock is held by a different thread. This faulty magic fairy dust was sprinkled on the time.strptime() function which internally loads _strptime module. If you access time.strptime() when another thread holdsthe import lock, the call will fail because of a failed import.

So there. As more people update to Python 3.3.0 hopefully this magic fairy dust workaround will see some daylight.

Thanks for the context, but that actually seems to be describing the
situation from Python 3.2. Python 3.3 introduced module-level import
locking with improved deadlock detection that looks for actual cycles
and is supposed to fix the unnecessary ImportErrors described in
#8098, rendering the PyImport_ImportModuleNoBlock "fairy dust"
unnecessary -- in 3.3, I understand that PyImport_ImportModuleNoBlock
is now just a synonym for PyImport_ImportModule. But you're reporting
that you're still seeing issues with this in Python 3.3. Diving into
the code, I think I start to understand why. Take a look at this
function from Lib/importlib/_bootstrap.py:

def _lock_unlock_module(name):
"""Release the global import lock, and acquires then release the
module lock for a given module name.
This is used to ensure a module is completely initialized, in the
event it is being imported by another thread.

Should only be called with the import lock taken."""
lock = _get_module_lock(name)
_imp.release_lock()
try:
lock.acquire()
except _DeadlockError:
# Concurrent circular import, we'll accept a partially initialized
# module object.
pass
else:
lock.release()


Normally, it just acquires and releases the module lock to ensure that
the module has been completely initialized. If a deadlock is
detected, however, then it skips that step and instead of raising an
ImportError (which would seem to me to be the right thing to do here)
it allows the importing code to simply proceed with a module that may
not be fully imported yet. Since your error message concerned missing
module-level attributes within the _strptime module, that seems to be
what you're running into. The time.strptime function attempts to
import _strptime, which is already being imported by another thread;
for some unknown reason it detects a cyclical deadlock as a result of
this; it then white-washes the deadlock and tries to use the _strptime
module anyway, causing a different error to occur. The question then
is how you're managing to get an import cycle by using strptime.

 

[Gnarlodious]

Thank you for a reasonable discussion of the problem, although I don't really understand what you said. This is a WSGI webapp configured for one-process and one-thread.

I can imagine my unorthodox handling of imported modules is suspect. To explain, my webapp first loads modules from a dict of allowed pages:

__init__
for module in self.allowedPages:
try:
self.Pages[module]=getattr(__import__(module), module)


This gives me a dict of objects like this:
self.Pages
{'Yijing': <Yijing.YijingProto object at 0x1067a4f90>, 'Strand': <Strand.StrandProto object at 0x1067b52d0>, 'Gnomon': <Gnomon.GnomonProto object at 0x10675a450>, 'Lexicon': <Lexicon.LexiconProto object at 0x1067c4a50>, 'Dream': <Dream.DreamProto object at 0x106800a50>, 'Grid': <Grid.GridProto object at 0x1067ac310>}


I can then extract the first query string variable name and get the page content:
__call__
page='Gnomon'
xml=self.Pages[page](EnvDict, qList) # sends environ & query list


I don't know how several modules are normally handled in a CGI webapp, but this homegrown system has served me well. Threadlocking, however, may find this sort of referencing problematic. That would be my first suspicion.

-- Gnarlie

 

[Gnarlodious]

Thank you for a reasonable discussion of the problem, although I don't really understand what you said. This is a WSGI webapp configured for one-process and one-thread.

I can imagine my unorthodox handling of imported modules is suspect. To explain, my webapp first loads modules from a dict of allowed pages:

__init__
for module in self.allowedPages:
try:
self.Pages[module]=getattr(__import__(module), module)


This gives me a dict of objects like this:
self.Pages
{'Yijing': <Yijing.YijingProto object at 0x1067a4f90>, 'Strand': <Strand.StrandProto object at 0x1067b52d0>, 'Gnomon': <Gnomon.GnomonProto object at 0x10675a450>, 'Lexicon': <Lexicon.LexiconProto object at 0x1067c4a50>, 'Dream': <Dream.DreamProto object at 0x106800a50>, 'Grid': <Grid.GridProto object at 0x1067ac310>}


I can then extract the first query string variable name and get the page content:
__call__
page='Gnomon'
xml=self.Pages[page](EnvDict, qList) # sends environ & query list


I don't know how several modules are normally handled in a CGI webapp, but this homegrown system has served me well. Threadlocking, however, may find this sort of referencing problematic. That would be my first suspicion.

-- Gnarlie