Problem with mixing doctest with gettext _()

[Pierre Rouleau]

I have a problem writing self-testable modules using doctest when these
modules have internationalized strings using gettext _('...').

- The main module of an application (say app.py) calls gettext.install()
to install the special _ function inside Python builtin. Other modules,
taken from a general purpose collection of Python modules, also support
internationalisation and doctest testing.

For example:

utstring.py would contain some code like this:

def onOffStr(isOn) :
"""Return the "ON" string for True, "OFF" for False.

**Example**
"""
if isOn:
return _(u"ON") # notice the underscore
else:
return _(u"OFF")



The utstring module does not call any of the gettext calls, because some
other module does it when the application runs. So the doctest fails:

*****************************************************************
Failure in example: onOffStr(True)
from line #4 of utstring.onOffStr
Exception raised:
Traceback (most recent call last):
File "C:\Python23\Lib\doctest.py", line 442, in _run_examples_inner
compileflags, 1) in globs
File "<string>", line 1, in ?
File "C:\dev\python\utstring.py", line 513, in onOffStr
return _(u"ON")
TypeError: 'tuple' object is not callable
*****************************************************************

I tried to define a _() function when testing with the code below but
the doctest still fails. The following code is at the end of my
utstring.py module

def _test():
"""_test() perform docstring test"""

import doctest, utstring
return doctest.testmod(utstring)

if __name__ == "__main__":
def _(aString):
# dummy _() attempting to get doctest to pass.
return aString

_test()


----------

Does anyone know why the doctest still fails when I define the dummy _()
function?


Thanks in advance.

Pierre

 

[Skip Montanaro]

Pierre> I tried to define a _() function when testing with the code
Pierre> below but the doctest still fails. The following code is at the
Pierre> end of my utstring.py module
...

I suspect it's because your dummy _ is not in builtins. Why not just call
gettext.install() where you are currently defining _?

Skip

 

[Pierre Rouleau]

Pierre> I tried to define a _() function when testing with the code
Pierre> below but the doctest still fails. The following code is at the
Pierre> end of my utstring.py module
...

I suspect it's because your dummy _ is not in builtins. Why not just call
gettext.install() where you are currently defining _?

Skip, this looks like the way to go.

I was trying to avoid it because the translation files for these library
modules are constructed at the application level somewhere else. But I
was also considering creating a translation library for these module, so
I guess that is one more incentive to do so...

Thanks

Pierre

 

[Skip Montanaro]

Pierre> Skip, this looks like the way to go.

Pierre> I was trying to avoid it because the translation files for these
Pierre> library modules are constructed at the application level
Pierre> somewhere else. But I was also considering creating a
Pierre> translation library for these module, so I guess that is one
Pierre> more incentive to do so...

If you really want a dummy _() you can also stuff your version into
builtins:
'hi'

Skip

 

[Pierre Rouleau]

Skip Montanaro wrote:

If you really want a dummy _() you can also stuff your version into
builtins:

'hi'

I tried that, but it only works for the first call...

[Shell buffer started: python.exe]
Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):


Pierre

 

[Peter Otten]

I tried that, but it only works for the first call...

Setting __builtin__._ to the result of the last calculation is a side effect
of sys.displayhook. Therefore you need to change that too:

Python 2.3.3 (#1, Jan 3 2004, 13:57:08)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information..... if a is not None: sys.stdout.write("%r\n" % a)
....
Peter

 

[Pierre Rouleau]

Skip, this looks like the way to go.

I was trying to avoid it because the translation files for these library
modules are constructed at the application level somewhere else. But I
was also considering creating a translation library for these module, so
I guess that is one more incentive to do so...

I implemented the dictionary and use it in the code just fine, but the
doctest still fails

My teststr.py module is:

#--[---------------------------------------------------------------
if __name__ == "__main__":
# install gettext for testing this module because of the string
translation
# performed in the code.
import gettext
gettext.install('impathpl', './locale', unicode=False)
presLan_en = gettext.translation('impathpl', "./locale",
languages=['en'])
presLan_en.install()

def onOffStr(isOn) :
"""Return the "ON" string for True, "OFF" for False.

**Example**
"""
if isOn:
return _("ON")
else:
return _("OFF")


def _test():
"""_test() perform docstring test"""

import doctest, teststr
return doctest.testmod(teststr)

if __name__ == "__main__":
_test()

#--]-------------------------------------------------


Running the following script shows that the module runs OK:

[Shell buffer started: python.exe]
Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.

>>> import teststr
>>> import gettext
>>> gettext.install('impathpl', './locale', unicode=False)
>>> presLan_en = gettext.translation('impathpl', "./locale", languages=['en'])
>>> presLan_en.install()
>>> teststr.onOffStr(True) 'ON'
>>>

BUT, the doctest fails:

D:\dev\python>teststr
*****************************************************************
Failure in example: onOffStr(False)
from line #6 of teststr.onOffStr
Exception raised:
Traceback (most recent call last):
File "c:\Python23\lib\doctest.py", line 442, in _run_examples_inner
compileflags, 1) in globs
File "<string>", line 1, in ?
File "teststr.py", line 23, in onOffStr
return _("OFF")
TypeError: 'str' object is not callable
*****************************************************************
1 items had failures:
1 of 2 in teststr.onOffStr
***Test Failed*** 1 failures.


I'm still puzzled...

Pierre

 

[Skip Montanaro]

Pierre> I tried that, but it only works for the first call...



Pierre> [Shell buffer started: python.exe]
Pierre> Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on
Pierre> win32
Pierre> Type "help", "copyright", "credits" or "license" for more information. Pierre> Traceback (most recent call last):

That's true. In interactive mode _ is assigned the value of the last
expression evaluated. Try it from a script.

Skip

 

[Skip Montanaro]

Pierre> BUT, the doctest fails:
...

Looks like Peter Otten's sys.displayhook hack is required for doctest.

Skip

 

[Pierre Rouleau]

Pierre Rouleau wrote:

Skip Montanaro wrote:



I tried that, but it only works for the first call...

Setting __builtin__._ to the result of the last calculation is a side effect
of sys.displayhook. Therefore you need to change that too:

Python 2.3.3 (#1, Jan 3 2004, 13:57:08)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

... if a is not None: sys.stdout.write("%r\n" % a)
...

...

'hi'

'hello'

Thanks Peter, it does work!

 

[Pierre Rouleau]

Pierre Rouleau wrote:

Skip Montanaro wrote:


If you really want a dummy _() you can also stuff your version into
builtins:

import __builtin__
def foo(s): return s
...
__builtin__._ = foo
_
<function foo at 0x1d6670>
_("hi")
'hi'


I tried that, but it only works for the first call...

Setting __builtin__._ to the result of the last calculation is a side
effect
of sys.displayhook. Therefore you need to change that too:

Python 2.3.3 (#1, Jan 3 2004, 13:57:08)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import sys
def mydisplayhook(a):

... if a is not None: sys.stdout.write("%r\n" % a)
...

def foo(s): return s

...

sys.displayhook = mydisplayhook
import __builtin__
__builtin__._ = foo
_("hi")

'hi'

_("hello")

'hello'

Thanks Peter, it does work!

It worked, BUT only for a simple function, it fails if I add a another
simple function:

My updated teststr.py script:

#--[--------------------------------------
def onOffStr(isOn) :
"""Return the "ON" string for True, "OFF" for False.

**Example**
"""
if isOn:
return _(u"ON")
else:
return _(u"OFF")


def inList(longString, stringList) :
"""Return True if one of the string in `stringList` is inside
`longString`.

Also return the list index.
**Example**

>>> L = ["**", "/*"]
>>> inList("aksdkajshd",L) (False, 0)
>>> inList("aksdkajsh**d",L)

(True, 0)

"""

theIndex = 0
for theStr in stringList:
if longString.find(theStr) >= 0:
return (True,theIndex)
theIndex +=1
return (False,0)


def _test():
"""_test() perform docstring test"""

import doctest, teststr
return doctest.testmod(teststr)

if __name__ == "__main__":
import sys

def test_displayhook(a):
if a is not None: sys.stdout.write("%r\n" % a)

def test_translator(aString):
return aString

sys.displayhook = test_displayhook

import __builtin__
__builtin__._ = test_translator
_test()

#--]--------------------------------------

Running the test fails:

D:\dev\python>teststr
*****************************************************************
Failure in example: inList("aksdkajshd",L)
from line #6 of teststr.inList
Exception raised:
Traceback (most recent call last):
File "c:\Python23\lib\doctest.py", line 442, in _run_examples_inner
compileflags, 1) in globs
File "<string>", line 1, in ?
File "D:\dev\python\teststr.py", line 50, in test_displayhook
if a is not None: sys.stdout.write("%r\n" % a)
TypeError: not all arguments converted during string formatting
*****************************************************************
Failure in example: inList("aksdkajsh**d",L)
from line #8 of teststr.inList
Exception raised:
Traceback (most recent call last):
File "c:\Python23\lib\doctest.py", line 442, in _run_examples_inner
compileflags, 1) in globs
File "<string>", line 1, in ?
File "D:\dev\python\teststr.py", line 50, in test_displayhook
if a is not None: sys.stdout.write("%r\n" % a)
TypeError: not all arguments converted during string formatting
*****************************************************************
1 items had failures:
2 of 3 in teststr.inList
***Test Failed*** 2 failures.

#------------------------------------------

So far, I don't have a solution for writing internationalized Python
that support doctest. Surely, I am not the first one trying to do that...


Pierre

 

[Peter Otten]

It worked, BUT only for a simple function, it fails if I add a another
simple function:

Haven't read your post completely, but judging from the error message, it's
just a minor glitch in mydisplayhook(). Try the following instead:

.... if a is not None:
.... sys.stdout.write("%r\n" % (a,))
....

That should be able to cope with tuples as commandline results.

Peter

 

[Pierre Rouleau]

Pierre Rouleau wrote:




Haven't read your post completely, but judging from the error message, it's
just a minor glitch in mydisplayhook(). Try the following instead:




... if a is not None:
... sys.stdout.write("%r\n" % (a,))
...

That should be able to cope with tuples as commandline results.

Peter

You're right! It does work. I must admit, that I don't see why though.
(a,) makes a tuple out of the `a` argument. Does the %r conversion
require a tuple?

 

[Peter Otten]

You're right! It does work. I must admit, that I don't see why though.
(a,) makes a tuple out of the `a` argument. Does the %r conversion
require a tuple?

The formatting operator behaves differently depending on whether the
righthand argument is a tuple or something else.

formatstr % tuple

ensures that there is a corresponding format expression - e.g. "%s" or "%d"
- for every item in the tuple, whereas

formatstr % nontuple

expects exactly one format expression. Therefore

"%r\n" % a

raises an exception when a is a tuple with more or less than one item and
wrongly prints the item's representation instead of the tuple's for
one-tuples.
Wrapping it like so (a,) fixes the problem because now we have always a
tuple with one item - where this item is sometimes a tuple.

An alternative approach would be to ensure that the righthand side is always
a nontuple:

"%s\n" % repr(a)

Peter

 

[Pierre Rouleau]

The formatting operator behaves differently depending on whether the
righthand argument is a tuple or something else.

formatstr % tuple

ensures that there is a corresponding format expression - e.g. "%s" or "%d"
- for every item in the tuple, whereas

formatstr % nontuple

expects exactly one format expression. Therefore

"%r\n" % a

raises an exception when a is a tuple with more or less than one item and
wrongly prints the item's representation instead of the tuple's for
one-tuples.
Wrapping it like so (a,) fixes the problem because now we have always a
tuple with one item - where this item is sometimes a tuple.

An alternative approach would be to ensure that the righthand side is always
a nontuple:

"%s\n" % repr(a)

Thanks for this clear explanation!

Pierre