gmpy moving to code.google.com

[Alex Martelli]

If you're interested in gmpy (the Python wrapper of GMP, for
unlimited-precision arithmetic, rationals, random number generation,
number-theoretical functions, etc), please DO check out
http://code.google.com/p/gmpy/ -- gmpy 1.02 is there (as far as I can
tell) in a workable state. Source on Subversion (and a prerelease
zipfile too), downloadable binaries for MacOSX (download and read the
README file first!) and Windows (for Python 2.4 and 2.5 only, built and
minimally tested on a shaky Win2K+mingw -- on -- Parallels/MacOSX
setup... I have no other Windows machine to check 'em out...!).

Please help me check that the move-and-upgrade went OK -- download some
or all of the pieces (including an svn checkout of the sources), build,
install, test, try it out. I will HEARTILY welcome feedback (mail
(e-mail address removed)) telling me what worked and/or what didn't so I can
finalize this release -- and hopefully move on to a future 1.03 (I won't
aim to 1.03 until I'm convinced that 1.02 is OK...).


Thanks in advance,

Alex

 

[Daniel Nogradi]

If you're interested in gmpy (the Python wrapper of GMP, for

unlimited-precision arithmetic, rationals, random number generation,
number-theoretical functions, etc), please DO check out
http://code.google.com/p/gmpy/ -- gmpy 1.02 is there (as far as I can
tell) in a workable state. Source on Subversion (and a prerelease
zipfile too), downloadable binaries for MacOSX (download and read the
README file first!) and Windows (for Python 2.4 and 2.5 only, built and
minimally tested on a shaky Win2K+mingw -- on -- Parallels/MacOSX
setup... I have no other Windows machine to check 'em out...!).

Please help me check that the move-and-upgrade went OK -- download some
or all of the pieces (including an svn checkout of the sources), build,
install, test, try it out. I will HEARTILY welcome feedback (mail
(e-mail address removed)) telling me what worked and/or what didn't so I can
finalize this release -- and hopefully move on to a future 1.03 (I won't
aim to 1.03 until I'm convinced that 1.02 is OK...).


Thanks in advance,

Alex

Svn checkout, compilation and installation went okay but some tests
failed, this is the output of 'python test_gmpy.py':


Unit tests for gmpy 1.02 release candidate
on Python 2.5 (r25:51908, Nov 1 2006, 11:42:37)
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)]
Testing gmpy 1.02 (GMP 4.1.4), default caching (20, 20, -2..11)
gmpy_test_cvr 270 tests, 0 failures
gmpy_test_rnd 26 tests, 0 failures
gmpy_test_mpf 155 tests, 0 failures
gmpy_test_mpq 264 tests, 0 failures
**********************************************************************
File "/home/nogradi/tmp/gmpy/test/gmpy_test_mpz.py", line ?, in
gmpy_test_mpz.__test__.number
Failed example:
_memsize()-_siz
Expected:
0
Got:
-4
**********************************************************************
1 items had failures:
2 of 132 in gmpy_test_mpz.__test__.number
***Test Failed*** 2 failures.
gmpy_test_mpz 388 tests, 2 failures
**********************************************************************
1 items had failures:
2 of 132 in gmpy_test_mpz.__test__.number
***Test Failed*** 2 failures.
gmpy_test_dec 16 tests, 0 failures
7 items had no tests:
gmpy_test_cvr._test
gmpy_test_dec._test
gmpy_test_mpf._test
gmpy_test_mpq._test
gmpy_test_mpz._memsize
gmpy_test_mpz._test
gmpy_test_rnd._test
30 items passed all tests:
6 tests in gmpy_test_cvr
12 tests in gmpy_test_cvr.__test__.misc_stuff
252 tests in gmpy_test_cvr.__test__.user_errors
1 tests in gmpy_test_dec
15 tests in gmpy_test_dec.__test__.elemop
1 tests in gmpy_test_mpf
32 tests in gmpy_test_mpf.__test__.binio
33 tests in gmpy_test_mpf.__test__.cmpr
49 tests in gmpy_test_mpf.__test__.elemop
34 tests in gmpy_test_mpf.__test__.format
6 tests in gmpy_test_mpf.__test__.newdiv
2 tests in gmpy_test_mpq
26 tests in gmpy_test_mpq.__test__.binio
62 tests in gmpy_test_mpq.__test__.cmpr
66 tests in gmpy_test_mpq.__test__.elemop
54 tests in gmpy_test_mpq.__test__.format
12 tests in gmpy_test_mpq.__test__.newdiv
18 tests in gmpy_test_mpq.__test__.power
24 tests in gmpy_test_mpq.__test__.qdiv
4 tests in gmpy_test_mpz
34 tests in gmpy_test_mpz.__test__.binio
62 tests in gmpy_test_mpz.__test__.bitops
48 tests in gmpy_test_mpz.__test__.cmpr
42 tests in gmpy_test_mpz.__test__.elemop
36 tests in gmpy_test_mpz.__test__.format
14 tests in gmpy_test_mpz.__test__.index
12 tests in gmpy_test_mpz.__test__.newdiv
4 tests in gmpy_test_mpz.factorize
1 tests in gmpy_test_rnd
25 tests in gmpy_test_rnd.__test__.rand
**********************************************************************
1 items had failures:
2 of 132 in gmpy_test_mpz.__test__.number
1119 tests in 38 items.
1117 passed and 2 failed.
***Test Failed*** 2 failures.



HTH,
Daniel

 

[Alex Martelli]

Svn checkout, compilation and installation went okay but some tests
failed, this is the output of 'python test_gmpy.py':

Thanks for the report! That test is trying to check that a previously
existing memory leak went away, and unfortunately this requires
measuring the memory used by the process which is kind of flaky... in
particular:

Failed example:
_memsize()-_siz
Expected:
0
Got:
-4

....this means that (after a loop of the operation that used to leak)
your process is consuming 4KB _less_ than it was before the loop (?!).
Guess I'll have to make that test more "tolerant" of such small
flakiness... [done -- it's now in revision 10 of the SVN repository].

1 items had failures:
2 of 132 in gmpy_test_mpz.__test__.number
***Test Failed*** 2 failures.

That's actually just the one failure (all tests are repeated with and
without caching enabled).


Thanks again for your report!


Alex

 

[Daniel Nogradi]

Hi Alex,

I did another test, this time with python 2.4 on suse and things are
worse than in the previous case (which was python 2.5 on fedora 3),
ouput of 'python gmp_test.py' follows:


Unit tests for gmpy 1.02 release candidate
on Python 2.4 (#1, Mar 22 2005, 21:42:42)
[GCC 3.3.5 20050117 (prerelease) (SUSE Linux)]
Testing gmpy 1.02 (GMP 4.1.3), default caching (20, 20, -2..11)
gmpy_test_cvr 270 tests, 0 failures
gmpy_test_rnd 26 tests, 0 failures
gmpy_test_mpf 155 tests, 0 failures
gmpy_test_mpq 264 tests, 0 failures
gmpy_test_mpz 376 tests, 0 failures
**********************************************************************
File "/home/nogradi/tmp/gmpy/test/gmpy_test_dec.py", line ?, in
gmpy_test_dec.__test__.elemop
Failed example:
print f+d
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.4/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
File "<doctest gmpy_test_dec.__test__.elemop[3]>", line 1, in ?
print f+d
File "/usr/lib/python2.4/decimal.py", line 906, in __add__
other = _convert_other(other)
File "/usr/lib/python2.4/decimal.py", line 2863, in _convert_other
raise TypeError, "You can interact Decimal only with int, long
or Decimal data types."
TypeError: You can interact Decimal only with int, long or Decimal
data types.
**********************************************************************
File "/home/nogradi/tmp/gmpy/test/gmpy_test_dec.py", line ?, in
gmpy_test_dec.__test__.elemop
Failed example:
print d+f
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.4/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
File "<doctest gmpy_test_dec.__test__.elemop[4]>", line 1, in ?
print d+f
File "/usr/lib/python2.4/decimal.py", line 906, in __add__
other = _convert_other(other)
File "/usr/lib/python2.4/decimal.py", line 2863, in _convert_other
raise TypeError, "You can interact Decimal only with int, long
or Decimal data types."
TypeError: You can interact Decimal only with int, long or Decimal
data types.
**********************************************************************
File "/home/nogradi/tmp/gmpy/test/gmpy_test_dec.py", line ?, in
gmpy_test_dec.__test__.elemop
Failed example:
print q+d
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.4/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
File "<doctest gmpy_test_dec.__test__.elemop[5]>", line 1, in ?
print q+d
File "/usr/lib/python2.4/decimal.py", line 906, in __add__
other = _convert_other(other)
File "/usr/lib/python2.4/decimal.py", line 2863, in _convert_other
raise TypeError, "You can interact Decimal only with int, long
or Decimal data types."
TypeError: You can interact Decimal only with int, long or Decimal
data types.
**********************************************************************
File "/home/nogradi/tmp/gmpy/test/gmpy_test_dec.py", line ?, in
gmpy_test_dec.__test__.elemop
Failed example:
print d+q
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.4/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
File "<doctest gmpy_test_dec.__test__.elemop[6]>", line 1, in ?
print d+q
File "/usr/lib/python2.4/decimal.py", line 906, in __add__
other = _convert_other(other)
File "/usr/lib/python2.4/decimal.py", line 2863, in _convert_other
raise TypeError, "You can interact Decimal only with int, long
or Decimal data types."
TypeError: You can interact Decimal only with int, long or Decimal
data types.
**********************************************************************
File "/home/nogradi/tmp/gmpy/test/gmpy_test_dec.py", line ?, in
gmpy_test_dec.__test__.elemop
Failed example:
print z+d
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.4/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
File "<doctest gmpy_test_dec.__test__.elemop[7]>", line 1, in ?
print z+d
File "/usr/lib/python2.4/decimal.py", line 906, in __add__
other = _convert_other(other)
File "/usr/lib/python2.4/decimal.py", line 2863, in _convert_other
raise TypeError, "You can interact Decimal only with int, long
or Decimal data types."
TypeError: You can interact Decimal only with int, long or Decimal
data types.
**********************************************************************
File "/home/nogradi/tmp/gmpy/test/gmpy_test_dec.py", line ?, in
gmpy_test_dec.__test__.elemop
Failed example:
print d+z
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.4/doctest.py", line 1243, in __run
compileflags, 1) in test.globs
File "<doctest gmpy_test_dec.__test__.elemop[8]>", line 1, in ?
print d+z
File "/usr/lib/python2.4/decimal.py", line 906, in __add__
other = _convert_other(other)
File "/usr/lib/python2.4/decimal.py", line 2863, in _convert_other
raise TypeError, "You can interact Decimal only with int, long
or Decimal data types."
TypeError: You can interact Decimal only with int, long or Decimal
data types.
**********************************************************************
1 items had failures:
6 of 15 in gmpy_test_dec.__test__.elemop
***Test Failed*** 6 failures.
gmpy_test_dec 16 tests, 6 failures
7 items had no tests:
gmpy_test_cvr._test
gmpy_test_dec._test
gmpy_test_mpf._test
gmpy_test_mpq._test
gmpy_test_mpz._memsize
gmpy_test_mpz._test
gmpy_test_rnd._test
29 items passed all tests:
6 tests in gmpy_test_cvr
12 tests in gmpy_test_cvr.__test__.misc_stuff
252 tests in gmpy_test_cvr.__test__.user_errors
1 tests in gmpy_test_dec
1 tests in gmpy_test_mpf
32 tests in gmpy_test_mpf.__test__.binio
33 tests in gmpy_test_mpf.__test__.cmpr
49 tests in gmpy_test_mpf.__test__.elemop
34 tests in gmpy_test_mpf.__test__.format
6 tests in gmpy_test_mpf.__test__.newdiv
2 tests in gmpy_test_mpq
26 tests in gmpy_test_mpq.__test__.binio
62 tests in gmpy_test_mpq.__test__.cmpr
66 tests in gmpy_test_mpq.__test__.elemop
54 tests in gmpy_test_mpq.__test__.format
12 tests in gmpy_test_mpq.__test__.newdiv
18 tests in gmpy_test_mpq.__test__.power
24 tests in gmpy_test_mpq.__test__.qdiv
4 tests in gmpy_test_mpz
34 tests in gmpy_test_mpz.__test__.binio
62 tests in gmpy_test_mpz.__test__.bitops
48 tests in gmpy_test_mpz.__test__.cmpr
42 tests in gmpy_test_mpz.__test__.elemop
36 tests in gmpy_test_mpz.__test__.format
12 tests in gmpy_test_mpz.__test__.newdiv
134 tests in gmpy_test_mpz.__test__.number
4 tests in gmpy_test_mpz.factorize
1 tests in gmpy_test_rnd
25 tests in gmpy_test_rnd.__test__.rand
**********************************************************************
1 items had failures:
6 of 15 in gmpy_test_dec.__test__.elemop
1107 tests in 37 items.
1101 passed and 6 failed.
***Test Failed*** 6 failures.


HTH,
Daniel

 

[Alex Martelli]

Hi Alex,

I did another test, this time with python 2.4 on suse and things are
worse than in the previous case (which was python 2.5 on fedora 3),
ouput of 'python gmp_test.py' follows:

Interesting! gmpy interacts with decimal.Decimal by "monkey-
patching" that class on the fly; clearly the monkey-patching isn't
working with 2.4 on SuSE, so all the addition attempts are failing
(all 6 of them).

So the issue is finding out why this strategy is failing there, while
succeeding on other Linux distros, Mac, and Windows.

To that end, I have added a feature in the latest svn trunk (revision
11) to set gmpy's own options.debug flag from an environment variable
named GMPY_DEBUG at startup (if that environment variable is set).

If you could please svn up and rebuild, then running with GMPY_DEBUG
set in the environment should give something like:

brain:~/gmpy alex$ GMPY_DEBUG=1 python -c 'import gmpy' 2>&1 | tail -5
mp_allocate( 4->8 )
mp_allocate( 4->8 ) ->0x60b8b0
gmpy_module at 0x63390
gmpy_module imported decimal OK
gmpy_module tweaked decimal OK

This is the expected behavior when module decimal is present, while,
when it's absent, you should see something like:

brain:~/gmpy alex$ GMPY_DEBUG=1 python2.3 -c 'import gmpy' 2>&1 |
tail -5
Initing new not in zcache
mp_allocate( 4->8 )
mp_allocate( 4->8 ) ->0x3017d0
gmpy_module at 0x6de70
gmpy_module could not import decimal

In each case, what matters is the last line or two after "gmpy_module
at" -- the rest of the copious debugging output is irrelevant here.

Module decimal is expected to be present from 2.4 forward (though it
could also be explicitly installed on 2.3). However, from the test
failures you report, it looks like SuSE's 2.4 does not have decimal
available for importing when gmpy is starting up -- so I'd like to
confirm that first.

If it's confirmed, it will be interesting to discover how this
happens (but that requires a SuSE installation, which I don't have)
and secondarily what is gmpy to do about it -- maybe offer a warning
on startup when module decimal is unexpectedly not importable but
then skip the relevant unittests in that case? (Note that the tests
ARE already skipped if module decimal is not importable _at test
time_ -- what's really weird is that apparently on SuSE decimal CAN
be imported by the tests but CANNOT be imported earlier while gmpy is
loading, which is truly puzzling).

I'd appreciate reports about this behavior on as many Python/Linux
installations as possible -- yours, of course (to check my guess that
the problem is that decimal can't be imported while gmpy is loading),
but also others' (to see how widespread the problem is, etc, etc).

Thanks again for reporting this, and thanks in advance to anybody
who'll lend a hand in tracking down this little strangeness!!!


Alex

 

[Daniel Nogradi]

Hi Alex,

Interesting! gmpy interacts with decimal.Decimal by "monkey-
patching" that class on the fly; clearly the monkey-patching isn't
working with 2.4 on SuSE, so all the addition attempts are failing
(all 6 of them).

So the issue is finding out why this strategy is failing there, while
succeeding on other Linux distros, Mac, and Windows.

To that end, I have added a feature in the latest svn trunk (revision
11) to set gmpy's own options.debug flag from an environment variable
named GMPY_DEBUG at startup (if that environment variable is set).

If you could please svn up and rebuild, then running with GMPY_DEBUG
set in the environment should give something like:

brain:~/gmpy alex$ GMPY_DEBUG=1 python -c 'import gmpy' 2>&1 | tail -5
mp_allocate( 4->8 )
mp_allocate( 4->8 ) ->0x60b8b0
gmpy_module at 0x63390
gmpy_module imported decimal OK
gmpy_module tweaked decimal OK

This is the expected behavior when module decimal is present, while,
when it's absent, you should see something like:

brain:~/gmpy alex$ GMPY_DEBUG=1 python2.3 -c 'import gmpy' 2>&1 |
tail -5
Initing new not in zcache
mp_allocate( 4->8 )
mp_allocate( 4->8 ) ->0x3017d0
gmpy_module at 0x6de70
gmpy_module could not import decimal

In each case, what matters is the last line or two after "gmpy_module
at" -- the rest of the copious debugging output is irrelevant here.

Module decimal is expected to be present from 2.4 forward (though it
could also be explicitly installed on 2.3). However, from the test
failures you report, it looks like SuSE's 2.4 does not have decimal
available for importing when gmpy is starting up -- so I'd like to
confirm that first.

If it's confirmed, it will be interesting to discover how this
happens (but that requires a SuSE installation, which I don't have)
and secondarily what is gmpy to do about it -- maybe offer a warning
on startup when module decimal is unexpectedly not importable but
then skip the relevant unittests in that case? (Note that the tests
ARE already skipped if module decimal is not importable _at test
time_ -- what's really weird is that apparently on SuSE decimal CAN
be imported by the tests but CANNOT be imported earlier while gmpy is
loading, which is truly puzzling).

I'd appreciate reports about this behavior on as many Python/Linux
installations as possible -- yours, of course (to check my guess that
the problem is that decimal can't be imported while gmpy is loading),
but also others' (to see how widespread the problem is, etc, etc).

Thanks again for reporting this, and thanks in advance to anybody
who'll lend a hand in tracking down this little strangeness!!!

Okay, here is a summary. The notation is:

platform version python version debug unittest

where debug will be either 'pass' or 'fail' if that particular
platform and python version passes or fails the GMPY_DEBUG=1
importing/tweaking test. Unittest will also be 'pass' or 'fail' if the
full unittesting passes or fails respectively.

Fedora 3 python 2.5 pass pass
Fedora 3 python 2.4 pass fail
Fedora 3 python 2.3.4 fail pass
Suse 9.3 python 2.4 pass fail
Suse 10 python 2.4.1 pass pass
Suse 10.2(64bit) python 2.5 pass fail

The output from each test is zipped and you can grab it from here:
http://lorentz.leidenuniv.nl/~nogradi/gmpy_test.zip

HTH,
Daniel

 

[Ziga Seilnacht]

Interesting! gmpy interacts with decimal.Decimal by "monkey-
patching" that class on the fly; clearly the monkey-patching isn't
working with 2.4 on SuSE, so all the addition attempts are failing
(all 6 of them).

So the issue is finding out why this strategy is failing there, while
succeeding on other Linux distros, Mac, and Windows.

This is a bug in Python's decimal module in release 2.4.0. It was
fixed
in release 2.4.1:

http://svn.python.org/view?rev=38708&view=rev

Ziga

 

[aleaxit]

Okay, here is a summary. The notation is:

platform version python version debug unittest

where debug will be either 'pass' or 'fail' if that particular
platform and python version passes or fails the GMPY_DEBUG=1
importing/tweaking test. Unittest will also be 'pass' or 'fail' if the
full unittesting passes or fails respectively.

Fedora 3 python 2.5 pass pass
Fedora 3 python 2.4 pass fail
Fedora 3 python 2.3.4 fail pass
Suse 9.3 python 2.4 pass fail
Suse 10 python 2.4.1 pass pass
Suse 10.2(64bit) python 2.5 pass fail

The output from each test is zipped and you can grab it from here:http://lorentz.leidenuniv.nl/~nogradi/gmpy_test.zip

Thanks for all the testing! So, Ziga's next post explains that the
2.4 problems are 2.4.0 bugs fixed since 2.4.1 -- I don't think I can
do anything about in gmpy, nor (I think) even if I could should I go
to great complications to support old, known-buggy, easily upgradable
(I hope) releases of Python.

The 2.5 issues have nothing to do with decimal -- they _do_ have to do
with 64-bit builds:

For the hash(z) issue, I think I know what to do -- instead of
claiming I know what the hash value should be (which clearly I don't
when it can be either a 32-bit or 64-bit build, I'll just claim
that the hash value
of z is the same as the the hash value of long(z).

I have svn committed that change (svn repository for gmpy is now at
revision 14) -- could you please svn up and run the tests again
(ideally after updating the Python 2.4 installations to 2.4.1 or
better, but that's minor, what I'd really need to see would be the
results on the 64-bit build... thanks!


I _am_ a bit surprised at the other failure, the one of:

ix(_g.mpz(sys.maxint)) == sys.maxint

where ix is operator.index -- particularly considering that the other
tests around that one succeed (e.g., the identical one using
sys.maxint + 1). Maybe that's an actual bug of 1.02 on a 64-bit build
of Python.

Could you please tell me, on that 64-bit build, what happens with:
???

i.e., what are the values correspondiing to the ??? occurrences when
you actually do that...?


Thanks again


Alex

 

[aleaxit]

This is a bug in Python's decimal module in release 2.4.0. It was
fixed
in release 2.4.1:

http://svn.python.org/view?rev=38708&view=rev

Thanks! This was really helpful and saved me a lot of head-
scratching.

I'm wondering if I should specifically test for 2.4.0 in the tests,
and give a clear error message about decimal interoperability with
other types being broken in that release and suggesting an upgrade,
rather than producing "mysterious" test breakages... done -- I think
this may simplify things in the future.


Thanks,

Alex

 

[Daniel Nogradi]

Could you please tell me, on that 64-bit build, what happens with:

???

i.e., what are the values correspondiing to the ??? occurrences when
you actually do that...?

Here it goes (python 2.5 - 64bit):
0

I don't pretent to fully understand what's going on, but
gmpy.mpz(sys.maxint)==gmpy.mpz(-1) is more than suspicious

 

[Daniel Nogradi]

Could you please tell me, on that 64-bit build, what happens with:

Here it goes (python 2.5 - 64bit):


9223372036854775808L>>>gmpy.mpz(sys.maxint+1)

mpz(9223372036854775808L)>>> operator.index(gmpy.mpz(sys.maxint)+1)

0

I don't pretent to fully understand what's going on,
butgmpy.mpz(sys.maxint)==gmpy.mpz(-1) is more than suspicious

[directing Alex's message back to the list]


Yes, clearly there is some 32-vs-64 bit assumption around. When I
create an mpz from a Python int (a PyObject* i which passes Python's
int typecheck) I do that by calling mpz_from_c_long(PyInt_AsLong(i))
-- depending on what "long" means in C, I guess that might be the root
of the problem. There might be similar problems in creating mpq and
mpf from Python ints > 2**31-1 on 64-bit platforms, I guess.

I could try flailing around in the dark to fix it, but, without access
to a 64-bit Python platform, I don't really see any chance of me
fixing this.

If somebody can give me ssh access to such a platform, I'll be glad to
work to give it a try; of course, if anybody who does have access to
such a platform wants to send me patches or join the gmpy project,
that would also be great; failing either of these options, I don't see
much that I could do beyond pointing out in the docs that gmpy does
not correctly support 64-bits builds of Python, checking for them in
the unittests, etc.


Thanks,

Alex