R
revuesbio
Hi
Does anyone have the python version of the conversion from msbin to
ieee?
Thank u
Does anyone have the python version of the conversion from msbin to
ieee?
Thank u
G
Gabriel Genellina
I imagine this will be done just once - msbin is a really old format.
Instead of coding the conversion in Python, you could:
- write a small QuickBasic program using the functions CVSMBF, CVDMBF to
do the conversion
- download this DLL http://www.microdexterity.com/products.html
J
John Machin
Yes, Google has it. Google is your friend. Ask Google. It will lead
you to such as:
http://mail.python.org/pipermail/python-list/2005-August/337817.html
HTH,
John
R
revuesbio
Thank you,
I've already read it but the problem is always present. this script is
for double precision MBF format ( 8 bytes).
I try to adapt this script for single precision MBF format ( 4 bytes)
but i don't find the right float value.
for example : 'P\xad\x02\x95' will return '0.00024924660101532936'
J
John Machin
It would have been somewhat more helpful had you said what you had
done so far, even posted your code ...
If you know what the *correct* value is, you might like to consider
shifting left by log2(correct_value/erroneous_value)
Do you have any known correct pairs of (mbf4 string, decimal_float
value)? My attempt is below -- this is based on a couple of
descriptive sources that my friend Google found, with no test data. I
believe the correct answer for the above input is 1070506.0 i.e. you
are out by a factor of 2 ** 32
def mbf4_as_float(s):
m0, m1, m2, m3 = [ord(c) for c in s]
exponent = m3
if not exponent:
return 0.0
sign = m2 & 0x80
m2 |= 0x80
mant = (((m2 << 8) | m1) << 8) | m0
adj = 24 + 128
num = mant * 2.0 ** (exponent - adj)
if sign:
return -num
return num
HTH,
John
R
revuesbio
It would have been somewhat more helpful had you said what you had
done so far, even posted your code ...
If you know what the *correct* value is, you might like to consider
shifting left by log2(correct_value/erroneous_value)
Do you have any known correct pairs of (mbf4 string, decimal_float
value)? My attempt is below -- this is based on a couple of
descriptive sources that my friend Google found, with no test data. I
believe the correct answer for the above input is 1070506.0 i.e. you
are out by a factor of 2 ** 32
def mbf4_as_float(s):
m0, m1, m2, m3 = [ord(c) for c in s]
exponent = m3
if not exponent:
return 0.0
sign = m2 & 0x80
m2 |= 0x80
mant = (((m2 << 8) | m1) << 8) | m0
adj = 24 + 128
num = mant * 2.0 ** (exponent - adj)
if sign:
return -num
return num
HTH,
John
well done ! it's exactly what i'm waiting for !!
my code was:
b=bytes[:]
sign = bytes[-2] & 0x80
b[-2] |= 0x80
exp = bytes[-1] - 0x80 - 56
acc = 0L
for i,byte in enumerate(b[:-1]):
acc |= (long(byte)<<(i*8))
return (float(acc)*2.0**exp)*((1.,-1.)[sign!=0])
0.00024924660101532936
this script come from google groups but i don't understand bit-string
manipulation (I'm a newbie). informations about bit-string
manipulation with python is too poor on the net.
thank you very much for your script.
A.
J
John Machin
If you know what the *correct* value is, you might like to consider
shifting left by log2(correct_value/erroneous_value)
well done ! it's exactly what i'm waiting for !!
my code was:>>> from struct import *
b=bytes[:]
sign = bytes[-2] & 0x80
b[-2] |= 0x80
exp = bytes[-1] - 0x80 - 56
acc = 0L
for i,byte in enumerate(b[:-1]):
acc |= (long(byte)<<(i*8))
return (float(acc)*2.0**exp)*((1.,-1.)[sign!=0])
Apart from the 2**32 problem, the above doesn't handle *any* of the
2**24 different representations of zero. Try feeding \0\0\0\0' to it
and see what you get.
The basic operations (and, or, exclusive-or, shift) are not specific
to any language. Several languages share the same notation (& | ^ <<
Don't thank me, publish some known correct pairs of values so that we
can verify that it's not just accidentally correct for 1 pair of
values.
R
revuesbio
Hi
Does anyone have the python version of the conversion from msbin to
ieee?
Thank u
Yes, Google has it. Google is your friend. Ask Google. It will lead
you to such as:
http://mail.python.org/pipermail/python-list/2005-August/337817.html
HTH,
John
Thank you,
I've already read it but the problem is always present. this script is
for double precision MBF format ( 8 bytes).
It would have been somewhat more helpful had you said what you had
done so far, even posted your code ...
I try to adapt this script for single precision MBF format ( 4 bytes)
but i don't find the right float value.
for example : 'P\xad\x02\x95' will return '0.00024924660101532936'
If you know what the *correct* value is, you might like to consider
shifting left by log2(correct_value/erroneous_value)
Do you have any known correct pairs of (mbf4 string, decimal_float
value)? My attempt is below -- this is based on a couple of
descriptive sources that my friend Google found, with no test data. I
believe the correct answer for the above input is 1070506.0 i.e. you
are out by a factor of 2 ** 32
def mbf4_as_float(s):
m0, m1, m2, m3 = [ord(c) for c in s]
exponent = m3
if not exponent:
return 0.0
sign = m2 & 0x80
m2 |= 0x80
mant = (((m2 << 8) | m1) << 8) | m0
adj = 24 + 128
num = mant * 2.0 ** (exponent - adj)
if sign:
return -num
return num
HTH,
John
Apart from the 2**32 problem, the above doesn't handle *any* of the
2**24 different representations of zero. Try feeding \0\0\0\0' to it
and see what you get.
The basic operations (and, or, exclusive-or, shift) are not specific
to any language. Several languages share the same notation (& | ^ <<
Don't thank me, publish some known correct pairs of values so that we
can verify that it's not just accidentally correct for 1 pair of
values.
pairs of values :
(bytes string, mbf4_as_float(s) result) right
float value
('P\xad\x02\x95', 1070506.0)
1070506.0
('\x00\x00\x00\x02', 5.8774717541114375e-039) 0.0
('\x00\x00\x00\x81', 1.0)
1.0
('\x00\x00\x00\x82', 2.0)
2.0
('\x00\x00@\x82', 3.0)
3.0
('\x00\x00\x00\x83', 4.0)
4.0
('\x00\x00 \x83', 5.0)
5.0
('\xcd\xcc\x0c\x81', 1.1000000238418579) 1.1
('\xcd\xcc\x0c\x82', 2.2000000476837158) 2.2
('33S\x82', 3.2999999523162842) 3.3
('\xcd\xcc\x0c\x83', 4.4000000953674316) 4.4
('\x00\x00z\x8a', 1000.0)
1000.0
J
John Machin
Hi
Does anyone have the python version of the conversion from msbin to
ieee?
Thank u
Yes, Google has it. Google is your friend. Ask Google. It will lead
you to such as:
http://mail.python.org/pipermail/python-list/2005-August/337817.html
HTH,
John
Thank you,
I've already read it but the problem is always present. this script is
for double precision MBF format ( 8 bytes).
It would have been somewhat more helpful had you said what you had
done so far, even posted your code ...
I try to adapt this script for single precision MBF format ( 4 bytes)
but i don't find the right float value.
for example : 'P\xad\x02\x95' will return '0.00024924660101532936'
If you know what the *correct* value is, you might like to consider
shifting left by log2(correct_value/erroneous_value)
Do you have any known correct pairs of (mbf4 string, decimal_float
value)? My attempt is below -- this is based on a couple of
descriptive sources that my friend Google found, with no test data. I
believe the correct answer for the above input is 1070506.0 i.e. you
are out by a factor of 2 ** 32
def mbf4_as_float(s):
m0, m1, m2, m3 = [ord(c) for c in s]
exponent = m3
if not exponent:
return 0.0
sign = m2 & 0x80
m2 |= 0x80
mant = (((m2 << 8) | m1) << 8) | m0
adj = 24 + 128
num = mant * 2.0 ** (exponent - adj)
if sign:
return -num
return num
HTH,
John
well done ! it's exactly what i'm waiting for !!
my code was:>>> from struct import *
x = list(unpack('BBBB','P\xad\x02\x95'))
x
[80, 173, 2, 149]
def conversion1(bytes):
b=bytes[:]
sign = bytes[-2] & 0x80
b[-2] |= 0x80
exp = bytes[-1] - 0x80 - 56
acc = 0L
for i,byte in enumerate(b[:-1]):
acc |= (long(byte)<<(i*8))
return (float(acc)*2.0**exp)*((1.,-1.)[sign!=0])
pairs of values :
(bytes string, mbf4_as_float(s) result) right
float value
('P\xad\x02\x95', 1070506.0)
1070506.0
('\x00\x00\x00\x02', 5.8774717541114375e-039) 0.0
There is no way that \x00\x00\x00\x02' could represent exactly zero.
What makes you think it does? Rounding?
It is not apparent whether you regard the output from the function as
correct or not.
4.4 "converted" to mbf4 format is '\xcd\xcc\x0c\x83' which is
4.4000000953674316 which is the closest possible mbf4 representation
of 4.4 (difference is 9.5e-008).
The next lower mbf4 value '\xcc\xcc\x0c\x83' is 4.3999996185302734
(difference is -3.8e-007).
Note that floating-point representation of many decimal fractions is
inherently inexact. print repr(4.4) produces 4.4000000000000004
Have you read this:
http://docs.python.org/tut/node16.html
?
If you need decimal-fraction output that matches what somebody typed
into the original software, or saw on the screen, you will need to
know/guess the precision that was involved, and round the numbers
accordingly -- just like the author of the original software would
have needed to do.
['4', '4.4', '4.40', '4.400', '4.4000', '4.40000', '4.400000',
'4.4000001', '4.40000010', '4.400000095']
HTH,
John
R
revuesbio
Hi
Does anyone have the python version of the conversion from msbin to
ieee?
Thank u
Yes, Google has it. Google is your friend. Ask Google. It will lead
you to such as:
http://mail.python.org/pipermail/python-list/2005-August/337817.html
HTH,
John
Thank you,
I've already read it but the problem is always present. this script is
for double precision MBF format ( 8 bytes).
It would have been somewhat more helpful had you said what you had
done so far, even posted your code ...
I try to adapt this script for single precision MBF format ( 4 bytes)
but i don't find the right float value.
for example : 'P\xad\x02\x95' will return '0.00024924660101532936'
If you know what the *correct* value is, you might like to consider
shifting left by log2(correct_value/erroneous_value)
Do you have any known correct pairs of (mbf4 string, decimal_float
value)? My attempt is below -- this is based on a couple of
descriptive sources that my friend Google found, with no test data. I
believe the correct answer for the above input is 1070506.0 i.e. you
are out by a factor of 2 ** 32
def mbf4_as_float(s):
m0, m1, m2, m3 = [ord(c) for c in s]
exponent = m3
if not exponent:
return 0.0
sign = m2 & 0x80
m2 |= 0x80
mant = (((m2 << 8) | m1) << 8) | m0
adj = 24 + 128
num = mant * 2.0 ** (exponent - adj)
if sign:
return -num
return num
HTH,
John
well done ! it's exactly what i'm waiting for !!
my code was:>>> from struct import *
x = list(unpack('BBBB','P\xad\x02\x95'))
x
[80, 173, 2, 149]
def conversion1(bytes):
b=bytes[:]
sign = bytes[-2] & 0x80
b[-2] |= 0x80
exp = bytes[-1] - 0x80 - 56
acc = 0L
for i,byte in enumerate(b[:-1]):
acc |= (long(byte)<<(i*8))
return (float(acc)*2.0**exp)*((1.,-1.)[sign!=0])
Apart from the 2**32 problem, the above doesn't handle *any* of the
2**24 different representations of zero. Try feeding \0\0\0\0' to it
and see what you get.
conversion1(x)
0.00024924660101532936
this script come from google groups but i don't understand bit-string
manipulation (I'm a newbie). informations about bit-string
manipulation with python is too poor on the net.
The basic operations (and, or, exclusive-or, shift) are not specific
to any language. Several languages share the same notation (& | ^ <<
), having inherited it from C.
thank you very much for your script.
Don't thank me, publish some known correct pairs of values so that we
can verify that it's not just accidentally correct for 1 pair of
values.
There is no way that \x00\x00\x00\x02' could represent exactly zero.
What makes you think it does? Rounding?
It is not apparent whether you regard the output from the function as
correct or not.
4.4 "converted" to mbf4 format is '\xcd\xcc\x0c\x83' which is
4.4000000953674316 which is the closest possible mbf4 representation
of 4.4 (difference is 9.5e-008).
The next lower mbf4 value '\xcc\xcc\x0c\x83' is 4.3999996185302734
(difference is -3.8e-007).
Note that floating-point representation of many decimal fractions is
inherently inexact. print repr(4.4) produces 4.4000000000000004
Have you read this:
http://docs.python.org/tut/node16.html
?
If you need decimal-fraction output that matches what somebody typed
into the original software, or saw on the screen, you will need to
know/guess the precision that was involved, and round the numbers
accordingly -- just like the author of the original software would
have needed to do.
['4', '4.4', '4.40', '4.400', '4.4000', '4.40000', '4.400000',
'4.4000001', '4.40000010', '4.400000095']
HTH,
John
another couples and round number corresponding to the right value
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('0\x8b\x01\x95', 1061222.0, '1061222.000')
('\xb8\x1e=\x83', 5.9099998474121094, '5.910')
(')\\O\x83', 6.4800000190734863, '6.480')
('\x9a\x99A\x83', 6.0500001907348633, '6.050')
('\x00\x00P\x83', 6.5, '6.500')
('8BY\x95', 1779783.0, '1779783.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('\xe0\xa0\x02\x95', 1070108.0, '1070108.000')
('33{\x83', 7.8499999046325684, '7.850')
('q=z\x83', 7.820000171661377, '7.820')
('33s\x83', 7.5999999046325684, '7.600')
(')\\\x7f\x83', 7.9800000190734863, '7.980')
('\x00\x9aX\x92', 221800.0, '221800.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('0\xa1\x02\x95', 1070118.0, '1070118.000')
('\x85\xebq\x83', 7.559999942779541, '7.560')
('\x14\xaeo\x83', 7.4899997711181641, '7.490')
('\xcd\xccT\x83', 6.6500000953674316, '6.650')
('\x00\x00p\x83', 7.5, '7.500')
('\x00\xa4N\x92', 211600.0, '211600.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('\x90\xa1\x02\x95', 1070130.0, '1070130.000')
('\xaeGa\x83', 7.0399999618530273, '7.040')
('\xc3\xf5p\x83', 7.5300002098083496, '7.530')
('\x8f\xc2e\x83', 7.179999828338623, '7.180')
('H\xe1b\x83', 7.0900001525878906, '7.090')
('\xc0\xe27\x93', 376598.0, '376598.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('\x08\xa4\x02\x95', 1070209.0, '1070209.000')
('\x9a\x99a\x83', 7.0500001907348633, '7.050')
('\xd7\xa3x\x83', 7.7699999809265137, '7.770')
('H\xe1r\x83', 7.5900001525878906, '7.590')
('{\x14v\x83', 7.690000057220459, '7.690')
('\x80.W\x93', 440692.0, '440692.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('h\xa4\x02\x95', 1070221.0, '1070221.000')
('\x8f\xc2\x01\x84', 8.1099996566772461, '8.110')
('=\n\x03\x84', 8.1899995803833008, '8.190')
('\xcd\xcc\x00\x84', 8.0500001907348633, '8.050')
('ffv\x83', 7.6999998092651367, '7.700')
('\x80X\x1a\x94', 632200.0, '632200.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('\x08\xa7\x02\x95', 1070305.0, '1070305.000')
('33s\x83', 7.5999999046325684, '7.600')
('q=r\x83', 7.570000171661377, '7.570')
('\\\x8fj\x83', 7.3299999237060547, '7.330')
('33k\x83', 7.3499999046325684, '7.350')
('\xc0a\r\x94', 579100.0, '579100.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('X\xa7\x02\x95', 1070315.0, '1070315.000')
('\xcd\xcc|\x83', 7.9000000953674316, '7.900')
('q=z\x83', 7.820000171661377, '7.820')
('\x00\x00p\x83', 7.5, '7.500')
('\x00\x00p\x83', 7.5, '7.500')
('\x00\x1b7\x92', 187500.0, '187500.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('\xb8\xa7\x02\x95', 1070327.0, '1070327.000')
('{\x14~\x83', 7.940000057220459, '7.940')
('\xcd\xcc\x04\x84', 8.3000001907348633, '8.300')
('\xe1z\x00\x84', 8.0299997329711914, '8.030')
('\xcd\xcc\x10\x84', 9.0500001907348633, '9.050')
('\x00R\x00\x95', 1051200.0, '1051200.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('P\xaa\x02\x95', 1070410.0, '1070410.000')
('R\xb8\x1e\x84', 9.9200000762939453, '9.920')
('\xd7\xa3\x1c\x84', 9.7899999618530273, '9.790')
('\x85\xeb\x19\x84', 9.619999885559082, '9.620')
('\x9a\x99\x19\x84', 9.6000003814697266, '9.600')
('\x98\x1c\x0c\x95', 1147795.0, '1147795.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('\xa0\xaa\x02\x95', 1070420.0, '1070420.000')
('=\n\x0f\x84', 8.9399995803833008, '8.940')
('ff\x0e\x84', 8.8999996185302734, '8.900')
('\xe1z\x0c\x84', 8.7799997329711914, '8.780')
('\x1f\x85\x0f\x84', 8.9700002670288086, '8.970')
('\x00\x1d&\x92', 170100.0, '170100.000')
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000')
('8\xad\x02\x95', 1070503.0, '1070503.000')
('\xf6(\x0c\x84', 8.7600002288818359, '8.760')
('\xe1z\x14\x84', 9.2799997329711914, '9.280')
all is ok.
thank u
J
John Machin
Hi
Does anyone have the python version of the conversion from msbin to
ieee?
Thank u
Yes, Google has it. Google is your friend. Ask Google. It will lead
you to such as:
http://mail.python.org/pipermail/python-list/2005-August/337817.html
HTH,
John
Thank you,
I've already read it but the problem is always present. this script is
for double precision MBF format ( 8 bytes).
It would have been somewhat more helpful had you said what you had
done so far, even posted your code ...
I try to adapt this script for single precision MBF format ( 4 bytes)
but i don't find the right float value.
for example : 'P\xad\x02\x95' will return '0.00024924660101532936'
If you know what the *correct* value is, you might like to consider
shifting left by log2(correct_value/erroneous_value)
Do you have any known correct pairs of (mbf4 string, decimal_float
value)? My attempt is below -- this is based on a couple of
descriptive sources that my friend Google found, with no test data. I
believe the correct answer for the above input is 1070506.0 i.e. you
are out by a factor of 2 ** 32
def mbf4_as_float(s):
m0, m1, m2, m3 = [ord(c) for c in s]
exponent = m3
if not exponent:
return 0.0
sign = m2 & 0x80
m2 |= 0x80
mant = (((m2 << 8) | m1) << 8) | m0
adj = 24 + 128
num = mant * 2.0 ** (exponent - adj)
if sign:
return -num
return num
HTH,
John
well done ! it's exactly what i'm waiting for !!
my code was:>>> from struct import *
x = list(unpack('BBBB','P\xad\x02\x95'))
x
[80, 173, 2, 149]
def conversion1(bytes):
b=bytes[:]
sign = bytes[-2] & 0x80
b[-2] |= 0x80
exp = bytes[-1] - 0x80 - 56
acc = 0L
for i,byte in enumerate(b[:-1]):
acc |= (long(byte)<<(i*8))
return (float(acc)*2.0**exp)*((1.,-1.)[sign!=0])
Apart from the 2**32 problem, the above doesn't handle *any* of the
2**24 different representations of zero. Try feeding \0\0\0\0' to it
and see what you get.
conversion1(x)
0.00024924660101532936
this script come from google groups but i don't understand bit-string
manipulation (I'm a newbie). informations about bit-string
manipulation with python is too poor on the net.
The basic operations (and, or, exclusive-or, shift) are not specific
to any language. Several languages share the same notation (& | ^ <<
), having inherited it from C.
thank you very much for your script.
Don't thank me, publish some known correct pairs of values so that we
can verify that it's not just accidentally correct for 1 pair of
values.
pairs of values :
(bytes string, mbf4_as_float(s) result) right
float value
('P\xad\x02\x95', 1070506.0)
1070506.0
('\x00\x00\x00\x02', 5.8774717541114375e-039) 0.0
another couples and round number corresponding to the right value
('\x00\x00\x00\x02', 5.8774717541114375e-039, '0.000') [snip]
all is ok.
thank u
I have not yet found a comprehensive let alone authoritative
description of the Microsoft binary floating format. However I've seen
enough to form a view that in general converting '\x00\x00\x00\x02' to
0.0 would be a mistake, and that 5.8774717541114375e-039 is the
correct answer.
Why do I think so? There's a Borland/Inprise document on the
wotsit.org website that gives C functions for conversion both ways
between MBF and IEEE formats (both 32 bits and 64 bits). They are
supposed to mimic functions that were in the MS C runtime library at
one stage. The _fieeetomsbin (32 bits) function does NOT make a
special case of IEEE 0.0; it passes it through the normal what is the
exponent, what is the mantissa routine, and produces
'\x00\x00\x00\x02' (ms exponent field == 2). The converse routine
regards any MBF number with exponent 0 as being 0.0, and puts anything
else through the normal cycle -- which is a nonsense with MBF exponent
== 1, by the way (because of the offset of 2, the result in IEEE-32-
bit is an exponent of -1 which becomes 255 which tags the result as
infinity or NaN (not a number)). The lack of round-trip sameness for
0.0 is so astonishing that it this were true one would have expected
it to be remarked on somewhere.
So: It is probably sufficient for your application to round everything
to 3 decimal places, but I thought I'd better leave this note to warn
anyone else who might want to use the function.
I am curious as to what software created your MBF-32-bit numbers ...
care to divulge?
Cheers,
John
R
revuesbio
Hi,
I found something interresting.
First, MBF Files come from metastock software but i use another one
(MLDownloader) to get quotes and convert them to MBF format probably
using functions you've just described (C, borland,...). In final, all
my files are created by mldownloader.
2nd, I've tried to modify quotes directly in metastock.
And when you read bytes corresponding to "zero" ... :
'\x00\x00\x00\x00' !
cheers,
Antoine
I found something interresting.
First, MBF Files come from metastock software but i use another one
(MLDownloader) to get quotes and convert them to MBF format probably
using functions you've just described (C, borland,...). In final, all
my files are created by mldownloader.
2nd, I've tried to modify quotes directly in metastock.
And when you read bytes corresponding to "zero" ... :
'\x00\x00\x00\x00' !
cheers,
Antoine
I
imageguy
Not sure if this helps, but I think this thread has the answer;
http://groups.google.com/group/comp...?lnk=gst&q=geskerrett&rnum=2#ce76d5fcd887a47d
Check out the response from Bengt Richter. His function did the right
thing.
J
John Machin
Yes, Bengt's function did the right thing on the input for that
particular problem, which involved IEEE 64-bit floating point numbers
stored in little-endian format.
The current problem involves 32-bit MBF (Microsoft Binary/Basic
Floating-point/Format) numbers.
Different problem.