Module python-magic on/for Windows?

[Larry Hale]

I've heard tell of a Python binding for libmagic (file(1) *nixy
command; see http://darwinsys.com/file/). Generally, has anybody
built this and worked with it under Windows?

The only thing I've been able to find is the python-magic module at
http://hupp.org/adam/hg/python-magic/.

[1] Is this "THE" python-magic module. (It seems to be to me, but
obviously I don't know.

[2] Has anybody been able to build THIS version under Windows?

I've gotten as far as completing the "setup.py install" process.
(After many troubles; I'll post the top-to-bottom HowTo-like info if/
when I ever get it to work.

At this point, there -is- a "magic" module that can be imported
(attempted-to, that is), but it fails.

If I go to a cmd window, run the Py interpreter, and do "import magic"
I get:

[1] an error pop-up (Windows window, which is blocking [as opposed
to "non-blocking", not "obscuring", though it does that, too] the cmd
window):

(X) This application has failed to start because magic1.dll
was not found. Re-installing the application may fix this problem.
[OK]

[2] then, within the interpreter, I get:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build\bdist.win32\egg\magic.py", line 2, in <module>
File "build\bdist.win32\egg\cmagic.py", line 7, in <module>
File "build\bdist.win32\egg\_cmagic.py", line 7, in <module>
File "build\bdist.win32\egg\_cmagic.py", line 6, in
__bootstrap__
ImportError: DLL load failed: The specified module could not
be found.

I'm using Python 2.5 on Windows XP Pro. I've got CYGWIN installed
(more info can be provided if necessary) for a copy of file.exe (and
libmagic.a @ 357KB and libmagic.dll.a @ 25KB in C:\cygwin\lib). I
also have GNUWin32, also for file.exe alternatively (and
libmagic.dll.a @ ~ 8KB in C:\Program Files\GnuWin32\lib).

magic.py in C:\Program Files\Python25\Lib\site-packages\magic-0.1-
py2.5-win32.egg imports cmagic.py (also in this egg), which imports
_cmagic.py, which has the following:

def __bootstrap__():
global __bootstrap__, __loader__, __file__
import sys, pkg_resources, imp
__file__ =
pkg_resources.resource_filename(__name__,'_cmagic.pyd')
del __bootstrap__, __loader__
imp.load_dynamic(__name__,__file__)
__bootstrap__()

Now I *presume* my problem (at this point) is that I need to have
libmagic named as "magic1.dll" -wherever- this module is looking for
it. I'm just not sure, let alone if this is true, WHERE Python/
modules expect to find such things.

Also, which version(s)/file(s) should be placed where, or...??


Thanks for any/all help/pointers. I apologize up-front if I left out
any pertinent info; I'm paranoid about putting in too much (and some
that may be worthless) already...



Cheers,
Larry Hale

 

[Michael Torrie]

Now I *presume* my problem (at this point) is that I need to have
libmagic named as "magic1.dll" -wherever- this module is looking for
it. I'm just not sure, let alone if this is true, WHERE Python/
modules expect to find such things.

Also, which version(s)/file(s) should be placed where, or...??

The dll you are looking for is here:

http://downloads.sourceforge.net/gnuwin32/file-4.21-bin.zip?modtime=1180175868&big_mirror=1

The .dll.a file you mention is just a linker stub for when you want to
compile against it (as you are doing when building python-magic).

The magic1.dll file in the file-4.21-bin.zip package should be placed
anywhere in the path. Since you'll be using it with python, you could
probably stick it in the python25/bin folder (where ever your python
system is installed).

 

[Larry Hale]

(quoted text below my reply)

THANK YOU SO MUCH, Michael!

Criminitly! Figures! HA! I had it the WHOLE TIME, but didn't
realize it! (As mentioned, I had FILE installed from GnuWin32.)
DUH! I just neglected to do a search for "magic1.dll"... the most
obvious of the obvious!

I copied it from C:\Program Files\GnuWin32\bin\magic1.dll to C:
\Program Files\python25\libs.

Imports fine now. (I'll *test* it later. Then write-up my HowTo... I
think I'll even see if I can make an EGG for others... SOUNDS "easy"
with EasyInstall [http://peak.telecommunity.com/DevCenter/
EasyInstall]... I'll -see-... )

Ah, sometimes one CAN'T see the forest for the trees; a few hours
hacking away at getting things going to THIS point will do it to ya...
err, or at least "to me"!


Thanks, again, Michael...
-Larry

 

[Larry Hale]

After putting the magic1.dll file to C:\Program Files\Python25\DLLs
(looking at the directory structure, that seemed as good a place as
any), now I can *import* the module, but...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build\bdist.win32\egg\magic.py", line 32, in __init__
File "build\bdist.win32\egg\magic.py", line 72, in _check_error
magic.MagicException: could not find any magic files!

I've placed

474,146 magic
1,037,440 magic.mgc
32,203 magic.mime
45,696 magic.mime.mgc
77,312 magic1.dll

into

C:\windows\system32
in Windows/System PATH
AND (simultaneously)
C:\Program Files\python25\DLLs
in sys.path/PYTHONPATH

(I've also tried putting copies into different dirs in both PYTHONPATH
and system PATH, to no avail.)

Obviously, magic1.dll/magic.py (et al) can't FIND the needed file(s),
but I'm not sure how to go about discerning WHERE it/they are looking
- any thought(s)/suggestion(s), anyone??


Thanks in advance!
-Larry


P.S.: Thanks, again, Michael! - L

 

[Larry Hale]

I've heard tell of a Python binding for libmagic (file(1) *nixy
command; seehttp://darwinsys.com/file/). Generally, has anybody
built this and worked with it under Windows?

The only thing I've been able to find is the python-magic module athttp://hupp.org/adam/hg/python-magic/.

[1] Is this "THE" python-magic module. (It seems to be to me, but
obviously I don't know.

[2] Has anybody been able to build THIS version under Windows?

I've gotten as far as completing the "setup.py install" process.
(After many troubles; I'll post the top-to-bottom HowTo-like info if/
when I ever get it to work.

At this point, there -is- a "magic" module that can be imported
(attempted-to, that is), but it fails.

If I go to a cmd window, run the Py interpreter, and do "import magic"
I get:

[1] an error pop-up (Windows window, which is blocking [as opposed
to "non-blocking", not "obscuring", though it does that, too] the cmd
window):

(X) This application has failed to start because magic1.dll
was not found. Re-installing the application may fix this problem.
[OK]

[2] then, within the interpreter, I get:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build\bdist.win32\egg\magic.py", line 2, in <module>
File "build\bdist.win32\egg\cmagic.py", line 7, in <module>
File "build\bdist.win32\egg\_cmagic.py", line 7, in <module>
File "build\bdist.win32\egg\_cmagic.py", line 6, in
__bootstrap__
ImportError: DLL load failed: The specified module could not
be found.

I'm using Python 2.5 on Windows XP Pro. I've got CYGWIN installed
(more info can be provided if necessary) for a copy of file.exe (and
libmagic.a @ 357KB and libmagic.dll.a @ 25KB in C:\cygwin\lib). I
also have GNUWin32, also for file.exe alternatively (and
libmagic.dll.a @ ~ 8KB in C:\Program Files\GnuWin32\lib).

magic.py in C:\Program Files\Python25\Lib\site-packages\magic-0.1-
py2.5-win32.egg imports cmagic.py (also in this egg), which imports
_cmagic.py, which has the following:

def __bootstrap__():
global __bootstrap__, __loader__, __file__
import sys, pkg_resources, imp
__file__ =
pkg_resources.resource_filename(__name__,'_cmagic.pyd')
del __bootstrap__, __loader__
imp.load_dynamic(__name__,__file__)
__bootstrap__()

Now I *presume* my problem (at this point) is that I need to have
libmagic named as "magic1.dll" -wherever- this module is looking for
it. I'm just not sure, let alone if this is true, WHERE Python/
modules expect to find such things.

Also, which version(s)/file(s) should be placed where, or...??

Thanks for any/all help/pointers. I apologize up-front if I left out
any pertinent info; I'm paranoid about putting in too much (and some
that may be worthless) already...

Cheers,
Larry Hale

ALSO: I've even tried putting the 4 "magic" files INTO the .egg
file... still no-go. :/


Thanks, all!
-Larry

 

[Michael Torrie]

ALSO: I've even tried putting the 4 "magic" files INTO the .egg
file... still no-go. :/

It's often the custom of programs ported from unix to windows to use the
dll location as a key to find the other files that are typically in
share, or etc. GTK, for example uses ../share and ../etc from the
location where the dlls are stored (if the dlls are in a folder called bin).

In this case I'd try making a folder in the Python25 folder called
"share" and put the contents of the gnuwin32 share/file stuff in there.
Should look something like this:

c:/python25/share/file/magic.mime.mgc
c:/python25/share/file/magic
c:/python25/share/file/magic.mgc
c:/python25/share/file/magic.mime

Or, judging by a string I found in the dll itself, you might have to put
the files here:

c:/progra~1/File/share/file/magic

Although if such a path really is hardcoded into the dll, this is
certainly a bug, since it will fail on certain non-english windows systems.

Anyway, who knows. Give these options a try.

 

[Larry Hale]

THANKS, AGAIN, for the reply!

Okay, now I -really- feel silly... :>

So, when I was going to try what you'd suggested, I noticed something
peculiar: I hadn't "changed" anything on my system, but now, when I
tried to "import magic" I got a missing DLL error. For a DIFFERENT
DLL! This got me to thinking...

Turns out, when I was having "success" earlier, I was -in- the C:
\Program Files\GnuWin32\bin directory (where file.exe and related .dll
files are).

So I added C:\Program Files\GnuWin32\bin to the (system/Windows)
PATH. Everything worked. I removed all the copious copies of the
DLL, the magic files, etc., littering my hard drive in every other
location. Still worked.

'Course, I don't honestly know why it didn't work with the files
copied to other locales contained in the sys/Py paths... perhaps it
was, as you mentioned, wanting the tree-structure to be the same? (I
haven't the heart to check this out fully right now... *maybe* at a
later time... )

Yeah, I know. But HONESTLY! I could have -sworn- I'd already tried
THAT... you know, all the "obvious"/"I've been programming/working in
computers (albeit 'Windows' until lately) for ~25 years; yeah, I'm a
'hacker'..." stuff. I -swear-!

[EGG ON FACE]

[SIGH] Well, I'd say "live and learn", but I'm still scratching my
head. But, 'nough of that. Thanks for your replies, Michael, and I'm
off to work _with_ the module now, then work-up my HowTo (and
experiment with making an "egg"?? )...


Caio!
-Larry

 

[Larry Hale]

THANKS, AGAIN, for the reply!

Okay, now I -really- feel silly... :>

So, when I was going to try what you'd suggested, I noticed something
peculiar: I hadn't "changed" anything on my system, but now, when I
tried to "import magic" I got a missing DLL error. For a DIFFERENT
DLL! This got me to thinking...

Turns out, when I was having "success" earlier, I was -in- the C:
\Program Files\GnuWin32\bin directory (where file.exe and related .dll
files are).

So I added C:\Program Files\GnuWin32\bin to the (system/Windows)
PATH. Everything worked. I removed all the copious copies of the
DLL, the magic files, etc., littering my hard drive in every other
location. Still worked.

'Course, I don't honestly know why it didn't work with the files
copied to other locales contained in the sys/Py paths... perhaps it
was, as you mentioned, wanting the tree-structure to be the same? (I
haven't the heart to check this out fully right now... *maybe* at a
later time... )

Yeah, I know. But HONESTLY! I could have -sworn- I'd already tried
THAT... you know, all the "obvious"/"I've been programming/working in
computers (albeit 'Windows' until lately) for ~25 years; yeah, I'm a
'hacker'..." stuff. I -swear-!

[EGG ON FACE]

[SIGH] Well, I'd say "live and learn", but I'm still scratching my
head. But, 'nough of that. Thanks for your replies, Michael, and I'm
off to work _with_ the module now, then work-up my HowTo (and
experiment with making an "egg"?? )...

Caio!
-Larry

It's often the custom of programs ported from unix to windows to use the
dll location as a key to find the other files that are typically in
share, or etc. GTK, for example uses ../share and ../etc from the
location where the dlls are stored (if the dlls are in a folder called bin).

In this case I'd try making a folder in the Python25 folder called
"share" and put the contents of the gnuwin32 share/file stuff in there.
Should look something like this:

Or, judging by a string I found in the dll itself, you might have to put
the files here:

Although if such a path really is hardcoded into the dll, this is
certainly a bug, since it will fail on certain non-english windows systems.

Anyway, who knows. Give these options a try.

UPDATE: HOORAY!!! Well, actually, was about two steps forward, one
back. But I've gotten it all sorted out now!

I added C:\Program Files\GnuWin32\bin to the system/Windows PATH.
This takes care of the various required DLLs/libs.

The file source (previously linked from http://hupp.org/adam/hg/python-magic/)
has the man pages. I'd overlooked these as they were only modified as
necessary; the "BSD" part made me think they'd be the same as the same
online versions I'd already read (that were *nix specific).
Obviously, I was wrong. These had the default paths to the magic*
files listed; C:\Program Files\File\share\file\.

If one creates said directory structure and moves/copies/unpacks the
magic ("database") files THERE, then one may do:
Star Trek game file here to test with...
'MS-DOS executable, MZ for MS-DOS' # <-- This is what's
returned...

Alternately, if one wishes to leave/place the magic files elsewhere,
do like:
\share\\file\\magic' ) # <-- spec where/what the file is

NOTE: Even if the "magic_file" location *is* specified, "mime = True"
still works fine. (Haven't checked, but I presume the source simply
tacks ".mime" to the filename automagically.) Obviously/also, one
needn't specify the argument names if one uses proper argument order:
magic.Magic( mime, magic_file )


THANKS SO MUCH, Michael, for your answers and helping me alone the
way...


Cheers,
-Larry

 

[Larry Hale]

THANKS, AGAIN, for the reply!

Okay, now I -really- feel silly... :>

So, when I was going to try what you'd suggested, I noticed something
peculiar: I hadn't "changed" anything on my system, but now, when I
tried to "import magic" I got a missing DLL error. For a DIFFERENT
DLL! This got me to thinking...

Turns out, when I was having "success" earlier, I was -in- the C:
\Program Files\GnuWin32\bin directory (where file.exe and related .dll
files are).

So I added C:\Program Files\GnuWin32\bin to the (system/Windows)
PATH. Everything worked. I removed all the copious copies of the
DLL, the magic files, etc., littering my hard drive in every other
location. Still worked.

'Course, I don't honestly know why it didn't work with the files
copied to other locales contained in the sys/Py paths... perhaps it
was, as you mentioned, wanting the tree-structure to be the same? (I
haven't the heart to check this out fully right now... *maybe* at a
later time... )

Yeah, I know. But HONESTLY! I could have -sworn- I'd already tried
THAT... you know, all the "obvious"/"I've been programming/working in
computers (albeit 'Windows' until lately) for ~25 years; yeah, I'm a
'hacker'..." stuff. I -swear-!

[EGG ON FACE]

[SIGH] Well, I'd say "live and learn", but I'm still scratching my
head. But, 'nough of that. Thanks for your replies, Michael, and I'm
off to work _with_ the module now, then work-up my HowTo (and
experiment with making an "egg"?? )...

Caio!
-Larry

It's often the custom of programs ported from unix to windows to use the
dll location as a key to find the other files that are typically in
share, or etc. GTK, for example uses ../share and ../etc from the
location where the dlls are stored (if the dlls are in a folder called bin).

In this case I'd try making a folder in the Python25 folder called
"share" and put the contents of the gnuwin32 share/file stuff in there.
Should look something like this:

Or, judging by a string I found in the dll itself, you might have to put
the files here:

Although if such a path really is hardcoded into the dll, this is
certainly a bug, since it will fail on certain non-english windows systems.

Anyway, who knows. Give these options a try.

UPDATE: HOORAY!!! Well, actually, was about two steps forward, one
back. But I've gotten it all sorted out now!

I added C:\Program Files\GnuWin32\bin to the system/Windows PATH.
This takes care of the various required DLLs/libs.

The file source (previously linked from http://hupp.org/adam/hg/python-magic/)
has the man pages. I'd overlooked these as they were only modified as
necessary; the "BSD" part made me think they'd be the same as the same
online versions I'd already read (that were *nix specific).
Obviously, I was wrong. These had the default paths to the magic*
files listed; C:\Program Files\File\share\file\.

If one creates said directory structure and moves/copies/unpacks the
magic ("database") files THERE, then one may do:
Star Trek game file here to test with...
'MS-DOS executable, MZ for MS-DOS' # <-- This is what's
returned...

Alternately, if one wishes to leave/place the magic files elsewhere,
do like:
\share\\file\\magic' ) # <-- spec where/what the file is

NOTE: Even if the "magic_file" location *is* specified, "mime = True"
still works fine. (Haven't checked, but I presume the source simply
tacks ".mime" to the filename automagically.) Obviously/also, one
needn't specify the argument names if one uses proper argument order:
magic.Magic( mime, magic_file )


THANKS SO MUCH, Michael, for your answers and helping me alone the
way...


Cheers,
-Larry

 

[Larry Hale]

The file source (previously linked from http://hupp.org/adam/hg/python-magic/)
has the man pages...

Err... I meant "http://downloads.sourceforge.net/gnuwin32/file-4.21-
bin.zip?modtime=1180175868&big_mirror=1"!

 

[Larry Hale]

The file source (previously linked from http://hupp.org/adam/hg/python-magic/)
has the man pages...

Err... I meant "http://downloads.sourceforge.net/gnuwin32/file-4.21-
bin.zip?modtime=1180175868&big_mirror=1"!

 

[Michael Torrie]

In this case I'd try making a folder in the Python25 folder called
"share" and put the contents of the gnuwin32 share/file stuff in there.
Should look something like this:

c:/python25/share/file/magic.mime.mgc
c:/python25/share/file/magic
c:/python25/share/file/magic.mgc
c:/python25/share/file/magic.mime

I used linux's strace command to trace the file calls of file.exe via
wine (fun!) and verified that magic1.dll indeed will first in
c:/progra~1/File/share/file/magic and then in ../share/file

 

[Michael Torrie]

Alternately, if one wishes to leave/place the magic files elsewhere,
do like:

\share\\file\\magic' ) # <-- spec where/what the file is

NOTE: Even if the "magic_file" location *is* specified, "mime = True"
still works fine. (Haven't checked, but I presume the source simply
tacks ".mime" to the filename automagically.) Obviously/also, one
needn't specify the argument names if one uses proper argument order:
magic.Magic( mime, magic_file )


THANKS SO MUCH, Michael, for your answers and helping me alone the
way...

Glad it's working for you.