64-bit gotcha

[Roedy Green]

A Java Web Start program I wrote a long time ago suddenly stopped
working. It sets the System clock to an atomic time server.

I discovered the problem was it uses a 32--bit DLL via JNI and my
default Java is 64 bit. It cannot use 32-bit DLLs.

That surprises me. I would have thought MS would have had to come up
with a way of doing that since so many DLLs would not yet be available
in 32-bit. Maybe it can in general, but just not in Java.
--
Roedy Green Canadian Mind Products
http://mindprod.com
For me, the appeal of computer programming is that
even though I am quite a klutz,
I can still produce something, in a sense
perfect, because the computer gives me as many
chances as I please to get it right.

 

[Arne Vajhøj]

A Java Web Start program I wrote a long time ago suddenly stopped
working. It sets the System clock to an atomic time server.

I discovered the problem was it uses a 32--bit DLL via JNI and my
default Java is 64 bit. It cannot use 32-bit DLLs.

That surprises me. I would have thought MS would have had to come up
with a way of doing that since so many DLLs would not yet be available
in 32-bit. Maybe it can in general, but just not in Java.

32 bit or 64 bit is a per process thing.

You can't have part of a process using 32 bit addresses and
another part using 64 bit addresses.

The JVM and native DLL's must have same bitness.

It is not Java specific.

If you are using C# then you can specify AnyCPU (32 bit or 64 bit),
x86 (32 bit only) and x64 (64 bit only) to stamp programs with
requirements.

Arne

 

[Roedy Green]

It's a fair question to ask why Microsoft didn't do the same for 32-bit
DLLs executing in 64-bit processes.

I don't personally have the answer.

The reason might be the decline of assembler. Upgrading 16 bit DLLs
to 32 bit written in assembler would be a mighty undertaking.
Upgrading a 32-bit C program to 64-bit would be fairly
straightforward, Doing the same for Java is transparent. Further,
providing a bridge would just delay the introduction of 64 bit.

I am glad I was born early enough to have fun seeing what you could
pull off in assembler in 1K, pushing the limits of what was possible
with a given CPU. I am even old enough to have used two generations
of programming before that, plug cords and machine hex.

I don't think I would enjoy programming today without some
understanding of what goes on at the chip level and inside the OS.
I would hope every CS student would write at least one device driver
to get a feel for what an OS does for a living.
--
Roedy Green Canadian Mind Products
http://mindprod.com
For me, the appeal of computer programming is that
even though I am quite a klutz,
I can still produce something, in a sense
perfect, because the computer gives me as many
chances as I please to get it right.

 

[Martin Gregorie]

I don't think I would enjoy programming today without some understanding
of what goes on at the chip level and inside the OS. I would hope every
CS student would write at least one device driver to get a feel for what
an OS does for a living.

This is somewhat OTT, but....

I related thought has occurred here in the UK to some people with the
money to do something about it. They realized that many of the current
bunch of IT people in this country got their start as teenagers by
hacking about with the Sinclair Spectrum and the BBC Model B - both
fairly inexpensive machines and both designed so they could be
experimented with - the Beeb even had a patch panel area. They have come
up with a current equivalent, the RaspberryPi, which uses an ARM chip and
runs Linux, so an ARM port of the JVM should be right at home. The
machine is a small, simple board with 128 or 256 MB RAM, an SD card to
boot from and USB, HDMI, audio and Ethernet interfaces plus a patch panel
area and should sell for around $US 32. String an ethernet cable to
anything that can run telnet or ssh (Kermit should be good too), plug in
the power, and you're in business. Its due out early next year and, as
its being done by a non-profit, prices should fall as volumes ramp up.

See http://www.raspberrypi.org/ for more detail if you're interested.

Disclaimer: I have no connection whatever with RaspberryPi, but am
planning to get one to hack about with.

 

[Jeff Higgins]

This is somewhat OTT, but....

I related thought has occurred here in the UK to some people with the
money to do something about it. They realized that many of the current
bunch of IT people in this country got their start as teenagers by
hacking about with the Sinclair Spectrum and the BBC Model B - both
fairly inexpensive machines and both designed so they could be
experimented with - the Beeb even had a patch panel area. They have come
up with a current equivalent, the RaspberryPi, which uses an ARM chip and
runs Linux, so an ARM port of the JVM should be right at home.
The
machine is a small, simple board with 128 or 256 MB RAM, an SD card to
boot from and USB, HDMI, audio and Ethernet interfaces plus a patch panel
area and should sell for around $US 32.

Geesh. Wish I had held off on the BeagleBoard. May explain why we now
have the BeagleBone!
<http://beagleboard.org/>

String an ethernet cable to

 

[Arved Sandstrom]

This is somewhat OTT, but....

I related thought has occurred here in the UK to some people with the
money to do something about it. They realized that many of the current
bunch of IT people in this country got their start as teenagers by
hacking about with the Sinclair Spectrum and the BBC Model B - both
fairly inexpensive machines and both designed so they could be
experimented with - the Beeb even had a patch panel area. They have come
up with a current equivalent, the RaspberryPi, which uses an ARM chip and
runs Linux, so an ARM port of the JVM should be right at home. The
machine is a small, simple board with 128 or 256 MB RAM, an SD card to
boot from and USB, HDMI, audio and Ethernet interfaces plus a patch panel
area and should sell for around $US 32. String an ethernet cable to
anything that can run telnet or ssh (Kermit should be good too), plug in
the power, and you're in business. Its due out early next year and, as
its being done by a non-profit, prices should fall as volumes ramp up.

See http://www.raspberrypi.org/ for more detail if you're interested.

Disclaimer: I have no connection whatever with RaspberryPi, but am
planning to get one to hack about with.

Sounds interesting, I'll keep an eye out for availability. I'm in that
mindset - I've been hacking for a while now with a MSP430
microcontroller development kit from TI
(http://www.ti.com/tool/msp-exp430g2). A lot of fun.

AHS

 

[Travers Naran]

But as with Java, you cannot call native DLLs using the wrong bitness of
a .NET program. A .NET program running as a 32-bit process can call only
32-bit DLLs, and a .NET program running as a 64-bit process can call
only 64-bit DLLs. Other than the way that the choice is made for what
kind of bitness the program is running in, it's pretty much the same
situation.

That also applies to ADO.Net drivers, as we discovered at work.

"I included the Oracle .net driver in the installation package, I don't
know what happened!"

"Did you remember Oracle's .Net driver will attempt to open DLLs already
installed in the system over the ones in the program's directory?"

"..."

"You're app is compiled & packaged as 32-bit but running on a laptop
with 64-bit Windows 7 which already has Oracle drivers installed."

".... Ohhhhh!"

So don't forget to ask what OTHER DLL's your DLL is opening.

 

[Joshua Cranmer]

True, Windows does not support that. But it's technically feasible, and
in fact 32-bit Windows supported 32-bit processes calling 16-bit DLLs
(e.g. http://support.microsoft.com/kb/155763).

It's a fair question to ask why Microsoft didn't do the same for 32-bit
DLLs executing in 64-bit processes.

I believe the short answer is that x86-64 is surprisingly different from
x86-32; note that, given some experience with so-called "16-bit" x86
code, it was much closer to the same ISA.

My very limited experience with some Win16 code indicates that the
primary difference is that 16-bit code relies on segmentation a lot more
(CS/DS/ES/SS concerns), but it's still in the same processor mode: if
not directly the same, the code would still act identically in the two
modes. By contrast, x86-64 moves 32-bit and 64-bit code into different
processor modes, and the same byte sequence would do different things.
For example, e9 is the opcode for a JMP <rel16> in 32-bit code but a JMP
<rel32> in 64-bit code--it can't act as both at the same time.

It still wouldn't be impossible to make 32-from-64 to work, but it would
certainly require IPC between a 64-bit real process and a 32-bit shadow
process; given undecidability of pointer aliasing (plus loads of pointer
punning), it would probably require nontrivial tailoring the 32-bit
process to get this to work--which raises the question of why not
translate it to 64-bit code in the first place.

Now, it is possible to engineer a 64-bit codebase to spin off the 32-bit
process itself and manage all of this itself.

 

[Jeff Higgins]

On 12/04/2011 07:06 AM, Martin Gregorie wrote:

Geesh. Wish I had held off on the BeagleBoard. May explain why we now
have the BeagleBone!
<http://beagleboard.org/>

Neato:
"Sun SPOT Hardware Developers Kits are available for purchase in the
Oracle Store!"

 

[Martin Gregorie]

Neato:
"Sun SPOT Hardware Developers Kits are available for purchase in the
Oracle Store!"
<http://sunspotworld.com/>

Looks interesting, though maybe not interesting enough at $400 per kit
for anything I'd do with it, even allowing for it including three
SunSpots. Pity there's not a version with touch sensitive screen: I could
have a use for that.

 

[Jeff Higgins]

Looks interesting, though maybe not interesting enough at $400 per kit
for anything I'd do with it, even allowing for it including three
SunSpots. Pity there's not a version with touch sensitive screen: I could
have a use for that.

Yep. I just installed the SDK, and will play with the emulator for a
while for free.

 

[Arne Vajhøj]

True, Windows does not support that. But it's technically feasible, and
in fact 32-bit Windows supported 32-bit processes calling 16-bit DLLs
(e.g. http://support.microsoft.com/kb/155763).

There are a few things worth noting:
1) this so called flat thunking only works on Win 95/98/ME
2) It does require glue code on both 16 and 32 bit side
(non transparent)

It's a fair question to ask why Microsoft didn't do the same for 32-bit
DLLs executing in 64-bit processes.

Well there is no 64 bit 95/98/ME.

And the NT family has never supported flat thunking.

So it simply would not work.

The NT family has so called generic thunking, but that
was 16 bit app -> 32 bit DLL and using a special API. So
again non transparent.

But saying "you can't do it" doesn't really seem to answer the question.
Roedy already knows he can't do it, at least in Java, and implicit in
his post is the question as to why Windows does not support calling
32-bit DLLs in 64-bit processes.

Because there is an address problem.

There are no transparent call across different bitness
on Windows and I don't know any other OS either that
can do that.

But as with Java, you cannot call native DLLs using the wrong bitness of
a .NET program. A .NET program running as a 32-bit process can call only
32-bit DLLs, and a .NET program running as a 64-bit process can call
only 64-bit DLLs. Other than the way that the choice is made for what
kind of bitness the program is running in, it's pretty much the same
situation.

You should get an error at startup instead of later when
the DLL is needed.

Arne

 

[Arne Vajhøj]

The reason might be the decline of assembler. Upgrading 16 bit DLLs
to 32 bit written in assembler would be a mighty undertaking.

True.

But then it happens that people only have binaries.

Arne

 

[Arne Vajhøj]

I believe the short answer is that x86-64 is surprisingly different from
x86-32; note that, given some experience with so-called "16-bit" x86
code, it was much closer to the same ISA.

My very limited experience with some Win16 code indicates that the
primary difference is that 16-bit code relies on segmentation a lot more
(CS/DS/ES/SS concerns), but it's still in the same processor mode: if
not directly the same, the code would still act identically in the two
modes.

Besides segmented vs flat address space there is also:
* real mode vs protected mode
* size of registers

It still wouldn't be impossible to make 32-from-64 to work, but it would
certainly require IPC between a 64-bit real process and a 32-bit shadow
process; given undecidability of pointer aliasing (plus loads of pointer
punning), it would probably require nontrivial tailoring the 32-bit
process to get this to work--which raises the question of why not
translate it to 64-bit code in the first place.

They could have provide an API for WOW64 similar to the
API for the old WOW.

But since it is non transparent, then the benefits
would be very limited.

Arne