C++ produced ASM code

[Joris]

Hi

I'm not sure if this is the correct newsgroup for this question
as it involves both ASM and c++, but here goes.

I have written a function in C++ that has to replace one of
a process in memory (to change functionality).
The original function's address is retrieved from 1 offset.
When the application starts I write the address of my new
function to that offset. So far so good. I found out the original
function has 1 argument (4-byte) and returns a 4-byte value
(1 or 0, BOOL). This fails. I took a look at the code c++
had generated, in a debugger. I saw c++ ends the new
function with "retn". When I changed this to "retn 4"
the process ran without any problems.

Why does "retn 4" work and "retn" not? Why does
Visual C++ compile "retn" and not "retn 4"?

Thanks
Joris

 

[Alf P. Steinbach]

* Joris:

I'm not sure if this is the correct newsgroup for this question
as it involves both ASM and c++, but here goes.

I have written a function in C++ that has to replace one of
a process in memory (to change functionality).
The original function's address is retrieved from 1 offset.
When the application starts I write the address of my new
function to that offset. So far so good. I found out the original
function has 1 argument (4-byte) and returns a 4-byte value
(1 or 0, BOOL). This fails. I took a look at the code c++
had generated, in a debugger. I saw c++ ends the new
function with "retn". When I changed this to "retn 4"
the process ran without any problems.

The problem is C-style machine code calling convention versus Pascal-like
style machine code calling convention.

Standard C++ does not help out with machine code calling conventions.

The closest you get in standard C++ is 'extern "C"' versus 'extern "C++"',
but for your problem, consult the compiler's documentation for language
extensions (OT: for MSVC the language extension is 'declspec').

Why does "retn 4" work and "retn" not?

See above.

Why does Visual C++ compile "retn" and not "retn 4"?

See above.

 

[Victor Bazarov]

I'm not sure if this is the correct newsgroup for this question
as it involves both ASM and c++, but here goes.

It's not. Your question is compiler-specific. It should be asked
in the newsgroup for that compiler. 'microsoft.public.vc.language'
is probably the one you need.

I have written a function in C++ that has to replace one of
a process in memory (to change functionality).
The original function's address is retrieved from 1 offset.
When the application starts I write the address of my new
function to that offset. So far so good. I found out the original
function has 1 argument (4-byte) and returns a 4-byte value
(1 or 0, BOOL). This fails. I took a look at the code c++
had generated, in a debugger. I saw c++ ends the new
function with "retn". When I changed this to "retn 4"
the process ran without any problems.

Why does "retn 4" work and "retn" not? Why does
Visual C++ compile "retn" and not "retn 4"?

<offtopic>
You probably didn't declare your function appropriately. There exist
several modifiers in Visual C++ (they call them "calling conventions")
that can change how the function is called, how the arguments are passed,
and how the function returns. See 'CALLBACK', 'WINAPI', '__stdcall',
etc.
</offtopic>

V

 

[Joris]

Thank you.


"Alf P. Steinbach" <[email protected]> schreef in bericht
* Joris:

I'm not sure if this is the correct newsgroup for this question
as it involves both ASM and c++, but here goes.

I have written a function in C++ that has to replace one of
a process in memory (to change functionality).
The original function's address is retrieved from 1 offset.
When the application starts I write the address of my new
function to that offset. So far so good. I found out the original
function has 1 argument (4-byte) and returns a 4-byte value
(1 or 0, BOOL). This fails. I took a look at the code c++
had generated, in a debugger. I saw c++ ends the new
function with "retn". When I changed this to "retn 4"
the process ran without any problems.

The problem is C-style machine code calling convention versus Pascal-like
style machine code calling convention.

Standard C++ does not help out with machine code calling conventions.

The closest you get in standard C++ is 'extern "C"' versus 'extern "C++"',
but for your problem, consult the compiler's documentation for language
extensions (OT: for MSVC the language extension is 'declspec').

Why does "retn 4" work and "retn" not?

See above.

Why does Visual C++ compile "retn" and not "retn 4"?

See above.