[ANN] au3 0.1.1 released

[Marvin Gülker]

au3 0.1.1 has been released. au3 is a library that allows you to
automate and simulate (fake) user input to a Windows system by using the
DLL interface of AutoIt behind the scenes. Abilities include:
* Cursor movement
* Key pressing
* Window interaction
* Clipboard access
* ...

Example:
------------------------------------
require "au3"
#Simulate the three letters A, B and C
AutoItX3.send_keys("ABC")
#Simulate A and B, then [ESC] and then C.
AutoItX3.send_keys("AB{ESC}C")
#Move the mouse cursor to position (100|100)
AutoItX3.move_mouse(100, 100)
#Get access to a window named "au3"
win = AutoItX3::XWindow.new("au3")
#Move it to (50|75)
win.move(50, 75)
#Close it
win.close
------------------------------------

As the name implies, it's mainly a wrapper for AutoItX, so you'll need
to have the AutoItX3.dll file somewhere in your path. See au3's README
for more details.

RubyForge project: http://rubyforge.org/projects/auto/
Gemcutter page: http://www.gemcutter.org/gems/au3
GitHub wiki: http://wiki.github.com/Quintus/Automations
GitHub issue tracker: http://github.com/Quintus/Automations/issues

Marvin

 

[Marvin Gülker]

win = AutoItX3::XWindow.new("au3")

Ouch. That should have read

win = AutoItX3::Window.new("au3")

Marvin

 

[Roger Pack]

au3 0.1.1 has been released. au3 is a library that allows you to
automate and simulate (fake) user input to a Windows system by using the
DLL interface of AutoIt behind the scenes. Abilities include:
* Cursor movement
* Key pressing
* Window interaction
* Clipboard access
* ...

Oh wow. This feels one step closer to an AutoHotKey written in Ruby...

-r

 

[Marvin Gülker]

Oh wow. This feels one step closer to an AutoHotKey written in Ruby...

-r

Thank you.

Indeed, I've struggled a lot with the Windows API to get rid of the
dependency of AutoItX, but I've given up on it, because I wasn't able to
master the #pack and #unpack methods. Many functions of the WinAPI want
unions of structs of structs of ... of... and till today I don't know
how to pack a struct into a struct. Easy procedures like the
GetCursorPos() function just take a POINT struct that I have to pack
before and unpack after the call again - but when it comes to keyboard
simulation, I have to provide a whole bunch of structs in structs,
because I can't take advantage of the struct's default values in Ruby
(I'm using the win32-api gem for such tasks). If I provide a packed
struct that misses some parameters, hoping the missing ones will be
filled by the default ones, I only get error responses. Also, my C
knowledge is not the best. Maybe you've noticed au3's first release,
which has been a C extension - but since I never programmed in C before
the style has been horrible. ;-)

So, if you could help me out with #pack and #unpack, especially with
structs in structs (I would look for an example from the WinAPI, if you
want) I would start working on that again.

Marvin

 

[Edward Middleton]

So, if you could help me out with #pack and #unpack, especially with
structs in structs (I would look for an example from the WinAPI, if you
want) I would start working on that again.

I haven't used Win32API, I just connected directly from dl. Examples of
the functions/structs you were having problems with would help.

Edward

 

[Marvin Gülker]

I haven't used Win32API, I just connected directly from dl. Examples of
the functions/structs you were having problems with would help.

Edward

OK - SendInput() is the main function to simulate keyboard and mouse
input: http://msdn.microsoft.com/en-us/library/ms646310(VS.85).aspx
(from there you can also get to the descriptions of INPUT and
KEYBDINPUT)
It takes the size of a C array containing INPUT structs, a C array
containing INPUT structs and the size of an INPUT structure.
The INPUT struct itself wants a description wheather it is a keyboard or
a mouse struct, and depending on that it chooses the MOUSEINPUT oder
KEYBDINPUT struct from its second, an union, parameter.
I tried with keyboard simulation first, so I have to build up the
KEYBDINPUT struct: That takes a virtual key code describing the key to
simulate - unless you want to simulate unicode characters -, than an
unicode character as a WORD if you want to simulate a unicode char. Next
parameter is the input simulation type, e.g. unicode, followed by a time
stamp and a pointer to extra message info.

That isn't as complicated as it sounds, but there are some problems I
cannot get around. First, how to obtain the size of the INPUT structure?
There's no sizeof keyword in Ruby... Second, how to transform a unicode
char (Windows usually uses UTF-16LE I've found out when working on au3)
into a WORD? As far as I know, this is an integer value, but a unicode
char like ä consists of more than one byte? Third, as said, how to pack
a struct into a struct, in this example: How to pack the KEYBDINPUT into
the INPUT struct? And last but not least, how to build up a C array in
Ruby?

That said, my quite incomplete code looks like this:
------------------------------------
SendInput = Win32::API.new("SendInput", 'IPI', 'I', "user32")
INPUT_KEYBOARD = 1 #We're using keyboard simulation
KEYEVENTF_UNICODE = 0x0004 #We want to simulate unicode chars

#INPUT struct: LP
#KEYBDINPUT struct: IILLP

keybdinput = [0, "a".encode("UTF16-LE").pack('I'), KEYEVENTF_UNICODE, 0,
nil].pack('IILLP')
input = [INPUT_KEYBOARD, keybdinput].pack('LP')
#Yes, I know that the following line produces more than one error.
SendInput.call(1, [input], ??) #How to obtain the size of an INPUT?

 

[Marvin Gülker]

the MOUSEINPUT oder KEYBDINPUT struct from its second

Oh no - you see, I'm a german speaker, so replace this "oder" with
"or"... :!

Marvin

 

[Edward Middleton]

... That isn't as complicated as it sounds, but there are some problems I
cannot get around. First, how to obtain the size of the INPUT structure?
There's no sizeof keyword in Ruby...

pack converts and ruby array to a ruby string which in ruby 1.8 is a
sequence of bytes. The size of the string will give you the size in
bytes of the struct.

Second, how to transform a unicode char (Windows usually uses UTF-16LE I've found out when working on au3) into a WORD? As far as I know, this is an integer value, but a unicode char like ä consists of more than one byte?

You use MultiByteToWideChar[1] and WideCharToMultiByte[2], the msdn
unicode[3] page has more details on this. I think I mostly input
everything in UTF-8 then converted to UTF-16LE because there were issues
with BOM using UTF-16LE

Third, as said, how to pack a struct into a struct, in this example: How to pack the KEYBDINPUT into the INPUT struct? And last but not least, how to build up a C array in Ruby?

The easiest way is to flatten struct into one. i.e.

INPUT {

WORD type

WORD wVk
WORD wScan
DWORD dwFlags
DWORD Time
ULONG_PTR

}

I don't have a windows machine to test this on but I believe this is
mostly correct.

Edward

1. http://msdn.microsoft.com/en-us/library/dd319072(VS.85).aspx
2. http://msdn.microsoft.com/en-us/library/dd374130(VS.85).aspx
3. http://msdn.microsoft.com/en-us/library/dd374081(VS.85).aspx

 

[Marvin Gülker]

Thank you, Edward, that helped me out a lot.
My code now looks like this (to simplify this for the moment, I sticked
to the non-unicode variant):
---------------------------------
#Encoding: UTF-8
require "win32/api"
GetLastError = Win32::API.new("GetLastError", '', 'I', "kernel32")
SendInput = Win32::API.new("SendInput", 'IPI', 'I', "user32")
#Needed for the KEYBDINPUT struct's last parameter
GetMessageExtraInfo = Win32::API.new("GetMessageExtraInfo", '', 'P',
"user32")
INPUT_KEYBOARD = 1
KEY_A = 0x41
#KEYEVENTF_UNICODE = 0x0004

input = [
INPUT_KEYBOARD,
KEY_A,
0,
0,
0,
GetMessageExtraInfo.call
].pack('issiiP')

#While the program pauses, I click into an editor window
#to see wheather I get the 'a' or not.
sleep 3

SendInput.call(1, input, input.size)
p GetLastError.call
---------------------------------
I'm using Ruby 1.9.1-p243 at the moment (will update to -p376 soon), but
I've checked that Array#pack returns a BINARY-encoded string, so there
shouldn't occur a problem. Nevertheless, I get error code 87, what means
"One of the parameters was invalid.". I suppose it's the C array I don't
provide, even if it contains only one element. Do you know how to build
up one?

Marvin

 

[Maxim Ap]

Hi Edward,

I have similar problem with the WinAPI call.
I am trying to get tree control item rectangle. C code for this looks
like

#define TVM_GETITEMRECT (TV_FIRST + 4)
#define TreeView_GetItemRect(hwnd, hitem, prc, code) \
(*(HTREEITEM *)prc = (hitem), (BOOL)SNDMSG((hwnd), TVM_GETITEMRECT,
(WPARAM)(code), (LPARAM)(RECT *)(prc)))

It is not a problem to call SendMessage(SNDMSG), but the problem is how
to transfer parameters(hitem) and get response in this case.

It would be great if you can help with this.

Thanks,
Maxim

 

[Edward Middleton]

input = [
INPUT_KEYBOARD,
KEY_A,
0,
0,
0,
GetMessageExtraInfo.call
].pack('issiiP')
...
Nevertheless, I get error code 87, what means
"One of the parameters was invalid.". I suppose it's the C array I don't
provide, even if it contains only one element.

Yes thats the problem. I was forgetting it was an array. That means
that the struct can't be flattened, or more specifically the parent
struct contains a pointer to the child struct[1] rather the then the
struct itself. I am not really sure how to get a pointer to the data of
a string in ruby. I think there was a DL method to do this though.

Edward

 

[Marvin Gülker]

"One of the parameters was invalid.". I suppose it's the C array I don't
provide, even if it contains only one element.

Yes thats the problem. I was forgetting it was an array. That means
that the struct can't be flattened, or more specifically the parent
struct contains a pointer to the child struct[1] rather the then the
struct itself. I am not really sure how to get a pointer to the data of
a string in ruby. I think there was a DL method to do this though.

Edward

Why can't it be flattened? The SendInput() function expects an array of
INPUT structs, the INPUT struct itself doesn't expect an array of
KEYBDINPUT structs. Maybe I should write a small C extension that just
puts my Ruby string into a one-element C array. To provide binaries for
such a small extension should be possible. On the other hand, I'm not
sure how MinGW will cope with the types used by the Windows API; as far
as I know, MinGW is using an old C runtime and many things have changed
in the WinAPI since then.

I am not really sure how to get a pointer to the data of
a string in ruby.

The documentation for Array#pack says this:

P | Pointer to a structure (fixed-length string)

I've used that already for packing the result of the
GetMessageExtraInfo() call (which returns a LPARAM), but I'm not sure
about how it works.

Marvin

 

[Phillip Gawlowski]

On the other hand, I'm not
sure how MinGW will cope with the types used by the Windows API; as far
as I know, MinGW is using an old C runtime and many things have changed
in the WinAPI since then.

If you find a Windows API call, or for that matter, an MS DOS 1.0 API
call that doesn't work as expected, MS treats that as a bug. The API
won't change. Once it's documented somewhere, it continues to take the
arguments it did the moment it was introduced. Implementation can
change, of course.

MS takes that so far, that they introduced compatibility shims to keep
SimCity 2000 running on newer OSes.

There'll have been changes in the driver model for Windows introduced
with Vista (or rather: Windows 6.0), but since I doubt you want to
access the WDDM, I doubt this is a major issue.

That said: I have copy of Win 7 and Visual Studio 2008 Pro available, so
I can offer (limited) assistance: I can test builds for you against the
current Platform SDK (which works outside of VS, but needs Windows), as
long as you send me a Visual Studio solution.

 

[Roger Pack]

So, if you could help me out with #pack and #unpack, especially with

structs in structs (I would look for an example from the WinAPI, if you
want) I would start working on that again.

Interesting.
You may want to take a look at ffi.

http://wiki.github.com/ffi/ffi

I was able to use it to pack some structs into windows calls and it
worked (plus the community is friendly to help with problems, plus it's
cross VM compatible)

Cheers!

-r

 

[Edward Middleton]

Why can't it be flattened? The SendInput() function expects an array of
INPUT structs, the INPUT struct itself doesn't expect an array of
KEYBDINPUT structs.

Sorry, yes you are right, I should have checked the msdn. What does
GetMessageExtraInfo(VOID) return?

Edward

 

[Edward Middleton]

Interesting.
You may want to take a look at ffi.

http://wiki.github.com/ffi/ffi

I was able to use it to pack some structs into windows calls and it
worked (plus the community is friendly to help with problems, plus it's
cross VM compatible)

Yes ffi is pretty good, it is a bit slow but this isn't going to be a
problem with this sort of application.

Edward

 

[Marvin Gülker]

Sorry, yes you are right, I should have checked the msdn. What does
GetMessageExtraInfo(VOID) return?

Edward

That function returns a LPARAM, which is, according to the explanations
on the FFI wiki page ( http://wiki.github.com/ffi/ffi/types ), a
pointer. MSDN docs:
http://msdn.microsoft.com/en-us/library/ms644937(VS.85).aspx

@roger: Looks good, I will try working with that.

@philip: Thank you, but I'd like to continue using MinGW + MSYS for
building C extensions (when I really do this, what is quite rarely the
case). I'm not good at C, so FFI sounds really nice to me.

Marvin

 

[Phillip Gawlowski]

@Philip:

It's "Phillip".

Thank you, but I'd like to continue using MinGW + MSYS for
building C extensions (when I really do this, what is quite rarely the
case).

Since that is what the RubyInstaller uses to build Ruby, this is also
the preferred way.

 

[Heesob Park]

Hi,

Thank you, Edward, that helped me out a lot.
My code now looks like this (to simplify this for the moment, I sticked
to the non-unicode variant):
---------------------------------
#Encoding: UTF-8
require "win32/api"
GetLastError =3D Win32::API.new("GetLastError", '', 'I', "kernel32")
SendInput =3D Win32::API.new("SendInput", 'IPI', 'I', "user32")
#Needed for the KEYBDINPUT struct's last parameter
GetMessageExtraInfo =3D Win32::API.new("GetMessageExtraInfo", '', 'P',
"user32")
INPUT_KEYBOARD =3D 1
KEY_A =3D 0x41
#KEYEVENTF_UNICODE =3D 0x0004

input =3D [
=C2=A0INPUT_KEYBOARD,
=C2=A0KEY_A,
=C2=A00,
=C2=A00,
=C2=A00,
=C2=A0GetMessageExtraInfo.call
].pack('issiiP')

#While the program pauses, I click into an editor window
#to see wheather I get the 'a' or not.
sleep 3

SendInput.call(1, input, input.size)
p GetLastError.call
---------------------------------
I'm using Ruby 1.9.1-p243 at the moment (will update to -p376 soon), but
I've checked that Array#pack returns a BINARY-encoded string, so there
shouldn't occur a problem. Nevertheless, I get error code 87, what means
"One of the parameters was invalid.". I suppose it's the C array I don't
provide, even if it contains only one element. Do you know how to build
up one?

For Future Reference, here is a working code.

--------------------------------------------------------------------------
#Encoding: UTF-8
require "win32/api"
GetLastError =3D Win32::API.new("GetLastError", '', 'I', "kernel32")
SendInput =3D Win32::API.new("SendInput", 'IPI', 'I', "user32")
#Needed for the KEYBDINPUT struct's last parameter
GetMessageExtraInfo =3D Win32::API.new("GetMessageExtraInfo", '', 'P', "use=
r32")
INPUT_KEYBOARD =3D 1
KEY_A =3D 0x0061 # unicode for 'a' cf. 0x00E4 for '=C3=A4'
KEYEVENTF_UNICODE =3D 0x0004
KEYEVENTF_KEYUP =3D 0x0002

#While the program pauses, I click into an editor window
#to see wheather I get the 'a' or not.
sleep 3

#key down
input =3D [
INPUT_KEYBOARD,
0,
KEY_A,
KEYEVENTF_UNICODE,
0,
nil
].pack('issiiP') + 0.chr * 8 # 8bytes dummy to make 28bytes
SendInput.call(1, input, input.size)

# key up
input =3D [
INPUT_KEYBOARD,
0,
KEY_A,
KEYEVENTF_UNICODE|KEYEVENTF_KEYUP,
0,
nil
].pack('issiiP') + 0.chr * 8
SendInput.call(1, input, input.size)

 

[Marvin Gülker]

Regards,

Park Heesob

Works great! But I have a question about the 0.chr line: Why do you need
to append 8 extra bytes to achieve a size of 28 bytes? I guess, it has
something to do with C's array treatment?

It's "Phillip".

Sorry.

Marvin