Examples of Python driven Microsoft UI Automation wanted


T

TheSeeker

Hi,

I am embarking on teaching myself Microsoft UI Automation using Python
as the scripting language.

I have asked some questions in the IronPython users group, but have
yet to get a response, so I thought I would broaden the audience by
asking here.

Specifically, I have a WinForms application I will be wanting to
automate. Does anyone have some Python examples of driving Microsoft
UI Automation they could share with me to get me started? The
structure of the UI automation classes etc. seem quite convoluted, and
I am having difficulty getting my brain wrapped around it.

Alternatives to Microsoft's UI Automation are welcome too, but I have
tried using winguiauto and watsup (along with AutoIt), and there seems
to be severe limitations when using these tools with WinForm
applications.

Thanks in advance,
Duane
 
Ad

Advertisements

T

Tim Harig

Specifically, I have a WinForms application I will be wanting to
automate. Does anyone have some Python examples of driving Microsoft
UI Automation they could share with me to get me started? The
structure of the UI automation classes etc. seem quite convoluted, and
I am having difficulty getting my brain wrapped around it.

If you find a way to work through the UI using the keyboard (tabs, etc);
then, you can send it keyboard commands using WScript.WshShell.SendKeys():

http://msdn.microsoft.com/en-us/library/8c6yea83(VS.85).aspx
 
T

Tim Harig

Unfortunately, I need to be able to find out the contents of a few
text-boxes as well,
so SendKeys isn't all I need.

Then you will need some way to import your application as an object so that
you can gain access to the methods and properties directly. Using
IronPython you should be able to access anyting in the GAC. If you
register your UI componets then you should be able to access them. If not,
or using cpython, you might be able to do something like:

http://msdn.microsoft.com/en-us/library/fh1h056h(VS.71).aspx

to get a mashalled COM object.
 
D

DuaneKaufman

Then you will need some way to import your application as an object so that
you can gain access to the methods and properties directly.  Using
IronPython you should be able to access anyting in the GAC.  If you
register your UI componets then you should be able to access them.  If not,
or using cpython, you might be able to do something like:

http://msdn.microsoft.com/en-us/library/fh1h056h(VS.71).aspx

to get a mashalled COM object.

Thanks again Tim.

I believe I probably mis-communicated my requirements (or at least,
was not explicit enough)

The application I wish to interact with is not my own, but an ERP
system GUI front-end.

With MS utilities like UISpy and the like, I can 'see' the controls in
the application, but I
do not seem to be able to manipulate them programatically, and I
believe it us simply due
to my not understanding of the UI Automation API.

Hence, my request for example code to boot-strap myself.

Thanks,
Duane
 
P

Paul McGuire

The application I wish to interact with is not my own, but an ERP
system GUI front-end.

I have used pywinauto to drive a Flash game running inside of an
Internet Explorer browser - that's pretty GUI!

-- Paul
 
Ad

Advertisements

D

DuaneKaufman

I have used pywinauto to drive a Flash game running inside of an
Internet Explorer browser - that's pretty GUI!

-- Paul

Hi,

Could you share some code examples?

Thanks,
Duane (duanek (at) chorus (dot) net)
 
P

Paul McGuire

Hi,

Could you share some code examples?

Thanks,
Duane (duanek (at) chorus (dot) net)

I just went on a brief fishing expedition through two disk backups,
and no luck. I guess it's been a while since I worked on this.

The work I did was entirely graphical, which is to say, my script
interacted with the Flash program by using PIL to take image snapshots
of the window, and then sifting through the bitmap looking for the
status of a "time remaining" thermometer-style gauge in the game.
Then the script could click on X-Y coordinates within the window,
which would get picked up by the Flash game, and the script would
monitor the "time remaining" gauge some more, and so on.

I'm not sure how well pywinauto would work in allowing you to access
controls such as textboxes within a form. I remember that I had to
access the IE window using a caption name, and then found the embedded
Flash program as an embedded control of some sort, again, I probably
needed to indicate that it was some sort of "Adobe.FlashWidget1"
object or something.

I may have another, older disk backup at home, I can look for it later
this evening.

-- Paul
 
M

Michael Torrie

DuaneKaufman said:
With MS utilities like UISpy and the like, I can 'see' the controls in
the application, but I
do not seem to be able to manipulate them programatically, and I
believe it us simply due
to my not understanding of the UI Automation API.

You're probably better off using a free app like AutoIt3 rather than
roll your own. AutoIt's scripting language isn't as nice as Python, but
it does the job and it turing-complete.

http://www.autoitscript.com/autoit3/
 
D

DuaneKaufman

http://www.autoitscript.com/forum/index.php?showtopic=96752-- looks
like recent betas have much better support for WinForms, at least from
the change list.

Hi,

Thanks for the information. Boy, you are away from a utility for a
couple of
weeks, and it starts growing things one needs!

Unfortunately, while it looks like AutoIt is gaining the ability to
'see' WinForms controls,
this beta doesn't seem to be able to retrieve the text say, in a
TextBox. Darn.

Thanks,
Duane
 
Ad

Advertisements

Ad

Advertisements

M

Mark

Hi,

I am the author of pywinauto (http://pywinauto.openqa.org/).

It doesn't have specific functionality built in for winforms - but
many winforms controls are built on top of win32 functionality and
pywinauto can work with them for that.


Wow - cool :)


You might have a look at http://pywinauto.pbworks.com/ (the site is no
longer used - but there is a cool movie giving an example of using
pywinauto at the interactive prompt)

Other examples (Notepad, mspaint) are included with the install.

PyPi - http://pypi.python.org/pypi/pywinauto/0.3.8


Good luck! :)
Mark
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top