Examples of Python driven Microsoft UI Automation wanted

[TheSeeker]

Hi,

I am embarking on teaching myself Microsoft UI Automation using Python
as the scripting language.

I have asked some questions in the IronPython users group, but have
yet to get a response, so I thought I would broaden the audience by
asking here.

Specifically, I have a WinForms application I will be wanting to
automate. Does anyone have some Python examples of driving Microsoft
UI Automation they could share with me to get me started? The
structure of the UI automation classes etc. seem quite convoluted, and
I am having difficulty getting my brain wrapped around it.

Alternatives to Microsoft's UI Automation are welcome too, but I have
tried using winguiauto and watsup (along with AutoIt), and there seems
to be severe limitations when using these tools with WinForm
applications.

Thanks in advance,
Duane

 

[Tim Harig]

Specifically, I have a WinForms application I will be wanting to
automate. Does anyone have some Python examples of driving Microsoft
UI Automation they could share with me to get me started? The
structure of the UI automation classes etc. seem quite convoluted, and
I am having difficulty getting my brain wrapped around it.

If you find a way to work through the UI using the keyboard (tabs, etc);
then, you can send it keyboard commands using WScript.WshShell.SendKeys():

http://msdn.microsoft.com/en-us/library/8c6yea83(VS.85).aspx

 

[DuaneKaufman]

If you find a way to work through the UI using the keyboard (tabs, etc);
then, you can send it keyboard commands using WScript.WshShell.SendKeys():

http://msdn.microsoft.com/en-us/library/8c6yea83(VS.85).aspx

Thanks for the link.

Unfortunately, I need to be able to find out the contents of a few
text-boxes as well,
so SendKeys isn't all I need.

Thanks again,
Duane

 

[Tim Harig]

Unfortunately, I need to be able to find out the contents of a few
text-boxes as well,
so SendKeys isn't all I need.

Then you will need some way to import your application as an object so that
you can gain access to the methods and properties directly. Using
IronPython you should be able to access anyting in the GAC. If you
register your UI componets then you should be able to access them. If not,
or using cpython, you might be able to do something like:

http://msdn.microsoft.com/en-us/library/fh1h056h(VS.71).aspx

to get a mashalled COM object.

 

[DuaneKaufman]

Then you will need some way to import your application as an object so that
you can gain access to the methods and properties directly.  Using
IronPython you should be able to access anyting in the GAC.  If you
register your UI componets then you should be able to access them.  If not,
or using cpython, you might be able to do something like:

http://msdn.microsoft.com/en-us/library/fh1h056h(VS.71).aspx

to get a mashalled COM object.

Thanks again Tim.

I believe I probably mis-communicated my requirements (or at least,
was not explicit enough)

The application I wish to interact with is not my own, but an ERP
system GUI front-end.

With MS utilities like UISpy and the like, I can 'see' the controls in
the application, but I
do not seem to be able to manipulate them programatically, and I
believe it us simply due
to my not understanding of the UI Automation API.

Hence, my request for example code to boot-strap myself.

Thanks,
Duane

 

[Paul McGuire]

The application I wish to interact with is not my own, but an ERP
system GUI front-end.

I have used pywinauto to drive a Flash game running inside of an
Internet Explorer browser - that's pretty GUI!

-- Paul

 

[DuaneKaufman]

I have used pywinauto to drive a Flash game running inside of an
Internet Explorer browser - that's pretty GUI!

-- Paul

Hi,

Could you share some code examples?

Thanks,
Duane (duanek (at) chorus (dot) net)

 

[Paul McGuire]

Hi,

Could you share some code examples?

Thanks,
Duane (duanek (at) chorus (dot) net)

I just went on a brief fishing expedition through two disk backups,
and no luck. I guess it's been a while since I worked on this.

The work I did was entirely graphical, which is to say, my script
interacted with the Flash program by using PIL to take image snapshots
of the window, and then sifting through the bitmap looking for the
status of a "time remaining" thermometer-style gauge in the game.
Then the script could click on X-Y coordinates within the window,
which would get picked up by the Flash game, and the script would
monitor the "time remaining" gauge some more, and so on.

I'm not sure how well pywinauto would work in allowing you to access
controls such as textboxes within a form. I remember that I had to
access the IE window using a caption name, and then found the embedded
Flash program as an embedded control of some sort, again, I probably
needed to indicate that it was some sort of "Adobe.FlashWidget1"
object or something.

I may have another, older disk backup at home, I can look for it later
this evening.

-- Paul

 

[Michael Torrie]

With MS utilities like UISpy and the like, I can 'see' the controls in
the application, but I
do not seem to be able to manipulate them programatically, and I
believe it us simply due
to my not understanding of the UI Automation API.

You're probably better off using a free app like AutoIt3 rather than
roll your own. AutoIt's scripting language isn't as nice as Python, but
it does the job and it turing-complete.

http://www.autoitscript.com/autoit3/

 

[Michael Torrie]

Alternatives to Microsoft's UI Automation are welcome too, but I have
tried using winguiauto and watsup (along with AutoIt), and there seems
to be severe limitations when using these tools with WinForm
applications.

http://www.autoitscript.com/forum/index.php?showtopic=96752 -- looks
like recent betas have much better support for WinForms, at least from
the change list.

 

[DuaneKaufman]

http://www.autoitscript.com/forum/index.php?showtopic=96752-- looks
like recent betas have much better support for WinForms, at least from
the change list.

Hi,

Thanks for the information. Boy, you are away from a utility for a
couple of
weeks, and it starts growing things one needs!

Unfortunately, while it looks like AutoIt is gaining the ability to
'see' WinForms controls,
this beta doesn't seem to be able to retrieve the text say, in a
TextBox. Darn.

Thanks,
Duane

 

[Kelie]

The University of North Carolina at Chapel Hill has a great Python package
called pyAA that does exactly this:

http://www.cs.unc.edu/Research/assi...trove.info/articles/gui-automation-with-pyaa/

Thanks Tim. From this page http://sourceforge.net/projects/uncassist/files/,
there isn't a installer for Python2.5 or higher. Didn't find the
source code there either.

 

[Mark]

Hi,

I am the author of pywinauto (http://pywinauto.openqa.org/).

It doesn't have specific functionality built in for winforms - but
many winforms controls are built on top of win32 functionality and
pywinauto can work with them for that.


Wow - cool


You might have a look at http://pywinauto.pbworks.com/ (the site is no
longer used - but there is a cool movie giving an example of using
pywinauto at the interactive prompt)

Other examples (Notepad, mspaint) are included with the install.

PyPi - http://pypi.python.org/pypi/pywinauto/0.3.8


Good luck!
Mark