Simulating Mouse Dragging

[Erik Veenstra]

Is it possible to simulate a mouse drag event in a browser (or
any program in general) with Ruby? Something like:

dragmouse(x1, y1, x2, y2)

or

dragmouse("windowname", x1, y1, x2, y2)

Either Windows or Linux (Xfree86).

Why? On the 'Net, there are a lot of sites which offer maps.
You can scroll these maps by dragging the image, virtually
flying over the map. You can look at such a map as one very,
very big image, of which you see only one part at a time.
Collecting these parts gives us little overlapping maps of a
certain region. These overlapping maps can be converted to
non-overlapping maps by cropping and saving with ImageMagick.
The resulting "tiles" are zipped in a file, which can be
handled with TIV (Tiled Image Viewer).

This is no theory, it actually works. I've made TIV in Ewe [1]
(kind of Java...). It runs on Windows (natively, fast), Linux
(Java, slow) and on my iPAQ (natively, fast enough). I did the
conversion from the overlapping maps to the non-overlapping
maps with ImageMagick. Not difficult, but it takes some
processing time. Saving the screenshot can be done with
"import" (part of ImageMagick). I glued everything together
with Ruby, of course.

I once got the images by hacking the URL's of one of those
sites, but they changed it... Grabbing the images by taking
screenshots is much better, because it works for every
"vendor"... The only missing part is the programmatically
dragging of the images in the browser.

Anybody able to help? Other ideas? (Except "buying" a
product...)

gegroet,
Erik V. - http://www.erikveen.dds.nl/

[1] http://www.ewesoft.com/

 

[Kero]

Is it possible to simulate a mouse drag event in a browser (or

any program in general) with Ruby? Something like:

dragmouse(x1, y1, x2, y2)

or

dragmouse("windowname", x1, y1, x2, y2)

Either Windows or Linux (Xfree86).

Could recording + analyzing mouse data help? Then you can simulate it
and let X11 consume it. `gpm -R` may help.

+--- Kero ------------------------- kero@chello@nl ---+
| all the meaningless and empty words I spoke |
| Promises -- The Cranberries |
+--- M38c --- http://members.chello.nl/k.vangelder ---+

 

[Erik Veenstra]

Could recording + analyzing mouse data help? Then you can
simulate it and let X11 consume it. `gpm -R` may help.

"Recording" the mouse data by reading /dev/gpmdata actually
works. Good start. Generating such a data stream is possible.
But what do we have to do to let X11 "consume" that stream?
"cat test > /dev/gpmdata" doesn't work...

gegroet,
Erik V. - http://www.erikveen.dds.nl/

 

[Kero]

Is it possible to simulate a mouse drag event in a browser

"Recording" the mouse data by reading /dev/gpmdata actually
works. Good start. Generating such a data stream is possible.
But what do we have to do to let X11 "consume" that stream?
"cat test > /dev/gpmdata" doesn't work...

disclaimer: I have never tried anything like this myself.

In what sense does it not work?
(assuming you made X11 read /dev/gpmdata one way or the other).
I would guess that `cat test` is "too fast".

+--- Kero ------------------------- kero@chello@nl ---+
| all the meaningless and empty words I spoke |
| Promises -- The Cranberries |
+--- M38c --- http://members.chello.nl/k.vangelder ---+

 

[Erik Veenstra]

Is it possible to simulate a mouse drag event in a

disclaimer: I have never tried anything like this myself.

In what sense does it not work? (assuming you made X11 read
/dev/gpmdata one way or the other). I would guess that `cat
test` is "too fast".

I experimented a bit, yesterday night. I didn't configure X11
to read from /dev/gpmdata. But I found that one. After that,
writing to /dev/gpmdata with a properly (?) filled memory
structure, actually did move the mouse. Clicking was possible,
too. But dragging somehow still doesn't work.

The protocol is IMPS/2.

I attached my experiment.

Somebody able and willing to help?

gegroet,
Erik V. - http://www.erikveen.dds.nl/

----------------------------------------------------------------

# http://www.mit.edu:8001/people/gassend/protocols/intellimouse/
# http://www.microsoft.com/whdc/device/input/mcompat.mspx

class Mouse
Yov = 0x80
Xov = 0x40
Y8 = 0x20
X8 = 0x10
BIT4 = 0x08
MB = 0x04
RB = 0x02
LB = 0x01

def initialize
@handle = File.new("/dev/gpmdata", "wb")
@handle.sync = true
@button = 0
end

def move(dx, dy)
@bytes = []

byte1 = (BIT4 | @button)
byte2 = (dx.to_i)
byte3 = (dy.to_i)
byte4 = 0

if dx < 0
byte1 |= X8
byte2 += 256
end

if dy < 0
byte1 |= Y8
byte3 += 256
end

@bytes << byte1
@bytes << byte2
@bytes << byte3
@bytes << byte4

send
end

def down(button)
@button = button
move(0, 0)
end

def up
@button = 0
move(0, 0)
end

def drag(dx, dy, button)
down(button)
move(dx, dy)
up
end

def click(button)
down(button)
up
end

def stream
@bytes.pack("c%d" % @bytes.length)
end

def inspect
@bytes.collect{|s| s.chr}.join("").unpack("H2"*@bytes.length)
end

def send
p inspect
@handle << stream
end
end

#system "sudo /etc/init.d/gpm stop" or puts "Oops"

mouse = Mouse.new

mouse.drag(-100, 0, Mouse::LB)
mouse.move(+100, 0)

#system "sudo /etc/init.d/gpm start" or puts "Oops"

----------------------------------------------------------------

 

[Detlef Reichl]

Am Samstag, den 30.07.2005, 05:01 +0900 schrieb Erik Veenstra:

Is it possible to simulate a mouse drag event in a browser (or
any program in general) with Ruby? Something like:

dragmouse(x1, y1, x2, y2)

or

dragmouse("windowname", x1, y1, x2, y2)

Either Windows or Linux (Xfree86).

If you realy want to do it in a browser the simplest way would be to
capture the MOUSEMOVE events in javascript.

Another way it to build an app with Gtk+. There you have complete mouse
controll and allso got image handling. The advantage over the browser
solution is, that it would run on Linux, OS X and Windoze. Within
javascript you have to program around the pittfalls of the different DOM
models.

Cheers
detlef

 

[William James]

Is it possible to simulate a mouse drag event in a browser (or
any program in general) with Ruby? Something like:

dragmouse(x1, y1, x2, y2)

or

dragmouse("windowname", x1, y1, x2, y2)

Either Windows or Linux (Xfree86).

I've never used it, but AutoIt for windows may be able to do what
you want: http://www.autoitscript.com/autoit3/index.php