Wikipedia Bot

[Peter Marsh]

I'm planning on writing a simple Wikipedia Bot, but I'm having a little
difficulty. The bot needs to be able to log in to Wikipedia, but I'm not
sure if this code is working. It returns the Main Page, which does
happen after you log in, but I'm not entirely sure if this means if I've
actually logged in or not. Here's what I've used (ripped from:
http://www.ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html):


require 'net/http'
require 'uri'

res =
Net::HTTP.post_form(URI.parse('http://en.wikipedia.org/w/index.php?title=Special:Userlogin&returnto=Main_Page'),{'wpName'=>'username_not_this_one',
'wpPassword'=>'password_not_this_one'})
puts res.body

My user & password were correct.

 

[Dan Zwell]

I'm planning on writing a simple Wikipedia Bot, but I'm having a little
difficulty. The bot needs to be able to log in to Wikipedia, but I'm not
sure if this code is working. It returns the Main Page, which does
happen after you log in, but I'm not entirely sure if this means if I've
actually logged in or not. Here's what I've used (ripped from:
http://www.ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html):


require 'net/http'
require 'uri'

res =
Net::HTTP.post_form(URI.parse('http://en.wikipedia.org/w/index.php?title=Special:Userlogin&returnto=Main_Page'),{'wpName'=>'username_not_this_one',
'wpPassword'=>'password_not_this_one'})
puts res.body

My user & password were correct.

On the main page, see the text where it says "Sign in / create account"?
Search the body for that text. It won't say that if the login was
successful.

Dan

 

[Peter Marsh]

On the main page, see the text where it says "Sign in / create account"?
Search the body for that text. It won't say that if the login was
successful.

Dan

That's the thing, I've tried that but I can't find 'sign in' 'sign out'
or the my username, which usually appears in the top right. Also, I've
tried looking at the pade via IE after I've run the code, and that says
I'm not signed in as well.

Any ideas?

 

[Marcin Raczkowski]

That's the thing, I've tried that but I can't find 'sign in' 'sign out'
or the my username, which usually appears in the top right. Also, I've
tried looking at the pade via IE after I've run the code, and that says
I'm not signed in as well.

Any ideas?

I recomend using scrubyt - there was example how to log into site, and post
comment - i guess it can be easly changed to wiki - but don't spam please

 

[Ryan Davis]

I'm planning on writing a simple Wikipedia Bot, but I'm having a
little
difficulty. The bot needs to be able to log in to Wikipedia, but
I'm not
sure if this code is working. It returns the Main Page, which does
happen after you log in, but I'm not entirely sure if this means if
I've
actually logged in or not.

Guess I'm going to be the one to ask: why do you need a wikipedia bot?

 

[Rick DeNatale]

Check if there is a cookie involved that stores your session. As far as
I understand the HTTP protocol, it is possible that WP is sending you a
redirect together with your session cookie, which it expects to be sent
back when (automatically) trying to GET the MainPage.

It is just an idea, I never looked at the login process of WP and
neither fully understand how much of the cookie handling is done for you
with the Net::HTTP module.

I'm pretty sure that mediawiki (the software behind wikipedia) does in
fact use a cookie for authorization. It's been a while since I looked
at the code.

The OP might want to have a look at the pywikipediabot framework and
see how they do it. This is the 'official' bot framework for
wikipedia. It's written in python as the name would indicate, but
python code isn't THAT hard to read. <G>

It would be cool if there were a ruby equivalent to pywikipedia.

 

[Peter Marsh]

Guess I'm going to be the one to ask: why do you need a wikipedia bot?

I don't need one at all personally, but there are loads of requests for
bots of all sorts from the community. I thought that it'd be a nice
project to start, and would actually be useful! I'm not upto no good

Pywikipedia looks pretty promising - since I'm learning Ruby I may as
well learn Python!

 

[Stephan Maka]

--lEGEL1/lMxI0MVQ2
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

It would be cool if there were a ruby equivalent to pywikipedia.

We have written what you're looking for, but a year ago and we've not
taken care of it since then. But it should be still working with recent
MediaWikis.

Be aware that most documentation is in german but the code itself is
english.

Homepage:
https://wiki.c3d2.de/Ruby-MediaWiki

Automatic RDoc in a Wiki:
https://wiki.c3d2.de/Ruby-MediaWiki/Documentation

Repository:
svn://svn.c3d2.de/ruby-mediawiki/trunk

Source browser:
http://trac.c3d2.de/ruby-mediawiki/browser/trunk/


Stephan

--lEGEL1/lMxI0MVQ2
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGUxoQuKMfCHUX8x0RAluqAJ9mvpciR5DtdAzVvhXRtzNF7EsKRwCeKv7X
man0x/a9m0PNIy/Q9lacbaw=
=Iruc
-----END PGP SIGNATURE-----

--lEGEL1/lMxI0MVQ2--

 

[Stephan Maka]

--9Ek0hoCL9XbhcSqy
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Be aware that most documentation is in german but the code itself is
english.

Err. Actually only the homepage is german, the RDoc is english.

--9Ek0hoCL9XbhcSqy
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGUxtBuKMfCHUX8x0RAumyAJ0Zkity/57Aa16G0n3BSNrOxLb2cQCfZQ0x
akLjo721Nvy8Yy4yxqGyWYA=
=Ex6E
-----END PGP SIGNATURE-----

--9Ek0hoCL9XbhcSqy--

 

[Peter Marsh]

Err. Actually only the homepage is german, the RDoc is english.

Excellent, I'll have a look!

 

[Rick DeNatale]

We have written what you're looking for, but a year ago and we've not
taken care of it since then.

Cool, if I once again need it I'll take a look.

But it should be still working with recent
MediaWikis.

There's the rub*, when I was using the pywikipediabot the team had a
time keeping up with the evolution of mediawiki. It's a problem with
screen-scraping, you need to keep up the the screen you're scraping.

*pre-s: It's happening to me again. For years I had the tendency to
type smalltalk automatically whenever I typed small. In fact it
lingers on sporadically even today. Just now I typed: There's the
ruby <G>

 

[benjohn]

*pre-s: It's happening to me again. For years I had the tendency to

type smalltalk automatically whenever I typed small. In fact it
lingers on sporadically even today. Just now I typed: There's the
ruby <G>

I get that now and then, but quite often I can't track down where it's
come from. Weird.

 

[Dan Zwell]

I get that now and then, but quite often I can't track down where it's
come from. Weird.

for a time, I couldn't type "myself" without first typing "mysql". Way
too much php scripting that summer.

 

[Kristoffer Lundén]

T24gNS8yMi8wNywgUGV0ZXIgTWFyc2ggPGV2aWxfZ3J1bmdlckBob3RtYWlsLmNvbT4gd3JvdGU6
Cj4KPiBQeXdpa2lwZWRpYSBsb29rcyBwcmV0dHkgcHJvbWlzaW5nIC0gc2luY2UgSSdtIGxlYXJu
aW5nIFJ1YnkgSSBtYXkgYXMKPiB3ZWxsIGxlYXJuIFB5dGhvbiEKPgoKTGFzdCBJIGxvb2tlZCBh
dCBQeXdpa2lwZWRpYSB0aGUgZGVzaWduIHdhcyBob3JyaWJseSBicm9rZW4gLSBkb24ndApyZWNh
bGwgZGV0YWlscyBleGFjdGx5LCBidXQgaXQgd2FzIHJlYWxseSByZWFsbHkgcGFpbmZ1bCB0byB0
cnkgYW5kCmV4dGVuZCBvciBlbmhhbmNlIGl0IGluIGFueSB3YXkuIFB1dHRpbmcgdXAgYSBzaW1w
bGUgUnVieS1ib3QgdGhhdCBkaWQKYSBiZXR0ZXIgam9iIHdhcyBhIG1hdHRlciBvZiBob3Vycy4g
VXNpbmcgc29tZSBvZiB0aGUgaGlnaC1sZXZlbCBIVFRQCmF1dG9tYXRpb24gbGlicyB0aGF0IGV4
aXN0IGl0IHNob3VsZCBiZSBldmVuIGZhc3RlciwgYW5kIHZlcnkKbWFpbnRhaW5hYmxlIGlmIHlv
dSBwdXQgc29tZSB0aG91Z2h0IGludG8gaXQuCgpEdXBsaWNhdGluZyBlZmZvcnQgaXMgdGhlIHJp
Z2h0IHdheSB0byBnbyBhdCB0aW1lcy4gOikKCi0tIEsKCi0tIApLcmlzdG9mZmVyIEx1bmTDqW4K
4pyJIGtyaXN0b2ZmZXIubHVuZGVuQGdtYWlsLmNvbQrinIkga3Jpc3RvZmZlci5sdW5kZW5AZ2Ft
ZW1ha2VyLm51Cmh0dHA6Ly93d3cuZ2FtZW1ha2VyLm51LwrimI4gMDcwNCA0OCA5OCA3Nwo=

 

[Peter Marsh]

Last I looked at Pywikipedia the design was horribly broken - don't
recall details exactly, but it was really really painful to try and
extend or enhance it in any way. Putting up a simple Ruby-bot that did
a better job was a matter of hours. Using some of the high-level HTTP
automation libs that exist it should be even faster, and very
maintainable if you put some thought into it.

Duplicating effort is the right way to go at times.

-- K

Well, from what I've seen so far it doesn't actually seem that bad -
however I'm really new to Pywikipedia/Ruby/Python so you probally know
better than me . The biggest pain in the arse at the moment is
actually getting permission to run a bot!

 

[Kristoffer Lundén]

T24gNS8yNS8wNywgUGV0ZXIgTWFyc2ggPGV2aWxfZ3J1bmdlckBob3RtYWlsLmNvbT4gd3JvdGU6
Cj4KPiBXZWxsLCBmcm9tIHdoYXQgSSd2ZSBzZWVuIHNvIGZhciBpdCBkb2Vzbid0IGFjdHVhbGx5
IHNlZW0gdGhhdCBiYWQgLQo+IGhvd2V2ZXIgSSdtIHJlYWxseSBuZXcgdG8gUHl3aWtpcGVkaWEv
UnVieS9QeXRob24gc28geW91IHByb2JhbGx5IGtub3cKPiBiZXR0ZXIgdGhhbiBtZSA6KS4gVGhl
IGJpZ2dlc3QgcGFpbiBpbiB0aGUgYXJzZSBhdCB0aGUgbW9tZW50IGlzCj4gYWN0dWFsbHkgZ2V0
dGluZyBwZXJtaXNzaW9uIHRvIHJ1biBhIGJvdCEKPgoKSWYgeW91IGp1c3Qgd2FudCB0byBkbyB3
aGF0IGl0IGFscmVhZHkgZG9lcywgc3VjaCBhcyBoZWxwaW5nIHdpdGgKY3Jvc3MtbGFuZ3VhZ2Ug
cmVmZXJlbmNpbmcsICBpdCdzIHByb2JhYmx5IGZpbmUuIEl0J3Mgd2hlbiB5b3Ugd2FudCB0bwpk
byBzb21ldGhpbmcgbmV3LCBvciBzb21ldGhpbmcgbW9yZSBhZHZhbmNlZCBpdCBnZXRzIHJlYWxs
eSB0cmlja3kuCgotLSBLCgoKPiAtLQo+IFBvc3RlZCB2aWEgaHR0cDovL3d3dy5ydWJ5LWZvcnVt
LmNvbS8uCj4KPgoKCi0tIApLcmlzdG9mZmVyIEx1bmTDqW4K4pyJIGtyaXN0b2ZmZXIubHVuZGVu
QGdtYWlsLmNvbQrinIkga3Jpc3RvZmZlci5sdW5kZW5AZ2FtZW1ha2VyLm51Cmh0dHA6Ly93d3cu
Z2FtZW1ha2VyLm51LwrimI4gMDcwNCA0OCA5OCA3Nwo=