Wikipedia Bot

Discussion in 'Ruby' started by Peter Marsh, May 21, 2007.

  1. Peter Marsh

    Peter Marsh Guest

    I'm planning on writing a simple Wikipedia Bot, but I'm having a little
    difficulty. The bot needs to be able to log in to Wikipedia, but I'm not
    sure if this code is working. It returns the Main Page, which does
    happen after you log in, but I'm not entirely sure if this means if I've
    actually logged in or not. Here's what I've used (ripped from:
    http://www.ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html):


    require 'net/http'
    require 'uri'

    res =
    Net::HTTP.post_form(URI.parse('http://en.wikipedia.org/w/index.php?title=Special:Userlogin&returnto=Main_Page'),{'wpName'=>'username_not_this_one',
    'wpPassword'=>'password_not_this_one'})
    puts res.body

    My user & password were correct.

    --
    Posted via http://www.ruby-forum.com/.
     
    Peter Marsh, May 21, 2007
    #1
    1. Advertising

  2. Peter Marsh

    Dan Zwell Guest

    Peter Marsh wrote:
    > I'm planning on writing a simple Wikipedia Bot, but I'm having a little
    > difficulty. The bot needs to be able to log in to Wikipedia, but I'm not
    > sure if this code is working. It returns the Main Page, which does
    > happen after you log in, but I'm not entirely sure if this means if I've
    > actually logged in or not. Here's what I've used (ripped from:
    > http://www.ruby-doc.org/stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html):
    >
    >
    > require 'net/http'
    > require 'uri'
    >
    > res =
    > Net::HTTP.post_form(URI.parse('http://en.wikipedia.org/w/index.php?title=Special:Userlogin&returnto=Main_Page'),{'wpName'=>'username_not_this_one',
    > 'wpPassword'=>'password_not_this_one'})
    > puts res.body
    >
    > My user & password were correct.
    >


    On the main page, see the text where it says "Sign in / create account"?
    Search the body for that text. It won't say that if the login was
    successful.

    Dan
     
    Dan Zwell, May 21, 2007
    #2
    1. Advertising

  3. Peter Marsh

    Peter Marsh Guest

    Dan Zwell wrote:
    > Peter Marsh wrote:
    >>
    >> res =
    >> Net::HTTP.post_form(URI.parse('http://en.wikipedia.org/w/index.php?title=Special:Userlogin&returnto=Main_Page'),{'wpName'=>'username_not_this_one',
    >> 'wpPassword'=>'password_not_this_one'})
    >> puts res.body
    >>
    >> My user & password were correct.
    >>

    >
    > On the main page, see the text where it says "Sign in / create account"?
    > Search the body for that text. It won't say that if the login was
    > successful.
    >
    > Dan


    That's the thing, I've tried that but I can't find 'sign in' 'sign out'
    or the my username, which usually appears in the top right. Also, I've
    tried looking at the pade via IE after I've run the code, and that says
    I'm not signed in as well.

    Any ideas?

    --
    Posted via http://www.ruby-forum.com/.
     
    Peter Marsh, May 21, 2007
    #3
  4. On Monday 21 May 2007 20:52, Peter Marsh wrote:
    > Dan Zwell wrote:
    > > Peter Marsh wrote:
    > >> res =
    > >> Net::HTTP.post_form(URI.parse('http://en.wikipedia.org/w/index.php?title
    > >>=Special:Userlogin&returnto=Main_Page'),{'wpName'=>'username_not_this_one
    > >>', 'wpPassword'=>'password_not_this_one'})
    > >> puts res.body
    > >>
    > >> My user & password were correct.

    > >
    > > On the main page, see the text where it says "Sign in / create account"?
    > > Search the body for that text. It won't say that if the login was
    > > successful.
    > >
    > > Dan

    >
    > That's the thing, I've tried that but I can't find 'sign in' 'sign out'
    > or the my username, which usually appears in the top right. Also, I've
    > tried looking at the pade via IE after I've run the code, and that says
    > I'm not signed in as well.
    >
    > Any ideas?


    I recomend using scrubyt - there was example how to log into site, and post
    comment - i guess it can be easly changed to wiki - but don't spam please :D

    --
    Marcin Raczkowski
    ---
    Friends teach what you should know
    Enemies Teach what you have to know
     
    Marcin Raczkowski, May 21, 2007
    #4
  5. Peter Marsh

    Ryan Davis Guest

    On May 21, 2007, at 10:27 , Peter Marsh wrote:

    > I'm planning on writing a simple Wikipedia Bot, but I'm having a
    > little
    > difficulty. The bot needs to be able to log in to Wikipedia, but
    > I'm not
    > sure if this code is working. It returns the Main Page, which does
    > happen after you log in, but I'm not entirely sure if this means if
    > I've
    > actually logged in or not.


    Guess I'm going to be the one to ask: why do you need a wikipedia bot?
     
    Ryan Davis, May 22, 2007
    #5
  6. On 5/22/07, Christian Luginbuehl <> wrote:
    > > That's the thing, I've tried that but I can't find 'sign in' 'sign out'
    > > or the my username, which usually appears in the top right. Also, I've
    > > tried looking at the pade via IE after I've run the code, and that says
    > > I'm not signed in as well.
    > >
    > > Any ideas?

    >
    > Check if there is a cookie involved that stores your session. As far as
    > I understand the HTTP protocol, it is possible that WP is sending you a
    > redirect together with your session cookie, which it expects to be sent
    > back when (automatically) trying to GET the MainPage.
    >
    > It is just an idea, I never looked at the login process of WP and
    > neither fully understand how much of the cookie handling is done for you
    > with the Net::HTTP module.


    I'm pretty sure that mediawiki (the software behind wikipedia) does in
    fact use a cookie for authorization. It's been a while since I looked
    at the code.

    The OP might want to have a look at the pywikipediabot framework and
    see how they do it. This is the 'official' bot framework for
    wikipedia. It's written in python as the name would indicate, but
    python code isn't THAT hard to read. <G>

    It would be cool if there were a ruby equivalent to pywikipedia.
    --
    Rick DeNatale

    My blog on Ruby
    http://talklikeaduck.denhaven2.com/
     
    Rick DeNatale, May 22, 2007
    #6
  7. Peter Marsh

    Peter Marsh Guest

    Ryan Davis wrote:
    > On May 21, 2007, at 10:27 , Peter Marsh wrote:
    >
    >> I'm planning on writing a simple Wikipedia Bot, but I'm having a
    >> little
    >> difficulty. The bot needs to be able to log in to Wikipedia, but
    >> I'm not
    >> sure if this code is working. It returns the Main Page, which does
    >> happen after you log in, but I'm not entirely sure if this means if
    >> I've
    >> actually logged in or not.

    >
    > Guess I'm going to be the one to ask: why do you need a wikipedia bot?


    I don't need one at all personally, but there are loads of requests for
    bots of all sorts from the community. I thought that it'd be a nice
    project to start, and would actually be useful! I'm not upto no good :p

    Pywikipedia looks pretty promising - since I'm learning Ruby I may as
    well learn Python!

    --
    Posted via http://www.ruby-forum.com/.
     
    Peter Marsh, May 22, 2007
    #7
  8. Peter Marsh

    Stephan Maka Guest

    [ANN] Ruby-MediaWiki (was: Wikipedia Bot)

    --lEGEL1/lMxI0MVQ2
    Content-Type: text/plain; charset=us-ascii
    Content-Disposition: inline

    Rick DeNatale wrote:
    > It would be cool if there were a ruby equivalent to pywikipedia.


    We have written what you're looking for, but a year ago and we've not
    taken care of it since then. But it should be still working with recent
    MediaWikis.

    Be aware that most documentation is in german but the code itself is
    english.

    Homepage:
    https://wiki.c3d2.de/Ruby-MediaWiki

    Automatic RDoc in a Wiki:
    https://wiki.c3d2.de/Ruby-MediaWiki/Documentation

    Repository:
    svn://svn.c3d2.de/ruby-mediawiki/trunk

    Source browser:
    http://trac.c3d2.de/ruby-mediawiki/browser/trunk/


    Stephan

    --lEGEL1/lMxI0MVQ2
    Content-Type: application/pgp-signature
    Content-Disposition: inline

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFGUxoQuKMfCHUX8x0RAluqAJ9mvpciR5DtdAzVvhXRtzNF7EsKRwCeKv7X
    man0x/a9m0PNIy/Q9lacbaw=
    =Iruc
    -----END PGP SIGNATURE-----

    --lEGEL1/lMxI0MVQ2--
     
    Stephan Maka, May 22, 2007
    #8
  9. Peter Marsh

    Stephan Maka Guest

    Re: [ANN] Ruby-MediaWiki (was: Wikipedia Bot)

    --9Ek0hoCL9XbhcSqy
    Content-Type: text/plain; charset=us-ascii
    Content-Disposition: inline

    Stephan Maka wrote:
    > Be aware that most documentation is in german but the code itself is
    > english.


    Err. Actually only the homepage is german, the RDoc is english.

    --9Ek0hoCL9XbhcSqy
    Content-Type: application/pgp-signature
    Content-Disposition: inline

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFGUxtBuKMfCHUX8x0RAumyAJ0Zkity/57Aa16G0n3BSNrOxLb2cQCfZQ0x
    akLjo721Nvy8Yy4yxqGyWYA=
    =Ex6E
    -----END PGP SIGNATURE-----

    --9Ek0hoCL9XbhcSqy--
     
    Stephan Maka, May 22, 2007
    #9
  10. Peter Marsh

    Peter Marsh Guest

    Re: Ruby-MediaWiki (was: Wikipedia Bot)

    Stephan Maka wrote:
    > Stephan Maka wrote:
    >> Be aware that most documentation is in german but the code itself is
    >> english.

    >
    > Err. Actually only the homepage is german, the RDoc is english.


    Excellent, I'll have a look!

    --
    Posted via http://www.ruby-forum.com/.
     
    Peter Marsh, May 22, 2007
    #10
  11. Re: [ANN] Ruby-MediaWiki (was: Wikipedia Bot)

    On 5/22/07, Stephan Maka <> wrote:
    > Rick DeNatale wrote:
    > > It would be cool if there were a ruby equivalent to pywikipedia.

    >
    > We have written what you're looking for, but a year ago and we've not
    > taken care of it since then.


    Cool, if I once again need it I'll take a look.

    > But it should be still working with recent
    > MediaWikis.


    There's the rub*, when I was using the pywikipediabot the team had a
    time keeping up with the evolution of mediawiki. It's a problem with
    screen-scraping, you need to keep up the the screen you're scraping.

    *pre-s: It's happening to me again. For years I had the tendency to
    type smalltalk automatically whenever I typed small. In fact it
    lingers on sporadically even today. Just now I typed: There's the
    ruby <G>
    --
    Rick DeNatale

    My blog on Ruby
    http://talklikeaduck.denhaven2.com/
     
    Rick DeNatale, May 22, 2007
    #11
  12. Peter Marsh

    Guest

    Re: [ANN] Ruby-MediaWiki (was: Wikipedia Bot)

    > *pre-s: It's happening to me again. For years I had the tendency to
    > type smalltalk automatically whenever I typed small. In fact it
    > lingers on sporadically even today. Just now I typed: There's the
    > ruby <G>


    I get that now and then, but quite often I can't track down where it's
    come from. Weird.
     
    , May 22, 2007
    #12
  13. Peter Marsh

    Dan Zwell Guest

    Re: [ANN] Ruby-MediaWiki

    wrote:
    >> *pre-s: It's happening to me again. For years I had the tendency to
    >> type smalltalk automatically whenever I typed small. In fact it
    >> lingers on sporadically even today. Just now I typed: There's the
    >> ruby <G>

    >
    > I get that now and then, but quite often I can't track down where it's
    > come from. Weird.
    >
    >
    >

    for a time, I couldn't type "myself" without first typing "mysql". Way
    too much php scripting that summer.
     
    Dan Zwell, May 23, 2007
    #13
  14. T24gNS8yMi8wNywgUGV0ZXIgTWFyc2ggPGV2aWxfZ3J1bmdlckBob3RtYWlsLmNvbT4gd3JvdGU6
    Cj4KPiBQeXdpa2lwZWRpYSBsb29rcyBwcmV0dHkgcHJvbWlzaW5nIC0gc2luY2UgSSdtIGxlYXJu
    aW5nIFJ1YnkgSSBtYXkgYXMKPiB3ZWxsIGxlYXJuIFB5dGhvbiEKPgoKTGFzdCBJIGxvb2tlZCBh
    dCBQeXdpa2lwZWRpYSB0aGUgZGVzaWduIHdhcyBob3JyaWJseSBicm9rZW4gLSBkb24ndApyZWNh
    bGwgZGV0YWlscyBleGFjdGx5LCBidXQgaXQgd2FzIHJlYWxseSByZWFsbHkgcGFpbmZ1bCB0byB0
    cnkgYW5kCmV4dGVuZCBvciBlbmhhbmNlIGl0IGluIGFueSB3YXkuIFB1dHRpbmcgdXAgYSBzaW1w
    bGUgUnVieS1ib3QgdGhhdCBkaWQKYSBiZXR0ZXIgam9iIHdhcyBhIG1hdHRlciBvZiBob3Vycy4g
    VXNpbmcgc29tZSBvZiB0aGUgaGlnaC1sZXZlbCBIVFRQCmF1dG9tYXRpb24gbGlicyB0aGF0IGV4
    aXN0IGl0IHNob3VsZCBiZSBldmVuIGZhc3RlciwgYW5kIHZlcnkKbWFpbnRhaW5hYmxlIGlmIHlv
    dSBwdXQgc29tZSB0aG91Z2h0IGludG8gaXQuCgpEdXBsaWNhdGluZyBlZmZvcnQgaXMgdGhlIHJp
    Z2h0IHdheSB0byBnbyBhdCB0aW1lcy4gOikKCi0tIEsKCi0tIApLcmlzdG9mZmVyIEx1bmTDqW4K
    4pyJIGtyaXN0b2ZmZXIubHVuZGVuQGdtYWlsLmNvbQrinIkga3Jpc3RvZmZlci5sdW5kZW5AZ2Ft
    ZW1ha2VyLm51Cmh0dHA6Ly93d3cuZ2FtZW1ha2VyLm51LwrimI4gMDcwNCA0OCA5OCA3Nwo=
     
    Kristoffer Lundén, May 25, 2007
    #14
  15. Peter Marsh

    Peter Marsh Guest

    Kristoffer Lundén wrote:
    > On 5/22/07, Peter Marsh <> wrote:
    >>
    >> Pywikipedia looks pretty promising - since I'm learning Ruby I may as
    >> well learn Python!
    >>

    >
    > Last I looked at Pywikipedia the design was horribly broken - don't
    > recall details exactly, but it was really really painful to try and
    > extend or enhance it in any way. Putting up a simple Ruby-bot that did
    > a better job was a matter of hours. Using some of the high-level HTTP
    > automation libs that exist it should be even faster, and very
    > maintainable if you put some thought into it.
    >
    > Duplicating effort is the right way to go at times. :)
    >
    > -- K


    Well, from what I've seen so far it doesn't actually seem that bad -
    however I'm really new to Pywikipedia/Ruby/Python so you probally know
    better than me :). The biggest pain in the arse at the moment is
    actually getting permission to run a bot!

    --
    Posted via http://www.ruby-forum.com/.
     
    Peter Marsh, May 25, 2007
    #15
  16. T24gNS8yNS8wNywgUGV0ZXIgTWFyc2ggPGV2aWxfZ3J1bmdlckBob3RtYWlsLmNvbT4gd3JvdGU6
    Cj4KPiBXZWxsLCBmcm9tIHdoYXQgSSd2ZSBzZWVuIHNvIGZhciBpdCBkb2Vzbid0IGFjdHVhbGx5
    IHNlZW0gdGhhdCBiYWQgLQo+IGhvd2V2ZXIgSSdtIHJlYWxseSBuZXcgdG8gUHl3aWtpcGVkaWEv
    UnVieS9QeXRob24gc28geW91IHByb2JhbGx5IGtub3cKPiBiZXR0ZXIgdGhhbiBtZSA6KS4gVGhl
    IGJpZ2dlc3QgcGFpbiBpbiB0aGUgYXJzZSBhdCB0aGUgbW9tZW50IGlzCj4gYWN0dWFsbHkgZ2V0
    dGluZyBwZXJtaXNzaW9uIHRvIHJ1biBhIGJvdCEKPgoKSWYgeW91IGp1c3Qgd2FudCB0byBkbyB3
    aGF0IGl0IGFscmVhZHkgZG9lcywgc3VjaCBhcyBoZWxwaW5nIHdpdGgKY3Jvc3MtbGFuZ3VhZ2Ug
    cmVmZXJlbmNpbmcsICBpdCdzIHByb2JhYmx5IGZpbmUuIEl0J3Mgd2hlbiB5b3Ugd2FudCB0bwpk
    byBzb21ldGhpbmcgbmV3LCBvciBzb21ldGhpbmcgbW9yZSBhZHZhbmNlZCBpdCBnZXRzIHJlYWxs
    eSB0cmlja3kuCgotLSBLCgoKPiAtLQo+IFBvc3RlZCB2aWEgaHR0cDovL3d3dy5ydWJ5LWZvcnVt
    LmNvbS8uCj4KPgoKCi0tIApLcmlzdG9mZmVyIEx1bmTDqW4K4pyJIGtyaXN0b2ZmZXIubHVuZGVu
    QGdtYWlsLmNvbQrinIkga3Jpc3RvZmZlci5sdW5kZW5AZ2FtZW1ha2VyLm51Cmh0dHA6Ly93d3cu
    Z2FtZW1ha2VyLm51LwrimI4gMDcwNCA0OCA5OCA3Nwo=
     
    Kristoffer Lundén, May 25, 2007
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Scott Allen
    Replies:
    4
    Views:
    1,075
    bruce barker
    Jan 14, 2005
  2. Lilian BRUN
    Replies:
    2
    Views:
    516
    Lilian BRUN
    Mar 5, 2005
  3. Nobody
    Replies:
    0
    Views:
    1,536
    Nobody
    Sep 21, 2005
  4. Nicola Larosa

    The snake bot

    Nicola Larosa, Aug 22, 2003, in forum: Python
    Replies:
    0
    Views:
    369
    Nicola Larosa
    Aug 22, 2003
  5. gabor

    is there a python icq bot?

    gabor, Jun 16, 2004, in forum: Python
    Replies:
    0
    Views:
    341
    gabor
    Jun 16, 2004
Loading...

Share This Page