urllib and threadings

Discussion in 'Python' started by jango2, Aug 8, 2008.

  1. jango2

    jango2

    Joined:
    Aug 8, 2008
    Messages:
    1
    Hi there

    I am trying to get pages that containg special word after posting vales.

    how can i make this script work in threading about 10 thread ..

    below is my code

    # -*- coding: cp949 -*-
    import csv, sys
    import urllib
    import urllib2
    import time
    import string
    import threading
    URL = sys.argv[2]

    def get_page(no,year):
    try:
    url = sys.argv[2]
    user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
    values = {no_value : no,year_value : year }
    data = urllib.urlencode(values)
    headers = { 'User-Agent' : user_agent }
    req = urllib2.Request(url, data, headers)

    for line in urllib2.urlopen(req):
    if sys.argv[3] in line:
    print no, year

    except:
    pass

    f = open(sys.argv[1], 'r')
    for line in f:
    no, year= line.strip().split(':')
    get_page(no,year)

    # I run this script like below

    # get.py list.txt http://www.google.com/post.php "keyword"
    # if the pages contains sys.argv[3]
    it print out "no" and "year" to the promtpt

    # list.txt contains like this (no:year)
    1:1999
    2:2000
    3:2001
    4:2002
    5:2003
    #####################

    help me to get this script work in thread mode

    thanks...
     
    Last edited: Aug 8, 2008
    jango2, Aug 8, 2008
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. john
    Replies:
    1
    Views:
    386
    Ng Pheng Siong
    Aug 1, 2003
  2. zxo102
    Replies:
    2
    Views:
    254
    zxo102
    Jul 19, 2007
  3. Jonathan Gardner

    Asynchronous urllib (urllib+asyncore)?

    Jonathan Gardner, Feb 26, 2008, in forum: Python
    Replies:
    1
    Views:
    481
    Terry Jones
    Feb 27, 2008
  4. Chris McDonald
    Replies:
    0
    Views:
    312
    Chris McDonald
    Nov 1, 2010
  5. Marcus
    Replies:
    8
    Views:
    64
    Roy Smith
    Mar 3, 2014
Loading...

Share This Page