how to retrieve data from given websites

Discussion in 'ASP .Net' started by zEE, Jun 3, 2006.

  1. zEE

    zEE Guest

    let say u have a website
    string url="http://www.hotmail.com";

    now u can use MSXML dll

    XMLHTTP ObjHttp = new XMLHTTP();
    //Connecting to site
    ObjHttp.open("post",url,false,null, null);
    //Sending parameters (may be u dont required it)
    ObjHttp.send("case="+TicketNum");

    if (ObjHttp.status.ToString() == "200") //successfull response
    {
    return ObjHttp.responseText;
    }

    "ObjHttp.responseText" this will be string having html of the page
    u need to parse it using Regex to get ur desired data

    Hope it will help u
     
    zEE, Jun 3, 2006
    #1
    1. Advertising

  2. Yoshitha,
    Check the documentation for the WebClient class. This is a simplified
    wrapper over HttpWebRequest / Response. It has four basic methods. There's
    plenty of good sample code in the MSDN documentation.
    Peter

    --
    Co-founder, Eggheadcafe.com developer portal:
    http://www.eggheadcafe.com
    UnBlog:
    http://petesbloggerama.blogspot.com




    "Yoshitha" wrote:

    > Hi
    >
    > I want to Extracet(retrieve) data from some of websites and the retrieved
    > data has to be saved in excel or.csv files.
    >
    >
    > I need to develop this in ASP.net(using C#.net Vs 2003).
    >
    > Can any one tell me the idea of how to get data from websites?
    >
    > Thanx in advance
    > Yoshitha
    >
    >
    >
     
    =?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=, Jun 3, 2006
    #2
    1. Advertising

  3. zEE

    Yoshitha Guest

    Hi

    I want to Extracet(retrieve) data from some of websites and the retrieved
    data has to be saved in excel or.csv files.


    I need to develop this in ASP.net(using C#.net Vs 2003).

    Can any one tell me the idea of how to get data from websites?

    Thanx in advance
    Yoshitha
     
    Yoshitha, Jun 3, 2006
    #3
  4. zEE

    zEE Guest

    all depends on ur needs.. what u really want to achieve from ur desired
    website
     
    zEE, Jun 5, 2006
    #4
  5. zEE

    Ray Booysen Guest

    mmmm screen scraping. Do you have permission to use the information
    you'll be taking from this site?

    Yoshitha wrote:
    > I need to get contact details of doctors, dentisit and so on from a site.
    > and the gathered details has to be stored in .xle or .csv file.
    > i need to gather data from this site
    > http://www.mydr.com.au/hcp/hcp.asp?mode=Pharmacist
    > http://www.mydr.com.au/hcp/hcp.asp?mode=Specialist
    > like wise i have some more urls.
    > the above links gives you teh search page in that we need to enter some
    > names for eg(AD) in surname text box and click on search button then it will
    > display some hyperlnks and when we click on hyperlink it will give contact
    > details.
    > and i need to get those details and store them into .xls or csv files.
    > here i used AD but we need to search with all posible combinations from A to
    > Z
    > and need to retrieve contact details of doctors etc.
    >
    > that is my requirement
    > i need to achieve this using any method.
    >
    > one more thing, for exmple if the page contains contact details directly of
    > all teh doctors and if that page is having paging(page 1,2...) if i want to
    > get data from all these pages how can be this achieved can you tell me.
    >
    > REgards
    > Yoshitha
    >
    >
    >
    >
    >
    > "zEE" <> wrote in message
    > news:...
    >> all depends on ur needs.. what u really want to achieve from ur desired
    >> website
    >>

    >
    >
     
    Ray Booysen, Jun 5, 2006
    #5
  6. in VB's myobject you can find urldownloadtofile
    Must be present in c# as well imo search..
    It's the easiest way.



    "Yoshitha" <> schreef in bericht
    news:%...
    > yes we do have permission
    > we can use either the urls i mentioned or can use yellowpages also
    > http://www.yellowpages.com.au/search/postCategorySearch.do?headingCode=23051
    > &businessType=dentist&stateId=9.
    >
    > can you tell me how to get data from yellow pages link?
    >
    >
    > "Ray Booysen" <> wrote in message
    > news:#...
    >> mmmm screen scraping. Do you have permission to use the information
    >> you'll be taking from this site?
    >>
    >> Yoshitha wrote:
    >> > I need to get contact details of doctors, dentisit and so on from a

    > site.
    >> > and the gathered details has to be stored in .xle or .csv file.
    >> > i need to gather data from this site
    >> > http://www.mydr.com.au/hcp/hcp.asp?mode=Pharmacist
    >> > http://www.mydr.com.au/hcp/hcp.asp?mode=Specialist
    >> > like wise i have some more urls.
    >> > the above links gives you teh search page in that we need to enter
    >> > some
    >> > names for eg(AD) in surname text box and click on search button then it

    > will
    >> > display some hyperlnks and when we click on hyperlink it will give

    > contact
    >> > details.
    >> > and i need to get those details and store them into .xls or csv files.
    >> > here i used AD but we need to search with all posible combinations from

    > A to
    >> > Z
    >> > and need to retrieve contact details of doctors etc.
    >> >
    >> > that is my requirement
    >> > i need to achieve this using any method.
    >> >
    >> > one more thing, for exmple if the page contains contact details
    >> > directly

    > of
    >> > all teh doctors and if that page is having paging(page 1,2...) if i
    >> > want

    > to
    >> > get data from all these pages how can be this achieved can you tell me.
    >> >
    >> > REgards
    >> > Yoshitha
    >> >
    >> >
    >> >
    >> >
    >> >
    >> > "zEE" <> wrote in message
    >> > news:...
    >> >> all depends on ur needs.. what u really want to achieve from ur
    >> >> desired
    >> >> website
    >> >>
    >> >
    >> >

    >
    >
     
    Edwin Knoppert, Jun 5, 2006
    #6
  7. zEE

    Yoshitha Guest

    hi
    is it possible to retrieve data from other websites uing XTerm, XPath
    Technology of ASP.net with regular Expressions.

    "zEE" <> wrote in message
    news:...
    >
    > let say u have a website
    > string url="http://www.hotmail.com";
    >
    > now u can use MSXML dll
    >
    > XMLHTTP ObjHttp = new XMLHTTP();
    > //Connecting to site
    > ObjHttp.open("post",url,false,null, null);
    > //Sending parameters (may be u dont required it)
    > ObjHttp.send("case="+TicketNum");
    >
    > if (ObjHttp.status.ToString() == "200") //successfull response
    > {
    > return ObjHttp.responseText;
    > }
    >
    > "ObjHttp.responseText" this will be string having html of the page
    > u need to parse it using Regex to get ur desired data
    >
    > Hope it will help u
    >
     
    Yoshitha, Jun 5, 2006
    #7
  8. zEE

    Yoshitha Guest

    I need to get contact details of doctors, dentisit and so on from a site.
    and the gathered details has to be stored in .xle or .csv file.
    i need to gather data from this site
    http://www.mydr.com.au/hcp/hcp.asp?mode=Pharmacist
    http://www.mydr.com.au/hcp/hcp.asp?mode=Specialist
    like wise i have some more urls.
    the above links gives you teh search page in that we need to enter some
    names for eg(AD) in surname text box and click on search button then it will
    display some hyperlnks and when we click on hyperlink it will give contact
    details.
    and i need to get those details and store them into .xls or csv files.
    here i used AD but we need to search with all posible combinations from A to
    Z
    and need to retrieve contact details of doctors etc.

    that is my requirement
    i need to achieve this using any method.

    one more thing, for exmple if the page contains contact details directly of
    all teh doctors and if that page is having paging(page 1,2...) if i want to
    get data from all these pages how can be this achieved can you tell me.

    REgards
    Yoshitha





    "zEE" <> wrote in message
    news:...
    > all depends on ur needs.. what u really want to achieve from ur desired
    > website
    >
     
    Yoshitha, Jun 5, 2006
    #8
  9. zEE

    Yoshitha Guest

    yes we do have permission
    we can use either the urls i mentioned or can use yellowpages also
    http://www.yellowpages.com.au/search/postCategorySearch.do?headingCode=23051
    &businessType=dentist&stateId=9.

    can you tell me how to get data from yellow pages link?


    "Ray Booysen" <> wrote in message
    news:#...
    > mmmm screen scraping. Do you have permission to use the information
    > you'll be taking from this site?
    >
    > Yoshitha wrote:
    > > I need to get contact details of doctors, dentisit and so on from a

    site.
    > > and the gathered details has to be stored in .xle or .csv file.
    > > i need to gather data from this site
    > > http://www.mydr.com.au/hcp/hcp.asp?mode=Pharmacist
    > > http://www.mydr.com.au/hcp/hcp.asp?mode=Specialist
    > > like wise i have some more urls.
    > > the above links gives you teh search page in that we need to enter some
    > > names for eg(AD) in surname text box and click on search button then it

    will
    > > display some hyperlnks and when we click on hyperlink it will give

    contact
    > > details.
    > > and i need to get those details and store them into .xls or csv files.
    > > here i used AD but we need to search with all posible combinations from

    A to
    > > Z
    > > and need to retrieve contact details of doctors etc.
    > >
    > > that is my requirement
    > > i need to achieve this using any method.
    > >
    > > one more thing, for exmple if the page contains contact details directly

    of
    > > all teh doctors and if that page is having paging(page 1,2...) if i want

    to
    > > get data from all these pages how can be this achieved can you tell me.
    > >
    > > REgards
    > > Yoshitha
    > >
    > >
    > >
    > >
    > >
    > > "zEE" <> wrote in message
    > > news:...
    > >> all depends on ur needs.. what u really want to achieve from ur desired
    > >> website
    > >>

    > >
    > >
     
    Yoshitha, Jun 5, 2006
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Robert Maas, see http://tinyurl.com/uh3t

    In servlet: Given session-ID, how to retrieve session object?

    Robert Maas, see http://tinyurl.com/uh3t, Jun 5, 2005, in forum: Java
    Replies:
    2
    Views:
    5,928
    Robert Maas, see http://tinyurl.com/uh3t
    Jun 26, 2005
  2. Lord0
    Replies:
    1
    Views:
    581
    Thomas Weidenfeller
    Apr 19, 2006
  3. chiara
    Replies:
    6
    Views:
    490
    Barry Schwarz
    Oct 6, 2005
  4. 2Barter.net
    Replies:
    0
    Views:
    380
    2Barter.net
    Dec 13, 2006
  5. Casey Hawthorne
    Replies:
    385
    Views:
    5,867
    ng2010
    Apr 4, 2010
Loading...

Share This Page