What am I doing wrong? curl

Discussion in 'C Programming' started by Shutdownrunner, Sep 5, 2005.

  1. I want to store result of curl in a variable, which means to store a
    webpage in a variable in order to parse it later and get our some useful
    information. But unfortunately I'm not too experienced in C and I'm
    making some stupid mistake. Could someone help me solve it?
    #include <string.h>
    #include <curl/curl.h>


    size_t write_data(const char *buffer, size_t size, size_t nmemb, char
    *userp)
    {
    char *string = userp;
    size_t len;
    len = size * nmemb;
    strncat(string, buffer,len); // I get segfault here. Why????
    return len;
    }



    int main(int argc, char **argv)
    {
    char *tablica="";
    CURL *curl;
    CURLcode success;
    curl = curl_easy_init();
    if(curl) {
    curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
    sample url.
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
    success = curl_easy_perform(curl);
    curl_easy_cleanup(curl);
    }
    return 0;
    }

    Thanks in advance for your help.
     
    Shutdownrunner, Sep 5, 2005
    #1
    1. Advertising

  2. On Mon, 05 Sep 2005 21:18:29 +0200, Shutdownrunner
    <"shutdownrunner[NOSPAM]"@o2.pl> wrote:

    >I want to store result of curl in a variable, which means to store a
    >webpage in a variable in order to parse it later and get our some useful
    >information. But unfortunately I'm not too experienced in C and I'm
    >making some stupid mistake. Could someone help me solve it?
    >#include <string.h>
    >#include <curl/curl.h>


    And we know what is in this header because ...

    >
    >
    >size_t write_data(const char *buffer, size_t size, size_t nmemb, char
    >*userp)
    >{
    > char *string = userp;
    > size_t len;
    > len = size * nmemb;
    > strncat(string, buffer,len); // I get segfault here. Why????


    write_data is called only through curl_easy_setopt. We have no idea
    no idea how that is accomplished. What do buffer and userp point to?

    > return len;
    >}
    >
    >
    >
    >int main(int argc, char **argv)
    > {
    > char *tablica="";
    > CURL *curl;
    > CURLcode success;
    > curl = curl_easy_init();
    > if(curl) {
    > curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
    >sample url.


    curl_easy_setopt is a function taking three arguments. The third one
    has a type consistent with pointer to char. It could be pointer to
    void or const qualified.

    > curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);


    Here you call curl_easy_setopt and the third argument is a function
    pointer. This is completely inconsistent unless curl_easy_setopt is a
    variadic function. Is it?

    > curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
    > success = curl_easy_perform(curl);
    > curl_easy_cleanup(curl);
    > }
    > return 0;
    >}
    >
    >Thanks in advance for your help.



    <<Remove the del for email>>
     
    Barry Schwarz, Sep 5, 2005
    #2
    1. Advertising

  3. Barry Schwarz napisa³(a):
    >> curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

    >
    >
    > Here you call curl_easy_setopt and the third argument is a function
    > pointer. This is completely inconsistent unless curl_easy_setopt is a
    > variadic function. Is it?

    I don't know if I understand you correctly, because I haven't heard
    about variadic functions before. There's a paragraph in libcurl which says:
    "Let's assume for a while that you want to receive data as the URL
    identifies a remote resource you want to get here. Since you write a
    sort of application that needs this transfer, I assume that you would
    like to get the data passed to you directly instead of simply getting it
    passed to stdout. So, you write your own function that matches this
    prototype:

    size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp);

    You tell libcurl to pass all data to this function by issuing a function
    similar to this:

    curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, write_data); "

    Userp can be FILE, but I wanted it to be a variable.

    >> curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);


    I'm passing tablica to write_data, because I want all that is downloaded
    by curl to be stored in this variable. I still have a lot to read about
    local and global variables, so it might be not that necessary as I
    think. Please tell me if there's any better solution to do what I want
    to do.

    I'm not getting any errors anymore. Unfortunately I only found a
    temporary solution.
    char tablica[100000];
    tablica[0] = '\0';
    I'm so used to PHP and not being forced to worry about memory
    management:( Is it possible to change variables size at runtime? Because
    I never know how big a webpage will be. E.g. I declare an empty
    variable, read the buffer, see that it is currently e.g. 1000bytes long,
    so I make my variable 1000bytes long, then I read buffer again and I see
    than now it's 2000bytes long, so I add another 2000 bytes to my variable
    (to make it 3000 bytes long), and so on and so forth. How do I do that?

    Best regards,
    Jacek
     
    Shutdownrunner, Sep 5, 2005
    #3
  4. Shutdownrunner

    elzacho Guest


    > I'm not getting any errors anymore. Unfortunately I only found a
    > temporary solution.
    > char tablica[100000];
    > tablica[0] = '\0';
    > I'm so used to PHP and not being forced to worry about memory
    > management:( Is it possible to change variables size at runtime? Because
    > I never know how big a webpage will be. E.g. I declare an empty
    > variable, read the buffer, see that it is currently e.g. 1000bytes long,
    > so I make my variable 1000bytes long, then I read buffer again and I see
    > than now it's 2000bytes long, so I add another 2000 bytes to my variable
    > (to make it 3000 bytes long), and so on and so forth. How do I do that?


    So this is not too hard to do if you can tell how big the buffer needs
    to be before you copy it to tablica. You can manage memory on the fly
    by using malloc calls like so...

    char *tablica;
    ....
    tablica = (char *) malloc(buffersize * sizeof(char));
    ....
    /* To make the buffer bigger: */
    /* Free old memory */
    free(tablica);
    /* Allocate new memory */
    tablica = (char *) malloc(newbuffersize * sizeof(char));

    Ofcourse this loses all the stored values in tablica. To preserve this
    data you would need to create a new buffer pointed to by a temp
    varialbe and copy the memory over, then free the old buffer and have
    tablica point to the new buffer.

    Zach
     
    elzacho, Sep 5, 2005
    #4
  5. On 5 Sep 2005 15:16:48 -0700, "elzacho" <> wrote:

    >
    >> I'm not getting any errors anymore. Unfortunately I only found a
    >> temporary solution.
    >> char tablica[100000];
    >> tablica[0] = '\0';
    >> I'm so used to PHP and not being forced to worry about memory
    >> management:( Is it possible to change variables size at runtime? Because
    >> I never know how big a webpage will be. E.g. I declare an empty
    >> variable, read the buffer, see that it is currently e.g. 1000bytes long,
    >> so I make my variable 1000bytes long, then I read buffer again and I see
    >> than now it's 2000bytes long, so I add another 2000 bytes to my variable
    >> (to make it 3000 bytes long), and so on and so forth. How do I do that?

    >
    >So this is not too hard to do if you can tell how big the buffer needs
    >to be before you copy it to tablica. You can manage memory on the fly
    >by using malloc calls like so...
    >
    >char *tablica;
    >...
    >tablica = (char *) malloc(buffersize * sizeof(char));


    Casting the return from malloc only serves to hide real errors.

    >...
    >/* To make the buffer bigger: */
    >/* Free old memory */
    >free(tablica);
    >/* Allocate new memory */
    >tablica = (char *) malloc(newbuffersize * sizeof(char));
    >
    >Ofcourse this loses all the stored values in tablica. To preserve this
    >data you would need to create a new buffer pointed to by a temp
    >varialbe and copy the memory over, then free the old buffer and have
    >tablica point to the new buffer.


    Consider the realloc() function.



    <<Remove the del for email>>
     
    Barry Schwarz, Sep 6, 2005
    #5
  6. On Mon, 05 Sep 2005 23:41:28 +0200, Shutdownrunner
    <"shutdownrunner[NOSPAM]"@o2.pl> wrote:

    >Barry Schwarz napisa³(a):
    >>> curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

    >>
    >>
    >> Here you call curl_easy_setopt and the third argument is a function
    >> pointer. This is completely inconsistent unless curl_easy_setopt is a
    >> variadic function. Is it?

    >I don't know if I understand you correctly, because I haven't heard
    >about variadic functions before. There's a paragraph in libcurl which says:
    >"Let's assume for a while that you want to receive data as the URL
    >identifies a remote resource you want to get here. Since you write a
    >sort of application that needs this transfer, I assume that you would
    >like to get the data passed to you directly instead of simply getting it
    >passed to stdout. So, you write your own function that matches this
    >prototype:
    >

    snip

    You need to discuss this in a newsgroup where libcurl is on-topic.


    <<Remove the del for email>>
     
    Barry Schwarz, Sep 6, 2005
    #6
  7. "elzacho" <> writes:
    >> I'm not getting any errors anymore. Unfortunately I only found a
    >> temporary solution.
    >> char tablica[100000];
    >> tablica[0] = '\0';
    >> I'm so used to PHP and not being forced to worry about memory
    >> management:( Is it possible to change variables size at runtime? Because
    >> I never know how big a webpage will be. E.g. I declare an empty
    >> variable, read the buffer, see that it is currently e.g. 1000bytes long,
    >> so I make my variable 1000bytes long, then I read buffer again and I see
    >> than now it's 2000bytes long, so I add another 2000 bytes to my variable
    >> (to make it 3000 bytes long), and so on and so forth. How do I do that?

    >
    > So this is not too hard to do if you can tell how big the buffer needs
    > to be before you copy it to tablica. You can manage memory on the fly
    > by using malloc calls like so...
    >
    > char *tablica;
    > ...
    > tablica = (char *) malloc(buffersize * sizeof(char));


    Better:

    tablica = malloc(buffersize);

    > /* To make the buffer bigger: */
    > /* Free old memory */
    > free(tablica);
    > /* Allocate new memory */
    > tablica = (char *) malloc(newbuffersize * sizeof(char));
    >
    > Ofcourse this loses all the stored values in tablica. To preserve this
    > data you would need to create a new buffer pointed to by a temp
    > varialbe and copy the memory over, then free the old buffer and have
    > tablica point to the new buffer.


    Use realloc() for this.

    One thing to watch out for is that realloc(), like malloc() can fail.
    If you want to be able to recover from such an error, you need to
    assign the result of realloc() to a separate variable; otherwise you
    could lose your original data.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Sep 6, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris

    cURL in ASP.NET

    Chris, Dec 17, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    8,255
    Joerg Jooss
    Dec 19, 2004
  2. ß Ø ® G

    Page Curl Javascript wtd

    ß Ø ® G, Jan 26, 2004, in forum: HTML
    Replies:
    10
    Views:
    6,805
    supermann
    Jun 7, 2007
  3. curl and popen2

    , Feb 2, 2005, in forum: Python
    Replies:
    1
    Views:
    612
    Robey Holderith
    Feb 2, 2005
  4. post data using curl

    , Feb 12, 2007, in forum: Python
    Replies:
    0
    Views:
    392
  5. Replies:
    1
    Views:
    387
    =?ISO-8859-1?Q?Stefan_N=E4we?=
    Nov 1, 2005
Loading...

Share This Page