What am I doing wrong? curl

[Shutdownrunner]

I want to store result of curl in a variable, which means to store a
webpage in a variable in order to parse it later and get our some useful
information. But unfortunately I'm not too experienced in C and I'm
making some stupid mistake. Could someone help me solve it?
#include <string.h>
#include <curl/curl.h>


size_t write_data(const char *buffer, size_t size, size_t nmemb, char
*userp)
{
char *string = userp;
size_t len;
len = size * nmemb;
strncat(string, buffer,len); // I get segfault here. Why????
return len;
}



int main(int argc, char **argv)
{
char *tablica="";
CURL *curl;
CURLcode success;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
sample url.
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
success = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return 0;
}

Thanks in advance for your help.

 

[Barry Schwarz]

I want to store result of curl in a variable, which means to store a
webpage in a variable in order to parse it later and get our some useful
information. But unfortunately I'm not too experienced in C and I'm
making some stupid mistake. Could someone help me solve it?
#include <string.h>
#include <curl/curl.h>

And we know what is in this header because ...

size_t write_data(const char *buffer, size_t size, size_t nmemb, char
*userp)
{
char *string = userp;
size_t len;
len = size * nmemb;
strncat(string, buffer,len); // I get segfault here. Why????

write_data is called only through curl_easy_setopt. We have no idea
no idea how that is accomplished. What do buffer and userp point to?

return len;
}



int main(int argc, char **argv)
{
char *tablica="";
CURL *curl;
CURLcode success;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
sample url.

curl_easy_setopt is a function taking three arguments. The third one
has a type consistent with pointer to char. It could be pointer to
void or const qualified.

curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

Here you call curl_easy_setopt and the third argument is a function
pointer. This is completely inconsistent unless curl_easy_setopt is a
variadic function. Is it?

curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
success = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return 0;
}

Thanks in advance for your help.

<<Remove the del for email>>

 

[Shutdownrunner]

Barry Schwarz napisa³(a):

Here you call curl_easy_setopt and the third argument is a function
pointer. This is completely inconsistent unless curl_easy_setopt is a
variadic function. Is it?

I don't know if I understand you correctly, because I haven't heard
about variadic functions before. There's a paragraph in libcurl which says:
"Let's assume for a while that you want to receive data as the URL
identifies a remote resource you want to get here. Since you write a
sort of application that needs this transfer, I assume that you would
like to get the data passed to you directly instead of simply getting it
passed to stdout. So, you write your own function that matches this
prototype:

size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp);

You tell libcurl to pass all data to this function by issuing a function
similar to this:

curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, write_data); "

Userp can be FILE, but I wanted it to be a variable.

I'm passing tablica to write_data, because I want all that is downloaded
by curl to be stored in this variable. I still have a lot to read about
local and global variables, so it might be not that necessary as I
think. Please tell me if there's any better solution to do what I want
to do.

I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?

Best regards,
Jacek

 

[elzacho]

I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?

So this is not too hard to do if you can tell how big the buffer needs
to be before you copy it to tablica. You can manage memory on the fly
by using malloc calls like so...

char *tablica;
....
tablica = (char *) malloc(buffersize * sizeof(char));
....
/* To make the buffer bigger: */
/* Free old memory */
free(tablica);
/* Allocate new memory */
tablica = (char *) malloc(newbuffersize * sizeof(char));

Ofcourse this loses all the stored values in tablica. To preserve this
data you would need to create a new buffer pointed to by a temp
varialbe and copy the memory over, then free the old buffer and have
tablica point to the new buffer.

Zach

 

[Barry Schwarz]

I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?

So this is not too hard to do if you can tell how big the buffer needs
to be before you copy it to tablica. You can manage memory on the fly
by using malloc calls like so...

char *tablica;
...
tablica = (char *) malloc(buffersize * sizeof(char));

Casting the return from malloc only serves to hide real errors.

...
/* To make the buffer bigger: */
/* Free old memory */
free(tablica);
/* Allocate new memory */
tablica = (char *) malloc(newbuffersize * sizeof(char));

Ofcourse this loses all the stored values in tablica. To preserve this
data you would need to create a new buffer pointed to by a temp
varialbe and copy the memory over, then free the old buffer and have
tablica point to the new buffer.

Consider the realloc() function.



<<Remove the del for email>>

 

[Barry Schwarz]

Barry Schwarz napisa³(a):
I don't know if I understand you correctly, because I haven't heard
about variadic functions before. There's a paragraph in libcurl which says:
"Let's assume for a while that you want to receive data as the URL
identifies a remote resource you want to get here. Since you write a
sort of application that needs this transfer, I assume that you would
like to get the data passed to you directly instead of simply getting it
passed to stdout. So, you write your own function that matches this
prototype:

snip

You need to discuss this in a newsgroup where libcurl is on-topic.


<<Remove the del for email>>

 

[Keith Thompson]

I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?

So this is not too hard to do if you can tell how big the buffer needs
to be before you copy it to tablica. You can manage memory on the fly
by using malloc calls like so...

char *tablica;
...
tablica = (char *) malloc(buffersize * sizeof(char));

Better:

tablica = malloc(buffersize);

/* To make the buffer bigger: */
/* Free old memory */
free(tablica);
/* Allocate new memory */
tablica = (char *) malloc(newbuffersize * sizeof(char));

Ofcourse this loses all the stored values in tablica. To preserve this
data you would need to create a new buffer pointed to by a temp
varialbe and copy the memory over, then free the old buffer and have
tablica point to the new buffer.

Use realloc() for this.

One thing to watch out for is that realloc(), like malloc() can fail.
If you want to be able to recover from such an error, you need to
assign the result of realloc() to a separate variable; otherwise you
could lose your original data.