Fetching binary file linked in URL

[David Jacques]

I have to write a console application which will fetch a zip on a daily
basis.
This zip file is hotlinked in a web page. The name of this .zip file will
change every day. I've been able to start the session, open the URL,
read each line, and get the line with the .zip file URL in it no problem.
Where I am running intro trouble is writing the zip file to disk as binary.
I'm following the example from the VC++ Online Help reference to
CFile::Open(). Can anyone see what I am doing wrong ?

thanks in advance.

#include <afx.h>
#include <afxwin.h>
#include <afxinet.h>
#include <iostream>
#include <stdlib.h>

int main(int argc, _TCHAR* argv[])
{
CStdioFile* pRemoteURLFile;
CStdioFile* pRemoteZipFile;
CFile* pLocalZipFile;
CInternetSession pSessionURL("URLSession");
CInternetSession pSessionZip("ZipSession");
char* ziptag = "shapefile";
CString sURL;
CString sZipURL;
CString sReadLine;
char* pFileName = "test.dat";
int iFound;
int iNumLines;
UINT nBytesRead;
char pbuf[100];
CFileException cFileErr;
pRemoteURLFile = NULL;
pRemoteZipFile = NULL;
pLocalZipFile = NULL;
iNumLines = nBytesRead = 0;
sURL = "http://activefiremaps.fs.fed.us/fireptdata/modisfire_2004.htm";
pRemoteURLFile =
pSessionURL.OpenURL(sURL,1,INTERNET_FLAG_TRANSFER_BINARY,NULL,NULL);
while (pRemoteURLFile->ReadString(sReadLine))
{
iNumLines ++;
iFound = sReadLine.Find("shapefile");
if (iFound > 0)
{
sZipURL = getZIPURL(sReadLine,ziptag,sURL);
}
}
delete pRemoteURLFile;
pSessionURL.Close;

pSessionZip.OpenURL(sZipURL,1,INTERNET_FLAG_TRANSFER_BINARY,NULL,NULL);
pRemoteZipFile =
pSessionZip.OpenURL(sZipURL,1,INTERNET_FLAG_TRANSFER_BINARY,NULL,NULL);

//** The below line fails **
if( pLocalZipFile->Open(pFileName, CFile::modeCreate | CFile::modeWrite |
CFile::typeBinary, &cFileErr ) )
{
while (pRemoteZipFile->Read(pbuf,100))
{
//count bytes read to see if size of file is correct
nBytesRead = nBytesRead + sizeof(pbuf);

//** write binary date to pLocalZipFile here **
}
printf("%d bytes read from remote file %s\n",nBytesRead,sZipURL);
delete pRemoteZipFile;
pSessionZip.Close;
return 0;
}
}

 

[Howard]

I have to write a console application which will fetch a zip on a daily
basis.
This zip file is hotlinked in a web page. The name of this .zip file will
change every day. I've been able to start the session, open the URL,
read each line, and get the line with the .zip file URL in it no problem.
Where I am running intro trouble is writing the zip file to disk as
binary.
I'm following the example from the VC++ Online Help reference to
CFile::Open(). Can anyone see what I am doing wrong ?

thanks in advance.

CFile appears to be an MFC class. As such, you need to ask in an mfc or
vc++ newsgroup for specifics on dealing with it. This group deals only with
standard C++ language issues, not platform or implementation-specific
issues.

-Howard

 

[Jim Howard]

What is coming back in cFileErr from
if( pLocalZipFile->Open(pFileName, CFile::modeCreate | CFile::modeWrite |
CFile::typeBinary, &cFileErr ) ) ?

Also, are your certain your app has write permission to pFileName?

Jim H

 

[David Jacques]

In the watch window after breaking it shows the following:

cFileErr {cause=0 OS Error=m_lOsError} CFileException

I've also tried to use a plain FILE* instead of a CSdtioFile for the
destination file to write locally using fopen(), fwrite() and fclose().
However this did produce a corrupt zip file of the incorrect size.
Somehow my file is larger than the zip if downloaded manually via IE.
Though I think this has to do with how I am reading the pRemoteZipFile.

What is coming back in cFileErr from
if( pLocalZipFile->Open(pFileName, CFile::modeCreate | CFile::modeWrite |
CFile::typeBinary, &cFileErr ) ) ?

Also, are your certain your app has write permission to pFileName?

Jim H

I have to write a console application which will fetch a zip on a daily
basis.
This zip file is hotlinked in a web page. The name of this .zip file will
change every day. I've been able to start the session, open the URL,
read each line, and get the line with the .zip file URL in it no problem.
Where I am running intro trouble is writing the zip file to disk as
binary.
I'm following the example from the VC++ Online Help reference to
CFile::Open(). Can anyone see what I am doing wrong ?

thanks in advance.

#include <afx.h>
#include <afxwin.h>
#include <afxinet.h>
#include <iostream>
#include <stdlib.h>

int main(int argc, _TCHAR* argv[])
{
CStdioFile* pRemoteURLFile;
CStdioFile* pRemoteZipFile;
CFile* pLocalZipFile;
CInternetSession pSessionURL("URLSession");
CInternetSession pSessionZip("ZipSession");
char* ziptag = "shapefile";
CString sURL;
CString sZipURL;
CString sReadLine;
char* pFileName = "test.dat";
int iFound;
int iNumLines;
UINT nBytesRead;
char pbuf[100];
CFileException cFileErr;
pRemoteURLFile = NULL;
pRemoteZipFile = NULL;
pLocalZipFile = NULL;
iNumLines = nBytesRead = 0;
sURL = "http://activefiremaps.fs.fed.us/fireptdata/modisfire_2004.htm";
pRemoteURLFile =
pSessionURL.OpenURL(sURL,1,INTERNET_FLAG_TRANSFER_BINARY,NULL,NULL);
while (pRemoteURLFile->ReadString(sReadLine))
{
iNumLines ++;
iFound = sReadLine.Find("shapefile");
if (iFound > 0)
{
sZipURL = getZIPURL(sReadLine,ziptag,sURL);
}
}
delete pRemoteURLFile;
pSessionURL.Close;

pSessionZip.OpenURL(sZipURL,1,INTERNET_FLAG_TRANSFER_BINARY,NULL,NULL);
pRemoteZipFile =
pSessionZip.OpenURL(sZipURL,1,INTERNET_FLAG_TRANSFER_BINARY,NULL,NULL);

//** The below line fails **
if( pLocalZipFile->Open(pFileName, CFile::modeCreate | CFile::modeWrite
|
CFile::typeBinary, &cFileErr ) )
{
while (pRemoteZipFile->Read(pbuf,100))
{
//count bytes read to see if size of file is correct
nBytesRead = nBytesRead + sizeof(pbuf);

//** write binary date to pLocalZipFile here **
}
printf("%d bytes read from remote file %s\n",nBytesRead,sZipURL);
delete pRemoteZipFile;
pSessionZip.Close;
return 0;
}
}

 

[Karl Heinz Buchegger]

I have to write a console application which will fetch a zip on a daily
basis.
This zip file is hotlinked in a web page. The name of this .zip file will
change every day. I've been able to start the session, open the URL,
read each line, and get the line with the .zip file URL in it no problem.
Where I am running intro trouble is writing the zip file to disk as binary.
I'm following the example from the VC++ Online Help reference to
CFile::Open(). Can anyone see what I am doing wrong ?

Why are you using so many pointers in your program?

CFile* pLocalZipFile;

This is a pointer to a CFile object

pLocalZipFile = NULL;

Here you set the pointer to 0.

//** The below line fails **
if( pLocalZipFile->Open(pFileName, CFile::modeCreate | CFile::modeWrite |
CFile::typeBinary, &cFileErr ) )

and here you program as if the pointer points to a CFile object, whih
of course is not the case.

WHy not simply

CFile LocalZipFile;

Now you have a CFile object

if( LocalZipFile.Open( pFileName, ....

and the CFile object will try to open the file.

Plain and simple:
Don't use pointers if you don't have to.

 

[David Jacques]

Thanks to all who responded.
I solved the problem.
I just used a FILE* to create and write to the local file.
(Along with a 1 time read and write of bytes).
Zip file is the same as if manually downloading with a right click of the
mouse.
Not the most elegant solution, but it works.

<<snipped>>

pRemoteZipFile =
pSessionZip.OpenURL(sZipURL,1,INTERNET_FLAG_TRANSFER_BINARY,NULL,NULL);
pFileName = getZipFileName(sZipURL);

if (( pLocalZipFile = fopen(pFileName,"wb")) != NULL)
{
//get # of bytes in whole file
dwActual = pRemoteZipFile->SeekToEnd();

//calculate size of buffer based on whole file size
pBuffer = (char*)calloc(sizeof(char),(dwActual));

//point file to beginning again to read
pRemoteZipFile->SeekToBegin();

//read whole file into this buffer once
lNumBytesRead = pRemoteZipFile->Read(pBuffer,dwActual);

//write the binary data locally all at once
fwrite(pBuffer,dwActual,1,pLocalZipFile);
}

fclose(pLocalZipFile);
delete pRemoteZipFile;
return 0;