Memory errors HELP!

[electrixnow]

I have the following test code that opens and reads a file.
The file is over 19000 lines long. The file contains two fields
on each line: "text,text\n"

I have placed a test that for loops up to 182 and the program errors
out.
I placed the error message after the code:

I am using VC++ Express Edition

#include <fstream>
#include <iostream>
#include <string>
#include <stdio.h>
#define MOVEX_QUERY "C:\\DWG_DATA\\DOCUMENTS_movex.csv"

int main(){
char * document;
char * edition;
char * str1;
char * next_token1;
int i=0;

using std::string;
using std::ifstream;
using std::cout;

ifstream inf(MOVEX_QUERY);

if (inf)
{
char namn[20000][30];
while ((inf.getline(namn, 30)) != NULL)++i;
for (int i = 0; i < 182; ++i){
str1 = namn;
cout << str1 << '\n';

document = strtok_s( str1, " ,\t\n", &next_token1);
edition = strtok_s( NULL, " ,\t\n", &next_token1);
std::string doc(document);
std::string edi(edition);
if ( edi == doc ) cout << "THIS WORKS" <<'\n';
printf( "%s\n", document );
printf( "%s\n", edition );
cout << i <<'\n';
}

}
else
{
cout << "Could not open file\n";
return 1;
}
cout << "PROCESSING COMPLETE\n";
return 0;
}


This is just test code, there are things that could be removed.
If I change the for loop to 182 or greater
I get a message as follows:


Unhandled Exception: System.AccessViolationException: Attempted to read
or write
protected memory. This is often an indication that other memory is
corrupt.

.{ctor}

(basic_string<char\,std::char_traits<char>\,std::allocator<char> >* ,
SByte* )
at main() in c:\documents and settings\grant\my documents\visual
studio 2005\
projects\documents\document1\document1\doc1.cpp:line 31
Press any key to continue . . .

Attempted to read or write protected memory

Please let me know what you think

Thanks

 

[int2str]

I have the following test code that opens and reads a file.
The file is over 19000 lines long. The file contains two fields
on each line: "text,text\n"

I have placed a test that for loops up to 182 and the program errors
out.
I placed the error message after the code:

I am using VC++ Express Edition

[snipped code]

Hard to tell for sure, but it looks like a memory error. You're trying
to allocate 20000 strings of 30 character length on the stack. That's
stretching it for sure.

The code you posted mixes a lot of "C style" with C++ functions and
classes. Why not go all the way and make your code look a little more
friendly?

Like so:
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
#include <vector>

using namespace std;

struct Doc
{
string name;
string edition;
};

int main()
{
ifstream ff( "test.dat" );
string line;

vector<Doc> documents;
documents.reserve( 10000 );

while( getline( ff, line ))
{
string::size_type comma_pos = line.find( ',' );
if ( string::npos == comma_pos )
continue;

Doc doc;
doc.name = line.substr( 0, comma_pos );
doc.edition = line.substr( comma_pos + 1 );

cout << doc.name << " " << doc.edition << endl;

documents.push_back( doc );
}
}

You can then process the documents vector in any way you like.

Cheers,
Andre

 

[Daniel T.]

I have the following test code that opens and reads a file.
The file is over 19000 lines long. The file contains two fields
on each line: "text,text\n"

I must say, you have a rather awkward mix of C and C++ code. I think you
would be much better off if you removed all the char*s

I have placed a test that for loops up to 182 and the program errors
out.
I placed the error message after the code:

I just ran it out to 1750 lines with no crash... I suspect that your
code is only reading in 182 lines. :-/

I am using VC++ Express Edition

#include <fstream>
#include <iostream>
#include <string>
#include <stdio.h>
#define MOVEX_QUERY "C:\\DWG_DATA\\DOCUMENTS_movex.csv"

int main(){
char * document;
char * edition;
char * str1;
char * next_token1;
int i=0;

Did you know you define two different variables both with the name 'i'?
(Here and in the for loop below.) Not a good idea and maybe the source
of your problem.

Change the line above to "int lines = 0;"

using std::string;
using std::ifstream;
using std::cout;

ifstream inf(MOVEX_QUERY);

if (inf)
{
char namn[20000][30];
while ((inf.getline(namn, 30)) != NULL)++i;

The above 'i's will have to be changed to 'lines' and a guard would
probably be a good idea.

while (lines < 20000 && inf.getline(namn[lines], 30) != NULL)
++lines;

I suspect that if you put:

cout << lines << '\n';

You will find that it outputs 182.

for (int i = 0; i < 182; ++i){

Change the '182' above to 'lines'

str1 = namn;
cout << str1 << '\n';

document = strtok_s( str1, " ,\t\n", &next_token1);
edition = strtok_s( NULL, " ,\t\n", &next_token1);
std::string doc(document);
std::string edi(edition);
if ( edi == doc ) cout << "THIS WORKS" <<'\n';
printf( "%s\n", document );
printf( "%s\n", edition );
cout << i <<'\n';
}

}
else
{
cout << "Could not open file\n";
return 1;
}
cout << "PROCESSING COMPLETE\n";
return 0;
}


This is just test code, there are things that could be removed.
If I change the for loop to 182 or greater
I get a message as follows:


Unhandled Exception: System.AccessViolationException: Attempted to read
or write
protected memory. This is often an indication that other memory is
corrupt.

.{ctor}

(basic_string<char\,std::char_traits<char>\,std::allocator<char> >* ,
SByte* )
at main() in c:\documents and settings\grant\my documents\visual
studio 2005\
projects\documents\document1\document1\doc1.cpp:line 31
Press any key to continue . . .

Attempted to read or write protected memory

Please let me know what you think

Thanks

 

[Daniel T.]

I have the following test code that opens and reads a file.
The file is over 19000 lines long. The file contains two fields
on each line: "text,text\n"

I have placed a test that for loops up to 182 and the program errors
out.
I placed the error message after the code:

I am using VC++ Express Edition

[snipped code]

Hard to tell for sure, but it looks like a memory error. You're trying
to allocate 20000 strings of 30 character length on the stack. That's
stretching it for sure.

The code you posted mixes a lot of "C style" with C++ functions and
classes. Why not go all the way and make your code look a little more
friendly?

Like so:
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
#include <vector>

using namespace std;

struct Doc
{
string name;
string edition;
};

int main()
{
ifstream ff( "test.dat" );
string line;

vector<Doc> documents;
documents.reserve( 10000 );

while( getline( ff, line ))
{
string::size_type comma_pos = line.find( ',' );
if ( string::npos == comma_pos )
continue;

Doc doc;
doc.name = line.substr( 0, comma_pos );
doc.edition = line.substr( comma_pos + 1 );

cout << doc.name << " " << doc.edition << endl;

documents.push_back( doc );
}
}

You can then process the documents vector in any way you like.

Or better yet...

int main( )
{
ifstream inf( MOVEX_QUERY );
vector<Document> documents;
Document doc;
while ( getline( inf, doc.name, ',' ) && getline( inf, doc.edition ) )
documents.push_back( doc );
// do what you will to the documents...
}

From what I think I know about the OP's problem to solve, a map may make
more sense...

int main( )
{
ifstream inf( MOVEX_QUERY );
map<string, string> documents;
string doc;
string edi;
while ( getline( inf, doc, ',' ) && getline( inf, edi ) ) {
if ( documents.count( doc ) == 0 )
documents.insert( make_pair( doc, edi ) );
else
cout << "duplicate entry found\n"; // what to do here?
}

The above will make it easer to compare the edition of a document from
this file, to the edition of the same document of the other file.

 

[electrixnow]

I have changed the code and it still blows up at 182 lines
I found that if I comment out the two string lines noted that it runs
to 19000 + lines. (e-mail address removed) suggested I was pushing it with
the array size but it works fine. I the array is full with no error
after the
while statement.

Any clue?

and I am mixing C and C++ because have just started using C++
with VC++ Express, it was free, and I have not found a good online
C++ reference. I use to program in C type shell languages in the 80's.
So I need a good online or electronic VC++ reference library. or
hardback
would work, any suggestions are welcome. Thanks for the help thus far.

#include <fstream>
#include <iostream>
#include <string>

#define MOVEX_QUERY "C:\\DWG_DATA\\DOCUMENTS_movex.csv" //

int main(){
char * document;
char * edition;
char * str1;
char * next_token1;
int lines=0;

using namespace std;

ifstream inf(MOVEX_QUERY);

if (inf)
{
char namn[20000][30];
while (lines < 20000 && (inf.getline(namn[lines], 30)) !=
NULL)++lines;
for (int i = 0; i < lines; ++i){
str1 = namn;
cout << str1 << '\n';

document = strtok_s( str1, " ,\t\n", &next_token1);
edition = strtok_s( NULL, " ,\t\n", &next_token1);
// string doc(document); // if these two lines are removed
// string edi(edition); // it will run to 19000 + lines
printf( "%s\n", document );
printf( "%s\n", edition );
cout << i <<'\n';
}

}
else
{
cout << "Could not open file\n";
return 1;
}
cout << "PROCESSING COMPLETE\n";
return 0;
}

Here is a sample of the data being input.

00-00008RA,0004,,,,,,
00-00009RA,0003,,,,,,
00-00010RA,0003,,,,,,
00-00019RA,0005,,,,,,
00-00021RA,0016,,,,,,
00-00036RA,0016,,,,,,
00-00041RA,0005,,,,,,
00-00042RA,0009,,,,,,
00-00043RA,0003,,,,,,
00-00045RA,0008,,,,,,
00-00047RA,0018,,,,,,
00-00051RA,0018,,,,,,
00-00052RA,0029,,,,,,
00-00054RA,0028,,,,,,
00-00055RA,0023,,,,,,
00-00056RA,0006,,,,,,
00-00057RA,0008,,,,,,
00-00058RA,0005,,,,,,
00-00059RA,0005,,,,,,
00-00076RA,0006,,,,,,
00-00085RA,0025,,,,,,
00-00086RA,0025,,,,,,

 

[Jay_Nabonne]

I have the following test code that opens and reads a file.
The file is over 19000 lines long. The file contains two fields
on each line: "text,text\n"

I have placed a test that for loops up to 182 and the program errors
out.

What is on line 182 and 183 of the text file?

I placed the error message after the code:

I am using VC++ Express Edition

#include <fstream>
#include <iostream>
#include <string>
#include <stdio.h>
#define MOVEX_QUERY "C:\\DWG_DATA\\DOCUMENTS_movex.csv"

int main(){
char * document;
char * edition;
char * str1;
char * next_token1;
int i=0;

using std::string;
using std::ifstream;
using std::cout;

ifstream inf(MOVEX_QUERY);

if (inf)
{
char namn[20000][30];

Could any line be longer than 30 characters?

while ((inf.getline(namn, 30)) != NULL)++i;
for (int i = 0; i < 182; ++i){
str1 = namn;
cout << str1 << '\n';

document = strtok_s( str1, " ,\t\n", &next_token1);
edition = strtok_s( NULL, " ,\t\n", &next_token1);
std::string doc(document);
std::string edi(edition);

What happens here if the line doesn't have a comma?
"document" or "edition" would be null, and initializing a string from that
will die the way you described.

if ( edi == doc ) cout << "THIS WORKS" <<'\n';
printf( "%s\n", document );
printf( "%s\n", edition );
cout << i <<'\n';
}

}
else
{
cout << "Could not open file\n";
return 1;
}
cout << "PROCESSING COMPLETE\n";
return 0;
}

Print out each line as you process it. See what the offending line looks
like.

- Jay

 

[Daniel T.]

I have changed the code and it still blows up at 182 lines
I found that if I comment out the two string lines noted that it runs
to 19000 + lines. (e-mail address removed) suggested I was pushing it with
the array size but it works fine. I the array is full with no error
after the
while statement.

Any clue?

and I am mixing C and C++ because have just started using C++
with VC++ Express, it was free, and I have not found a good online
C++ reference. I use to program in C type shell languages in the 80's.
So I need a good online or electronic VC++ reference library. or
hardback
would work, any suggestions are welcome. Thanks for the help thus far.

#include <fstream>
#include <iostream>
#include <string>

#define MOVEX_QUERY "C:\\DWG_DATA\\DOCUMENTS_movex.csv" //

int main(){
char * document;
char * edition;
char * str1;
char * next_token1;
int lines=0;

using namespace std;

ifstream inf(MOVEX_QUERY);

if (inf)
{
char namn[20000][30];
while (lines < 20000 && (inf.getline(namn[lines], 30)) !=
NULL)++lines;
for (int i = 0; i < lines; ++i){
str1 = namn;
cout << str1 << '\n';

document = strtok_s( str1, " ,\t\n", &next_token1);
edition = strtok_s( NULL, " ,\t\n", &next_token1);

I think the reason it crashes is because at line 182 in the file,
'document' is NULL. To prove it do this:

string doc;
string edi;
if ( document && edition ) {
doc = document;
edi = edition;
}
else { cout << "Would have crashed!\n"; }

 

[int2str]

I have changed the code and it still blows up at 182 lines
I found that if I comment out the two string lines noted that it runs
to 19000 + lines. (e-mail address removed) suggested I was pushing it with
the array size but it works fine. I the array is full with no error
after the
while statement.

Any clue?

Still unrecoverable code .
Using Daniels suggestions and your update on the format, here is a
nicer version:

#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
#include <map>

using namespace std;

int main()
{
ifstream ff( "test.dat" );
string document, edition, rest;

map<string,string> documents;

while( getline( ff, document, ',' )
&& getline( ff, edition, ',' )
&& getline( ff, rest ))
{
cout << document << " " << edition << endl;
documents.insert(make_pair(document, edition ));
}
}

 

[electrixnow]

I found the problem. Down in the file at the line it blows up at looks
like this.

083256,,,,,,,

I made that line up, but it has no data after the delimiter comma.

also I found that if I put at least 2 new lines in after the last data
line
it also blows up.

I am thinking of a way to fix.

Here is my real program I am working on, disreguard the old one.
-----------

#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <ostream>


#define MOVEX_QUERY "C:\\DWG_DATA\\DOCUMENTS_MOVEX.dat"
#define CONTROL_LIST "C:\\DWG_DATA\\DOCUMENTS_LIST.txt"
#define REQUEST_LIST "C:\\DWG_DATA\\REQUEST_LIST.txt"

using namespace std;

typedef vector<string> stringvec;

void upper_case(string in, stringvec &out) {
// Erase all entries from the vector
out.swap(stringvec());
int len = in.length();
for (int a=0; a<len; ++a)
in[a] = toupper(in[a]);
out.push_back(in);
}



int main()
{
char * document;
char * edition;
char input_line[31];
char * next_token1;
int i=0;
ifstream inf(MOVEX_QUERY);
ifstream inf2(CONTROL_LIST);
ofstream outf(REQUEST_LIST);
if (inf)
{
while ((inf.getline(input_line,30)) != NULL)
{
//while (! inf.eof()) // eof did'nt work
//inf.getline(input_line,30); // don't know why
document = strtok_s( input_line, " ,\t\n", &next_token1);
edition = strtok_s( NULL, " ,\t\n", &next_token1);
stringvec doc,edi;
upper_case(document,doc);
upper_case(edition,edi);
cout << "vector document " << doc[0] <<'\n'; // for test
cout << "vector edition " << edi[0] <<'\n'; // for test
cout << i <<'\n'; // for test


if ( doc[0] == edi[0] )
{

if (! outf)
{
cout << "File " << REQUEST_LIST;
cout << " could not be opened.";
return -1;
}
outf << doc[0]<<","<< edi[0]<< endl;
}
++i;
}
}
else
{
cout << "File " << MOVEX_QUERY;
cout << " could not be opened.";
return 1;
}
outf.close();
cout << "PROCESSING COMPLETE\n";
return 0;
}
-----------
Here is a short data test file I made. I won't work on the last line.
526104A, 526104a
526105-3, C
526106-ir ,D
526107-ap,,

 

[electrixnow]

Gotter FIXED
check for empty lines and missing data first!

THANKS FOR THE HELP

 

[Daniel T.]

I found the problem. Down in the file at the line it blows up at looks
like this.

083256,,,,,,,

I made that line up, but it has no data after the delimiter comma.

also I found that if I put at least 2 new lines in after the last data
line
it also blows up.

I am thinking of a way to fix.

I gave you a way to fix, test if document and edition is NULL before
trying to load them in a string.

Here is my real program I am working on, disreguard the old one.
-----------

#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <ostream>


#define MOVEX_QUERY "C:\\DWG_DATA\\DOCUMENTS_MOVEX.dat"
#define CONTROL_LIST "C:\\DWG_DATA\\DOCUMENTS_LIST.txt"
#define REQUEST_LIST "C:\\DWG_DATA\\REQUEST_LIST.txt"

using namespace std;

typedef vector<string> stringvec;

void upper_case(string in, stringvec &out) {
// Erase all entries from the vector
out.swap(stringvec());

The above shouldn't work, swap takes a non-const reference, and you are
sending it a temporary. Why not just out.clear()?

int len = in.length();
for (int a=0; a<len; ++a)
in[a] = toupper(in[a]);
out.push_back(in);
}



int main()
{
char * document;
char * edition;
char input_line[31];
char * next_token1;

What is the above for?

int i=0;
ifstream inf(MOVEX_QUERY);
ifstream inf2(CONTROL_LIST);
ofstream outf(REQUEST_LIST);
if (inf)
{
while ((inf.getline(input_line,30)) != NULL)

Why the extra parens?

{
//while (! inf.eof()) // eof did'nt work
//inf.getline(input_line,30); // don't know why
document = strtok_s( input_line, " ,\t\n", &next_token1);
edition = strtok_s( NULL, " ,\t\n", &next_token1);
stringvec doc,edi;

Don't use a stringvec here. Just change your upper_case function to
output a string or modify the cstring in place.

upper_case(document,doc);
upper_case(edition,edi);
cout << "vector document " << doc[0] <<'\n'; // for test
cout << "vector edition " << edi[0] <<'\n'; // for test
cout << i <<'\n'; // for test


if ( doc[0] == edi[0] )
{

if (! outf)
{
cout << "File " << REQUEST_LIST;
cout << " could not be opened.";
return -1;
}
outf << doc[0]<<","<< edi[0]<< endl;
}
++i;
}
}
else
{
cout << "File " << MOVEX_QUERY;
cout << " could not be opened.";
return 1;
}
outf.close();
cout << "PROCESSING COMPLETE\n";
return 0;
}
-----------
Here is a short data test file I made. I won't work on the last line.
526104A, 526104a
526105-3, C
526106-ir ,D
526107-ap,,

Below is my version of your program. Look it over carefully, and ask
questions about *any* line you don't understand.

#include <string>
#include <iostream>
#include <fstream>

using namespace std;

const char* const movex_query = "movex.dat";
const char* const request_list = "request_list.txt";

// strips whitespace off front and back of string,
// and converts string to uppercase.
void clean( string& str )
{
const char* const whitespace = " \t";
str.erase( 0, str.find_first_not_of( whitespace ) );
// strips whitespace off front

// strip whitespace off back
string::size_type pos = str.find_last_not_of( whitespace );
if ( pos != string::npos )
str.erase( pos + 1 );

// convert to uppercase
for ( int i = 0; i < str.length(); ++i )
str = toupper( static_cast<unsigned char>( str ) );
}

int main()
{
ifstream inf( movex_query );
ofstream out( request_list );
if ( inf && out ) {
int lines_processed = 0;
string document;
string edition;
// the below will break if there is a line with no commas,
// should I guard against that?
while ( getline( inf, document, ',' ) && getline( inf, edition ) )
{
clean( document );
try { edition.erase( edition.find( ',' ) ); } catch (...) { }
// above is for lines with extra commas: strips garbage
clean( edition );
if ( document == "" || edition == "" )
cout << "Bad record: " << document << ", "
<< edition << '\n';
else if ( document == edition ) {
out << document << ", " << edition << '\n';
}
++lines_processed;
}
cout << "Total lines processed: " << lines_processed
<< "\nProcessing complete!\n";
}
else {
// below is probably more fancy than necessary,
// but the output looks good.
cout << "Could not open file" << ( ( !inf && !out ) ? "s" : "" )
<< ": ";
if ( !inf )
cout << movex_query << ( ( !out ) ? ", and " : "" );
if ( !out )
cout << request_list;
cout << '\n';
}
}