Error with threads opening files

[lancer6238]

Hi all,

I'm using C and pthread on a Linux machine, and I'm having trouble
parallelizing a program.

I'm basically trying to take in a folder of data files, divide them
into groups, each group handled by a thread, and run a function on
each of the data file.

The way I'm doing this is I have a global char **filename variable,
where filename = filename of a data file. In the main function,
I'll read in the filenames of all the data files (minus "." and "..")
using scandir and put them in the filename variable. Then 4 (arbitrary
number) threads are created each calling the Process function. In
Process(), each thread only opens (using a FILE *fin declared in
Process()) and works on a portion of the data files using a
start_index and an end_index. For example, if there are 100 files,
then each thread will handle filename[0] to filename[24], filename[25]
to filename[49], filename[50] to filename[74] and filename[75] to
filename[99] respectively. After they're done, there is a pthread_join
in main() for all 4 threads.

I have checked that the filenames have been stored correctly, both in
main() and Process(). However, I keep getting segmentation fault here,
in Process():

for (i = start_index, i <= end_index ; i++)
fin = fopen(filename, "rb"); <--- Seg fault

I don't really get why there should be an error since none of the
threads are trying to open the same file.

Please advise.

Thank you.

Regards,
Rayne

 

[Ian Collins]

Hi all,

I'm using C and pthread on a Linux machine, and I'm having trouble
parallelizing a program.

I'm basically trying to take in a folder of data files, divide them
into groups, each group handled by a thread, and run a function on
each of the data file.

The way I'm doing this is I have a global char **filename variable,
where filename = filename of a data file. In the main function,
I'll read in the filenames of all the data files (minus "." and "..")
using scandir and put them in the filename variable. Then 4 (arbitrary
number) threads are created each calling the Process function. In
Process(), each thread only opens (using a FILE *fin declared in
Process()) and works on a portion of the data files using a
start_index and an end_index. For example, if there are 100 files,
then each thread will handle filename[0] to filename[24], filename[25]
to filename[49], filename[50] to filename[74] and filename[75] to
filename[99] respectively. After they're done, there is a pthread_join
in main() for all 4 threads.

I have checked that the filenames have been stored correctly, both in
main() and Process(). However, I keep getting segmentation fault here,
in Process():

for (i = start_index, i <= end_index ; i++)
fin = fopen(filename, "rb"); <--- Seg fault

I don't really get why there should be an error since none of the
threads are trying to open the same file.

Please advise.

The real problem is probably in the code you haven't posted. Post a
(short if possible) complete example.

 

[Nick]

[...] each thread only opens (using a FILE *fin declared in
Process()) and works on a portion of the data files using a
start_index and an end_index. For example, if there are 100 files,
then each thread will handle filename[0] to filename[24], filename[25]
to filename[49], filename[50] to filename[74] and filename[75] to
filename[99] respectively. After they're done, there is a pthread_join
in main() for all 4 threads.

I have checked that the filenames have been stored correctly, both in
main() and Process(). However, I keep getting segmentation fault here,
in Process():

for (i = start_index, i <= end_index ; i++)
fin = fopen(filename, "rb"); <--- Seg fault

Given the tiny amount of information available, my best guess can only
be a guess:

s/<=/<//

That looks a good guess. I've seen segfaults from fopen if a single
FILE * has been fclosed twice (no problem with the fclose).

 

[Andre Ramaciotti]

[...] each thread only opens (using a FILE *fin declared in
Process()) and works on a portion of the data files using a
start_index and an end_index. For example, if there are 100 files,
then each thread will handle filename[0] to filename[24], filename[25]
to filename[49], filename[50] to filename[74] and filename[75] to
filename[99] respectively. After they're done, there is a pthread_join
in main() for all 4 threads.

I have checked that the filenames have been stored correctly, both in
main() and Process(). However, I keep getting segmentation fault here,
in Process():

for (i = start_index, i <= end_index ; i++)
fin = fopen(filename, "rb"); <--- Seg fault

Given the tiny amount of information available, my best guess can only be a
guess:

s/<=/<//

I'll make another guess: s/,/;/ after start_index.

 

[Ben Bacarisse]

(e-mail address removed) wrote:

for (i = start_index, i <= end_index ; i++)
fin = fopen(filename, "rb"); <--- Seg fault

Given the tiny amount of information available, my best guess can only be a
guess:

s/<=/<//

I'll make another guess: s/,/;/ after start_index.

Unlikely -- just because I don't recall ever seeing a compiler that
would accept the for loop as presented. It is more likely a typing
error. The other reason this seems likely is that the loop, as shown,
is pointless since it overwrites the same fin variable every time.
There is a lot of evidence that this is "something like" the code
being debugged.

To the OP: make sure you include program text using some automatic
method rather than typing it in. At the very least, I'd want to see
the code that sets up the filename data and the code that chooses the
start and end indexes. If it not the fencepost error that Richard
Heathfield has suggested, it is most likely something to do with the
filename data, but people here need to see the real code.