Reading Simple Flatfiles with a COBOL Mindset

[KevinD]

thank you for your helpful explanations.

In my first note I forgot to mention that my simple flatfile is a text
file with a newline character at the end thus I able to get an entire
record [ie. empid, dept, name and hiredate with one fgets()
statement].


Dave V. - I understand your description but as described above, I get
all 5 data fields in one file read using fgets.


I will try the fopen in binary form and fread as describe.


My COBOL example is from my IBM mainframe days (EBCDIC) and my new C
programming is on a DEC VAX circa 1989. So EBCDIC will not be an
issue.

I thought I should be able to read all 58 bytes + 1 byte (newline)
into a generic string variable then move that data to a structure. I
would have thought that a fixed record layout (i.e. think back to an
old keypunch card) would be a universal file concept but I guessed
wrong.

thanks again
kevin

~~~~~~
......

01 employee-record.
03 emp-id pic 9(5).
03 emp-dept pic x(5).
03 emp-name.
05 emp-name-last pic x(20).
05 emp-name-first pic x(20).
03 emp-hire-date.
05 emp-hire-date-mm pic 9(2).
05 emp-hire-date-dd pic 9(2).
05 emp-hire-date-yy pic 9(4).

read employee-flatfile into employee-record.

 

[Chris Barts]

I would have thought that a fixed record layout (i.e. think back to an
old keypunch card) would be a universal file concept but I guessed
wrong.

Heh. C was designed by people who were damned glad they /weren't/ using
punch cards. It was created at the same time (and by some of the same
people) who were inventing the Unix operating system, a system that has
never supported the punch card the way IBM OSes from (roughly) the same
era did. (Unix, and C, like teletypes and other character-oriented
interactive input/output methods.)

Of course, C has grown up and left its single-OS roots behind, but it
still makes certain assumptions about how programmers want to see the
world. Elaborate support for records isn't one of them.

 

[Dave Vandervies]

thank you for your helpful explanations.

In my first note I forgot to mention that my simple flatfile is a text
file with a newline character at the end thus I able to get an entire
record [ie. empid, dept, name and hiredate with one fgets()
statement].


Dave V. - I understand your description but as described above, I get
all 5 data fields in one file read using fgets.

That makes the reading part of my suggestion easier, and leaves most of
the extracting data from the string that's been read in quite similar.
Just read it with a single call to fgets (if they're delimited by
newlines, this is Really Easy and not just easy; if there's only a
single newline at the end of the file, you can give the record length
(+1 for the '\0' to terminate the string) to fgets as its length argument
instead of the buffer size), and then extract the parts of the record
from that string.

I will try the fopen in binary form and fread as describe.

There's (usually) no good reason to open a text file in binary mode.
If you have a newline at the end of each record, you can read a record
at a time with fgets and then extract it.

My COBOL example is from my IBM mainframe days (EBCDIC) and my new C
programming is on a DEC VAX circa 1989. So EBCDIC will not be an
issue.

If you're still working with the original compiler that came with the
system, you'll most likely not be able to directly use a lot of the
code you'll see posted here, since it will be written to conform to
the standard published in 1989. If you have access to an (even mostly)
ANSI-compliant compiler for it, that will make working with most of the
examples you see rather easier.

I thought I should be able to read all 58 bytes + 1 byte (newline)
into a generic string variable then move that data to a structure.

01 employee-record.
03 emp-id pic 9(5).
03 emp-dept pic x(5).
03 emp-name.
05 emp-name-last pic x(20).
05 emp-name-first pic x(20).
03 emp-hire-date.
05 emp-hire-date-mm pic 9(2).
05 emp-hire-date-dd pic 9(2).
05 emp-hire-date-yy pic 9(4).

Is this the description of the record in code? (I had thought it was
the form as it appears in the file, which confused me a little bit.)

If your records as seen in the file look like this:
IIIIIDDDDDLLLLLLLLLLLLLLLLLLLLFFFFFFFFFFFFFFFFFFFFMMDDYYYY\n
(I=id, D=dept, L=lastname,F=firstname,MMDDYYYY=date)
you can read an entire record with fgets and then extract the components:
--------
fgets(buf,sizeof buf,in);
ret.emp_id=/*extract ID from buf[0]..buf[4], perhaps with sscanf*/
ret.emp_dept=/*extract dept from buf[5]..buf[9] (strncpy?)*/
/*...continue...
(Don't forget to COPY your strings, not just point into buf[])
*/
return ret;
--------

The internal representation in a struct can be easily made close to the
representation in the file, but it would be nontrivial to set up and
difficult to work with if you wanted to make it identical. So you're
better off repacking it into a struct that you can work with easily in
C when you read them, and repacking the struct back into the flat-file
format when you write them.

I
would have thought that a fixed record layout (i.e. think back to an
old keypunch card) would be a universal file concept but I guessed
wrong.

C's view of file operations tries to avoid working at a high enough
level to understand fixed record layouts.
(It is easier to fake them with the everything's-a-stream-of-bytes model
than the other way around, though. If you're restricted to a single I/O
abstraction to work with, stream-of-bytes is one of the better choices.)


dave

 

[kal]

In my first note I forgot to mention that my simple flatfile is a text
file with a newline character at the end thus I able to get an entire
record [ie. empid, dept, name and hiredate with one fgets()
statement].

It is difficult to undrestand what "I able" means, for the verb seems
to be missing. If you ARE able to get the entire record then I am
unable to understand what the problem is.

My apologies for making this post long.

FWIW the following are my 2 cents.

INPUT FILE:
-----------
00001USEXEWASHINGTON GEORGE 04301789
00002USEXEADAMS JOHN 03041797
00003USEXEJEFFERSON THOMAS 03041801

PROGRAM OUTPUT:
---------------

Input file: pres.txt

record 1 : 1,USEXE,GEORGE WASHINGTON,4/30/1789
record 2 : 2,USEXE,JOHN ADAMS,3/4/1797
record 3 : 3,USEXE,THOMAS JEFFERSON,3/4/1801

PROGRAM SOURCE:
---------------
#include <stdio.h>
#include <ctype.h>
#include <string.h>

long *get_long (long*, const char*, int);
char *get_string (char*, const char*, int);


int main (int argc, char *argv[])
{
FILE *fflat = NULL;
char buf[256], out[256];
long recno, id, mm, dd, yyyy;
char dept[6], namelast[21], namefirst[21];

do
{
if (argc != 2)
{ printf("Incorrect number of parameters.\n");
printf("Usage: cflat.exe <filename>\n");
break;
}

printf("\nInput file: %s\n\n",argv[1]);

fflat = fopen(argv[1],"rt");
if (fflat == NULL)
{ printf("File open failed\n");
break;
}

recno = 0;
while (fgets(buf,60,fflat))
{
recno++;
if (strlen(buf) != 59)
{ printf("Invalid record length %d\n",strlen(buf));
printf("record data: %s\n",buf);
break;
}
if (get_long (&id, buf+0, 5) == NULL ||
get_string(dept, buf+5, 5) == NULL ||
get_string(namelast, buf+10,20) == NULL ||
get_string(namefirst,buf+30,20) == NULL ||
get_long (&mm, buf+50,2 ) == NULL ||
get_long (&dd, buf+52,2 ) == NULL ||
get_long (&yyyy, buf+54,4 ) == NULL)
{ printf("record data error.\n");
printf("record data: %s\n",buf);
break;
}
sprintf(out,"%ld,%s,%s %s,%ld/%ld/%ld",
id,dept,namefirst,namelast,mm,dd,yyyy);
printf ("record %ld : %s\n",recno,out);
}

if (feof(fflat) == 0)
{ printf("Could not process the complete file.\n");
break;
}

} while(0);

if (fflat) fclose(fflat);

return 0;
}


long *get_long (long *dest, const char *buf, int size)
{
int i;

if (size < 1)
return NULL;

*dest = 0;
for (i=0; i != size; i++)
{
if (!isdigit(buf))
break;
*dest = (*dest * 10) + (buf - '0');
}
if (i != size)
return NULL;

return dest;
}


char *get_string (char *dest, const char *buf, int size)
{
int i;

if (size < 1)
return NULL;

memcpy(dest,buf,size);
for (i=size-1; i >= 0 && dest == ' '; i--)
;
dest[i+1] = 0;

return dest;
}