How can I Read/Write multiple sequential Binary/Text data files

Discussion in 'Python' started by Albert Tu, Mar 10, 2005.

  1. Albert Tu

    Albert Tu Guest

    Dear there,

    We have an x-ray CT system. The acquisition computer acquires x-ray
    projections and outputs multiple data files in binary format (2-byte
    unsigned integer) such as projection0.raw, projection1.raw,
    projection2.raw ... up to projection500.raw. Each file is
    2*1024*768-byte big.

    I would like to read those files and convert to ascii files in %5.0f/n
    format as projection0.data ... projection500.data so that our
    visualization software can undersatnd the projection images. I was
    trying to do this conversion using Python. However, I had troubles
    declaring the file names using the do-loop index. Anyone had previous
    experience?

    Thanks,
    Albert
    Albert Tu, Mar 10, 2005
    #1
    1. Advertising

  2. On 10 Mar 2005 09:41:05 -0800, rumours say that "Albert Tu"
    <> might have written:

    >Dear there,
    >
    >We have an x-ray CT system. The acquisition computer acquires x-ray
    >projections and outputs multiple data files in binary format (2-byte
    >unsigned integer) such as projection0.raw, projection1.raw,
    >projection2.raw ... up to projection500.raw. Each file is
    >2*1024*768-byte big.
    >
    >I would like to read those files and convert to ascii files in %5.0f/n
    >format as projection0.data ... projection500.data so that our
    >visualization software can undersatnd the projection images. I was
    >trying to do this conversion using Python. However, I had troubles
    >declaring the file names using the do-loop index. Anyone had previous
    >experience?


    Regular expressions could help, but if you *know* that these are the filenames,
    you can (untested code):

    PREFIX= "projection"
    SUFFIX_I= ".raw"
    SUFFIX_O= ".data"

    import glob, struct

    for filename in glob.glob("%s*%s" % (PREFIX, SUFFIX_I)):
    number= filename[len(PREFIX):-len(SUFFIX_I)]
    fpi= open(filename, "rb")
    fpo= open("%s%s%s" % (PREFIX, number, SUFFIX_O), "w")
    while 1:
    datum= fpi.read(2)
    if not datum: break
    fpo.write("%5d\n" % struct.unpack("H", datum)) # check endianness!!!
    fpi.close()
    fpo.close()
    --
    TZOTZIOY, I speak England very best.
    "Be strict when sending and tolerant when receiving." (from RFC1958)
    I really should keep that in mind when talking with people, actually...
    Christos TZOTZIOY Georgiou, Mar 10, 2005
    #2
    1. Advertising

  3. Albert Tu

    John Machin Guest

    On Thu, 10 Mar 2005 20:06:29 +0200, Christos "TZOTZIOY" Georgiou
    <> wrote:

    >On 10 Mar 2005 09:41:05 -0800, rumours say that "Albert Tu"
    ><> might have written:
    >
    >>Dear there,
    >>
    >>We have an x-ray CT system. The acquisition computer acquires x-ray
    >>projections and outputs multiple data files in binary format (2-byte
    >>unsigned integer) such as projection0.raw, projection1.raw,
    >>projection2.raw ... up to projection500.raw. Each file is
    >>2*1024*768-byte big.
    >>
    >>I would like to read those files and convert to ascii files in %5.0f/n
    >>format as projection0.data ... projection500.data so that our
    >>visualization software can undersatnd the projection images. I was
    >>trying to do this conversion using Python. However, I had troubles
    >>declaring the file names using the do-loop index. Anyone had previous
    >>experience?

    >
    >Regular expressions could help, but if you *know* that these are the filenames,
    >you can (untested code):
    >
    >PREFIX= "projection"
    >SUFFIX_I= ".raw"
    >SUFFIX_O= ".data"
    >
    > import glob, struct


    import array

    DIFFERENT_ENDIAN = True/False

    >
    >for filename in glob.glob("%s*%s" % (PREFIX, SUFFIX_I)):
    > number= filename[len(PREFIX):-len(SUFFIX_I)]
    > fpi= open(filename, "rb")
    > fpo= open("%s%s%s" % (PREFIX, number, SUFFIX_O), "w")
    > while 1:
    > datum= fpi.read(2)
    > if not datum: break
    > fpo.write("%5d\n" % struct.unpack("H", datum)) # check endianness!!!


    If the OP knows that each input file is small enough (1.5Mb each as
    stated), then the agony of file.read(2) can be avoided by reading the
    whole file in one hit. The agony of struct.unpack() on each datum can
    be avoided by using the array module. E.g. replace the whole 'while'
    loop by this:

    ary = array.array('H', fpi.read())
    if DIFFERENT_ENDIAN:
    ary.byteswap()
    for datum in ary:
    fpo.write("%5d\n" % datum)

    Even if the input files were too large to fit in memory, they could
    still be processed fast enough by reading a big chunk at a time.


    > fpi.close()
    > fpo.close()
    John Machin, Mar 10, 2005
    #3
  4. On 10 Mar 2005 09:41:05 -0800, "Albert Tu" <> wrote:

    >Dear there,
    >
    >We have an x-ray CT system. The acquisition computer acquires x-ray
    >projections and outputs multiple data files in binary format (2-byte
    >unsigned integer) such as projection0.raw, projection1.raw,
    >projection2.raw ... up to projection500.raw. Each file is
    >2*1024*768-byte big.
    >
    >I would like to read those files and convert to ascii files in %5.0f/n
    >format as projection0.data ... projection500.data so that our
    >visualization software can undersatnd the projection images. I was

    Is there no chance of fixing the visualization software instead? The format seems
    easy and efficient, and it seems a shame to make a redundant bloated copy
    of the same info. What next? XML tags surrounding your ascii floats?

    What platform are you on? What is the visualization software's method of
    accessing data? Only files as you describe? Are you visualizing interactively,
    or setting up batch processing?

    >trying to do this conversion using Python. However, I had troubles
    >declaring the file names using the do-loop index. Anyone had previous
    >experience?
    >

    Are you the same person who posted re this format some time ago?

    Regards,
    Bengt Richter
    Bengt Richter, Mar 10, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Doug
    Replies:
    3
    Views:
    4,353
    Bruce Barker
    Nov 4, 2005
  2. Rhonda
    Replies:
    7
    Views:
    11,182
    Rhonda
    Sep 17, 2005
  3. Replies:
    4
    Views:
    937
    M.E.Farmer
    Feb 13, 2005
  4. utab
    Replies:
    3
    Views:
    829
  5. Ray Leon
    Replies:
    25
    Views:
    1,560
    John W Kennedy
    Jul 13, 2008
Loading...

Share This Page