Complex CSV parsing

Discussion in 'Ruby' started by Stuart Clarke, Jan 13, 2009.

  1. I am trying to parse data from a file where the values are common
    seperated, however there is slightly more to the file than just commas,
    see below

    for
    (;;);{"t":"msg","c":"p_114000000","ms":[{"type":"msg","msg":{"text":"you
    around"}]}

    From reading around fasterCSV seems the way forward therefore I have
    this code

    require 'rubygems'
    require 'faster_csv'

    FasterCSV.foreach("C:\\Documents and
    Settings\\sjc\\Desktop\\p_1149549999=2[1].txt", :row_sep => ",") do
    |row|
    puts row[0]
    break
    end

    However I am getting an error like this

    C:/Program
    Files/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:1
    650:in `shift': Illegal quoting on line 1.
    (FasterCSV::MalformedCSVError)
    from C:/Program
    Files/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/fa
    ster_csv.rb:1568:in `loop'
    from C:/Program
    Files/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/fa
    ster_csv.rb:1568:in `shift'
    from C:/Program
    Files/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/fa
    ster_csv.rb:1513:in `each'
    from C:/Program
    Files/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/fa
    ster_csv.rb:1017:in `foreach'
    from C:/Program
    Files/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/fa
    ster_csv.rb:1191:in `open'
    from C:/Program
    Files/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/fa
    ster_csv.rb:1016:in `foreach'
    from C:/Documents and Settings/sjc/Desktop/test.rb:4

    Can anyone help me out with this?

    Many thanks
    --
    Posted via http://www.ruby-forum.com/.
     
    Stuart Clarke, Jan 13, 2009
    #1
    1. Advertising

  2. Stuart Clarke wrote:
    > I am trying to parse data from a file where the values are common
    > seperated, however there is slightly more to the file than just commas,
    > see below
    >
    > for
    > (;;);{"t":"msg","c":"p_114000000","ms":[{"type":"msg","msg":{"text":"you
    > around"}]}


    That is not valid CSV, so FasterCSV won't help you.

    I think you'll need to describe more carefully how you want this input
    line broken up, and give some more examples.

    It looks to me like a nested structure. If every line has exactly the
    same set of fields you may get away with a regexp. But if not, you may
    have to write a full-blown parser for this language.

    However, this may be sufficiently close to JSON that you could use an
    existing JSON parser. http://www.json.org/

    (But in that case, I don't know what the "for(;;);" is doing on the
    front)
    --
    Posted via http://www.ruby-forum.com/.
     
    Brian Candler, Jan 13, 2009
    #2
    1. Advertising

  3. Stuart Clarke

    Mark Thomas Guest

    On Jan 13, 8:42 am, Stuart Clarke <> wrote:
    > I am trying to parse data from a file where the values are common
    > seperated, however there is slightly more to the file than just commas,
    > see below
    >
    > for
    > (;;);{"t":"msg","c":"p_114000000","ms":[{"type":"msg","msg":{"text":"you
    > around"}]}


    That's not a CSV file. It looks like some sort of serialized data
    structure. If you know how it was serialized, you should be able to
    easily restore the structure.

    If not, you can use Treetop to specify the grammar including the
    balanced delimiters {}, (), and [], which apparently take precedence
    over the commas, and perform the parsing.
     
    Mark Thomas, Jan 13, 2009
    #3
  4. In article <>,
    Stuart Clarke <> wrote:
    >I am trying to parse data from a file where the values are common
    >seperated, however there is slightly more to the file than just commas,
    >see below
    >
    >for
    >(;;);{"t":"msg","c":"p_114000000","ms":[{"type":"msg","msg":{"text":"you
    >around"}]}


    It looks more JSON than CSV, try using a JSON parser instead.
    --
    Ollivier ROBERT -=- EEC/RIF/SEU -=-
    Systems Engineering Unit
     
    Ollivier Robert, Jan 14, 2009
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. GIMME
    Replies:
    2
    Views:
    930
    GIMME
    Feb 11, 2004
  2. news.amnet.net.au
    Replies:
    1
    Views:
    604
    =?UTF-8?b?TMSByrtpZSBUZWNoaWU=?=
    Apr 13, 2004
  3. Stanimir Stamenkov
    Replies:
    2
    Views:
    788
    Stanimir Stamenkov
    Oct 25, 2005
  4. Robert Mark Bram
    Replies:
    0
    Views:
    716
    Robert Mark Bram
    Feb 4, 2007
  5. Kottiyath

    How complex is complex?

    Kottiyath, Mar 18, 2009, in forum: Python
    Replies:
    22
    Views:
    799
Loading...

Share This Page