Bug in 1.9.0 csv parser

Discussion in 'Ruby' started by Nicko Kaltner, Jan 10, 2008.

  1. Hi,
    I've found a bug in the 1.9.0 csvparser. I've got a script and data
    that effectively breaks it, but it is 567kb. Is that too large for this
    list?

    The ruby instance takes 100% of cpu while processing this file, and I
    had to stop it after 5 minutes..

    the code is

    #!/usr/local/bin/ruby1.9.0

    require 'csv'

    filename = 'broken.csv'

    CSV.foreach(filename) do |row|
    STDERR.puts row.length
    row.each do |entry|
    puts entry
    end
    puts "\n####################################\n"
    end


    I would try and debug it further, but the debugger seems broken in 1.9.0.


    Regards,
    Nicko
     
    Nicko Kaltner, Jan 10, 2008
    #1
    1. Advertising

  2. Nicko Kaltner

    James Gray Guest

    On Jan 10, 2008, at 2:15 AM, Nicko Kaltner wrote:

    > I've found a bug in the 1.9.0 csvparser. I've got a script and data =20=


    > that effectively breaks it, but it is 567kb. Is that too large for =20=


    > this list?


    Probably, but you are welcome to email it to me privately. I maintain =20=

    the CSV library in Ruby 1.9.

    > The ruby instance takes 100% of cpu while processing this file, and =20=


    > I had to stop it after 5 minutes..


    I'm 95% percent sure this is an issue of your CSV file being invalid. =20=

    Because of the way the format works, a simple file like:

    "=8510 Gigs of data without another quote=85

    can only be detected invalid by reading the entire thing. I've =20
    considered putting limits in the library for controlling how far it =20
    would read ahead, but these would break some valid data. Then the =20
    problem becomes, do I make the limits default to on? That's the only =20=

    way they would have helped here, but that would break some otherwise =20
    valid code. It's a tough problem to solve correctly.

    James Edward Gray II=
     
    James Gray, Jan 10, 2008
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michal Mikolajczyk
    Replies:
    0
    Views:
    690
    Michal Mikolajczyk
    Feb 13, 2004
  2. Skip Montanaro
    Replies:
    0
    Views:
    761
    Skip Montanaro
    Feb 13, 2004
  3. Tintin92
    Replies:
    1
    Views:
    1,806
    Andrew Thompson
    Feb 14, 2007
  4. jliu66
    Replies:
    0
    Views:
    559
    jliu66
    Oct 19, 2007
  5. sso
    Replies:
    20
    Views:
    2,770
    Martin Gregorie
    Apr 26, 2009
Loading...

Share This Page