Bug in 1.9.0 csv parser

Discussion in 'Ruby' started by Nicko Kaltner, Jan 10, 2008.

  1. Hi,
    I've found a bug in the 1.9.0 csvparser. I've got a script and data
    that effectively breaks it, but it is 567kb. Is that too large for this
    list?

    The ruby instance takes 100% of cpu while processing this file, and I
    had to stop it after 5 minutes..

    the code is

    #!/usr/local/bin/ruby1.9.0

    require 'csv'

    filename = 'broken.csv'

    CSV.foreach(filename) do |row|
    STDERR.puts row.length
    row.each do |entry|
    puts entry
    end
    puts "\n####################################\n"
    end


    I would try and debug it further, but the debugger seems broken in 1.9.0.


    Regards,
    Nicko
     
    Nicko Kaltner, Jan 10, 2008
    #1
    1. Advertisements

  2. Nicko Kaltner

    James Gray Guest

    On Jan 10, 2008, at 2:15 AM, Nicko Kaltner wrote:

    > I've found a bug in the 1.9.0 csvparser. I've got a script and data =20=


    > that effectively breaks it, but it is 567kb. Is that too large for =20=


    > this list?


    Probably, but you are welcome to email it to me privately. I maintain =20=

    the CSV library in Ruby 1.9.

    > The ruby instance takes 100% of cpu while processing this file, and =20=


    > I had to stop it after 5 minutes..


    I'm 95% percent sure this is an issue of your CSV file being invalid. =20=

    Because of the way the format works, a simple file like:

    "=8510 Gigs of data without another quote=85

    can only be detected invalid by reading the entire thing. I've =20
    considered putting limits in the library for controlling how far it =20
    would read ahead, but these would break some valid data. Then the =20
    problem becomes, do I make the limits default to on? That's the only =20=

    way they would have helped here, but that would break some otherwise =20
    valid code. It's a tough problem to solve correctly.

    James Edward Gray II=
     
    James Gray, Jan 10, 2008
    #2
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mike Bridge

    Need CSV Parser

    Mike Bridge, May 11, 2004, in forum: ASP .Net
    Replies:
    4
    Views:
    2,921
    clintonG
    May 11, 2004
  2. Michal Mikolajczyk
    Replies:
    0
    Views:
    908
    Michal Mikolajczyk
    Feb 13, 2004
  3. Tintin92
    Replies:
    1
    Views:
    2,121
    Andrew Thompson
    Feb 14, 2007
  4. jliu66
    Replies:
    0
    Views:
    747
    jliu66
    Oct 19, 2007
  5. sso
    Replies:
    20
    Views:
    3,159
    Martin Gregorie
    Apr 26, 2009
  6. Tim
    Replies:
    1
    Views:
    458
    Peter Otten
    Jul 5, 2010
  7. Li Chen
    Replies:
    18
    Views:
    984
    Azmi Farih
    Mar 23, 2010
  8. Sacha Rook

    csv read clean up and write out to csv

    Sacha Rook, Nov 2, 2012, in forum: Python
    Replies:
    2
    Views:
    433
    Hans Mulder
    Nov 2, 2012
Loading...