multiline string split problem and fix

Discussion in 'Javascript' started by Brian, Dec 17, 2006.

  1. Brian

    Brian Guest

    I have been working on a data reception system.. I am still finding my
    way around Javascript, though I am accomplishing much.

    I just fixed a flaw that was really hard to find. The symptoms are
    this:

    I get a multiline string returned to Javascript from a Proxy+Google
    Maps API GDownloadUrl()
    The data, when added to a DOM table looked fine, about 20 lines in CSV
    format

    Sunrise,-119.098,35.345,0.0<br>
    SwanDancer,-119.345,35.567,1.0<br>
    .... etc

    (I don't know why the <br>'s are there, but that's what it looks like)
    So using a suggestion from this newsgroup, I perform two subsequent
    split()'s

    var index, index2;
    var strCSVFile = data;
    var arrayCSVFile;

    arrayCSVFile = strCSVFile.split( "<br>" );

    for ( index = 0; index < arrayCSVFile.length; index++ )
    {
    arrayCSVFile[ index ] = arrayCSVFile[ index ].split( ',' );
    // do stuff to the elements
    }

    I use both strCSVFile *and* arrayCSVFile to be doubly sure I wasn't
    somehow clobbering something, though in theory there needs to be only
    the original string. At any rate, what I see is this (after HOURS of
    trying and finally using str.charCodeAt())

    10|32|32|32|32|32|32|32|32|83|117|110|114|105|115|101| len=16
    10|10|32|32|32|32|32|32|32|32|83|119|97|110|68|97|110|99|101|114|
    len=20
    .... etc

    %^!@#$^%@ <- that's cursing, people
    So I am now hand clipping some number of LF and SPACE chars using
    str.charCodeAt(). On top of that, my furtive attempts at RegEx
    replacements along the way had been SILENTLY FAILING. Probably because
    of the leading LF(s). I had no idea, and it took valuable time..

    I looked for split() gotcha's but never found anything like this. I
    thought I tried changing the split to "<BR>\r" at one point, but I
    probably did the return instead of line feed... Also, that would NOT
    handle the first line case ?!

    This is what is happening, and I now have tedious code to handle it.
    Looking back on the original recv'd data, it does indeed have a leading
    LF|SPACE's, with two LF's on every subsequent row. I never saw them.
    How could I ? When I aded them to the HTML page to check the data, they
    didn't show
    This was awful.

    FYI
     
    Brian, Dec 17, 2006
    #1
    1. Advertising

  2. Brian

    Brian Guest

    btw- I just added

    strCSVFile.replace( / /g, '');
    strCSVFile.replace( / \n/g, '');
    strCSVFile.replace( / \r/g, '');

    to clean the data (the whole block before the split()'s. Am I making a
    mistake in the RegEx? they don't work...
     
    Brian, Dec 17, 2006
    #2
    1. Advertising

  3. Brian

    Brian Guest

    >
    > strCSVFile.replace( / /g, '');
    > strCSVFile.replace( / \n/g, '');
    > strCSVFile.replace( / \r/g, '');
    >
    >


    hmmm, very late night typing.. I meant

    strCSVFile.replace( / +/g, '');
    strCSVFile.replace( /\n+/g, '');
    strCSVFile.replace( /\r+/g, '');
     
    Brian, Dec 17, 2006
    #3
  4. Brian

    mick white Guest

    Brian wrote:

    >
    > hmmm, very late night typing.. I meant
    >
    > strCSVFile.replace( / +/g, '');
    > strCSVFile.replace( /\n+/g, '');
    > strCSVFile.replace( /\r+/g, '');
    >

    Perhaps you need:
    someVariable=strCSVFile.replace(/\s/g,'');

    No need for "+", since you are using the "g" modifier. And why not
    assign the result of the statement to a variable? (Unless you want to
    destroy the original string.)
    Mick
     
    mick white, Dec 17, 2006
    #4
  5. Brian

    RobG Guest

    Brian wrote:
    > I have been working on a data reception system.. I am still finding my
    > way around Javascript, though I am accomplishing much.
    >
    > I just fixed a flaw that was really hard to find. The symptoms are
    > this:
    >
    > I get a multiline string returned to Javascript from a Proxy+Google
    > Maps API GDownloadUrl()
    > The data, when added to a DOM table looked fine, about 20 lines in CSV
    > format
    >
    > Sunrise,-119.098,35.345,0.0<br>
    > SwanDancer,-119.345,35.567,1.0<br>
    > ... etc


    Try something like:

    var strCSVFile = 'Sunrise,-119.098,35.345,0.0<br>'
    + 'SwanDancer,-119.345,35.567,1.0<br>';

    /* Remove any leading or trailing br elements */
    strCSVFile = strCSVFile.replace(/(^<br>)|(<br>$)/g,'');

    var arrayCSVFile = strCSVFile.split('<br>');
    var recordElement;

    /* arrayCSVFile is now an array with two elements:
    *
    * ['Sunrise,-119.098,35.345,0.0',
    * 'SwanDancer,-119.345,35.567,1.0']
    */
    for (var i=0, len=arrayCSVFile.length; i<len; i++){
    arrayCSVFile = arrayCSVFile.split(',');

    /* arrayCSVFile is still an array with two elements,
    * but each is now an array of 4 elements:
    *
    * [
    * ['Sunrise', '-119.098', '35.345', '0.0'],
    * ['SwanDancer', '-119.345', '35.567', '1.0']
    * ]
    */
    for (var j=0, len2=arrayCSVFile.length; j<len2; j++){
    recordElement = arrayCSVFile[j];

    /* recordElement will be each element in turn, i.e.
    * 'Sunrise', then '-119.098', then '35.345', and so on
    */

    alert('Record: ' + (i+1) + ' of ' + len
    + '\nElement: ' + (j+1) + ' of ' + len2
    + '\nValue: ' + recordElement);
    }
    }

    If you want to remove all whitespace (all spaces, tabs, linefeeds,
    returns, the lot) than add .replace(/\s/g, '') to the end of the line
    where the leading and trailing br's are replaced.


    --
    Rob
     
    RobG, Dec 17, 2006
    #5
  6. Brian

    Brian Guest

    RobG wrote:
    > Brian wrote:...
    > Try something like:
    >
    > var strCSVFile = 'Sunrise,-119.098,35.345,0.0<br>'
    > + 'SwanDancer,-119.345,35.567,1.0<br>';
    >
    > /* Remove any leading or trailing br elements */
    > strCSVFile = strCSVFile.replace(/(^<br>)|(<br>$)/g,'');
    >
    > var arrayCSVFile = strCSVFile.split('<br>');
    > var recordElement;
    >
    > /* arrayCSVFile is now an array with two elements:
    > *
    > * ['Sunrise,-119.098,35.345,0.0',
    > * 'SwanDancer,-119.345,35.567,1.0']
    > */
    > for (var i=0, len=arrayCSVFile.length; i<len; i++){
    > arrayCSVFile = arrayCSVFile.split(',');
    >
    > /* arrayCSVFile is still an array with two elements,
    > * but each is now an array of 4 elements:
    > *
    > * [
    > * ['Sunrise', '-119.098', '35.345', '0.0'],
    > * ['SwanDancer', '-119.345', '35.567', '1.0']
    > * ]
    > */
    > for (var j=0, len2=arrayCSVFile.length; j<len2; j++){
    > recordElement = arrayCSVFile[j];
    >
    > /* recordElement will be each element in turn, i.e.
    > * 'Sunrise', then '-119.098', then '35.345', and so on
    > */
    >
    > alert('Record: ' + (i+1) + ' of ' + len
    > + '\nElement: ' + (j+1) + ' of ' + len2
    > + '\nValue: ' + recordElement);
    > }
    > }
    >



    hey, Rob, I think you have dome something like this before (!) The
    clarity of your formatting alone is helpful. I, also, split twice to
    end up with CSV[][].

    I ended up going a different direction at the end though. Rather than
    make a single, flat line of elements, why not use the JS Object/perl
    hash/ObjectiveC collection(?)/STL Map idiom. that is, and array of
    objects, a series of name/value pairs. years ago I called it a
    dictionary, or associative array (name/value pairs), but didn't use it
    much. These days, it seems to have sprung into popularity, with
    language support to just 'toss in' elements as needed (though I am not
    using it that way here)

    So the final lines of the my version turns into:
    // Obj w/named fields, the result of processing the CSV
    var csvDataObj = {};
    csvDataObj.recArray = [];

    // process each record
    for ( index = 0; index < arrayCSVFile.length; index++ )
    {
    arrayCSVFile[ index ] = arrayCSVFile[ index ].split( ',' );

    // build data container to pass out to other processes
    var tRecObj = {};
    tRecObj.recName = arrayCSVFile[index][0];
    tRecObj.recLat = arrayCSVFile[index][1];
    tRecObj.recLng = arrayCSVFile[index][2];
    tRecObj.recAlt = arrayCSVFile[index][3];

    csvDataObj.recArray.push( tRecObj);
    }

    // Now do something with arrayCSVFile[][]
    gMyCore.processAllDevices( csvDataObj);

    On the other end, I could "discover" the elements, but since its only
    Javascript ;-) I use direct knowledge of the contents in the code.

    I wrote it, worked the first time. So some things are going ok :)
     
    Brian, Dec 18, 2006
    #6
  7. Brian

    Brian Guest

    mick white wrote:
    > Brian wrote:
    > >
    > > strCSVFile.replace( / +/g, '');
    > > strCSVFile.replace( /\n+/g, '');
    > > strCSVFile.replace( /\r+/g, '');
    > >

    > Perhaps you need:
    > someVariable=strCSVFile.replace(/\s/g,'');
    >
    > No need for "+", since you are using the "g" modifier. And why not
    > assign the result of the statement to a variable? (Unless you want to
    > destroy the original string.)


    I would happily destry the original string! but, I am reticent to
    admit, those simple replace's were failing silently. Haven't revisited
    it quite yet.. on a deadline.. but will look soon
     
    Brian, Dec 18, 2006
    #7
  8. In comp.lang.javascript message
    <erehh.4447$>, Sun, 17 Dec 2006 16:41:46,
    mick white <> wrote:
    >Brian wrote:
    >
    >> hmmm, very late night typing.. I meant
    >> strCSVFile.replace( / +/g, '');
    >> strCSVFile.replace( /\n+/g, '');
    >> strCSVFile.replace( /\r+/g, '');
    >>

    > Perhaps you need:
    >someVariable=strCSVFile.replace(/\s/g,'');
    >
    >No need for "+", since you are using the "g" modifier. And why not
    >assign the result of the statement to a variable? (Unless you want to
    >destroy the original string.)



    Str.replace should have no effect on Str's existence. The .replace
    method generates a new string, but does not destroy the old one.

    AFAIK, the only type of Object which has Methods provided to change its
    value is the Date Object.


    I would expect, if there are instances of multiple /r, for the /r+
    version to be slightly faster, since it calls for fewer replacements.
    That could be implementation-dependent. A quick test shows a slight
    gain in speed when using +.

    It's a good idea to read the newsgroup and its FAQ. See below.

    --
    (c) John Stockton, Surrey, UK. ?@merlyn.demon.co.uk Turnpike v6.05 IE 6
    <URL:http://www.jibbering.com/faq/> A FAQ for news:comp.lang.javascript.
    <URL:http://www.merlyn.demon.co.uk/js-index.htm> jscr maths, dates, sources.
    <URL:http://www.merlyn.demon.co.uk/> TP/BP/Delphi/jscr/&c, FAQ items, links.
     
    Dr J R Stockton, Dec 18, 2006
    #8
  9. Brian

    mick white Guest

    Dr J R Stockton wrote:

    > mick white <> wrote:


    >> Perhaps you need:
    >> someVariable=strCSVFile.replace(/\s/g,'');
    >>
    >> No need for "+", since you are using the "g" modifier. And why not
    >> assign the result of the statement to a variable? (Unless you want to
    >> destroy the original string.)

    >
    >
    > Str.replace should have no effect on Str's existence. The .replace
    > method generates a new string, but does not destroy the old one.


    var StringA="A B C"
    var StringA=StringA.replace(/\s/g,'');
    alert(StringA);
    But you're correct, /technically/.

    Mick
     
    mick white, Dec 18, 2006
    #9
  10. In comp.lang.javascript message
    <jOChh.4643$>, Mon, 18 Dec 2006 20:24:47,
    mick white <> wrote:
    >Dr J R Stockton wrote:


    >> Str.replace should have no effect on Str's existence. The .replace
    >>method generates a new string, but does not destroy the old one.

    >
    >var StringA="A B C"
    >var StringA=StringA.replace(/\s/g,'');
    >alert(StringA);
    >But you're correct, /technically/.


    It is not the .replace that destroys "A B C", but the subsequent
    assignment.

    --
    (c) John Stockton, Surrey, UK. ?@merlyn.demon.co.uk Turnpike v6.05 IE 6
    news:comp.lang.javascript FAQ <URL:http://www.jibbering.com/faq/index.html>.
    <URL:http://www.merlyn.demon.co.uk/js-index.htm> jscr maths, dates, sources.
    <URL:http://www.merlyn.demon.co.uk/> TP/BP/Delphi/jscr/&c, FAQ items, links.
     
    Dr J R Stockton, Dec 19, 2006
    #10
  11. Brian

    Brian Guest

    Dr J R Stockton wrote:
    > ...
    > It's a good idea to read the newsgroup and its FAQ. See below.
    >
    > --
    > (c) John Stockton, Surrey, UK. ?@merlyn.demon.co.uk Turnpike v6.05 IE 6
    > <URL:http://www.jibbering.com/faq/> A FAQ for news:comp.lang.javascript.
    > <URL:http://www.merlyn.demon.co.uk/js-index.htm> jscr maths, dates, sources.
    > <URL:http://www.merlyn.demon.co.uk/> TP/BP/Delphi/jscr/&c, FAQ items, links.


    Searching the three FAQs listed in your signature for the term
    'replace' yields nothing at all about string replaces...

    Rather than empathy or insight, a blind "read the FAQs" with no
    specific pointer, and in this case FAQs that have no reference to my
    problems. is in itself less than helpful.

    page 526 of Javascript, the Definitive Guide, 4th Ed. does say that the
    string replace() returns a new string with the replacement, but fails
    to emphasize that the original argument is untouched.

    On another note - Had this been C, would have checked the binary
    contents of the array returned remotely immediately, by rote, in any
    decent development envirnment. Since its web, and .js, and I'm still
    getting used to it, I got caught by an annoying situation. Now I know.
    I trust the thread will be helpful to someone someday.
     
    Brian, Dec 21, 2006
    #11
  12. In comp.lang.javascript message
    <>, Wed, 20 Dec 2006
    18:37:29, Brian <> wrote:
    >
    >Dr J R Stockton wrote:
    >> ...
    >> It's a good idea to read the newsgroup and its FAQ. See below.


    >Searching the three FAQs listed in your signature for the term
    >'replace' yields nothing at all about string replaces...


    If I had wished to give you advice about your problem, I would have done
    so in an article *directly* following-up to one of yours.

    >Rather than empathy or insight, a blind "read the FAQs" with no
    >specific pointer, and in this case FAQs that have no reference to my
    >problems. is in itself less than helpful.


    If you have indeed searched all three URLs, you should have learned
    quite a bit that can be useful to you in the future. You should also
    have seen no case showing that the string parameter of .replace is
    changed, and many cases in which the value of S.replace(a, b) is
    assigned for use. You should also have seen a couple of references to
    ECMA-262.

    >page 526 of Javascript, the Definitive Guide, 4th Ed. does say that the
    >string replace() returns a new string with the replacement, but fails
    >to emphasize that the original argument is untouched.


    I'll take your word for that; but what do other pages say? However, if
    it is clear that a value is RETURNed, is it not then unreasonable to
    expect the original string (which is not an argument) to be altered?
    That would be superfluous.


    It's a good idea to read the newsgroup and its FAQ. See below.

    --
    (c) John Stockton, Surrey, UK. ?@merlyn.demon.co.uk Turnpike v6.05 IE 6
    news:comp.lang.javascript FAQ <URL:http://www.jibbering.com/faq/index.html>.
    <URL:http://www.merlyn.demon.co.uk/js-index.htm> jscr maths, dates, sources.
    <URL:http://www.merlyn.demon.co.uk/> TP/BP/Delphi/jscr/&c, FAQ items, links.
     
    Dr J R Stockton, Dec 21, 2006
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.

Share This Page