Using external text file to do search and replace?

Discussion in 'Ruby' started by Dan George, Aug 22, 2007.

  1. Dan George

    Dan George Guest

    I recently started to take an interest in Ruby programming and with
    the help of some people and by reading Learn to program by Chris Pine
    I made myself a little program that has come to a halt because I don't
    know if or how I can use an external file where I store the strings I
    want to search and replace in text type files.

    This is what I have so far:

    txt_files = Dir.glob('**/*.txt').each do |path|
    puts path
    txts = path.to_s
    file = File.open(txts).readlines.each { |line|
    if line.match(/PROMOS:/)
    then line.gsub!(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
    end }
    file2=File.open(txts, "w")
    file2.write( file )
    end

    What I want is to be able to use an external file where I store the
    values I want to search and replace with "line.gsub!" eg.: "/PROMOn=[A-
    Za-z0-9]+/, 'PROMOn=some_other_text'".
    I will need to use RubyScript2Exe because I'm not sure that on some
    other machines I will have ruby installed and it's easier to just put
    the strings I want to search and replace in an external file that is
    located in the same folder as the script or on a predefined path.

    Any ideas, hints, improvements and critiques are highly welcome.
     
    Dan George, Aug 22, 2007
    #1
    1. Advertising

  2. Alle mercoled=EC 22 agosto 2007, Dan George ha scritto:
    > I recently started to take an interest in Ruby programming and with
    > the help of some people and by reading Learn to program by Chris Pine
    > I made myself a little program that has come to a halt because I don't
    > know if or how I can use an external file where I store the strings I
    > want to search and replace in text type files.
    >
    > This is what I have so far:
    >
    > txt_files =3D Dir.glob('**/*.txt').each do |path|
    > puts path
    > txts =3D path.to_s
    > file =3D File.open(txts).readlines.each { |line|
    > if line.match(/PROMOS:/)
    > then line.gsub!(/PROMO1=3D[A-Za-z0-9]+/, 'PROMO1=3Dsome_text')
    > end }
    > file2=3DFile.open(txts, "w")
    > file2.write( file )
    > end
    >
    > What I want is to be able to use an external file where I store the
    > values I want to search and replace with "line.gsub!" eg.: "/PROMOn=3D[A-
    > Za-z0-9]+/, 'PROMOn=3Dsome_other_text'".
    > I will need to use RubyScript2Exe because I'm not sure that on some
    > other machines I will have ruby installed and it's easier to just put
    > the strings I want to search and replace in an external file that is
    > located in the same folder as the script or on a predefined path.
    >
    > Any ideas, hints, improvements and critiques are highly welcome.


    =46irst some comments about your code:
    * path is already a string, so calling to_s on it does nothing.
    * You can replace the File.open(txts).readlines part with=20
    =46ile.readlines(txts), which is (in my opinion) clearer and doesn't force =
    you=20
    to remember to close the file (which by the way, you don't do).
    * When you write to file2, you can use the block form of File.open, which=20
    takes care of closing the file for you.

    Here's how what I'd have written:

    Dir.glob('**/*.txt').each do |path|
    lines =3D File.readlines(path)
    if line.match(/PROMOS:/)
    lines.map!{|l| l.gsub(/PROMO1=3D[A-Za-z0-9]+/, 'PROMO1=3Dsome_text')=20
    end
    File.open(path, 'w'){|f| f.write lines}
    end

    As you can see, I've also replaced the each/gsub! combination with map!/gsu=
    b,=20
    which, in my opinion makes clearer what you're doing (changing the contents=
    =20
    of the array).

    As for storing the search/replacement pairs on a file, I'd use YAML. It's=20
    included in the standard library, so there shouldn't be problems with=20
    RubyScript2Exe. You can get information on yaml for ruby at=20
    http://yaml4r.sourceforge.net/ (look in particular at the cookbook and doc=
    =20
    sections). A simple example could be this:

    'PROMO1=3D[A-za-z0-9]+': 'PROMO1=3Dsome_text'
    'PROMO2=3D[A-za-z0-9]+': 'PROMO1=3Dsome_other_text'
    =2E..

    When read into ruby using YAML.load, this would return the following hash:

    {
    'PROMO1=3D[A-za-z0-9]+' =3D> 'PROMO1=3Dsome_text',
    'PROMO2=3D[A-za-z0-9]+' =3D> 'PROMO1=3Dsome_other_text'
    }

    You could then create regexps using Regexp.new. (Actually, you can also sto=
    re=20
    the regexps directly in the yaml file, prefixing them with the=20
    string !ruby/regexp, but I think the file is easier to read/write this way).

    I hope this helps.

    Stefano
     
    Stefano Crocco, Aug 22, 2007
    #2
    1. Advertising

  3. On Aug 22, 6:55 am, Dan George <> wrote:
    > I recently started to take an interest in Ruby programming and with
    > the help of some people and by reading Learn to program by Chris Pine
    > I made myself a little program that has come to a halt because I don't
    > know if or how I can use an external file where I store the strings I
    > want to search and replace in text type files.
    >
    > This is what I have so far:
    >
    > txt_files = Dir.glob('**/*.txt').each do |path|
    > puts path
    > txts = path.to_s
    > file = File.open(txts).readlines.each { |line|
    > if line.match(/PROMOS:/)
    > then line.gsub!(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
    > end }
    > file2=File.open(txts, "w")
    > file2.write( file )
    > end
    >
    > What I want is to be able to use an external file where I store the
    > values I want to search and replace with "line.gsub!" eg.: "/PROMOn=[A-
    > Za-z0-9]+/, 'PROMOn=some_other_text'".
    > I will need to use RubyScript2Exe because I'm not sure that on some
    > other machines I will have ruby installed and it's easier to just put
    > the strings I want to search and replace in an external file that is
    > located in the same folder as the script or on a predefined path.
    >
    > Any ideas, hints, improvements and critiques are highly welcome.


    # First line of file is regular expression; second is
    # substitution.
    temp = IO.readlines( "substitute" ).map{|s| s.chomp }
    reg_exp = Regexp.new( temp[0] )
    new_text = temp[1]

    Dir[ "**/*.txt" ].each{|path|
    p path
    changed = false
    lines = IO.readlines( path ).map{|s|
    if s =~ /PROMOS:/
    old_s = s.dup
    s.gsub!( reg_exp, new_text )
    changed = true if s != old_s
    end
    s
    }
    # Don't write to file unless there was a change.
    if changed
    File.open( path, "w" ){|f| f.puts lines }
    puts " ---> changed"
    end
    }
     
    William James, Aug 22, 2007
    #3
  4. Dan George

    Dan George Guest

    Thanks everyone for your reply.

    I'm visiting some friends at the moment so I can't try anything yet
    but as soon as I get home I'll try your suggestions, even if this will
    have to wait a few days. Maybe I can read the cookbook and docs for
    YAML till now.

    Thank you again!
     
    Dan George, Aug 25, 2007
    #4
  5. Dan George

    Dan George Guest

    On Aug 22, 5:43 pm, Stefano Crocco <> wrote:
    > Alle mercoledì 22 agosto 2007, Dan George ha scritto:
    >
    >
    >
    > > I recently started to take an interest in Ruby programming and with
    > > the help of some people and by reading Learn to program by Chris Pine
    > > I made myself a little program that has come to a halt because I don't
    > > know if or how I can use anexternalfile where I store the strings I
    > > want to search and replace in text type files.

    >
    > > This is what I have so far:

    >
    > > txt_files = Dir.glob('**/*.txt').each do |path|
    > > puts path
    > > txts = path.to_s
    > > file = File.open(txts).readlines.each { |line|
    > > if line.match(/PROMOS:/)
    > > then line.gsub!(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
    > > end }
    > > file2=File.open(txts, "w")
    > > file2.write( file )
    > > end

    >
    > > What I want is to be able to use anexternalfile where I store the
    > > values I want to search and replace with "line.gsub!" eg.: "/PROMOn=[A-
    > > Za-z0-9]+/, 'PROMOn=some_other_text'".
    > > I will need to use RubyScript2Exe because I'm not sure that on some
    > > other machines I will have ruby installed and it's easier to just put
    > > the strings I want to search and replace in anexternalfile that is
    > > located in the same folder as the script or on a predefined path.

    >
    > > Any ideas, hints, improvements and critiques are highly welcome.

    >
    > First some comments about your code:
    > * path is already a string, so calling to_s on it does nothing.
    > * You can replace the File.open(txts).readlines part with
    > File.readlines(txts), which is (in my opinion) clearer and doesn't force you
    > to remember to close the file (which by the way, you don't do).
    > * When you write to file2, you can use the block form of File.open, which
    > takes care of closing the file for you.
    >
    > Here's how what I'd have written:
    >
    > Dir.glob('**/*.txt').each do |path|
    > lines = File.readlines(path)
    > if line.match(/PROMOS:/)
    > lines.map!{|l| l.gsub(/PROMO1=[A-Za-z0-9]+/, 'PROMO1=some_text')
    > end
    > File.open(path, 'w'){|f| f.write lines}
    > end
    >
    > As you can see, I've also replaced the each/gsub! combination with map!/gsub,
    > which, in my opinion makes clearer what you're doing (changing the contents
    > of the array).
    >
    > As for storing the search/replacement pairs on a file, I'd use YAML. It's
    > included in the standard library, so there shouldn't be problems with
    > RubyScript2Exe. You can get information on yaml for ruby athttp://yaml4r.sourceforge.net/(look in particular at the cookbook and doc
    > sections). A simple example could be this:
    >
    > 'PROMO1=[A-za-z0-9]+': 'PROMO1=some_text'
    > 'PROMO2=[A-za-z0-9]+': 'PROMO1=some_other_text'
    > ...
    >
    > When read into rubyusingYAML.load, this would return the following hash:
    >
    > {
    > 'PROMO1=[A-za-z0-9]+' => 'PROMO1=some_text',
    > 'PROMO2=[A-za-z0-9]+' => 'PROMO1=some_other_text'
    >
    > }
    >
    > You could then create regexpsusingRegexp.new. (Actually, you can also store
    > the regexps directly in the yaml file, prefixing them with the
    > string !ruby/regexp, but I think the file is easier to read/write this way).
    >
    > I hope this helps.
    >
    > Stefano


    I tried what you said and it shows it works and I get an Exit code: 0
    but nothing is modified.

    Here's what I have now:

    require 'yaml'
    promo = File.open('promo.yaml')
    yp = YAML::load_documents(promo) do |item|
    txt_files = Dir.glob('**/*.txt').each do |path|
    puts path
    file = File.open(path).readlines.each { |line|
    if line.match(/PROMOS/)
    then line.gsub!(item['search'], item['sub'])
    end }
    File.open(path, 'w'){|f| f.write file}
    end
    end

    And my YAML file looks like this:

    ---
    search: /PROMO1=[A-Za-z0-9]+/
    sub: PROMO1=some_text

    Can anyone please tell me what's wrong with it?
     
    Dan George, Sep 6, 2007
    #5
  6. Alle gioved=EC 6 settembre 2007, Dan George ha scritto:
    > I tried what you said and it shows it works and I get an Exit code: 0
    > but nothing is modified.
    >
    > Here's what I have now:
    >
    > require 'yaml'
    > promo =3D File.open('promo.yaml')
    > yp =3D YAML::load_documents(promo) do |item|
    > txt_files =3D Dir.glob('**/*.txt').each do |path|
    > puts path
    > =A0 file =3D File.open(path).readlines.each { |line|
    > =A0 =A0 if line.match(/PROMOS/)
    > =A0 =A0 =A0 then line.gsub!(item['search'], item['sub'])
    > =A0 =A0 end }
    > =A0 File.open(path, 'w'){|f| f.write file}
    > end
    > end
    >
    > And my YAML file looks like this:
    >
    > ---
    > search: /PROMO1=3D[A-Za-z0-9]+/
    > sub: PROMO1=3Dsome_text
    >
    > Can anyone please tell me what's wrong with it?


    The reason your code doesn't work is that the search string is stored in th=
    e=20
    YAML file as a string, not as a regexp. To use it as a regexp, you need to=
    =20
    create a regexp from it: remove the delimiting / in the regexp in the YAML=
    =20
    file and replace the call to gsub! with:

    gsub(Regexp.new(item['search']), item['sub'])

    and it should work.=20

    Without creating the regexp, gsub! would look for a=20
    literal '/PROMO1=3D[A-Za-z0-9]+/' in the argument, not for a Regexp. (As I=
    =20
    wrote in my previous post, you can store directly a in the YAML file:

    search: !ruby/regexp /PROMO1=3D[A-Za-z0-9]+/
    ).

    By the way, you don't need to store the return value of YAML::load_document=
    s=20
    in a variable.

    I hope this helps

    Stefano
     
    Stefano Crocco, Sep 6, 2007
    #6
  7. Dan George

    Dan George Guest

    On Sep 6, 8:06 pm, Stefano Crocco <> wrote:
    > Alle giovedì 6 settembre 2007, Dan George ha scritto:
    >
    >
    >
    > > I tried what you said and it shows it works and I get an Exit code: 0
    > > but nothing is modified.

    >
    > > Here's what I have now:

    >
    > > require 'yaml'
    > > promo = File.open('promo.yaml')
    > > yp = YAML::load_documents(promo) do |item|
    > > txt_files = Dir.glob('**/*.txt').each do |path|
    > > puts path
    > > file = File.open(path).readlines.each { |line|
    > > if line.match(/PROMOS/)
    > > then line.gsub!(item['search'], item['sub'])
    > > end }
    > > File.open(path, 'w'){|f| f.write file}
    > > end
    > > end

    >
    > > And my YAML file looks like this:

    >
    > > ---
    > > search: /PROMO1=[A-Za-z0-9]+/
    > > sub: PROMO1=some_text

    >
    > > Can anyone please tell me what's wrong with it?

    >
    > The reason your code doesn't work is that the search string is stored in the
    > YAML file as a string, not as a regexp. To use it as a regexp, you need to
    > create a regexp from it: remove the delimiting / in the regexp in the YAML
    > file and replace the call to gsub! with:
    >
    > gsub(Regexp.new(item['search']), item['sub'])
    >
    > and it should work.
    >
    > Without creating the regexp, gsub! would look for a
    > literal '/PROMO1=[A-Za-z0-9]+/' in the argument, not for a Regexp. (As I
    > wrote in my previous post, you can store directly a in the YAML file:
    >
    > search: !ruby/regexp /PROMO1=[A-Za-z0-9]+/
    > ).
    >
    > By the way, you don't need to store the return value of YAML::load_documents
    > in a variable.
    >
    > I hope this helps
    >
    > Stefano


    Thanks a lot for your help Stefano! It works great!

    Cheers :)
     
    Dan George, Sep 6, 2007
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dorsa
    Replies:
    0
    Views:
    464
    Dorsa
    Dec 23, 2003
  2. Replies:
    7
    Views:
    118
    Emmanuel Oga
    Apr 4, 2007
  3. mscir
    Replies:
    0
    Views:
    317
    mscir
    Oct 12, 2005
  4. Chris Angelico
    Replies:
    9
    Views:
    234
    Andrew Cooper
    Jul 29, 2012
  5. Tim Chase
    Replies:
    10
    Views:
    385
    Robert Miles
    Aug 31, 2012
Loading...

Share This Page