Creating lots of folders on ftp server

Discussion in 'Ruby' started by Raimon Fs, Oct 6, 2009.

  1. Raimon Fs

    Raimon Fs Guest

    Hello,


    I have to check some folders on a ftp server, if they don't exist,
    create it.


    The main folder is:

    project

    the project has some folders, like 1,2,3,4,5,...

    and each folder have 12 more folders, one for each month, and this
    folders have many folders as days the current month have.


    My code works, but I think is pretty slow and always, I'm getting an
    EOFError: end of file reached after creating 50 folders, always.

    How can I trace to find the error ?

    thanks,

    r.


    date=Date.new(2008,1,1)
    res.each{ |tuple|

    row_current+=1

    print "\n\n Row "+row_current.to_s+"/"+rows_to_fetch.to_s+" => "

    print tuple['id'], ' ', tuple['name']

    project=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
    date.month+"/"+'%02d' % date.day
    project_month=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d'
    % date.month
    project_year=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s
    project_id=ftp_acct+"/"+tuple['id']


    begin # project_id
    ftp.chdir(project_id)
    rescue Net::FTPPermError, NameError => boom
    ftp.mkdir("/"+project_id)

    print "Folder #{project_id} created"
    end


    begin # project_year
    ftp.chdir(project_year)
    rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
    create it
    ftp.mkdir("/"+project_year)
    print "Folder #{project_year} created"
    end


    for i in 1..12

    project_month=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
    i

    begin # project_month
    ftp.chdir(project_month)
    rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
    create it
    ftp.mkdir("/"+project_month)
    print "Folder #{project_month} created"
    end


    if i == 12
    d=Date.new(date.year+1,1,1)-1
    else
    d=Date.new(date.year.to_i,i+1,1)-1
    end



    puts "\n #{d.to_s}"


    for days in 1..d.day

    project_day=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
    i+"/"+'%02d' % days

    begin # project_day
    ftp.chdir(project_day)
    rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
    create it
    ftp.mkdir("/"+project_day)
    print "Folder #{project_day} created"
    end

    print " \nFolder #{project_day} exists "


    end # for days



    end # for i in 1..12


    } # res.each{ |tuple|
    --
    Posted via http://www.ruby-forum.com/.
    Raimon Fs, Oct 6, 2009
    #1
    1. Advertising

  2. Paste the error you get. You should see line numbers indicating the
    method call(s) that bailed on you.

    Incidentally I think you could do this more simply using some recursion
    and a couple more ruby idioms.

    m0wf
    --
    Posted via http://www.ruby-forum.com/.
    Chris Mowforth, Oct 6, 2009
    #2
    1. Advertising

  3. Raimon Fs

    Raimon Fs Guest

    Chris Mowforth wrote:
    > Paste the error you get. You should see line numbers indicating the
    > method call(s) that bailed on you.
    >
    > Incidentally I think you could do this more simply using some recursion
    > and a couple more ruby idioms.
    >
    > m0wf


    The error I get is from the ftp library: reach end of line

    As it starts creating folders, is getting slower and slower and finally,
    at 50, end of reach line

    and yes, I think I can do it more 'ruby like', but I'm coming from a
    really bored language ...

    :)

    thanks,

    r.
    --
    Posted via http://www.ruby-forum.com/.
    Raimon Fs, Oct 6, 2009
    #3
  4. On Mon, Oct 5, 2009 at 11:48 PM, Raimon Fs <> wrote:

    > I have to check some folders on a ftp server, if they don't exist,
    > create it.


    This may not fit your use case, but it might be easier and faster to
    create the directory structure locally and rsync it to the server.

    Just a thought.
    --
    Hassan Schroeder ------------------------
    twitter: @hassan
    Hassan Schroeder, Oct 6, 2009
    #4
  5. Raimon Fs

    Raimon Fs Guest

    Chris Mowforth wrote:
    > Paste the error you get. You should see line numbers indicating the
    > method call(s) that bailed on you.



    EOFError: end of file reached

    method readline in ftp.rb at line 211
    method getline in ftp.rb at line 211
    method getmultiline in ftp.rb at line 221
    method getresp in ftp.rb at line 235
    method voidresp in ftp.rb at line 251
    method voidcmd in ftp.rb at line 274
    method synchronize in monitor.rb at line 242
    method voidcmd in ftp.rb at line 272
    method chdir in ftp.rb at line 674
    at top level in createFoldersOnFTP.rb at line 161
    method each in createFoldersOnFTP.rb at line 156
    at top level in createFoldersOnFTP.rb at line 156
    method each in createFoldersOnFTP.rb at line 133
    at top level in createFoldersOnFTP.rb at line 133
    method each in createFoldersOnFTP.rb at line 96
    at top level in createFoldersOnFTP.rb at line 96


    maybe could be a memory leak, as it goes slower, slower, slower, and
    after 50 folders created, the error appears...


    >
    > Incidentally I think you could do this more simply using some recursion
    > and a couple more ruby idioms.


    can you give me some hints on this, please ?


    thanks,


    regards,

    r.
    --
    Posted via http://www.ruby-forum.com/.
    Raimon Fs, Oct 8, 2009
    #5
  6. Raimon Fs

    Raimon Fs Guest

    I have more information ....

    Just create an array and add elements with the folders names you want to
    create.


    ftp_folder_create.each{ |folder|

    print "\n Creating folder ",folder

    ftp.mkdir(folder)

    end


    }

    All works great, if they folders doesn't exist and you can create them.


    Add error control and check before creating the folder, if it really
    exists, and you'll get the error after creating 50 folders.


    ftp_folder_create.each{ |folder|

    print "\n Creating folder ",folder

    begin

    ftp.chdir(folder)

    rescue Net::FTPPermError, NameError => boom # it doesn't exist => create
    it

    ftp.mkdir(folder)

    end


    }


    Anyone can confirm this ?

    thanks,


    r.
    --
    Posted via http://www.ruby-forum.com/.
    Raimon Fs, Oct 8, 2009
    #6
  7. Sounds reasonable.

    If you HAVE to use ftp, personally I'd start creating a nested array
    representing the folder structure, then start trying to get Net::FTP to
    iterate through it.

    Alternatively if you could store the absolute paths as array elements, I
    think it'd be much more efficient than calling #chdir all the time. Of
    course this conflates the folder structure with the ruby array.

    And try puts rather than print, might save you some typing.
    --
    Posted via http://www.ruby-forum.com/.
    Chris Mowforth, Oct 9, 2009
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laurentiu

    Transfer folders to ftp

    Laurentiu, Jul 10, 2007, in forum: Python
    Replies:
    0
    Views:
    277
    Laurentiu
    Jul 10, 2007
  2. brad
    Replies:
    9
    Views:
    367
    Bruno Desthuilliers
    Jun 19, 2008
  3. michela rossi

    Synchronizing two ftp sites/folders

    michela rossi, Jul 8, 2003, in forum: Perl Misc
    Replies:
    4
    Views:
    81
    Perusion Hostmaster
    Jul 12, 2003
  4. D. Buck
    Replies:
    2
    Views:
    479
    D. Buck
    Jun 29, 2004
  5. coolneo
    Replies:
    9
    Views:
    186
    coolneo
    Jan 30, 2007
Loading...

Share This Page