Re: files,folders,paths

Discussion in 'Python' started by Dave Angel, Feb 14, 2011.

  1. Dave Angel

    Dave Angel Guest

    (You forgot to include the list in your response, and you neglected to
    properly quote the message you were replying to. That makes it quite
    hard to separate your remarks from mine)

    On 02/13/2011 07:19 PM, jon hayes wrote:
    > c:\users\name\backup\ #win7
    >> c:\docs and settings\name\app data\backup\ #winxp
    >> /home/name/backup #linux and mac

    >
    > In what way do those imply a need for chdir() ? What's wrong with os.path.join() ?
    > the only commonality is the \backup
    > the chdir() is there to build the path name of the destination.
    > so c:\users\name\backup\folder1\folder2\file3
    > in my code, doing the substution.


    Well, since chdir() is illegal inside the os.walk() context and
    unnecesary, I suggest you debug your original technique. Build the
    names with os.path.join(), as I've suggested twice before, and others
    have mentioned as well.

    >
    > these 3 lines
    > os.chdir(path)
    > cwd = os.getcwd()
    > leftover = cwd.replace(source, '')
    > in this case leftover = \folder1\folder2
    > which gets added to the path for the destination in
    > shutil.copytree(os.getcwd(), destination+leftover)
    >
    > i looked at the relpath, it did not do exacly what i wanted. in version 11 of the code this works out well.


    I can't see how os.path.relpath() would be useful here.



    >
    > "And I quote from facbac-012a.py:
    > os.chdir("v:")"
    > i don't understand. is this a question?


    It is in context. If you read my message, I had just quoted you saying
    you did not change drive letters.

    >
    > "I suggest you actually try writing a simple loop using os.walk to just
    > display the names of all the files in and under a specified directory.
    > Then maybe you'll understand what path, dirs, and files actually are."
    > i started with that. look at the attached file, date-linuxversion.py.
    > the relative paths are not, quite, right.


    Well file you enclosed shows that you don't understand os.walk(). So
    learn it before trying to use it in a more complex setting. Why would
    you use the following sequence?

    filestring = str(files)
    filestring = filestring.strip("[")
    filestring = filestring.strip("]")
    filestring = filestring.strip("'")

    This is no way to process a list of filenames.


    > the thing with the copytree is, the line above is sopposed to check whether that folder exists. if it does not exist at all, then why not copy the whole thing?
    > same sort of thing with files. if not exist then do copy. that way nothing should get clobbered.
    > this is the observed behavior in version 11.
    >


    Why not copy the whole tree? Because once the code works, you'll be
    copying it many times.

    DaveA
    Dave Angel, Feb 14, 2011
    #1
    1. Advertising

  2. Dave Angel

    ecu_jon Guest

    On Feb 13, 8:48 pm, Dave Angel <> wrote:
    > (You forgot to include the list in your response, and you neglected to
    > properly quote the message you were replying to.  That makes it quite
    > hard to separate your remarks from mine)
    >
    > On 02/13/2011 07:19 PM, jon hayes wrote:
    >
    > > c:\users\name\backup\  #win7
    > >> c:\docs and settings\name\app data\backup\  #winxp
    > >> /home/name/backup  #linux and mac

    >
    > > In what way do those imply a need for chdir() ?  What's wrong with os..path.join() ?
    > > the only commonality is the \backup
    > > the chdir() is there to build the path name of the destination.
    > > so c:\users\name\backup\folder1\folder2\file3
    > > in my code, doing the substution.

    >
    > Well, since chdir() is illegal inside the os.walk() context and
    > unnecesary, I suggest you debug your original technique.  Build the
    > names with os.path.join(), as I've suggested twice before, and others
    > have mentioned as well.
    >
    >
    >
    > > these 3 lines
    > > os.chdir(path)
    > > cwd = os.getcwd()
    > > leftover = cwd.replace(source, '')
    > > in this case leftover = \folder1\folder2
    > > which gets added to the path for the destination in
    > > shutil.copytree(os.getcwd(), destination+leftover)

    >
    > > i looked at the relpath, it did not do exacly what i wanted. in version 11 of the code this works out well.

    >
    > I can't see how os.path.relpath() would be useful here.
    >
    >
    >
    > > "And I quote from facbac-012a.py:
    > >          os.chdir("v:")"
    > > i don't understand. is this a question?

    >
    > It is in context.  If you read my message, I had just quoted you saying
    > you did not change drive letters.
    >
    >
    >
    > > "I suggest you actually try writing a simple loop using os.walk to just
    > > display the names of all the files in and under a specified directory.
    > > Then maybe you'll understand what  path, dirs, and files actually are.."
    > > i started with that. look at the attached file, date-linuxversion.py.
    > > the relative paths are not, quite, right.

    >
    > Well file you enclosed shows that you don't understand os.walk().  So
    > learn it before trying to use it in a more complex setting.  Why would
    > you use the following sequence?
    >
    >      filestring = str(files)
    >      filestring = filestring.strip("[")
    >      filestring = filestring.strip("]")
    >      filestring = filestring.strip("'")
    >
    > This is no way to process a list of filenames.
    >
    > > the thing with the copytree is, the line above is sopposed to check whether that folder exists. if it does not exist at all, then why not copy the whole thing?
    > > same sort of thing with files. if not exist then do copy. that way nothing should get clobbered.
    > > this is the observed behavior in version 11.

    >
    > Why not copy the whole tree?  Because once the code works, you'll be
    > copying it many times.
    >
    > DaveA


    if i do not change the directory how do i get the relative folder
    path. for example,
    starting with c:\users\name\backup\folder1\folder2\file
    as the oswalk drills down through the folders, when it is in the
    'folder2' folder, if you do something like
    print dirs
    it would output just 'folder2' not the whole path or any of the other
    parts.
    so how do i know where to write this to on the destination relative
    to
    \\servername\username\week[1-4]\ or possibly that folder mounted as a
    drive in windows as
    v:\week[1-4]\ ?
    how do i build the output folder name.
    bear in mind it works great in version 011.
    i spent weeks, and tried maybe 7 different other ways before going
    down this path, so to speak.
    i thought about using a list. or an array. but you wouldnt know when
    oswalk starts bak from top level to go int another folder. let me
    think if i can say it another way. ...
    i think what you guys want me to do the osjoin on is this line
    destination = r"\\\\Mothera\\"+username+"\\"+weekstr or
    shutil.copytree(os.getcwd(), destination+leftover)
    please clarify.
    ecu_jon, Feb 14, 2011
    #2
    1. Advertising

  3. Dave Angel

    ecu_jon Guest

    this is a reply to both Dave Angel and Ben Finney. this version of
    testing i think incorperates what you guys are saying.
    <a href="http://thanksforallthefish.endofinternet.net/
    testing1.py">testin1.py</a>
    except maybe the os.join.path in the last function.
    here is the traceback
    Traceback (most recent call last):
    File "I:\college\spring11\capstone-project\testing.py", line 88, in
    <module>
    backupall()
    File "I:\college\spring11\capstone-project\testing.py", line 84, in
    backupall
    shutil.copy2(filesource, filedest1)
    File "C:\Python27\lib\shutil.py", line 127, in copy2
    copyfile(src, dst)
    File "C:\Python27\lib\shutil.py", line 82, in copyfile
    with open(dst, 'wb') as fdst:
    IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
    \college\\spring11\\capstone-project\\changelog.txt'

    it looks like it needs os.walk and the other for line to dig through
    the folders.
    this being the whole path
    I:\\college\\spring11\\capstone-project\\changelog.txt of source.
    ecu_jon, Feb 14, 2011
    #3
  4. Dave Angel

    ecu_jon Guest

  5. Dave Angel

    Dave Angel Guest

    On 01/-10/-28163 02:59 PM, ecu_jon wrote:
    > this is a reply to both Dave Angel and Ben Finney. this version of
    > testing i think incorperates what you guys are saying.
    > <a href="http://thanksforallthefish.endofinternet.net/
    > testing1.py">testin1.py</a>
    > except maybe the os.join.path in the last function.
    > here is the traceback
    > Traceback (most recent call last):
    > File "I:\college\spring11\capstone-project\testing.py", line 88, in
    > <module>
    > backupall()
    > File "I:\college\spring11\capstone-project\testing.py", line 84, in
    > backupall
    > shutil.copy2(filesource, filedest1)
    > File "C:\Python27\lib\shutil.py", line 127, in copy2
    > copyfile(src, dst)
    > File "C:\Python27\lib\shutil.py", line 82, in copyfile
    > with open(dst, 'wb') as fdst:
    > IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
    > \college\\spring11\\capstone-project\\changelog.txt'
    >
    > it looks like it needs os.walk and the other for line to dig through
    > the folders.
    > this being the whole path
    > I:\\college\\spring11\\capstone-project\\changelog.txt of source.
    >


    Simplify the code.


    def weekChoice()
    return 1 + (datetime.now().day -1) // 7 #weeks go from 1 to 5

    def backupall():
    source = source1()
    destination = destination2()
    print "copy tree ", source, destination
    if os.path.exists(destination):
    if os.path.isdir(destination):
    shutil.rmtree(destination)
    else:
    os.unlink(destination)
    shutil.copytree(source, destination)
    return

    All that nonsense with curdir was messing up your thinking.


    I'm not going to try to debug your destination2() function, but I would
    point out that when you use os.path.join(), please let it do the work
    for you. It takes any number of arguments, and those arguments should
    not have extra slashes or backslashes in them. The leading slashes are
    okay for the first node, if you want a unc.

    It would have been much easier if you had made source and destination
    arguments to the backupall() function. It then might have become clear
    that it's simply copytree(), with the extra requirement of deleting
    whatever used to be there.

    Now if you aren't allowed to delete what was already there, then you
    can't use copytree, and need to start over with os.walk().

    DaveA
    Dave Angel, Feb 14, 2011
    #5
  6. Dave Angel

    ecu_jon Guest

    On Feb 14, 4:43 am, Dave Angel <> wrote:
    > On 01/-10/-28163 02:59 PM, ecu_jon wrote:
    >
    >
    >
    > > this is a reply to both Dave Angel and Ben Finney. this version of
    > > testing i think incorperates what you guys are saying.
    > > <a href="http://thanksforallthefish.endofinternet.net/
    > > testing1.py">testin1.py</a>
    > > except maybe the os.join.path in the last function.
    > > here is the traceback
    > > Traceback (most recent call last):
    > >    File "I:\college\spring11\capstone-project\testing.py", line 88, in
    > > <module>
    > >      backupall()
    > >    File "I:\college\spring11\capstone-project\testing.py", line 84, in
    > > backupall
    > >      shutil.copy2(filesource, filedest1)
    > >    File "C:\Python27\lib\shutil.py", line 127, in copy2
    > >      copyfile(src, dst)
    > >    File "C:\Python27\lib\shutil.py", line 82, in copyfile
    > >      with open(dst, 'wb') as fdst:
    > > IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
    > > \college\\spring11\\capstone-project\\changelog.txt'

    >
    > > it looks like it needs os.walk and the other for line to dig through
    > > the folders.
    > > this being the whole path
    > > I:\\college\\spring11\\capstone-project\\changelog.txt of source.

    >
    > Simplify the code.
    >
    > def weekChoice()
    >      return 1 + (datetime.now().day -1) // 7   #weeks go from 1 to 5
    >
    > def backupall():
    >      source = source1()
    >      destination = destination2()
    >      print "copy tree ", source, destination
    >      if os.path.exists(destination):
    >          if os.path.isdir(destination):
    >              shutil.rmtree(destination)
    >         else:
    >              os.unlink(destination)
    >      shutil.copytree(source, destination)
    >      return
    >
    > All that nonsense with curdir was messing up your thinking.
    >
    > I'm not going to try to debug your destination2() function, but I would
    > point out that when you use os.path.join(), please let it do the work
    > for you.  It takes any number of arguments, and those arguments should
    > not have extra slashes or backslashes in them.  The leading slashes are
    > okay for the first node, if you want a unc.
    >
    > It would have been much easier if you had made source and destination
    > arguments to the backupall() function.  It then might have become clear
    > that it's simply copytree(), with the extra requirement of deleting
    > whatever used to be there.
    >
    > Now if you aren't allowed to delete what was already there, then you
    > can't use copytree, and need to start over with os.walk().
    >
    > DaveA


    and dest2 was me trying different ways to write to the sever, unc or
    mounted drive. that's for later :p
    i missed this "as missing parent directories."
    shutil.copytree(src, dst[, symlinks=False[, ignore=None]])¶

    Recursively copy an entire directory tree rooted at src. The
    destination directory, named by dst, must not already exist; it will
    be created as well as missing parent directories.

    so for testing i did the copy once. deleted the files in top folder,
    and deleted a folder deeper in.
    gave error when tried to copytree the deeper in folder.

    is there something like mkdir?
    ecu_jon, Feb 14, 2011
    #6
  7. Dave Angel

    ecu_jon Guest

    On Feb 14, 4:43 am, Dave Angel <> wrote:
    > On 01/-10/-28163 02:59 PM, ecu_jon wrote:
    >
    >
    >
    > > this is a reply to both Dave Angel and Ben Finney. this version of
    > > testing i think incorperates what you guys are saying.
    > > <a href="http://thanksforallthefish.endofinternet.net/
    > > testing1.py">testin1.py</a>
    > > except maybe the os.join.path in the last function.
    > > here is the traceback
    > > Traceback (most recent call last):
    > >    File "I:\college\spring11\capstone-project\testing.py", line 88, in
    > > <module>
    > >      backupall()
    > >    File "I:\college\spring11\capstone-project\testing.py", line 84, in
    > > backupall
    > >      shutil.copy2(filesource, filedest1)
    > >    File "C:\Python27\lib\shutil.py", line 127, in copy2
    > >      copyfile(src, dst)
    > >    File "C:\Python27\lib\shutil.py", line 82, in copyfile
    > >      with open(dst, 'wb') as fdst:
    > > IOError: [Errno 2] No such file or directory: '//Mothera/week2\\jonI:\
    > > \college\\spring11\\capstone-project\\changelog.txt'

    >
    > > it looks like it needs os.walk and the other for line to dig through
    > > the folders.
    > > this being the whole path
    > > I:\\college\\spring11\\capstone-project\\changelog.txt of source.

    >
    > Simplify the code.
    >
    > def weekChoice()
    >      return 1 + (datetime.now().day -1) // 7   #weeks go from 1 to 5
    >
    > def backupall():
    >      source = source1()
    >      destination = destination2()
    >      print "copy tree ", source, destination
    >      if os.path.exists(destination):
    >          if os.path.isdir(destination):
    >              shutil.rmtree(destination)
    >         else:
    >              os.unlink(destination)
    >      shutil.copytree(source, destination)
    >      return
    >
    > All that nonsense with curdir was messing up your thinking.
    >
    > I'm not going to try to debug your destination2() function, but I would
    > point out that when you use os.path.join(), please let it do the work
    > for you.  It takes any number of arguments, and those arguments should
    > not have extra slashes or backslashes in them.  The leading slashes are
    > okay for the first node, if you want a unc.
    >
    > It would have been much easier if you had made source and destination
    > arguments to the backupall() function.  It then might have become clear
    > that it's simply copytree(), with the extra requirement of deleting
    > whatever used to be there.
    >
    > Now if you aren't allowed to delete what was already there, then you
    > can't use copytree, and need to start over with os.walk().
    >
    > DaveA


    i thought i had it. i think im going to sleep on it.
    here is what im working with right now. jsut trying to make folders
    stuff work.
    def backupall():
    week = weekChoice()
    weekstr = "week"+str(week)
    source = source1() #was backupdir
    destination = destination1() #was remotedir
    #destinationweek = os.path.join(destination, weekstr)
    for (path, dirs, files) in os.walk(source):
    for fname in dirs:
    os.chdir(path)
    leftover = os.getcwd().replace(source, '')
    #print "leftover is:",leftover
    currentdir = destination+leftover
    if not os.path.isdir(currentdir):
    print "leftover is:",leftover
    print "currentdir1:",currentdir
    print "i should copy teh
    folderz",os.path.join(currentdir,fname)
    os.mkdir(os.path.join(currentdir,fname),0755)

    it is not making the top level folder, then tries to create deeper
    folders and fails.
    part of the traceback.
    os.mkdir(os.path.join(currentdir,fname),0755)
    WindowsError: [Error 3] The system cannot find the path specified: 'C:\
    \Users\\jon\\backup2\\week2\\configs\\squid_files'
    ecu_jon, Feb 14, 2011
    #7
  8. Dave Angel

    ecu_jon Guest

    well i think i will close this out. redid recursive foldering now for
    3rd or 4th time. below script works from a windows to another folder
    (destination1) or a unc path(destination2). i tried it on my linux
    box, and the error deals with writing to a remote smb share. i think
    if it was mounted somewhere, it would work fine with something like
    dest1, but changing the path(or mounting the remote drive to backup2
    for the lazy ).

    #!/usr/bin/python
    import wx,os,string,fnmatch,shutil,tempfile,time,getpass
    from datetime import *
    from os.path import join, getsize


    def weekChoice():
    #week = 0
    dateNum = datetime.now().day
    if dateNum <=7:
    week = 1
    elif (dateNum >= 8) and (dateNum <= 14):
    week = 2
    elif (dateNum >= 15) and (dateNum <= 21):
    week = 3
    elif dateNum > 22:
    week = 4
    else:
    print "error!!!!"
    return week

    def getusername():
    username = getpass.getuser()
    return username

    def homeDir():
    homedir = os.path.expanduser('~')
    try:
    from win32com.shell import shellcon, shell
    homedir = shell.SHGetFolderPath(0, shellcon.CSIDL_APPDATA, 0,
    0)
    except ImportError:
    homedir = os.path.expanduser("~")
    return homedir

    def source1():
    week = weekChoice()
    homedir = homeDir()
    #changed backupdir to source
    source = os.path.join(homedir, "backup")
    return source

    def destination1():
    week = weekChoice()
    homedir = homeDir()
    remotedir = os.path.join(homedir, "backup2")
    weekstr = "week"+str(week)
    destination = os.path.join(remotedir, weekstr)
    return destination

    def destination2():
    username = getusername()
    week = weekChoice()
    weekstr = "week"+str(week)
    destination = os.path.join("//Mothera",username,weekstr) #need to
    mount the smb share somewhere
    return destination

    def backupall():
    week = weekChoice()
    weekstr = "week"+str(week)
    source = source1()
    destination = destination2() # change this around to suit needs
    for (path, dirs, files) in os.walk(source):
    for fname in dirs:
    leftover = path.replace(source, '')
    #print "leftover = ",leftover
    currentdir = destination+leftover
    #print "currentdir = ",currentdir
    try:
    #print "dest folder to
    make",os.path.join(currentdir,fname)
    os.mkdir(os.path.join(currentdir,fname),0755)
    except:
    print "error folder"
    for fname in files:
    leftover = path.replace(source, '')
    #print "leftover = ",leftover
    currentdir = destination+leftover
    #print "currentdir = ",currentdir
    try:
    #print "dest file to
    make",os.path.join(currentdir,fname)

    shutil.copy2(os.path.join(path,fname),os.path.join(currentdir,fname))
    except:
    print "error file"
    backupall()
    ecu_jon, Feb 15, 2011
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mauro

    Uploading Files and Folders

    Mauro, Jan 9, 2004, in forum: ASP .Net
    Replies:
    5
    Views:
    378
    George Ter-Saakov
    Jan 9, 2004
  2. Noah
    Replies:
    5
    Views:
    760
  3. Replies:
    5
    Views:
    737
    Juan T. Llibre
    Jan 26, 2007
  4. ecu_jon

    files,folders,paths

    ecu_jon, Feb 13, 2011, in forum: Python
    Replies:
    13
    Views:
    548
    ecu_jon
    Feb 14, 2011
  5. Ohad Lutzky

    Paths, gentleman, paths

    Ohad Lutzky, Nov 6, 2006, in forum: Ruby
    Replies:
    2
    Views:
    180
    David Vallner
    Nov 7, 2006
Loading...

Share This Page