"system" with [ ] in filename

Discussion in 'Perl Misc' started by Richard, Mar 5, 2009.

  1. Richard

    Richard Guest

    Hi all,

    this is keeping me awake for the last days. Can someone shed some
    light, or point me somewhere else if appropriate?

    (Windows XP SP1, Perl v5.8.7 MSWin32-x86-multi-thread)

    In a perl script I do this:

    $cmd = "convert \"$source\" -resize \"160x160>\"
    \"${thumbnail_prefix}${photonr}\"";
    print "Command for shell is: ".$cmd."\n";
    system ($cmd);

    This works fine, as long as ${photonr} does NOT contain any square
    brackets.
    If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    resulting system command takes an incredible lpng time to complete,
    something like 5 to 10 minutes. (it does complete in the end...)
    If no [] are present, then all is done in around half a second.

    This is printed to the screen and looks ok to me:
    Command for shell is: convert "C:\wwwroot\images\Import/104 Weekly Pix
    from Supplier/07-090225 Handicraft Fair2/IMG_8699.jpg" -resize
    "160x160>" "C:\wwwroot\images\ThumbNails\th_IMG_8699[5].jpg"

    The question now is:
    Do the [ ] have a special meaning to Perl in this context?
    Or should I look for the problem in the way Windows handles the [5] in
    the filename?
    Or anything else to investigate?
    I am sort of stuck. :(

    Some more info:
    The target directory has a lot of files, around 200.000
    I used File Monitor (Sysinternals) to trace, and see that this
    "convert" process is checking LOTS of files but can not find those.
    Its mostly looking in the current dir (where the script is). Is this
    an indication of anything?

    This has been working for over a year, but has never given this
    strange behaviour ( as far as I know !!) .

    Anyone a hint?

    Muchos thanks already,
    Richard
    Richard, Mar 5, 2009
    #1
    1. Advertising

  2. "Richard" <root@localhost> wrote in news:49b04f21$0$3118$dbd4d001
    @news.euronet.nl:

    > Hi all,
    >
    > this is keeping me awake for the last days. Can someone shed some
    > light, or point me somewhere else if appropriate?
    >
    > (Windows XP SP1, Perl v5.8.7 MSWin32-x86-multi-thread)
    >
    > In a perl script I do this:
    >
    > $cmd = "convert \"$source\" -resize \"160x160>\"
    > \"${thumbnail_prefix}${photonr}\"";
    > print "Command for shell is: ".$cmd."\n";
    > system ($cmd);
    >
    > This works fine, as long as ${photonr} does NOT contain any square
    > brackets.
    > If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    > resulting system command takes an incredible lpng time to complete,
    > something like 5 to 10 minutes. (it does complete in the end...)
    > If no [] are present, then all is done in around half a second.


    What happens if you bypass the shell?

    system convert => $source, '-resize', '160x160',
    $thumbnail_prefix.$photonr;

    Sinan

    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://www.rehabitation.com/clpmisc/
    A. Sinan Unur, Mar 5, 2009
    #2
    1. Advertising

  3. Richard

    Richard Guest

    "A. Sinan Unur" <> wrote in message
    news:Xns9BC5B9C728EF8asu1cornelledu@127.0.0.1...
    > "Richard" <root@localhost> wrote in news:49b04f21$0$3118$dbd4d001
    > @news.euronet.nl:
    >
    >> Hi all,
    >>
    >> this is keeping me awake for the last days. Can someone shed some
    >> light, or point me somewhere else if appropriate?
    >>
    >> (Windows XP SP1, Perl v5.8.7 MSWin32-x86-multi-thread)
    >>
    >> In a perl script I do this:
    >>
    >> $cmd = "convert \"$source\" -resize \"160x160>\"
    >> \"${thumbnail_prefix}${photonr}\"";
    >> print "Command for shell is: ".$cmd."\n";
    >> system ($cmd);
    >>
    >> This works fine, as long as ${photonr} does NOT contain any square
    >> brackets.
    >> If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    >> resulting system command takes an incredible lpng time to complete,
    >> something like 5 to 10 minutes. (it does complete in the end...)
    >> If no [] are present, then all is done in around half a second.

    >
    > What happens if you bypass the shell?
    >
    > system convert => $source, '-resize', '160x160',
    > $thumbnail_prefix.$photonr;
    >
    > Sinan
    >
    > --
    > A. Sinan Unur <>
    > (remove .invalid and reverse each component for email address)
    >
    > comp.lang.perl.misc guidelines on the WWW:
    > http://www.rehabitation.com/clpmisc/


    Hi,
    I have just tried to execute the command in the shell directly, and
    exactly the same thing happens :(
    So, it has now become a shell / Windows issue, not Perl.

    Thanks anyway.

    Richard
    Richard, Mar 5, 2009
    #3
  4. "Richard" <root@localhost> wrote in news:49b060e8$0$24239$dbd49001
    @news.euronet.nl:

    >
    > "A. Sinan Unur" <> wrote in message
    > news:Xns9BC5B9C728EF8asu1cornelledu@127.0.0.1...
    >> "Richard" <root@localhost> wrote in news:49b04f21$0$3118$dbd4d001
    >> @news.euronet.nl:
    >>

    ....
    >>> In a perl script I do this:
    >>>
    >>> $cmd = "convert \"$source\" -resize \"160x160>\"
    >>> \"${thumbnail_prefix}${photonr}\"";
    >>> print "Command for shell is: ".$cmd."\n";
    >>> system ($cmd);

    ....
    >>
    >> What happens if you bypass the shell?
    >>
    >> system convert => $source, '-resize', '160x160',
    >> $thumbnail_prefix.$photonr;


    [ don't quote sigs ]

    >
    > Hi,
    > I have just tried to execute the command in the shell directly, and
    > exactly the same thing happens :(
    > So, it has now become a shell / Windows issue, not Perl.


    So, did you try bypassing the shell?

    Sinan


    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://www.rehabitation.com/clpmisc/
    A. Sinan Unur, Mar 5, 2009
    #4
  5. "Richard" <root@localhost> wrote in news:49b060e8$0$24239$dbd49001
    @news.euronet.nl:

    >
    > "A. Sinan Unur" <> wrote in message
    > news:Xns9BC5B9C728EF8asu1cornelledu@127.0.0.1...
    >> "Richard" <root@localhost> wrote in news:49b04f21$0$3118$dbd4d001
    >> @news.euronet.nl:
    >>

    ....

    >>> If no [] are present, then all is done in around half a second.


    ....

    > I have just tried to execute the command in the shell directly, and
    > exactly the same thing happens :(
    > So, it has now become a shell / Windows issue, not Perl.


    cmd /?

    The special characters that require quotes are:
    <space>
    &()[]{}^=;!'+,`~

    I would still recommend using the list argument to system to bypass the
    shell.

    Sinan

    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://www.rehabitation.com/clpmisc/
    A. Sinan Unur, Mar 5, 2009
    #5
  6. Richard

    Richard Guest

    "A. Sinan Unur" <> wrote in message
    news:Xns9BC5BD6864764asu1cornelledu@127.0.0.1...
    > "Richard" <root@localhost> wrote in news:49b060e8$0$24239$dbd49001
    > @news.euronet.nl:
    >
    >>
    >> "A. Sinan Unur" <> wrote in message
    >> news:Xns9BC5B9C728EF8asu1cornelledu@127.0.0.1...
    >>> "Richard" <root@localhost> wrote in news:49b04f21$0$3118$dbd4d001
    >>> @news.euronet.nl:
    >>>

    > ...
    >>>> In a perl script I do this:
    >>>>
    >>>> $cmd = "convert \"$source\" -resize \"160x160>\"
    >>>> \"${thumbnail_prefix}${photonr}\"";
    >>>> print "Command for shell is: ".$cmd."\n";
    >>>> system ($cmd);

    > ...
    >>>
    >>> What happens if you bypass the shell?
    >>>
    >>> system convert => $source, '-resize', '160x160',
    >>> $thumbnail_prefix.$photonr;

    >
    > [ don't quote sigs ]
    >
    >>
    >> Hi,
    >> I have just tried to execute the command in the shell directly, and
    >> exactly the same thing happens :(
    >> So, it has now become a shell / Windows issue, not Perl.

    >
    > So, did you try bypassing the shell?


    system convert => $source, '-resize', '160x160>',
    $thumbnail_prefix.$photonr;

    gives exactly the same result. :(

    Richard
    Richard, Mar 5, 2009
    #6
  7. "Richard" <root@localhost> wrote in news:49b06715$0$24259$dbd49001
    @news.euronet.nl:

    >
    > "A. Sinan Unur" <> wrote in message
    > news:Xns9BC5BD6864764asu1cornelledu@127.0.0.1...
    >> "Richard" <root@localhost> wrote in news:49b060e8$0$24239$dbd49001
    >> @news.euronet.nl:
    >>
    >>>
    >>> "A. Sinan Unur" <> wrote in message
    >>> news:Xns9BC5B9C728EF8asu1cornelledu@127.0.0.1...
    >>>> "Richard" <root@localhost> wrote in news:49b04f21$0$3118$dbd4d001
    >>>> @news.euronet.nl:
    >>>>

    >> ...
    >>>>> In a perl script I do this:
    >>>>>
    >>>>> $cmd = "convert \"$source\" -resize \"160x160>\"
    >>>>> \"${thumbnail_prefix}${photonr}\"";
    >>>>> print "Command for shell is: ".$cmd."\n";
    >>>>> system ($cmd);

    >> ...
    >>>>
    >>>> What happens if you bypass the shell?
    >>>>
    >>>> system convert => $source, '-resize', '160x160',
    >>>> $thumbnail_prefix.$photonr;

    >>
    >> [ don't quote sigs ]
    >>
    >>>
    >>> Hi,
    >>> I have just tried to execute the command in the shell directly, and
    >>> exactly the same thing happens :(
    >>> So, it has now become a shell / Windows issue, not Perl.

    >>
    >> So, did you try bypassing the shell?

    >
    > system convert => $source, '-resize', '160x160>',
    > $thumbnail_prefix.$photonr;
    >
    > gives exactly the same result. :(


    I am unable to replicate the problem:

    C:\DOCUME~1\asu1\LOCALS~1\Temp\q> cat c.pl
    #!/usr/bin/perl

    use strict;
    use warnings;

    my $source = 'test.jpg';
    my $target = 'test[1].jpg';

    my @cmd = (convert => $source, -resize => '1024x1024', $target);

    print "@cmd\n";

    system @cmd;

    C:\DOCUME~1\asu1\LOCALS~1\Temp\q> timethis c.pl

    convert test.jpg -resize 1024x1024 test[1].jpg

    TimeThis : Command Line : c.pl
    TimeThis : Elapsed Time : 00:00:00.546

    C:\DOCUME~1\asu1\LOCALS~1\Temp\q> cat c.pl
    #!/usr/bin/perl

    use strict;
    use warnings;

    my $source = 'test.jpg';
    my $target = 'test[1].jpg';

    my $cmd = qq{convert $source -resize 1024x1024 $target};

    print "$cmd\n";

    system $cmd;


    C:\DOCUME~1\asu1\LOCALS~1\Temp\q> timethis c.pl

    convert test.jpg -resize 1024x1024 test[1].jpg

    TimeThis : Command Line : c.pl
    TimeThis : Elapsed Time : 00:00:00.531



    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://www.rehabitation.com/clpmisc/
    A. Sinan Unur, Mar 6, 2009
    #7
  8. Richard wrote:
    > Hi all,
    >
    > this is keeping me awake for the last days. Can someone shed some
    > light, or point me somewhere else if appropriate?
    >
    > (Windows XP SP1, Perl v5.8.7 MSWin32-x86-multi-thread)
    >
    > In a perl script I do this:
    >
    > $cmd = "convert \"$source\" -resize \"160x160>\"
    > \"${thumbnail_prefix}${photonr}\"";
    > print "Command for shell is: ".$cmd."\n";
    > system ($cmd);
    >
    > This works fine, as long as ${photonr} does NOT contain any square
    > brackets.
    > If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    > resulting system command takes an incredible lpng time to complete,
    > something like 5 to 10 minutes. (it does complete in the end...)
    > If no [] are present, then all is done in around half a second.
    >
    > This is printed to the screen and looks ok to me:
    > Command for shell is: convert "C:\wwwroot\images\Import/104 Weekly Pix
    > from Supplier/07-090225 Handicraft Fair2/IMG_8699.jpg" -resize
    > "160x160>" "C:\wwwroot\images\ThumbNails\th_IMG_8699[5].jpg"
    >
    > The question now is:
    > Do the [ ] have a special meaning to Perl in this context?
    > Or should I look for the problem in the way Windows handles the [5] in
    > the filename?
    > Or anything else to investigate?
    > I am sort of stuck. :(
    >
    > Some more info:
    > The target directory has a lot of files, around 200.000
    > I used File Monitor (Sysinternals) to trace, and see that this
    > "convert" process is checking LOTS of files but can not find those.
    > Its mostly looking in the current dir (where the script is). Is this
    > an indication of anything?
    >
    > This has been working for over a year, but has never given this
    > strange behaviour ( as far as I know !!) .
    >
    > Anyone a hint?
    >
    > Muchos thanks already,
    > Richard
    >
    >



    an easier solution would be to move this to Linux whereby you would not
    have this problem ;)
    Michael Austin, Mar 6, 2009
    #8
  9. Ben Morrow <> wrote in
    news::

    >
    > Quoth "A. Sinan Unur" <>:
    >>
    >> I would still recommend using the list argument to system to bypass
    >> the shell.

    >
    > The OP is on Win32, and you can't bypass the shell on Win32. Win32
    > doesn't have an argc/argv-style calling convention, instead parameter
    > splitting happens inside the libc startup code. system LIST under
    > Win32 will attempt to quote the arguments passed correctly, but it's
    > not any safer than system STRING with correct quoting.


    I am perplexed then:

    #!/usr/bin/perl

    use strict;
    use warnings;

    system 'gvim %temp%';

    scalar <STDIN>;

    system gvim => '%temp%';

    __END__

    The first call opens the directory browser in

    C:/Documents and Settings/asu1/Local Settings/Temp

    The second call opens and allows me to write to the file

    C:\DOCUME~1\asu1\LOCALS~1\Temp\%temp%

    I always took this behavior to mean that system with a list argument
    bypassed the shell on Win32 systemsas well.

    Sinan

    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://www.rehabitation.com/clpmisc/
    A. Sinan Unur, Mar 6, 2009
    #9
  10. Richard

    Richard Guest

    "Michael Austin" <> wrote in message
    news:4o_rl.24253$...
    > Richard wrote:
    >> Hi all,
    >>
    >> this is keeping me awake for the last days. Can someone shed some
    >> light, or point me somewhere else if appropriate?
    >>
    >> (Windows XP SP1, Perl v5.8.7 MSWin32-x86-multi-thread)
    >>
    >> In a perl script I do this:
    >>
    >> $cmd = "convert \"$source\" -resize \"160x160>\"
    >> \"${thumbnail_prefix}${photonr}\"";
    >> print "Command for shell is: ".$cmd."\n";
    >> system ($cmd);
    >>
    >> This works fine, as long as ${photonr} does NOT contain any square
    >> brackets.
    >> If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    >> resulting system command takes an incredible lpng time to complete,
    >> something like 5 to 10 minutes. (it does complete in the end...)
    >> If no [] are present, then all is done in around half a second.
    >>
    >> This is printed to the screen and looks ok to me:
    >> Command for shell is: convert "C:\wwwroot\images\Import/104 Weekly
    >> Pix from Supplier/07-090225 Handicraft Fair2/IMG_8699.jpg" -resize
    >> "160x160>" "C:\wwwroot\images\ThumbNails\th_IMG_8699[5].jpg"
    >>
    >> The question now is:
    >> Do the [ ] have a special meaning to Perl in this context?
    >> Or should I look for the problem in the way Windows handles the [5]
    >> in the filename?
    >> Or anything else to investigate?
    >> I am sort of stuck. :(
    >>
    >> Some more info:
    >> The target directory has a lot of files, around 200.000
    >> I used File Monitor (Sysinternals) to trace, and see that this
    >> "convert" process is checking LOTS of files but can not find those.
    >> Its mostly looking in the current dir (where the script is). Is
    >> this an indication of anything?
    >>
    >> This has been working for over a year, but has never given this
    >> strange behaviour ( as far as I know !!) .
    >>
    >> Anyone a hint?
    >>
    >> Muchos thanks already,
    >> Richard

    >
    >
    > an easier solution would be to move this to Linux whereby you would
    > not have this problem ;)


    Ok, deal!
    Come over and you make it work on Linux.
    The Linux machine is right next to this one, so you need only one
    chair.

    I have now narrowed it down to the "convert.exe". It is the
    ImageMagick convert.
    Maybe I try the PerlMagick module see if it makes a diff...

    Richard
    Richard, Mar 6, 2009
    #10
  11. Richard wrote:
    > "Michael Austin" <> wrote in message
    > news:4o_rl.24253$...
    >> Richard wrote:
    >>> Hi all,
    >>>
    >>> this is keeping me awake for the last days. Can someone shed some
    >>> light, or point me somewhere else if appropriate?
    >>>
    >>> (Windows XP SP1, Perl v5.8.7 MSWin32-x86-multi-thread)
    >>>
    >>> In a perl script I do this:
    >>>
    >>> $cmd = "convert \"$source\" -resize \"160x160>\"
    >>> \"${thumbnail_prefix}${photonr}\"";
    >>> print "Command for shell is: ".$cmd."\n";
    >>> system ($cmd);
    >>>
    >>> This works fine, as long as ${photonr} does NOT contain any square
    >>> brackets.
    >>> If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    >>> resulting system command takes an incredible lpng time to complete,
    >>> something like 5 to 10 minutes. (it does complete in the end...)
    >>> If no [] are present, then all is done in around half a second.
    >>>
    >>> This is printed to the screen and looks ok to me:
    >>> Command for shell is: convert "C:\wwwroot\images\Import/104 Weekly
    >>> Pix from Supplier/07-090225 Handicraft Fair2/IMG_8699.jpg" -resize
    >>> "160x160>" "C:\wwwroot\images\ThumbNails\th_IMG_8699[5].jpg"
    >>>
    >>> The question now is:
    >>> Do the [ ] have a special meaning to Perl in this context?
    >>> Or should I look for the problem in the way Windows handles the [5]
    >>> in the filename?
    >>> Or anything else to investigate?
    >>> I am sort of stuck. :(
    >>>
    >>> Some more info:
    >>> The target directory has a lot of files, around 200.000
    >>> I used File Monitor (Sysinternals) to trace, and see that this
    >>> "convert" process is checking LOTS of files but can not find those.
    >>> Its mostly looking in the current dir (where the script is). Is
    >>> this an indication of anything?
    >>>
    >>> This has been working for over a year, but has never given this
    >>> strange behaviour ( as far as I know !!) .
    >>>
    >>> Anyone a hint?
    >>>
    >>> Muchos thanks already,
    >>> Richard

    >>
    >> an easier solution would be to move this to Linux whereby you would
    >> not have this problem ;)

    >
    > Ok, deal!
    > Come over and you make it work on Linux.
    > The Linux machine is right next to this one, so you need only one
    > chair.
    >
    > I have now narrowed it down to the "convert.exe". It is the
    > ImageMagick convert.
    > Maybe I try the PerlMagick module see if it makes a diff...
    >
    > Richard
    >
    >


    The only reason that a windows box generates the [n] format is a
    half-baked attempt to create a new version of the same filename -
    similar to the VMS OS disk:[dir.subdir]file.ext;version. Most Open
    Source ports to VMS generally turns this into /disk/dir/subdir/file.ext
    where the OS will automatically return the latest "version" of the
    file unless otherwise specified and automatically increments the version
    upon "creating" a "duplicate" filename. example below: (login.com is
    similar to autoexec.bat(windows) and .profile (*N*X))

    Directory somedevice1:[sometopdir.someusername]

    LOGIN.COM;37
    LOGIN.COM;36
    LOGIN.COM;35
    LOGIN.COM;34
    LOGIN.COM;33
    LOGIN.COM;32
    LOGIN.COM;31
    LOGIN.COM;30

    Total of 8 files.


    So, the issue is not just that *Magick convert.exe has a problem dealing
    with the file name and creating a perceived performance issue, you have
    an issue where (to me at least - and I could be wrong) the file you are
    generating - say th_IMG_8699[5].jpg could potentially not be the same
    image as th_IMG_8699.jpg, th_IMG_8699[1].jpg, th_IMG_8699[2].jpg ....

    You now have a "data" issue. While these are unique, but similar names,
    they potentially do not represent the same image. I do not see your code
    where you transform/read/determine the filename to be written, but a
    simple routine to make it entirely unique - without the "[n]" would not
    be out of the question here and would solve the overall problem - giving
    you time for submitting a bug report to the *magick folks.
    Michael Austin, Mar 6, 2009
    #11
  12. Richard

    Richard Guest

    "Michael Austin" <> wrote in message
    news:3Q0sl.24266$...
    > Richard wrote:
    >> "Michael Austin" <> wrote in message
    >> news:4o_rl.24253$...
    >>> Richard wrote:
    >>>> Hi all,
    >>>>
    >>>> this is keeping me awake for the last days. Can someone shed some
    >>>> light, or point me somewhere else if appropriate?
    >>>>
    >>>> (Windows XP SP1, Perl v5.8.7 MSWin32-x86-multi-thread)
    >>>>
    >>>> In a perl script I do this:
    >>>>
    >>>> $cmd = "convert \"$source\" -resize \"160x160>\"
    >>>> \"${thumbnail_prefix}${photonr}\"";
    >>>> print "Command for shell is: ".$cmd."\n";
    >>>> system ($cmd);
    >>>>
    >>>> This works fine, as long as ${photonr} does NOT contain any
    >>>> square brackets.
    >>>> If ${photonr} turns out to be for example: IMG_8699[1].jpg then
    >>>> the resulting system command takes an incredible lpng time to
    >>>> complete, something like 5 to 10 minutes. (it does complete in
    >>>> the end...)
    >>>> If no [] are present, then all is done in around half a second.
    >>>>
    >>>> This is printed to the screen and looks ok to me:
    >>>> Command for shell is: convert "C:\wwwroot\images\Import/104
    >>>> Weekly Pix from Supplier/07-090225 Handicraft
    >>>> Fair2/IMG_8699.jpg" -resize "160x160>"
    >>>> "C:\wwwroot\images\ThumbNails\th_IMG_8699[5].jpg"
    >>>>
    >>>> The question now is:
    >>>> Do the [ ] have a special meaning to Perl in this context?
    >>>> Or should I look for the problem in the way Windows handles the
    >>>> [5] in the filename?
    >>>> Or anything else to investigate?
    >>>> I am sort of stuck. :(
    >>>>
    >>>> Some more info:
    >>>> The target directory has a lot of files, around 200.000
    >>>> I used File Monitor (Sysinternals) to trace, and see that this
    >>>> "convert" process is checking LOTS of files but can not find
    >>>> those. Its mostly looking in the current dir (where the script
    >>>> is). Is this an indication of anything?
    >>>>
    >>>> This has been working for over a year, but has never given this
    >>>> strange behaviour ( as far as I know !!) .
    >>>>
    >>>> Anyone a hint?
    >>>>
    >>>> Muchos thanks already,
    >>>> Richard
    >>>
    >>> an easier solution would be to move this to Linux whereby you
    >>> would not have this problem ;)

    >>
    >> Ok, deal!
    >> Come over and you make it work on Linux.
    >> The Linux machine is right next to this one, so you need only one
    >> chair.
    >>
    >> I have now narrowed it down to the "convert.exe". It is the
    >> ImageMagick convert.
    >> Maybe I try the PerlMagick module see if it makes a diff...
    >>
    >> Richard
    >>
    >>

    >
    > The only reason that a windows box generates the [n] format is a
    > half-baked attempt to create a new version of the same filename -
    > similar to the VMS OS disk:[dir.subdir]file.ext;version. Most
    > Open Source ports to VMS generally turns this into
    > /disk/dir/subdir/file.ext where the OS will automatically return the
    > latest "version" of the file unless otherwise specified and
    > automatically increments the version upon "creating" a "duplicate"
    > filename. example below: (login.com is similar to
    > autoexec.bat(windows) and .profile (*N*X))
    >
    > Directory somedevice1:[sometopdir.someusername]
    >
    > LOGIN.COM;37
    > LOGIN.COM;36
    > LOGIN.COM;35
    > LOGIN.COM;34
    > LOGIN.COM;33
    > LOGIN.COM;32
    > LOGIN.COM;31
    > LOGIN.COM;30
    >
    > Total of 8 files.
    >
    >
    > So, the issue is not just that *Magick convert.exe has a problem
    > dealing with the file name and creating a perceived performance
    > issue, you have an issue where (to me at least - and I could be
    > wrong) the file you are generating - say th_IMG_8699[5].jpg could
    > potentially not be the same image as th_IMG_8699.jpg,
    > th_IMG_8699[1].jpg, th_IMG_8699[2].jpg ....
    >
    > You now have a "data" issue. While these are unique, but similar
    > names, they potentially do not represent the same image. I do not
    > see your code where you transform/read/determine the filename to be
    > written, but a simple routine to make it entirely unique - without
    > the "[n]" would not be out of the question here and would solve the
    > overall problem - giving you time for submitting a bug report to the
    > *magick folks.


    Hi, thanks for the thoughts.
    The output filename $photonr is a unique / non existing filename. Just
    before this conversion it has been generated and tested by the Perl
    script itself, not by Windows.
    I have suspected a Windows mechanism here, but it turned out not to be
    the case.
    One was the generation of short-filenames, the
    shell-file-folder-extension mechanism, or wildcard issues.
    I have now excluded all of these.

    I have just discovered GriphicsMagic
    http://sourceforge.net/tracker/index.php?func=detail&aid=1878992&group_id=73485&atid=537937

    which is derived from ImageMagick.
    Its a fixed bug/issue, but I think only for the input filename, and
    not (as in my case) for the output filename.
    Will check with them, and also with ImageMagick.

    Richard.
    Richard, Mar 6, 2009
    #12
  13. On 2009-03-06, Ben Morrow <> wrote:
    >
    > Quoth "A. Sinan Unur" <>:
    >>
    >> I would still recommend using the list argument to system to bypass the
    >> shell.

    >
    > The OP is on Win32, and you can't bypass the shell on Win32.


    Of course one can.

    > Win32 doesn't have an argc/argv-style calling convention, instead
    > parameter splitting happens inside the libc startup code.


    Correct.

    > system LIST under Win32 will attempt to quote the arguments passed
    > correctly


    No. It would not "attempt to quote". It would just "quote".

    > but it's not any safer than system STRING with correct quoting.


    This does not make any sense to me. One can always implement a
    Turing machine emulator in your script, and then program the Turing
    machine. It will work as far as your emulation and the Turing machine
    program are "correct".

    The problem is HOW to make it "correct". And if you do not care about
    "how", and just want correct operation, use multi-arg system().

    [As you saw, Sinan could not find "correct" quoting to make one-arg
    call equivalent to multi-arg one...]

    Hope this helps,
    Ilya
    Ilya Zakharevich, Mar 6, 2009
    #13
  14. Richard

    ilovelinux Guest

    On 5 mrt, 22:16, "Richard" <root@localhost> wrote:

    > In a perl script I do this:
    >
    > $cmd = "convert \"$source\" -resize \"160x160>\"
    > \"${thumbnail_prefix}${photonr}\"";
    > print "Command for shell is: ".$cmd."\n";
    > system ($cmd);
    >
    > This works fine, as long as ${photonr} does NOT contain any square
    > brackets.
    > If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    > resulting system command takes an incredible lpng time to complete,
    > something like 5 to 10 minutes. (it does complete in the end...)
    > If no [] are present, then all is done in around half a second.


    The square brackets have special meaning to ImageMagick's convert. See
    http://www.imagemagick.org/script/command-line-processing.php and look
    for "Selecting Frames". Why this results in this incredibly long
    processing time, I don't know, however.
    ilovelinux, Mar 6, 2009
    #14
  15. Richard

    Richard Guest

    "ilovelinux" <> wrote in message
    news:...
    > On 5 mrt, 22:16, "Richard" <root@localhost> wrote:
    >
    >> In a perl script I do this:
    >>
    >> $cmd = "convert \"$source\" -resize \"160x160>\"
    >> \"${thumbnail_prefix}${photonr}\"";
    >> print "Command for shell is: ".$cmd."\n";
    >> system ($cmd);
    >>
    >> This works fine, as long as ${photonr} does NOT contain any square
    >> brackets.
    >> If ${photonr} turns out to be for example: IMG_8699[1].jpg then the
    >> resulting system command takes an incredible lpng time to complete,
    >> something like 5 to 10 minutes. (it does complete in the end...)
    >> If no [] are present, then all is done in around half a second.

    >
    > The square brackets have special meaning to ImageMagick's convert.
    > See
    > http://www.imagemagick.org/script/command-line-processing.php and
    > look
    > for "Selecting Frames". Why this results in this incredibly long
    > processing time, I don't know, however.
    >


    Hi,
    thank you for the link. I did not know about this...

    It also seems any [ ] in the filename causes IM to "glob", unfortunate
    in this case because there are over 200.000 images in the dir.

    I dont want to give up the [ ] for now, I'll try a temp dir as output
    and move the file afterwards.

    Thanks again.

    Richard
    Richard, Mar 6, 2009
    #15
  16. Richard wrote:
    >
    > It also seems any [ ] in the filename causes IM to "glob", unfortunate
    > in this case because there are over 200.000 images in the dir.
    >
    > I dont want to give up the [ ] for now, I'll try a temp dir as output
    > and move the file afterwards.


    Since you are creating the filename yourself inside the script, you can
    always create a filename without [] for IM and rename the file after it
    successfully complete.

    --
    -brian
    Brian Helterline, Mar 6, 2009
    #16
  17. On 2009-03-06 05:36, Ilya Zakharevich <> wrote:
    > On 2009-03-06, Ben Morrow <> wrote:
    >> Quoth "A. Sinan Unur" <>:
    >>>
    >>> I would still recommend using the list argument to system to bypass the
    >>> shell.

    >>
    >> The OP is on Win32, and you can't bypass the shell on Win32.

    >
    > Of course one can.
    >
    >> Win32 doesn't have an argc/argv-style calling convention, instead
    >> parameter splitting happens inside the libc startup code.

    >
    > Correct.
    >
    >> system LIST under Win32 will attempt to quote the arguments passed
    >> correctly

    >
    > No. It would not "attempt to quote". It would just "quote".


    I think you omitted the most important word here: "system LIST ... will
    attempt to quote ... *correctly*". I.e., what Ben tried to express is
    that it will quote, and that the quoting does attempt to preserve all
    arguments exactly but that this may not be possible in some cases (which
    seems unavoidable to me: If the parsing is done by each program instead
    of by a shell, each program may use different quoting conventions, so
    system would have to know the quoting conventions of each specific
    program).


    >
    >> but it's not any safer than system STRING with correct quoting.

    >
    > This does not make any sense to me.


    It isn't any safer because it is the same thing. Whether system does the
    quoting or the applicaton programmer does it, the result is the same (if
    they use the same quoting conventions).

    > One can always implement a Turing machine emulator in your script, and
    > then program the Turing machine. It will work as far as your
    > emulation and the Turing machine program are "correct".


    Huh? What do Turing machines have to do with the quoting conventions of
    Windows programs?

    hp
    Peter J. Holzer, Mar 7, 2009
    #17
  18. On 2009-03-07, Peter J. Holzer <> wrote:
    > attempt to quote ... *correctly*". I.e., what Ben tried to express is
    > that it will quote, and that the quoting does attempt to preserve all
    > arguments exactly but that this may not be possible in some cases (which
    > seems unavoidable to me: If the parsing is done by each program instead
    > of by a shell, each program may use different quoting conventions, so
    > system would have to know the quoting conventions of each specific
    > program).


    Quoting is done by C runtime. Unquoting is done by C runtime.

    Of course, the *intent* of quoting is to invert unquoting. So a
    correctly designed CRTL would quote in such a way that unquoting will
    produce the initial array (and visa versa).

    In principle, different CRTL could provide different (un)quoting
    rules. However, I myself know only one bullet-proof way (= no
    information loss, and allows "typical" usage of DOSISH systems):

    0) Not-in-quotes whitespace is ignored, and separates arguments;

    a) "\x25" (="\"") starts/stops quoting mode IFF it is preceeded by
    even number of backslashes (including the case of 0 backspaces).

    b) If "\"" is preceeded by 2n or 2n+1 backslashes, n backslashes are
    included in the corresponding argument.

    >>> but it's not any safer than system STRING with correct quoting.

    >>
    >> This does not make any sense to me.

    >
    > It isn't any safer because it is the same thing. Whether system does the
    > quoting or the applicaton programmer does it, the result is the same (if
    > they use the same quoting conventions).


    Sorry, I cannot read this as a statement given any thought...

    a) "quoted" 1-arg system() is NOT equivalent to multi-arg system()
    (but, given enough intelligence, it MIGHT be possible to give the
    final result of running these to be the same -- however, I do not
    remember anyone implementing this);

    b) You assume that it is humanly possible for "an application
    programmer" to do it correctly. I do not think this is a
    practically meaningful assumption.

    >> One can always implement a Turing machine emulator in your script, and
    >> then program the Turing machine. It will work as far as your
    >> emulation and the Turing machine program are "correct".


    > Huh? What do Turing machines have to do with the quoting conventions of
    > Windows programs?


    It is MUCH easier to write

    $b = sin $a;

    "correctly" than to

    a) implement an emulator of a Turing machine in Perl;

    b) write a Turing machine program which calculates sin(),

    and do it "correctly". I would even say that the second task is
    impossible without a major collection of test cases on which one would
    be able to debug the code.

    [ Writing a correct "quoter" for 1-arg system must be much easier;
    on the other hand, getting a good test suite without having the
    code for CMD.EXE may be much trickier... ]

    Hope this helps,
    Ilya
    Ilya Zakharevich, Mar 8, 2009
    #18
  19. Richard

    Marc Lucksch Guest

    Ben Morrow schrieb:
    > For a simple example, it
    > appears to me from the code that
    >
    > system q/echo/, q/"foo"/;
    >
    > does the same as
    >
    > system q/echo/, q/foo/;
    >

    No, at least not for me

    C:\Users\Maluku>perl
    system q/echo/, q/foo/;
    system q/echo/, q/"foo"/;
    __END__
    foo
    "foo"

    C:\Users\Maluku>

    (Vista, x64)

    Marc "Maluku" Lucksch
    Marc Lucksch, Mar 8, 2009
    #19
  20. On 2009-03-08, Ben Morrow <> wrote:
    >
    > Quoth Ilya Zakharevich <>:
    >> On 2009-03-07, Peter J. Holzer <> wrote:
    >> > attempt to quote ... *correctly*". I.e., what Ben tried to express is
    >> > that it will quote, and that the quoting does attempt to preserve all
    >> > arguments exactly but that this may not be possible in some cases (which
    >> > seems unavoidable to me: If the parsing is done by each program instead
    >> > of by a shell, each program may use different quoting conventions, so
    >> > system would have to know the quoting conventions of each specific
    >> > program).

    >>
    >> Quoting is done by C runtime.

    >
    > No it's not. It's done by win32.c:create_command_line.


    Really??? Hmm, I knew that Perl development goes down the drain now,
    but I would not expect this...

    > know. For another, the code makes no attempt to quote arguments with
    > special characters other than space in, so presumably


    With *correct* design, only whitespace, quote char and backslash need
    to be treated specially. The code you discuss looks like they want to
    use SHELL for multi-arg system().

    I sincerely hope that this is not the main branch of the
    implementation of system(), and is used only if the command is not
    found on PATH!

    > The code in there
    > is sufficiently convoluted, and contains a sufficient number of comments
    > about things not working entirely as documented, as to make me very
    > unsure of its correctness in every case. For a simple example, it
    > appears to me from the code that
    >
    > system q/echo/, q/"foo"/;
    >
    > does the same as
    >
    > system q/echo/, q/foo/;
    >
    > which is absolutely not the case under argc/argv systems


    Did you try it? Remove echo executable from path, and try this on
    UNIX. The result is going to be the same.

    Likewise, I sincerely hope that if echo.exe is present on PATH on
    windows system, their behavior is different on Windows (again, same as
    on UNIX)...

    >> a) "quoted" 1-arg system() is NOT equivalent to multi-arg system()
    >> (but, given enough intelligence, it MIGHT be possible to give the
    >> final result of running these to be the same -- however, I do not
    >> remember anyone implementing this);

    >
    > MakeMaker's MM->quote_literal is a decent attempt at a cross-platform
    > quoter, though it has some oddities to do with passing through make
    > variables.


    There is ABSOLUTELY no way to do cross-platform quoting. To have a
    SINGLE PLATFORM quoting is supposed to go through shell, you need to
    have the complete documentation of how the shell treats its metachars.

    I would be very surprised if CMD.EXE had such documentation (even IF
    its behaviour would NOT depend on the version of CMD.EXE and a lot of
    configuration variables [same shit as with tcsh]).

    >> b) You assume that it is humanly possible for "an application
    >> programmer" to do it correctly. I do not think this is a
    >> practically meaningful assumption.


    > It's obviously possible to get any given case right.


    For me, it is obviuosly impossible without a lot of knowledge (unless
    you assume Bourne semantic, which was SPECIALLY designed to allow
    simple quoting)
    s/'/'"'"'/g for @foo;
    q(') . join(q[' '], @foo) . q(').

    All other cases (including csh derivatives) are, AFAIK, hopeless to
    get bullet-proof.

    > My point was that even multi-arg system still isn't *safe* under Win32,
    > and still doesn't guarantee cmd.exe (and/or whatever you've got in
    > %PERL5SHELL%) won't be invoked.


    It was always my POV that

    *) multi-arg system() should NEVER involve shell (the programmer
    can, of course, prepend $ENV{COMPSEC}, q(/c) to the @args), AND

    *) that one should a port of Bourne shell (and default to it for
    one-arg system()) on all non-UNIXISH platforms.

    (This is what I do on OS/2 port I maintain.) As you see, this POV was
    consistently ignored...

    Thanks for correcting me,
    Ilya
    Ilya Zakharevich, Mar 10, 2009
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?Sm9l?=

    Extract filename from a filename typed by user

    =?Utf-8?B?Sm9l?=, Aug 23, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    1,016
    Travis Murray
    Aug 24, 2004
  2. Replies:
    1
    Views:
    1,460
    Roland de Ruiter
    Jun 15, 2006
  3. Ed
    Replies:
    10
    Views:
    45,835
    alok000707
    Jul 13, 2010
  4. Beauregard T. Shagnasty

    Re: filename.gif or filename.gif.jpg?

    Beauregard T. Shagnasty, May 30, 2008, in forum: HTML
    Replies:
    1
    Views:
    743
    Jonathan N. Little
    May 30, 2008
  5. Bergamot
    Replies:
    0
    Views:
    450
    Bergamot
    May 30, 2008
Loading...

Share This Page