macros: return or exit

[Marc Girod]

Hello,

I consider making a change to a module, which will significantly alter
its client interface.
Until now, client functions could exit, die, or exec: no further
action was meant to take place.
Alternatively, they could return 0, and a generic behaviour would
follow.

I want now to introduce a loop, i.e. to retain control.
I control one such client, and am not sure whether there are actually
more than one other.

My problem is however to maintain a branch of the module and its known
clients, at least in order to investigate it and possibly to get
feedback. This could take a year or so. Or more.

What I did in the main loop, is:

< my $rc = &$cmd(@ARGV);
< # ... and exit unless it returned zero.
< exit $rc if $rc;
---

my ($rc, $done) = &$cmd(@ARGV);
# ... and return unless it returned zero and not done.
return $rc if $done || $rc;

The change in the client code is thus ( a couple of examples out of
many):

< exit $?;
---

return($?, 'done');

or

return($lsp->system, 'done');

etc.
What are my best options to handle this?
I could sketch a few overrides such as e.g. for 'exit':

$legacy? exit $_ : return($_, 'done');

....but this ought to be a macro, not a function, for 'return' to
return to the right context.
What should I look for in CPAN or in the FAQ?
How to make my question shorter so that it fits in such a query?

If you are curious (you are fully entitled to), the modules are (on
CPAN):
ClearCase::Wrapper (the base module)
ClearCase::Wrapper::MGi (my own client)
ClearCase::Wrapper:SB (the other known client)

Thanks,
Marc

 

[Marc Girod]

Thanks Ben,

It's not clear what procedure you're trying to follow here. Are you
going to create your own fork of the module, and possibly try to feed
the changes upstream, or are you trying to find a way to avoid that?

It is not clear to me.
I'll make my choice based on the options.
If I maintain a branch, I won't publish it to CPAN: I'll keep it in my
Google Code site.
Until I have something good enough to convince the author?
Earlier if the solution is not invasive?
Never if it ends up shooting the performance down or similar.

The syntax $cmd->(@ARGV) is usually clearer, and generalises better to
situations like $cmd{$which}->(@ARGV).

Thanks. I agree.
I'll remember it if I push a change to the base module.

One option is to accept that the module you are using will terminate the
process, and fork before calling it in order to retain control. You will
need to think carefully about whether the exit calls are going to do
anything unpleasant to your filehandle buffers.

I don't think it is an option: I already have one fork, and I'd very
much like the forked process to remain a single shared background
server.
And it is a proprietary binary, to which I talk via stdin/stdout (I
mean that it won't 'select').

Perl doesn't have macros, I'm afraid.

This far, I knew...

If you can afford to maintain a fork of the module,
a simple search-and-replace in your text editor
would seem the easiest option.

Many nasty merges for every change I'll do...
Or generate the branched version every time? Not impossible...
But double testing... double environment, double installs...

Otherwise, one nasty option (and you're pretty much into nasty options
if you need to wrap a module with such an unpleasant interface) would be
to use the (undocumented and unexported) Want::double_return function
provided you the Want module. This sets up the next return to return
'twice', so you could do something like

Should I say that this is exactly the kind of answer I was hoping?
I have to investigate it a bit.
It even seems like an example for making my own, if I could not use
this
one precisely...

Thanks again.
Marc

 

[Uri Guttman]

MG> And it is a proprietary binary, to which I talk via stdin/stdout (I
MG> mean that it won't 'select').

huh? do you mean select as in the single arg select which handle
gets print by default? or 4 arg select which multiplexes handles for
i/o? if you mean the 4 arg, you can definitely use that on the subproc's
stdio as that is a common way to manage i/o from a forked process.

uri

 

[Marc Girod]

huh?

Sorry for the confusion.
The problem is probably that I didn't explain what binary I was
talking about.
I don't know how to make this story short, and it goes beyond our
scope.
If you are interested, you can read the docs on CPAN, for the modules
I mentioned, plus ClearCase::Argv which they all use.
But I doubt it is worth your interest unless you may try them, and
this depends on your having access to a ClearCase installation...

Marc

 

[Uri Guttman]

MG> Sorry for the confusion.
MG> The problem is probably that I didn't explain what binary I was
MG> talking about.

it is irrelevant about which binary. the issue is how do you manage its
i/o via its stdio. that is the same problem for any binary subprocess
you want to run with its stdio. now you never answered the question
about which select function you meant.

MG> I don't know how to make this story short, and it goes beyond our
MG> scope. If you are interested, you can read the docs on CPAN, for
MG> the modules I mentioned, plus ClearCase::Argv which they all use.
MG> But I doubt it is worth your interest unless you may try them, and
MG> this depends on your having access to a ClearCase installation...

no chance in hell of my having clearcase.

uri

 

[Marc Girod]

it is irrelevant about which binary. the issue is how do you manage its
i/o via its stdio. that is the same problem for any binary subprocess
you want to run with its stdio. now you never answered the question
about which select function you meant.

The C function that the cleartool binary does not use.

I was not saying my code doing select in any way. I meant its *not*
doing select.
It may be me who do not understand...
If I fork part of my script, there will be two instances sharing the
same background process.
I cannot see this work.

Does it make better sense now?

Thanks,
Marc

 

[Uri Guttman]

MG> The C function that the cleartool binary does not use.

MG> I was not saying my code doing select in any way. I meant its *not*
MG> doing select.
MG> It may be me who do not understand...
MG> If I fork part of my script, there will be two instances sharing the
MG> same background process.
MG> I cannot see this work.

MG> Does it make better sense now?

not at all. when you fork, why not pass some flag to tell one of the
procs not to deal with the background process? in the other process,
close off the handles to that background process.

uri

 

[Marc Girod]

not at all. when you fork, why not pass some flag to tell one of the
procs not to deal with the background process? in the other process,
close off the handles to that background process.

Not to deal with the background process ?
But what should it do then ?
The whole purpose is to interface it !

It could compute pi decimals, or have dinner.
Once it has a fork...

Marc

 

[Uri Guttman]

MG> Not to deal with the background process ?
MG> But what should it do then ?
MG> The whole purpose is to interface it !

MG> It could compute pi decimals, or have dinner.
MG> Once it has a fork...

i am lost as to the big picture now. i either need to read all the older
emails again or you need to explain your goals (and not your
design). you seem to want to run some binary process and manage its
i/o. why does this need to be in the background? what about a simple
fork of it and manage its i/o in an event loop in the parent? this is
fairly easy and there are modules that do all the hard lifting. if this
design isn't good enough explain why you need a more complex
architecture.

uri

 

[Marc Girod]

i am lost as to the big picture now.

I am not surprised, and I plead guilty.
But I am not writing something from scratch, and I am not going to
redesign anything too drasticly: I play as if there would be users,
and unpublished packages built upon this framework, which has been
public for many years.

So, I did describe my goals, and you did help me.
I appreciate that it is frustrating, after providing useful answers,
and already investing some effort, to be replied RTFM, or "sorry, I
won't disclose more".
It is just that I don't know where to cut: whereever I do, there will
be a next scope. For quite a long time.
Thanks.
Marc

 

[Marc Girod]

Does this seems a reasonable approach now, or is there reason this is
impossible?

I get it now: there will only be one instance.
However, it still poses some hard problems--although I can see that
you (collective of course) are not short of resources...

I'll have to take care that the pipes get properly transfered to the
child,
so that the communication goes on.
But, then, I maintain a reference count of (objects) clients of the
background server. There may be several per function, and the number
is dynamic. The last one turns the light off (send a 'quit' to the
server).
Well... the child will inherit the current count, and after it
completes, the count should drop back to where it was...?

I cannot turn this suggestion down: it seems valid too.
And maybe better than the first?
Forking is relatively expensive, though? The first should be faster?

Thanks again...
Food to thought. I should try and measure... Work... (No, that wasn't
Yuk!)

Marc

 

[Marc Girod]

    require Want;
    *Nasty::Module::exit = sub {
        my ($rv) = @_;
        Want::double_return();
        return $rv, "done";
    };

This would need to be done before loading Nasty::Module, so the override
gets properly picked up.

OK. I installed Want, and it works on a demo example but so far not in
my test case.
I had to compile without -fstack-protector, which probably requires a
newer compiler than the one I have on cygwin. It may also be that this
would not be supported on cygwin or whatever.
Anyway, as I told, my demo seems to work, and all the tests passed.

The main problem is however that I can see now that double_return is
too unflexible: exit, exec and die may be invoked at any depth level
in the call stack, not necessarily at level 2.
What I would need with this strategy is a longjump...
I may still try to write it myself...

Or to investigate the other option: based on fork...

Marc

 

[Uri Guttman]

MG> The main problem is however that I can see now that double_return is
MG> too unflexible: exit, exec and die may be invoked at any depth level
MG> in the call stack, not necessarily at level 2.
MG> What I would need with this strategy is a longjump...
MG> I may still try to write it myself...

again, i am baffled as to your big picture. one day ...

but longjump in perl is spelled die/eval. if you wrap your stack of calls
with eval blocks, you can pop up the stack with dies until you hit the
one that handles that particular die. you can tell by passing in a die
string and checking it at each eval and propogating it up if it isn't
handled there.

uri

 

[Marc Girod]

again, i am baffled as to your big picture. one day ...

'Big picture' is a philosophical assumption.
In general, there is none. What there are, are bigger pictures,
but they depend upon where you start.
So, we'd have to agree on a starting point, and learn gradually
what hidden (to us) assumptions we made, and why they break.

but longjump in perl is spelled die/eval. if you wrap your stack of calls
with eval blocks, you can pop up the stack with dies until you hit the
one that handles that particular die. you can tell by passing in a die
string and checking it at each eval and propagating it up if it isn't
handled there.

Thanks again. This is a very valid answer.
I recognize it as something I knew, and didn't think of:
shame on me! I like this sens of after-the-fact obviousness.
I can very well investigate this path...

Marc

 

[Marc Girod]

Hello,

I am only now back to this old thread of mine,
and to investigating Uri's suggestion...

I can very well investigate this path...

eval deals nicely with die, but not with exit or exec calls.

a> perl -le '$rc = eval{die"foo"};if($@){print$@}else{print$rc}'
foo at -e line 1.

a> perl -le '$rc = eval{exec q(/bin/date)};if($@){print$@}else{print
$rc}'
Sun Feb 21 19:21:49 GMTST 2010
a> perl -le '$rc = eval{exit 25};if($@){print$@}else{print$rc}'
a> echo $?
25

Do I miss something?

Marc

 

[Marc Girod]

Do I miss something?

May I override *::exit and *::exec with a function doing die?
Or may I only override Package local functions?

a> perl -le '*::exit=\$die;$rc = eval{exit 25};if($@){print$@}
else{print$rc}'
a> echo $?
25
a> perl -le '*::exec=\$die;$rc = eval{exec "foo"};if($@){print$@}
else{print$rc}'
0

Marc

 

[Marc Girod]

A piece of reusable code shouldn't normally call exit or exec (except
after a fork). What is the actual problem you are trying to solve?

I am trying to reuse a piece of code which has not been
designed to be reused in that way.
It is a wrapper around a proprietary tool, giving
command line access to a database.
The original tool is mostly used to run one-shot
commands, but it also has a mode in which it stays up
and reads commands in a loop. This mode is convenient
to start the tool in the background, and feed it with
multiple commands.
This comes handy to the kind of enhancements I want to
implement in my own wrapper, which need to perform
numerous database accesses, and gain in performance by
not having to initialize and finalize the tool for
each of them.

Now, I write my wrapper using a framework designed for
this purpose, but not supporting the 'shell' function
described above. So wrappers written so far using this
framework have been emulating the tool in one-shot
commands, but not in the ability to install itself in
the background.

I wish my wrapper to overcome this limitation.

Now, the frustration in describing what intentions
comes from the tool being proprietary, which results
in the fact that I assume you (perl gurus) could not
in general run the code examples I could give.

The database is IBM Rational ClearCase.
The CLI tool is cleartool.
The CPAN perl package giving access to cleartool is
ClearCase::Argv (itself using Argv, and IPC::Open3
among others).
The CPAN wrapper infrastructure is ClearCase::Wrapper.
My own CPAN wrapper is ClearCase::Wrapper::MGi.

The changes I try to implement are for
ClearCase::Wrapper to support the latter, and would
go more specifically to cleartool.plx.

Does this help you to consider my questions?

Thanks,
Marc

 

[Marc Girod]

May I override *::exit and *::exec with a function doing die?

Maybe a typo of mine didn't help:

release> perl -le '*::exit=\die;$rc = eval{exit "foo"};if($@){print$@}
else{print$rc}'
Died at -e line 1.
release> perl -le '*::exec=\die;$rc = eval{exec "foo"};if($@){print$@}
else{print$rc}'
Died at -e line 1.

So, at least these work.
Now, I can see that it may not be a good idea to redefine such globale
functions.
E.g. for the case when some other package below would also attempt to
redefine them (not a better idea, probably, and without much brighter
perspectives).

I can check that this work as well:

release> perl -le 'package Foo; *Foo::exit=\die;$rc = eval{exit
"foo"};if($@){print$@}else{print$rc}'
Died at -e line 1.

So, I got the answer to my own questions.
I'll try to make better ones next time.
Thanks,
Marc

 

[Marc Girod]

Err... no. You can't take a ref to a builtin. The \die expression just
calls die then and there. Also, you can only override a builtin from
within a different package, and the override must happen at compile
time.

    BEGIN { package Foo; *main::exit = sub { die } }

Thanks...
For both remarks. I guess I would soon have noticed the first, but I
didn't yet.

Note that an override for &main::exit only applies to code compiled in
package main. Other code will still see CORE::exit.

But it may be inherited?

Marc

 

[Marc Girod]

No... not unless you call ->exit as a method on a subclass. The
builtin-override logic just looks in the current package, and
CORE::GLOBAL::.

OK... Week-end, so I am back to my home work.
I set in the ClearCase::Argv.pm package (among
others...):

BEGIN {
if ($ENV{CLEARTOOL_PLX_LOOP}) {
package ClearCase::Argv;
*main::exit = sub { die @_, "\n" };
*main::exec = sub { die system(@_), "\n" };
}
}

This package does an 'exit' as part of its own
implementation of an 'exec' member.
I hope this 'ClearCase::Argv::exec' would not
collide with the 'main::exec' which we are
overriding here?

Well, I get errors:

BEGIN not safe after errors--compilation aborted at /usr/lib/
perl5/5.10/Carp/Heavy.pm line 11.
Compilation failed in require at /usr/lib/perl5/5.10/Carp.pm line 33.
Attempt to reload Carp/Heavy.pm aborted.
Compilation failed in require at /usr/lib/perl5/5.10/Carp.pm line 33.
Attempt to reload Carp/Heavy.pm aborted.
....

I read perldebug and set PERLDB_OPTS=AutoTrace
prior to starting my debug session.
This gives among the (long) output an indication
that the errors relate indeed to my change:

....
ClearCase::Argv::CODE(0x1f267b8)(/usr/lib/perl5/site_perl/5.10/
ClearCase/Argv.pm:19):
19: if ($ENV{CLEARTOOL_PLX_LOOP}) {
20: package ClearCase::Argv;
ClearCase::Argv::CODE(0x1f267b8)(/usr/lib/perl5/site_perl/5.10/
ClearCase/Argv.pm:21):
21: *main::exit = sub { die @_, "\n" };
Attempt to reload Carp/Heavy.pm aborted.
Compilation failed in require at /usr/lib/perl5/5.10/Carp.pm line 33.
ClearCase::Argv::CODE(0x1f267b8)(/usr/lib/perl5/site_perl/5.10/
ClearCase/Argv.pm:22):
22: *main::exec = sub { die system(@_), "\n" };
Attempt to reload Carp/Heavy.pm aborted.
Compilation failed in require at /usr/lib/perl5/5.10/Carp.pm line 33.
ClearCase::Argv::CODE(0x1f1b150)(/usr/lib/perl5/site_perl/5.10/
ClearCase/Argv.pm:26):
....

And I am lost again...

Marc

P.S. Why must I specify the package in the BEGIN block?
Or must I? On my command line test, it does seem so...