crisis Perl

[cartercc]

Over the past four years, I've written a fair amount of Perl. Some of
it has been 'crisis Perl'. This results in scripts that solve a
problem but are thrown together in a hurry with inefficient, untested,
and confusing code. When the crisis resolved, I wanted to go back, do
real testing, and rewrite the script, but have been told on almost
every occasion to leave it alone. The fact that it seemed to worked
was good enough, and most of this code found its way into production.

In the past month or so, I've had to look at four scripts I have
written this way. One script was over two years old, and the newest
was several months old. Needless to say, dealing with confusing,
uncommented, and inefficient code is a problem! It would have been
much easier to clean up the code when it was written than to rewrite
it months after the fact.

I actually knew better than to not clean up the code, but it was
easier at the time not to pick a fight with my managers. This isn't an
excuse, but an explanation.

How do you deal with a manager who tells you to leave a script alone,
when you know good and well that it's such poorly written code that it
will be extremely hard to maintain, and perhaps will be buggy as well?
Getting another job isn't an option, and firing the manager isn't an
option, either.

CC

 

[Josef Moellers]

How do you deal with a manager who tells you to leave a script alone,
when you know good and well that it's such poorly written code that it
will be extremely hard to maintain, and perhaps will be buggy as well?
Getting another job isn't an option, and firing the manager isn't an
option, either.

Just give it to him in writing. If you later get into trouble, at least
you told him before! Maybe one day he'll listen.

Josef

 

[Ted Zlatanov]

c> How do you deal with a manager who tells you to leave a script alone,
c> when you know good and well that it's such poorly written code that it
c> will be extremely hard to maintain, and perhaps will be buggy as well?
c> Getting another job isn't an option, and firing the manager isn't an
c> option, either.

This is why you should not write poor, unclear code even in a crisis.
The real problem happened during the crisis, when you compromised the
quality of your work. The situation now was inevitable after the choice
you made back then.

The best you can do is build a convincing case for the necessity of a
rewrite. Focus on the bugs in the code, not its beauty. It will make
you look bad, though, since it was your code anyhow.

Next time, don't give in to the pressure of "do it now." Besides the
consequences you see above, it's also a great way to get into an even
deeper crisis if you write buggy code in haste.

Ted

 

[Andrew DeFaria]

I actually knew better than to not clean up the code, but it was
easier at the time not to pick a fight with my managers. This isn't an
excuse, but an explanation.

You should start telling your manager that you don't have the time to do
it over so you're gonna take the time to do it right in the first place!

How do you deal with a manager who tells you to leave a script alone,
when you know good and well that it's such poorly written code that
it will be extremely hard to maintain,

The quick answer is that you don't. You just fix the code. Your manager
is not charged with maintaining your code so it's not his problem - it's
your problem! You don't ask your manager for time to tidy your desk -
it's your desk and you care for it as you see fit.

Getting another job isn't an option,

Getting another job is *always* an option.

and firing the manager isn't an option, either.

Getting another job *is* firing the manager - hell the whole company!

 

[RedGrittyBrick]

Over the past four years, I've written a fair amount of Perl. Some of
it has been 'crisis Perl'. This results in scripts that solve a
problem but are thrown together in a hurry with inefficient, untested,
and confusing code. When the crisis resolved, I wanted to go back, do
real testing, and rewrite the script,

How do you deal with a manager who tells you to leave a script alone,
when you know good and well that it's such poorly written code that it
will be extremely hard to maintain, and perhaps will be buggy as well?
Getting another job isn't an option, and firing the manager isn't an
option, either.

Firstly, I'd try to practice Perl more frequently so that when a crisis
occurs, I am better able to write good Perl under pressure.

Secondly, I'd make it my aim, that in a crisis, I'd stay calm, take my
time, and attempt to write clear well-documented Perl from the start.
This is, of course, much easier said than done. However, I think that if
you make this a goal and remind yourself of it regularly, then you are
more likely to be able to achieve a result closer to your goal.

If the above is achievable then your aims won't come into conflict with
those of your manager and there won't be a problem.

It's also worth bearing in mind that your manager probably has
legitimate concerns that you don't. For example he or she might have a
budgetary or priority driven need to avoid spending resources polishing
something that is already operational and performing adequately. I'd
find out what my managers concerns were, then go away and think about
other ways those concerns might be addressed. Then you might be able to
propose some approach that meets both your concerns. At the end of the
day though, the manager has the duty and authority to manage. You have
to accept that.

Just my ¤0.02 worth.

 

[sln]

Over the past four years, I've written a fair amount of Perl. Some of
it has been 'crisis Perl'. This results in scripts that solve a
problem but are thrown together in a hurry with inefficient, untested,
and confusing code. When the crisis resolved, I wanted to go back, do
real testing, and rewrite the script, but have been told on almost
every occasion to leave it alone. The fact that it seemed to worked
was good enough, and most of this code found its way into production.

In the past month or so, I've had to look at four scripts I have
written this way. One script was over two years old, and the newest
was several months old. Needless to say, dealing with confusing,
uncommented, and inefficient code is a problem! It would have been
much easier to clean up the code when it was written than to rewrite
it months after the fact.

I actually knew better than to not clean up the code, but it was
easier at the time not to pick a fight with my managers. This isn't an
excuse, but an explanation.

How do you deal with a manager who tells you to leave a script alone,
when you know good and well that it's such poorly written code that it
will be extremely hard to maintain, and perhaps will be buggy as well?
Getting another job isn't an option, and firing the manager isn't an
option, either.

CC

First, I just want to tell you, none of the other replies to your
post you should even consider. Not even the one recommending you
write better code or learn from your mistakes.

You should not mention, nor try to coerce any response from your
manager, concerning the desire to redo something you know you could
do better the second time around, whatsover!

If your manager contacts you about a bug that comes up with your
legacy code, act shocked, don't say anything at all, look at the
code, find the bug and fix it... AND NOTHING ELSE !!!
Don't try to rewrite sections that look shabby or any crap like that.

Fix it, test it, then LET IT GO man! Give the impression to your
manager the code is otherwise perfect.

Remember, if your company does validation, you don't wan't to change
anything unless you have to after its been in production. To do so
would risk getting fired, and I'm sure you don't wan't that.

If your code falls under the umbrella of an overhaul, then by all means,
if you have the time, redo as much as you feel comfortable with.
But remember, it will have to be validated again.

I won't mean jack to your company how much you 'clean up' old code,
they only care about money ....

sln

 

[xahlee]

Over the past four years, I've written a fair amount of Perl. Some of
it has been 'crisis Perl'. This results in scripts that solve a
problem but are thrown together in a hurry with inefficient, untested,
and confusing code. When the crisis resolved, I wanted to go back, do
real testing, and rewrite the script, but have been told on almost
every occasion to leave it alone. The fact that it seemed to worked
was good enough, and most of this code found its way into production.

In the past month or so, I've had to look at four scripts I have
written this way. One script was over two years old, and the newest
was several months old. Needless to say, dealing with confusing,
uncommented, and inefficient code is a problem! It would have been
much easier to clean up the code when it was written than to rewrite
it months after the fact.

I actually knew better than to not clean up the code, but it was
easier at the time not to pick a fight with my managers. This isn't an
excuse, but an explanation.

How do you deal with a manager who tells you to leave a script alone,
when you know good and well that it's such poorly written code that it
will be extremely hard to maintain, and perhaps will be buggy as well?
Getting another job isn't an option, and firing the manager isn't an
option, either.

in general, that's how real word code works. In my experience, almost
all production code are like that. Live with it.

The time when your desire to rewrite is appreciated, is when things
broke or things need change. When that happens, they will come to you
first, or you may have already moved on.

Maybe one day you'll become a manager, and you'll probably make the
same choice.

Xah
∑ http://xahlee.org/

☄

 

[Charlton Wilbur]

cc> How do you deal with a manager who tells you to leave a script
cc> alone, when you know good and well that it's such poorly written
cc> code that it will be extremely hard to maintain, and perhaps
cc> will be buggy as well? Getting another job isn't an option, and
cc> firing the manager isn't an option, either.

Educate the manager. Keeping shoddy code in production is a gamble:
you're gambling that the cost of fixing the code *now* is higher than
the cost of fixing it when it breaks or when there's a crisis. This is
almost never the case, and a competent manager will realize this. But
the risk of fixing the code now is that you'll break something that's
working, and that's often the key factor in "if it ain't broke, don't
fix it" decisions.

Put together a test suite that tests the existing code for correct
behavior, after establishing exactly what the desired behavior is. Be
complete in your tests, and use any available tools to make sure that
the code is all tested. Then replace the scripts one by one, making
sure that they pass the test suite. You'll probably find a lot of bugs
this way.

The test suite is an asset to the code base, especially when it's
automated, because you can verify that any code you change doesn't break
anything that was previously broken. And when you *do* break something
that was previously broken but wasn't tested, you can add to the test
suite.

And then, the next time you have a crisis, you can add tests
immediately. Or, when you're doing development at a sane pace, you can
write the tests first.

Charlton

 

[Josef Moellers]

Following up on myself, after having given it some more thoughts:

Just give it to him in writing. If you later get into trouble, at least
you told him before! Maybe one day he'll listen.

Maybe the best advice, which will not help you in this case, is that you
should force yourself to write good, well documented code in the first
place. Make it a habit, even for Q&D programlets: write down what the
program does, add a comment every time you have to think even for the
fraction of a second about how to do something, use boilerplates for
subs, ... the lot.. Depending upon your workload, you'll never get the
chance to clean it up afterwards, but you'll most likely will have to
return and fix it.
Writing good, well documented code does not take much longer than
hacking together bad, sloppy, undocumented code, but the resulting code
will be of much better quality, as you will have given it a second
thought when you wrote the documentation (i.e. comments, POD, ...).

I may sound sarcastic, but maybe it's a lesson that you have learned the
hard way.

I, for one, have.

Josef

 

[cartercc]

Thank you all. Every post had something valuable, and I appreciate all
the input.

As to writing good code to begin with, it's easier said than done.
When it's almost midnight, and you've been at work for about 14 hours,
and you have people (not just your manager, but his boss, the big
boss, and the guy in charge of the project) calling you every 15
minutes, and the people you had promised things mad at you because you
were tasked with doing a job THAT THEY KNEW ABOUT FOUR WEEKS AGO BUT
DIDN'T GIVE YOU A HEADS UP (!!!) ... well, pretty code takes a back
seat.

Particularly when the guy who should be showering you with praise for
performing over and beyond the call of duty will the very next day
will blame the programmer for the project being late, omitting the
fact that the programmer wasn't even told about the project until the
day of the deadline.

My solution is to teach my manager (who came by his position as a
result of a management background and has no grounding in IT ...
couldn't program Hello World if his life depended on it) how to
program under the explanation that he should know what's involved. The
result: he hasn't learned how to program but he HAS learned how to
look at a project from a technical point of view.

    cc> How do you deal with a manager who tells you to leave a script
    cc> alone, when you know good and well that it's such poorly written
    cc> code that it will be extremely hard to maintain, and perhaps
    cc> will be buggy as well?  Getting another job isn't an option, and
    cc> firing the manager isn't an option, either.

Educate the manager.  Keeping shoddy code in production is a gamble:
you're gambling that the cost of fixing the code *now* is higher than
the cost of fixing it when it breaks or when there's a crisis.  This is
almost never the case, and a competent manager will realize this.  But
the risk of fixing the code now is that you'll break something that's
working, and that's often the key factor in "if it ain't broke, don't
fix it" decisions.

I mostly grab data from one place, massage it, and send it to another
place. The format of the data I get, and/or the format of the final
form, changes several times a year. It's not that the code breaks a
lot, but that the specifications change. Since we can't 'fix' the
specifications, we have to 'fix' the code, and guess who gets blamed
if the code isn't 'fixed.'

Thanks, CC.

 

[Charlton Wilbur]

cc> As to writing good code to begin with, it's easier said than
cc> done. When it's almost midnight, and you've been at work for
cc> about 14 hours, and you have people (not just your manager, but
cc> his boss, the big boss, and the guy in charge of the project)
cc> calling you every 15 minutes,

Ah, this is fixable. Use your backbone: "I am working on solving the
problem as fast as I can. Every minute I spend explaining the status to
four different people is a minute I am not spending solving the problem.
Every two-minute interruption because of a phone call means a
five-minute delay into getting back to where I was in the work. Until
the problem is solved, I am going to ignore the phone, and give periodic
updates to $manager, who can update $boss, $big_boss, and
$guy_in_charge."

cc> Particularly when the guy who should be showering you with
cc> praise for performing over and beyond the call of duty will the
cc> very next day will blame the programmer for the project being
cc> late, omitting the fact that the programmer wasn't even told
cc> about the project until the day of the deadline.

This sounds like a toxic work environment. If it's a poor manager, the
solution is probably to grin and bear it -- the other people who work
with him don't believe him any more than you do. If the rot goes all
the way to the top, though, it's probably time to polish your resume.

cc> I mostly grab data from one place, massage it, and send it to
cc> another place. The format of the data I get, and/or the format
cc> of the final form, changes several times a year. It's not that
cc> the code breaks a lot, but that the specifications change. Since
cc> we can't 'fix' the specifications, we have to 'fix' the code,
cc> and guess who gets blamed if the code isn't 'fixed.'

So you write your test suite to have an example of data in the format
you expect to receive and in the format you expect to produce.

And you need to start building an expectation of a certain lag between
you being notified of the specification change and the new system being
in place. If there's actually *blame* being assigned for an obvious
managerial defect, this sounds even more like a toxic work environment.

I've been there, I've done that, and I've got scars to show for it. My
solution was to painstakingly document every process I was responsible
for, so that the manager-types could see that there was 30+ pages of
information about how data was retrieved from point A, massaged, and
provided at point B. This prevented them from thinking it was trivial;
and when my resume was fully polished and I left that job, no doubt the
documentation was a great comfort to my replacement.

Charlton

 

[Andrew DeFaria]

As to writing good code to begin with, it's easier said than done.

It's really not that difficult.

When it's almost midnight, and you've been at work for about 14 hours,
and you have people (not just your manager, but his boss, the big
boss, and the guy in charge of the project) calling you every 15
minutes, and the people you had promised things mad at you because you
were tasked with doing a job THAT THEY KNEW ABOUT FOUR WEEKS AGO BUT
DIDN'T GIVE YOU A HEADS UP (!!!) ... well, pretty code takes a back seat.

You switched from "good code" to "pretty code". It's not necessarily the
same! What's that saying again? Oh yes "A lack of preparation on your
part does not constitute an emergency on my part!". Get a backbone and
tell them that. Stand your ground or quite frankly get out. Life is too
short to put up with other people's crap if they really are behaving as
you describe.

Particularly when the guy who should be showering you with praise for
performing over and beyond the call of duty will the very next day
will blame the programmer for the project being late, omitting the
fact that the programmer wasn't even told about the project until the
day of the deadline.

Such a manager should give you the time the next day to clean stuff up.
If he doesn't then get a new manager!

 

[cartercc]

You switched from "good code" to "pretty code". It's not necessarily the
same! What's that saying again? Oh yes "A lack of preparation on your
part does not constitute an emergency on my part!". Get a backbone and
tell them that. Stand your ground or quite frankly get out. Life is too
short to put up with other people's crap if they really are behaving as
you describe.

This is an academic environment. The people are pretty good,
particularly the IT dept (all hardware guys except for me). Most of
our requirements come from the academic side, faculty, deans,
provosts, vice chancellors, and the like, and they commonly don't know
the right questions to ask or sometimes even what they want!

It's a good job, little stress, lots of time off, interesting work,
the only drawback is the pay but there are always tradeoffs.

Such a manager should give you the time the next day to clean stuff up.
If he doesn't then get a new manager!

I don't think this is intentional, but more of not understanding the
process. I quite routinely get my work out real quick, and my clients
have come to expect results well within the hour of making the
request. I've automated so much of what I do that much of it is
mindlessly pushing buttons. From their point of view, if I take a long
time with something, it's obviously my fault -- but this isn't is bad
as it sounds because I'm the only guy who knows how to do what I do
(yes, we are one deep) and the praises I get sometimes are unearned.

A story: We generate several thousand contracts five times a year,
using a script I wrote. I got a panic call a couple of hours ago
because the data file had the contract amounts all as zeros. So I
started at the bottom of the stack, checking the inputs of the
functions. Yep. All zeros. Worked my way up the stack to the very top.
Zeros all the way up. Then, I looked at the input file and wouldn't
you know it -- the script expects an eleven column input file but the
actual input file contained ten columns. The reason we were getting
zeros was that the relevant column had been shifted one to the left.
Once we revised the input file to the correct format, all was fine.
Yeah, this is a routine event in the World of Work, but it's kind of
interesting that we get to deal with people issues as well as logical
and language issues, "If you would match the input file to the
documentation you would have seen that you were using the wrong input
file, and saved both yourself and me a panic attack and a wasted
morning."

CC

 

[Tad J McClellan]

die("Invalid input line, columns != 10") unless $#line == 9;

written more maintainably:

die("Invalid input line, columns != 10") unless @line == 10;

 

[cartercc]

This is a nice example to emphasize the "write good code in the first
place" point.
Your code probably looked something like this:

while(<STDIN>){
    my @line  = split("/;/");
    generate_contract($line[0], $line[3], $line[9]);

}

Very close, except that multiple columns need massaging, and the logic
to calculate the contract amounts is over 500 LOC.

By only investing 15 seconds more to write e.g.

while(<STDIN>){
    my @line  = split("/;/");
    die("Invalid input line, columns != 10") unless $#line == 9;
    generate_contract($line[0], $line[3], $line[9]);

}

Absolutely, except in this case I wrote both the Perl script and the
query code, and had the confidence that the query code was rock solid.
Which it was, but the user had inadvertently used a previous query
script. This is why I started out at the bottom -- I felt that the
database file could not be corrupted. I was wrong. :-(

you would immediately have seen what the problem is and spared
yourself from a panic attack. Then, you would have been able to spend
the morning doing something fun, e.g. replacing die() with a call to
logging framework that notifies you by email if an error happens.

Which I'm not going to do, mostly because I've used that technique a
lot in the past, and the first thing I do in the morning delete
emails. The SOP here is to generate successful emails because if
somethings happens to the Windows server (most of the automated stuff
runs on Windows) nothing will happen, and the people I work for want
to be notified of success so that if they don't get an email they will
worry.

Next time, I'll do it myself. Take five minutes, maybe, and I'll be
done with it.

CC

 

[xhoster]

A story: We generate several thousand contracts five times a year,
using a script I wrote. I got a panic call a couple of hours ago
because the data file had the contract amounts all as zeros. So I
started at the bottom of the stack, checking the inputs of the
functions. Yep. All zeros. Worked my way up the stack to the very top.
Zeros all the way up. Then, I looked at the input file and wouldn't
you know it -- the script expects an eleven column input file but the
actual input file contained ten columns. The reason we were getting
zeros was that the relevant column had been shifted one to the left.
Once we revised the input file to the correct format, all was fine.
Yeah, this is a routine event in the World of Work, but it's kind of
interesting that we get to deal with people issues as well as logical
and language issues, "If you would match the input file to the
documentation you would have seen that you were using the wrong input
file, and saved both yourself and me a panic attack and a wasted
morning."

This is a nice example to emphasize the "write good code in the first
place" point.
Your code probably looked something like this:

while(<STDIN>){
my @line =3D split("/;/");
generate_contract($line[0], $line[3], $line[9]);
}

By only investing 15 seconds more to write e.g.

while(<STDIN>){
my @line =3D split("/;/");
die("Invalid input line, columns !=3D 10") unless $#line =3D=3D 9;
generate_contract($line[0], $line[3], $line[9]);
}

you would immediately have seen what the problem is and spared
yourself from a panic attack.

If course you have the benefit of already knowing the unknown-at-the-time
failure mode. What if two columns were swapped rather than one being
missing?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[J. Gleixner]

rc wrote:
[...]

Your code probably looked something like this:

while(<STDIN>){
my @line = split("/;/");

[...]

Hopefully it was actually written better than that example.

 

[RedGrittyBrick]

A story: We generate several thousand contracts five times a year,
using a script I wrote. I got a panic call a couple of hours ago
because the data file had the contract amounts all as zeros. So I
started at the bottom of the stack, checking the inputs of the
functions. Yep. All zeros. Worked my way up the stack to the very top.
Zeros all the way up. Then, I looked at the input file and wouldn't
you know it -- the script expects an eleven column input file but the
actual input file contained ten columns. The reason we were getting
zeros was that the relevant column had been shifted one to the left.
Once we revised the input file to the correct format, all was fine.
Yeah, this is a routine event in the World of Work, but it's kind of
interesting that we get to deal with people issues as well as logical
and language issues, "If you would match the input file to the
documentation you would have seen that you were using the wrong input
file, and saved both yourself and me a panic attack and a wasted
morning."

This is a nice example to emphasize the "write good code in the first
place" point.
Your code probably looked something like this:

while(<STDIN>){
my @line =3D split("/;/");
generate_contract($line[0], $line[3], $line[9]);
}

By only investing 15 seconds more to write e.g.

while(<STDIN>){
my @line =3D split("/;/");
die("Invalid input line, columns !=3D 10") unless $#line =3D=3D 9;
generate_contract($line[0], $line[3], $line[9]);
}

you would immediately have seen what the problem is and spared
yourself from a panic attack.

If course you have the benefit of already knowing the unknown-at-the-time
failure mode. What if two columns were swapped rather than one being
missing?

I suspect a regex would often catch a greater proportion of incorrect input?

if (/^(\d{2,5}) ... ([A-Z]+)$/) {
$contract_no=$1; ...
} else {
die "Invalid input line ... ";
}

Everything is easier in hindsight. I do think it valid to ponder
validating input and consider whether that is a habit you wish to
practise such that it becomes almost automatic, even under pressure.

I find the more you practice something when there is no pressure, the
easier it becomes when there *is* pressure.

I'm pretty sure the earlier you catch bad data, the less confusing the
failure mode will be.

Just my ¤0.02 worth, I'm sure I've written *lots* of code with similar
weaknesses (or worse).

 

[Jürgen Exner]

If course you have the benefit of already knowing the unknown-at-the-time
failure mode. What if two columns were swapped rather than one being
missing?

That's why you use column headers and explicitely check their values
before processing the input file. Now, that still doesn't help if the
user puts the street address in the column with the header "Order
Number", but _that_ mistake is on a level which even a manager can
understand.

However, all that requires some thought which requires some time which
is exactly what you don't have in the typical emergency.

jue

 

[cartercc]

That's why you use column headers and explicitely check their values
before processing the input file. Now, that still doesn't help if the
user puts the street address in the column with the header "Order
Number", but _that_ mistake is on a level which even a manager can
understand.

I get a lot of files with headers, so I have developed the habit of
using the header line to build a hash table rather than throwing it
away (since it can't be used as data). $line{ADDRESS}, $line{CITY},
$line{ZIP} is a lot easier to deal with than $line[67], $line[68],
$line[69].

Unfortunately, most of my data files don't come with headers.

However, all that requires some thought which requires some time which
is exactly what you don't have in the typical emergency.

It's funny (or not) how quickly someone else's emergency can become
your own.

CC