perl to fortran thoughts

[Jim Burns]

Hi all,

I need some feedback on an issue. I use a perl script provided by a third
party that processes an html form. When I recreated the html form on my
end, initially identical to the third party form, things worked perfectly.
But then I set out to rearrange the html form elements in a better order on
the web page and the form broke inconsistently.

In chasing down this problem I discovered the form /never/ broke if the html
form elements were in the original order.

Now this has be terribly confused. I've been writing software for 15 yrs
and this immediately bothered me since the UI layers input elements should
have no effect on any backend processing.

But, allowing that I'm not web/cgi/perl expert, I must be missing something
on the technical end here.

My premise is that since any perl script would decode the input to get
access to the $name,$value pairs that make up the html's UI control names
and data, that it was unconscionable that the perl script ignore the UI
control names and process the data in an expected order. This is just
against all best practices in my experience. After all, in a normal
programming environment we don't tend to have data outside variables to hold
on to them with (variables, references, memory addresses).

I wrote the third party nicely asking for help and explaining what I thought
was the problem and the short answer I received was,

I briefly examined this perl script and determined tthat, in
this case, the order is important because the perl is piping
the input directly into a Fortran program, which is doing the
real work.

Ok so the only way I can see this explanation working is if the perl script
is somehow, literally, a mere conduit for the browser input and passes it on
to this fortran program without ever looking at, considering or touching the
data. This would be like using data without the variables that tell us the
type of that data or what that data is for.

Can this be done? I don't think so, can anyone provide some explanation if
so?

And, even so, then aren't we still back at the same point and doesn't my
same assertion hold? If the perl script merely streams the input from the
browser directly into another process then logically one could subtract the
perl script from the analysis and assert that the fortran program should be
properly decoding the input, using the UI element names AND their data and
processing the input properly based on the data name rather than expecting a
certain order.

In the end I just can't seem to justify how this third party can suggest
that it's not good practice at their end to be handling the input in this
way. And I assert there can be no way in which they MUST do it this way.

Thoughts?

If you want to see an example, you can visit,

my page: <http://www.ascentaviation.net/test_sunmoon.php>
their original: <http://aa.usno.navy.mil/data/docs/RS_OneDay.html#forma>


Many, many thanks in advance.


Jim

 

[Bart Van der Donck]

I need some feedback on an issue. I use a perl script provided by a third
party that processes an html form. When I recreated the html form on my
end, initially identical to the third party form, things worked perfectly.
But then I set out to rearrange the html form elements in a better order on
the web page and the form broke inconsistently.
[...]

One workaround is to define 2 forms on your web page: one intended for
screen, and one intended for the submit action. Both forms can then be
populated with elements in the order you like. Something like this:

<!-- form that is submitted -->
<form method="get" action="http://www.google.com/" name="f1">
<input type="hidden" name="year">
<input type="hidden" name="month">
<input type="hidden" name="day">
<input type="hidden" name="state">
<input type="hidden" name="city">
</form>
<!-- form shown on screen -->
<form method="get" onSubmit="return false;" name="f2">
<input type="text" name="day">
<input type="text" name="month">
<input type="text" name="year">
<input type="text" name="city">
<input type="text" name="state">
<input type="button" value="Get data" onClick="
document.forms['f1'].year.value = document.forms['f2'].year.value;
document.forms['f1'].month.value = document.forms['f2'].month.value;
document.forms['f1'].day.value = document.forms['f2'].day.value;
document.forms['f1'].city.value = document.forms['f2'].city.value;
document.forms['f1'].state.value = document.forms['f2'].state.value;
document.forms['f1'].submit();
">
</form>

Hope this helps,

 

[Greger]

Hi all,

I need some feedback on an issue. I use a perl script provided by a third
party that processes an html form. When I recreated the html form on my
end, initially identical to the third party form, things worked perfectly.
But then I set out to rearrange the html form elements in a better order
on the web page and the form broke inconsistently.

In chasing down this problem I discovered the form /never/ broke if the
html form elements were in the original order.

Now this has be terribly confused. I've been writing software for 15 yrs
and this immediately bothered me since the UI layers input elements should
have no effect on any backend processing.

But, allowing that I'm not web/cgi/perl expert, I must be missing
something on the technical end here.

My premise is that since any perl script would decode the input to get
access to the $name,$value pairs that make up the html's UI control names
and data, that it was unconscionable that the perl script ignore the UI
control names and process the data in an expected order. This is just
against all best practices in my experience. After all, in a normal
programming environment we don't tend to have data outside variables to
hold on to them with (variables, references, memory addresses).

I wrote the third party nicely asking for help and explaining what I
thought was the problem and the short answer I received was,

I briefly examined this perl script and determined tthat, in
this case, the order is important because the perl is piping
the input directly into a Fortran program, which is doing the
real work.

Ok so the only way I can see this explanation working is if the perl
script is somehow, literally, a mere conduit for the browser input and
passes it on to this fortran program without ever looking at, considering
or touching the
data. This would be like using data without the variables that tell us
the type of that data or what that data is for.

Can this be done? I don't think so, can anyone provide some explanation
if so?

And, even so, then aren't we still back at the same point and doesn't my
same assertion hold? If the perl script merely streams the input from the
browser directly into another process then logically one could subtract
the perl script from the analysis and assert that the fortran program
should be properly decoding the input, using the UI element names AND
their data and processing the input properly based on the data name rather
than expecting a certain order.

In the end I just can't seem to justify how this third party can suggest
that it's not good practice at their end to be handling the input in this
way. And I assert there can be no way in which they MUST do it this way.

Thoughts?

If you want to see an example, you can visit,

my page: <http://www.ascentaviation.net/test_sunmoon.php>
their original: <http://aa.usno.navy.mil/data/docs/RS_OneDay.html#forma>


Many, many thanks in advance.


Jim

if the order of elements really is dependent of the successful outcome of
the fortran program then throw it away. No matter if you POST or GET to the
program the order of parameters should not matter. If you just want to
process a simple form then this is done dead simple in perl, and also php
of course.

 

[Greger]

if the order of elements really is dependent of the successful outcome of
the fortran program then throw it away. No matter if you POST or GET to
the program the order of parameters should not matter. If you just want to
process a simple form then this is done dead simple in perl, and also php
of course.

errr, the other way*grin*

 

[Jim Burns]

One workaround is to define 2 forms on your web page: one intended for
screen, and one intended for the submit action. Both forms can then be
populated with elements in the order you like. Something like this:

Very cool idea, thanks. It doesn't help me understand why the USNO wants to
assert the way it's processing the form MUST be done that way but I like the
workaround.


/jim

 

[Jim Burns]

I need some feedback on an issue. I use a perl script provided by a third
....

In the end I just can't seem to justify how this third party can suggest
that it's not good practice at their end to be handling the input in this
way. And I assert there can be no way in which they MUST do it this way.

Thanks for the responses so far. I replied to my message because in
re-reading it I noticed this error in the above paragraph,

In the end I just can't seem to justify how this third
party can suggest that it is good practice at their end
to be handling the input in this way.

I had my logic backwards.

As far as my complaint and Greger's response goes, I think we're pretty much
seeing it the same way. But, while I usually speak to things directly and
am a stickler for the abolutes of best practices first and the discernment
to known when to bend or break them (rather than the ugly alternative of
sloppy first and tidying up later as needed), I've learned that there's
usually no absolute that usually can't be broken, either by re-thinking or
re-defining something. (see Bart's idea. Neat!)

But I'm just trying to see what I may not be seeing. Could it be simply a
matter of them being dependent upon the ill-designed fortran script without
any power or inclination to fix or modify it? I believe it could be. But I
also received,

Also, we have a very limited staff and resources, so we
cannot allocate much effort to research a problem that
appears to be on your end

which is primarilly what drives me to want to provide a thoughtful response.
So far from what I've been able to determine, it's fundamentally wrong to
suggest it's a problem "at my end."

Thanks for any additional thoughts,


/jim

 

[xhoster]

Hi all,

I need some feedback on an issue. I use a perl script provided by a
third party that processes an html form. When I recreated the html form
on my end, initially identical to the third party form, things worked
perfectly. But then I set out to rearrange the html form elements in a
better order on the web page and the form broke inconsistently.

In chasing down this problem I discovered the form /never/ broke if the
html form elements were in the original order.

If you hire and pay the programmers, you can fire them for not following
best (or even good) practices. Otherwise, tough tatas. It is what it is.

Xho

 

[Bart Van der Donck]

[...]
But I'm just trying to see what I may not be seeing. Could it be simply a
matter of them being dependent upon the ill-designed fortran script without
any power or inclination to fix or modify it? I believe it could be. But I
also received,

Also, we have a very limited staff and resources, so we
cannot allocate much effort to research a problem that
appears to be on your end

which is primarilly what drives me to want to provide a thoughtful response.
So far from what I've been able to determine, it's fundamentally wrong to
suggest it's a problem "at my end."

The problem is not at your end. The Perl script should read out the
offered data by parsing the name/value pairs; and that procedure should
not depend on the order of those pairs (which would be at least risky
IMO).

But there's a chance Perl does the job right. It could maybe go wrong
when Perl passes the data further to the Fortran program. Difficult to
tell without looking at the code, of course.

 

[Jim Burns]

If you hire and pay the programmers, you can fire them for not following
best (or even good) practices. Otherwise, tough tatas. It is what it is.

Well that's a childish remark. For starters, I'm trying to understand their
response; I'm allowing that they see something I don't. So on that level
alone your response is valueless as it has nothing to do with what I've
actually asked of this group. Great job!

Moreover, there's nothing wrong with me providing feedback to the third
party as to what I see wrong with their script and its form processing.
Again, perfectly valid and acceptable. Which begs the question of whether
sticking your nose in and not contributing to the point is merely for your
warped enjoyment. Again, great job!


/jim

 

[Sherm Pendley]

[...]
But I'm just trying to see what I may not be seeing. Could it be simply a
matter of them being dependent upon the ill-designed fortran script without
any power or inclination to fix or modify it? I believe it could be. But I
also received,

Also, we have a very limited staff and resources, so we
cannot allocate much effort to research a problem that
appears to be on your end

which is primarilly what drives me to want to provide a thoughtful response.
So far from what I've been able to determine, it's fundamentally wrong to
suggest it's a problem "at my end."

The problem is not at your end. The Perl script should read out the
offered data by parsing the name/value pairs; and that procedure should
not depend on the order of those pairs (which would be at least risky
IMO).

It sounds to me like the Perl script is written to loop through the request
inputs with something like this:

foreach $input ( @{$req->params} ) {

That is, it sounds like it's getting the list of input names from the form
itself, instead of using a hard-coded list. It also sounds (to me) like the
downstream Fortran app was written to rely on the arbitrary ordering that
happened to appear in the first version of the form.

The *correct* solution would obviously to fix the broken Fortran so that it's
not dependent on the order of the input items. Apparently that's not possible,
maybe even for political, financial or other non-technical reasons.

So, a workaround might be to simply hard-code the list of inputs to pass
along, in the order in which the Fortran app expects them. If you go with
this solution, an explanatory comment is definitely in order! For example:

# Warning! Change this list at your own risk!
# Inputs must be passed to the Fortran app in this exact order
foreach $input ( qw/foo bar baz/ ) {

sherm--

 

[xhoster]

Well that's a childish remark.

Ask a childish question...

For starters, I'm trying to understand
their response;

What is there to understand? You want them to fix their code, they don't
want to.

I'm allowing that they see something I don't. So on that
level alone your response is valueless as it has nothing to do with what
I've actually asked of this group. Great job!

I answered what seemed to be your question. No, it isn't good practise,
but it is what it is. Isn't that you were asking?

Moreover, there's nothing wrong with me providing feedback to the third
party as to what I see wrong with their script and its form processing.
Again, perfectly valid and acceptable.

Feel free to provide your feedback to them. But why you are providing your
feedback to them to us? We are not them.


Xho

 

[Jim Burns]

It sounds to me like the Perl script is written to loop through the request
inputs with something like this:

foreach $input ( @{$req->params} ) {

That is, it sounds like it's getting the list of input names from the form
itself, instead of using a hard-coded list. It also sounds (to me) like the
downstream Fortran app was written to rely on the arbitrary ordering that
happened to appear in the first version of the form.

The *correct* solution would obviously to fix the broken Fortran so that it's
not dependent on the order of the input items. Apparently that's not possible,
maybe even for political, financial or other non-technical reasons.

As I thought about it, not being good with perl, I figured there was a way
via an associative array to peel off the values of the input without regard
to the names of the UI controls, the variables if you will, that provide
context and meaning.

Alternatively, in my limited perl experience, I've also decoded the input
and grabbed BOTH the $name and $value parts of the input. In php I'd simply
refer to $_REQUEST['control_name'] to retrieve the data for that
control_name.

So, even with the fortran, if the perl script decodes the form input but, as
I descibed first, ignores the $name portion, then I still fault the perl
script since whether or not the fortran requires a particular order the perl
script in this case only plays into this by passing the need to handle the
order up to the UI level.

We call on services all the time, be they simply library calls or OS
services, etc. And the order of our parameters to a function matters.
That's not the problem in and of itself. But if the perl script does decode
the input it should've been written to fully pull apart the names and value
from the input (UI) and call the fortran correctly. Right?


Alternatively, based on the response I got, I was allowing that somehow the
perl script was literally doing /nothing/ but passing on the input. Now,
not being an expert in what this data stream really looks like, I'll just
accept it, but then that means the fortran code a) understands the input
from the web browser, which at least generally dates it with regard to any
expectation that it's still maintainable and b) if a is true, begs the
question of why the need for the perl script in the middle, and c) now has
the responsibility to parse the input.


If it's not clear, my /reason/ for writing was to understand the response I
received from them since I have trouble believing it's accurate. Aside from
assuming the third party no longer can maintain the fortran code, is there
anything I may be missing to suggest that what they're telling me doesn't
constitute bad decision making on their end. After all, they've blamed me,
which is as logically short-sighted as attacking me for trying to understand
what's going on here. Quite simply I am considering their side and trying
to find any way which might validate their assertion that it's at my end or,
by implication, that I'm simply asking for too much (i.e. the simpleton
response of "it is what it is"). For them to have come this far in
providing this script but then constrain me to a particular order at the UI
level is not only bad design but constraining. I /am/ able to assume they
/want/ to provide the service since they are, so it should not be
unacceptable to provide them with constructive criticism on that service or
how it might be better provided. Forcing the UI layer to maintain a
specific order to that the data is passed from browser to server in a
particular order is almost offensive. In fact, is it even wise with regard
to the specification that one place a dependency indirectly upon the
browser's form layout by placing a dependency on the order that form data is
passed to the server? I've not consulted the spec on this but I'd guess
it's not.


Thanks,

/jim

 

[xhoster]

As I thought about it, not being good with perl, I figured there was a
way via an associative array to peel off the values of the input without
regard to the names of the UI controls, the variables if you will, that
provide context and meaning.

Sure, that is possible, but you probably wouldn't use an associative array
to do it that way.

Alternatively, in my limited perl experience, I've also decoded the input
and grabbed BOTH the $name and $value parts of the input. In php I'd
simply refer to $_REQUEST['control_name'] to retrieve the data for that
control_name.

Sure, something like that is also possible, using CGI.pm for example.

So, even with the fortran, if the perl script decodes the form input but,
as I descibed first, ignores the $name portion, then I still fault the
perl script since whether or not the fortran requires a particular order
the perl script in this case only plays into this by passing the need to
handle the order up to the UI level.
OK.

We call on services all the time, be they simply library calls or OS
services, etc. And the order of our parameters to a function matters.
That's not the problem in and of itself. But if the perl script does
decode the input it should've been written to fully pull apart the names
and value from the input (UI) and call the fortran correctly. Right?

In an ideal world, sure.

Alternatively, based on the response I got, I was allowing that somehow
the perl script was literally doing /nothing/ but passing on the input.
Now, not being an expert in what this data stream really looks like, I'll
just accept it, but then that means the fortran code a) understands the
input from the web browser,

Well, understand it well enough to work under a certain restricted set of
circumstances.

which at least generally dates it with regard
to any expectation that it's still maintainable and

I have no idea what that means.

b) if a is true, begs
the question of why the need for the perl script in the middle,

Maybe there is no need for it but they did it that way because they were
used to writing Perl CGI and not Fortran CGI. Maybe they use the Perl to
format the output or something.

and c)
now has the responsibility to parse the input.

Yeah, OK, for sufficiently loose meanings of "responsibility".

If it's not clear, my /reason/ for writing was to understand the response
I received from them since I have trouble believing it's accurate.

There is no particular reason to think it is not, and there is no
particular reason to think that it is.

Aside
from assuming the third party no longer can maintain the fortran code, is
there anything I may be missing to suggest that what they're telling me
doesn't constitute bad decision making on their end.

I don't think they ever invited you to critique their decision making.
And if the decision was between "slap something together in 2 hours, or
don't do it at all" they very well may have made the right decision.

After all, they've
blamed me,

They said the problem was on your end, which is not the same thing as
blaming you. And they are right. Their code works from their form. It
doesn't work from your form. In a perfect world, it would work from your
form, but that doesn't change the fact that the problem is illicited by
your form and not by theirs. Especially consider that we've seen nothing
to suggest that they encourage or welcome other people's forms pointing to
their CGI.

which is as logically short-sighted as attacking me for trying
to understand what's going on here. Quite simply I am considering their
side and trying to find any way which might validate their assertion that
it's at my end or, by implication, that I'm simply asking for too much

If you are truly considering their side, then how much validation does it
take that you are asking too much other than them saying that you are
asking too much?

(i.e. the simpleton response of "it is what it is"). For them to have
come this far in providing this script but then constrain me to a
particular order at the UI level is not only bad design but constraining.
I /am/ able to assume they /want/ to provide the service since they are,

You are justified in assuming they want to provide that service *from their
form*. That is all. Or at least that is all you have shown us.

so it should not be unacceptable to provide them with constructive
criticism on that service or how it might be better provided.

You have done so, they responded, and now you are whining here because you
didn't like their response.

Forcing
the UI layer to maintain a specific order to that the data is passed from
browser to server in a particular order is almost offensive. In fact, is
it even wise with regard to the specification that one place a dependency
indirectly upon the browser's form layout by placing a dependency on the
order that form data is passed to the server? I've not consulted the
spec on this but I'd guess it's not.

I haven't consulted the spec either, but it probably isn't wise.

Xho

 

[Jim Burns]

On 06 Jul 2006 18:40:36 GMT, (e-mail address removed) wrote:

--snip--

I started to answer your comments inline but as I trimmed the reply,
deciding what I wanted to respond to, I found myself realizing you've turned
this thread toward the irrelevant BS you seem to want to bitch about.
Through your replies you've misrepresented, whether intentionally or
ignorantly, the context, full meaning and intention of my remarks. Ignoring
my explanations, you've created your own contexts to discuss points of your
own design and derailed this thread. In contrast, I've found every other
reply helpful, maybe you should take note.

In my conversation it doesn't matter whether they "invited" my critique or
not, you're just being an ass here. Their abbreviated response is not
accurate your drivel on the matter notwithstanding. Of course their code
works on their form, everything balanced just as they expect, I never argued
this but the software is still flawed and to consider fixing it is valid
your philosophizing tripe aside. They don't have to change or fix their
form at all, I've never argued otherwise but a design bug is a design bug
regardless of your blather and they opened further discussion by providing
an inaccurate, or at the very least dismissive response. The difference
between the best decision and a right decision is above your head I see.

You have done so, they responded, and now you are whining here because you
didn't like their response.

Actually, I'm not whining at all, I'm just exploring the issue /I/ wanted to
explore. And since I have to make some assumptions about what their real
constraints may be I came here first. And unless you're planning a trip to
Houston and are asking for my address please don't start puffing out your
chest. And unless that's the case, I'm done with you.


/jim

 

[axel]

As I thought about it, not being good with perl, I figured there was a way
via an associative array to peel off the values of the input without regard
to the names of the UI controls, the variables if you will, that provide
context and meaning.

No need to even involve a hash... it just needs a QUERY_STRING
(assuming that method is being used) to be split and the values
then split from the names. From the sounds of it this or something
similar is what is happening.

Have you tried experimenting with the form... e.g. keeping the original
order of the elements and seeing if the application still works?

So, even with the fortran, if the perl script decodes the form input but, as
I descibed first, ignores the $name portion, then I still fault the perl
script since whether or not the fortran requires a particular order the perl
script in this case only plays into this by passing the need to handle the
order up to the UI level.

It is obviously a very badly written Perl CGI script and probably has
little or no data verification... which into question its security.

We call on services all the time, be they simply library calls or OS
services, etc. And the order of our parameters to a function matters.
That's not the problem in and of itself. But if the perl script does decode
the input it should've been written to fully pull apart the names and value
from the input (UI) and call the fortran correctly. Right?

Yes and run sanity checks on the data.

Alternatively, based on the response I got, I was allowing that somehow the
perl script was literally doing /nothing/ but passing on the input. Now,
not being an expert in what this data stream really looks like, I'll just
accept it, but then that means the fortran code a) understands the input
from the web browser, which at least generally dates it with regard to any
expectation that it's still maintainable and b) if a is true, begs the
question of why the need for the perl script in the middle, and c) now has
the responsibility to parse the input.

It sounds as if the Fortran program is a legacy application, perhaps
involved in some heavy number crunching.

I cannot imagine anyone being perverse enough to write a CGI interface
in Fortran since it is a language totally unsuited for this purpose so
(a) is extremely unlikely.

Axel

 

[axel]

On 06 Jul 2006 16:27:05 GMT, (e-mail address removed) wrote:

Well that's a childish remark. For starters, I'm trying to understand their
response; I'm allowing that they see something I don't. So on that level
alone your response is valueless as it has nothing to do with what I've
actually asked of this group. Great job!

Why should you only expect follow-ups that specifically answer your
questions only?

Moreover, there's nothing wrong with me providing feedback to the third
party as to what I see wrong with their script and its form processing.
Again, perfectly valid and acceptable. Which begs the question of whether
sticking your nose in and not contributing to the point is merely for your
warped enjoyment. Again, great job!

Perhaps you should not post to Usenet if you do not want other people
to make post post-followups that do not specifically answer your
questions or that raise matters that are not of interest to you only.

Xho was perfectly correct in what he said.

Essentially you are using a third-party application which breaks
when you try to change one part of it. Of course you can provide
feedback, but trying to understand exactly how the data is being
processed is not really your concern... it is obviously a proprietary
system and most people have no reason to disclose how such systems
work and very good reasons to keep the workings as a black box to
outsiders... security being one of them.

Axel

 

[anno4000]

On 06 Jul 2006 18:40:36 GMT, (e-mail address removed) wrote:

--snip--

I started to answer your comments inline but as I trimmed the reply,
deciding what I wanted to respond to, I found myself realizing you've turned
this thread toward the irrelevant BS you seem to want to bitch about.
Aha.

Through your replies you've misrepresented, whether intentionally or
ignorantly, the context, full meaning and intention of my remarks.

Xho has, intentionally without a doubt, taken a step back and put your
problem in a broader perspective. If you don't like the view, that's
your problem.

[...]

And since I have to make some assumptions about what their real
constraints may be I came here first.

What has coming first to do with anything?

And unless you're planning a trip to
Houston and are asking for my address please don't start puffing out your
chest. And unless that's the case, I'm done with you.

We love you too.

Anno

 

[anno4000]

Well that's a childish remark. For starters, I'm trying to understand their
response; I'm allowing that they see something I don't. So on that level
alone your response is valueless as it has nothing to do with what I've
actually asked of this group. Great job!

So the group is, in your opinion, bound to answer your question or
shut up? That's not how Usenet works. If anything is childish around
here, it's your assumption that this is somehow "your" thread and you
get to say what belongs here and what doesn't.

Anno