A
Anne & Lynn Wheeler
A. G. McDowell said:I would be very interested to hear more about increasing the
effectiveness of testing beyond all recognition. I am a professional
programmer in an area where we routinely estimate the testing effort as
about equal to the programming effort (in terms of staff time, but not
necessarily staff cost). Do you have references? As a token of sincerity
I will provide references for what we seem to agree is commercial
practice (whether it should be or not):
when we were doing the original payment gateway
http://www.garlic.com/~lynn/aadsm5.htm#asrn2
http://www.garlic.com/~lynn/aadsm5.htm#asrn3
we set up a test matrix ... not for the sotware ... but for the
service. nominal payment infrastructure trouble desk did 5 minute
problem first level problem determination ... however that was for an
infrastructure that was almost exclusively circuit based.
while it was possible to translate the (payment) message formats from
a circuit based infrastructure to a packet based infrastructure ...
translating the circuit-based service operation to a packet-based
infrastructure was less clear cut (merchant/webhost complains that
payments aren't working ... expects internet/packet connection to be
much less expensive than direct circuit ... but at the same time
expects compareable availability).
The claim has been that coding for a service operation is 4-10 times
that of a straight application implementation and ten times the effort
because of needing to understand all possible failure modes
.... regardless of whether they are characteristic of the software or
hardware or some totally unrelated environmental characteristic.
in any case, one of the issues was detailed analysis of existing
trouble desk circuit-based problem determination procedures and being
able to translate that into a packet-based (internet) environment and
still attempt to come close to the goal of being able to perform first
level problem determination in five minutes. When we started there
were cases of trouble ticket being close NTF (no trouble found) after
3hrs of manual investigation.
of course this was also at a time ... when it was difficult to find
any ISP that even knew how to spell "service level agreement".
aka ... it is possible for software to perform flawlessly and still be
useless.
some of this came from doing ha/cmp
http://www.garlic.com/~lynn/subtopic.html#hacmp
misc. related past threads
http://www.garlic.com/~lynn/2000g.html#50 Egghead cracked, MS IIS again
http://www.garlic.com/~lynn/2001e.html#48 Where are IBM z390 SPECint2000 results?
http://www.garlic.com/~lynn/2001f.html#75 Test and Set (TS) vs Compare and Swap (CS)
http://www.garlic.com/~lynn/2001k.html#18 HP-UX will not be ported to Alpha (no surprise)exit
http://www.garlic.com/~lynn/2001n.html#85 The demise of compaq
http://www.garlic.com/~lynn/2001n.html#91 Buffer overflow
http://www.garlic.com/~lynn/2001n.html#93 Buffer overflow
http://www.garlic.com/~lynn/2002.html#28 Buffer overflow
http://www.garlic.com/~lynn/2002.html#29 Buffer overflow
http://www.garlic.com/~lynn/2002e.html#73 Blade architectures
http://www.garlic.com/~lynn/2002f.html#24 Computers in Science Fiction
http://www.garlic.com/~lynn/2002h.html#11 Why did OSI fail compared with TCP-IP?
http://www.garlic.com/~lynn/2002h.html#12 Why did OSI fail compared with TCP-IP?
http://www.garlic.com/~lynn/2002n.html#11 Wanted: the SOUNDS of classic computing
http://www.garlic.com/~lynn/2003b.html#53 Microsoft worm affecting Automatic Teller Machines
http://www.garlic.com/~lynn/2003g.html#62 IBM says AMD dead in 5yrs ... -- Microsoft Monopoly vs. IBM
http://www.garlic.com/~lynn/2003j.html#15 A Dark Day
http://www.garlic.com/~lynn/2003l.html#49 Thoughts on Utility Computing?
http://www.garlic.com/~lynn/2003p.html#37 The BASIC Variations