Systems software versus applications software definitions

[Nick Maclaren]

|> (e-mail address removed) (Matt) wrote in message |> > How do we define systems programs?
|>
|> One view (certainly not the only one) distinguishes between programs that need
|> or expect some privilege (e.g. supervisor state) from those that don't (though
|> they may use privilege-requiring services through a system-call interface).
|>
|> In this view, system programs *must* be written carefully, since a malfunction
|> can have global effects; application programs should never be able to do worse
|> than shoot themselves in the foot (local and bounded side-effects only). This
|> last assumption depends of course on a properly fenced execution environment,
|> which most people don't enjoy.

And which most systems don't enable :-(

Even when they do, there is the concept of programs that neither
need nor expect ANY privilege, but are likely to be used at a
relatively high privilege level. Even the proponents of that
view don't claim that a shell or basic utility need not be
implemented to the same standard as a 'system' program.


Regards,
Nick Maclaren.

 

[Sander Vesik]

Plz explain your first point.
For web servers, to my opinion, they're widely using TCP/IP stack, which
is OS built-in functions. Therefore, at least they are not *such* high
level apps.

In which case so are (n)curses text editors, as they use low level terminal
i/o a lot. You definition basicly makes any network app be systems software,
something that I don't think is a good thing.

 

[Charlie Gordon]

Here in comp.lang.c,

Systems programmers count in hex.

[Snip]

Application programmers count in decimal.

I count in octal -- what does that make me?

Old, like me <g>.

(If I'm counting on my fingers I use Gray code...)

I find that hard to believe !

 

[Chris Croughton]

Here in comp.lang.c,
"Alex McDonald" <[email protected]> spake unto us, saying:

Systems programmers count in hex.

[Snip]

Application programmers count in decimal.

I count in octal -- what does that make me?

Old, like me <g>.

(If I'm counting on my fingers I use Gray code...)

I find that hard to believe !

For numbers expected to be greater than five, it's easier to use a
binary method of some kind (I dislike having to use the other hand to
keep track of how many fives I've had). Gray code has the advantage
that only one bit changes at a time.

I learnt it as a teenager (I 'inherited' some optical shaft encoders
which used it), and have done it automatically for so many years that I
find it harder to use binary. The conversion back to 'straight' binary
(if one has forgotten the tables, only 32 values for a single hand) is
simple:

Find the leftmost '1' in the value to be converted and invert each bit
to the right of it.

Repeat with the digits of the original value (not the partially
converted one) until you run out of digits.

Converting binary to Gray code is similar (and slightly simpler, because
you don't need to remember the original value):

Find the leftmost '1' and invert each bit to the right of it.

Repeat with the digits of the partially converted value until you run
out of digits.

Incrementing a value doesn't need propagation of 'carry', it's just:

If the parity is even, invert the bottom bit.

If the parity is odd, invert the bit immediately to the left of the
rightmost '1' bit;

Or in C:

unsigned int fromGray(unsigned int val)
{
unsigned int i = 1U << (sizeof(val)*CHAR_BIT - 1);
unsigned int b = val;
while (i)
{
if (val & i)
b ^= i - 1;
i >>= 1;
}
return b;
}

unsigned int toGray(unsigned int b)
{
unsigned int i = 1U << (sizeof(b)*CHAR_BIT - 1);
while (i)
{
if (b & i)
b ^= i - 1;
i >>= 1;
}
return b;
}

(Is there an easier portable way of setting the top bit of a uint than
calculating the number of bits using sizeof and CHAR_BIT? If UINT_MAX
is guaranteed to be 2^n - 1 then (UINT_MAX - UINT_MAX/2) would work, but
that's almost as non-transparent.)

Chris C

 

[George Neuner]

Kind of off topic, but... IE can be "safely" removed from the OS.

You can remove the IE executable, but that program is not Internet
Explorer. What you interact with is just a thin shell around a COM
control that provides the browser functions. The IE program does
little more than instantiate that control and provide a window for it
to scribble on.

You can hunt down and remove the various incarnations of the
"WebBrowser" control, but doing so may break the Explorer shell if
"Active Desktop" functions are enabled and can render the shell
completely unusable. The help systems in 2K and XP, MS Office and
many 3rd party applications also use the browser control. And the
browser control itself relies on other controls which can't be
removed.

Microsoft exaggerated but they were not lying. Removing IE results in
reduced functionality ... that is if the system still works at all.

George

 

[Anne & Lynn Wheeler]

i've frequently claimed that to take a straight forward written
application and turn it into a "service" ... it takes 4-10 times the
code and ten times the work.

.... recent slashdot
http://developers.slashdot.org/developers/04/12/04/1551255.shtml?tid=221&tid=1

points to the acm queue interview article
http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=233

Designing for failure may be the key to success.
Engineering for Failure

it was called system/r ... developed on vm/370 base and initially
transferred from sjr to endicott for sql/ds. later tech transfer
to stl for db2. misc past system/r, sql, etc posts
http://www.garlic.com/~lynn/subtopic.html#systemr

and related to the subject was ha/cmp
http://www.garlic.com/~lynn/subtopic.html#hacmp

where we had to do detailed total system vulnerability and failure
mode investigation.

 

[Nick Maclaren]

My viewpoint is slightly different, and I dissent. If you spend
3 times the effort in design, BEFORE you start, you can reduce the
amount of code to about twofold and the amount to effort to about
threefold.

But I agree that your figures are correct if the application wasn't
designed to be a service in the first place. At at least 3 other
people (including, I think, Fred Brooks) have said the same.

... recent slashdot
http://developers.slashdot.org/developers/04/12/04/1551255.shtml?tid=221&tid=1

points to the acm queue interview article
http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=233

Designing for failure may be the key to success.
Engineering for Failure

Well, as I know, I agree with that. But you also know that I am
convinced that the IT world is STILL heading away from there :-(


Regards,
Nick Maclaren.

 

[Anne & Lynn Wheeler]

Well, as I know, I agree with that. But you also know that I am
convinced that the IT world is STILL heading away from there :-(

when i got to do the resource manager
http://www.garlic.com/~lynn/subtopic.html#fairshare
http://www.garlic.com/~lynn/subtopic.html#wsclock

one of the supporting processes was an automated test & benchmarking
process
http://www.garlic.com/~lynn/subtopic.html#bench

and eventually did one sequence of 2000 benchmarks taking 3 months
elapsed time before first customer ship.

with the benchmarking process ... was able to define almost any sort
of workload characteristics ... including very stressful ones that
turned out were guarenteed to crash the system. early on as a result of
this ... one of the side-tracks was to go in and completely redesign
and rewrite the kernel serialization infrastructure ... that resulted
in the elimination of all stress-test induced system failures as well
as all observed situations of hung/zombie processes .... some of
hung/zombies as well as other fault diagnostic stuff
http://www.garlic.com/~lynn/subtopic.html#dumprx

the main product release after the release of the resource manager
included SMP support ... which had a lot of dependencies on the
resource manager code ... which then created a business problem.
http://www.garlic.com/~lynn/subtopic.html#smp

the resource manager wss the first forey into priced kernel software
with guidelines that direct hardware support kernel software was
still free. having smp "free" software dependent on priced (resource
manager) software violated those business guidelines. As a result ..
something like 80percent of code in original resource manager was
removed and incorporated into the "base, free" kernel software.

so problem reporting and fix process had a procedures that assigned
a number (that was incremented sequentially) and fixes for specific
problems were given the same number. with the very first version
of vm370, the numbering started ... and as far as i know may never
have been reset (i think i've recently seen references to sequential
numbers in the 60k range?).

Any way ... if you have to be dilligent to maintain kernel integrity
(failures because of dangling activities after processes have gone
away) and/or hung/zombie processes (waiting for some activity
to complete). A couple releases after the resource manager went out,
there was a fix introduced into the dispatcher to ignore various
kinds of events under certain circumstances; this had a fix number of
something like 15,3xx (or maybe 15,1xx?). In any case, it resulted
in re-introducing hung/zombie processes.

This occured after i had redone the i/o system to make it bullet proof
so the disk enginneering lab could do their work in an operating
system environment ... instead of doing everything with dedicated,
stand-alone machines:
http://www.garlic.com/~lynn/subtopic.html#disk

I created an update that removed the effects of the fix that
re-introduced hung/zombie processes ... and tried to find out what was
the original justification for generating it in the first place.

of course, all the ha/cmp work was pretty much trying to figure out
how to retrofit availability andd assurance to existing
infrastructures
http://www.garlic.com/~lynn/subtopic.html#hacmp

the stuff for electronic commerce was someplace in-between ... characterize
all possible failure modes anywhere in the infrastructure ... and then
define recovery and/or diagnostic processes to handle every possible
scenario. ... aka the previous electronic commerce ref
http://www.garlic.com/~lynn/2004p.html#23

however, in approx, the same electronic commerce time-frame ... we
looked at taking a more systemic approach. we had a jad with taligent
about their environment ... taking the analogy from the original
os/360 system services .... what taligent characteristics would be
needed for assurance and availability system services. we had a one
week JAD where we walked thru all of the taligent infrastructure,
specifying what needed to be added/changed for assurance and
availability. at the end, we had come up with two new
availability/assurance frameworks ... and, in addition, about a 30%
hit to the existing taligent frameworks

you would still need the up-front failure analysis ... but could
possibly reduce application code lines from a 4-10 times increase to
possibly only a 1.5 to 2.0 times increase (with some higher level
assurance and availability abstrations being provided by the taligent
infrastructure).

random past taligent references:
http://www.garlic.com/~lynn/2000e.html#46 Where are they now : Taligent and Pink
http://www.garlic.com/~lynn/2000e.html#48 Where are they now : Taligent and Pink
http://www.garlic.com/~lynn/2000.html#10 Taligent
http://www.garlic.com/~lynn/2001j.html#36 Proper ISA lifespan?
http://www.garlic.com/~lynn/2001n.html#93 Buffer overflow
http://www.garlic.com/~lynn/2002.html#24 Buffer overflow
http://www.garlic.com/~lynn/2002i.html#60 Unisys A11 worth keeping?
http://www.garlic.com/~lynn/2002j.html#76 Difference between Unix and Linux?
http://www.garlic.com/~lynn/2002m.html#60 The next big things that weren't
http://www.garlic.com/~lynn/2003d.html#45 IBM says AMD dead in 5yrs ... -- Microsoft Monopoly vs. IBM
http://www.garlic.com/~lynn/2003e.html#28 A Speculative question
http://www.garlic.com/~lynn/2003g.html#62 IBM says AMD dead in 5yrs ... -- Microsoft Monopoly vs. IBM
http://www.garlic.com/~lynn/2003j.html#15 A Dark Day
http://www.garlic.com/~lynn/2004c.html#53 defination of terms: "Application Server" vs. "Transaction Server"
http://www.garlic.com/~lynn/2004l.html#49 "Perfect" or "Provable" security both crypto and non-crypto?

 

[Anne & Lynn Wheeler]

ref:
http://www.garlic.com/~lynn/2004p.html#63
http://www.garlic.com/~lynn/2004p.html#64

another dimension of assurance is this stuff:
http://www.software.org/quagmire/

something like 2167a can increase the straight-forward application
development costs by a factor of ten times ... and frequently this
sort of stuff can't be retrofitted after the fact (has to be done up
front before coding ever starts)

 

[Nick Maclaren]

|>
|> something like 2167a can increase the straight-forward application
|> development costs by a factor of ten times ... and frequently this
|> sort of stuff can't be retrofitted after the fact (has to be done up
|> front before coding ever starts)

Yes and no. I obviously agree with the latter, but not the former,
which is true but incorrect in context!

If you design for failure at the start, yes, it increases the
initial cost of both design and coding. But it very often reduces
the testing cost and it usually reduces the cost of finding bugs
in the field (i.e. support and maintenance) by a large factor. I
could give many examples.

The point is that the initial coding cost is often less than that
of even the initial testing, and is almost always a small fraction
of the support and maintenance costs. However, those are almost
always indirect, and typically come out of separate budgets anyway.


Regards,
Nick Maclaren.

 

[Joona I Palaste]

Juhan Leemet <[email protected]> scribbled the following

FWIW, I seem to recall being able to multiply hex digits (rote memory, I
know) when I was crawling through IBM mainframe dumps as a uni student. I
had a professor who joked about how I was playing the front panel switches
of a PDP-8 "like a piano" when I toggled in the bootstrap loader. Ah...

You're much more learned than I am, then. The only thing almost a
decade of writing toy machine language programs to see what the
Commodore 64 can do has taught me in this regard is being able to
convert any integer from 0 to 255 from decimal to hexadecimal or back
in my head in a couple of seconds. Well, it amazed my little brother
for a couple of minutes.

 

[Anne & Lynn Wheeler]

You're much more learned than I am, then. The only thing almost a
decade of writing toy machine language programs to see what the
Commodore 64 can do has taught me in this regard is being able to
convert any integer from 0 to 255 from decimal to hexadecimal or
back in my head in a couple of seconds. Well, it amazed my little
brother for a couple of minutes.

besides learning to read hex from mainframe dumps ... i also learned
to read it from the front console lights as well as the punch holes in
cards (output of assembler and compiler binary/txt decks) ... both
hex->instructions/addresses and hex->character (or in the case of
hex punch cards, holes->hex->instructions/addresses and
holes->hex->ebcdic).

In the past I had made (the mistake of?) posts about the TSM lineage
from a file backup/archive program
http://www.garlic.com/~lynn/subtopic.html#backup

that I had written for internal use that then went thru 3-4 (internal)
releases, eventually packaged as customer product called workstation
datasave facility, and then its morphing into ADSM and now TSM (tivoli
storage manager).

so a couple days ago ... i get email from somebody trying to decode a
TSM tape; included was hex dump of the first 1536 bytes off the tape
.... asking me to tell them what TSM had on the tape.

well way back in the dark ages ... you could choose your physical tape
block size ... and the "standard label" tape convention started with
three 80-byte records; vol1, hdr1, hdr2.

so the first 1536 bytes was three 512byte records ... and i recognize
the first 80 bytes of each (512byte) record as starting vol1, hdr1,
hdr2.

the hex dump had included the hex->character translation ... but for
ascii ... and of course the tsm heritage is from ebcdic mainframe
.... not ascii (aka it was the ebcdic hex for vol1, hdr1, hdr2) It
didn't even get to the TSM part of the tape data ... it was still all
the os standard label convention.

 

[Dan Koren]

How do we define systems programs? when we say systems programming,
does it necessary mean that the programs we write need to interact
with hardware directly? For example, OS, compiler, kernel, drivers,
network protocols, etc...? Couple years ago, yes, I understand this is
definitely true. However, as the software applications become more and
more complicated, some people try to argue that. Some people argue the
definition of systems programs depend on the level of abstractions. I
heard people saying that web server is a systems software, which I
feel confused. I think web server is an application software. Yes,
other applications run on top of web server.

Please advise and discuss. thanks!!

One of the oldest jokes in this profession:

Q: What is the difference between application
programmers and system programmers?

A: The latter can make other people's programs
crash!



dk

 

[Richard Bos]

No.

I wouldn't comment either way on the safety of removing any part of MS
Windows (unless you remove all parts at the same time...), but why on
earth was this cross-posted to comp.lang.c? It has nothing to do with
that language. Please be a little more discerning in your crossposts
next time.

Richard

 

[Keith Thompson]

I wouldn't comment either way on the safety of removing any part of MS
Windows (unless you remove all parts at the same time...), but why on
earth was this cross-posted to comp.lang.c? It has nothing to do with
that language. Please be a little more discerning in your crossposts
next time.

According to Google, both of Maxim's messages were cross-posted to
comp.lang.c about 2 years ago. I have no idea why they've shown up
again. Maybe some news server somewhere belched.

 

[CBFalconer]

According to Google, both of Maxim's messages were cross-posted to
comp.lang.c about 2 years ago. I have no idea why they've shown up
again. Maybe some news server somewhere belched.

Nothing prior to Richard Bos' article ever appeared here, so the
glitch was quite local.

--
"Our enemies are innovative and resourceful, and so are we.
They never stop thinking about new ways to harm our country
and our people, and neither do we." -- G. W. Bush.
"The people can always be brought to the bidding of the
leaders. All you have to do is tell them they are being
attacked and denounce the pacifists for lack of patriotism
and exposing the country to danger. It works the same way
in any country." --Hermann Goering.

 

[Keith Thompson]

Nothing prior to Richard Bos' article ever appeared here, so the
glitch was quite local.

Richard and I, who are on opposite sides of the planet, both saw two
articles by Maxim S. Shatskih. It's not local, but apparently it's
not universal either. I have no idea what happened, and I don't care
enough to try to track it down.