New paradigm of introspective OO development


J

Jared Thirsk

== Introduction to DAF ==

I am interested in the next generation of approaches to software
development. Over the last decades, there has been an obvious shift
in how our most complex software is created. We started off with
punch cards (or so I am told -- I was born in 1981 so apologies if my
history is off or my ideas are naive), punching in a stream of ones
and zeros. The code contained only 0 and 1 and everything else was
abstracted in the mind of the programmer, even the machine-level
instructions. Then we moved to assembly language, where code more or
less explicitly contains the machine-level instructions, while the
meaning of variables, and the control structures were still largely
abstract. From there, with the advent of C, control structures and
variable names are now in the code. With C++, objects are in the
code, while system-level architecture, design patterns, and object
interoperability are still mostly abstracted.

The general trend is a move towards more intelligent code -- from
1's and 0's, to objects with explicit inheritance, interfaces, and
data members. Each time more abstractions are made explicit in the
code, the code becomes easier to implement, test, and reuse.

I am interested in exploring the next step (perhaps even determining
the nature of or actualizing the final step), of moving more
abstractions to code, making the code more intelligent, and easier to
develop, as the developer needs to carry less abstraction in his/her
own head.

What I am proposing is a strictly minimalistic paradigm that
facilitates more intelligent code. More intelligent code facilitates
better and easier-to-implement RAD design tools, as well as more
intelligently adaptive and self-aware software components. The
working name used here is DAF, which stands for Dynamic Application
Framework.

== Basic Concept ==

The idea I have is for a purer form of object-oriented development,
or really, aiming for objectively pure objects that perfectly
represent any conceptual abstraction a software programmer or
architect could make, on the object / component level. (On a system
level, abstraction arises from the interactions of the objects, and so
is beyond the scope of object definition, or object oriented
programming, but arises out of it anyway.)

Conceptually speaking, what is an object? It is a concept, an idea.
It only has meaning once things are attached to it. The decorator
pattern (or at least my concept of it), in its general conceptual
form, can be used to attach capability to objects. Objects have two
primary capabilities: they can accept input from their environment,
and they affect their environment. Automata such as computer code
only meaningfully affects its external runtime environment in response
to input. (Whether various "agents" such as humans or software agents
act spontaneously is a philosophical matter requiring a theory of
reality, like the CTMU, or a semantic issue, for people who like to
think they are writing spontaneously intelligent agents.) Taking this
into consideration, we get the general principles of interface (which
is basically equivalent to environmental input or event handling) and
implementation (internal aspect of doing something that affects the
outer world) and their natural separation.

Thus each object intrinsically has decorators that provide
interfaces, along with implementation of those interfaces. I think
that the progression towards easier programming will inevitably lead
and is leading towards smarter programming tools, eventually towards
the point where the programming objects themselves will support
reflexive self-modification, which could potentially be dropped at
either compilation time or runtime for the sake of proprietary opacity
or memory/CPU efficiency, where desired, or left in for the sake of
dynamically intelligent, self-aware and/or user runtime-extendable
applications.

(There is also a shift towards using scripting languages where
certain application behavior is treated more as data than as
statically hard-wired information. The divide between data vs code is
shrinking and will continue to shrink as our software demands more
cross-integration and adaptive capability such as self-repair,
self-optimization, and transparency of object distribution.)

Self-modifying objects therefore require two basic decorators: one
for managing interfaces, and one for managing decorators (which
encapsulate implementors). In DAF, there are three fundamental types
of objects, data, executable, and generic.
- Data primitives contain a data decorator, and provide get/set
interfaces for compatible types in the host language (char*, long,
etc.), with the implementation possibly converting numeric strings to
binary integers, for example.
- Executable primitives are guaranteed to implement an "execute"
method, which executes code in a supported language (perhaps a call to
shared library, or C code, or a lua script).
- Generic objects are not primitives in that they contain no Data or
Executable that interface with the host machine architecture. They
are completely abstract objects, containing nothing and having no
capability by default.

== Extending Capability with Decorators ==

Generic, or non-primitive objects must be extended to be useful.
The DAF development environment contains a executable primitive to
create a Generic with self-modifying capabilities. Its inextricable
interfaces provide access to both the interface manager, and the
decorator manager. From here, all imaginable capability can be added.

By adding a string decorator to an object, for example, the string
decorator factory will invoke operations in the Interface Manager to
add the supported Get/Set accessor interfaces to the object. The
decorator factory will also use the object's Decorator Manager to add
a string decorator, which is responsible for handling the
implementation of the string accessor methods appropriately, and
ensuring the value is stored in memory. (Prototypes for efficient
creation of frequently used objects is mentioned later.)

== Refining Capability via Chain of Command ==

Another decorator could be added, to handle thread synchronization
for the object. A 'synchronization' decorator would be attached,
containing some kind of mutex or semaphore as its private
implementation data, and the existing interface methods for accessing
the string would be replaced by a chain of command: when GetString, or
whatever is invoked, the Interface Manager will pass execution to
locking/waiting implementation in the sync decorator, followed by the
string get implementation in the string decorator, followed by the
unlocking implementation.

Another decorator could be added to control permissions of the
object, inserting a permission check at the beginning fo the chain of
command. The implementation would verify a security token object's
security domain (referenced inside the decorator, described in a
central location) before the chain was allowed to continue, if at all.

All sorts of kinds of decorators could be added to objects, such as
ones that implement transparent access of objects over the net,
time-variable caching from net or disk, database access, child
containers, parent accessors, observer pattern, documentation info,
etc. As such, this could be seen as a sort of middleware that could
compete (once it evolves to this point, of course) with the likes of
CORBA, DCOP, ZeroC ICE, etc. (ICE's feature list is probably the
closest match I've seen to what I envision here.) The distinction
between DAF and these being that all capability for distributed
objects, as well as everything else, is optional, not constrained to
or focused on any one application.

== Typing ==

Objects are not described by a single type, but rather conform to
one or more schemata which indicate their capabilities. A schema
comprises a set of one or more interfaces (or interface groups, not
described here), or one or more decorators, or both, with the
openendedness of DAF allowing more complex schema systems to be
implemented, such as the one used by XML (.xsd). For convenience of
organization, a schema may include other sub-schemata.

Schemata can be generated at runtime, against which introspective
objects can be validated. Alternately, non-introspective objects or
non-modifiable objects can be hardcoded to indicate they implement a
certain schema.

== Prototyping / Flyweighting ==

Factories may be fed a particular schema to use to create an object.
Detailed schemata may effectively be prototypes, which factories
could automatically generate for efficient creation, or which user
could manually specify.
I won't go through all the details, but with DAF there is
opportunity to set up the creation flyweights as well as copy-on-write
objects, transparently and conveniently for the developer.

== Full Reflexivity and Self-Documentation ==

Documentation decorators could be added to every object prototype as
desired, at runtime (in a RAD environment) with the documentation
being physically stored in a separate location as the code. DAF
allows for multiple group (and parent accessor) decorators,
facilitating multi-dimensional heirarchies.
For example, a schema object could simultaneously be all of:
- the member of a multi-level index of all schemas currently
available,
- the member of a index of schemas provided by a certain pluggable
library,
- the dereference point of another object's reference (perhaps
hardcoded, or cached from a schema check) indicating that object
conforms to the schema,
- the parent object of a documentation object describing that schema,
the member of an index containing the set of objects authoritative by
an object server,
- a member of a revision control system tracking this version of the
object
- or any other number of examples.

When subsystems are dropped or become irrelevant, such as a
development subsystem, all related decorators are dropped (revision
control, code documentation, etc.), and may be added again at runtime
(depending on how much introspective capabilities the objects have).

The intent is to establish one way (via group/reference decorators)
and traceable one way (via group/parent decorators) connectivity
between objects in a way that explicitly captures the abstract
relationships between objects, moving the intelligence into the code.

The resulting codebase becomes a self-documenting, self-accessible
network with broad scope, allowing a developer to traverse it similar
to how TheBrain traverses broad information rich networks.

== Example Applications ==

Theoretically there should be no restriction on what couldn't be done,
the idea being that at some point, runtime dynamicity and reflection
can be traded in for runtime functionality and performance comparable
to a C program. That said, here are a few random examples where the
dynamic capabilities might be helpful:

- A web based knowledge base that allows arbitrary relationships
providing multilevel heirarchies. Nodes of information could have
discussions linked to them, people's votes on appeal or validity,
external links, etc.
- A multi-server scale or peer-to-peer computer game requiring
transparent relocation of objects, benefitting from adaptive load
balancing and self-optimization.
- Any application where introspection is needed, applications that
learn, system doctor programs, better versions of clippy.
- Any application benefitting from a minimal or maximal
object-oriented approach, and design patterns.

== Flexibility ==

The idea is that DAF is a simple abstraction layer, and does not
prevent the developer from making use of existing libraries and their
APIs, or from designing 80% of their application in Java or C++ or
perl and abstracting the 20% of the high-level stuff and scripting
abilities with DAF.

In theory, DAF is just a way to abstract concepts, and should not
impose any restrictions on design. Any run-time dynamic
self-modification and introspective abilities should be optional
similar to how C++ RTTI is optional.

== Facilitation of Adaptivity ==

A quick example of how introspection could facilitate Adaptive Agent
Intelligence: a group decorator (container of keyed child objects),
could have an AI agent attach itself to the "find object" interface
chain of command to profile a single or multiple algorithms for
looking up children, and either a) switch to another algorithm /
storage mechanism that is known or believed to be better, or b)record
performance of algorithms to disk so that common executions of the
application can reveal the best algorithm to use (hash table (of
varying sizes and algorithms), b-tree, map, etc.).
In general, this kind of introspection allows profiling that can be
more automated and facilitate intelligent run-time adaptivity in code
execution.

== Implementation ==

DAF is a simple and obvious way of defining application in terms of
objects, interfaces and decorators. The matter of actually executing
code is arbitrary and could be handled by any compiled or interpreted
language with sufficient bindings to the DAF specification. If DAF
actually gets implemented, and it's up to me, I'm thinking that a BSD
or GPL licensed DAF runtime environment would be implemented using
C++, with support for executing shared libraries on Win32 / Linux /
other OS's, and bindings to a scripting language like Lua. At some
point, it may make sense to create a dedicated language for working
with DAF for the sake of syntax convenience and aesthetics (so far
I've been creatively overloading C++ operators.)

== Summary ==

DAF is a proposal to facilitate and make explicit the implementation
of several important design patterns, including decorators, facade,
chain-of-command, command objects (instead of parameters to executable
objects). Generic support for observers, marshals, factories and
probably several other patterns will be provided as a part of DAF's
core decorator libraries, as will commonly useful applications such as
threading, network/database transparency, load balancing, etc.

DAF aims to ease the development and debugging bottlenecks by leverage
today's increased CPU and memory availability to make applications
fully interactive, both to use and to develop introspectively without
necessarily enforcing any of these features. Introspection
facilitates adaptively intelligent program dynamicity to the point
where applications can start to become intelligently self-analytical.

== Conclusion ==

I am a recent SW engineering grad without several years of serious
programming work experience, so I present this with humility and an
open mind. I am hoping that people can direct me to a) why this is a
bad idea, b) what is out there that already does this, or c) that this
is an avenue worth exploring, or d) same as c and that they're
interested in helping make it or e) something else. In addition to
being someone who would like better RAD tools for software
development, I am also an independent researcher in the realm of
psychology and cognitive science, and would like to see intuitive
methods of programming that more closely represents good theory of
mind (requiring less overall time and effort from the software
developer), as well as a theory of mind (AI) that is easier to
implement in and interact with software.

I will try to monitor the forums where I post this, but if you want to
be sure to get through to me, email me at (e-mail address removed)
(removing the 123 for my actual address).

Regards,
Jared Thirsk
http://daf.melsib.net
 
Ad

Advertisements

H

H. S. Lahman

Responding to Thirsk...
== Introduction to DAF ==

I am interested in the next generation of approaches to software
development. Over the last decades, there has been an obvious shift
in how our most complex software is created. We started off with
punch cards (or so I am told -- I was born in 1981 so apologies if my
history is off or my ideas are naive), punching in a stream of ones
and zeros. The code contained only 0 and 1 and everything else was
abstracted in the mind of the programmer, even the machine-level
instructions. Then we moved to assembly language, where code more or
less explicitly contains the machine-level instructions, while the
meaning of variables, and the control structures were still largely
abstract. From there, with the advent of C, control structures and
variable names are now in the code. With C++, objects are in the
code, while system-level architecture, design patterns, and object
interoperability are still mostly abstracted.

FYI, paper tape preceded punched cards and plug boards preceded paper
tape. By the time punched cards arrived on the scene there had already
been a lot of automation in the form of loaders, linkers, and Assembly
compilers. In the Plug Board Era software was quite literally an
extension of the hardware. One "wrote" a program by using a jumper to a
power rail to selectively turn on/off the grid in triode tubes.

[Doing that and changing vacuum tubes was such a character building
experience that I didn't do software again for a decade when 3GLs and
solid state computers were commonplace (late '60s). B-)]
The general trend is a move towards more intelligent code -- from
1's and 0's, to objects with explicit inheritance, interfaces, and
data members. Each time more abstractions are made explicit in the
code, the code becomes easier to implement, test, and reuse.

True. There has been a steady migration from right to left along the
continuum:

Requirements -> Analysis -> Design -> 3GL code -> Assembly -> Executable

Today everything to the right of 3GL code is fully automated unless one
is doing embedded systems or developing compilers. But it took nearly
three decades to get there. Since the '70s things have migrated further
in specialized niches like RAD IDEs for client/server applications. (In
at least once <rather narrow> arena, ATLAS (IEEE 716), a text
requirements specification language is routinely compiled directly into
an executable.)

However,...


I am interested in exploring the next step (perhaps even determining
the nature of or actualizing the final step), of moving more
abstractions to code, making the code more intelligent, and easier to
develop, as the developer needs to carry less abstraction in his/her
own head.

In the late '80s the technology arrived to do exactly what you want to
do. It is called translation-based development. Tools have existed for
nearly two decades that will do 100% code generation from an OOA model.
Today all such tools employ UML as the notation and OMG's current MDA
effort is providing much needed standardization. You might want to
check out the way MDA employs meta models to provide mapping between
different model notations in the normal stages of software development.

In addition, you might want to check out these tool sites for examples
of full code generators from OOA models:

ARTiSAN www.artisansw.com
Bridgepoint www.projtech.com
ObjectBench www.ses.com
Rhapsody www.i-logix.com
Rose/RT www.rational.com
System Architect www.popkin.com
xtUML www.kc.com

In addition Pathfinder (www.pathfindermda.com) provides code generation
rear ends for several UML drawing tools, including Rose. These are just
the vendors that have been around for a decade or so. Since the MDA
initiative was launched translation tool vendors are popping up like
tulips. The Bridgepoint, xtUML, and Pathfinder sites all have several
white papers on translation available (albeit mostly at the Executive
Summary level).

[Some are not pure translation vendors because they use an augmented 3GL
as an behavior description language rather than an abstract action
language (UML v1.5). Also, some cater to both the translation and
round-trip markets so some foraging of their web sites to find the
translation stuff may be necessary.]

Your approach seems interesting based upon the summary. Unforunately
there wasn't enough in the summary to indicate how attractive it would
be versus the existing technologies for translation. My concern is that
you may be reinventing a lot of work that has already been done. That
is, the "next step" has already been taken.


*************
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
(e-mail address removed)
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
(888)-OOA-PATH
 
C

Cristiano Sadun

(e-mail address removed) (Jared Thirsk) wrote in
I am a recent SW engineering grad without several years
of serious programming work experience, so I present this
with humility and an open mind. I am hoping that people
can direct me to a) why this is a bad idea,

Hi there. I answer since I also have (used to have :) some
academic background in cognitive science, so I also have a
symphatetic ear. :)

It's not that what you say is a bad idea - for what I've got
of it, of course :) - just that it usually takes both more
and less to talk of "new paradigm".

Specifically, less general talk, and more details - formal,
or at least obviously formalizable - which show where your
approach is structurally different from the bunch of stuff
that's already there. Formalization is essential, imho,
since makes things compact and easier to read, it
allows easy comparison between theories and generally,
saying it academically, cuts the crap. :)

For talking of a "new paradigm" you shall - of course -
start with a paradigm; and then have either one big
fundamental difference from which you descend a theory (say,
"the earth revolves around the sun and not the opposite":);
or most often a number of smaller differences (for example
"we bundle objects in self contained packages which
incapsulate complex behaviour and expose it in terms of
service and properties" is one of the jumps between OO and
component-oriented approach) whose overall consequences
produce a structural difference in an existing theory.

"Structural" meaning that you go down to the details and the
difference is still there - and u cannot take it away. In
other words, you dont want just to say what's desireable,
but also how to do it and why and in which way that "how" is
different or new. In jargon, you want a quantitative
approach rather than just qualitative (that, we leave to
philosophers - ok, kidding ;-).

For a number of reasons, it is less relevant whether or not
one paradigm can be implemented by another; but is important
to identify exactly which mechanism is new (but can be
translated/realized into some computation in another
paradigm) and which structurally different. Now, in your
post you have some heading like "Typing" (which tends to be
a quite precise, formalized and throroughly explored
concept) at the same level as "Flexibility" (which is in
general a far from rigorous idea, and difficult to
quantify).

Take stuff like, for example, literate programming; or
simply the differences between procedural and OO,
functional/declarative approaches. What you have in each
case is that you select some fundamental mechanisms for
describing your problem space, and take the (more or less
extreme) consequences.
b) what is out there that already does this, or c) that
this is an avenue worth exploring, or d) same as c and
that they're interested in helping make it or e) something
else. In addition to being someone who would like better
RAD tools for software development,

Ideas are always worth exploring - keep in mind that fixing
'em or finding out that they're wrong is a good result in
itself. What is imho important is the execution - the
relenteless cutting, refinement and discarding of what is
irrelevant, chaotic or not otherwise to the point.

Not that different from what makes a good software engineer,
all in all :)
I am also an independent researcher in the realm of
psychology and cognitive science, and would like to see
intuitive methods of programming that more closely
represents good theory of mind (requiring less overall
time and effort from the software developer),

Yep, that's what most recent paradigms are all about. Not
more computational power - which we probably can't anyway -
but better efficiency. I'd like the start trek approach
("Computer, do this") but I guess it'll take a while. :)
as well as a theory of mind (AI) that is easier to
implement in and interact with software.

On this issue (which is probably somewhat OT here), there
have been a number of efforts (of which you probably know
more than me, since I havent done much reasarch in the
fields since mid-90s). The key element I figured out at the
time was language - specifically models to compute semantics
of natural languages - which could have an huge impact:
imagine software tools which could help you out defining
models starting from a NL description, as much as a good
s.e. does when talking to a customer. I won't be holding my
breath tough. :)
 
C

Charles-A. Rovira

Smalltalk is allready capable in all these respects.

It's just not ''used'' that way.

Lets just say that the addition of Relationship classes, their
instantiations in Connection objects, which only respond to
disconnect, and the use of a Schema are all that required.

The rest is material but not as important as ''using'' Smalltalk the
way it was __meant__ to be used.

-Charles-A
 
Ad

Advertisements

S

Shashank

Hi Jared,

You should visit omg.org to know the current technologies being promoted
by them.

CCM - CORBA Component Model that relieves application (writing only
business logic within a logical unit i.e Component) developer from
deployment environment development (read as infrastructure development
like fault-tolerance, load-balancing, life-cycle management etc.)

UML- Technology to model a software. With latest specification now you can
even model real-time requirements, patterns, QoS, constraints etc.

MDA- Technology aimed at making writing code thing of past. Only models,
codes will be generated from models etc. Currently from UML model most of
the code, (80-90% may vary depending on kind of application) may be
generated.

This is already practiced by some of the organization and their success
stories are compiled at omg.org site. Do look there.

So now think about way of developing a software systems not by writing
code but by modeling your requirement.

Now does this passes your wildest imagination?

regards,
Shashank
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top