choices regarding where to place code - in the database or middle tier

D

Daniel Morgan

Joe said:
Please try one more time. In your first sentence you imply you'll take
several concepts one at a time. Your next sentence lists three possible
candidate 'concepts' all together. Then your last sentence only refers to
two, presumably security and performance. (What happened to scalability?).
I don't understand what you're saying. Your first sentence doesn't make
sense to me, and I think we unintentionally misunderstand this below:

In my meaning,

"Complete DBMS *dependence* means utilizing (all) those DBMS-vendor
specific
functions that optimize or implement security, performance, and scalability
(and other stuff)." Perhaps it also means "Relying on at least one DBMS
vendor-
specific feature in a way that makes the system practically unable to adapt
to another DBMS".

Complete DBMS independence means that a system is not bound to a given
DBMS, because it uses only the functionality offered by the DBMS that is
accessible via DBMS-neutral syntax, that syntax which provides the same
semantics in any DBMS.

It is also true that a system could be completely dependent on a given
middle tier product if it used all the vendor-specific features of
that product. What we have been talking about is what degree of
DBMS independence a system should maintain. *Some* independence is
possible and good, and maybe even unavoidable, such as the common
semantics (for the most part) of the shared SQL language. Independence
is good because it broadens the market for the system and lowers the cost
usually. Lots of such successful systems exist.
Some dependence is unavoidable, such as having something in the DBMS
client that speaks the client-DBMS wire protocol. Some
dependence/independence
is optional, such as a vendor's stored procedures. Should a system use
them, or stick to fresh SQL92? I would generally expect stored procedures
would be better, even if the system had to have multiple analogous
subsystems to attempt to duplicate functionality to different DBMSes.
On the other hand, there are some functions that the DBMS can do,
in a vendor-specific way, that I would advise against. Indeed, there
are some functions that the DBMS can do in a completely generic way,
that should nevertheless be done elsewhere. Oracle's top TPC-C benchmark
is an example of this last category.
Joe

We are going to have to end this one with a total disagreement. Having
spent more than 33 years in IT and more than 15 of them with relational
databases I have yet to ever see an example of:

"Complete DBMS *dependence* means utilizing (all) those DBMS-vendor
specific functions that optimize or implement security, performance, and
scalability (and other stuff)"

and

"Complete DBMS independence means that a system is not bound to a given
DBMS, because it uses only the functionality offered by the DBMS that is
accessible via DBMS-neutral syntax"

What you suggest is a logical impossibility.

In Oracle, for example, there is no better place to put code than in a
package; for numerous reasons. No other database vendor has the concept
.... therefore throw packages away.

In Oracle and DB2 the best way to do numering is with a sequence. SQL
Server and Sybase don't have them so throw sequences away. Of course SQL
Server and Sybase have autonumbering that Oracle doesn't have so throw
autonumbering away too. Guess we won't have any surrogate keys except by
wrapping your own ... now that's generic. All we need to do now is
recompile for every operating system.

Sorry Joe ... but your attempt to construe development to sell your
company's product is a great way to do some things. Creating high
performance scalable databases isn't one of them. Not once has Oracle
won a benchmark contest with BEA using generic code. No one else will
either.

--
Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
(e-mail address removed)
(replace 'x' with a 'u' to reply)
 
J

Joe Weinstein

We are going to have to end this one with a total disagreement. Having
spent more than 33 years in IT and more than 15 of them with relational
databases I have yet to ever see an example of:

"Complete DBMS *dependence* means utilizing (all) those DBMS-vendor
specific functions that optimize or implement security, performance, and
scalability (and other stuff)"

I can't understand your inability to grok this. If a product is written
using DBMS-vendor-specific functionality in a way which makes the product
unportable to another DBMS, it is DBMS-specific and dependent on that DBMS.
and

"Complete DBMS independence means that a system is not bound to a given
DBMS, because it uses only the functionality offered by the DBMS that is
accessible via DBMS-neutral syntax"

A simple-minded product that connected to any JDBC-accessible DBMS, and
sent only fresh SQL queries is an example. Not powerful but certainly
DBMS-vendor independent.
What you suggest is a logical impossibility.

How did I do this time? What logical failure is there in my simple examples
above?
In Oracle, for example, there is no better place to put code than in a
package; for numerous reasons. No other database vendor has the concept
... therefore throw packages away.
In Oracle and DB2 the best way to do numering is with a sequence. SQL
Server and Sybase don't have them so throw sequences away. Of course SQL
Server and Sybase have autonumbering that Oracle doesn't have so throw
autonumbering away too. Guess we won't have any surrogate keys except by
wrapping your own ... now that's generic. All we need to do now is
recompile for every operating system.

No! You seem to be working on false presumptions. Repeating ad nauseum, I
advise against complete DBMS-independence. Using oracle packages would not
impose a severe enough problem with the generality of the DBMS client part
of the application for me to suggest that they not be used. If you need
numbering, you have to do it DBMS-specifically, or wrap your own. I don't
understand about the recompile comment. Java, for instance doesn't need it.
The other point is that there *are* some things that you should not have the
DBMS do. The reason may be that the functionality requires too much
specificity in the product outside the DBMS, or it may be simply be because
the part of the product outside the DBMS can do it better.
Sorry Joe ... but your attempt to construe development to sell your
company's product is a great way to do some things. Creating high
performance scalable databases isn't one of them. Not once has Oracle
won a benchmark contest with BEA using generic code. No one else will
either.

No one ever stated that the current best practice is generic code. You
are busily beating a horse that only you brought to the race. The big
point you can't address, is that even using all Oracle's capabilities,
generic or proprietary, it goes into standard benchmarks with middleware
to make it go fast. This means that there are some things you don't want
the DBMS to do whether they are generic or not. Even if some of these
things are usually thought of as central or integral to a generic DBMS
product.
Proof:
http://www.tpc.org/tpcc/results/tpcc_perf_results.asp?resulttype=all
 
S

Stu Charlton

Daniel Morgan said:
"Complete DBMS *dependence* means utilizing (all) those DBMS-vendor
specific functions that optimize or implement security, performance, and
scalability (and other stuff)"

and

"Complete DBMS independence means that a system is not bound to a given
DBMS, because it uses only the functionality offered by the DBMS that is
accessible via DBMS-neutral syntax"

What you suggest is a logical impossibility.

I think I understand completely what Joe is trying to say, so I doubt
it is a logical impossibliity. I suggest it's a communications gap.
In Oracle, for example, there is no better place to put code than in a
package; for numerous reasons. No other database vendor has the concept
... therefore throw packages away.

That's not the suggestion at all. Packages are good things and should
be used in any case you have stored procedures.
In Oracle and DB2 the best way to do numering is with a sequence. SQL
Server and Sybase don't have them so throw sequences away. Of course SQL

Again, that's not the case. Keep your sequences.
Sorry Joe ... but your attempt to construe development to sell your
company's product is a great way to do some things. Creating high
performance scalable databases isn't one of them. Not once has Oracle
won a benchmark contest with BEA using generic code. No one else will
either.

I think the point is not about generic vs. specific code. Genericity
is a trade-off against performance, and isn't always applicable.

The point, I believe, is about trade-offs of what features to use when
you are building a commercial distributed system that involves several
product sets. Performance is usually the primary consideration with
any large system, but it rarely is the only consideration (except,
perhaps, for benchmarks).

Having said that, one might look at the TPC benchmark, as Joe has
pointed out, and wonder why they used BEA Tuxedo instead of Pro*C with
Oracle Shared Server / MTS. Tuxedo performs the same feature of
pooling / dispatching. Does it do it better than Shared Server? I
don't know - perhaps it's used out of inertia. But it is used, and
may successful systems have been built with such an approach. This is
an example of choosing the feature set of one product over Oracle's
equivalent.

Another example: sometimes, for skills support and maintainence
reasons, it makes sense to develop the presentation features of a
system (i.e. the dynamic web page generation) into a language such as
Java or C# -- something other than PL/SQL. Object oriented languages
are useful alternatives to procedural languages. In fact, this is
something that Oracle supports out of the box with Oracle 9i's HTTP
server and JSP runtime. This is not to say that it couldn't be done
with PL/SQL, it is to say the context of the situation may involve
tradeoffs.

Is the introduction of a middle tier an unscalable approach? Well
this is really a question of how you would like your parallelism - in
the database engine, or across server platforms (i.e. database +
middleware). Perhaps now with RAC, everything (including the web
server) could be on the same cluster and be more scalable. But
historically it has made sense to use different product sets for their
different strengths - TP monitors for their presentation libraries and
multiplexing support, Web Servers for their static file-serving
capabilities, and Oracle for its ability to chug through massive
amounts of data.

As for benchmarks...The TPC-C benchmark isn't a great example of how
to design an application - if you look at it, it actually doesn't
really use stored procedures that much, most of its SQL statements are
in C macros. It also uses multiple block sizes to try to get some
kind of performance enhancement, which as has been discussed in this
newsgroup, is of questionable value. Plus the TPC spec itself has a
rather odd setup scenario which has had an unrealistic affinity
towards shared-nothing clusters (notwithstanding the fact that a
shared-disk RAC configuration now holds the #1 spot).

Cheers
Stu
 
D

Daniel Morgan

Stu said:
I think I understand completely what Joe is trying to say, so I doubt
it is a logical impossibliity. I suggest it's a communications gap.




That's not the suggestion at all. Packages are good things and should
be used in any case you have stored procedures.




Again, that's not the case. Keep your sequences.




I think the point is not about generic vs. specific code. Genericity
is a trade-off against performance, and isn't always applicable.

The point, I believe, is about trade-offs of what features to use when
you are building a commercial distributed system that involves several
product sets. Performance is usually the primary consideration with
any large system, but it rarely is the only consideration (except,
perhaps, for benchmarks).

Please acknowledge that once your code is in packages and you are using
sequences ... the code absolutely can not be compatible with code
written for any other database product: No exceptions. It just will not
be compatible and no quantity of bubble gum, paper clips, or rubber
bands is going to make it compatible.

So while I will gladly acknowledge a communication gap I still can't get
past thinking what you guys are proposing is a logical impossibility. If
you disagree, and I suspect you do/will, then please provide an example
of how you could use packages and sequences in Oracle and meet your
stated objective.

--
Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
(e-mail address removed)
(replace 'x' with a 'u' to reply)
 
A

Adam Jenkins

Daniel said:
We are going to have to end this one with a total disagreement. Having
spent more than 33 years in IT and more than 15 of them with relational
databases I have yet to ever see an example of:

"Complete DBMS *dependence* means utilizing (all) those DBMS-vendor
specific functions that optimize or implement security, performance, and
scalability (and other stuff)"

and

"Complete DBMS independence means that a system is not bound to a given
DBMS, because it uses only the functionality offered by the DBMS that is
accessible via DBMS-neutral syntax"

What you suggest is a logical impossibility.

Perhaps your having spent 33 years in IT prevents you from actually
considering what someone else writes on the subject. Joe isn't actually
advocating either "complete DBMS dependance" or "complete DBMS
independance", as you seem to think, he's just defining the two
extremes. Then he goes on to explain some of the pros and cons of
different compromises between the two extremes. I don't see why you
keep bringing up this straw man of complete DBMS independance and
resulting terrible performance; noone in this thread is advocating that.

As I understand your argument, you're saying that since it's not
possible to write *completely* DBMS independant code without losing a
lot of performance and robustness offered by proprietary features, the
whole idea of DBMS independance is nonsense. A more reasonable approach
is to have DBMS independance as an ideal, but to recognize that gains in
performance and robustness can be had by using some proprietary and/or
non-universally supported DBMS features. So you take into account the
advantages of using a certain non-standard feature, and weigh it against
the extra cost of supporting it on the different DBMSes that you want to
support. Then you wrap the non-portable functionality that you decide
to use in an integration layer which needs to be reimplemented for each
DBMS. This is similar to the approach used for graphics APIs,
filesystems, network protocols, etc.

Adam
 
D

Daniel Morgan

Adam said:
Perhaps your having spent 33 years in IT prevents you from actually
considering what someone else writes on the subject. Joe isn't actually
advocating either "complete DBMS dependance" or "complete DBMS
independance", as you seem to think, he's just defining the two
extremes. Then he goes on to explain some of the pros and cons of
different compromises between the two extremes. I don't see why you
keep bringing up this straw man of complete DBMS independance and
resulting terrible performance; noone in this thread is advocating that.

As I understand your argument, you're saying that since it's not
possible to write *completely* DBMS independant code without losing a
lot of performance and robustness offered by proprietary features, the
whole idea of DBMS independance is nonsense. A more reasonable approach
is to have DBMS independance as an ideal, but to recognize that gains in
performance and robustness can be had by using some proprietary and/or
non-universally supported DBMS features. So you take into account the
advantages of using a certain non-standard feature, and weigh it against
the extra cost of supporting it on the different DBMSes that you want to
support. Then you wrap the non-portable functionality that you decide
to use in an integration layer which needs to be reimplemented for each
DBMS. This is similar to the approach used for graphics APIs,
filesystems, network protocols, etc.

Adam

All of this fluff, and I don't mean it in a prejorative way provides no
guideline. How would you make the decision as to which proprietary
features such as packages and sequences you would use and which ones not
to use.

Not once have I ever had a design spec. that said do a mediocre job of
implementing security, scalability, and performance.

Statements like "then you wrap the non-portable functionality that you
decide to use in an integration layer" are, from where I sit more fluff.
How do you write a wrapper around using a sequence versus using an
autonumbered column? I don't think it is technically possible.

I don't know you but strongly suspect your expertise is all front-end
.... Java ... and that you really have substantial database experience.

--
Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
(e-mail address removed)
(replace 'x' with a 'u' to reply)
 
S

Stu Charlton

Daniel Morgan said:
Please acknowledge that once your code is in packages and you are using
sequences ... the code absolutely can not be compatible with code
written for any other database product: No exceptions. It just will not
be compatible and no quantity of bubble gum, paper clips, or rubber
bands is going to make it compatible.
Acknowledged.

So while I will gladly acknowledge a communication gap I still can't get
past thinking what you guys are proposing is a logical impossibility. If
you disagree, and I suspect you do/will, then please provide an example
of how you could use packages and sequences in Oracle and meet your
stated objective.

Ok.

Requirement: A web-page that displays all of the employees in a given
department.

Solutions that are completely dependent on the database's feature set:
a. Build it with Marvel / HTML DB
b. Build it with mod_plsql and Oracle HTTP Server

Solutions that while still dependent on the database's feature set
(i.e. packages and stored procedures), it uses another technology for
its presentation logic:
a. Build it with mod_perl
b. Build it with JSP
c. Build it with ASP.NET

So, here's an example with ASP.NET (which I actually took and modified
into C# .NET from http://osi.oracle.com/~tkyte/ResultSets/index.html).
It's untested pseudocode, so probably has minor bugs in the ASP page.

CREATE OR REPLACE
PACKAGE DEPARTMENT AS
TYPE  CURSOR_TYPE IS REF CURSOR;
PROCEDURE GET_EMPS (I_DEPTNO     IN  NUMBER,
                O_RESULT_SET OUT CURSOR_TYPE);
END;
/

CREATE OR REPLACE
PACKAGE BODY DEPARTMENT AS
PROCEDURE GET_EMPS (I_DEPTNO     IN  NUMBER,
                     O_RESULT_SET OUT CURSOR_TYPE)
AS
BEGIN
OPEN O_RESULT_SET FOR
     SELECT EMPNO, ENAME
     FROM EMP
     WHERE DEPTNO = I_DEPTNO;
END;
END;

employees.aspx:

<%@ Page language="c#" %>
<%@ Import Namespace="System.Data" %>
<html>
<body>
 <h2>Query Employees in Department Code</h2>
 <form id="Form1" method="post" runat="server">
<asp:textbox id="DeptCode" runat="server"/>
<asp:button id="Submit" runat="server" Text="Run Query"/>
<asp:DataGrid id="MyDataGrid" runat="server" width="800" />
</form>
<script language="c#" runat="server">
void Page_Load(object src, EventArgs e) {
Submit += new System.EventHandler( RunQuery );
}
void RunQuery(object src, EventArgs e) {
OracleConnection conn = new OracleConnection();
OracleCommand cmd = new OracleCommand();
conn.ConnectionString = "data
source="oracle.server;uid=scott;password=tiger";
cmd.Connection = conn;
cmd.CommandText = "SCOTT.DEPARTMENT.GET_EMPS";
cmd.CommandType = CommandType.StoredProcedure;
OracleParameter i_deptno =
cmd.Parameters.Add("I_DEPTNO",
OracleClient.OracleType.Number);
i_deptno.Direction = ParameterDirection.Input;
OracleParameter o_resultset =
cmd.Parameters.Add("O_RESULT_SET",
OracleClient.OracleType.Cursor);
o_resultset.Direction = ParameterDirection.Output;
conn.Open();

i_deptno.Value = Int32.Parse ( DeptCode.Text );
OracleDataReader results = cmd.ExecuteReader();
MyDataSet.DataSource = results;
MyDataSet.DataBind();
}
</script>
</body>
</html>

The whole point of this is that we *didn't* use the Oracle-built in
features of HTML DB or mod_plsql, even though many successful people
CAN do so. It's really a matter of tradeoff of available skills,
future maintenance, and performance. In this case, I don't really
think the ASP.NET approach is any less scalable than Oracle-only
approaches.

The problem that too many Java or .NET folks cause is that they
re-invent database features all the time in the misguided attempt to
be "generic" because they don't want to be tied to their database
economically. This is not appropriate and is usually too drastic a
trade-off.

Anyhow, I can't speak for Joe, but this is my understanding of the
viewpoint.

Cheers
Stu
 
D

Daniel Morgan

Stu said:
Ok.

Requirement: A web-page that displays all of the employees in a given
department.
Cheers
Stu

First let me say you have my admiration. You put in effort and made an
attempt at answering my query. But the problem is that your example very
carefully, perhpas even artfully, dodged the issue I put in front of you
like a charging rhinoceros. What I asked was "please provide an example
of how you could use PACKAGES and SEQUENCES in Oracle and meet your
stated objective" ... and you didn't.

By selecting as your example: "A web-page that displays" you eliminated
anything having to do with sequences. What I was trying to get you to
deal with was issues related to insert, update, and delete. Issues that
involve sequences but also the transaction and locking models that are
totally different between commercial database products.

So the response while impressive is unsatisfactory in that it didn't
address the issue. And the stuff about Project Marvel ... as marvelous
as it may be ... has nothing to do with the back-end database ...
HTML_DB is a front-end tool. Could I impose upon you to try again this
time addressing this issues around transaction processing.

I still content that you can't have your cake and eat it too.

Thanks.

--
Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
(e-mail address removed)
(replace 'x' with a 'u' to reply)
 
V

Vladimir Andreev

There is NOTHING in a middle-tier that couldn't be done by a DBMS

There is, Sybrand. Connection pooling, for one. Just the other day I
saw a database that was serving hundreds of logins per minute. Guess
what kind of queries popped on the top of my StatsPack report?
Recursive SQL. Pure overhead.

Flado
 
D

Daniel Morgan

Joe said:
I can't understand your inability to grok this. If a product is written
using DBMS-vendor-specific functionality in a way which makes the product
unportable to another DBMS, it is DBMS-specific and dependent on that DBMS.
Agreed.

Agreed.

A simple-minded product that connected to any JDBC-accessible DBMS, and
sent only fresh SQL queries is an example. Not powerful but certainly
DBMS-vendor independent.

Everytime someone disputes my point-of-view they do so with examples
that don't involve transactions. My challenge to you, made as blatantly
obvious as an unwashed purple rhinoceros in the middle of a living room
has involved 'sequences' for a reason. Anyone can come up with a
vendor-neutral select statement. I am challenging you to deal with
transactions and locking.
How did I do this time? What logical failure is there in my simple examples
above?

As I've said so many times and I am beginning to wonder if I need to
SCREAM it ... you, and others that have agreed with you, have not once
responded with a transaction ... only with read-only SELECT statements.

Now perhaps it is just me ... but every time I connect a relational
database to a web front-end I expect it to be able to insert, update,
and delete ... not just select. I expect it to be able to create new
records, to lock records, to commit and rollback. Is my usage that unusual?

If you agree that a N-tier application should be able to create insert,
update, and delete, transactions ... then explain to me how you can be
vendor neutral and WITH THE SAME CODE handle Oracle with multi-version
conurrency (readers don't block writers and writers don't block readers)
and also support other RDBMS implementations? Same goes for sequences
vs. autonumbering columns. What are you goint to wrap this in?

Sure you can write your own table and increment a value in serial
fashion ... but that's not using native functionality. That's returning
to technoloogy I implemented back in the 1970s.
No! You seem to be working on false presumptions. Repeating ad nauseum, I
advise against complete DBMS-independence.

I repeat it because so far it has not been addressed except in read-only
examples involving select statements.

Using oracle packages would not
impose a severe enough problem with the generality of the DBMS client part
of the application for me to suggest that they not be used.

But as packages don't exist in any other RDBMS implementation the
front-end would need to be recoded. Agreed?

If you need
numbering, you have to do it DBMS-specifically, or wrap your own.

At which point you are 100% vendor dependent. Any wrap-your-own will
create serialization which greatly limits scalability. Agreed?
The reason may be that the functionality requires too much
specificity in the product outside the DBMS, or it may be simply be because
the part of the product outside the DBMS can do it better.

If anything can be done better outside the DBMS then I agree that
doesn't belong in the DBMS except as it may be required to guarantee
security and integrity if someone accesses the data with another tool.
Agreed?

I am not a database bigot. I put functionality in the front-end, in
middle tiers, anywhere that it makes sense. The issue from my standpoint
is not where functionality resides ... on that we are basically in
agreement.

My dispute is with the thought that I can be both vendor neutral and
vendor dependant. You can not have your cake and eat it too.

--
Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
(e-mail address removed)
(replace 'x' with a 'u' to reply)
 
J

JEDIDIAH

["Followup-To:" header set to comp.database.oracle.]
Hi Joe,
I am a Siebel configurator/programmer (Siebel is a "Customer
Relationship Management" software, which can be considered analogous
to SAP). My personal experience with the issue which interests you is
that as much as possible should be stored in the database. Siebel, by
some twisted reasoning about compatibility of code on various
databases (it runs on Oracle, SQL Server and DB2), decided to keep

That's just plain silly. Given that all of those products are variants
on a single specification based on SQL, the DDL is the least likely
part of these products to present cross-platform development issues.
Niggling quirks with datatypes (like date/time) are going to be far
more interesting.

My last project involved a product that was ported from msSQL to Oracle
(and then back) with another DB2 port in progress. Outside of stored
procedures, it was pretty simple to use perl to convert an Oracle schema
into something suitable for msSQL.

[deletia]
 
J

JEDIDIAH

["Followup-To:" header set to comp.database.oracle.]
I appreciate your opinion and your honesty that your perspective comes
from selling that middle tier but I completely disagree.

The 'lets push more bytes down the pipe and across all those routers'
thinking is not going to lead to performance. You may be scalable but
performance will suffer. And you will be no more scalable than a thinner
client.

Render under to database everything you can do in the database and let
the middle tier do what it does best ... fail-over, load levelling, and
serving up the front-end.

You are painting with far too broad a brush. Some of your examples below
represent absurd examples of middle-tier database processing. They would
(and do) horrify anti-centrists as much as they do you. Some of them
demonstrate that even good ideas can be poorly implemented.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,769
Messages
2,569,582
Members
45,063
Latest member
StormyShuf

Latest Threads

Top