Is perl better? :(((

T

Tom Dyess

kjc said:
Dude, I think you badly want to retract your comments.
The code posted by the OP is correct along one dimension.
He is setting a query value 2K times FIRST, THEN, issuing the query.


Your code example sends a query 2K times to the DB server.
Defintely, NOT a good idea.

Dude! Thanks for that memo from the department of the obvious. Lol. I'm
trying to determine which he wants. Trying to understand the idea behind
what he is trying to do without actually seeing the full code. Based on his
comments, hitting the SQL 2000 times seems like that's what he wants to do,
which is yes, a bad idea, but if that's a requirement, that's a requirement.
In that case, NO I don't want to retract the statement, because to
troubleshoot, you need to completely understand a) what's currently
happening, b) what the desired effect is and in this case c) how to achieve
the desired effect in a much faster manner.

Consider this snippit: Tell me, what's the point of setting a prepared
statement 2000 times without executing the statement? Does that seem strange
to you? It does to me. When I see that and not the proper factory creation
for a prepared statement, connection.prepareStatement(String), it sends a
red flag to which I then presented that reply for further clarification. I
wasn't suggesting running the query 2000 times to increase the speed, I was
merely asking if that is what he wanted based on the code that I saw. KJC,
please read the entire thread first before blurting out. Alex, please
clarify.

conection.getPreparedStatement("select 'key', sysdate+? from
dual");

for (int i=0; i<2000; i++)
ps.setInt(1, i);

rs = ps.executeQuery();

Tom Dyess
OraclePower.com
 
A

akizub

Sorry. Of course I mistyped this code and forgot one { after for.
I think it's obvious because I had odd number of braces and there is no
sense to set parameter 2000 times and then do one execution.

OK. Here is "exact" (almost) code.
Point is not in few new Double or SQL tune.
It is exactly the same in perl.
Point is significant difference in time.
Java is about 2.5 slower!
Probably this is OCI Orcale connection for perl.
I can't install it for JDBC (don't have rights).

If I use more simple SQL (something like select sysdate from dual)
then perl and Java have the same time.
Which is obvious because bottle neck is Oracle.
But even in this example Oracle is main time eater!
And why Java realization is slower I have no idea.


//================ DA.java ===============
import java.sql.*;
import java.util.*;
import java.io.*;
import java.net.*;

public class DA {
String url;
String user;
String password;
public Connection con;
public Statement stmt;

/** Load Oracle driver one time for application */
static {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
} catch (Exception ex) {
System.out.println("static driver not found: " + ex.getMessage());
}
}

public DA () {
}

public String openCon(String url, String user, String password) {
if (url instanceof String
&& url.length() > 0
&& user instanceof String
&& user.length() > 0
&& password instanceof String
&& password.length() > 0) {
this.url = url;
this.user = user;
this.password = password;
try {
con = DriverManager.getConnection(url, user, password);
if (con == null)
return "ERROR: DBAccess: connection is null!";
stmt = con.createStatement();
if (stmt == null)
return "ERROR: DBAccess: statement is null!";
} catch (Exception ex) {
System.out.println(
"Connection failed "
+ ex.getMessage()
+ "\n"
+ url
+ "\n"
+ user
+ " "
+ password);
return "Connection failed " + ex.getMessage() + "\n" + url + "\n";
}
return "OK";
} else
return "Error.";
}

/** Close current database connectionand statements */
public String closeCon() {
try {
stmt.close();
con.close();
return "OK";
} catch (Exception ex) {
return ("close connection " + ex.getMessage());
}

}

public static void main(String[] args) {
System.out.println("DBAccess test START!");

new DA1().run();

System.out.println("DBAccess test END!
Java="+System.getProperty("java.version","Not defined"));

}

public void run() {
openCon("jdbc:eek:racle:thin:mad:yahoo.com:1521:users",
"scott","tiger");

try {
ResultSet rs = null;
PreparedStatement ps =
getPreparedStatement("select sysdate+? from dual");
ps.setInt(1, 365);

rs = ps.executeQuery();
if (rs.next()) {
System.out.println("Prepared statement " + rs.getString(1));
}
rs.close();

} catch (Exception ex) {
System.out.println("SQL run Exception " + ex.getMessage());
}

closeCon();
}

public PreparedStatement getPreparedStatement(String sql)
throws java.sql.SQLException {
return con.prepareStatement(sql);
}

//////////// method itself ///////////////
public Vector table; // vector of String[5]; 2000 elements
// collecting code, month, userid, start, end,
public Hashtable totaled_hrs=new Hashtable();
public void getData()
{
try{
DA da=new DA();
ResultSet rs;
Properties pr=new Properties();
pr.load("/config/common.properties");

da.url=pr.getProperty("URL");
da.user=pr.getProperty("User");
da.password=pr.getProperty("Password");

if (!"OK".equals(da.openCon(da.url,da.user,da.password)))
System.out.println( "Error1");

double value=0;
String tmp[]=null;
double total=0,time=0;
String month="00";

String sql="SELECT SUM(DECODE(TO_CHAR(e.date, 'MM'), ?, e.hours,
0)) hours, \n"+
"d.person_type \n" +
"FROM projects a, tasks b, assignments c, resources d, hours e,
\n"+
"super_projects h, users g \n"+
"WHERE \n"+
"d.id = ? \n"+
"AND (e.date >= TO_DATE(?, 'mm/dd/yyyy')) \n"+
"AND (e.date <= TO_DATE(?, 'mm/dd/yyyy')) \n"+
"AND a.id = b.projectid \n"+
"AND b.id = c.taskid \n"+
"AND c.resourceid = d.id \n"+
"AND c.id = e.assignmentid \n"+
"AND e.approved = 'yes' \n"+
"AND a.superid = h.id \n"+
"AND h.name like 'Billed %' \n"+
"AND a.name not like 'Local %' \n"+
"GROUP BY d.person_type \n";

PreparedStatement ps = da.getPreparedStatement(sql);

for(int i=0; i<table.size(); i++)
{
tmp=(String[])(table.get(i));

ps.setString(1, tmp[1]);
ps.setString(2, tmp[2]);
ps.setString(3, tmp[3]);
ps.setString(4, tmp[4]);
rs = ps.executeQuery();

if (rs.next())
{
value=rs.getDouble(1);

if ("volunteer".equals(rs.getString(2)))
{
time=0;
}
else
{
time=value;
}
}
total+=time;
Double d=(Double)totaled_hrs.get(tmp[0]);
if (d!=null)
{
time += d.doubleValue();
}
totaled_hrs.put(tmp[0], new Double(adjusted_summed_time));

}
rs.close();
}
da.closeCon();
System.out.println("Hours: "+total);
} catch (Exception e){
System.out.println( "Error "+e); }

}


}
################# perl ########################
#!/perl/5.6/bin/perl
use lib "/perl/5.6/DBD/oracle8.1.6/lib/site_perl/5.6.0/sun4-solaris";
use Oraperl;

.... open connection ....

sub getData{

my(@persons)=@_;
$total=0;
%totaled_hrs=();

foreach $key(@persons){

@one_person=split(/:/,$key);

$sql=...;

$db_list=&ora_open($lda,$sql) || die "\nCan't open cursor
1: $ora_errstr\n";

($time, $type)=&ora_fetch($db_list);

if($type eq "volunteer"){
$time=0;
}
$totaled_hrs{$one_person[0]}+=$time;
$total+=$time;

}


Alex Kizub.
 
J

John C. Bollinger

It's not big deal. I have 2000 SQL so I have 2000 new Doubles.
Sure it's not the point to lost 10 minutes.
I'm talking about significant difference.

You don't know how much of a difference it makes because you refuse to
profile. I agree that creating 2000 Doubles ought not to take any
significant fraction of 10 minutes, but then your whole test program
ought not to take more than a few seconds unless there is a bottleneck
elsewhere (e.g. network, DB).


John Bollinger
(e-mail address removed)
 
J

John C. Bollinger

Here is code. Exactly. Except the SQL.

Evidently it isn't. I observe, for instance, that you misspelled
"prepareStatement" as "getPreparedStatement":
PreparedStatement ps =
conection.getPreparedStatement("select 'key', sysdate+? from
dual");

There is at least one other likely error in your code that has already
pointed out elsewhere (the extent of your loop), but I don't know
whether that's in the real code or whether it's a typo.


John Bollinger
(e-mail address removed)
 
J

John C. Bollinger

Of course creating of thousand of threads could be more useful.
Unlikely.

But
what about poor Oracle? To be honest, our DBAs already called me and
asked to stop my SQL and run in the middle of the night. It was first
call from them in 5 years.

This should be a gigantic red flag telling you that your Java program is
doing something _significantly_ different from what your Perl script
does. (The timing difference should be another, smaller flag.) If the
Java program were generating the same SQL traffic that the Perl program
does then the DBA wouldn't be able to tell the difference. Does the
JDBC driver have an option by which you can enable some kind of
client-side logging of traffic with the DB? I suspect the results would
be quite revealing.


John Bollinger
(e-mail address removed)
 
R

Robert Klemme

Sorry. Of course I mistyped this code and forgot one { after for.
I think it's obvious because I had odd number of braces and there is no
sense to set parameter 2000 times and then do one execution.

OK. Here is "exact" (almost) code.

Apparently not. I suggest you post code that compiles and is properly
indented. If you want feedback from people around you have to do at least
a bit to make people's life's simpler.

Regards

robert
 
A

akizub

Apparently not. I suggest you post code that compiles and is properly
indented.

This code works.
Unfortunately, when it was posted to the news/forum it was broken.
But you can see the idea. And see critical parts. And see that
everything is ok.

When I post short cut from code people want to see whole. When I post
whole code people don't like because it's too big.
Why do you suspect me that I miss double quote or curly braces and
start this big discussion?

Code is simple in both languages.
And this is not about
- tune SQL
- make connection
- improve performance
- change hardware, software, database or it's structure or algorithm
- do my work

Question is simple:
Why for the same SQL, hardware, database, network, time and so on
for small amount of 2000 SQL perl does it job for 17 minutes when java
does for 43?

If you divide these 43 miniutes for 2000 you can see that bottle neck
is definitely not couple new Doubles but connection to Oracle. And as
you can see SQL are too slow.
But why they are faster for perl? There is only one connection, as you
can see in source. What slows SQLs so much?

Agian. I don't like this database. It s... I don't like this SQL. I
don't like this project which was created ASAP (As Stupid As Possible)
and so on so on.

But qestion remains: why it is so significant difference for two
languages for all the same environment/etc...?

Alex Kizub.
 
K

kjc

indented.

This code works.
Unfortunately, when it was posted to the news/forum it was broken.
But you can see the idea. And see critical parts. And see that
everything is ok.

When I post short cut from code people want to see whole. When I post
whole code people don't like because it's too big.
Why do you suspect me that I miss double quote or curly braces and
start this big discussion?

Code is simple in both languages.
And this is not about
- tune SQL
- make connection
- improve performance
- change hardware, software, database or it's structure or algorithm
- do my work

Question is simple:
Why for the same SQL, hardware, database, network, time and so on
for small amount of 2000 SQL perl does it job for 17 minutes when java
does for 43?

If you divide these 43 miniutes for 2000 you can see that bottle neck
is definitely not couple new Doubles but connection to Oracle. And as
you can see SQL are too slow.
But why they are faster for perl? There is only one connection, as you
can see in source. What slows SQLs so much?

Agian. I don't like this database. It s... I don't like this SQL. I
don't like this project which was created ASAP (As Stupid As Possible)
and so on so on.

But qestion remains: why it is so significant difference for two
languages for all the same environment/etc...?

Alex Kizub.
Alex,
the best thing to do is run it through a profiler.
After revewing the output from a profiler, you should have your answer.
 
A

akizub

Of course creating of thousand of threads could be more useful.
Unlikely.
Who knows. We can discuss it in separate thread. What I wanted to say
that perl and Java do it in the same way. 2000 single SQLs and store
results.
If the Java program were generating the same SQL traffic that the Perl
program
does then the DBA wouldn't be able to tell the difference.

It was exactly perl program running when I had this call.
I just wanted to say that Oracle was evidently overworked and using the
treads could not be big difference.

Does the JDBC driver have an option by which you can enable some
kind of
client-side logging of traffic with the DB?

Don't know. Never tried. It's the production server and production
database so I don't have enough power to do something.
What I can do is to print SQL, results, time.
Good enough for me to be sure that it's probably the same.
And, BTW, perl uses SQL created on the fly and Java uses prepared
statement so they can't be compared. Of course I could and did use
creating SQLs on the fly in Java too.
SQLs were the same.

Again. My goal is to use Java. Not perl.
But how can I do this when I have such bad comparison results.
Alex Kizub.
 
D

Dimitri Maziuk

Chris Smith sez:
I don't know about that; a little slower than C or FORTRAN in the
average case, sure. A little slower than Perl, though?

JVM startup time and garbage collection.
In any case, something is clearly wrong here besides the language.

Yep. I'd look into CLOSE_CURSORS_ON_COMMIT (or whatever it's
called) if I were OP.

Dima
 
C

Chris Smith

100% that all is the same.
100% that Java is not wasting the time.
And I'm sure in 100% that it is something with JDBC and Java
middleware.

So, in other words, you haven't bothered to use a profiler yet. Stop
wasting eveyone's time, then. If you wish to struggle against the
unknown and refuse to use tools that will help you, that's your problem.
Just go away and do it on your own. It's becoming clear that you aren't
willing to work with anyone to solve your problem.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
L

Luke Webber

Tor said:
Perl has (in theory at least) both of those as well, though.

And we're talking about a process that runs for as much as ten minutes,
so it should not be significant.

Luke
 
L

Luke Webber

indented.

This code works.
Unfortunately, when it was posted to the news/forum it was broken.
But you can see the idea. And see critical parts. And see that
everything is ok.

Nope, not a bit. I've gotta tell tou, that code isn't even close to
running. What are all these references to DA1? Where is "table"
initialised (hint: it's not). Also, you have too many right-braces in
your getData method before the catch statement, "adjusted_summed_time"
is not defined, the call to pr.load(InputStream) is passing a filename
instead of an InputStream...
When I post short cut from code people want to see whole. When I post
whole code people don't like because it's too big.
Why do you suspect me that I miss double quote or curly braces and
start this big discussion?

Calm down. It's not suspicion, we're trying to help, but if you want us
to help, you need to be clear on what you're trying to do. And you've
been very unclear to date.
Code is simple in both languages.

Actually, the PERL code is a lot simpler. I don't speak PERL myself, but
if that's all there is to the PERL, you're doing a lot more in the Java
version.
And this is not about
- tune SQL
- make connection
- improve performance
- change hardware, software, database or it's structure or algorithm
- do my work

Question is simple:
Why for the same SQL, hardware, database, network, time and so on
for small amount of 2000 SQL perl does it job for 17 minutes when java
does for 43?

Show us working code and maybe we can tell you. Perhaps you could put it
up on a web server somewhere for us to download. And /please/ make sure
that it's complete, or you're wasting the time of people who are trying
to help you. For free, I might add.

Luke
 
A

akizub

Luke:
we're trying to help
And I really appreciate it.
I'm only a little bit upset when people say that I'm unwiling to
work...

Let's start again with willings.

Profile doesn't have sense for me. It's obvious.
We are waiting for Oracle answer:
java -Xrunhprof:cpu=samples -classpath .:eek:jdbc14.jar DA
rank self accum count trace method
1 74.08% 74.08% 889 102 java.net.SocketInputStream.socketRead0
2 5.42% 79.50% 65 133
java.net.SocketOutputStream.socketWrite0
3 1.33% 80.83% 16 145 oracle.jdbc.ttc7.TTCItem.unmarshal
4 1.17% 82.00% 14 4 java.lang.ClassLoader.defineClass0
5 0.92% 82.92% 11 19
java.lang.ClassLoader.findBootstrapClass
6 0.83% 83.75% 10 143
oracle.jdbc.ttc7.MAREngine.unmarshalCLRforREFS
7 0.58% 84.33% 7 159
oracle.jdbc.ttc7.TTCAdapter.createNonPlsqlTTCColumnArray
8 0.58% 84.92% 7 150
oracle.jdbc.ttc7.TTC7Protocol.createDBItem
9 0.50% 85.42% 6 158 java.lang.StringBuffer.toString

I do not believe in profiling. It takes time.
And during this time Oracle is working too!
So, I prefer more simple way - just count milliseconds.
For whole program, for database, (for sql), for cycle.
See code.

And here is exact code (except sql):
//========= DA.java =============

import java.io.*;
import java.sql.*;
import java.util.*;

public class DA {
public Connection con;
public Statement stmt;

static {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
} catch (Exception ex) {
System.out.println("static driver not found: "
+ ex.getMessage());
}
}

public void openCon(String url,
String user, String password) {
try {
con = DriverManager.getConnection(url, user, password);
stmt = con.createStatement();
} catch (Exception ex) {
System.out.println(ex);
}
}

public void closeCon() {
try {
stmt.close();
con.close();
} catch (Exception ex) {
System.out.println(ex);
}
}

public static void main(String[] args) {
new DA().getData();
System.out.println("Java="+System.getProperty("java.version"));
}

public PreparedStatement getPreparedStatement(String sql)
throws java.sql.SQLException {
return con.prepareStatement(sql);
}

//////////// method itself ///////////////
public Vector table; // vector of String[4]; 2000 elements
// collecting_code|userid|start|end
public Hashtable totaled_hrs=new Hashtable();
public void getData()
{
try{

double mth_hours=23*7.75; // for December 2004
String oldCode=null;

table=new Vector();
String tmp[]=null;
BufferedReader br=new BufferedReader(new FileReader("load"));
String line;
while ((line=br.readLine())!=null){
StringTokenizer st=new StringTokenizer(line,"|");
tmp=new String[4];
tmp[0]=st.nextToken();
tmp[1]=st.nextToken();
tmp[2]=st.nextToken();
tmp[3]=st.nextToken();
table.add(tmp);
}
br.close();
System.out.println(table.size());
long tall,tdb,tsql,tcycle,v2,v3;
tall=tdb=tsql=tcycle=0;

DA da=new DA();
ResultSet rs;
Properties pr=new Properties();
pr.load(new FileInputStream("common.properties"));

//openCon("jdbc:eek:racle:thin:mad:yahoo.com:1521:users",
// "scott","tiger");

da.openCon(pr.getProperty("url"),
pr.getProperty("user"),pr.getProperty("password"));

double value=0,total=0;
String type="";

String sql=
"SELECT SUM(DECODE(TO_CHAR(e.date, 'MM'), \n"+
" '12', e.hours,0)) hours, \n"+
" d.person_type \n" +
" FROM projects a, tasks b, assignments c, \n"+
" resources d, hours e, \n"+
" super_projects h, users g \n"+
" WHERE \n"+
" d.id = ? \n"+
" AND (e.date >= TO_DATE(?, 'mm/dd/yyyy')) \n"+
" AND (e.date <= TO_DATE(?, 'mm/dd/yyyy')) \n"+
" AND a.id = b.projectid \n"+
" AND b.id = c.taskid \n"+
" AND c.resourceid = d.id \n"+
" AND c.id = e.assignmentid \n"+
" AND e.approved = 'yes' \n"+
" AND a.superid = h.id \n"+
" AND h.name like 'Billed %' \n"+
" AND a.name not like 'Local %' \n"+
" GROUP BY d.person_type \n";

PreparedStatement ps = da.getPreparedStatement(sql);
tall=System.currentTimeMillis();

v2=System.currentTimeMillis();

for(int i=0; i<table.size(); i++)
{
tmp=(String[])(table.get(i));
v3=System.currentTimeMillis();
tcycle+=v3-v2;

ps.setString(1, tmp[1]);
ps.setString(2, tmp[2]);
ps.setString(3, tmp[3]);
rs = ps.executeQuery();
tsql+=System.currentTimeMillis()-v3;
if (!rs.next()) continue;
value=rs.getDouble(1);
type=rs.getString(2);
rs.close();

v2=System.currentTimeMillis();
tdb += v2-v3;

if (value>mth_hours&&"Billed".equals(type)) value=mth_hours;

total+=value;
MyDouble d=(MyDouble)totaled_hrs.get(tmp[0]);
if (d!=null) d.value+=value;
else {
d=new MyDouble();
d.value=value;
totaled_hrs.put(tmp[0],d);
}
if (!(tmp[0].equals(oldCode))&&oldCode!=null)
System.out.println(
oldCode+" "+
((MyDouble)totaled_hrs.get(oldCode)).value+
" "+total);
oldCode=tmp[0];
}// end cycle

da.closeCon();
tall=System.currentTimeMillis()-tall;

System.out.println("Hours: "+total+
" time all="+tall+
" time db="+tdb+
" (sql="+tsql+")"+
" time cycle="+tcycle);
} catch (Exception e){
e.printStackTrace();
}
}

class MyDouble{public double value=0;}

}
//////////// end DA.java /////////////

I hope it will post well. I'll double check it and
if it will be bad I'll put it somewhere in my web site.

And here is perl code:
################# perl ########################
#!/perl/5.6/bin/perl
use lib "/perl/5.6/DBD/oracle8.1.6/lib/site_perl/5.6.0/sun4-solaris";
use Oraperl;

.... open connection ....

sub getData{

my(@persons)=@_;
$total=0;
%totaled_hrs=();

foreach $key(@persons){

@one_person=split(/:/,$key);

$sql=...;

$db_list=&ora_open($lda,$sql) || die "\nCan't open cursor
1: $ora_errstr\n";

($time, $type)=&ora_fetch($db_list);

if ($time>178.25) {
if ( "Billed" eq $type ) {
$time=178.25;
}}

$totaled_hrs{$one_person[0]}+=$time;
$total+=$time;

}
############# end of perl ################

Result of java is
Hours: 154930.95 time all=5436908 time db=5297897 (sql=5434009) time
cycle=243005
Java=1.4.2_01

Which means 90 minutes for whole run.
Where 88 minutes for Oracle waiting.
Total Java expenses 4.5 minutes.
(Please, don't be too strict to numbers.
I just cut&paste them. Milliseconds are a little big time period.
Be indulgent to this quick and not precision way.)

perl result is:
Hours: 154930.95 for 1487 seconds.
Which is 25 minutes today.
Which is 3.6 times faster.

So, I'm open to any suggestions how to improve my Java code
or how to force managers to use Java in this project.

Alex Kizub.
 
A

akizub

I do not believe in profiling. It takes time.
You ask experts for advice, and then refuse to take it. Your choice.
Read the post! I did profile! And it was as I expected.
Program itself is so simple that do not require profile.
It's obvious that problem is somewhere in Oracle connection.
Which means that it doesn't matter how good is your program -
connection kills it.
Very pity.

I gave up and left this application on perl.

Alex Kizub.
 
R

Robert Klemme

Luke:
And I really appreciate it.
I'm only a little bit upset when people say that I'm unwiling to
work...

Let's start again with willings.

Profile doesn't have sense for me. It's obvious.
We are waiting for Oracle answer:
java -Xrunhprof:cpu=samples -classpath .:eek:jdbc14.jar DA
rank self accum count trace method
1 74.08% 74.08% 889 102 java.net.SocketInputStream.socketRead0
2 5.42% 79.50% 65 133
java.net.SocketOutputStream.socketWrite0
3 1.33% 80.83% 16 145 oracle.jdbc.ttc7.TTCItem.unmarshal
4 1.17% 82.00% 14 4 java.lang.ClassLoader.defineClass0
5 0.92% 82.92% 11 19
java.lang.ClassLoader.findBootstrapClass
6 0.83% 83.75% 10 143
oracle.jdbc.ttc7.MAREngine.unmarshalCLRforREFS
7 0.58% 84.33% 7 159
oracle.jdbc.ttc7.TTCAdapter.createNonPlsqlTTCColumnArray
8 0.58% 84.92% 7 150
oracle.jdbc.ttc7.TTC7Protocol.createDBItem
9 0.50% 85.42% 6 158 java.lang.StringBuffer.toString

Did you feed that into an analysis tool like HPJmeter? Maybe there's
something different going on on the network.

Also, for the db it might be more efficient if you include the data you
draw from variable "table" and execute a slightly modified SQL just once.

robert
I do not believe in profiling. It takes time.
And during this time Oracle is working too!
So, I prefer more simple way - just count milliseconds.
For whole program, for database, (for sql), for cycle.
See code.

And here is exact code (except sql):
//========= DA.java =============

import java.io.*;
import java.sql.*;
import java.util.*;

public class DA {
public Connection con;
public Statement stmt;

static {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
} catch (Exception ex) {
System.out.println("static driver not found: "
+ ex.getMessage());
}
}

public void openCon(String url,
String user, String password) {
try {
con = DriverManager.getConnection(url, user, password);
stmt = con.createStatement();
} catch (Exception ex) {
System.out.println(ex);
}
}

public void closeCon() {
try {
stmt.close();
con.close();
} catch (Exception ex) {
System.out.println(ex);
}
}

public static void main(String[] args) {
new DA().getData();
System.out.println("Java="+System.getProperty("java.version"));
}

public PreparedStatement getPreparedStatement(String sql)
throws java.sql.SQLException {
return con.prepareStatement(sql);
}

//////////// method itself ///////////////
public Vector table; // vector of String[4]; 2000 elements
// collecting_code|userid|start|end
public Hashtable totaled_hrs=new Hashtable();
public void getData()
{
try{

double mth_hours=23*7.75; // for December 2004
String oldCode=null;

table=new Vector();
String tmp[]=null;
BufferedReader br=new BufferedReader(new FileReader("load"));
String line;
while ((line=br.readLine())!=null){
StringTokenizer st=new StringTokenizer(line,"|");
tmp=new String[4];
tmp[0]=st.nextToken();
tmp[1]=st.nextToken();
tmp[2]=st.nextToken();
tmp[3]=st.nextToken();
table.add(tmp);
}
br.close();
System.out.println(table.size());
long tall,tdb,tsql,tcycle,v2,v3;
tall=tdb=tsql=tcycle=0;

DA da=new DA();
ResultSet rs;
Properties pr=new Properties();
pr.load(new FileInputStream("common.properties"));

//openCon("jdbc:eek:racle:thin:mad:yahoo.com:1521:users",
// "scott","tiger");

da.openCon(pr.getProperty("url"),
pr.getProperty("user"),pr.getProperty("password"));

double value=0,total=0;
String type="";

String sql=
"SELECT SUM(DECODE(TO_CHAR(e.date, 'MM'), \n"+
" '12', e.hours,0)) hours, \n"+
" d.person_type \n" +
" FROM projects a, tasks b, assignments c, \n"+
" resources d, hours e, \n"+
" super_projects h, users g \n"+
" WHERE \n"+
" d.id = ? \n"+
" AND (e.date >= TO_DATE(?, 'mm/dd/yyyy')) \n"+
" AND (e.date <= TO_DATE(?, 'mm/dd/yyyy')) \n"+
" AND a.id = b.projectid \n"+
" AND b.id = c.taskid \n"+
" AND c.resourceid = d.id \n"+
" AND c.id = e.assignmentid \n"+
" AND e.approved = 'yes' \n"+
" AND a.superid = h.id \n"+
" AND h.name like 'Billed %' \n"+
" AND a.name not like 'Local %' \n"+
" GROUP BY d.person_type \n";

PreparedStatement ps = da.getPreparedStatement(sql);
tall=System.currentTimeMillis();

v2=System.currentTimeMillis();

for(int i=0; i<table.size(); i++)
{
tmp=(String[])(table.get(i));
v3=System.currentTimeMillis();
tcycle+=v3-v2;

ps.setString(1, tmp[1]);
ps.setString(2, tmp[2]);
ps.setString(3, tmp[3]);
rs = ps.executeQuery();
tsql+=System.currentTimeMillis()-v3;
if (!rs.next()) continue;
value=rs.getDouble(1);
type=rs.getString(2);
rs.close();

v2=System.currentTimeMillis();
tdb += v2-v3;

if (value>mth_hours&&"Billed".equals(type)) value=mth_hours;

total+=value;
MyDouble d=(MyDouble)totaled_hrs.get(tmp[0]);
if (d!=null) d.value+=value;
else {
d=new MyDouble();
d.value=value;
totaled_hrs.put(tmp[0],d);
}
if (!(tmp[0].equals(oldCode))&&oldCode!=null)
System.out.println(
oldCode+" "+
((MyDouble)totaled_hrs.get(oldCode)).value+
" "+total);
oldCode=tmp[0];
}// end cycle

da.closeCon();
tall=System.currentTimeMillis()-tall;

System.out.println("Hours: "+total+
" time all="+tall+
" time db="+tdb+
" (sql="+tsql+")"+
" time cycle="+tcycle);
} catch (Exception e){
e.printStackTrace();
}
}

class MyDouble{public double value=0;}

}
//////////// end DA.java /////////////

I hope it will post well. I'll double check it and
if it will be bad I'll put it somewhere in my web site.

And here is perl code:
################# perl ########################
#!/perl/5.6/bin/perl
use lib "/perl/5.6/DBD/oracle8.1.6/lib/site_perl/5.6.0/sun4-solaris";
use Oraperl;

... open connection ....

sub getData{

my(@persons)=@_;
$total=0;
%totaled_hrs=();

foreach $key(@persons){

@one_person=split(/:/,$key);

$sql=...;

$db_list=&ora_open($lda,$sql) || die "\nCan't open cursor
1: $ora_errstr\n";

($time, $type)=&ora_fetch($db_list);

if ($time>178.25) {
if ( "Billed" eq $type ) {
$time=178.25;
}}

$totaled_hrs{$one_person[0]}+=$time;
$total+=$time;

}
############# end of perl ################

Result of java is
Hours: 154930.95 time all=5436908 time db=5297897 (sql=5434009) time
cycle=243005
Java=1.4.2_01

Which means 90 minutes for whole run.
Where 88 minutes for Oracle waiting.
Total Java expenses 4.5 minutes.
(Please, don't be too strict to numbers.
I just cut&paste them. Milliseconds are a little big time period.
Be indulgent to this quick and not precision way.)

perl result is:
Hours: 154930.95 for 1487 seconds.
Which is 25 minutes today.
Which is 3.6 times faster.

So, I'm open to any suggestions how to improve my Java code
or how to force managers to use Java in this project.

Alex Kizub.
 
A

akizub

Did you feed that into an analysis tool like HPJmeter?
I used HPjmeter-1.6. It's a little old. Latest version is for 2003. And
supports
JAVA PROFILE 1.0 when latest java produces JAVA PROFILE 1.0.1.
Nevertheless you can see original text file which is obvious enough.
Maybe there's something different going on on the network.
Problem is that both perl and java are working in the same environment,
the same time,
the same database and the same sql. And result is always the same.
perl at least twice faster. If java runs 90 minutes then perl runs 25.
If java runs 43 minyes then perl 17. And so on.
Also, for the db it might be more efficient if you include the data
you
draw from variable "table" and execute a slightly modified SQL just
once.

Of course I can modify SQL. But I have to do it for both langauges. So
this is not the point for difference. And, I assume, that 2000 SQLs are
good enough for statistic's measure.

Disappointing point is that if I use simple SQL like "select 1 from
dual' then both languages have the same result. Which means for me that
overhead charges are the same. When I change results (like increase
key numbers for hastable, make them unique = 2000 all different, but
for the same simple query) then java is faster.
Which means calculation itself are better in java.

But database connection kills everything.
I had another experience with IBM original JDBC connection to mainframe
DB2.
select for 1,000,000 items table was about 5 minutes.
But I thought that IBM forced programers to use COBOL not Java.
And gave up for it too. My Java world shrinks so rapidly....
:(((((((((((((((

Alex Kizub.
 
L

Luke Webber

Read the post! I did profile! And it was as I expected.
Program itself is so simple that do not require profile.
It's obvious that problem is somewhere in Oracle connection.
Which means that it doesn't matter how good is your program -
connection kills it.
Very pity.

I gave up and left this application on perl.

It seems clear that you're correct about the time being spent in the
JDBC connection logic, but I'm wondering just which part of the JDBC
code is taking all the time. The figures aren't fine-grained enough.

I have one off-the-wall suggestion. Why not try removing the
"rs.close()" statement and see how it goes? I have a niggling feeling
that it might be making an extra round trip to the database server for
some reason.

Cheers,
Luke
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,769
Messages
2,569,582
Members
45,070
Latest member
BiogenixGummies

Latest Threads

Top