.net Application Performance

[skotapal]

Hello

I manage a web based VB .net application. This application has 3
components:
1. Webapp (this calls the executibles)
2. database
3. business logic is contained in individual exe application that get
called in a sequence to do some heavy calculations (mainly DB
operations with in memory datasets)

I am having an issue with performance with this architecture. the exes
themselved are multi-threaded and can have multiple instances (this
means that there are these mutliple datasets on the app server). Are
there any guidelines that can help me establish how I can improve
performance - increasing memory, adding an app server (clustering),
adding cpus, etc. Any advice from you will be helpful!

Thanks

KS

 

[Michael Nemtsev]

Hello (e-mail address removed),

I manage a web based VB .net application. This application has 3
components:
1. Webapp (this calls the executibles)
2. database
3. business logic is contained in individual exe application that get
called in a sequence to do some heavy calculations (mainly DB
operations with in memory datasets)

What does "business logic is contained in individual exe" mean?
Who starts them, where are they hosted?

I am having an issue with performance with this architecture.

How do u get this?

the exes themselved are multi-threaded and can have multiple instances
(this means that there are these mutliple datasets on the app server).

Have u tried to use caching?

Are there any guidelines that can help me establish how I can improve
performance - increasing memory, adding an app server (clustering),
adding cpus, etc. Any advice from you will be helpful!

Firstly use some kind of performance tool to understand the bottleneck.

---
WBR,
Michael Nemtsev [C# MVP] :: blog: http://spaces.live.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

 

[skotapal]

Mike

Thanks for you resonse.

1. "business logic is contained in individual exe" means that the
webapp does not contain the actual calculations. These are kicked off
from the web app on a button click by the user - we call each run a
process.

2. The business logic itself reads the data from files... validates
them pumps them into the DB (SQL Server) - this happens pretty quickly.
Then reads the data from the DB (selects) into in-memory tables.
Applies some rules on them, and makes inserts into another in-memory
tables and finally inserts and updates the SQL Server DB tables. The
part that takes long when multiple users kick off their own processes
is the actual calculation where there maybe in-memory tables from
multiple processes.

3. The reason I say that there is a performance issue is that the time
to do the actual calculation increases when there are multiple
processes running simultaeneously.

4. No we have not yet tries cacheing. We were concentrating on
optimizing the sql queries till now. Will look into that now.

5. Yes, I am looking into that as well. Are there some that you can
suggest? I have ANTS so far that I think will do the job and is
reasonably priced.

Thanks much for you help.

rgds

KS

Hello (e-mail address removed),

I manage a web based VB .net application. This application has 3
components:
1. Webapp (this calls the executibles)
2. database
3. business logic is contained in individual exe application that get
called in a sequence to do some heavy calculations (mainly DB
operations with in memory datasets)

What does "business logic is contained in individual exe" mean?
Who starts them, where are they hosted?

I am having an issue with performance with this architecture.

How do u get this?

the exes themselved are multi-threaded and can have multiple instances
(this means that there are these mutliple datasets on the app server).

Have u tried to use caching?

Are there any guidelines that can help me establish how I can improve
performance - increasing memory, adding an app server (clustering),
adding cpus, etc. Any advice from you will be helpful!

Firstly use some kind of performance tool to understand the bottleneck.

---
WBR,
Michael Nemtsev [C# MVP] :: blog: http://spaces.live.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

 

[Michael Nemtsev]

Hello (e-mail address removed),

s> 1. "business logic is contained in individual exe" means that the
s> webapp does not contain the actual calculations. These are kicked
s> off from the web app on a button click by the user - we call each run
s> a process.

So it's just the EXE process? hmmmm.... what's the reason of this and not
the some kind of appservices, like enterpriseServices or WebServices?

s> 2. The business logic itself reads the data from files... validates
s> them pumps them into the DB (SQL Server) - this happens pretty
s> quickly.
s> Then reads the data from the DB (selects) into in-memory tables.
s> Applies some rules on them, and makes inserts into another in-memory
s> tables and finally inserts and updates the SQL Server DB tables.

Which DB is used? I'm asking this because in case of SQLServer 2005 you can
save your resources performing reading data and applying rules completly
on the DB size with using SQL .NET components

BTW, have u considered any possible cased to load data from files directly
to db and process all work there?

s> The
s> part that takes long when multiple users kick off their own processes
s> is the actual calculation where there maybe in-memory tables from
s> multiple processes.

Yep, so consider sharing for the data, if they are the same for the all users
(MS Cache Application Blocks as the case)
BTW, I'd look at the object pooling too (for example in EnterpriseServices
it's can be turned on/off too easy)

s> 4. No we have not yet tries cacheing. We were concentrating on
s> optimizing the sql queries till now. Will look into that now.

Right, the more calculation next to DB the better performance

s> 5. Yes, I am looking into that as well. Are there some that you can
s> suggest? I have ANTS so far that I think will do the job and is
s> reasonably priced.

CLRProfiler, Compuware DevPartnet, JetBrains .Trace


s> Michael Nemtsev wrote:
s>

Hello (e-mail address removed),

I manage a web based VB .net application. This application has 3
components:
1. Webapp (this calls the executibles)
2. database
3. business logic is contained in individual exe application that
get
called in a sequence to do some heavy calculations (mainly DB
operations with in memory datasets)

What does "business logic is contained in individual exe" mean? Who
starts them, where are they hosted?

I am having an issue with performance with this architecture.

How do u get this?

the exes themselved are multi-threaded and can have multiple
instances (this means that there are these mutliple datasets on the
app server).

Have u tried to use caching?

Are there any guidelines that can help me establish how I can
improve performance - increasing memory, adding an app server
(clustering), adding cpus, etc. Any advice from you will be
helpful!

Firstly use some kind of performance tool to understand the
bottleneck.

---
WBR,
Michael Nemtsev [C# MVP] :: blog: http://spaces.live.com/laflour
"At times one remains faithful to a cause only because its opponents
do not cease to be insipid." (c) Friedrich Nietzsche

---
WBR,
Michael Nemtsev [C# MVP] :: blog: http://spaces.live.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche

 

[Brennan Stehling]

rgds,

A simple way to test the performance of your application is to log the
time before and after blocks of code in your application. Once you can
identify the most time-consuming piece you can focus more on it and
eventually identify the best place to apply some optimization
techniques.

Until you know the which specific pieces are the cause for the delays
it will be hard to know what techniques to apply to resolve your
issues.

In general, even though the SQL and programming logic is not in a
code-behind, it sounds like you still have the code running either in a
class library or a console application which is not on the same machine
as the database server. If you are transferring the data from the
server to a remote client then you will find the time to transfer all
of that data is significant. A simple way to cut down on that is to
make sure you are not selecting all rows and instead trim it down to
just the columns you need.

Next you have to consider the costs of pulling in that data and parsing
it into a form you can use. The cost of casting a Integer from the
Object that the DataSet provides can add up. If at all possible, place
more code into your database as stored procedures. You can do quite a
bit with T/SQL, such as working with table variables and while loops to
look at the data to implement your business logic. Normally you do not
want a lot of business logic in your stored procedures, but when
performance is more important as it seems here, it can be the right
option.

I just moved a good deal of logic into a T/SQL script this week and it
went from running for 1 hour to 3 minutes. And that was likely due to
eliminating the data transfer and type casting costs.

Lastly, review each of your select statements and table indexes. Make
sure when you are joining tables you are doing so with indexed values.
And if you do join across many tables, consider a data warehousing
approach where you pre-fill a denormalized table with the contents of
the expensive query so that when you use it later it comes back very
quickly.


Brennan Stehling
http://brennan.offwhite.net/blog/

Mike

Thanks for you resonse.

1. "business logic is contained in individual exe" means that the
webapp does not contain the actual calculations. These are kicked off
from the web app on a button click by the user - we call each run a
process.

2. The business logic itself reads the data from files... validates
them pumps them into the DB (SQL Server) - this happens pretty quickly.
Then reads the data from the DB (selects) into in-memory tables.
Applies some rules on them, and makes inserts into another in-memory
tables and finally inserts and updates the SQL Server DB tables. The
part that takes long when multiple users kick off their own processes
is the actual calculation where there maybe in-memory tables from
multiple processes.

3. The reason I say that there is a performance issue is that the time
to do the actual calculation increases when there are multiple
processes running simultaeneously.

4. No we have not yet tries cacheing. We were concentrating on
optimizing the sql queries till now. Will look into that now.

5. Yes, I am looking into that as well. Are there some that you can
suggest? I have ANTS so far that I think will do the job and is
reasonably priced.

Thanks much for you help.

rgds

KS

Hello (e-mail address removed),

I manage a web based VB .net application. This application has 3
components:
1. Webapp (this calls the executibles)
2. database
3. business logic is contained in individual exe application that get
called in a sequence to do some heavy calculations (mainly DB
operations with in memory datasets)

What does "business logic is contained in individual exe" mean?
Who starts them, where are they hosted?

I am having an issue with performance with this architecture.

How do u get this?

the exes themselved are multi-threaded and can have multiple instances
(this means that there are these mutliple datasets on the app server).

Have u tried to use caching?

Are there any guidelines that can help me establish how I can improve
performance - increasing memory, adding an app server (clustering),
adding cpus, etc. Any advice from you will be helpful!

Firstly use some kind of performance tool to understand the bottleneck.

---
WBR,
Michael Nemtsev [C# MVP] :: blog: http://spaces.live.com/laflour

"At times one remains faithful to a cause only because its opponents do not
cease to be insipid." (c) Friedrich Nietzsche