Memory usage and large number of pages

[Guest]

I have migrated my asp application to asp.net some time ago - but I am still
having some difficulties in understanding the best way to mange some tasks.

I currently have a page that loads a aspx web page - this page is
continually refreshed - every 5 seconds or so. To do this I use the download
behavior on the client to call a particular page - say newchart.aspx. The
newchart.aspx than calls a custom component (vb app on another machine using
dcom) and this app goes and creates a chartXXX.aspx file in the temp
directory on the web server. newchart.aspx than calls this chart123.aspx and
returns the result of this page back to the client. chartXXX.aspx is unique
for every call.

On each call the memory on the server increases - until the point it resets.
I used this method in asp extensively and never had these problems - but the
changes in asp.net means that this is not be best way to do this.

Can anyone suggest to me a better way of managing something like this??

My custom app on the remote computer has the smarts of what needs to be
returned to the client and sometimes it simply writes a html page to the temp
directory and this gets returned to the client - this causes no problem - the
difficulty happens when a aspx file is written.

 

[Karl Seguin]

Matthew:
Your process seems complicated. I don't know enough to say that there's a
better way to do everything, I merely state that to indicate how hard this
is going to be to fix things. ASP.Net will automatically recycle itself
after it just-in-time compiles X amount of pages. From what you describe,
it seems like every 5 seconds for every user a new page is created, forcing
ASP.Net to have to recompile it. I would suspect that you'd quickly run
into problems this way (however, I doubt that's, in any way, associated with
your memory problem). Just a comment of something you should be aware of.

You might want to build your entire reload/dynamic creation framework on a
much lighterweight protocol. Again, I don't fully understand what you are
doing, but it would seem AJAX might be able to help you here. This would
let you asynchronously send whatever data back to your original page
without having to reload or recreate new pages. You can learn more about
this from: http://ajax.schwarz-interactive.de/csharpsample/default.aspx

Karl

 

[Guest]

Karl,

Thanks for the response - it was helpfull.

The AJAX system looks good and I will consider using this in the future - I
have already provided similiar functionality to this by using a webservice -
this works well and I am currently looking at this as an solution to this
specific problem.

The true problem I am dealing with is actually a little more critical then
this issue - as I have been trying to discuss on another thread for some-time
- The problem I mainly have is dealing with very large numbers of files.

The main issue I have is that I have over 10,000 files - most of which
represent invoices to clients - I have to keep the original copies of each
invoice and it is not practical to re-create the invoice each time it is
viewed by the client. Currently what I do is save these files else where and
when they are requested I copy them to a temp directory on the server and
redirect the customer to the page using response.redirect.

The problem is the overhead of compiling these pages is huge - and because
there are so many there is no way to include them into the application. I
have also found that for each file that is compiled the memory increases.

Many of the files also needed to be ASPX as they have server-side logic
within them - this is used to provide a interactive charting in each invoice.

Most people have simply said that it is a bad design and I should start
again - but I can't see any other way of doing this.

What I wanted to know - is it possible to pre-compile a page somehow and
maybee save a copy of the pre-compiled class file. When the page is
requested I could copy accross a copy of this file along with the original -
when the system goes to access the file it would somehow recongize that it
has already been compiled and simply get on with processing the page. This
would mean I would need to save twice as many files - but I don't see this as
a major issue.

This solution would fix my speed issues - but does not provide an immediate
solution to the memory issue. but one thing at a time - is what I have
suggested possible????

And to extend this further - once a class has been defined - is it possible
to remove this definition - or does this definition remain available until
the process is next recycled?

 

[Hans Kesting]

Karl,

Thanks for the response - it was helpfull.

The AJAX system looks good and I will consider using this in the
future - I have already provided similiar functionality to this by
using a webservice - this works well and I am currently looking at
this as an solution to this specific problem.

The true problem I am dealing with is actually a little more critical
then this issue - as I have been trying to discuss on another thread
for some-time - The problem I mainly have is dealing with very large
numbers of files.

The main issue I have is that I have over 10,000 files - most of which
represent invoices to clients - I have to keep the original copies of
each invoice and it is not practical to re-create the invoice each
time it is viewed by the client. Currently what I do is save these
files else where and when they are requested I copy them to a temp
directory on the server and redirect the customer to the page using
response.redirect.

The problem is the overhead of compiling these pages is huge - and
because there are so many there is no way to include them into the
application. I have also found that for each file that is compiled
the memory increases.

Many of the files also needed to be ASPX as they have server-side
logic within them - this is used to provide a interactive charting in
each invoice.

Most people have simply said that it is a bad design and I should
start again - but I can't see any other way of doing this.

What I wanted to know - is it possible to pre-compile a page somehow
and maybee save a copy of the pre-compiled class file. When the page
is requested I could copy accross a copy of this file along with the
original - when the system goes to access the file it would somehow
recongize that it has already been compiled and simply get on with
processing the page. This would mean I would need to save twice as
many files - but I don't see this as a major issue.

This solution would fix my speed issues - but does not provide an
immediate solution to the memory issue. but one thing at a time - is
what I have suggested possible????

And to extend this further - once a class has been defined - is it
possible to remove this definition - or does this definition remain
available until the process is next recycled?

One way you could investigate:
Don't store those invoices as aspx pages, complete with controls
for charting, but store only the data IN the invoices (as XML files,
maybe even in the database). Instead of redirecting to the (copied)
invoice-aspx file, use a standard aspx with a parameter to select the
invoice data. Then build you invoice, complete with charts, from
that data.

Hans Kesting

 

[Juan T. Llibre]

I'm not sure if what I'm about to suggest would meet your company's
apparently very strict rules but, instead of saving original code as
aspx pages in case you have to retrieve the same information again,
wouldn't it be a lot simpler to have a dynamic page ( aspx ) which
creates the invoices you need, and publish the resulting invoices to
PDF files, which you could then classify/retrieve at will ?

That would eliminate your excessive aspx files problem,
while still meeting the requirement that the original invoices be saved.

 

[Karl Seguin]

Both Hans and Juan make the same suggestion I would have. Store the data
(database, xml, ...) and simply have an invoice.aspx which takes an
?invoiceId=xxx and dynamically generates the invoice.. That's certainly
the more "traditional" way of doing thing.

Karl

 

[Karl Seguin]

You can look at:
http://www.dotnet247.com/247reference/msgs/18/92782.aspx

Ken Cox suggest that it's possible to do it via vbc or csc (vb compiler, c#
compiler) command line. I would agree with him that it's possible, but I
don't know how.

Karl

 

[Guest]

Hi,

The runtime compiling can work but it is more complex than necessary in a
ASP.Net app, maybe V2 will make it easier.

It sounds like you are not disposing of the DCOM call properly, so I would
focus on that.

Regards
Brucek

 

[Guest]

Thank you for you response.
I did notice that if I publish the page the changes do show up. Since I am
not really ready to publish this page to the world yet, I just published to a
local folder on my hard drive.

The funny thing is that with FP 2002, I NEVER published and didn't have to
in order to see the changes. And I did not have to open up every single file
that had the page include and save them.

Strange happenings are afoot, but I least they are figured out.

Thanks again.