Really slow, painfullu slow asp.net web app

Discussion in 'ASP .Net' started by michael@randallk.com, Feb 14, 2006.

  1. Guest

    Hi,

    I am trying to write an ASP.NET web app, in Visual Basic.NET, using
    Visual Studio.NET 2004, .NET framework 1.1.4322 SP1
    Running the project/app on localhost while in dev/write/debug stage

    When I say "trying", I do have it written, and it works ... sort of,
    for some cases.
    The problems/issues?

    1. it runs very slowly
    2. it seems to either hang up/lock up aspnet_wp.exe
    3. or timesouts way to soon
    4. if/when it hangs/locks aspnet_wp.exe, "End Process" in Task Manager
    is only way to get aspnet_wp.exe to stop (and it restarts) thus
    bringing VS.NET back into "write mode" from debug mode (closing web
    page or hitting stop debugging in VS.NET does not actually detach/stop
    aspnet_wp.exe)

    In psuedo code form my app takes several images (ideally)
    for each image
    go through each pixel and gets its color
    if the color for the pixel is already on a dataset.table then
    increment the frequency count for that color (ARGB)
    if not on the table then add that color (ARGB) to the
    dataset.table
    next pixel
    when done with that image then store the values/rows (colors &
    frequencies) in the SQL Server 2000 database

    next image

    FIRST - let me stress it is not pulling th data from SQL Server - it is
    finding/crunching the data in the image and then when done processing
    the image/pixels, then at that point it rights the results to database
    SECOND - it is not the writing to the db that is slow - have, in
    attempt to pin issues down commented out actual writing to db

    If I run my app on an image of size 100x100 pixels it runs pretty fast
    Can even do a batch with 3 or 4 smaller images

    But if I run it on an image of say 600 x 600 pixels ... it just bogs
    down.

    I can understand a larger image would have more colors, and as it loops
    through the pixels, it would take longer to sort the table for that
    color (to see if it needs to add or increment)

    But it just comes to a grinding halt.
    As it never actually finishes a larger image don't lnow exactly how
    many colors it is finding, and therefore how many rows the tabel has,
    but near as I can tell, on one image there is about 70,000 colors,
    meaning the table would have about 70,000 lines.

    Is that too big? As in too many lines for ASP.NET to handle?

    And if I either:
    1. just check aspnet_wp.exe while running, or
    2. close the web page, or
    3. stop debugging in VS.net

    Mem usage for aspnet_wp.exe is psikingat like 98% of memory.
    Stopping debugging does not stop aspnet_wp.exe from spiking ... even if
    I give the computer/VS>NET an hour or more to actually stop debugging.

    I have to go into Task Manager and end process on aspnet_wp.exe then
    vs.net actually stops debugging.

    I did write sort of a prototype of this as a windows app and it goes
    very fast, even with very large files & a large number of files (and my
    code for this protoype is way less efficient)

    I would expect some slowness moving to a web app, but not for it to
    just simply hang (and if I let it run long enough ... it just times out
    .... with aspnet_wp.exe spiking for literally days ... e.g. I let the
    app run over the weekend, came back today ... and timeout, no data and
    aspnet_wp.exe still consuming 98% of mem.)

    Any help appreciated
    Mike
     
    , Feb 14, 2006
    #1
    1. Advertising

  2. Let me guess: You're using the Bitmap.GetPixel method, right? If you want to
    process an image pixel by pixel, you want to use unsafe code and pointer
    arithmetic, which you won't be able to do using VB.Net. The GetPixel method
    is safe, and slow. Also, depending upon how you're using your Dataset, you
    may be running into some performance issues there as well. Regardless of
    whether you're writing to the database all at once, you may be running into
    some type-checking issues, again, depending upon how you're using the
    DataSet. But I'd be willing to guess that 90% of your problem is with your
    image processing.

    --
    HTH,

    Kevin Spencer
    Microsoft MVP
    ..Net Developer
    We got a sick zebra a hat,
    you ultimate tuna.


    <> wrote in message
    news:...
    > Hi,
    >
    > I am trying to write an ASP.NET web app, in Visual Basic.NET, using
    > Visual Studio.NET 2004, .NET framework 1.1.4322 SP1
    > Running the project/app on localhost while in dev/write/debug stage
    >
    > When I say "trying", I do have it written, and it works ... sort of,
    > for some cases.
    > The problems/issues?
    >
    > 1. it runs very slowly
    > 2. it seems to either hang up/lock up aspnet_wp.exe
    > 3. or timesouts way to soon
    > 4. if/when it hangs/locks aspnet_wp.exe, "End Process" in Task Manager
    > is only way to get aspnet_wp.exe to stop (and it restarts) thus
    > bringing VS.NET back into "write mode" from debug mode (closing web
    > page or hitting stop debugging in VS.NET does not actually detach/stop
    > aspnet_wp.exe)
    >
    > In psuedo code form my app takes several images (ideally)
    > for each image
    > go through each pixel and gets its color
    > if the color for the pixel is already on a dataset.table then
    > increment the frequency count for that color (ARGB)
    > if not on the table then add that color (ARGB) to the
    > dataset.table
    > next pixel
    > when done with that image then store the values/rows (colors &
    > frequencies) in the SQL Server 2000 database
    >
    > next image
    >
    > FIRST - let me stress it is not pulling th data from SQL Server - it is
    > finding/crunching the data in the image and then when done processing
    > the image/pixels, then at that point it rights the results to database
    > SECOND - it is not the writing to the db that is slow - have, in
    > attempt to pin issues down commented out actual writing to db
    >
    > If I run my app on an image of size 100x100 pixels it runs pretty fast
    > Can even do a batch with 3 or 4 smaller images
    >
    > But if I run it on an image of say 600 x 600 pixels ... it just bogs
    > down.
    >
    > I can understand a larger image would have more colors, and as it loops
    > through the pixels, it would take longer to sort the table for that
    > color (to see if it needs to add or increment)
    >
    > But it just comes to a grinding halt.
    > As it never actually finishes a larger image don't lnow exactly how
    > many colors it is finding, and therefore how many rows the tabel has,
    > but near as I can tell, on one image there is about 70,000 colors,
    > meaning the table would have about 70,000 lines.
    >
    > Is that too big? As in too many lines for ASP.NET to handle?
    >
    > And if I either:
    > 1. just check aspnet_wp.exe while running, or
    > 2. close the web page, or
    > 3. stop debugging in VS.net
    >
    > Mem usage for aspnet_wp.exe is psikingat like 98% of memory.
    > Stopping debugging does not stop aspnet_wp.exe from spiking ... even if
    > I give the computer/VS>NET an hour or more to actually stop debugging.
    >
    > I have to go into Task Manager and end process on aspnet_wp.exe then
    > vs.net actually stops debugging.
    >
    > I did write sort of a prototype of this as a windows app and it goes
    > very fast, even with very large files & a large number of files (and my
    > code for this protoype is way less efficient)
    >
    > I would expect some slowness moving to a web app, but not for it to
    > just simply hang (and if I let it run long enough ... it just times out
    > ... with aspnet_wp.exe spiking for literally days ... e.g. I let the
    > app run over the weekend, came back today ... and timeout, no data and
    > aspnet_wp.exe still consuming 98% of mem.)
    >
    > Any help appreciated
    > Mike
    >
     
    Kevin Spencer, Feb 14, 2006
    #2
    1. Advertising

  3. Mike Guest

    Hi Kevin,

    Re:
    "Let me guess: You're using the Bitmap.GetPixel method, right?"

    Yes, using bitmap.getpixel.

    "But I'd be willing to guess that 90% of your problem is with your
    image processing. "

    Ordinarily, a good safe bet. ;-)
    But not in this case.

    I noticed, even in winapp version that bitmap.getpixel is slow, so I
    inserted a function that reads the bitmap color info into an array and
    then work with the array.
    Reading the bitmpa into the array is lightening fast, and then doing
    the crunching on the pixel data in the array is not where I seem to be
    taking the hit.

    "depending upon how you're using the DataSet."

    I am pretty sure this is where the issue lies ... but can't seem to pin
    it down as to what about how I am using the ds is doing it.
    Right now, for the part causing the slow down/hang up it is a typed
    dataset.
    I actually instatiate 2 instances of the table:
    1. holds the intermediate, per pixel data, a temptable if you will
    2. then I check to see if that intermediate color/info is already on
    the complete/permanent table, if that color is there increment
    frequency, if not there then add it.

    How?
    I have tried every way I can think of, using select on table,
    dataviews, datarowviews, defaultview, filtering, looping through, etc.
    etc. etc. nothing seems to make any difference (which leads me back to
    thinking there is either a bug in aspnet_wp.exe or I have a bad install
    of it ... given it hangs/gets stuck).

    Here is the code, as I have it now (this is about the 30th version) of
    the section/procedure that seems to be causing the traffic jam:

    Private Shared Sub AddPixelColorDataToTable(ByRef DtlRowView As
    System.Data.DataRowView _
    , ByRef RawDV As DataView _
    , ByRef dsMCT As WhatIsBlue.MaterialColorTemp _
    , ByRef arrayMatColorTempRows() As
    WhatIsBlue.MaterialColorTemp.MaterialColorTempRow _
    , ByRef DtlRowCount As Integer _
    , ByRef MCTDV As System.Data.DataView)

    Dim StartLoop As Integer = 1
    If (DtlRowCount = 1) Then
    StartLoop = 0
    End If

    Dim foundrow As Integer = -1
    Dim useMCTIndex As Int64

    For i As Integer = StartLoop To UBound(arrayMatColorTempRows)
    If (i = 0) Then
    useMCTIndex = RawDV(0)("MCTIndex")
    Else
    useMCTIndex = DtlRowView("MCTIndex")
    End If

    MCTDV.RowFilter = "MCTIndex = '" & useMCTIndex.ToString &
    "'"


    foundrow = MCTDV.Find(New Object() {useMCTIndex _
    , arrayMatColorTempRows(i).Alpha,
    arrayMatColorTempRows(i).Red _
    , arrayMatColorTempRows(i).Green,
    arrayMatColorTempRows(i).Blue})

    'CurrRow = dsMCT.MaterialColorTemp.Rows.Find(New Object()
    {useMCTIndex _
    ' , arrayMatColorTempRows(i).Alpha,
    arrayMatColorTempRows(i).Red _
    ' , arrayMatColorTempRows(i).Green,
    arrayMatColorTempRows(i).Blue})

    'If (CurrRow Is Nothing) Then
    If (foundrow = -1) Then
    Dim CurrRow As
    WhatIsBlue.MaterialColorTemp.MaterialColorTempRow

    CurrRow = dsMCT.MaterialColorTemp.NewRow

    CurrRow.MCTIndex = useMCTIndex
    CurrRow.Alpha = arrayMatColorTempRows(i).Alpha
    CurrRow.Red = arrayMatColorTempRows(i).Red
    CurrRow.Green = arrayMatColorTempRows(i).Green
    CurrRow.Blue = arrayMatColorTempRows(i).Blue
    CurrRow.WeightedPct =
    arrayMatColorTempRows(i).WeightedPct
    CurrRow.RawPct = arrayMatColorTempRows(i).RawPct
    CurrRow.WeightedFrequency =
    arrayMatColorTempRows(i).WeightedFrequency
    CurrRow.RawFrequency =
    arrayMatColorTempRows(i).RawFrequency
    CurrRow.Accent = arrayMatColorTempRows(i).Accent
    CurrRow.Accept = arrayMatColorTempRows(i).Accept

    dsMCT.MaterialColorTemp.Rows.Add(CurrRow)

    Else
    MCTDV(foundrow)("RawFrequency") =
    MCTDV(foundrow)("RawFrequency") + 1
    MCTDV(foundrow)("WeightedFrequency") =
    MCTDV(foundrow)("WeightedFrequency") +
    arrayMatColorTempRows(i).WeightedFrequency

    'CurrRow("RawFrequency") = CurrRow("RawFrequency") + 1
    'CurrRow("WeightedFrequency") =
    CurrRow("WeightedFrequency") +
    arrayMatColorTempRows(i).WeightedFrequency
    End If
    Next

    End Sub

    Yes a bit odd ... as I said, sort of trying this & that.
    BTW: it actually gets, and here tries to store 2 values for each pixel,
    one is the raw/as is color data for the pixel (from the array) and the
    other is the calculated color (for our needs, sort of have to do an
    averaging of color values in the area around a pixel)

    Yes, eliminating the raw data does indeedy speed things up, a bit, but
    on the larger images, it still comes to a grinding halt, hangs, need to
    "end process" on aspnet_wp.exe

    Really, puzzled on this ... just comes to a grinding halt ... I'd even
    be happy if it took a couple of hours to do an image ... at least it
    does the image (even if slow) but it just comes to a grinding halt ...
    left it run over night a couple of times now, on 1 image and each
    morning get the generic "The page cannot be displayed" message in web
    browser

    When I say generic page ... this is after it has been processing for a
    while and the page suggest things like hitting refresh, trying again
    later, check my internet settings, yada yada.

    It just simply comes to a grinding halt ... slow would be better than a
    grinding halt.

    On the smaller images z(100x100) it does them in about 2 minutes ...
    larger ... comes to a halt.

    Thanks for the help.

    Mike
     
    Mike, Feb 14, 2006
    #3
  4. Mike Guest

    Hi,

    Fixed it.

    As one might see from above code, I was using DataView's &
    DataRowView's to filter the tables (which gets fairly large as it
    processes an image).

    Don't use DataView's or DRV's (even though documentation stresses them
    heavily).
    Not saying, never use them, but if you are adding data, large amounts
    of it, to tables via/thru DVs, it will bog down.

    I moved code to a windows app, just to see if it was ASP.NET related
    issue.
    But windows app, alos slowed way down (oh, btw, in prototype windows I
    was not using tables but rather storing data in arrays, hashtables,
    collections, etc. which is why prototype was fast).

    End up, in trying one thing after another, creating several (variable
    number depending upon how an image is to be processed) seperate tables,
    then using table.select methods to find/sort/add and it zips right
    along.
    Then moved it back to a web app.

    With DV method ran it twice overnight ... always bombed out, yet
    aspnet_wp.exe still churning away in the background ... 16 hours or so
    .... nver finishing, web page ceasing to display, aspnet_wp.exe churning
    away ....

    By working directly with tables (an array of them) it can process the
    image 3 times, essentially, in 3.75 minutes, with no lock up, boom
    done.

    HTH someone.

    Mike
     
    Mike, Feb 15, 2006
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Alex

    asp.net web app slow

    Alex, Oct 15, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    362
    Oliver
    Oct 15, 2003
  2. Rod
    Replies:
    2
    Views:
    1,376
  3. dev648237923
    Replies:
    3
    Views:
    535
    Steven Cheng[MSFT]
    Feb 9, 2007
  4. Headless Rider

    Control asp.net app from other asp.net app

    Headless Rider, Aug 18, 2003, in forum: ASP .Net Security
    Replies:
    0
    Views:
    118
    Headless Rider
    Aug 18, 2003
  5. Mat Schaffer

    String#chop slow? REALLY slow?

    Mat Schaffer, Jul 27, 2006, in forum: Ruby
    Replies:
    11
    Views:
    338
    Caio Chassot
    Jul 27, 2006
Loading...

Share This Page