Response Filter losing data when response is buffered

[Brad]

I have a response filter which injects "standard" html into my pages. The
filter works fine when the initial stream is small enough not to
buffer...or....if I have a large unbuffered stream (i.e. I set buffer=false
on a large page). Now the problem: If I turn on buffering on a large
page, the page output (to the browser) is correct a few times (sometines
just once, sometime 2-3 times...on the same page) then I seem to either
lose data or get corrputed data (I'm not sure which) in the middle of the
stream. The problem seems to persist until the page in my local web servers
cache (?) is cleared out i.e. I browser to some other pages then browse
back to the problem page.

I've compared the size of the buffer at the start and end of my filter's
write sub on the calls when it works when it does not and the buffers are
the same size which make it look like I'm not droping characters. The large
page has embedded encrypted client script from a third party control and
I've wondered if that script could be getting corrupted from the
UTF8Encoding.UTF8.GetString and/or the later UTF8Encoding.UTF8.GetBytes in
the write sub.

Anyone have thoughts or suggestions as to what to look for?

Thanks

Brad

 

[Steven Cheng[MSFT]]

Hi Brad,



Thanks for posting in the community!
From your description, you used a certain Custom Response Filter to add
some certain html code into the response stream. However, you found the
page which has applied the Filter will output incompleted or corrupted
response data when the page's response content is large enough to buffer
and the Response's Buffer is set as true, yes?
If there is anything I misunderstood, please feel free to let me know.

I'm not sure the detailed operations in your filter, can you repro this
problem via certain code logic and steps? Or just a certain simple sample
Filter class to repro this problem?
Have you tried applied another certain filter class on the page's response
to see whether this problem remains?
If so , I think I can do some further tests on my side on this issue.
Please check the above items and let me know if you have any findings.


Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

Get Preview at ASP.NET whidbey
http://msdn.microsoft.com/asp.net/whidbey/default.aspx

 

[Brad]

Steven -
I've resolved the problem. Here's what I found and how I resolved it:

First, I simplified the filter's Write sub to the following and still had
the problem.
=============================
Friend Class ResponseFilter
Inherits Stream
Private responseStream As Stream

Public Overrides Sub Write(ByVal buffer() As Byte, ByVal offset As
Integer, ByVal count As Integer)
Dim sBuffer As String =
System.Text.UTF8Encoding.UTF8.GetString(buffer, offset, count)

' Here is where code could be placed to manipulate content of
sBuffer before writing to stream

buffer = System.Text.UTF8Encoding.UTF8.GetBytes(sBuffer)
responseStream.Write(buffer, offset, buffer.Length)
End Sub

=============================
I resolved the problem, by appending the writes buffer content into a
stringbuilder. When I detect </html> in the buffer I know I've received the
last buffer (last call to write). I make my changes to the contents of the
stringbuilder and write the entire content to the stream.

Friend Class ResponseFilter
Inherits Stream
Private responseStream As Stream
Private html As StringBuilder = New StringBuilder
...
Public Overrides Sub Write(ByVal buffer() As Byte, ByVal offset As
Integer, ByVal count As Integer)
Dim sBuffer As String =
System.Text.UTF8Encoding.UTF8.GetString(buffer, offset, count)
html.Append(sBuffer)

Dim oEndFile As Regex = New Regex("</html>",
RegexOptions.IgnoreCase)

If (oEndFile.IsMatch(sBuffer)) Then
Dim tempResponse As String = html.ToString()
buffer = System.Text.UTF8Encoding.UTF8.GetBytes(tempResponse)
responseStream.Write(buffer, 0, buffer.Length)
End If
End Sub
=============================

If you see anything bad still or have suggestions please let me know.

Thanks

Brad

 

[Steven Cheng[MSFT]]

Hi Brad,


Thanks for your followup. I'm glad that you've figured out this problem.
The StringBuilder will provide high performance when doing string
manipulation. Here is the description in MSDN:
----------------------------------------------------------
The String object is immutable. Every time you use one of the methods in
the System.String class, you create a new string object in memory, which
requires a new allocation of space for that new object. In situations where
you need to perform repeated modifications to a string, the overhead
associated with creating a new String object can be costly. The
System.Text.StringBuilder class can be used when you want to modify a
string without creating a new object. For example, using the StringBuilder
class can boost performance when concatenating many strings together in a
loop.
----------------------------------------------------------
Also, here are some tech articles discussing on this:
#HOW TO: Improve String Concatenation Performance in Visual Basic .NET
http://support.microsoft.com/?id=306821

#Efficient String Manipulations with StringBuilder
http://www.ondotnet.com/pub/a/dotnet/2002/07/15/string.html


Maybe the large page with buffer turn on cause string to realloc again and
again cause the unexpected problem. But I'm still so surprised at this
result. Any way, thanks again for providing us your experience.


Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

Get Preview at ASP.NET whidbey
http://msdn.microsoft.com/asp.net/whidbey/default.aspx