StreamReader doesn't pick up special characters

[Max]

I'm using StreamReader.ReadToEnd() to populate and string from a file and
then display it as a literal on my web site. The problem is I'm losing all
the special characters like Æ,Ø, and Å that originates in the file I'm
reading. Do I have to tell streamreader what characterset I'm using or
something?

-Max

 

[David Browne]

I'm using StreamReader.ReadToEnd() to populate and string from a file and
then display it as a literal on my web site. The problem is I'm losing all
the special characters like Æ,Ø, and Å that originates in the file I'm
reading. Do I have to tell streamreader what characterset I'm using or
something?

Yes.

http://msdn.microsoft.com/library/d.../frlrfsystemiostreamreaderclassctortopic4.asp

David

 

[Eliahu]

when using Unicode all those problems should disappear. what coding do you
use for your file? you should use the same one when you read your file.

 

[Max]

This is what I came up with, adding the latin code now displays chars in
Norway, and the English versions seem to be unaffected. I'm reading them in
using FileStream now, instead of StreamReader.

If anyone has anyway to improve performance, please correct me -- I need the
best performance out of reading these text files, since they are read in and
displayed to the page per each page load.

Dim strFileContents As String
Dim enc As System.Text.Encoding
enc = System.Text.Encoding.GetEncoding(1252)
Dim fs As System.IO.FileStream = New System.IO.FileStream(strFilePath,
FileMode.Open, FileAccess.Read, FileShare.Read)

Dim buffer(4096) As Byte
While fs.Read(buffer, 0, 4096)
strFileContents = strFileContents & enc.GetString(buffer)
End While

-Max

Think I need an fs.close in there too.

 

[David Browne]

This is what I came up with, adding the latin code now displays chars in
Norway, and the English versions seem to be unaffected. I'm reading them
in using FileStream now, instead of StreamReader.

If anyone has anyway to improve performance, please correct me -- I need
the best performance out of reading these text files, since they are read
in and displayed to the page per each page load.

Dim strFileContents As String
Dim enc As System.Text.Encoding
enc = System.Text.Encoding.GetEncoding(1252)
Dim fs As System.IO.FileStream = New System.IO.FileStream(strFilePath,
FileMode.Open, FileAccess.Read, FileShare.Read)

Dim buffer(4096) As Byte
While fs.Read(buffer, 0, 4096)
strFileContents = strFileContents & enc.GetString(buffer)
End While

Yuck,

All that string concatenation will kill you.

Try this instead, it's short and faster.

Dim strFilePath As String
Dim sr As New StreamReader(strFilePath,
System.Text.Encoding.GetEncoding(1252))
Dim strFileContents As String = sr.ReadToEnd
sr.Close()

For large files you can improve performance a little more if you know how
many bytes per char your encoding has (1252 Latin 1 is a single-byte
encoding), and how large the file is.

Dim fs As System.IO.FileStream = New System.IO.FileStream(strFilePath,
FileMode.Open, FileAccess.Read, FileShare.Read)
Dim bytes As Integer = CInt(fs.Length)
Dim sr As New StreamReader(fs, System.Text.Encoding.GetEncoding(1252),
False, 4096)
Dim sb As New Text.StringBuilder(bytes)
Dim charBuf(4096) As Char
Dim charsRead As Integer = sr.ReadBlock(charBuf, 0, 4096)
Do While charsRead > 0
sb.Append(charBuf, 0, charsRead)
charsRead = sr.ReadBlock(charBuf, 0, 4096)
Loop
sr.Close()
Dim strFileContents As String = sb.ToString


The StringBuilder allows you to pre-allocate the char array for the string.
Then StringBuilder.ToString returns a string which shares that char buffer.

David