Reply to thread

Message: [QUOTE="Steven D'Aprano, post: 4157776"] What's "fairly large"? Large to you is probably not large to your computer. Anything less than a few dozen megabytes is small enough to be read entirely into memory. If all you are doing is replacing single characters, then there's no need for the 80lb sledgehammer of regular expressions when all you need is a delicate tack hammer. Instead of this: * read the file as bytes * search for pairs of bytes like chr(195)+chr(130) using a regex * replace them with single bytes like 'A' do this: * read the file as a Unicode * search for characters like Ã‚ * replace them with single characters like A using unicode.translate() (or str.translate() in Python 3.x) The only gotcha is that you need to know (or guess) the encoding to read the file correctly. [/QUOTE]