feature request / max size limit on StringIO

C

Clark C. Evans

Hello. I've not been able to use cStringIO since I have the need to
ensure that the memory buffers created are bounded within a resonable
limit set by specifications. No, this code does not properly belong
in my application as the modules that use "files" should not have
to care about any resource limitations that may be imposed.

class LimitedBuffer(StringIO):
def __init__(self, buffer = None, maxsize = 5 * 1000 * 1000):
StringIO.__init__(self,buffer)
self.cursize = 0
self.maxsize = maxsize
def write(self,str):
self.cursize += len(str)
if self.cursize > self.maxsize:
raise IOError("allocated buffer space exceeded")
return StringIO.write(self,str)

Kind Regards,

Clark Evans
 
B

Bengt Richter

Hello. I've not been able to use cStringIO since I have the need to
ensure that the memory buffers created are bounded within a resonable
limit set by specifications. No, this code does not properly belong
in my application as the modules that use "files" should not have
to care about any resource limitations that may be imposed.

class LimitedBuffer(StringIO):
def __init__(self, buffer = None, maxsize = 5 * 1000 * 1000):
StringIO.__init__(self,buffer)
self.cursize = 0
self.maxsize = maxsize
def write(self,str):
self.cursize += len(str)
if self.cursize > self.maxsize:
raise IOError("allocated buffer space exceeded")
return StringIO.write(self,str)

You might want to use StringIO's own knowledge of its writing position (self.pos or self.tell())
and then you wouldn't need cursize, nor to worry about whether seek has been used to
reposition, maybe writing the same section of file over and over for some reason.

I'm not sure whether seek beyond the end actually causes allocation, but I don't think so.
So you should be ok just checking self.pos+len(strarg)> self.maxsize.
There's also some interesting things going on with self.buf and self.buflist attributes, which
probably is not doing as simple a job of buffering as you might think. Maybe you'll want to wrap
a byte array or an mmap instance to store your info, depending on what you are doing?

Regards,
Bengt Richter
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,898
Latest member
BlairH7607

Latest Threads

Top