Using logging module to log into flash drive

[kretel]

Hi All,

I am trying to implement the following functionality:
1. log messages to the flash drive
2. if the flash drive is not available, switch handler to the
BufferringHandler and log into buffer,
3. once the flash drive is plugged in and available store the logs
from BufferHandler into that flash drive and switch the handler into
RotateFileHandler.

Which approach would you suggest to use while implementing this
functionality? One that come into my mind is to have one process or
thread to check periodically if the flashdrive is available, and have
a flag that will indicate that we can store the logs into the flash
drive. But I don't particularly like this approach.
Would you do it different way?
Any suggestions are appreciated.

Cheers,
Krzysztof

 

[A. Cavallo]

Hi,
the problem screams for a separate thread.

Anyway there's a TimedRotatingFileHandler handler in the logging package:
you can derive from it and change the emit/doRollover pair to hold the records
until a device is not ready.


Regards,
Antonio

 

[Krzysztof Retel]

Hi,
the problem screams for a separate thread.

I was thinking about that, as mentioned in the first post. Although, I
was wonder if there is another way to tackle the problem.

Anyway there's a TimedRotatingFileHandler handler in the logging package:
you can derive from it and change the emit/doRollover pair to hold the records
until a device is not ready.

Hm, that might be the way to go. Will have a try.

If anyone has other thoughts, please share them.

Thanks,
Krzysztof

 

[Krzysztof Retel]

Anyway there's a TimedRotatingFileHandler handler in the logging package:

Hm, that might be the way to go. Will have a try.

I had another look at the logging package.
The class TimedRotatingFileHandler within the initialisation method
uses the FileHandler.__init__. Whereas the FileHandler.__init__ method
opens the stream in the 'append' mode (mode='a'). If the flash drive
is not available while you start the script, which equals that you
can't open a file in the append mode, then the
TimedRotatingFileHandler (or FileHandler) fails.
In my opinion the only way to go through this is to use two different
handlers and once the flash drive is available then start to use the
FileHandler.

Still not sure if I want to use separate thread for checking the flash
drive availability. Probably I need to build it this way.

Regards
Krzysztof

 

[Carl Banks]

Hi All,

I am trying to implement the following functionality:
1. log messages to the flash drive
2. if the flash drive is not available, switch handler to the
BufferringHandler and log into buffer,
3. once the flash drive is plugged in and available store the logs
from BufferHandler into that flash drive and switch the handler into
RotateFileHandler.

Which approach would you suggest to use while implementing this
functionality? One that come into my mind is to have one process or
thread to check periodically if the flashdrive is available, and have
a flag that will indicate that we can store the logs into the flash
drive. But I don't particularly like this approach.
Would you do it different way?
Any suggestions are appreciated.

I'd refactor the steps this way:

1. log messages to a buffer
2. periodically flush the buffer to the flash drive, if it's available

The "periodically" part could be accomplished with a thread or
scheduled delays, however suits your application. It might not be a
final if you need to log messages promptly, but that's how I'd begin.


Carl Banks

 

[Krzysztof Retel]

I'd refactor the steps this way:

1. log messages to a buffer
2. periodically flush the buffer to the flash drive, if it's available

The "periodically" part could be accomplished with a thread or
scheduled delays, however suits your application.  It might not be a
final if you need to log messages promptly, but that's how I'd begin.

Carl Banks

Hi Carl,

Thanks for the advice. I haven't think about it this way, but it looks
like that might be the starting point.

Thanks,
Krzysztof

 

[Jorgen Grahn]

[top-posting fixed]

First, to ignore the words "flash drive" and think in terms of "can I
open the file named so-and-so for writing?". Unless you absolutely
must avoid storing your file on a file system based on some other
storage technology.

Hi,
the problem screams for a separate thread.

I don't hear any screaming. It seems simpler to just

def log(something):
if logging_to_memore() and seconds_since_i_last_checked() > N:
try_to_switch_to_file()
do_the_actual_logging(something)

unless it's vital that as many of these logs as possible survive a
crash (in which case I guess the program would also refuse to exit
until the user finds the physical flash memory device somewhere and
mounts it correctly -- flashback to ancient floppy-based Macs).

Yes, I find the requirements quite odd.

/Jorgen