Split a very large log file

[Terry.Riegel]

I have a 9gig log file, I am trying to do something with it. I tried
to run it through awstats and after about 8 hours I got a segmentation
fault.

I think if I could break it up into 2 files I could process one , then
process the other. I am not very good at PERL, but have to believe
this would be a 5-10 line program.

Any ideas?

Thanks,

Terry Riegel

 

[J. Gleixner]

I have a 9gig log file, I am trying to do something with it. I tried
to run it through awstats and after about 8 hours I got a segmentation
fault.

I think if I could break it up into 2 files I could process one , then
process the other. I am not very good at PERL, but have to believe
this would be a 5-10 line program.

Yep.. why not post what you've tried.

Also, if you're on *nix? There are a bunch of
commands that will do it:

man split
man head
man tail

 

[xhoster]

I have a 9gig log file, I am trying to do something with it. I tried
to run it through awstats and after about 8 hours I got a segmentation
fault.

I think if I could break it up into 2 files I could process one , then
process the other.

I would guess that doing this will lead to the wrong answers. If awstats
is at all well written, then it wouldn't be loading all the data into
memory (which I'm guessing is what precipitated the seg fault) unless that
is what it needed to do to get the right answer.

I am not very good at PERL, but have to believe
this would be a 5-10 line program.

I'd use GNU "split" rather than using Perl at all, but anyway you just
need to look in $. to decide if you should reopen to a new output
file.

Xho