+972 52-548-6969
Accidental code review
time to read 1 min | 113 words
I’m trying to get a better insight on a set of log files sent by a customer. So I turned to find a tool that can do that, and I found Inidihiang. There is a x86 vs x64 issue that I had to go through, but then it was just sitting there trying to parse a 34MB log file.
I got annoyed enough that I actually checked, and this is the reason why:
Sigh…
I gave up on this and wrote my own stuff.
Comments
It's hard to believe there isn't a decent big log file viewer out there. Maybe it's time to respond to your challenge and write that damn tool and put it out there Apache 2.0 style.
Something along the lines of what the Scalyr guys did? http://blog.scalyr.com/2014/05/searching-20-gbsec-systems-engineering-before-algorithms/
I don't know about the specifics of your scenario, but what about the ELK stack?
Njy, All I wanted was to look for a bit at a single log file, nothing more.
Logparser might be worth having a look at http://blogs.technet.com/b/exchange/archive/2012/03/07/introducing-log-parser-studio.aspx. The article is on Exchange, it works for IIS too.
"for k=0" as a code smell.
...and the reassignment to listDump[j] is ________.
so yeah. everything in memory is variation on a theme.
There are decent parsers for logfiles but you need to start thinking about logfiles as files and start treating them as data; logstash / graylog2 / elasticsearch is fun and free.
It wasn't so much the "for k=0", it was the Cartesian product problem of j max was 131470 and k max was 102964.
So it reduced to for(int j = 0 ; j < 131470; j++) { for(int k = 0; k < 102964; k++) { } }
That's 13,536,677,080 iterations, plus all the work inside each iteration.
@Terry - indeed.
The "for k=0" merely raises the spidey sense that further investigation is warranted.
I'd have been more worried if it arbitrarily said, "for k = 93;".
I am 100% sure that, there should be an outer loop which starts as follows :)
for (int i=0;i<....
Comment preview