Your code is buggy

allStats = new int[4][];
...
for (var i = 0; i < stats.Length; i++)
 {
           var value = stats[i][0] + stats[i][1] + stats[i][2] + stats[i][3];

stats.Length is 4.

Also, when you fix this bug, you will have to deal with another problem: there is no warranty stats arrays be the same size, so the following code can fail with IndexOutOfRangeException:

var value = stats[0][i] + stats[1][i] + stats[2][i] + stats[3][i];

You forgot to mention one optimization you made in the final version.

The code now handles the case where the year and month are the same, but the days are different. There's no need to worry about leap year in this case, so the code just has to deal with different calculating the difference of days in seconds as well.

The bookkeeping code I added showed the following for the sample data: same day count = 5,454,657 same month, different day count = 333,066 other = 11,104 (calls the full, slow datetime parse code)

Jesus's bug report means that the summary.txt file is empty since there are no non-zero values in the first 4 slots.

Once you fix Jesus's bug, the file contents are wrong because of another bug.

As I commented in yesterday's article, there's a bug in the Parse() method.

the first line of Parse has the start and end positions swapped:

        public static void Parse(byte* buffer, out int id, out int duration)
        {
            duration = DiffTimesInSecond(buffer + 20, buffer);
            id = ParseInt8(buffer + 40);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static int DiffTimesInSecond(byte* start, byte* end)
        {

There's a third bug, this time in the WriteFormattedTime() method.

In the output, the timespan numbers are supposed to be separated by colons, but WriteFormattedTime() uses the wrong indices for writing the seconds value, thereby overwriting the second colon:

        private static void WriteFormattedTime(TimeSpan ts, byte[] temp, int pos)
        {
            var hours = ts.Hours;
            temp[pos] = (byte)(hours / 10 + '0');
            temp[pos + 1] = (byte)(hours % 10 + '0');

            var min = ts.Minutes;
            temp[pos + 3] = (byte)(min / 10 + '0');
            temp[pos + 4] = (byte)(min % 10 + '0');

            var sec = ts.Seconds;
            temp[pos + 5] = (byte)(sec / 10 + '0');  // <<< should be + 6
            temp[pos + 6] = (byte)(sec % 10 + '0');  // <<< should be + 7
        }
    }

After fixing the above-mentioned 3 bugs, app runtime is ~135ms.

You can reduce the runtime further by not creating a Timespan object for each value output in the WriteOutput() method. Instead you calculate the seconds, minutes and hours directly. Along with inlining the WriteFormattedTime() function, this reduces the runtime by about 10-15ms in the parallel version.

With the above change, the runtime is around 120-125ms on my machine.

N.B. I re-read today's entry and found where you mentioned the optimization for the same month - I was distracted by the pretty pix of source code. :)

I found a fourth bug in the code.

In the Main() function, the code that calculates the start and end ranges for each Task to process is wrong.

The integer division used does an implicit floor() of the division result. That messes up the calculation for the end point since the bias is to round down to the nearest integer.

Here's debug output of the start, end, and total entries in the file for each Task.

Task | Start   | End    | Total
-----+---------+--------+--------
 0   | 0       |1449706 |5798827
 1   | 1449706 |2899412 |5798827
 2   | 2899412 |4349118 |5798827
 3   | 4349118 |5798824 |5798827

The previous Task's range end should be the next Task's range start.

The problem is that the last Task's range end stops short of the actual entries end, so the code doesn't handle all the entries in the file.

Fixing the bug shouldn't affect the runtime significantly.

Thanks for writing all the blog entries. It's been an interesting read.

Some more optimizations which reduce the Parse runtime by about 20--25%

For DiffTimeInSeconds, time conversion code can be replaced with

                var st = (*(long*)(start + 11)) & 0x0F07000F07000F03;
                var st1 = (st * 2561) >> 8;
                var st2 = ((st1 & 0x000000003F00001F) * 0xE1000003C) >> 24;
                var st3 = (st1 >> 48);
                var startTime = (int)(st3 + st2) & 0x1FFFF;

                var en = (*(long*)(end + 11)) & 0x0F07000F07000F03;
                var en1 = (en * 2561) >> 8;
                var en2 = ((en1 & 0x000000003F00001F) * 0xE1000003C) >> 24;
                var en3 = (en1 >> 48);
                var endTime = (int)(en3 + en2) & 0x1FFFF;

                var diff = endTime - startTime;

For ParseInt8, replace with

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        private static int ParseInt8(byte* buffer)
        {
            var n = (*(long*)(buffer)) & 0x0F0F0F0F0F0F0F0F;

            var n1 = (n) * 2561 >> 8;
            var n2 = (n1 & 0x00FF00FF00FF00FF) * 6553601 >> 16;
            var num = (int)((n2 & 0x0000FFFF0000FFFF) * 42949672960001 >> 32);

            return num;
        }

See https://johnnylee-sde.github.io/Fast-numeric-string-to-int/

and https://johnnylee-sde.github.io/Fast-time-string-to-seconds/