Profile the interpreter

[wilhelma]

Use an appropriate profiler to profile the interpreter in order to find bottlenecks.

[wilhelma]

The bottleneck could actually be the DebugTool, since it's using io operations massively.

[pdigiglio]

After a quick analysis with callgrind, I saw that a lot of time was spent in DBInterpreter::getCallID() to scan through the CallTable_ searching for which call was issued on which instruction ID. So I added a std::map<INS_ID, CAL_ID> member variable to the CallTable_. This seems to have improved the performance but I can't tell yet by how much.

I'll also have a look at the DebugTool, of course!

[wilhelma]

Use std::vectors with .reserve()

[pdigiglio]

IDs in the database are not always contiguous, so I can't arrange the database rows in a std::vector. Also, it would be nice to have all the IDs starting at 0.

[pdigiglio]

@wilhelma I implemented the parallel database reading and a string class called FastString that is just a rapper for an array of characters. Here are the results of the runs.

• The string type does not affect much the execution. I believe this is due to the fact that there are not many TEXT fields in the database. The reason why you got a 2-second (or something like that) speedup in the parceive-workspace is that you choose to load the Reference table.
• There's not much speedup in the parallel version. However I'm running the interpreter on a computer with an SSD disk.

With std::string and 1 thread:

     1 ms : TRACE >> Rows in Thread:         1
   311 ms : TRACE >> Rows in Segment:        1587559
   781 ms : TRACE >> Rows in Reference:      1722791
  1454 ms : TRACE >> Rows in LoopIteration:  1658
  1519 ms : TRACE >> Rows in LoopExecution:  264269
  1521 ms : TRACE >> Rows in Loop:           28
  2510 ms : TRACE >> Rows in Instruction:    5182320
  2512 ms : TRACE >> Rows in Function:       42
  2513 ms : TRACE >> Rows in File:           3
  3157 ms : TRACE >> Rows in Call:           528709
  7496 ms : TRACE >> Rows in Access:         2930897

With std::string and 4 threads:

     1 ms : TRACE >> Rows in Thread:         1
   355 ms : TRACE >> Rows in Segment:        1587559
   813 ms : TRACE >> Rows in Reference:      1722791
   815 ms : TRACE >> Rows in LoopIteration:  1658
   881 ms : TRACE >> Rows in LoopExecution:  264269
   882 ms : TRACE >> Rows in Loop:           28
  1895 ms : TRACE >> Rows in Instruction:    5182320
  1897 ms : TRACE >> Rows in Function:       42
  1898 ms : TRACE >> Rows in File:           3
  2524 ms : TRACE >> Rows in Call:           528709
  6894 ms : TRACE >> Rows in Access:         2930897

With FastString and 1 thread:

     1 ms : TRACE >> Rows in Thread:         1
   559 ms : TRACE >> Rows in Segment:        1587559
  1448 ms : TRACE >> Rows in Reference:      1722791
  1450 ms : TRACE >> Rows in LoopIteration:  1658
  1553 ms : TRACE >> Rows in LoopExecution:  264269
  1554 ms : TRACE >> Rows in Loop:           28
  3433 ms : TRACE >> Rows in Instruction:    5182320
  3434 ms : TRACE >> Rows in Function:       42
  3435 ms : TRACE >> Rows in File:           3
  4172 ms : TRACE >> Rows in Call:           528709
  9149 ms : TRACE >> Rows in Access:         2930897

With FastString and 4 threads:

     1 ms : TRACE >> Rows in Thread:         1
   298 ms : TRACE >> Rows in Segment:        1587559
   748 ms : TRACE >> Rows in Reference:      1722791
   750 ms : TRACE >> Rows in LoopIteration:  1658
   853 ms : TRACE >> Rows in LoopExecution:  264269
   856 ms : TRACE >> Rows in Loop:           28
  1781 ms : TRACE >> Rows in Instruction:    5182320
  1783 ms : TRACE >> Rows in Function:       42
  1784 ms : TRACE >> Rows in File:           3
  2429 ms : TRACE >> Rows in Call:           528709
  6756 ms : TRACE >> Rows in Access:         2930897

[pdigiglio]

@wilhelma Do we by chance know that the strings in the database are shorter than a certain number N? This would allow me to use std::array<char, N> as underlying FastString type and spare me the troubles of heap memory management.

[pdigiglio]

I implemented a move constructor for FastString and used it to load data in the database records (i.e. access_t, call_t, and so on). I was able to get about a half-second speedup over the previous times:

     2 ms : TRACE >> Rows in Thread:         1
   227 ms : TRACE >> Rows in Segment:        1587559
   581 ms : TRACE >> Rows in Reference:      1722791
   583 ms : TRACE >> Rows in LoopIteration:  1658
   631 ms : TRACE >> Rows in LoopExecution:  264269
   633 ms : TRACE >> Rows in Loop:           28
  1389 ms : TRACE >> Rows in Instruction:    5182320
  1391 ms : TRACE >> Rows in Function:       42
  1392 ms : TRACE >> Rows in File:           3
  1986 ms : TRACE >> Rows in Call:           528709
  6160 ms : TRACE >> Rows in Access:         2930897