Is there a max iteration for evaluating predicate?

[brandonshiyay]

I was running queries on a Java project and it didn't seem to end. Normally it'd take about less than an hour at most to finish those queries, but this time it has been almost 12 hours and still going. I was running it on a 8-core CPU with 8 GBs of memory machine.

I checked execute-queries.log and found some predicates are being evaluated on the 4000th iteration.

I tried to do some research, but couldn't find much related. I'm not sure if I'm running into an infinite loop or the project I'm trying to analyze is too large as the database directory is about 2.6 GBs large.

But regardless, I'm also curious to find out if the iteration amount I mentioned above is normal, and if so, is there a maximum iteration?

[smowton]

How much of the time has been spent analysing queries in the ModulusAnalysis:: space? There isn't an iteration limit, nor (barring bugs) can there be infinite loops in codeql, only very large result sets, but if the same project has previously been analysed in a reasonable timeframe then that's likely a regression we want to look at. What is your position regarding sharing the log or the database for investigation?

[brandonshiyay]

I want to clarify that it was my first time analyzing this very project. When I was analyzing other projects it usually wouldn't take long, but this one took an absurd amount time. Since this is an internal project so unfortunately I cannot provide the database at this point, but I will be more than happy to share the log with you guys. However, before that, I just want to make sure may this just very well be the issue of large database instead of the performance issue of codeql?

[smowton]

That's certainly a remarkably large database, so it's possible it really is intractable to analyse. If that's the case, you should consider building a subset of the project to analyse (see https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language if using Code Scanning, or use the -c option if working from the command-line).

On the other hand your particular project could have found a pathological case in our modulus analysis in particular, in which case we'd be interested to see the log in case there's an easy fix we can identify.

[brandonshiyay]

Thanks for the explanation! Is there a specific log file you want to look at? Or will all the *.log files be helpful?

[smowton]

The file you quote in part in the screenshot

[brandonshiyay]

execute-queries.log

For some privacy reasons, I omitted some command-line info.