High load when showing History or Bookmarks menu

[ALMh]

Displaying the "History" or "Bookmarks" menu takes a few seconds and a high load on the CPU on my old netbook.
Qupzilla 1.8.6
WebKit 538.1
Qt 5.4.1
Fedora 21

[nowrep]

That's known issue when you have a LOT of bookmarks.

[ALMh]

Anyway this menu opens slower, than window "Organize Bookmarks".

2015-10-02 23:27 GMT+08:00 David Rosca [email protected]:

That's known issue when you have a LOT of bookmarks.

—
Reply to this email directly or view it on GitHub
#1679 (comment).

[GreenLunar]

Turn the library into a standalone binary. The library would handle bookmarks, history (read only) and web feeds (atom, rdf, rss), QupZilla would handle browsing history, maybe via library, as it currently does.

If Mozilla would have done this with their own fancy places manager, they probably would have more audiences by now.

[Martii]

@GreenLunar
That covers desktops relatively easily with MIME/Media Types... so what about portable devices? e.g. they would need a guaranteed exchange mechanism for loading urls into a specific app I think.

[GreenLunar]

so what about portable devices?

I don't know, but if it is not manageable on portable devices, then perhaps using a local bookmarks/history database(s) would suffice.

Personally, I am biased in this concern. I have been using Firefox solely as a bookmark manager, and I have asked for a somewhat standalone bookmarks manager on another project bookieio/Bookie#550

[OlafLostViking]

I'm encountering the very same problem on my intel i7-3740QM / Samsung 830 SSD with 162 bookmarks. I hope this isn't "a LOT" ;-)

[OlafLostViking]

I SIGINTed QupZilla running in a gdb session while it was hanging/busy after opening the bookmarks menu:

Thread 1 "qupzilla" received signal SIGINT, Interrupt.
0x00007ffff1154b43 in pread64 () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff1154b43 in pread64 () from /usr/lib/libc.so.6
#1  0x00007fffc5162255 in ?? () from /usr/lib/libsqlite3.so.0
#2  0x00007fffc5165b31 in ?? () from /usr/lib/libsqlite3.so.0
#3  0x00007fffc51688f9 in sqlite3PagerGet () from /usr/lib/libsqlite3.so.0
#4  0x00007fffc5103f1f in ?? () from /usr/lib/libsqlite3.so.0
#5  0x00007fffc519a9b9 in ?? () from /usr/lib/libsqlite3.so.0
#6  0x00007fffc519b1f3 in sqlite3VdbeMemFromBtree () from /usr/lib/libsqlite3.so.0
#7  0x00007fffc519334d in sqlite3VdbeExec () from /usr/lib/libsqlite3.so.0
#8  0x00007fffc519447f in sqlite3_step () from /usr/lib/libsqlite3.so.0
#9  0x00007fffc53f0199 in ?? () from /usr/lib/qt/plugins/sqldrivers/libqsqlite.so
#10 0x00007fffc53f1363 in ?? () from /usr/lib/qt/plugins/sqldrivers/libqsqlite.so
#11 0x00007fffeee9c102 in QSqlQuery::exec() () from /usr/lib/libQt5Sql.so.5
#12 0x00007ffff7a8ee85 in IconProvider::imageForUrl(QUrl const&) () from /usr/lib/libQupZilla.so.2
#13 0x00007ffff7a8f5a4 in IconProvider::iconForUrl(QUrl const&) () from /usr/lib/libQupZilla.so.2
#14 0x00007ffff797fcc8 in BookmarkItem::icon() () from /usr/lib/libQupZilla.so.2
#15 0x00007ffff79a046a in BookmarksTools::addUrlToMenu(QObject*, Menu*, BookmarkItem*) () from /usr/lib/libQupZilla.so.2
#16 0x00007ffff79a192f in BookmarksTools::addFolderToMenu(QObject*, Menu*, BookmarkItem*) () from /usr/lib/libQupZilla.so.2
#17 0x00007ffff799ac17 in BookmarksMenu::refresh() () from /usr/lib/libQupZilla.so.2
#18 0x00007ffff799ac89 in BookmarksMenu::aboutToShow() () from /usr/lib/libQupZilla.so.2
#19 0x00007ffff7ab8725 in ?? () from /usr/lib/libQupZilla.so.2
#20 0x00007ffff1c6a659 in QMetaObject::activate(QObject*, int, int, void**) () from /usr/lib/libQt5Core.so.5
#21 0x00007ffff23508be in QMenu::popup(QPoint const&, QAction*) () from /usr/lib/libQt5Widgets.so.5
#22 0x00007ffff235c04b in ?? () from /usr/lib/libQt5Widgets.so.5
#23 0x00007ffff235e55f in ?? () from /usr/lib/libQt5Widgets.so.5
#24 0x00007ffff235ebb1 in QMenuBar::mousePressEvent(QMouseEvent*) () from /usr/lib/libQt5Widgets.so.5
#25 0x00007ffff222a76f in QWidget::event(QEvent*) () from /usr/lib/libQt5Widgets.so.5
#26 0x00007ffff235fe8b in QMenuBar::event(QEvent*) () from /usr/lib/libQt5Widgets.so.5
#27 0x00007ffff21e2e0c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib/libQt5Widgets.so.5
#28 0x00007ffff21eb43d in QApplication::notify(QObject*, QEvent*) () from /usr/lib/libQt5Widgets.so.5
#29 0x00007ffff1c3ede0 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib/libQt5Core.so.5
#30 0x00007ffff21e9aed in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool) () from /usr/lib/libQt5Widgets.so.5
#31 0x00007ffff2243fa0 in ?? () from /usr/lib/libQt5Widgets.so.5
#32 0x00007ffff2246743 in ?? () from /usr/lib/libQt5Widgets.so.5
#33 0x00007ffff21e2e0c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib/libQt5Widgets.so.5
#34 0x00007ffff21ea581 in QApplication::notify(QObject*, QEvent*) () from /usr/lib/libQt5Widgets.so.5
#35 0x00007ffff1c3ede0 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib/libQt5Core.so.5
#36 0x00007fffefd4d303 in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) ()
   from /usr/lib/libQt5Gui.so.5
#37 0x00007fffefd4ee85 in QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) ()
   from /usr/lib/libQt5Gui.so.5
#38 0x00007fffefd2ccab in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Gui.so.5
#39 0x00007fffdcbba390 in ?? () from /usr/lib/libQt5XcbQpa.so.5                                                                               
#40 0x00007fffedae4587 in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0                                                             
#41 0x00007fffedae47f0 in ?? () from /usr/lib/libglib-2.0.so.0                                                                                   
#42 0x00007fffedae489c in g_main_context_iteration () from /usr/lib/libglib-2.0.so.0                                                                
#43 0x00007ffff1c9370f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Core.so.5               
#44 0x00007ffff1c3d23a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Core.so.5                                     
#45 0x00007ffff1c4573c in QCoreApplication::exec() () from /usr/lib/libQt5Core.so.5                                                                    
#46 0x0000000000402a77 in ?? ()                                                                                                                             
#47 0x00007ffff109b291 in __libc_start_main () from /usr/lib/libc.so.6                                                                                      
#48 0x0000000000402d9a in ?? ()                                                                                                                                    

[nowrep]

@OlafLostViking Can you please run Tools -> Clear Recent History -> Optimize database and try again?

[OlafLostViking]

I already tried a manual vacuum of the database, which didn't help. Your proposed database optimization via QupZilla reduced the database size further from 50 MiB to 40 MiB but didn't bring any relevant changes, either. So I deleted the complete history and optimized the database down to 30 KiB. Now the bookmarks menu opens instantaneously.

(I made a backup of the database as I'd prefer to keep the history - in case there's another solution in the end ;). )

[nowrep]

Can you try with this patch, just to make sure it is due to icons:

diff --git a/src/lib/bookmarks/bookmarkitem.cpp b/src/lib/bookmarks/bookmarkitem.cpp
index f3c4b01..a6d08ef 100644
--- a/src/lib/bookmarks/bookmarkitem.cpp
+++ b/src/lib/bookmarks/bookmarkitem.cpp
@@ -78,7 +78,7 @@ QIcon BookmarkItem::icon()
     switch (m_type) {
     case Url:
         if (m_iconTime.isNull() || m_iconTime.elapsed() > iconCacheTime) {
-            m_icon = IconProvider::iconForUrl(m_url);
+//            m_icon = IconProvider::iconForUrl(m_url);
             m_iconTime.restart();
         }
         return m_icon;

[OlafLostViking]

I will try it later this day, if you still want me to do it; but I just deleted everything from the icons table in browsedata.db and the menu is now fast again.

[nowrep]

It would help if you can test this patch with your old browsedata.db (with all the history and icons), thanks.

[OlafLostViking]

Ah, I managed to access my "compile capable" ;-) machine earlier and I can confirm that your little patch mentioned above fixes/circumvents the problem.

[nowrep]

@OlafLostViking Can you please try it now with master?

[OlafLostViking]

Looks good, thanks. master opens the menu as fast as the version without icons.

The very first load I had a warning on the console, though. Unfortunately, I cannot reproduce it anymore. But I'll post it anyway to have it documented somewhere:

$ ./qupzilla 
[1211/105042:ERROR:browser_main_loop.cc(217)] Running without the SUID sandbox! See https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the sandbox on.
QupZilla: 0 extensions loaded
*** Error in `./qupzilla': corrupted double-linked list: 0x00007f70e806c640 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x70c4b)[0x7f71ad8f5c4b]
/usr/lib/libc.so.6(+0x76fe6)[0x7f71ad8fbfe6]
/usr/lib/libc.so.6(+0x77fbf)[0x7f71ad8fcfbf]
/usr/lib/libsqlite3.so.0(sqlite3_free+0x5d)[0x7f7181982e6d]
/usr/lib/libsqlite3.so.0(+0x98d28)[0x7f7181991d28]
/usr/lib/libsqlite3.so.0(+0x98e6c)[0x7f7181991e6c]
/usr/lib/libsqlite3.so.0(+0x98ed0)[0x7f7181991ed0]
/usr/lib/libsqlite3.so.0(sqlite3PagerClose+0x61)[0x7f718198b561]
/usr/lib/libsqlite3.so.0(sqlite3BtreeClose+0xab)[0x7f71819289fb]
/usr/lib/libsqlite3.so.0(sqlite3LeaveMutexAndCloseZombie+0x73)[0x7f718197f043]
/usr/lib/libsqlite3.so.0(+0x879fb)[0x7f71819809fb]
/usr/lib/qt/plugins/sqldrivers/libqsqlite.so(+0x7f80)[0x7f7181c11f80]
/home/olaflostviking/QUPZILLA/qupzilla/bin/libQupZilla.so.2(_ZN11SqlDatabaseD1Ev+0x97)[0x7f71b42ae907]
/home/olaflostviking/QUPZILLA/qupzilla/bin/libQupZilla.so.2(+0x286989)[0x7f71b42ae989]
/usr/lib/libc.so.6(+0x35990)[0x7f71ad8ba990]
/usr/lib/libc.so.6(+0x359ea)[0x7f71ad8ba9ea]
/usr/lib/libc.so.6(__libc_start_main+0xf8)[0x7f71ad8a5298]
./qupzilla[0x402d9a]

[nowrep]

@OlafLostViking Please report if you get this crash again.