Menu
▾
▴
tora-develop
|
From: Tomas S. <tom...@gm...> - 2010-12-12 10:33:20
Attachments:
schema_browser_cache.patch.bz2
|
Hello Currently TOra cache is not utilised enough and therefore TOra re-executes the same queries again and again. http://torasql.com/knowhow:cache What do you think about extending cache functionality in following way: 1. Update cache while TOra is running (currently cache is only updated when opening connection). This would mean saving results of queries performed by user, obsolete data when user updates information about some objects etc. 2. Make cache store information which was actually required at least once (and eventually stop querying information about all objects even if user does not need that). 3. Save cache when closing connection. Attached is an example patch which does this: 1. Saves date when a particular record in cache was updated (to be used later for targeted clean-up of outdated cache parts). 2. Saves information about list of tables, views, sequences etc. when they are queried from database in schema browser. 3. Uses cached lists in schema browser (currently list content is re-queried each time active tab is changed). 4. Saves cache when closing connection. Any ideas why I should not commit this? Something which must be added with this? == C++ == Can somebody give me ideas about these two: 1. In tocache.cpp I've removed include of boost preprocessor local.hpp as it was causing compilation errors. Is it OK? This include was copied from original toconnection.cpp. 2. Cache class does not know about a structure used to store data in toResultModel (and I cannot include that as it introduces some circular dependencies). Therefore cache returns data in simple QList<QList<QVariant>> which later has to be looped through and converted to QList<QList<toQValue>>... Is there any way to make this faster/more elegant? (see changes to toresultmodel.cpp) Thank you -- Tomas Straupis |
|
From: Neulinger, N. <nn...@ms...> - 2010-12-12 16:31:54
|
If significant additional use of the cache is introduced, I would like to see an easy to use button/menu-item (not buried) to clear/flush the cache - both for security/data purposes and for 'glitches' that are certain to occur. -- Nathan ------------------------------------------------------------ Nathan Neulinger nn...@ms... Missouri S&T Information Technology (573) 612-1412 System Administrator - Principal KD0DMH > -----Original Message----- > From: Tomas Straupis [mailto:tom...@gm...] > Sent: Sunday, December 12, 2010 4:33 AM > To: tora-develop > Subject: [Tora-develop] Cache > > Hello > > Currently TOra cache is not utilised enough and therefore TOra > re-executes the same queries again and again. > http://torasql.com/knowhow:cache > > What do you think about extending cache functionality in following > way: > 1. Update cache while TOra is running (currently cache is only > updated when opening connection). This would mean saving results of > queries performed by user, obsolete data when user updates information > about some objects etc. > 2. Make cache store information which was actually required at least > once (and eventually stop querying information about all objects even > if user does not need that). > 3. Save cache when closing connection. > > Attached is an example patch which does this: > 1. Saves date when a particular record in cache was updated (to be > used later for targeted clean-up of outdated cache parts). > 2. Saves information about list of tables, views, sequences etc. when > they are queried from database in schema browser. > 3. Uses cached lists in schema browser (currently list content is > re-queried each time active tab is changed). > 4. Saves cache when closing connection. > > Any ideas why I should not commit this? Something which must be > added with this? > > == C++ == > > Can somebody give me ideas about these two: > 1. In tocache.cpp I've removed include of boost preprocessor local.hpp > as it was causing compilation errors. Is it OK? This include was > copied from original toconnection.cpp. > > 2. Cache class does not know about a structure used to store data in > toResultModel (and I cannot include that as it introduces some > circular dependencies). Therefore cache returns data in simple > QList<QList<QVariant>> which later has to be looped through and > converted to QList<QList<toQValue>>... Is there any way to make this > faster/more elegant? (see changes to toresultmodel.cpp) > > Thank you > > -- > Tomas Straupis |
|
From: Tomas S. <tom...@gm...> - 2010-12-18 21:40:38
|
2010-12-12 Neulinger, Nathan: > If significant additional use of the cache is introduced, I would like to see > an easy to use button/menu-item (not buried) to clear/flush the cache - > both for security/data purposes and for 'glitches' that are certain to occur. Could you clarify, are current changes "significant"? Is current menu option to re-read the cache sufficient? For security reasons, should the option be added to say prevent userlist from being saved to disk (currently you can do that by disabling storing cache contents on disk altogether)? -- Tomas Straupis |
|
From: Neulinger, N. <nn...@ms...> - 2010-12-19 16:21:35
|
Does re-reading the cache re-populate it from the source database, or reread it from disk? If the former, that sounds fine. -- Nathan ------------------------------------------------------------ Nathan Neulinger nn...@ms... Missouri S&T Information Technology (573) 612-1412 System Administrator - Principal KD0DMH > -----Original Message----- > From: Tomas Straupis [mailto:tom...@gm...] > Sent: Saturday, December 18, 2010 3:40 PM > To: tora-develop > Subject: Re: [Tora-develop] Cache > > 2010-12-12 Neulinger, Nathan: > > If significant additional use of the cache is introduced, I would > like to see > > an easy to use button/menu-item (not buried) to clear/flush the cache > - > > both for security/data purposes and for 'glitches' that are certain > to occur. > > Could you clarify, are current changes "significant"? > Is current menu option to re-read the cache sufficient? > For security reasons, should the option be added to say prevent > userlist from being saved to disk (currently you can do that by > disabling storing cache contents on disk altogether)? > > -- > Tomas Straupis > > ----------------------------------------------------------------------- > ------- > Lotusphere 2011 > Register now for Lotusphere 2011 and learn how > to connect the dots, take your collaborative environment > to the next level, and enter the era of Social Business. > http://p.sf.net/sfu/lotusphere-d2d > _______________________________________________ > Tora-develop mailing list > Tor...@li... > https://lists.sourceforge.net/lists/listinfo/tora-develop |
|
From: Tomas S. <tom...@gm...> - 2010-12-19 16:48:54
|
2010-12-19 Neulinger, Nathan: > Does re-reading the cache re-populate it from the source database, or > reread it from disk? If the former, that sounds fine. Re-reading clears the cache and re-queries the database. -- Tomas Straupis |
|
From: Neulinger, N. <nn...@ms...> - 2010-12-19 17:39:02
|
That works for me! ------------------------------------------------------------ Nathan Neulinger nn...@ms... Missouri S&T Information Technology (573) 612-1412 System Administrator - Principal KD0DMH > -----Original Message----- > From: Tomas Straupis [mailto:tom...@gm...] > Sent: Sunday, December 19, 2010 10:48 AM > To: tora-develop > Subject: Re: [Tora-develop] Cache > > 2010-12-19 Neulinger, Nathan: > > Does re-reading the cache re-populate it from the source database, or > > reread it from disk? If the former, that sounds fine. > > Re-reading clears the cache and re-queries the database. > > -- > Tomas Straupis > > ----------------------------------------------------------------------- > ------- > Lotusphere 2011 > Register now for Lotusphere 2011 and learn how > to connect the dots, take your collaborative environment > to the next level, and enter the era of Social Business. > http://p.sf.net/sfu/lotusphere-d2d > _______________________________________________ > Tora-develop mailing list > Tor...@li... > https://lists.sourceforge.net/lists/listinfo/tora-develop |
|
From: Mike J. <mrj...@mi...> - 2010-12-18 22:22:41
|
I'm getting a segfault right now trying to open the schema browser on a
postgres connection:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7a157c6 in QString::operator==(QString const&) const () from
/usr/lib/libQtCore.so.4
(gdb) bt
#0 0x00007ffff7a157c6 in QString::operator==(QString const&) const () from
/usr/lib/libQtCore.so.4
#1 0x0000000000458243 in toCache::updateObjects (this=<value optimized
out>, owner=<value optimized out>, type=..., rows=<value optimized out>) at
tocache.cpp:524
#2 0x000000000066fcea in toBrowserSchemaTableView::updateCache (this=<value
optimized out>) at tobrowserschemawidget.cpp:98
#3 0x000000000070e930 in toBrowserSchemaTableView::qt_metacall
(this=0x15ac390, _c=QMetaObject::InvokeMetaMethod, _id=<value optimized
out>, _a=<value optimized out>)
at moc_tobrowserschemawidget.cpp:74
#4 0x00007ffff7adae3f in QMetaObject::activate(QObject*, QMetaObject
const*, int, void**) () from /usr/lib/libQtCore.so.4
#5 0x0000000000707014 in toResultTableView::qt_metacall (this=0x15ac390,
_c=QMetaObject::InvokeMetaMethod, _id=20721312, _a=0x7fffffffd070) at
moc_toresulttableview.cpp:112
#6 0x000000000070e918 in toBrowserSchemaTableView::qt_metacall
(this=0x13c2e98, _c=25189424, _id=20721312, _a=0x188f8c0) at
moc_tobrowserschemawidget.cpp:69
#7 0x00007ffff7adae3f in QMetaObject::activate(QObject*, QMetaObject
const*, int, void**) () from /usr/lib/libQtCore.so.4
#8 0x00000000004cdd8f in toResultModel::cleanup (this=0x7fffe4000da0) at
toresultmodel.cpp:163
#9 0x00000000004ce0e6 in toResultModel::readData (this=<value optimized
out>) at toresultmodel.cpp:263
#10 0x00000000004ce42c in toResultModel::fetchMore (this=0x7fffe4000da0) at
toresultmodel.cpp:782
#11 0x0000000000706d54 in toResultModel::qt_metacall (this=0x7fffe4000da0,
_c=QMetaObject::InvokeMetaMethod, _id=20721312, _a=0x7fffffffd370) at
moc_toresultmodel.cpp:100
#12 0x00007ffff7adae3f in QMetaObject::activate(QObject*, QMetaObject
const*, int, void**) () from /usr/lib/libQtCore.so.4
#13 0x00000000006fdb60 in toEventQuery::done (this=0x13c2e98,
_t1=0x7fffe4008cb0) at moc_toeventquery.cpp:138
#14 0x00000000006fdda4 in toEventQuery::qt_metacall (this=0x7fffe4008cb0,
_c=QMetaObject::InvokeMetaMethod, _id=20721312, _a=0x7fffe400ef30) at
moc_toeventquery.cpp:103
#15 0x00007ffff7ad7d49 in QObject::event(QEvent*) () from
/usr/lib/libQtCore.so.4
#16 0x00007ffff6e8122c in QApplicationPrivate::notify_helper(QObject*,
QEvent*) () from /usr/lib/libQtGui.so.4
#17 0x00007ffff6e876fb in QApplication::notify(QObject*, QEvent*) () from
/usr/lib/libQtGui.so.4
#18 0x00007ffff7ac806c in QCoreApplication::notifyInternal(QObject*,
QEvent*) () from /usr/lib/libQtCore.so.4
#19 0x00007ffff7aca7e7 in
QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) ()
from /usr/lib/libQtCore.so.4
#20 0x00007ffff7af19d3 in ?? () from /usr/lib/libQtCore.so.4
#21 0x00007ffff22db8c2 in g_main_dispatch (context=0xee2520) at
/build/buildd/glib2.0-2.24.1/glib/gmain.c:1960
#22 IA__g_main_context_dispatch (context=0xee2520) at
/build/buildd/glib2.0-2.24.1/glib/gmain.c:2513
#23 0x00007ffff22df748 in g_main_context_iterate (context=0xee2520,
block=<value optimized out>, dispatch=<value optimized out>, self=<value
optimized out>)
at /build/buildd/glib2.0-2.24.1/glib/gmain.c:2591
#24 0x00007ffff22df8fc in IA__g_main_context_iteration (context=0xee2520,
may_block=1) at /build/buildd/glib2.0-2.24.1/glib/gmain.c:2654
#25 0x00007ffff7af1513 in
QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
() from /usr/lib/libQtCore.so.4
#26 0x00007ffff6f3146e in ?? () from /usr/lib/libQtGui.so.4
#27 0x00007ffff7ac6992 in
QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from
/usr/lib/libQtCore.so.4
#28 0x00007ffff7ac6d6c in
QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from
/usr/lib/libQtCore.so.4
#29 0x00007ffff7acaaab in QCoreApplication::exec() () from
/usr/lib/libQtCore.so.4
#30 0x000000000043562c in main (argc=1, argv=<value optimized out>) at
main.cpp:284
On Sat, Dec 18, 2010 at 1:40 PM, Tomas Straupis <tom...@gm...>wrote:
> 2010-12-12 Neulinger, Nathan:
> > If significant additional use of the cache is introduced, I would like to
> see
> > an easy to use button/menu-item (not buried) to clear/flush the cache -
> > both for security/data purposes and for 'glitches' that are certain to
> occur.
>
> Could you clarify, are current changes "significant"?
> Is current menu option to re-read the cache sufficient?
> For security reasons, should the option be added to say prevent
> userlist from being saved to disk (currently you can do that by
> disabling storing cache contents on disk altogether)?
>
> --
> Tomas Straupis
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> Tora-develop mailing list
> Tor...@li...
> https://lists.sourceforge.net/lists/listinfo/tora-develop
>
--
Open source monitoring done right: http://www.leemba.com
|
|
From: Mike J. <mrj...@mi...> - 2010-12-18 22:23:51
|
Some more details:
(gdb) up
#1 0x0000000000458243 in toCache::updateObjects (this=<value optimized
out>, owner=<value optimized out>, type=..., rows=<value optimized out>) at
tocache.cpp:524
524 if ((*currentObjects).Name == (*newObjects).Name)
(gdb) list
519 {
520 while ((*currentObjects).Type != type && currentObjects
!= ObjectNames.end())
521 currentObjects++; // skip cached objects of other
types
522
523 //qDebug() << "iterating through (new:curr)" <<
(*newObjects).Name << (*currentObjects).Name;
524 if ((*currentObjects).Name == (*newObjects).Name)
525 {
526 //qDebug() << "Object is already in cache" <<
(*newObjects).Name;
527 currentObjects++;
528 newObjects++;
(gdb) print *currentObjects
$2 = (struct toCache::objectName &) @0x13c2e98: {Name = {d =
0xcdad8e7053371c02}, Owner = {d = 0x7159422c00000000}, Type = {d =
0x7fffe4017180}, Comment = {d = 0x7fffe4014840},
Synonyms = {<std::_List_base<QString, std::allocator<QString> >> = {
_M_impl = {<std::allocator<std::_List_node<QString> >> =
{<__gnu_cxx::new_allocator<std::_List_node<QString> >> = {<No data fields>},
<No data fields>}, _M_node = {
_M_next = 0x7fffe401d270, _M_prev = 0xf4}}}, <No data fields>},
Timestamp = {jd = 21462448}}
(gdb) print (*currentObjects).Name
$4 = {d = 0xcdad8e7053371c02}
(gdb) printq4string (*currentObjects).Name
Cannot access memory at address 0xcdad8e7053371c0a
On Sat, Dec 18, 2010 at 2:22 PM, Mike Johnson <mrj...@mi...> wrote:
> I'm getting a segfault right now trying to open the schema browser on a
> postgres connection:
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff7a157c6 in QString::operator==(QString const&) const () from
> /usr/lib/libQtCore.so.4
> (gdb) bt
> #0 0x00007ffff7a157c6 in QString::operator==(QString const&) const () from
> /usr/lib/libQtCore.so.4
> #1 0x0000000000458243 in toCache::updateObjects (this=<value optimized
> out>, owner=<value optimized out>, type=..., rows=<value optimized out>) at
> tocache.cpp:524
> #2 0x000000000066fcea in toBrowserSchemaTableView::updateCache
> (this=<value optimized out>) at tobrowserschemawidget.cpp:98
> #3 0x000000000070e930 in toBrowserSchemaTableView::qt_metacall
> (this=0x15ac390, _c=QMetaObject::InvokeMetaMethod, _id=<value optimized
> out>, _a=<value optimized out>)
> at moc_tobrowserschemawidget.cpp:74
> #4 0x00007ffff7adae3f in QMetaObject::activate(QObject*, QMetaObject
> const*, int, void**) () from /usr/lib/libQtCore.so.4
> #5 0x0000000000707014 in toResultTableView::qt_metacall (this=0x15ac390,
> _c=QMetaObject::InvokeMetaMethod, _id=20721312, _a=0x7fffffffd070) at
> moc_toresulttableview.cpp:112
> #6 0x000000000070e918 in toBrowserSchemaTableView::qt_metacall
> (this=0x13c2e98, _c=25189424, _id=20721312, _a=0x188f8c0) at
> moc_tobrowserschemawidget.cpp:69
> #7 0x00007ffff7adae3f in QMetaObject::activate(QObject*, QMetaObject
> const*, int, void**) () from /usr/lib/libQtCore.so.4
> #8 0x00000000004cdd8f in toResultModel::cleanup (this=0x7fffe4000da0) at
> toresultmodel.cpp:163
> #9 0x00000000004ce0e6 in toResultModel::readData (this=<value optimized
> out>) at toresultmodel.cpp:263
> #10 0x00000000004ce42c in toResultModel::fetchMore (this=0x7fffe4000da0) at
> toresultmodel.cpp:782
> #11 0x0000000000706d54 in toResultModel::qt_metacall (this=0x7fffe4000da0,
> _c=QMetaObject::InvokeMetaMethod, _id=20721312, _a=0x7fffffffd370) at
> moc_toresultmodel.cpp:100
> #12 0x00007ffff7adae3f in QMetaObject::activate(QObject*, QMetaObject
> const*, int, void**) () from /usr/lib/libQtCore.so.4
> #13 0x00000000006fdb60 in toEventQuery::done (this=0x13c2e98,
> _t1=0x7fffe4008cb0) at moc_toeventquery.cpp:138
> #14 0x00000000006fdda4 in toEventQuery::qt_metacall (this=0x7fffe4008cb0,
> _c=QMetaObject::InvokeMetaMethod, _id=20721312, _a=0x7fffe400ef30) at
> moc_toeventquery.cpp:103
> #15 0x00007ffff7ad7d49 in QObject::event(QEvent*) () from
> /usr/lib/libQtCore.so.4
> #16 0x00007ffff6e8122c in QApplicationPrivate::notify_helper(QObject*,
> QEvent*) () from /usr/lib/libQtGui.so.4
> #17 0x00007ffff6e876fb in QApplication::notify(QObject*, QEvent*) () from
> /usr/lib/libQtGui.so.4
> #18 0x00007ffff7ac806c in QCoreApplication::notifyInternal(QObject*,
> QEvent*) () from /usr/lib/libQtCore.so.4
> #19 0x00007ffff7aca7e7 in
> QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) ()
> from /usr/lib/libQtCore.so.4
> #20 0x00007ffff7af19d3 in ?? () from /usr/lib/libQtCore.so.4
> #21 0x00007ffff22db8c2 in g_main_dispatch (context=0xee2520) at
> /build/buildd/glib2.0-2.24.1/glib/gmain.c:1960
> #22 IA__g_main_context_dispatch (context=0xee2520) at
> /build/buildd/glib2.0-2.24.1/glib/gmain.c:2513
> #23 0x00007ffff22df748 in g_main_context_iterate (context=0xee2520,
> block=<value optimized out>, dispatch=<value optimized out>, self=<value
> optimized out>)
> at /build/buildd/glib2.0-2.24.1/glib/gmain.c:2591
> #24 0x00007ffff22df8fc in IA__g_main_context_iteration (context=0xee2520,
> may_block=1) at /build/buildd/glib2.0-2.24.1/glib/gmain.c:2654
> #25 0x00007ffff7af1513 in
> QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)
> () from /usr/lib/libQtCore.so.4
> #26 0x00007ffff6f3146e in ?? () from /usr/lib/libQtGui.so.4
> #27 0x00007ffff7ac6992 in
> QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from
> /usr/lib/libQtCore.so.4
> #28 0x00007ffff7ac6d6c in
> QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from
> /usr/lib/libQtCore.so.4
> #29 0x00007ffff7acaaab in QCoreApplication::exec() () from
> /usr/lib/libQtCore.so.4
> #30 0x000000000043562c in main (argc=1, argv=<value optimized out>) at
> main.cpp:284
>
>
>
> On Sat, Dec 18, 2010 at 1:40 PM, Tomas Straupis <tom...@gm...>wrote:
>
>> 2010-12-12 Neulinger, Nathan:
>> > If significant additional use of the cache is introduced, I would like
>> to see
>> > an easy to use button/menu-item (not buried) to clear/flush the cache -
>> > both for security/data purposes and for 'glitches' that are certain to
>> occur.
>>
>> Could you clarify, are current changes "significant"?
>> Is current menu option to re-read the cache sufficient?
>> For security reasons, should the option be added to say prevent
>> userlist from being saved to disk (currently you can do that by
>> disabling storing cache contents on disk altogether)?
>>
>> --
>> Tomas Straupis
>>
>>
>> ------------------------------------------------------------------------------
>> Lotusphere 2011
>> Register now for Lotusphere 2011 and learn how
>> to connect the dots, take your collaborative environment
>> to the next level, and enter the era of Social Business.
>> http://p.sf.net/sfu/lotusphere-d2d
>> _______________________________________________
>> Tora-develop mailing list
>> Tor...@li...
>> https://lists.sourceforge.net/lists/listinfo/tora-develop
>>
>
>
>
> --
> Open source monitoring done right: http://www.leemba.com
>
--
Open source monitoring done right: http://www.leemba.com
|
|
From: Tomas S. <tom...@gm...> - 2010-12-19 09:14:08
|
2010-12-19 Mike Johnson: > I'm getting a segfault right now trying to open the schema browser on a > postgres connection: Please update and try again. -- Tomas Straupis |
|
From: Mike J. <mrj...@mi...> - 2010-12-19 17:25:41
|
Yup, works now. Thanks! On Sun, Dec 19, 2010 at 1:13 AM, Tomas Straupis <tom...@gm...>wrote: > 2010-12-19 Mike Johnson: > > I'm getting a segfault right now trying to open the schema browser on a > > postgres connection: > > Please update and try again. > > -- > Tomas Straupis > > > ------------------------------------------------------------------------------ > Lotusphere 2011 > Register now for Lotusphere 2011 and learn how > to connect the dots, take your collaborative environment > to the next level, and enter the era of Social Business. > http://p.sf.net/sfu/lotusphere-d2d > _______________________________________________ > Tora-develop mailing list > Tor...@li... > https://lists.sourceforge.net/lists/listinfo/tora-develop > -- Open source monitoring done right: http://www.leemba.com |
Thanks for helping keep SourceForge clean.
X