On Sat, 2009-08-29 at 12:38 +0200, David Engster wrote:
> Eric M. Ludlam <eric@...> writes:
> > I'm starting with the premise that we want to avoid the searches in a
> > dereferencer. I'm sure the thing you wrote works fine, I'm just
> > optimizing here. In fact, I'm only trying to optimize a single line,
> > which was a global search across all tables.
>
> I'm not sure which line you are referring to.
>
> There is
>
> (semanticdb-find-tags-by-name name)
>
> in semantic-c-check-type-namespace-alias, and
>
> (semanticdb-find-tags-by-name (car typename))
>
> in semantic-c-dereference-type-alias. Both do a search for the aliased
> namespace. Then, there's
These two are troublesome because they search "everything". If there
was some search path to add to the search, then this wouldn't be a
problem.
I also know that this is important to do sometimes. For example,
semantic-ctxt-scoped-types has to do this a couple times to build the
scope. Fortunately in this case, the scope is cached, and can be
recycled between requests.
Actually, I wonder if the search for using statements in the scope can
be used instead. Maybe I was completely off track, and we could add
a :aliases slot to the scope?
> (semantic-deep-find-tags-by-name typename result)
This one is fine since you are passing the results of some other
search in. The tough ones are when you have to search all the active
tables.
> which searches for the type in the namespace.
>
> > The question is then, can we leave special tags behind in the typecache
> > during the scan? I'm not entirely sure after reading your mail. I was
> > hoping something like a typedef could be left behind while building the
> > cache. We would then need a dereferencer for the metatag, but that
> > dereferencer would look just like the typedef one which has no searches
> > in it.
> >
> > The trick is that if you find one of your special namespace entries, it
> > would leave behind something with a name that matches what would be
> > searched for. This tag would be found, and then the dereferences would
> > just return a new name of whatever it referenced.
> >
> > What makes me uncertain if this will work now is that this only works
> > for a fully qualified type. If I understand the problem right, your
> > aliases are for names in the middle of a fully qualified name.
>
> Well, they can be completely unqualified. Consider
>
> --- someheader ---
>
> namespace foo {
> using namespace std;
> }
>
> ------------------
>
> --- somefile ---
>
> #include "someheader"
>
> int main(void) {
> using namespace foo;
> vector<int> v;
> v.//complete here
> }
>
> Even if I had a metatag which tells me that foo is an alias for std, I'd
> still have to lookup 'vector' in std.
Yes. This is the key functionality of the metaclass dereference loop.
Your dereferencer only needs to convert foo::vector into std::vector.
The outer loop will then do the second lookup.
What I think I now understand is that you cannot convert foo::vector to
std::vector unless you search in std:: for vector to validate that
translation?
> > Or perhaps this is a premature optimization, and I'm just speculating.
> > Using ELP in the past has usually been the best thing.
> >
> > You could do before/after runs of the unit test suite, and see what the
> > delta is. That would be easy.
>
> There is no delta, since the dereferencer doesn't do anything for the
> unit tests, as they're all un-aliased types. Maybe we should a include a
> completion in the unit tests which deliberately fails, to see the impact
> when all dereferencers fire up and do their job.
Well, then that is good. It hadn't occurred to me that might happen.
Only complex code will suffer complex lookups. Or perhaps failures in
lookup will also go through this.
> I tried to create something like a 'worst case' scenario (see the
> attached file). I include almost the full STL, then create a namespace
> 'foo' which is an alias for 'std'. When I then fire up the analyzer, I
> get the following profile:
>
> semantic-analyze-current-context 1 15.577188 15.577188
> semantic-analyze-current-context-default 1 14.714472 14.714472
> semantic-analyze-find-tag-sequence 1 12.615496 12.615496
> semantic-analyze-find-tag-sequence-default 1 12.615482 12.615482
> semantic-sort-tags-by-name-then-type-increasing 436 10.639908000 0.0244034587
> semantic-tag-lessp-name-then-type 655951 3.1323869999 4.775...e-06
> semanticdb-find-translate-path 7 2.2029980000 0.3147140000
> semanticdb-find-translate-path-default 7 2.202899 0.3146998571
> semanticdb-find-translate-path-includes-default 7 2.2027309999 0.3146758571
> semanticdb-file-table-object 375 2.1800860000 0.0058135626
> semanticdb-find-tags-collector 5 2.1140149999 0.4228029999
>
> [...]
>
> semantic-c-dereference-namespace 2 0.027656 0.013828
> semantic-c-dereference-type-alias 1 0.026057 0.026057
>
> It seems the dereferencer doesn't really matter, but maybe the example
> is somewhat wrong; I'm not sure.
>
> Regards,
> David
>
> --- Test case:
>
> #include <algorithm>
> #include <ccomplex>
> #include <climits>
> #include <cstdarg>
> #include <ctgmath>
> #include <deque>
> #include <limits>
> #include <ostream>
> #include <stack>
> #include <tuple>
> #include <vector>
> #include <array>
> #include <cctype>
> #include <clocale>
> #include <cstdbool>
> #include <ctime>
> #include <exception>
> #include <iomanip>
> #include <list>
> #include <stdexcept>
> #include <typeinfo>
> #include <cerrno>
> #include <cmath>
> #include <cstddef>
> #include <cwchar>
> #include <locale>
> #include <queue>
> #include <streambuf>
> #include <cfenv>
> #include <complex>
> #include <cstdint>
> #include <cwctype>
> #include <iosfwd>
> #include <map>
> #include <random>
> #include <string>
> #include <unordered_map>
> #include <bitset>
> #include <cfloat>
> #include <cstdio>
> #include <iostream>
> #include <memory>
> #include <regex>
> #include <unordered_set>
> #include <cinttypes>
> #include <csetjmp>
> #include <cstdlib>
> #include <fstream>
> #include <istream>
> #include <new>
> #include <set>
> #include <utility>
> #include <cassert>
> #include <ciso646>
> #include <csignal>
> #include <cstring>
> #include <functional>
> #include <iterator>
> #include <numeric>
> #include <sstream>
> #include <valarray>
>
> namespace foo {
> using namespace std;
> }
>
> int main(void)
> {
> foo::vector<int> v;
> v.
> }
>
This certainly seems like a good way to exercise the system.
>
> ---- Profile:
>
>
> semantic-analyze-current-context 1 15.577188 15.577188
> semantic-analyze-current-context-default 1 14.714472 14.714472
> semantic-analyze-find-tag-sequence 1 12.615496 12.615496
> semantic-analyze-find-tag-sequence-default 1 12.615482 12.615482
> semantic-sort-tags-by-name-then-type-increasing 436 10.639908000 0.0244034587
> semantic-tag-lessp-name-then-type 655951 3.1323869999 4.775...e-06
These two are our typecache overhead.
> semanticdb-find-translate-path 7 2.2029980000 0.3147140000
> semanticdb-find-translate-path-default 7 2.202899 0.3146998571
> semanticdb-find-translate-path-includes-default 7 2.2027309999 0.3146758571
I think these used to be called once. It is needed for every search
that doesn't specify a search path. The stuff I added for creating
mini-scopes around a parent class to search for that parent's parent
class probably increased this number.
> semanticdb-file-table-object 375 2.1800860000 0.0058135626
> semanticdb-find-tags-collector 5 2.1140149999 0.4228029999
> semantic-analyze-scoped-types 4 2.0995730000 0.5248932500
> semantic-analyze-scoped-types-default 4 2.0995009999 0.5248752499
> semantic-ctxt-scoped-types 4 2.0994539999 0.5248634999
> semantic-ctxt-scoped-types-c++-mode 4 2.099407 0.52485175
These two are the overhead for the wide searches I talked about above
that gets cached in the local scope.
> semantic-calculate-scope 1 2.097087 2.097087
> semanticdb-find-tags-by-class 4 2.089528 0.522382
> semanticdb-find-translate-path-includes--internal 2 2.026349 1.0131745
> semanticdb-find-table-for-include 317 1.9044870000 0.0060078454
> semanticdb-find-table-for-include-default 317 1.8996039999 0.0059924416
These two show that you included lots of include files. :)
> semanticdb-get-database 14 1.856218 0.1325869999
> semanticdb-create-database 15 1.8510979999 0.1234065333
> semanticdb-load-database 12 1.8325850000 0.1527154166
These show you ran the profiler test before the idle timer caught all
the includes.
> semanticdb-find-load-unloaded 311 1.763112 0.0056691704
> semanticdb-find-load-unloaded-default 247 1.7506250000 0.0070875506
I struggled with these a lot. Huzzah for the small time on them. :)
> semantic-decorate-add-decorations 301 1.1829669999 0.0039301229
Hmmm.
> semantic-refresh-tags-safe 3 0.862528 0.2875093333
> semantic-fetch-tags 14 0.8625099999 0.0616078571
> semantic--set-buffer-cache 2 0.5229929999 0.2614964999
> semantic-decorate-tags-after-full-reparse 2 0.522069 0.2610345
Hmmm again. Did you use a variant of semantic-elp-analyze to run your
test? This fcn shouldn't show up unless you run the profiler test and
request by hand.
I apologize for not describing the profiling tool in Semantic earlier.
Unless you did, and something is broken leaking out the decorator. It
seems speedy enough though.
The semantic wrapper for elp is helpful because it forces many caches to
be deleted so it turns a given test into a worse case scenario. It will
also save the results in an save-file that can be reloaded and browsed
later, allowing comparisons between runs.
> semantic-decoration-on-includes-highlight 64 0.501624 0.007837875
> semantic-decoration-on-includes-highlight-default 64 0.5004510000 0.0078195468
> semantic-tag-copy 3811 0.4500199999 0.0001180844
> semantic-analyze-scoped-type-parts 7 0.3445380000 0.0492197142
> semantic-analyze-tag-type 1 0.340777 0.340777
> semantic-analyze-type 1 0.3407670000 0.3407670000
> semantic-analyze-dereference-metatype-stack 1 0.338855 0.338855
> semantic-analyze-dereference-metatype 2 0.338837 0.1694185
> semantic-analyze-dereference-metatype-c-mode 2 0.3387280000 0.1693640000
> semantic-analyze-scoped-inherited-tags 6 0.331399 0.0552331666
> semantic-analyze-scoped-inherited-tag-map 9 0.3313939999 0.0368215555
> semantic-c-dereference-member-of 2 0.3109969999 0.1554984999
> semantic-new-buffer-fcn 1 0.254285 0.254285
> semantic-decoration-mode 1 0.219424 0.219424
> semantic-decoration-mode-setup 1 0.219384 0.219384
> semantic--tag-put-property 5935 0.2038390000 3.434...e-05
> semanticdb-refresh-table 753 0.1861490000 0.0002472098
> semantic-something-to-tag-table 1086 0.1793570000 0.0001651537
> semantic-tag-with-position-p 4560 0.1777190000 3.897...e-05
> semanticdb-get-table-index 274 0.1762299999 0.0006431751
> semantic-tag-components-with-overlays 596 0.1717100000 0.0002881040
> semantic-tag-components-with-overlays-default 596 0.1645049999 0.0002760151
> semanticdb-find-search-index 235 0.1596429999 0.0006793319
> semantic-tag-make-plist 6181 0.1525790000 2.468...e-05
> semantic-tag-of-type-p 29117 0.1499450000 5.149...e-06
> semanticdb-find-need-cache-update-p 7 0.1377140000 0.0196734285
> semanticdb-find-incomplete-cache-entries-p 7 0.1373759999 0.0196251428
> semanticdb-abstract-table-child-p 557 0.1333470000 0.0002394021
> semanticdb-table-child-p 512 0.1320040000 0.0002578203
> semantic--tag-link-to-buffer 298 0.1134949999 0.0003808557
> semantic-parse-region 5 0.100303 0.0200606
> semantic-parse-region-c-mode 5 0.100206 0.0200412000
> semantic-parse-region-default 5 0.1001430000 0.0200286
> semantic-tag-file-name 5229 0.0970610000 1.856...e-05
> semantic-repeat-parse-whole-stream 5 0.093378 0.0186756
> semantic-tag-in-buffer-p 6034 0.0841019999 1.393...e-05
> semantic-parse-stream 75 0.0823349999 0.0010977999
> semanticdb-find-tags-by-class-method 524 0.0701199999 0.0001338167
> semantic-dependency-find-file-on-path 261 0.065377 0.0002504865
> semanticdb-file-table 234 0.0641979999 0.0002743504
> semantic-parse-stream-default 1 0.062927 0.062927
> semanticdb-full-filename 892 0.0556190000 6.235...e-05
> semantic-decorate-tag 155 0.0551050000 0.0003555161
> semantic-overlay-p 22775 0.0525950000 2.309...e-06
> semanticdb-needs-refresh-p 247 0.0509880000 0.0002064291
> semanticdb-in-buffer-p 1382 0.0505340000 3.656...e-05
> semanticdb-directory-loaded-p 233 0.0491140000 0.0002107896
> semantic-tag-boundary-highlight 91 0.0430130000 0.0004726703
> semantic--dependency-find-file-on-path 261 0.0426429999 0.0001633831
> semantic-tag-boundary-highlight-default 91 0.0420030000 0.0004615714
> semantic-tag-create-secondary-overlay 155 0.0418080000 0.0002697290
> semanticdb-get-tags 1007 0.0394749999 3.920...e-05
> semanticdb-refresh-references 2 0.037945 0.0189725
> semanticdb-add-reference 64 0.037379 0.0005840468
> semanticdb-semantic-init-hook-fcn 1 0.034149 0.034149
> semanticdb-table 218 0.0341469999 0.0001566376
> semantic--tag-link-cache-to-buffer 1 0.031986 0.031986
> semantic-c-dereference-namespace 2 0.027656 0.013828
> semantic-c-dereference-type-alias 1 0.026057 0.026057
Wow, I'm impressed you can have those searches in these routines, and
have them be this speedy.
I think you have proven your point well. Thanks for putting in the
extra effort. Unless my first comment about adding some data to the
scope calculation resonates with you, I think your patch is as safe to
do as anything else, and you can check it in.
Thanks!
Eric
|