Thread: [Pydev-code] Extend rename-refactoring to properly cover all 'uses' of a method
Brought to you by:
fabioz
From: Andreas P. <an...@fr...> - 2015-03-23 16:32:20
Attachments:
projects.zip
folder1.zip
|
Hi, I recently found that the renaming of functions with PyDev does not quite work as one would expect. Given the following setup there are two issues I see happening with 3.9.2: - 2 Projects shared and user - None of the two projects has its project directory set as a pydev source folder - The shared project has a linked folder that is set as a source folder - the linked folder has a single python module with a single function - The user project has two subfolders each is set up as a pydev source folder in that project - The user project has the shared project in its project references - Each of the subfolders has a test.py with a function and each imports and uses the function from the shared project's linked folder Now when renaming the shared function inside either of the test.py files it is being renamed in that file and in the module in the shared project, but it is not being renamed in the other test.py in the user project. If the renaming is initiated in the shared functions definition module in the shared project none of the references in the user project are adapted. I'm attaching two zip's one containing the two project the other one containing the linked folder. Since I saw several renaming tickets being in the backlog I wanted to see if I could find the culprit myself but couldn't. So I'd appreciate any pointers as to where to look for the part that gathers all 'places' that need to be adjusted when when renaming a function. Andreas -- Andreas Pakulat sq...@fr... froglogic GmbH - Automated UI and Web Testing |
From: Fabio Z. <fa...@gm...> - 2015-03-23 16:47:46
|
Hi Andreas, I recently changed some things in that area... (so, make sure you have the latest development branch before starting to check it there to avoid conflicts) the place which checks the modules we want to look for tokens is: com.python.pydev.analysis.additionalinfo.AbstractAdditionalDependencyInfo.getModulesWithToken(IProject, String, IProgressMonitor), so, that'd be the first thing to check (if the modules are being found there in that situation). Best Regards, Fabio On Mon, Mar 23, 2015 at 1:32 PM, Andreas Pakulat <an...@fr...> wrote: > Hi, > > I recently found that the renaming of functions with PyDev does not quite > work as one would expect. Given the following setup there are two issues I > see happening with 3.9.2: > > - 2 Projects shared and user > - None of the two projects has its project directory set as a pydev source > folder > - The shared project has a linked folder that is set as a source folder > - the linked folder has a single python module with a single function > - The user project has two subfolders each is set up as a pydev source > folder in that project > - The user project has the shared project in its project references > - Each of the subfolders has a test.py with a function and each imports > and uses the function from the shared project's linked folder > > Now when renaming the shared function inside either of the test.py files > it is being renamed in that file and in the module in the shared project, > but it is not being renamed in the other test.py in the user project. > > If the renaming is initiated in the shared functions definition module in > the shared project none of the references in the user project are adapted. > > I'm attaching two zip's one containing the two project the other one > containing the linked folder. > > Since I saw several renaming tickets being in the backlog I wanted to see > if I could find the culprit myself but couldn't. So I'd appreciate any > pointers as to where to look for the part that gathers all 'places' that > need to be adjusted when when renaming a function. > > Andreas > > -- > Andreas Pakulat sq...@fr... > froglogic GmbH - Automated UI and Web Testing > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for > all > things parallel software development, from weekly thought leadership blogs > to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > pydev-code mailing list > pyd...@li... > https://lists.sourceforge.net/lists/listinfo/pydev-code > > |
From: Andreas P. <an...@fr...> - 2015-03-24 17:03:04
|
Hi Fabio, thanks for the hint. I think the issue is that in my example both source folders use the same module name, that is my project referencing the shared module looks like this: user (not a source folder) - test1 (source folder) | - test.py | - test2 (source folder) - test.py And getModulesWithToken builds a treemap based on the module names. For both files in my example the module name is 'test' and hence the second one is ignored when considering which modules contain the function that is about to be renamed. A quick try changing ModulesKey.compareTo to consider the path if the name is the same helps, but it seems the purpose of the class is to compare only based on the module name so I'm not sure what a proper fix for that situation would look like. This is with development just updated 2 hours ago. I've now also tried a scenario thats closer to my real-world usecase, that has just 1 source folder in the user project and inside that source folder there are two subfolders test1 and test2 each with a test.py. In this case the reference in neither of the two test modules is being renamed, I guess the fact that the two folders have no __init__.py declaring them packages makes PyDev ignore the modules inside? The layout for this case is like: user (not a source folder) - base (source folder) - test1 (not a source folder) | - test.py | - test2 (not a source folder) - test.py I understand that neither may be a typical setup for PyDev projects but it would be great if I could make at least the last one work - possibly by allowing some extension mechanism to help determining that case and providing the modules that need to be considered. Andreas On 2015-03-23 17:47, Fabio Zadrozny wrote: > Hi Andreas, > > I recently changed some things in that area... (so, make sure you have the > latest development branch before starting to check it there to > avoid conflicts) the place which checks the modules we want to look for > tokens is: > > com.python.pydev.analysis.additionalinfo.AbstractAdditionalDependencyInfo.getModulesWithToken(IProject, > String, IProgressMonitor), so, that'd > be the first thing to check (if the modules are being found there in that > situation). > > Best Regards, > > Fabio > > On Mon, Mar 23, 2015 at 1:32 PM, Andreas Pakulat <an...@fr...> > wrote: > >> Hi, >> >> I recently found that the renaming of functions with PyDev does not quite >> work as one would expect. Given the following setup there are two >> issues I see happening with 3.9.2: >> >> - 2 Projects shared and user >> - None of the two projects has its project directory set as a pydev source >> folder >> - The shared project has a linked folder that is set as a source folder >> - the linked folder has a single python module with a single function >> - The user project has two subfolders each is set up as a pydev source >> folder in that project >> - The user project has the shared project in its project references >> - Each of the subfolders has a test.py with a function and each imports and >> uses the function from the shared project's linked folder >> >> Now when renaming the shared function inside either of the test.py files it >> is being renamed in that file and in the module in the shared >> project, but it is not being renamed in the other test.py in the user >> project. >> >> If the renaming is initiated in the shared functions definition module in >> the shared project none of the references in the user project are >> adapted. >> >> I'm attaching two zip's one containing the two project the other one >> containing the linked folder. >> >> Since I saw several renaming tickets being in the backlog I wanted to see >> if I could find the culprit myself but couldn't. So I'd >> appreciate any pointers as to where to look for the part that gathers all >> 'places' that need to be adjusted when when renaming a function. >> >> Andreas >> >> -- >> Andreas Pakulat sq...@fr... >> froglogic GmbH - Automated UI and Web Testing >> ------------------------------------------------------------------------------ >> Dive into the World of Parallel Programming The Go Parallel Website, >> sponsored >> by Intel and developed in partnership with Slashdot Media, is your hub for >> all >> things parallel software development, from weekly thought leadership blogs >> to >> news, videos, case studies, tutorials and more. Take a look and join the >> conversation now. http://goparallel.sourceforge.net/ [1] >> _______________________________________________ >> pydev-code mailing list >> pyd...@li... >> https://lists.sourceforge.net/lists/listinfo/pydev-code [2] > > > > Links: > ------ > [1] http://goparallel.sourceforge.net/ > [2] https://lists.sourceforge.net/lists/listinfo/pydev-code > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for > all > things parallel software development, from weekly thought leadership blogs > to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > pydev-code mailing list > pyd...@li... > https://lists.sourceforge.net/lists/listinfo/pydev-code -- Andreas Pakulat sq...@fr... froglogic GmbH - Automated UI and Web Testing |
From: Fabio Z. <fa...@gm...> - 2015-03-24 17:49:25
|
Hi Andreas, You're right in both cases: 1. Duplicate module names under the same project are not supported -- I do have plans to improve on that to support namespace-based packages (I actually dislike those and find them a hack in Python, but PyDev should probably support that -- the issue here is that the indexing should be per source folder and not only per-project), but there's no hard date on that. 2. Folders which don't have __init__.py files are really invisible to PyDev and that's what Python expects -- that'd usually be an error in the project structure, so, PyDev itself should probably not support it... although I'm Ok if you want to add that through some extension. Best Regards, Fabio On Tue, Mar 24, 2015 at 2:02 PM, Andreas Pakulat <an...@fr...> wrote: > Hi Fabio, > > thanks for the hint. I think the issue is that in my example both source > folders use the same module name, that is my project referencing the shared > module looks like this: > > user (not a source folder) > - test1 (source folder) > | - test.py > | > - test2 (source folder) > - test.py > > And getModulesWithToken builds a treemap based on the module names. For > both files in my example the module name is 'test' and hence the second one > is ignored when considering which modules contain the function that is > about to be renamed. > > A quick try changing ModulesKey.compareTo to consider the path if the name > is the same helps, but it seems the purpose of the class is to compare only > based on the module name so I'm not sure what a proper fix for that > situation would look like. > > This is with development just updated 2 hours ago. > > I've now also tried a scenario thats closer to my real-world usecase, that > has just 1 source folder in the user project and inside that source folder > there are two subfolders test1 and test2 each with a test.py. In this case > the reference in neither of the two test modules is being renamed, I guess > the fact that the two folders have no __init__.py declaring them packages > makes PyDev ignore the modules inside? > > The layout for this case is like: > > user (not a source folder) > - base (source folder) > - test1 (not a source folder) > | - test.py > | > - test2 (not a source folder) > - test.py > > I understand that neither may be a typical setup for PyDev projects but it > would be great if I could make at least the last one work - possibly by > allowing some extension mechanism to help determining that case and > providing the modules that need to be considered. > > Andreas > > > On 2015-03-23 17:47, Fabio Zadrozny wrote: > >> Hi Andreas, >> >> I recently changed some things in that area... (so, make sure you have >> the latest development branch before starting to check it there to >> avoid conflicts) the place which checks the modules we want to look for >> tokens is: >> >> com.python.pydev.analysis.additionalinfo.AbstractAdditionalDependencyIn >> fo.getModulesWithToken(IProject, String, IProgressMonitor), so, that'd >> be the first thing to check (if the modules are being found there in that >> situation). >> >> Best Regards, >> >> Fabio >> >> On Mon, Mar 23, 2015 at 1:32 PM, Andreas Pakulat <an...@fr...> >> wrote: >> >> Hi, >>> >>> I recently found that the renaming of functions with PyDev does not >>> quite work as one would expect. Given the following setup there are two >>> issues I see happening with 3.9.2: >>> >>> - 2 Projects shared and user >>> - None of the two projects has its project directory set as a pydev >>> source folder >>> - The shared project has a linked folder that is set as a source folder >>> - the linked folder has a single python module with a single function >>> - The user project has two subfolders each is set up as a pydev source >>> folder in that project >>> - The user project has the shared project in its project references >>> - Each of the subfolders has a test.py with a function and each imports >>> and uses the function from the shared project's linked folder >>> >>> Now when renaming the shared function inside either of the test.py files >>> it is being renamed in that file and in the module in the shared >>> project, but it is not being renamed in the other test.py in the user >>> project. >>> >>> If the renaming is initiated in the shared functions definition module >>> in the shared project none of the references in the user project are >>> adapted. >>> >>> I'm attaching two zip's one containing the two project the other one >>> containing the linked folder. >>> >>> Since I saw several renaming tickets being in the backlog I wanted to >>> see if I could find the culprit myself but couldn't. So I'd >>> appreciate any pointers as to where to look for the part that gathers >>> all 'places' that need to be adjusted when when renaming a function. >>> >>> Andreas >>> >>> -- >>> Andreas Pakulat sq...@fr... >>> froglogic GmbH - Automated UI and Web Testing >>> ------------------------------------------------------------ >>> ------------------ >>> Dive into the World of Parallel Programming The Go Parallel Website, >>> sponsored >>> by Intel and developed in partnership with Slashdot Media, is your hub >>> for all >>> things parallel software development, from weekly thought leadership >>> blogs to >>> news, videos, case studies, tutorials and more. Take a look and join the >>> conversation now. http://goparallel.sourceforge.net/ [1] >>> _______________________________________________ >>> pydev-code mailing list >>> pyd...@li... >>> https://lists.sourceforge.net/lists/listinfo/pydev-code [2] >>> >> >> >> >> Links: >> ------ >> [1] http://goparallel.sourceforge.net/ >> [2] https://lists.sourceforge.net/lists/listinfo/pydev-code >> >> ------------------------------------------------------------ >> ------------------ >> Dive into the World of Parallel Programming The Go Parallel Website, >> sponsored >> by Intel and developed in partnership with Slashdot Media, is your hub >> for all >> things parallel software development, from weekly thought leadership >> blogs to >> news, videos, case studies, tutorials and more. Take a look and join the >> conversation now. http://goparallel.sourceforge.net/ >> _______________________________________________ >> pydev-code mailing list >> pyd...@li... >> https://lists.sourceforge.net/lists/listinfo/pydev-code >> > > -- > Andreas Pakulat sq...@fr... > froglogic GmbH - Automated UI and Web Testing > |
From: Andreas P. <an...@fr...> - 2015-03-24 19:14:11
|
On 2015-03-24 18:48, Fabio Zadrozny wrote: > Hi Andreas, > > You're right in both cases: > > 1. Duplicate module names under the same project are not supported -- I do > have plans to improve on that to support namespace-based packages > (I actually dislike those and find them a hack in Python, but PyDev should > probably support that -- the issue here is that the indexing should > be per source folder and not only per-project), but there's no hard date on > that. I assume thats a more extensive change that a 'newcomer' like me is unlikely to be able to resolve? > 2. Folders which don't have __init__.py files are really invisible to PyDev > and that's what Python expects -- that'd usually be an error in > the project structure, so, PyDev itself should probably not support it... > although I'm Ok if you want to add that through some extension. I see, in our case the subfolders are actually testcase directories and each testcase has a 'driving' test.py with a main function that is being executed. Most users have the test.py filled with shared function calls and hence renaming one of them in only one of the tests is a small disaster :) If 1. would be solved I could adapt the setup so that each testcase folder is setup up separately as a source folder which should make an extension unneeded if I understand things correctly. That would also likely better reflect the runtime python path. Since the lookup is done per-project already that should also cover the shared folder within each project that we have (to share between testcases but not 'globally'). If you have any suggestions on where an extension for 2. would be best be inserted I'd be happy to hear them. Otherwise do you see any general problem with adjusting ModuleKey to consider name + file for compareTo/equal/hashCode - other than potential speed issues? We could carry as a custom patch for the time being. Andreas -- Andreas Pakulat sq...@fr... froglogic GmbH - Automated UI and Web Testing |
From: Fabio Z. <fa...@gm...> - 2015-03-24 19:37:57
|
On Tue, Mar 24, 2015 at 4:13 PM, Andreas Pakulat <an...@fr...> wrote: > On 2015-03-24 18:48, Fabio Zadrozny wrote: > >> Hi Andreas, >> >> You're right in both cases: >> >> 1. Duplicate module names under the same project are not supported -- I >> do have plans to improve on that to support namespace-based packages >> (I actually dislike those and find them a hack in Python, but PyDev >> should probably support that -- the issue here is that the indexing should >> be per source folder and not only per-project), but there's no hard date >> on that. >> > > I assume thats a more extensive change that a 'newcomer' like me is > unlikely to be able to resolve? > > 2. Folders which don't have __init__.py files are really invisible to >> PyDev and that's what Python expects -- that'd usually be an error in >> the project structure, so, PyDev itself should probably not support it... >> although I'm Ok if you want to add that through some extension. >> > > I see, in our case the subfolders are actually testcase directories and > each testcase has a 'driving' test.py with a main function that is being > executed. Most users have the test.py filled with shared function calls and > hence renaming one of them in only one of the tests is a small disaster :) > > If 1. would be solved I could adapt the setup so that each testcase folder > is setup up separately as a source folder which should make an extension > unneeded if I understand things correctly. That would also likely better > reflect the runtime python path. > > Since the lookup is done per-project already that should also cover the > shared folder within each project that we have (to share between testcases > but not 'globally'). > > If you have any suggestions on where an extension for 2. would be best be > inserted I'd be happy to hear them. Otherwise do you see any general > problem with adjusting ModuleKey to consider name + file for > compareTo/equal/hashCode - other than potential speed issues? We could > carry as a custom patch for the time being. > > I think changing ModuleKey may end up breaking other things (and the speed issues are likely to happen too on big projects), so, I'd recommend against it (and I can't really think of a better solution that's not the proper fix -- which would be fixing item 1)... As for item 1, I think the effort to make it work properly will be substantial and will really require many changes, (which is why I haven't tackled that so far...), so, I think it may be really harder to resolve on your side... Best Regards, Fabio |