Well, I went and changed the grid frame to be just a regular frame. That means you can keep a data display up and then also load a new set of data and bring it up at the same time. Now, having said that.... The program doesn't keep a seperate page of data for each view. So the single copy of data will be overwritten when you load a new copy, but your grid won't change. That means if you edit the data in one grid, it will re-insert that data into the main copy of the data. Before it wouldn't let you load new data by making a modal frame that prevented it. I would only load two data sets at the same time for some kind of visual comparison or whatever. Like you could line up two columns of data from different sets side by side and look at them. Beats me... use your imagination. I used it to look at two database tables at the same time and a csv file I had loaded from the disk drive.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The config dialog had some major funkyness to it. Now it detects if you type in a valid delimiter in the "other" box and also if you type in a valid header row. It will check the checkboxes as long as the input isn't garbage. AAAAAND if you have made changes to your file, it won't import new data unless you say it's OK. Remember, the config doesn't actually DO anything except control the view of the data. you can go and configure whatever you want there are no changes to your data. The dialog always makes changes when you click, even if you hit cancel. I probably want to change that hah hah.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
alright, now that I have most likely fixed all the strange coding problems lurking in there, I will make an interface change. So when you view the data, it will select the rows and columns that you have in the lists of selected rows/cols. I was thinking of making the rows/cols a different color instead of selecting them, but it's probably more logical to select them so the user can deselect if they want.
Selected rows/cols are filtered out when you select row/col filter. Editing your csv file will pad all the rows out to fit in the grid, so when it's saved, it will contain additional blank cells. It only pads if you are editing using the grid, not the main control window.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
well I commited the interface change. Now when you use FIND menu on the grid, you can pick row matches or column matches and it will select the previous matches you made if they are on the grid. If you pick accept, those new selections are applied, but if you click cancel it doesn't change them. this means:
if you view the data and select a row/col then accept, if you view the data again you can pick row/col matches and it will select the ones you had before. If they exist in the display. I used a dictionary to look up the actual row number stored in the labels. I will probably have to work on the column display code next. I should be able to get rid of all the demons in my code there.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
yes, been working on the column demons. I made another dictionary for the labels! Things are definitely coming along with recalling the previously selected rows/column so you can select more. All I think I have left to close the cycle is to update the dictionary when the headers/row labels update as well. Just reading and testing the code now.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
there's some annoying busywork the program is doing with the local copy of the data. I think it's time to expunge the strange fiddling with the local data copy and copying it onto the global one. That's like...multiple errors right there. Good question as to why I had changed course like that in the code. That's some kind of mess! I don't know if the data is ever referenced for display, or only searches. I'll clean that up. I think I had moved all the data manipulation code into the find window. That's not where it belongs, silly rabbit. It has to execute on the main data array in the control window. I guess all this code worked as long as you never viewed a subset of the data in question. hah hah mmmm lunacy. Things are coming together though. I will finally produce a finished version lol that works properly. It seems most of the csv EDITING code that I had added later was producing cpu busywork for no real reason. the viewing part worked perfectly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
after boring down into the code, there are several poorly implemented indexing schemes I was using. It's tied in with the local data fiddling code. So what I'm going to do, is focus on the initial implementation of the indexes. It just used a list of ints that linked back to the main data rows/cols. That will allow me to clean up all the time wasting code as well. It'll solve most of the problems I had implemented for data editing... I'm pretty sure. Because then it'lll be editing the rows in the data that was loaded not constructing a new set of data from the data in the grid and setting the main data to that...ugh! I think I had forgotten about the indexes. some kind of rush to get it all done in a day hah hah!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
wow, now we're cooking with gas! The reflecting I did on the poorly coded stuff has resulted in major improvements. Multiple data operations have been cancelled in the column delete routine. Proper row and column index is used for the grid, making a quick index lookup possible.
It looks like I was patching things onto the side of my code, and ignoring my original implementation. I blame the fact it was my first and original attempt to edit csv files and learn python at the same time. All the ideas were there for a good app, but I failed to find cohesion. Perhaps all the time spent step by step debugging other peoples major projects has helped me to visualize code execution. Soon all the cringeworthy code will be gone.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
well as soon as I can I'll push the changes to git, For some reason I can't push to sourceforge. Basically the index code is complete, so now I should be able to view rows not in sequence and still edit the main data in the proper location. No more building new data out of the grid, which is a serious problem. I should be able to put the interface limitations to rest. Also, I am finally reformatting the project so it uses tabs instead of a crazy mixture of tabs and spaces. Not sure how that happened, but I was editing it in windows and linux and a tab was 4 spaces instead of being a tab with adjustable width.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
up next I'll work on the cell editing. The things I have left to fix are: cell editing, column and row insert/append. Not sure if I should keep the sorting and stuff. Is it even of any use? And I have to test the data finding and display. Also, I will add an option to show all the selected rows/cols in a grid.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I know the last update seemed like lame busywork, but it's actually in prep to edit the main code for editing the cells. Hard to make changes to the code when it was all a mashup of tabs and spaces. Oh and I realized the code for loading the file was duplicated. Probably didn't know how to make/call python methods before.
Last edit: Mat L 2017-02-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
cell editing is working properly, using the indexes to tell the control window it needs to change value x,y to new value. Also, I broke out the exportdsv and importdsv. so I could erase many lines of junk code and replace them with good code. The most important parts are "complete" but of course I need to make sure nothing is referencing a stale copy of local data in the grid. I will probably make a new member to refresh the grid from the parent data. I think most of the globals and passing copies of data can be eliminated. By most I mean all.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
today, I think I will complete the code that adds a new row and column. That should finish off all the important functions/methods. Then I'll do some extensive testing! After that, I'll add in the methods and dialogs that I've really wanted to. Things it should have been able to do before, but never could because all my plans went off the rails. I'm still wondering how I lost the idea I had started with and then filled it all in with junk. I suppose because I was using the program to edit data and it just had to do it's thing, which was erasing rows and columns. I tacked on the data editing later hah hah. That's where it all went haywire. The online class on c# programming probably helped me structure my python code better and move the methods into a sane place. Still wondering how I was able to really read and understand the code now. Strange how a decade ago it seemed confusing and complex, but now it just reads like a novel. A few things changed I guess! Skills building and also I ditched that old medication from the 90s. I'm sure that had some kind of effect on my mind...like brain damage from dangerously low glucose levels. lol enough self-analysis and onto code analysis.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
instead I worked on the find data. and the display matches. I still have to put the code in to display the found data in the headers. So there's my warning. I will add the code to insert new column/row as soon as I can. It was always lame place-holder code that appended a new row/col to a local copy of the fiddled data. Almost all data fiddling is gone. Code sanity is being restored. Won't be long before it's all complete! exciting!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
display matches is working superb! It selects the grid cell that the matches when you scroll through them. I think originally it just made the cell visible. The dictionary I use to look up the data matches is going to be used in the parent as well. Then you can call up the matches again instead of searching again to see them. By the Way the program is using python/perl regex for searching. So it's more than a simple string search, and I removed the "case" checkbox. I mean...it's using regex, so how does using a case checkbox even make sense? Also, it's probably a good idea to ged rid of the data preview grid when you drop columns. What was that even there for? probably made that thing before I had the grid editing, and now it's useless.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ok I think I've finished off the find data and find in the grid. They run on the same code, one method calls the first and returns the matches. Now it remembers the matches you had in the previous viewing of the data and shows them if they are in the current view. This can lead to other functions like viewing only the previous matches instead of having to click on find data again. I am going to go back over the data fiddling code I have there in filter column. It's definitely due for an overhaul to use DSV.exportDSV.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
oh by the by, I'm also going to use this opportunity to start coding those globals out of the program. All globals have to go. I think I used the globals because it was easy. Makes for some really strange code though.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
deleted all globals. That's been on my wishlist for as long as I can remember. Instead now it uses a local copy of the found rows/cols that is a copy of it's parents. Phew going to give the coding a break for a bit and chill out. Then I can get to work on row/col insert. I'll keep testing the find data as I go. It seems really solid. Code sanity is on the rise. Oh, forgot about the column data fiddling lol
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
wow! what a run of code deleting! I deleted code wholesale, as there was code that deleted ranges of rows and code that filtered a list of rows. I consolidated that into the code that deletes a list of rows. Deleted a bunch of fiddling code. I think the only fiddling left is in the column viewing. Then I can move onto the find grid and de-fiddle that, the rest of the way. oh happy days! The program is really good at detecting when you deleted all the data. It replaces the data with "no data" and puts "no data" in the control. During the column delete it also scans the rows to see if they are empty. If they are, then it deletes the row as well. When it reaches 0 rows... it says no data. With the fixes to the gui and the main code, it'll be amazing! The code fixes provide way more flexibility as well as better operation.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
good news! I worked on the code to insert a new row. It creates a new row with delimiters and inserts that into the data. I will split the popup into insert row and append row, so you can rmb on the last row and select append row, to add one on the end or after any other row. Once I give the same treatment to columns, the program will be all complete on the csv editing end. After that... all there is left to do, is check over the other popup menu options and see what they are doing and what they need to complete their task or, get rid of them if the task is not productive. Phew... then it's off to testing with random data. See what it takes to make it break. I promise I tested this code and really put some thought into how to prevent errors from happening, instead of glossing the crud over with try except.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
inserting a new column is now reality! excitement! So now editing delimited data is complete, you can insert new rows and columns then edit the data and save it. I bet you're wondering...or not... why it took so little time to complete that part? Well I used the code copied from changing a value, and made it insert the column. Now, all I have to do is append row/col. Also, I will make sure the data is padded out properly for edge case scenarios. Like if you edit the data in the control window and make one line an element longer than the other, it never pads the lines to all the same length. Not good. I'll flush out all the little minor demons that are left. hooray! only minor demons! my entire wishlist of fixes is complete. All hastily constructed code lines have been erased. amazeballs.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
fixed the edge case of more data in rows than headers. Had some trickle down fixes in there as well. Editing the dsv file is a breeze. It even produces the right format when you save. Should be code complete in the next week. Definitely useable now!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
blah there are some unimportant ui bugs it reports an extra/less column/row exists because I forgot to update that variable. You can't view rows/cols that don't exist though, so it doesn't cause any real problem.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
now you can append a column. so it makes a new col next to the column you RMB click on, as indicated in the menu. next up append row. I will remove the column sorting, not sure I ever used it... But it can be added back in later, I guess. It's just garbage code, so when it's removed, things will be gasp complete. Except the minor bug testing ;)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
more amazing progress hah hah. I have the row append going and I deleted the sorting selection on the popup. I've tested it quite a few times, but I'm sure there aRE Some bugs I haven't found. The row and column properties are going to be "fixed" to use the proper data when checking values. They "work" but results can't be counted on. I call it pre-editing code. ohhh forgot about erasing the match results. I'll get to that!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, I went and changed the grid frame to be just a regular frame. That means you can keep a data display up and then also load a new set of data and bring it up at the same time. Now, having said that.... The program doesn't keep a seperate page of data for each view. So the single copy of data will be overwritten when you load a new copy, but your grid won't change. That means if you edit the data in one grid, it will re-insert that data into the main copy of the data. Before it wouldn't let you load new data by making a modal frame that prevented it. I would only load two data sets at the same time for some kind of visual comparison or whatever. Like you could line up two columns of data from different sets side by side and look at them. Beats me... use your imagination. I used it to look at two database tables at the same time and a csv file I had loaded from the disk drive.
The config dialog had some major funkyness to it. Now it detects if you type in a valid delimiter in the "other" box and also if you type in a valid header row. It will check the checkboxes as long as the input isn't garbage. AAAAAND if you have made changes to your file, it won't import new data unless you say it's OK. Remember, the config doesn't actually DO anything except control the view of the data. you can go and configure whatever you want there are no changes to your data. The dialog always makes changes when you click, even if you hit cancel. I probably want to change that hah hah.
alright, now that I have most likely fixed all the strange coding problems lurking in there, I will make an interface change. So when you view the data, it will select the rows and columns that you have in the lists of selected rows/cols. I was thinking of making the rows/cols a different color instead of selecting them, but it's probably more logical to select them so the user can deselect if they want.
Selected rows/cols are filtered out when you select row/col filter. Editing your csv file will pad all the rows out to fit in the grid, so when it's saved, it will contain additional blank cells. It only pads if you are editing using the grid, not the main control window.
well I commited the interface change. Now when you use FIND menu on the grid, you can pick row matches or column matches and it will select the previous matches you made if they are on the grid. If you pick accept, those new selections are applied, but if you click cancel it doesn't change them. this means:
if you view the data and select a row/col then accept, if you view the data again you can pick row/col matches and it will select the ones you had before. If they exist in the display. I used a dictionary to look up the actual row number stored in the labels. I will probably have to work on the column display code next. I should be able to get rid of all the demons in my code there.
yes, been working on the column demons. I made another dictionary for the labels! Things are definitely coming along with recalling the previously selected rows/column so you can select more. All I think I have left to close the cycle is to update the dictionary when the headers/row labels update as well. Just reading and testing the code now.
there's some annoying busywork the program is doing with the local copy of the data. I think it's time to expunge the strange fiddling with the local data copy and copying it onto the global one. That's like...multiple errors right there. Good question as to why I had changed course like that in the code. That's some kind of mess! I don't know if the data is ever referenced for display, or only searches. I'll clean that up. I think I had moved all the data manipulation code into the find window. That's not where it belongs, silly rabbit. It has to execute on the main data array in the control window. I guess all this code worked as long as you never viewed a subset of the data in question. hah hah mmmm lunacy. Things are coming together though. I will finally produce a finished version lol that works properly. It seems most of the csv EDITING code that I had added later was producing cpu busywork for no real reason. the viewing part worked perfectly.
after boring down into the code, there are several poorly implemented indexing schemes I was using. It's tied in with the local data fiddling code. So what I'm going to do, is focus on the initial implementation of the indexes. It just used a list of ints that linked back to the main data rows/cols. That will allow me to clean up all the time wasting code as well. It'll solve most of the problems I had implemented for data editing... I'm pretty sure. Because then it'lll be editing the rows in the data that was loaded not constructing a new set of data from the data in the grid and setting the main data to that...ugh! I think I had forgotten about the indexes. some kind of rush to get it all done in a day hah hah!
wow, now we're cooking with gas! The reflecting I did on the poorly coded stuff has resulted in major improvements. Multiple data operations have been cancelled in the column delete routine. Proper row and column index is used for the grid, making a quick index lookup possible.
It looks like I was patching things onto the side of my code, and ignoring my original implementation. I blame the fact it was my first and original attempt to edit csv files and learn python at the same time. All the ideas were there for a good app, but I failed to find cohesion. Perhaps all the time spent step by step debugging other peoples major projects has helped me to visualize code execution. Soon all the cringeworthy code will be gone.
well as soon as I can I'll push the changes to git, For some reason I can't push to sourceforge. Basically the index code is complete, so now I should be able to view rows not in sequence and still edit the main data in the proper location. No more building new data out of the grid, which is a serious problem. I should be able to put the interface limitations to rest. Also, I am finally reformatting the project so it uses tabs instead of a crazy mixture of tabs and spaces. Not sure how that happened, but I was editing it in windows and linux and a tab was 4 spaces instead of being a tab with adjustable width.
up next I'll work on the cell editing. The things I have left to fix are: cell editing, column and row insert/append. Not sure if I should keep the sorting and stuff. Is it even of any use? And I have to test the data finding and display. Also, I will add an option to show all the selected rows/cols in a grid.
I know the last update seemed like lame busywork, but it's actually in prep to edit the main code for editing the cells. Hard to make changes to the code when it was all a mashup of tabs and spaces. Oh and I realized the code for loading the file was duplicated. Probably didn't know how to make/call python methods before.
Last edit: Mat L 2017-02-21
cell editing is working properly, using the indexes to tell the control window it needs to change value x,y to new value. Also, I broke out the exportdsv and importdsv. so I could erase many lines of junk code and replace them with good code. The most important parts are "complete" but of course I need to make sure nothing is referencing a stale copy of local data in the grid. I will probably make a new member to refresh the grid from the parent data. I think most of the globals and passing copies of data can be eliminated. By most I mean all.
today, I think I will complete the code that adds a new row and column. That should finish off all the important functions/methods. Then I'll do some extensive testing! After that, I'll add in the methods and dialogs that I've really wanted to. Things it should have been able to do before, but never could because all my plans went off the rails. I'm still wondering how I lost the idea I had started with and then filled it all in with junk. I suppose because I was using the program to edit data and it just had to do it's thing, which was erasing rows and columns. I tacked on the data editing later hah hah. That's where it all went haywire. The online class on c# programming probably helped me structure my python code better and move the methods into a sane place. Still wondering how I was able to really read and understand the code now. Strange how a decade ago it seemed confusing and complex, but now it just reads like a novel. A few things changed I guess! Skills building and also I ditched that old medication from the 90s. I'm sure that had some kind of effect on my mind...like brain damage from dangerously low glucose levels. lol enough self-analysis and onto code analysis.
instead I worked on the find data. and the display matches. I still have to put the code in to display the found data in the headers. So there's my warning. I will add the code to insert new column/row as soon as I can. It was always lame place-holder code that appended a new row/col to a local copy of the fiddled data. Almost all data fiddling is gone. Code sanity is being restored. Won't be long before it's all complete! exciting!
display matches is working superb! It selects the grid cell that the matches when you scroll through them. I think originally it just made the cell visible. The dictionary I use to look up the data matches is going to be used in the parent as well. Then you can call up the matches again instead of searching again to see them. By the Way the program is using python/perl regex for searching. So it's more than a simple string search, and I removed the "case" checkbox. I mean...it's using regex, so how does using a case checkbox even make sense? Also, it's probably a good idea to ged rid of the data preview grid when you drop columns. What was that even there for? probably made that thing before I had the grid editing, and now it's useless.
ok I think I've finished off the find data and find in the grid. They run on the same code, one method calls the first and returns the matches. Now it remembers the matches you had in the previous viewing of the data and shows them if they are in the current view. This can lead to other functions like viewing only the previous matches instead of having to click on find data again. I am going to go back over the data fiddling code I have there in filter column. It's definitely due for an overhaul to use DSV.exportDSV.
oh by the by, I'm also going to use this opportunity to start coding those globals out of the program. All globals have to go. I think I used the globals because it was easy. Makes for some really strange code though.
deleted all globals. That's been on my wishlist for as long as I can remember. Instead now it uses a local copy of the found rows/cols that is a copy of it's parents. Phew going to give the coding a break for a bit and chill out. Then I can get to work on row/col insert. I'll keep testing the find data as I go. It seems really solid. Code sanity is on the rise. Oh, forgot about the column data fiddling lol
wow! what a run of code deleting! I deleted code wholesale, as there was code that deleted ranges of rows and code that filtered a list of rows. I consolidated that into the code that deletes a list of rows. Deleted a bunch of fiddling code. I think the only fiddling left is in the column viewing. Then I can move onto the find grid and de-fiddle that, the rest of the way. oh happy days! The program is really good at detecting when you deleted all the data. It replaces the data with "no data" and puts "no data" in the control. During the column delete it also scans the rows to see if they are empty. If they are, then it deletes the row as well. When it reaches 0 rows... it says no data. With the fixes to the gui and the main code, it'll be amazing! The code fixes provide way more flexibility as well as better operation.
good news! I worked on the code to insert a new row. It creates a new row with delimiters and inserts that into the data. I will split the popup into insert row and append row, so you can rmb on the last row and select append row, to add one on the end or after any other row. Once I give the same treatment to columns, the program will be all complete on the csv editing end. After that... all there is left to do, is check over the other popup menu options and see what they are doing and what they need to complete their task or, get rid of them if the task is not productive. Phew... then it's off to testing with random data. See what it takes to make it break. I promise I tested this code and really put some thought into how to prevent errors from happening, instead of glossing the crud over with try except.
inserting a new column is now reality! excitement! So now editing delimited data is complete, you can insert new rows and columns then edit the data and save it. I bet you're wondering...or not... why it took so little time to complete that part? Well I used the code copied from changing a value, and made it insert the column. Now, all I have to do is append row/col. Also, I will make sure the data is padded out properly for edge case scenarios. Like if you edit the data in the control window and make one line an element longer than the other, it never pads the lines to all the same length. Not good. I'll flush out all the little minor demons that are left. hooray! only minor demons! my entire wishlist of fixes is complete. All hastily constructed code lines have been erased. amazeballs.
fixed the edge case of more data in rows than headers. Had some trickle down fixes in there as well. Editing the dsv file is a breeze. It even produces the right format when you save. Should be code complete in the next week. Definitely useable now!
blah there are some unimportant ui bugs it reports an extra/less column/row exists because I forgot to update that variable. You can't view rows/cols that don't exist though, so it doesn't cause any real problem.
now you can append a column. so it makes a new col next to the column you RMB click on, as indicated in the menu. next up append row. I will remove the column sorting, not sure I ever used it... But it can be added back in later, I guess. It's just garbage code, so when it's removed, things will be gasp complete. Except the minor bug testing ;)
more amazing progress hah hah. I have the row append going and I deleted the sorting selection on the popup. I've tested it quite a few times, but I'm sure there aRE Some bugs I haven't found. The row and column properties are going to be "fixed" to use the proper data when checking values. They "work" but results can't be counted on. I call it pre-editing code. ohhh forgot about erasing the match results. I'll get to that!