linkbat-devel Mailing List for Linux Knowledge Base and Tutorial (Page 3)
Brought to you by:
jimmo
You can subscribe to this list here.
2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(47) |
Dec
(4) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2003 |
Jan
(9) |
Feb
(3) |
Mar
|
Apr
|
May
(8) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Shanta M. <sh...@fo...> - 2002-11-20 19:05:14
|
On Wed, 2002-11-20 at 12:01, James Mohr wrote: > > > > That is why I like to store my data in SQL. It allows me to manipulate > > the data as I see fit. This includes exporting data to cvs files. > > In principle I don't have a problem with that. However, it requires the > intermediate step to be SQL. If you have already inserted it into an SQL DB, > why bother exporting it to CSV? (unless you want to move it a server that has > no SQL DB) Although not necessarily efficient, I see a "friendlier" approach > to first parse the XML into CSV and then import that into SQL. > eXtropia puts things into the chosen DataSource. There is no need for the XML interface to do that. > > > > In my setup 0 change exempt one variable in the Global site setup. > > $DataSource = "file"; or $DataSource = "MySQL"; > > What about the code itself? If a single file = MySQL in terms of being a data > source, I do not see how you could get around the problem of having to redo A > LOT of code. If a file equaled a MySQL Table, then I see a 1:1 relationship > between fields in MySQL and fields in the file, so there would be very litle > code change necessary. Long ago I added the falowing code to my application files my @BASIC_DATASOURCE_CONFIG_PARAMS; if ($site eq "file"){ @BASIC_DATASOURCE_CONFIG_PARAMS = ( -TYPE => 'File', -FILE => "$APP_DATAFILES_DIRECTORY/$APP_NAME.dat", -FIELD_DELIMITER => '|', -COMMENT_PREFIX => '#', -CREATE_FILE_IF_NONE_EXISTS => 1, -FIELD_NAMES => \@DATASOURCE_FIELD_NAMES, -KEY_FIELDS => ['record_id'], -FIELD_TYPES => { record_id => 'Autoincrement', datetime => [ -TYPE => "Date", -STORAGE => 'y-m-d H:M:S', -DISPLAY => 'y-m-d H:M:S', ], }, ); } else{ @BASIC_DATASOURCE_CONFIG_PARAMS = ( -TYPE => 'DBI', -DBI_DSN => $DBI_DSN, -TABLE => 'linkbat_questions_tb', -USERNAME => $AUTH_MSQL_USER_NAME, -PASSWORD => $MySQLPW, -FIELD_NAMES => \@DATASOURCE_FIELD_NAMES, -KEY_FIELDS => ['username'], -FIELD_TYPES => { record_id => 'Autoincrement', datetime => [ -TYPE => "Date", -STORAGE => 'y-m-d H:M:S', -DISPLAY => 'y-m-d H:M:S', ], }, ); In the Global setup the variables are defined as they are used my all apps in the site only the table name needs to be changed in telling eXtropia how the data is stored. cvs file or SQL. > > > > > Validation, field > > > > contents, required fields and many other checking is already in > > > > eXtropia one just has to tell it what you which to check for. Don't > > > > know if this is what you are referring to with XML validation as I know > > > > little of XML. > > > > > > I'm refering to the actual XML files before they are imported into the > > > database. If someone forgets a closing tag or something else, we should > > > know about it before we try to read it into the database. > > > > Ok. but that is why one develops on a separate server. could be added to > > eXtropia but XML in the form View would likely be easier > > What about performance. An XML files is at least 10 times larger. Plus we have > the issues of the inter-relationships between the various KUs. To what extent > can this be handle by eXtropia when directly reading the XML files? > Not sure! As a Perl programmer I see XML as an more complex form of HTML. I would not chose to use XML to directly access the underlying data files but that is largely because I have not taken the time yet to look into XML as a programing language. In this context I was thinking more in the form of Javascripts ability to do checking in the browser. I don't even know if XML can do that. In eXtropia XML is placed in TTML files. This blends Perl into the "View" that gets exported into the browser. > Even if eXtropia could read the XML files directly and be able to present the > relationships correctly. I cannot see how that would **not** lock us into > eXtropia as the delivery mechanism. If in a CSV text file or SQL DB, then we > can do standard queries on the data. > In eXtropia the queries are done on ActionHandlers to separate the logic you are developing from the look and feel of the site. For the most part you just tell eXtropia what you want it make the quire to the DataSource so you don't have write new code to switch from SQL to CSV Here is an example $cgi->param( -NAME => 'raw_search', -VALUE => "project_code=='Requested'" ); There is a subroutine built into the system that forms a SQL quire and sends it to MySQL or parses the CSV File for the records that match your request. No rewriting of code to accommodate the change in DataSource. I you want to duplicate the mechanics of doing this you will take years to get it right. I don't have any desire to do that unless the default dose not do what I want then I will hack the code, Isolate it from the default code and call it as I need it. Selena, Gunther, and Peter are far better programmers than I am. Gunther worked for Lincoln Stein. They all program for banks using this application. > > ------------------------------------------------------- > This sf.net email is sponsored by: > Battle your brains against the best in the Thawte Crypto > Challenge. Be the first to crack the code - register now: > http://www.gothawte.com/rd521.html > _______________________________________________ > Linkbat-devel mailing list > Lin...@li... > https://lists.sourceforge.net/lists/listinfo/linkbat-devel -- Shanta McBain <sh...@fo...> |
From: James M. <lin...@ji...> - 2002-11-20 19:02:19
|
On Wednesday 20 November 2002 19:33, Shanta McBain wrote: > Hi All > > I have created the Questions table here is the details contained in the > Client news link on the left side bar this is only visible to > Linkbat_admin. I also have not converted this to the Linkbat View yet so > you will not get a left side bar or style sheets. (this is written with > View from the web) I have three links labled "Client news". None of them provide any questions. (At least nothing I have found.) > > Added Questions table > > Please note new addition in admin add table on the left side bar. This > link is to the table of questions. see details. > > The Question number no longer match the original table as the first > field in the MySQL table was set to auto incriminate so the table was > populated with a different set of numbers than the csv file. If this is > critical I can redo it. turning of auto inc then turning it on after > install. The actual question numbers (i.e. question IDs) are not important. What is important is the included page ID. This provides the link back to the page where the this question came from. > > Another note of importance is the category field . The drop list of > entries are contained in a table and filled via a ActionHandler. This > allows admin to add new categories by placing them in the table. I am > currently updating this system and will add the changes soon. This will > consist of an add form in the admin add links and a ActionHandler tuned > to Linkbats needs. I am missing the point of a form to input questions. I understand that this is something that eXtropia can do, but unless this mechanism can insert the question into a KU in the XML file and make the necessary references I am missing the point. regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: Shanta M. <sh...@fo...> - 2002-11-20 18:33:38
|
Hi All I have created the Questions table here is the details contained in the Client news link on the left side bar this is only visible to Linkbat_admin. I also have not converted this to the Linkbat View yet so you will not get a left side bar or style sheets. (this is written with View from the web) Added Questions table Please note new addition in admin add table on the left side bar. This link is to the table of questions. see details. The Question number no longer match the original table as the first field in the MySQL table was set to auto incriminate so the table was populated with a different set of numbers than the csv file. If this is critical I can redo it. turning of auto inc then turning it on after install. Another note of importance is the category field . The drop list of entries are contained in a table and filled via a ActionHandler. This allows admin to add new categories by placing them in the table. I am currently updating this system and will add the changes soon. This will consist of an add form in the admin add links and a ActionHandler tuned to Linkbats needs. Have Fun. -- Shanta McBain <sh...@fo...> |
From: James M. <lin...@ji...> - 2002-11-20 18:32:40
|
On Wednesday 20 November 2002 18:33, Shanta McBain wrote: > On Wed, 2002-11-20 at 10:46, James Mohr wrote: > > That's exactly it: "when a host that dose not have access to SQL" and > > that's really the only reason. A corollary is someone who does not have > > the skill or desire to set up an SQL database, but that is less important > > to me than someone who cannot set it up. However, it will be a while > > before it is a "product" that we can package and make available on > > SourceForge, etc. I just don't want to paint ourselves into a corner. > > That is why I like to store my data in SQL. It allows me to manipulate > the data as I see fit. This includes exporting data to cvs files. In principle I don't have a problem with that. However, it requires the intermediate step to be SQL. If you have already inserted it into an SQL DB, why bother exporting it to CSV? (unless you want to move it a server that has no SQL DB) Although not necessarily efficient, I see a "friendlier" approach to first parse the XML into CSV and then import that into SQL. > > > I will have the CSV s files into MySQL in a few hours. I have code to > > > do this by simply importing the contents into the SQL Table.PHP but it > > > works. From that point on to switch the code from on DataSource to > > > another takes a simple change in one variable in the site setup file. > > > Export the contents of the SQL table to a | delimited file though and > > > delimiter can be used. and there you have switched. > > > > Does that mean there would be little change in the eXtropia code to > > reflect a CSV data source as opposed to SQL? > > In my setup 0 change exempt one variable in the Global site setup. > $DataSource = "file"; or $DataSource = "MySQL"; What about the code itself? If a single file = MySQL in terms of being a data source, I do not see how you could get around the problem of having to redo A LOT of code. If a file equaled a MySQL Table, then I see a 1:1 relationship between fields in MySQL and fields in the file, so there would be very litle code change necessary. > > > Validation, field > > > contents, required fields and many other checking is already in > > > eXtropia one just has to tell it what you which to check for. Don't > > > know if this is what you are referring to with XML validation as I know > > > little of XML. > > > > I'm refering to the actual XML files before they are imported into the > > database. If someone forgets a closing tag or something else, we should > > know about it before we try to read it into the database. > > Ok. but that is why one develops on a separate server. could be added to > eXtropia but XML in the form View would likely be easier What about performance. An XML files is at least 10 times larger. Plus we have the issues of the inter-relationships between the various KUs. To what extent can this be handle by eXtropia when directly reading the XML files? Even if eXtropia could read the XML files directly and be able to present the relationships correctly. I cannot see how that would **not** lock us into eXtropia as the delivery mechanism. If in a CSV text file or SQL DB, then we can do standard queries on the data. I think I'll draw a diagram of what I was thinking about it and post it. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: <sh...@fo...> - 2002-11-20 18:32:08
|
Add Event. ---------- subject Added Questions table abstract Please note new addition in admin add table on the left side bar. This link is to the table of questions. see details full_text The Question number no longer match the original table as the first field in the MySQL table was set to auto incrimate so the table was populated with a diffrent set of numbers than the csv file. Another note of importance is the category field . The drop list of entries are contained in a table and filled via a ActionHandler. This alows the |
From: Shanta M. <sh...@fo...> - 2002-11-20 17:34:20
|
On Wed, 2002-11-20 at 10:46, James Mohr wrote: > That's exactly it: "when a host that dose not have access to SQL" and that's > really the only reason. A corollary is someone who does not have the skill > or desire to set up an SQL database, but that is less important to me than > someone who cannot set it up. However, it will be a while before it is a > "product" that we can package and make available on SourceForge, etc. I just > don't want to paint ourselves into a corner. > That is why I like to store my data in SQL. It allows me to manipulate the data as I see fit. This includes exporting data to cvs files. > > > > I will have the CSV s files into MySQL in a few hours. I have code to do > > this by simply importing the contents into the SQL Table.PHP but it > > works. From that point on to switch the code from on DataSource to > > another takes a simple change in one variable in the site setup file. > > Export the contents of the SQL table to a | delimited file though and > > delimiter can be used. and there you have switched. > > Does that mean there would be little change in the eXtropia code to reflect a > CSV data source as opposed to SQL? > In my setup 0 change exempt one variable in the Global site setup. $DataSource = "file"; or $DataSource = "MySQL"; DataSources currently supported by eXtropia are file, MySQL, Oracle and a few others but they are not important. > > Validation, field > > contents, required fields and many other checking is already in eXtropia > > one just has to tell it what you which to check for. Don't know if this > > is what you are referring to with XML validation as I know little of > > XML. > > I'm refering to the actual XML files before they are imported into the > database. If someone forgets a closing tag or something else, we should know > about it before we try to read it into the database. > Ok. but that is why one develops on a separate server. could be added to eXtropia but XML in the form View would likely be easier > > > Following are some thoughts and ideas about how the tables can be > > > created. I have no particular affection for any of these ideas, so feel > > > free to tell me they stink. > > > > > > The first thing we need to decided for the data model is how to store the > > > KUs themselves. Since different KU types have different attributes, I do > > > not see a way of storing them all in a single table. Instead, I would see > > > one table for each KU type and then a central table that contains the > > > unique KU ID and then the KU type. When making a cross reference from one > > > KU to another, we would then go through this central table. > > > > > > Next is the inter-relationship between the KUs, which is a core aspect of > > > linkbat. There is a potential that each KU (regardless of type) can > > > reference every other type of KU. However, I cannot really imagine an > > > index for each relationship. For example, the Concept KU has an index > > > listing all of the Concept KUs and with pointers to all of the referenced > > > Glossary KUs. Then another index with the MoreInfo KUs. In my mind that > > > would mean too many tables. If there are five KU types and each has a > > > index for the relationship to the others, then we would have 20 tables > > > (5x4). With 6 KU tables, 30 tables (6x5). Is my math right? > > > > > > However, the question is whether 20 or 30 indexes is "too many". Does > > > having seperate indexes for each relationship provide any advantages? > > > Quicker access? If so, does it compensate for the extra work to manage > > > 20-30 tables?? > > > > The SQL table takes care of that for another reason to store all data in > > SQL.You can assign indexes within each table. > > I'm aware of that. However, these are not quite the same indexes that an RDBMS > system like MySQL would generate automatically. It's not a index of a single > table used to speed-up access, but rather a table containing the relationship > between the KUs. These would be something we would have to define and fill > ourselves I'm just curous about the performance issue of having 1 or 30 > tables versus the administration of 30 tables. Is that really an issue. > > Based on the discusion following this part, I think only a single table will > be needed since MySQL should be able to handle the amount of data we are > dealing with. > > ------------------------------------------------------- > This sf.net email is sponsored by: To learn the basics of securing > your web site with SSL, click here to get a FREE TRIAL of a Thawte > Server Certificate: http://www.gothawte.com/rd524.html > _______________________________________________ > Linkbat-devel mailing list > Lin...@li... > https://lists.sourceforge.net/lists/listinfo/linkbat-devel -- Shanta McBain <sh...@fo...> |
From: James M. <lin...@ji...> - 2002-11-20 17:16:54
|
On Wednesday 20 November 2002 17:24, Shanta McBain wrote: <extropia stuff snipped> > I am a bit at a loss as to what is so special about CSV text files. I > see them only as a poor and limited (Hard to display code) form of data > storage When data is stored in SQL there is no need for them at all > except as an alternate storage system when a host that dose not have > access to SQL. That being said I still use SQL quires to search the > files for the records and contents of the records you what to display in > the "View". That's exactly it: "when a host that dose not have access to SQL" and that's really the only reason. A corollary is someone who does not have the skill or desire to set up an SQL database, but that is less important to me than someone who cannot set it up. However, it will be a while before it is a "product" that we can package and make available on SourceForge, etc. I just don't want to paint ourselves into a corner. > > Rama has volunteered to work on the code to convert the XML files into > > something the database can handle. Shanta is working on the "presentation > > layer", so obviously you two will be working closely together. When the > > code is done for the CSV->XML conversion, I would like Luu to start > > working on the code to validate the XML files. That is, making sure the > > right tags are there, tags match, etc. > > I will have the CSV s files into MySQL in a few hours. I have code to do > this by simply importing the contents into the SQL Table.PHP but it > works. From that point on to switch the code from on DataSource to > another takes a simple change in one variable in the site setup file. > Export the contents of the SQL table to a | delimited file though and > delimiter can be used. and there you have switched. Does that mean there would be little change in the eXtropia code to reflect a CSV data source as opposed to SQL? > Validation, field > contents, required fields and many other checking is already in eXtropia > one just has to tell it what you which to check for. Don't know if this > is what you are referring to with XML validation as I know little of > XML. I'm refering to the actual XML files before they are imported into the database. If someone forgets a closing tag or something else, we should know about it before we try to read it into the database. > > Following are some thoughts and ideas about how the tables can be > > created. I have no particular affection for any of these ideas, so feel > > free to tell me they stink. > > > > The first thing we need to decided for the data model is how to store the > > KUs themselves. Since different KU types have different attributes, I do > > not see a way of storing them all in a single table. Instead, I would see > > one table for each KU type and then a central table that contains the > > unique KU ID and then the KU type. When making a cross reference from one > > KU to another, we would then go through this central table. > > > > Next is the inter-relationship between the KUs, which is a core aspect of > > linkbat. There is a potential that each KU (regardless of type) can > > reference every other type of KU. However, I cannot really imagine an > > index for each relationship. For example, the Concept KU has an index > > listing all of the Concept KUs and with pointers to all of the referenced > > Glossary KUs. Then another index with the MoreInfo KUs. In my mind that > > would mean too many tables. If there are five KU types and each has a > > index for the relationship to the others, then we would have 20 tables > > (5x4). With 6 KU tables, 30 tables (6x5). Is my math right? > > > > However, the question is whether 20 or 30 indexes is "too many". Does > > having seperate indexes for each relationship provide any advantages? > > Quicker access? If so, does it compensate for the extra work to manage > > 20-30 tables?? > > The SQL table takes care of that for another reason to store all data in > SQL.You can assign indexes within each table. I'm aware of that. However, these are not quite the same indexes that an RDBMS system like MySQL would generate automatically. It's not a index of a single table used to speed-up access, but rather a table containing the relationship between the KUs. These would be something we would have to define and fill ourselves I'm just curous about the performance issue of having 1 or 30 tables versus the administration of 30 tables. Is that really an issue. Based on the discusion following this part, I think only a single table will be needed since MySQL should be able to handle the amount of data we are dealing with. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-20 16:26:25
|
Hi All! It just dawned on me that we might have a problem with the command KUs. Each command has a different number of options and examples. So I don't see a way of having one record per command. The way is see it is that we have a table for the options that has (at least three fields): COMMAND_KU_ID:OPTION:ACTION Where the OPTION is the text of the option and ACTION is a description of what the option does. We would also have a table for the examples: COMMAND_KU_ID:TEXT:DESCRIPTION Where TEXT is the syntax example and DESCRIPTION is a short description of that example. Commands also have other attributes like a synopsis, description and syntax (look at any man-page for what I am talking about). However, these will be consistant for all Commands KU. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: Shanta M. <sh...@fo...> - 2002-11-20 16:25:03
|
I have made limited comments in line. Could everyone go to http://webcthelpdesk.com/cgi-bin/Linkbat/linkbat.cgi?site=Linkbat click the logon link under the logo on the left and create an account. I will set the system to display so group normal will display all the links. Until I get you all changed to Linkbat_admin. The script mails me when an account is created. To bring everyone up to speed on this. This site is created in a web application delivery product known as eXtropia. The product takes care of Sessions, Authentication, DataSource (works transparently with CVS or SQL), Views, name space, and much much more. There is 15 or more years of development here with the last five for banks. I have been developing applications in this product now for 4 or 5 years. At first it was vary much a blackbox but now it is a resource I would not be without. Remember that this is not necessarily the end delivery engine. James has not at this time accepted this as the method of delivering the project but rather a Demo on what eXtropia can do. This site is also a blend of Linkbat and my admin services. It has tools in it that are not available to any but my clients at this time. Project tracking, todo's logs, FAQ etc. They are in the extra Admin link box. All these tool's contents are in MySQL. On Wed, 2002-11-20 at 07:02, James Mohr wrote: > Hi All! > > I think we have reached the point where we need to be talking about the actual > data/table structure in CSV text or a database. For now let's just work on > the database model, to make things simpler. However, we all need to keep in > mind that I eventually want to have the system work with CSV text files. > I am a bit at a loss as to what is so special about CSV text files. I see them only as a poor and limited (Hard to display code) form of data storage When data is stored in SQL there is no need for them at all except as an alternate storage system when a host that dose not have access to SQL. That being said I still use SQL quires to search the files for the records and contents of the records you what to display in the "View". > Rama has volunteered to work on the code to convert the XML files into > something the database can handle. Shanta is working on the "presentation > layer", so obviously you two will be working closely together. When the code > is done for the CSV->XML conversion, I would like Luu to start working on the > code to validate the XML files. That is, making sure the right tags are > there, tags match, etc. I will have the CSV s files into MySQL in a few hours. I have code to do this by simply importing the contents into the SQL Table.PHP but it works. From that point on to switch the code from on DataSource to another takes a simple change in one variable in the site setup file. Export the contents of the SQL table to a | delimited file though and delimiter can be used. and there you have switched. Validation, field contents, required fields and many other checking is already in eXtropia one just has to tell it what you which to check for. Don't know if this is what you are referring to with XML validation as I know little of XML. > Following are some thoughts and ideas about how the tables can be created. I > have no particular affection for any of these ideas, so feel free to tell me > they stink. > > The first thing we need to decided for the data model is how to store the KUs > themselves. Since different KU types have different attributes, I do not see > a way of storing them all in a single table. Instead, I would see one table > for each KU type and then a central table that contains the unique KU ID and > then the KU type. When making a cross reference from one KU to another, we > would then go through this central table. > > Next is the inter-relationship between the KUs, which is a core aspect of > linkbat. There is a potential that each KU (regardless of type) can reference > every other type of KU. However, I cannot really imagine an index for each > relationship. For example, the Concept KU has an index listing all of the > Concept KUs and with pointers to all of the referenced Glossary KUs. Then > another index with the MoreInfo KUs. In my mind that would mean too many > tables. If there are five KU types and each has a index for the relationship > to the others, then we would have 20 tables (5x4). With 6 KU tables, 30 > tables (6x5). Is my math right? > > However, the question is whether 20 or 30 indexes is "too many". Does having > seperate indexes for each relationship provide any advantages? Quicker > access? If so, does it compensate for the extra work to manage 20-30 tables?? > The SQL table takes care of that for another reason to store all data in SQL.You can assign indexes within each table. > Alternatively, I could imagine one index that keeps tracks of the type of KUs, > as well: > > UNIQUE_KU_ID:REFERENCED_TYPE:REFERENCED_KU_ID > > UNIQUE_KU_ID - The ID of the current KU > REFERENCED_TYPE - The type of KU that this KU is referencing > REFERENCED_KU_ID - The ID of that KU that is being refereced. > > A query could be done sorted by UNIQUE_KU_ID and then REFERENCED_TYPE. Once > we have that, the code to present a list of referenced KUs grouped by KU type > would be easy. Currently we are only dealing with a few thousands KUs. > However, what about when we get to tens of thousands?? Hopefully MySQL should > be in a position to deal with that **few** records. So, I would not see it as > a problem to have all of the KUs referenced in a single table like that. > > Is the REFERENCED_TYPE even necessary here? We obviously need a table that > contains the attributes of each KU and a unique ID, so the REFERENCED_KU_ID > can be used to get the type through the central table that I mentioned above > and we have the KU type. On the other hand, to get a list of all referenced > KUs in the correct order we could have a query like this: > > select REFERENCED_KU_ID from WHATEVER_TABLE where UNIQUE_KU_ID = > 'CURRENT_KU_ID" sort by REFERENCED_TYPE > > This is a single table access and is obviously faster than mutliple access. > Therefore, I think this would more than make up for having an extra field in > the index. > > Since we can expect that new data files will have mixed KU types, I am > thinking that a single index which also contains the KU type would be more > efficient. Each time we read a new KU, it is inserted into that single index, > so we don't have to deal with mulitple indexes. Comments? > > Although not KUs in the true sense, I think we need an index by topic and > skill level (if we decide to implement that). This would be used in > generateing the technical FAQs or lists of all info on a particular topic. > > What about the reverse references. For example a list of all Concept KUs that > reference a particular Glossary KU. This is not the same as the list of > Glossary KUs that this Concept KU references. We click on a link that > displays a Glossary KU, with a link to all the KUs that it references. I > would also like links to all of the KUs that reference this glossary term. > Although it seems logical that each Concept KU that references a particular > Glossary KU should also be referenced by that Glossary KU, it is likely that > we will forget the reverse references when create the XML files. Think about > the glossary terms on the content pages. Each time we add a new page we have > to update the referenced glossary KUs to point back to this page. > > I'm sure there is a lot more, but I should get this off to start the > discussion going. > > Regards, > > jimmo > -- > --------------------------------------- > "Be more concerned with your character than with your reputation. Your > character is what you really are while your reputation is merely what others > think you are." -- John Wooden > --------------------------------------- > Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info > > > ------------------------------------------------------- > This sf.net email is sponsored by: To learn the basics of securing > your web site with SSL, click here to get a FREE TRIAL of a Thawte > Server Certificate: http://www.gothawte.com/rd524.html > _______________________________________________ > Linkbat-devel mailing list > Lin...@li... > https://lists.sourceforge.net/lists/listinfo/linkbat-devel -- Shanta McBain <sh...@fo...> |
From: James M. <lin...@ji...> - 2002-11-20 13:33:16
|
Hi All! I think we have reached the point where we need to be talking about the actual data/table structure in CSV text or a database. For now let's just work on the database model, to make things simpler. However, we all need to keep in mind that I eventually want to have the system work with CSV text files. Rama has volunteered to work on the code to convert the XML files into something the database can handle. Shanta is working on the "presentation layer", so obviously you two will be working closely together. When the code is done for the CSV->XML conversion, I would like Luu to start working on the code to validate the XML files. That is, making sure the right tags are there, tags match, etc. Following are some thoughts and ideas about how the tables can be created. I have no particular affection for any of these ideas, so feel free to tell me they stink. The first thing we need to decided for the data model is how to store the KUs themselves. Since different KU types have different attributes, I do not see a way of storing them all in a single table. Instead, I would see one table for each KU type and then a central table that contains the unique KU ID and then the KU type. When making a cross reference from one KU to another, we would then go through this central table. Next is the inter-relationship between the KUs, which is a core aspect of linkbat. There is a potential that each KU (regardless of type) can reference every other type of KU. However, I cannot really imagine an index for each relationship. For example, the Concept KU has an index listing all of the Concept KUs and with pointers to all of the referenced Glossary KUs. Then another index with the MoreInfo KUs. In my mind that would mean too many tables. If there are five KU types and each has a index for the relationship to the others, then we would have 20 tables (5x4). With 6 KU tables, 30 tables (6x5). Is my math right? However, the question is whether 20 or 30 indexes is "too many". Does having seperate indexes for each relationship provide any advantages? Quicker access? If so, does it compensate for the extra work to manage 20-30 tables?? Alternatively, I could imagine one index that keeps tracks of the type of KUs, as well: UNIQUE_KU_ID:REFERENCED_TYPE:REFERENCED_KU_ID UNIQUE_KU_ID - The ID of the current KU REFERENCED_TYPE - The type of KU that this KU is referencing REFERENCED_KU_ID - The ID of that KU that is being refereced. A query could be done sorted by UNIQUE_KU_ID and then REFERENCED_TYPE. Once we have that, the code to present a list of referenced KUs grouped by KU type would be easy. Currently we are only dealing with a few thousands KUs. However, what about when we get to tens of thousands?? Hopefully MySQL should be in a position to deal with that **few** records. So, I would not see it as a problem to have all of the KUs referenced in a single table like that. Is the REFERENCED_TYPE even necessary here? We obviously need a table that contains the attributes of each KU and a unique ID, so the REFERENCED_KU_ID can be used to get the type through the central table that I mentioned above and we have the KU type. On the other hand, to get a list of all referenced KUs in the correct order we could have a query like this: select REFERENCED_KU_ID from WHATEVER_TABLE where UNIQUE_KU_ID = 'CURRENT_KU_ID" sort by REFERENCED_TYPE This is a single table access and is obviously faster than mutliple access. Therefore, I think this would more than make up for having an extra field in the index. Since we can expect that new data files will have mixed KU types, I am thinking that a single index which also contains the KU type would be more efficient. Each time we read a new KU, it is inserted into that single index, so we don't have to deal with mulitple indexes. Comments? Although not KUs in the true sense, I think we need an index by topic and skill level (if we decide to implement that). This would be used in generateing the technical FAQs or lists of all info on a particular topic. What about the reverse references. For example a list of all Concept KUs that reference a particular Glossary KU. This is not the same as the list of Glossary KUs that this Concept KU references. We click on a link that displays a Glossary KU, with a link to all the KUs that it references. I would also like links to all of the KUs that reference this glossary term. Although it seems logical that each Concept KU that references a particular Glossary KU should also be referenced by that Glossary KU, it is likely that we will forget the reverse references when create the XML files. Think about the glossary terms on the content pages. Each time we add a new page we have to update the referenced glossary KUs to point back to this page. I'm sure there is a lot more, but I should get this off to start the discussion going. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-20 09:05:09
|
On Wednesday 20 November 2002 02:31, Shanta McBain wrote: > Were is the data for FAQ's stored? If you are referring to the site FAQ, then it is pure text. That is the page is all hardcoded. It started out as a simply introduction with just a couple of paragraphs and grew into its current form. Currently there are no technical FAQs, but that will come out of the XML files. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-20 09:01:55
|
On Wednesday 20 November 2002 02:30, Shanta McBain wrote: > This line came from tyk.data. > > 1:3:Two different users can share the same program.:T:::: > > my guess > > RecordID auto inc. | quiz_id | Question | Answer :?:? :etc > Please give your current field names. I will create an access to the raw > data (this will become the Question db) and put the content of this file > into MySQL. This is obviously the quiz dataset. The format is: QUESTION_ID:PAGE_ID:QUESTION_TEXT:CORRECT_ANSWER:WRONG_ANSWER1:..:WRONG_ANSWER4 If the correct answer is either T or F, the code knows that it is a True/False question and the other answer is obviously wrong. The page ID is the reference into the page.data file. The "real" version of the DB also needs to account for a "reason" field for each answer. > I have on hand a copy of WebCT's they have a well developed quiz system > that is split into to parts. > > The actual quiz and the questions that it uses. It will be a good source > of code snippets and ideas( not open licence so mostly ideas.) This > product is a complex course delivery tool licenced to University's > across the world. If you have taken an on line course from a major > University it likly was delivered with WebCT software. But that another > story. The name sounds familiar, but I am not sure. I have seen a number of CBT systems that have some really nice features, one is obviously the user management. Another is proving a series of questions without revealing the answer until all questions are answered. Currently my system presents the correct answer immediately. However, that is useful too. One idea for the future is the dynamic question generation based on what has been answered wrong up to that point (what's the correct term?). If they get a question wrong in a particular area they keep getting more in that area. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: Shanta M. <sh...@fo...> - 2002-11-20 01:32:19
|
Were is the data for FAQ's stored? -- Shanta McBain <sh...@fo...> |
From: Shanta M. <sh...@fo...> - 2002-11-20 01:31:33
|
This line came from tyk.data. 1:3:Two different users can share the same program.:T:::: my guess RecordID auto inc. | quiz_id | Question | Answer :?:? :etc Please give your current field names. I will create an access to the raw data (this will become the Question db) and put the content of this file into MySQL. This is obviously the quiz dataset. I have on hand a copy of WebCT's they have a well developed quiz system that is split into to parts. The actual quiz and the questions that it uses. It will be a good source of code snippets and ideas( not open licence so mostly ideas.) This product is a complex course delivery tool licenced to University's across the world. If you have taken an on line course from a major University it likly was delivered with WebCT software. But that another story. Thanks -- Shanta McBain <sh...@fo...> |
From: James M. <lin...@ji...> - 2002-11-19 10:50:12
|
Hi All! Here is a new version of the dyk.data file (which is converted into Concept KUs). I have added 1-3 topics to each of them. These are the topics that I talked about in my last email. Luu, please integrate this into the code. I will be working on adding topics to the other data files, but I would like to have input on the list of topics before I do all of them. With this new dyk.data file, you can do the code to create the <Topics> container and the <TopicRef> entries. The code should not be any different for the other data files. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-19 09:59:38
|
Hi All! I would like some input about standard topics. Here are the ones that I came up with (in no special order): Networking Operating System Administration Working with the System Users Hardware Troubleshooting Security Linux-Windows Integration These are "related" to the chapter headings within the tutorial but not identical. Chapter headings tend to be more "wordy". Plus a single chapter can relate to multiple topics, plus a single topic could be spread across multiple chapters. I would like to have the list of topics as simple as possible, but cover the most important topics. I am pretty happy with this list, although "Linux-Windows Integration" **might** be a little too much. However, we must keep our audience in mind. There will be a lot of people who are moving from Windows to Linux and need information about integration, easing the transition, and so on. The term "integration" is not necessarily the best choice, so I am open to suggestions. Another one is "Working with the System". This refers to issue related to working with the system (what else), like shells, X-Windows, logging in, etc. This is not the same as "Users" which refers to topics related to user accounts (passwords, permissions, allowing remote access, etc.) Please let me hear from you as soon as possible. (Please, be sure to send to the list and not just to me!) Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-19 08:34:44
|
(Note this was sent to the list.) Hey everyone, the conversion is almost done (well, at least the code for it). Thanks Luu! However, there are some important questions to answer NOW, before we continue. PLEASE, please, please read this and give me your input. On Tuesday 19 November 2002 00:31, Luan Luu wrote: > according the the moreinfo.data, the format: > ID#:TYPE:DESCRIPTION:LOCATION > > the XML are > > <KnowledgeUnit> > <Atrributes> > <Type sub-type="[TYPE]" location="[LOCATION]">MoreInfo</Type> > <Text>[DESCRIPTION]</Text> > </Atrributes> > </KnowledgeUnit> > > In the reference to the brackets, is the pointer the the type, location, > and description like that? Perfect. The only question is whether we should actually do it that way or not. That is, should the sub-type and location be attributes within the <Type> tag or should they be seperate tags, i.e. <SubType> <Location>? By gut feeling is that they should be attributes within the <Type> tag. They not necessarily attributes of KU, but rather provide additional info for the type. Comments anyone? This needs to be answered before we continue. > is the url be absolute path with the http infront right? Yes. You will note that in the data file, they just begin with // and not http://. This was because I made an unwise decision to use the colon (:) as the field seperator. However, the colon appears frequently in Linux (especially with URLs) so it became a problem. I was going to change to something like a pipe (|) which comes less frequently. Regardless, we will have a problem since the odds are that whatever character we use, it will appear in text somewhere in the data. Obviously this is not a problem when we import directly into a database. However, as I said to Shanta, I don't have a problem with importing the files directly into a database for the first release. However, eventually I want the system to be independant of the data source (CSV, database) and independant of the presentation (eXtropia, other portal). Therefore, we need to consider a new seperator. Suggestions? > Inside the tyk xml question tags, there is a topicRef tag, which is the > reference of the PAGE_ID. So, do you want to put the Page_id in the > topicRef tags or the actual topic name in the page.data ? The TopicRef is a references to a topic, such as Administration, Networking, Security, etc. This is just text. There are PageRef tags and these contains the page *name* from page.data. However, I am nop longer sure we should do it that way (see below). This needs to be changed in tykToXML.pl. Keep in mind that the questions will exist only within a KU. This KU will have a primary page so we automatically have the primary page for the question. However, the question could reference multiple topics. We have a problem with some of the questions where an angle bracket is one of the answer ("What symbol is used to "pipe" two commands together?") This means we have two angle brackets together (<< or >>) which could confuse the XML parser. There are only a few and we can change them by hand. We just need to be aware of them. Also watch the format of the answers, even for the T/F questions: <Correct> <Text>T</Text> <Reason>why this answer is correct</Reason> </Correct> not just <Correct>T</Correct> I think that as much as possible it is better to have the same format for all types of questions. More than likely, "fill in the blank" type questions won't have an <Incorrect> answer, but I still want to have a <Reason> tag to provide an explanation why the answer is correct. However, since we do not yet have the reasons, I think you should simply leave as <Reason></Reason>. I think if we leave the text as "put your reason why is correct/incorrect. " we might forget to change it and then displaying that text would look silly. If the <Reason> tag is empty, we can just ignore it. OR we could simply not include the <Reason> tag at this point. What do you think? With the Glossary KUs please create a <GlossaryTerms> container with the GlossaryRefs to the other terms. These are the numbers at the end of each line in glossary.data. They are the ID numbers of the other glossary terms. Therefore, instead of reading in each line from glossary.data and processing it, you will need to read it all at once and put it into an array, then parse that array. EVERYONE PLEASE READ AND COMMENT: Currently the glossary.idx file contains a list of pages that contain each glossary item. This is created by an external script and is **not** done when the glossary item is loaded. That would take way too much time. The question is whether we should have PageRefs within the Glossary KU. Personally, I do not think so. We can create the index of glossary-page_id along with everything else. If we include the page ID/page name within the Glossary KU and add a new glossary item, then we would need to go looking for all of the pages that have that glossary term. Obviously we need to search for the pages to add the <Glossary> tag within the page. However, I just see it as unnecessary work to add PageRefs withing the Glossary KU since we can create the index by other, more efficient means. EVERYONE PLEASE READ AND COMMENT: It just hit me that we might be building a trap for ourselves. If we use the full path instead of the ID number, we will have problems if we ever rename the file, move it to a different directory, etc. I **expect** to be moving files to different directories real soon! I want to change the order of the files and their locations. As we get more content, I can imagine that we change locations again. I see three options: Use the full-path as the PageRef: - Easy to find/insert the reference we want - Tracking down the actual page from a KU is easy - In the display code we don't need to do a look up to display the page. - PROBLEM: moving/renaming the file. Since the XML files are text, we can use sed/perl to make a global change. Use just the page name without the path: - Once the file name is defined, it is less likely that the page name will be changed. PROBLEM: We must have *completely unique* page names. We cannot have a "Known Problems" in the Network section AND in the Printing section. They must be named "Known Problems-Network" and "Known Problems-Printing" (or something like that). Use the ID as the PageRef: - Remains constant, independant of the actual name of the file. - PROBLEM: Need to do a look-up to find the correct file. However, since page.data is current sorted by chapter/section, I have found that it is not al that hard. For the existing moreinfo and DYK entries one PageRef can be inserted automatically. Still, if we want to include more PageRefs, we will hve to do it by hand and look up the ID, but we will have to look it up any way to get the full path. So whether we lookup and insert the page name or the page ID it's the same amount of work. I still like the idea of using the full-path and NOT and ID number. You need to do a look-up anyway to find the ID or the correct text for the full path. Tracking down the original page from the XML file is straight forward. Making a change would be a simple matter of running a sed/perl script. We could even write it in advance and it becomes a part of our "utility" package: rename_page.pl [-f filename] original_name new_name It then scans all PageRefs in the named file and changes them accordingly. Using an ID number bothers me because makes the construct dependant on an external file or we are imposing a structure on it unnecessarily. Therefore, the knowledge base is not self-contained. EVERYONE PLEASE READ AND COMMENT: We have a similar problem with the MoreInfoRefs for the Page KUs. Currently they are referenced by their ID number and Luu did the same thing in her code. However, once again, I am not happy with idea of using ID numbers instead of text. So, do we reference the text of the MoreInfo KUs?? I have pretty much decided to go through the existing data files and add up to three topics. I will add these to the **end** of each line for all of the data files. So, Luu, could you change the code to create a <Topics> container and <TopicRefs> for all of the data files? Note that I will probably not list three topics for everything. Therefore, the code will need to be smart enough to recognized this. Since you are probably asleep already I can work on it today and send you at least one file with the topics, so you will see the format. Regards, jimmo -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-18 21:00:43
|
Hi Luan! (for the rest of you, please take a look at this as well, I would like some feedback). Wow! That was quick. First, note that I replied to the linkbat-devel mailing list. I think all of these discussions should be on the list. The rest of the comments are below. On Monday 18 November 2002 21:20, Luan Luu wrote: > 1. Do you want the code automatic create the output file, or we could > manually in the command line to concatenate to new xml file? I think you should just write to standard output. This gives us greater flexability to specify any file name we choose. > 2. Only one top level is allowed, so, i put another <KUs> tag wrap around > the <knowledgeUnit> tags. is that ok? In the specification, I had defined the top-level as being <KnowledgeUnits> (with the 's' at the end). In general, I had it so that the container were all plurals like <KnowledgeUnits> and <MoreInfos>. > check the first one out, see if it is correct. let me know. thanks. So far that looks great!!! However, a couple of things. The <PageRef> should contain the file name out of the page.data file. These numbers will more or less disappear once the conversion is made. Also, please put in an line between the KnowledgeUnits. What I was thinking about was going through the original data files and add topics. It would be fairly straight forward to define a handful of topics (administration, network, security, users, hardware, etc.) and add them to the CSV files. That would mean an extra field to parse, but the code is pretty much written. In fact, I could add multiple topics and if the field is empty, just don't print a topic tag. How does that sound? I am thinking that we could save a fair bit of time that way. If at least one (or maybe even 2 or 3) topics are already present and can be added automatically, then we don't need to do it by hand. Granted, sub-topics will probably need to be added, later. But we have saved some work. In the CVS state right now, it is alot easier to add information in bulk. Comments, anyone? Are there other references that we could add in bulk now? Regards, Jim ================= NOTE: I snipped most of the stuff. This is just an example to see the results and the original file. <KnowledgeUnit> <Attributes> <Type>Concept</Type> <Text>Linux can be started from any partition.</Text> </Attributes> <Pages> <PageRef primary="true">106</PageRef> </Pages> <Questions> </Questions> </KnowledgeUnit> <KnowledgeUnit> <Attributes> <Type>Concept</Type> <Text>Linux can combine multiple drives into a single RAID system, even if the drives are of different types.</Text> </Attributes> <Pages> <PageRef primary="true">127</PageRef> </Pages> <Questions> </Questions> </KnowledgeUnit> <KnowledgeUnit> <Attributes> <Type>Concept</Type> <Text>Unwanted cron output can be redirected just like any other command.</Text> </Attributes> <Pages> <PageRef primary="true">21</PageRef> </Pages> <Questions> </Questions> </KnowledgeUnit> <KnowledgeUnit> <Attributes> <Type>Concept</Type> <Text>Cron is started through an rc-script like most system daemons. </Text> </Attributes> <Pages> <PageRef primary="true">67</PageRef> </Pages> <Questions> </Questions> </KnowledgeUnit> <KnowledgeUnit> <Attributes> <Type>Concept</Type> <Text>Cron can be disabled through the /etc/rc.config file.</Text> </Attributes> <Pages> <PageRef primary="true">67</PageRef> </Pages> <Questions> 1:106:Linux can be started from any partition. 2:127:Linux can combine multiple drives into a single RAID system, even if the drives are of different types. 3:21:Unwanted cron output can be redirected just like any other command. 4:67:Cron is started through an rc-script like most system daemons. 5:67:Cron can be disabled through the /etc/rc.config file. #!perl #!/usr/bin/perl #convert the dyk.data into xml data. open(DATA,"<dyk.data"); $TYPE = "Concept"; #hardcoded. $TEXT = "put your text here"; $PAGE_ID = "your Page id from the file."; #required, only one top level is allowed. print "<KUs>\n"; while (<DATA>){ chomp; $READLINE=$_; ($ID,$PAGE_ID,$TEXT) = split(/:/,$READLINE); #print the output... print " <KnowledgeUnit>\n"; print " <Attributes>\n"; print " <Type>$TYPE</Type>\n"; print " <Text>$TEXT</Text>\n"; print " </Attributes>\n"; print " <Pages>\n"; print " <PageRef primary=\"true\">$PAGE_ID</PageRef>\n"; print " </Pages>\n"; print " <Questions>\n"; print " \n"; #blank for now, insert the questions later. print " </Questions>\n"; print " </KnowledgeUnit>\n"; } print "</KUs>\n"; close DATA; |
From: James M. <lin...@ji...> - 2002-11-18 18:35:38
|
Hi All, this is the second half of a conversation I have been having with Shanta McBain who is working on the presentation of the data (among other things). She reminded me that we need to have these discussions on the list and not in private. I have created a linkbat-writers mailing list to discuss actual content. Most of the people on that list will not be all too interested in the technical details of the actual implementation and I don't think you developers are all that interested in what goes into which article. However, I'll let you be on both lists if you want. As another reminder, please, please, please send me comments, questions, etc about the data model as soon as possible. On Monday 18 November 2002 17:41, Shanta McBain wrote: > > Here are the content pages for the tutorial. With the data files and code > > that I sent last week, you should have enough to get a working demo > > running using eXtropia. The actual data won't change much and the format > > of any CSV file should be *basically* the same. Even if the order of the > > fields is different or there are more fields, the basic principles are > > the same. > > I will over the course of the next few days convert your data to MySQL. > Hope to have the FAQ done tonight. Keep in mind that what is currently in the CSV files is only a small fraction of what will be there when we get all of the KUs defined. > > So, where do you want to go from here? Do you want to be responsible for > > the data extraction and presentation? Just the presentation? I have one > > person (Luan) already working on the conversion from the existing CSV to > > XML and two people who were interested in the coding from the other > > direction. However, they have not responded since their original email. I > > am going to ping them again to see if they are still interested. We also > > have a couple of new editors and content writers. > > XML is a subset of eXtropia views. I will have to show her how to put > her code into the views used to display the content of the MySQL tables. > Once the cvs is converted there is no longer any need for them. > Except if you what others to install the system on their server which I > don't see a need for. There is a misunderstanding here and I am not sure on which side. Luan is currently working on the code to generate the XML file from the existing CSV files. The other two are planned to write the code to covert the XML to either new, expanded CSV or insert them into a database. Once the XML files are generated their will a need for CSV files, in general. We need to provide the option of running the system without an SQL database or some other database other than MySQL. Although we can initially work toward this first public version being on MySQL, I don't want code written exclusively to import to MySQL. > > If you are doing the presentation component, could you also come up with > > a decent CSS concept? What I did was a kludge and doesn't look good on > > all systems, particularly MS Internet Exploder 5.x. With 6x it's okay, > > but 5 gives me problems. We could tell people to upgrade, but I think it > > is just as much a problem with the styles as it is with IE5. > > You will not have too much luck accommodating IE except to give them a > complete view done just for them. This is one reason that I have avoided > fancy dancy. My suggestion here is to code to HTML 4 get Bobby > certification and encourage users to upgrade to a gecko based browser > such as Netscape or the leading edge Mozilla by providing links to them. > Possibly a page giving reason why they should abandon IE for a net > standard browsers. If we can get it looking decent on Mozilla-based, Konquerer, Opera and any thing else that runs on Linux, plus fairly decent on IE 5/6 then I am happy. We have people using IE visiting the site. As long as no functionality is missing and something looks goofy, then I am happy. We cannot afford to alienate people by saying the site only works on non-MS browsers. That's shooting ourselves in the foot. > > I just what to re-emphasize that I don't want to turn this into an > > eXtropia project. That is, a project to get linkbat work explicitely with > > eXtropia. It should be completely independent of the front end we choose. > > I understand your enthusiasm with the product, but as much as possible > > the presentation layer of linkbat needs to be portable to any system. > > eXtropia is portable to any system that has Perl. The data can be in any > file or SQL. So if your code can access the files or SQL then you can > display the contents. eXtropia takes care of authentication, Sessions, > encryption, DataAccess (file, MySQL, PosGress, Oracle etc), translation, > Logging, to name a few. This is part of the base eXtropia distribution. Not eXtropia has to be portable, but linkbat. eXtropia handling the data access concerns me in that we will be dependant on eXtropia. If eXtropia access an "api" (one we define or is provided by someone else) that goes out and gets the data from MySQL, CSV, or whatever, then there is less of an issue. However, eventually (not necessarily in the first iteration), I want the Data Access to be presentation independant or at least have the ability to make it so. Obviously if it is MySQL and someone know how the tables are defined they can access the data. However, I do not see that as the same thing as us providing the "api". > > It will eventually need to be made available for download from > > SourceForge, so it needs to be in a form that people can install without > > hounding us to death. On the other hand, is there any **requirement** > > that says we need to make things available throguh source forge. Hmmmm. > > Still, eventually I would like at least the code and data files format > > available for download. I want to adher to the open source principles > > while at the same time not giving aware the farm (just yet). > > eXtropia is already available for down load and is vary easy to install. I was thinking more along the lines of our stuff getting available for download. What "our stuff" is and when it would be ready for download is still unclear. > They have their own support system so one can just pass eXtropia level > questions to them or my HelpDesk system. The part that will be: > > HTMLViews spacific to us How configurable are these? It is possible for users to change the color scheme, for example. What about the layout from and administration pespective? I want certain data presented in certain places. Take the Linux Tutorial as an example. It's a 3x3 table with each cell performing a particular function (i.e. displaying certain data). > The Data Files. I can export the current table contents into a cvs file > or SQL file for import into SQL. > > Any ActionHandlers that are unique to our project so far I don't see a > need for them. Please clarify for us (i.e. me ) what an ActionHandlers really does. If I click on a "More Info" link to see the more info related to the current page, wouldn't it be an ActionHandler that generates the new page? However, since we are dealing with "standard" queries a "standard" action handler could runn the query, something like this: select * from moreinfo_table where page_id = '1234' Is that what we are talking about? Let's say I want a list of topics and clicking on one gives me a list of all KUs that reference this topic. (See Topic Tree on the linkbat concept page). How would that be realized? Would that also be a simply query? > The setup files unique to the project. What do they look like? How are they configured? > I only code to open source though I do accept payment for modification > of code but the code anyone can have and use. Open source is one reason for doing it through SourceForge. Code is not the issue, but rather the data (i.e. the KUs). One reason I am concerned with releasing the data is that I see us opening up a whole can of worms to support raw data. If we have a complete "product" which includes instructions on how to configure eXtopia, perl code, XML files, etc I don't have a problem. I just don't want to deal with pieces. > > The more I think about it, the more I like th > > e idea of extra content at the > > various levels of "membership". In addition to the online output of > > commands, > > Easy to do in eXtropia. Out of curiosity, can you define different levels? > > I have been planning to include "pseudo-animated" graphics on the site. > > Using an IFRAME that refreshes itself at regular intervals, you can get > > some nice looking graphics. Since there is nothing on the site that needs > > 3D life-like animations, this works well. The images are pretty small > > (typically < 50K) so I don't see bandwidth problem. As an alternative an > > co-worker of mine showed me a trick to split up the graphics into small > > chunks. Large white spaces are s single byte that is blown up to fill the > > whole area. Probably not necessary, but something to consider. At any > > rate, these animations would probably be something for those people who > > have registered, whereas online command output would be for "paying" > > members as they would need to download it to their local systems. Members > > would still have the current updates and we could potentially have things > > like the quizes only available for registered members since I plan to > > keep track of the user's score, completed sections and so on. Hmmm. There > > are many possibilities. > > This is part of the views. And views can be user based, right? So a registed user could get the animation, where an anonymous person does not? Is that right? > > Have you had a chance to go through the concept and data description? I > > would really like some input as quickly as possible to finalize the > > format. Luan will need to know what the final format is before she can > > convert the CSV files. She is already working on the code (I think), but > > is using the "suggestions" I provided. Her code shouldn't change too > > much, however I would like to get moving. > > No I have been a bit busy and took a few hours off for myself this > weekend. I will get on them tonight. Get Luan in touch with me and move > this conversation to the list. If you don't have one I can create a list > for us on my system. I have access to my lists via Perl as well. Users > can use a web form to sign up. Please, please, please read the concept and data model. Also, if I don't hear back from the others who wanted to do the XML code, then I will work together with Luan on it. Assuming she wants to keep going. ;-) > I will be back in touch. Check the site tomorrow night I should have the > FAQ converted to MySQL by them. Will do. Regards, Jim -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-17 13:31:04
|
Hi All! Just a friendly remember that I would like (read: need) comments about the Linkbat data model. In reviewing the descriptions I have come up with several questions that need to be discussed and answered: - Does this data model make sense? - Are the other KU types that w should include? - Are there any relationships that its missing? - Do we need to have an ID within the <KnowledgeUnit> tag? Originally, it was intended to make sure each KU was unique and to use as a reference point. However, a unique ID can be generated when the XML files are parsed during the conversion to CSV or database. - Do we need to include a definition for the Page KU within the same files as the other KUs? Eventually we will be converting the HTML pages to XML and all of the necessary tags can be included in that file. At this point, I think it would be a lot of work to convert the HTML files to XML and parse them. Since we can automate the creation of Page KUs, I think we should go ahead and have the Page KU definition seperate from the actual content for the time being. That is, we should include the definition with the other KUs **for now**. - How do we define KU sub-types? For example, the "man-page" sub-type to the MoreInfo KU. Should we have a <Type> and <SubType> tag or is the sub-type an attribute of the Type tag: <Type sub-type="">? We also need to account for the specific location and other info. For other sites we need a location; for books, we need the ISBN, author and perhaps publisher? Should these be attributes of the <Type> tag or seperate tags? - Should be include a <Language> tag? Eventually we will have everything translated into other languages. We could create a special tag to identify the language the would allow us to have a single knowledge base in multiple language. Or we simply keep the languages seperate. - Should be include a <SkillLevel> tag? The content is getting to the point where is a lot of it might be overwhelming to newbies. Therefore, it would be nice to be able to configure the system to only display KUs that are appropriate for their skill level. This could be accomplished with a <SkillLevel> tag. Questions could also be marked with this tag and only questions appropriate for that skill level would be asked. - Do command KU need a <SeeAlso> tag? Like the man-pages the point to more information, but that's the point of simply referencing another KU. If it is important enough to mention as a "See Also", then it should have a KU, right? - When we create the Glossary KUs, should we include the page references? Currently, I have a script that searches through the content pages and creates a list of all pages that contain each glossary term. This is more efficient than including the PageRef within the Glossary KU. I see no need and it it is one more place to forget something. For example, if you add a new content page we could forget to update the appropriate glossary KUs. Scanning the content pages seams more efficient. MAYBE it would be necessary to create a "primary" page for the glossary entries, but I cannot convince myself that it is necessary. Well, I am sure there are more questions to answer, but these are the ones that seem to be most pressing. Regards, Jim -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: Hal F. G. <hal...@ur...> - 2002-11-15 13:14:33
|
-----Original Message----- From: James Mohr [mailto:lin...@ji...] Sent: Wednesday, November 13, 2002 3:41 PM To: lin...@li... Cc: Suriya Narayanan M S; hal gottfried; John Purdy; Rama; A .; Russell Cotten Subject: Linkbat Project I've got a couple of administrative issues to cover. I have not yet added everyone to the developers list because not everyone has confirmed that they want me to do it. Check out the project page on SourceForge to see if you are listed as a developer (http://sourceforge.net/projects/linkbat/). If you are not listed and do want to be, let me know. I will then put you on the developers mailing list, as well. HG> You have not listed me but I would like to be listed there as well. Most people do not have a public skill profile on SourceForge. That's okay, but I would like to know what exprience and skills people have, plus exactly what areas they want to work on. There is plenty of work in multiple areas, so don't be shy! Check out the task list on SourceForge. Most are related to content and not programming. HG> I will go fill that out for you today. Also please let me know what style of management you want from me. Do you expect me to hand out assignments to you or will you simply grab one? Do you need regular reminders? I realize that we are all doing this when other projects permit and sometimes we loose track of time. I will fit myself to your style. Since this is a team project, I really don't mind if you tell me that you think something is wrong. I believe that we are headed in the right direction, but please tell me before we drive over a cliff. HG> I am working on quite a few projects so you can tell me if you would like what part you'd like me to work on, or I can just grab something. You can send this kind of "personal" info to me directly, instead of the list. However, technical/project related info, I would like to go to the list. That way if someone else can/should comment on any topic, they can. Please give me an idea of you availability. I don't need the specific number of hours you can work per week, but I would really like to know what other projects you have going. Are you in school, working full time, whatever? If can no longer contribute for whatever reason please let me know, as soon as possible. If you have to take a break for whatever reason, please let me know, as well. I know this is an all volunteer project and I am very grateful for the help. However, communication is the only way this can be a success, so please communicate. HG> I am a corperate IT Trainer and I am currently on the road about 20 - 25 days a month but I can provide at least 4-5 hours per week, for this as I see it now. Also please let me know where you live. I am in Germany, so I am several hours ahead of the people in North America. Aside from simply being curious, it helps me to know when you will be reading email, etc. If I send a message early in the morning (before I go to work) , some of you might still be awake. If I send it later in the morning, I will expect to wait a while before you even read it. HG> I live in Pittsburgh USA, I used to live in Germany though. Up to this point, I have been encouraging visitors to support the tutorial by making a donation to the Jessie Arbogast Medical Fund. Since we are a team, I think we should all have an equal say in how to distribute any donations. At this point, I do **not** what to soliticate donations to pay us. Note the phrase "at this point". I don't want to count our chickens before they hatch. However, I think we can have a "care-ware" site, where we encourage donations to needy causes. Each person can make a suggestion and we can create a "donations page" or whatever to point people to the right place. Also, if the organization your want to support has a banner ad that we can use, it is really easy to add it to the system. There are a number already up. Now for the technical stuff. As soon as possible, please, please read both the Concept and Detailed KU Description on the linkbat site (http://linkbat.sourceforge.net). I want to finalize this as soon as possible. One nice thing about storing in XML is that we can more easily change the structure, but I would like to have as many questions answered and decisions made as possible before we continue. I would like to start the hardcore work no later than the beginning of next week. One person had expressed interest in helping, but could not shake the idea of linkbat as being a huge Linux glossary. It ain't! Although I am not trying for the ultimate knowledge base with artificial intelligence that can answer every question you input, I feel this data model can simplify access the knowledge necessary to get the answer (if not providing the answer directly). Also keep in mind that a knowledge base is useless without content and we can always use content. So, if there are content related tasks that you could do and need a break from coding or whatever, **please** grab one. Also, the tutorial can always use some editorial review (spellchecking, etc). Even if it is a single correction, it make the tutorial a little better. Once we have the data structure we can start moving the data from the current CSV into XML. While that is happening we can also work on the code to do the conversion back into CSV as well as into an database (i.e. SQL). The reason for the conversion back into CSV (as mentioned in the linkbat concept) is speed when online. XML is ten times larger and takes that much larger to read. Plus the indexes need to be created and there is no way we can do them in real-time. (Or does anyone see that differently?) Three people have said they could help doing the conversion. That might be overkill, but doing a quick conversation means we have the XML files for the coding. That means moving sooner to other tasks. Once I get people's comments I will "finalize" the model and create a DTD. I don't think it is necessary to wait to convert the data to XML before the DTD is finished. The DTD is just a stylized description of things we already decided upon. However for someone providing us content in the future, it would be useful. I guess that's it for now. I am sure I will have more to say real soon. Best Regards, Jim -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |
From: James M. <lin...@ji...> - 2002-11-13 19:13:05
|
Hi All! First, I want to welcome you all to the team. I really appreaciate your willingness to help. I am excited about this project and I am sure that it will be a success. Considering how successul the Linux Tutorial has been already, it can only get better. I've got a couple of administrative issues to cover. I have not yet added everyone to the developers list because not everyone has confirmed that they want me to do it. Check out the project page on SourceForge to see if you are listed as a developer (http://sourceforge.net/projects/linkbat/). If you are not listed and do want to be, let me know. I will then put you on the developers mailing list, as well. Most people do not have a public skill profile on SourceForge. That's okay, but I would like to know what exprience and skills people have, plus exactly what areas they want to work on. There is plenty of work in multiple areas, so don't be shy! Check out the task list on SourceForge. Most are related to content and not programming. Also please let me know what style of management you want from me. Do you expect me to hand out assignments to you or will you simply grab one? Do you need regular reminders? I realize that we are all doing this when other projects permit and sometimes we loose track of time. I will fit myself to your style. Since this is a team project, I really don't mind if you tell me that you think something is wrong. I believe that we are headed in the right direction, but please tell me before we drive over a cliff. You can send this kind of "personal" info to me directly, instead of the list. However, technical/project related info, I would like to go to the list. That way if someone else can/should comment on any topic, they can. Please give me an idea of you availability. I don't need the specific number of hours you can work per week, but I would really like to know what other projects you have going. Are you in school, working full time, whatever? If can no longer contribute for whatever reason please let me know, as soon as possible. If you have to take a break for whatever reason, please let me know, as well. I know this is an all volunteer project and I am very grateful for the help. However, communication is the only way this can be a success, so please communicate. Also please let me know where you live. I am in Germany, so I am several hours ahead of the people in North America. Aside from simply being curious, it helps me to know when you will be reading email, etc. If I send a message early in the morning (before I go to work) , some of you might still be awake. If I send it later in the morning, I will expect to wait a while before you even read it. Up to this point, I have been encouraging visitors to support the tutorial by making a donation to the Jessie Arbogast Medical Fund. Since we are a team, I think we should all have an equal say in how to distribute any donations. At this point, I do **not** what to soliticate donations to pay us. Note the phrase "at this point". I don't want to count our chickens before they hatch. However, I think we can have a "care-ware" site, where we encourage donations to needy causes. Each person can make a suggestion and we can create a "donations page" or whatever to point people to the right place. Also, if the organization your want to support has a banner ad that we can use, it is really easy to add it to the system. There are a number already up. Now for the technical stuff. As soon as possible, please, please read both the Concept and Detailed KU Description on the linkbat site (http://linkbat.sourceforge.net). I want to finalize this as soon as possible. One nice thing about storing in XML is that we can more easily change the structure, but I would like to have as many questions answered and decisions made as possible before we continue. I would like to start the hardcore work no later than the beginning of next week. One person had expressed interest in helping, but could not shake the idea of linkbat as being a huge Linux glossary. It ain't! Although I am not trying for the ultimate knowledge base with artificial intelligence that can answer every question you input, I feel this data model can simplify access the knowledge necessary to get the answer (if not providing the answer directly). Also keep in mind that a knowledge base is useless without content and we can always use content. So, if there are content related tasks that you could do and need a break from coding or whatever, **please** grab one. Also, the tutorial can always use some editorial review (spellchecking, etc). Even if it is a single correction, it make the tutorial a little better. Once we have the data structure we can start moving the data from the current CSV into XML. While that is happening we can also work on the code to do the conversion back into CSV as well as into an database (i.e. SQL). The reason for the conversion back into CSV (as mentioned in the linkbat concept) is speed when online. XML is ten times larger and takes that much larger to read. Plus the indexes need to be created and there is no way we can do them in real-time. (Or does anyone see that differently?) Three people have said they could help doing the conversion. That might be overkill, but doing a quick conversation means we have the XML files for the coding. That means moving sooner to other tasks. Once I get people's comments I will "finalize" the model and create a DTD. I don't think it is necessary to wait to convert the data to XML before the DTD is finished. The DTD is just a stylized description of things we already decided upon. However for someone providing us content in the future, it would be useful. I guess that's it for now. I am sure I will have more to say real soon. Best Regards, Jim -- --------------------------------------- "Be more concerned with your character than with your reputation. Your character is what you really are while your reputation is merely what others think you are." -- John Wooden --------------------------------------- Be sure to visit the Linux Tutorial: http://www.linux-tutorial.info |