Note: Many sections of this Wiki Page may have references to other websites for more information. Right Click on these Links and Open them in New Tab for comfortable reading and return back to this Wiki Page.
I am glad to announce the release of tabulate.zip version 2.11 with features like:
I am glad to announce the release of tabulate.zip version 2.10 with features like:
For using the awk command on windows natively, please watch : https://youtu.be/dakKlEkrOP4
I am glad to announce the release of tabulate.zip version 2.8 with features like:
I am glad to announce the release of tabulate.zip version 2.6 with features like:
I am glad to announce the release of tabulate.zip version 2.5 with features like:
I am glad to announce the release of tabulate.zip version 2.4 with features like:
I am glad to announce the release of tabulate.zip version 2.2 with features like:
A video demo here : https://www.youtube.com/watch?v=ZLQ8zRIzORM
I am glad to announce the release of tabulate.zip version 2.1 with features like:
I am glad to announce the release of tabulate.zip version 2.0 with features like:
I am glad to announce the release of tabulate.sh version 1.0 with features like conversion of any input data to html table.
It is very easy to process tabular data files with tabulate.sh, as shown in the examples on the main page. But, how to process files, where data is in multiple rows, instead of columns.
I will show a real life example that I faced for the same. I had multiple software info files ( i.e. text files ending with .info ) which were having rows, with a repeating data pattern like this:
Title:
Description:
Version:
Author:
Original-site:
Copying-policy:
Size:
Extension_by:
Tags:
Comments:
Change-log:
Current:
I had to write a small script which
declares a multiple Search_Pattern Variable, with each pattern starting with a character ( i.e. ^ ) and the above tags. You can include anything after the ^ character, including spaces, if your data requires.
reads all .info files, searches for lines with the Search_Pattern, substitutes the first delimiter found ( i.e. : here with % ), then picks up the data alone ( i.e. $2 ) after trimming leading / trailing spaces, removes leading delimiters, and finally prints every 8 lines to a single line, with the unique %%%%%%%% delimiter in between columns. Note : if your data does not contain any tags and repeats itself say every 7 rows, then just use this, for the second command : pr --columns=7 --across --length=1 --separator="%%%%%%%%" .info > allinfo.txt
send the allinfo.txt generated to tabulate.sh for html table conversion
Action Required:
On your terminal / console :
Search_Pattern='^Title|^Description|^Version|^Original-site|^Copying-policy|^Size|^Extension_by|^Tags'
cat *.info | env LC_ALL=C egrep "$Search_Pattern" | awk '{sub(/:/,"%")}1' | awk 'BEGIN{FS="%";OFS="%%%%%%%%"} {$1=""; gsub(/^[ \t]+|[ \t]+$/, "", $2)}1' | sed 's/^%%%%%%%%//' | pr -s'%%%%%%%%' -8 -l1 > allinfo.txt
cat allinfo.txt | ./tabulate.sh -d "%%%%%%%%" -t "My Report" -h "My Header" > allinfo.html
For even more complex data patterns, refer to the following webpages:
Also, exporting the data from your customer's software to a text file for processing, could be challenging, as well. Search Google with software name export data. If no known methods exist, then it is possible to create a Text File Printer from Control Panel and print to it from the software, thereby producing a text file output.
If your input data file already contains URLs or if you are planning to include columns that are referring to some https://www.domainname.com/filename or https://localmachinename/filename then you need to hyperlink them, to enable click and viewing of these links, in new tabs.
Latest Update : Sample.xlsx file is now included in the tabulate.zip itself. It contains Formulas for Hyperlinking Various Data Fields. Copying these formulas into your input data Spreadsheets, editing them to match the column names / numbers, then Copy / Paste Special - Values Only, and then Delete Unnecessary Columns and then Save As .CSV Type file with a Custom Delimiter like | ( pipe ) and then pass its data to tabulate.sh as usual. If you are new to Paste Special, then watch youtube videos : https://www.youtube.com/watch?v=cp1hVf5taTs ( for Libreoffice Calc ) and https://www.youtube.com/watch?v=PLRQ2YCQAAM ( for Excel )
Also, you can edit the tabulate.sh script / input data files / output html files, with a good text / code editor like sublime text ( https://www.sublimetext.com )
For a complete reference of options that can be used inside these html tags, please visit : https://www.w3schools.com/tags/default.asp
Also refer : https://stackoverflow.com/questions/62933581/how-to-make-images-size-equal-in-table-html
The CSS codes can be added to the style type="text/css"
section of the tabulate.sh script.
The below steps are given for information purpose only.
="<img src="&""""&A1&""""&">" where A1 contains picture1.png
, in the input data file, and then dos2unix somefile.csv
to convert line endings from windows to unix type.Eg. For displaying a picture in a table cell, the html syntax is:
<tr>
<td>
<img src="picture1.png">
</td>
</tr>
Only the line beginning with <img src="picture1.png">
should be prepared on a spreadsheet program. The rest of it like <tr><td></td> </tr>
etc. will be taken care of by tabulate.sh
Note: Testing some sample data with URLs is a must, before you actually start working on real life projects. You can open the output.html file in a text editor to check for the syntax generated, edit and save it and then view it on the browser. You can also dynamically alter the syntax using sed substitute command, described in a Ticket raised on this website above and also given in a section below.
For long data columns, which run out of page width, you need to give them a fixed size. For this, search for fixed-width in tabulate.sh ( atleast twice ) and follow instruction there in. After making suitable changes, Save the file and re-run it, as before.
To measure column-width in pixels, use a pixel ruler, like in firefox : https://addons.mozilla.org/en-US/firefox/addon/web-developer/ This contains a pixel ruler in its miscellaneous section. Also, pixel rulers are available as extension on Google Chrome browser : https://chrome.google.com/webstore/search/pixel%20ruler
If you wish to give a new appearance ( colors ) to each page, you generate, search for changing-colors in tabulate.sh ( many times ) and change colors there in. After making suitable changes, Save the file and re-run it, as before.
Both Firefox and Google Chrome has extensions called Color Picker, which can be used to know the Color Code used on any Web Page:
https://addons.mozilla.org/en-US/firefox/addon/color-picker-webext/
https://chrome.google.com/webstore/search/color%20picker
The list of color codes can be obtained from :
https://www.computerhope.com/htmcolor.htm#color-codes
and
https://www.w3schools.com/colors/colors_names.asp
Suppose you want to insert a paragraph in your generated html file, then open the file with a text editor, then add paragraphs, as shown in the following link : https://www.w3schools.com/Html/html_colors.asp especially those that start with <p
and After making suitable changes, Save the file and re-run it, as before.
Similarly, you can add blank lines : https://www.w3schools.com/tags/tag_br.asp
if your customer already has a domain name, purchased from a web hosting company, then the company provides a link to login and upload files. For eg. see https://in.godaddy.com/help/upload-files-3239
We can then create a hyperlink on its main index.html file or default.html file etc. ( by downloading & uploading it ) as specified in this tutorial : https://www.w3schools.com/html/html_links.asp
Otherwise, purchase a new domain name, along with web hosting, from any popular web hosting company of your area ( check google ) and login to your domain link, that they provide and upload files. If you upload your file with name index.html or default.html etc. , then it will be served automatically on visiting http://www.mydomainname.com
If you only need a local network webserver, then there are lots of free and popular webservers available for install : https://www.webdevelopersnotes.com/web-servers-list
Once installed, search google for webserver's document root location and transfer your generated html files there. You can now access your html file on the browser by providing address like http://machinename/filename.html from other machines too.
Also see : https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_image_link
Also learn html from : https://www.w3schools.com/html/ in your spare time. It will help.
Publishing & Maintaining data of schools, colleges, organizations, wholesale shops / supermarkets etc. on their intranet or internet, can be taken up part-time or full-time, with this tabulate script.
For eg. we can setup an information kiosk of their products, along with its details, in their own premises, for providing an easy search facility, for large amount of data, deployed and maintained from a centralized webserver location.
Please do share this news about tabulate script on your social media websites, to help struggling people, find new job opportunities.
The latest version of the tabulate script supports nearly all available and popular operating systems. So the below mentioned method may not be required.
To run the script, all you need is a small and free linux operating system like Puppy Linux ( http://puppylinux.com/ ) installed on to a small pen drive ( https://www.youtube.com/watch?v=QpzP2I8QPZE ) and boot from pen drive ( https://www.youtube.com/watch?v=xzd73vS9WXo and https://www.youtube.com/watch?v=IU0b48XpNsU ) on any desktop or a laptop computer. Also extract the files in tabulate.zip file to the pen drive, for using later. Start Puppy Linux, launch console/terminal on it, and copy/execute the tabulate.zip files on the data folder. The output html file generated must be right clicked on the file manager and choose Open with default viewer.
If your anti-virus scanner does not allow writing puppy linux files, then you can try my other project MultiBootUsb-Live, which can be downloaded here : https://sourceforge.net/projects/multibootusb-live/ and written to a DVD and boot through it. Watch https://www.youtube.com/watch?v=CNgVxKvOdxI for writing isofiles on to pen drives.
The tabulate.sh script's html output is known to work with most browser softwares like firefox, chrome, opera, safari, palemoon etc. However, very very old browser softwares will have a problem with the integrated search functionality, created with javascript.
For those users, using such older browsers, are encouraged to install and use newer versions of browser softwares, like the ones listed above.
The tabulate.sh script comes in the form of a zip file, with a search.png file, along with it. This search.png has to be copied to the locations, where you are copying the html generated by the script. This will help in displaying a nice search icon, on the left of the search input box.
The html file generated carries a default charset setting, in its fourth line, as follows : charset="ISO-8859-1"
If your data is still not getting displayed properly, then change this charset, using a text editor, to suit your input data. List of charsets are
given in the following links :
http://www.madore.org/~david/computers/unicode/cstab.html
https://www.iana.org/assignments/character-sets/character-sets.xhtml
A good alternate choice could be : charset="UTF-8"
Also, you can edit the tabulate.sh script and search for and change the charset setting temporarily or permanently.
Another technique to identify charset for a particular language is to do Google Search for that language websites, open them one by one and right click them to View Page Source ( on Google Chrome Browser ) and search for charset setting used on that page.
Special Characters List for complex delimiters are given in this wiki page here : [Special_Characters]
To achieve this, you have to add new input column with rating of High, Medium or Low. For this, read the wiki page : [How to Use Comparison Operators with Awk in Linux]
Before you write the final output.html file, you can do dynamic substitutions, by passing the output through a pipe, i.e. | and using tools like sed, awk, tr or grep commands, along with their options and pattern.
Also, you can modify your input data file dynamically too, using a pipe and these same tools. Eg. cat input.txt | sed or awk or tr or grep, along with their options and pattern.
For this, watch these easy youtube tutorial videos:
awk command:
https://www.youtube.com/watch?v=X-3AXAsU3Wo
https://www.youtube.com/watch?v=FbSpuZVb164
https://www.youtube.com/watch?v=9YOZmI-zWok
grep command:
https://www.youtube.com/watch?v=TvkBv1iUf80
Also, download the awk tutorial attachments from the top of this wiki page.
I have additionally collected and created wiki pages for these commands like :
[collection of linux one-liner commands]
[tr command in Unix-Linux with examples]
[AWK one-liner collection]
[grep command with examples]
The tickets or discussion webpages on this website provide additional inputs. So, please view them from time to time.
Example:
df -h | { cat ; echo ; } | sed 's/Mounted on/Mounted_on/g' | grep -v "^map" | ./tabulate.sh -d " " -t "My System" -h "Disk Free as of $(date '+%Y/%m/%d %H:%M:%S')" > diskfree_iv.html
This prints the header along with the date and time :
Disk Free as of 2022/03/09 04:01:36
Google Search for appropriate commands like this : linux terminal print date and time command
and enclose the commands within the $( ) structure, like the example above.
Another Example:
Gathering and tabulating CPU Statistics:
top -a -n20 | awk 'FNR>=11 && FNR<=31{print $0};FNR==31{exit}' > cpustat.txt
cat cpustat.txt | { cat ; echo ; } | ./tabulate.sh -d " " -t "My System" -h "CPU STATS as of $(date '+%Y/%m/%d %H:%M:%S')" > cpustat.html
LibreOffice Calc allows you to change field and string separator characters while saving as TEXT CSV type on Windows, Linux and MAC Operating Systems. It also supports wide range of file formats like Spreadsheets and Text files and can help in opening and converting from one to another format.
For this, watch the youtube video :
https://youtu.be/s8Uby1Rex_s for LibreOffice Calc File Save With Any Custom Delimiter
The official website:
LibreOffice is free to download for Linux, Windows and Mac.
https://www.libreoffice.org/download/
Google Search for :
to get the Portable Versions.
Search for scalc or soffice in all the subfolders
and launch these programs.
Also, for the Custom Puppy Linux Live CD mentioned at the top of this Wiki Page, LibreOffice.sfs files are available for download in the same location.
Note : Use field separators like | ( pipe ) or # ( hash ) etc. Verify that this character is not in your data already, with search on cell content.
To generate a CSV file from an Excel Sheet with Pipe Delimiter instead of Comma, please watch : https://www.youtube.com/watch?v=RM6L8sRIPpU
For introducing a Dynamically Sortable Table with Javascript, just download the Dynamically_Sort_Any_Table.zip
attachment, from the top of this Wiki Page, extract it, and then View the README_FIRST file for easy instructions.
Always, test your new features on different internet browsers on different operating systems, as well, to ensure that your
users get the best performance and features.
If you need to pre-sort your input data text files, before passing on the tabulate command, then refer these articles:
For Windows Users : You can get the needed binaries like head, tail, sort etc. from my other project :
https://sourceforge.net/projects/portable-linux-binary-windows/
Wiki: AWK one-liner collection
Wiki: How to Use Comparison Operators with Awk in Linux
Wiki: Special_Characters
Wiki: collection of linux one-liner commands
Wiki: grep command with examples
Wiki: tr command in Unix-Linux with examples