Menu

Home

Nathan S.R.
Attachments
HTML_tags.html (56646 bytes)
awk.zip (1242847 bytes)
html_tags.pdf (114972 bytes)
hyperlinks_newtab.pdf (16687 bytes)

Note: Many sections of this Wiki Page may have references to other websites for more information. Right Click on these Links and Open them in New Tab for comfortable reading and return back to this Wiki Page.

tabulate.zip version 2.11 released on 2022-08-16

I am glad to announce the release of tabulate.zip version 2.11 with features like:

  1. As the Windows Version of this script was occasionally generating a zero byte file, because of the usage of pipes,
    I have avoided them totally, in this version.
  2. This is also carried out in my other new project, which generates html charts, mentioned in the last release below.

tabulate.zip version 2.10 released on 2022-07-25

I am glad to announce the release of tabulate.zip version 2.10 with features like:

  1. A seperate For_Windows folder has been added with scripts and utilities to run on various windows systems
  2. A readme file inside that folder explains how to run the various scripts therein
  3. Prefer using Double Quotes instead of Single Quotes, while using most of these Binaries / Utilities.
  4. You can get additional Binaries / Utilities from my other project : https://sourceforge.net/projects/portable-linux-binary-windows/
  5. Please refer my other new youtube video : https://youtu.be/5Y8SdI-c7uU for learning about the features of this version.
  6. For Converting an Excel Sheet OR Command Outputs of Servers OR Database Text Files to a Professional HTML Chart, Please refer my other project : https://sourceforge.net/projects/command-output-to-html-chart/
  7. Also related to the working of this windows folder version is this article : https://stackoverflow.com/questions/9378972/how-do-i-detect-and-delete-a-file-if-it-is-empty-using-a-windows-batch-file
  8. For Command Output to Mathematical Statistic, please refer my other project : https://sourceforge.net/projects/command-output-to-statistics/

For using the awk command on windows natively, please watch : https://youtu.be/dakKlEkrOP4

tabulate.zip version 2.8 released on 2022/04/14

I am glad to announce the release of tabulate.zip version 2.8 with features like:

  1. The Server_Sample_Code_For_Scripts.sh has been improved to show headings easily

tabulate.zip version 2.6 released on 2022/04/01

I am glad to announce the release of tabulate.zip version 2.6 with features like:

  1. Index page generation is now automated with the release of the generate_default_index_webpage.sh script
  2. Please view its own README file for instructions

tabulate.zip version 2.5 released on 2022/03/31

I am glad to announce the release of tabulate.zip version 2.5 with features like:

  1. Server* files are included for Server Side Scripting and Automation
  2. With this, the job opportunities multiply for those willing to take up work on the Server / Client Side

tabulate.zip version 2.4 released on 2022/03/23

I am glad to announce the release of tabulate.zip version 2.4 with features like:

  1. tabulate.sh script can now handle HTML Syntax / Tags in the input text files directly
  2. Sample Files are now included in the tabulate.zip file itself
  3. Sample.xlsx file contains Formulas for Hyperlinking Various Data Fields. Copying these formulas into your input data Spreadsheets, editing them to match the column names / numbers, then Copy / Paste Special - Values Only, and then Delete Unnecessary Columns and then Save As .CSV Type file with a Custom Delimiter like | ( pipe ) and then pass its data to tabulate.sh as usual. If you are new to Paste Special, then watch youtube videos : https://www.youtube.com/watch?v=cp1hVf5taTs ( for Libreoffice Calc ) and https://www.youtube.com/watch?v=PLRQ2YCQAAM ( for Excel )
  4. View the Sample.csv and Sample.html to know the HTML features
  5. Also view the LibreOffice Calc File Save With Any Custom Delimiter.mp4 video in the tabulate folder
  6. Libreoffice calc spreadsheet program can be installed on all major operating systems freely. Refer section 16 below for it

tabulate.zip version 2.2 released on 2022-03-07

I am glad to announce the release of tabulate.zip version 2.2 with features like:

  1. Blank Data Cells or Fields are represented properly in the HTML Table
  2. A custom Puppy Linux Operating System has been created for this purpose of running the script on various client computers. You can download it here : https://sourceforge.net/projects/command-output-to-html-table/files/OS/ Wherever possible, give preference to the 64-bit system over the 32-bit system for better performance and features. You can copy and paste the code easily on the Terminal / Console program on this operating system, by using the keyboard shortcut keys like Ctrl+Insert for copying selected text and Shift+Insert for pasting copied text OR ELSE Middle Click Mouse Button / Wheel For Pasting on Terminal / Console. Also, you can choose Edit Menu - Paste or else Right Click Menu - Paste. Press F4 to launch the Terminal from any Folder on the File Manager. This will set your working directory automatically. Use your favorite text editor to edit commands as required and then copy / paste it on the terminal / console and then press the enter key to execute it.

A video demo here : https://www.youtube.com/watch?v=ZLQ8zRIzORM

tabulate.zip version 2.1 released on 2021-09-25

I am glad to announce the release of tabulate.zip version 2.1 with features like:

  1. /dev/stdin renamed to - character to support all terminals

tabulate.zip version 2.0 released on 2018-07-04

I am glad to announce the release of tabulate.zip version 2.0 with features like:

  1. Dynamic Searching / Filtering of all Rows, based on an input
  2. International Character Set Support
  3. Colored backgrounds for a pleasing appearance
  4. Hover color for easy identification of any row

tabulate.sh version 1.0 released on 2018-06-29

I am glad to announce the release of tabulate.sh version 1.0 with features like conversion of any input data to html table.

Some Technical Challenges:

Section 1: Processing Non-Tabular data files with tabulate.sh

It is very easy to process tabular data files with tabulate.sh, as shown in the examples on the main page. But, how to process files, where data is in multiple rows, instead of columns.

I will show a real life example that I faced for the same. I had multiple software info files ( i.e. text files ending with .info ) which were having rows, with a repeating data pattern like this:

Title:
Description:
Version:
Author:
Original-site:
Copying-policy:
Size:
Extension_by:
Tags:
Comments:
Change-log:
Current:

I had to write a small script which

  1. declares a multiple Search_Pattern Variable, with each pattern starting with a character ( i.e. ^ ) and the above tags. You can include anything after the ^ character, including spaces, if your data requires.

  2. reads all .info files, searches for lines with the Search_Pattern, substitutes the first delimiter found ( i.e. : here with % ), then picks up the data alone ( i.e. $2 ) after trimming leading / trailing spaces, removes leading delimiters, and finally prints every 8 lines to a single line, with the unique %%%%%%%% delimiter in between columns. Note : if your data does not contain any tags and repeats itself say every 7 rows, then just use this, for the second command : pr --columns=7 --across --length=1 --separator="%%%%%%%%" .info > allinfo.txt

  3. send the allinfo.txt generated to tabulate.sh for html table conversion

Action Required:

  1. Copy the below code block ( in between #### ) to a new text file called rows_to_columns.sh and edit it as required, for your search_pattern, data file names, delimiters, every n lines, etc. Save it to the folder, containing your data files.

On your terminal / console :

  1. cd my_data_files_folder
  2. chmod +x ./rows_to_columns.sh
  3. ./rows_to_columns.sh
######################### code begins here

Search_Pattern='^Title|^Description|^Version|^Original-site|^Copying-policy|^Size|^Extension_by|^Tags'

cat *.info | env LC_ALL=C egrep "$Search_Pattern" | awk '{sub(/:/,"%")}1' | awk 'BEGIN{FS="%";OFS="%%%%%%%%"} {$1=""; gsub(/^[ \t]+|[ \t]+$/, "", $2)}1' | sed 's/^%%%%%%%%//' | pr -s'%%%%%%%%' -8 -l1 > allinfo.txt

cat allinfo.txt | ./tabulate.sh -d "%%%%%%%%" -t "My Report" -h "My Header" > allinfo.html

######################### code ends here

For even more complex data patterns, refer to the following webpages:

  1. https://stackoverflow.com/questions/25317736/convert-rows-to-columns-with-bash
  2. https://unix.stackexchange.com/questions/258361/convert-rows-to-columns
  3. https://unix.stackexchange.com/questions/155709/use-sed-or-awk-to-move-rows-to-columns
  4. https://unix.stackexchange.com/questions/79642/transposing-rows-and-columns
  5. If the above links do not answer your requirements, then you can ask questions in these forums

Also, exporting the data from your customer's software to a text file for processing, could be challenging, as well. Search Google with software name export data. If no known methods exist, then it is possible to create a Text File Printer from Control Panel and print to it from the software, thereby producing a text file output.

If your input data file already contains URLs or if you are planning to include columns that are referring to some https://www.domainname.com/filename or https://localmachinename/filename then you need to hyperlink them, to enable click and viewing of these links, in new tabs.

Latest Update : Sample.xlsx file is now included in the tabulate.zip itself. It contains Formulas for Hyperlinking Various Data Fields. Copying these formulas into your input data Spreadsheets, editing them to match the column names / numbers, then Copy / Paste Special - Values Only, and then Delete Unnecessary Columns and then Save As .CSV Type file with a Custom Delimiter like | ( pipe ) and then pass its data to tabulate.sh as usual. If you are new to Paste Special, then watch youtube videos : https://www.youtube.com/watch?v=cp1hVf5taTs ( for Libreoffice Calc ) and https://www.youtube.com/watch?v=PLRQ2YCQAAM ( for Excel )

Also, you can edit the tabulate.sh script / input data files / output html files, with a good text / code editor like sublime text ( https://www.sublimetext.com )

For a complete reference of options that can be used inside these html tags, please visit : https://www.w3schools.com/tags/default.asp

Also refer : https://stackoverflow.com/questions/62933581/how-to-make-images-size-equal-in-table-html
The CSS codes can be added to the style type="text/css" section of the tabulate.sh script.

The below steps are given for information purpose only.

  1. For Hyperlinking, see the attached files called html_tags.pdf and hyperlinks_newtab.txt and then
  2. Concatenate the additional syntax needed with the following formula, in programs like Excel, Libreoffice Calc or Gnumeric : Eg. ="<img src="&""""&A1&""""&">" where A1 contains picture1.png, in the input data file, and then
  3. Save file as CSV or TEXT type ( refer Section 16 of this Wiki Page ) and then
  4. Open the file in Sublime Text Editor, change line endings using menu item View / Line Endings to Unix, Save the File and then
  5. Pass that revised data to tabulate.sh
  6. Instead of Sublime Text Editor, you can use the Terminal Command on Puppy Linux dos2unix somefile.csv to convert line endings from windows to unix type.

Eg. For displaying a picture in a table cell, the html syntax is:

 <tr> 
    <td>
    <img src="picture1.png">
  </td>
</tr>

Only the line beginning with <img src="picture1.png"> should be prepared on a spreadsheet program. The rest of it like <tr><td></td> </tr> etc. will be taken care of by tabulate.sh

Note: Testing some sample data with URLs is a must, before you actually start working on real life projects. You can open the output.html file in a text editor to check for the syntax generated, edit and save it and then view it on the browser. You can also dynamically alter the syntax using sed substitute command, described in a Ticket raised on this website above and also given in a section below.

Section 3: Fixing Column-Width to fit Page-Width

For long data columns, which run out of page width, you need to give them a fixed size. For this, search for fixed-width in tabulate.sh ( atleast twice ) and follow instruction there in. After making suitable changes, Save the file and re-run it, as before.

To measure column-width in pixels, use a pixel ruler, like in firefox : https://addons.mozilla.org/en-US/firefox/addon/web-developer/ This contains a pixel ruler in its miscellaneous section. Also, pixel rulers are available as extension on Google Chrome browser : https://chrome.google.com/webstore/search/pixel%20ruler

Section 4: Changing Colors

If you wish to give a new appearance ( colors ) to each page, you generate, search for changing-colors in tabulate.sh ( many times ) and change colors there in. After making suitable changes, Save the file and re-run it, as before.

Both Firefox and Google Chrome has extensions called Color Picker, which can be used to know the Color Code used on any Web Page:

https://addons.mozilla.org/en-US/firefox/addon/color-picker-webext/

https://chrome.google.com/webstore/search/color%20picker

The list of color codes can be obtained from :

https://www.computerhope.com/htmcolor.htm#color-codes

and

https://www.w3schools.com/colors/colors_names.asp

Section 5: Introducing Paragraphs and Line Breaks

Suppose you want to insert a paragraph in your generated html file, then open the file with a text editor, then add paragraphs, as shown in the following link : https://www.w3schools.com/Html/html_colors.asp especially those that start with <p
and After making suitable changes, Save the file and re-run it, as before.

Similarly, you can add blank lines : https://www.w3schools.com/tags/tag_br.asp

Section 6: New "World Wide Web" or Local Network Webserver Deployments

if your customer already has a domain name, purchased from a web hosting company, then the company provides a link to login and upload files. For eg. see https://in.godaddy.com/help/upload-files-3239

We can then create a hyperlink on its main index.html file or default.html file etc. ( by downloading & uploading it ) as specified in this tutorial : https://www.w3schools.com/html/html_links.asp

Otherwise, purchase a new domain name, along with web hosting, from any popular web hosting company of your area ( check google ) and login to your domain link, that they provide and upload files. If you upload your file with name index.html or default.html etc. , then it will be served automatically on visiting http://www.mydomainname.com

If you only need a local network webserver, then there are lots of free and popular webservers available for install : https://www.webdevelopersnotes.com/web-servers-list
Once installed, search google for webserver's document root location and transfer your generated html files there. You can now access your html file on the browser by providing address like http://machinename/filename.html from other machines too.

Also see : https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_image_link
Also learn html from : https://www.w3schools.com/html/ in your spare time. It will help.

Section 7: New Job Opportunities

Publishing & Maintaining data of schools, colleges, organizations, wholesale shops / supermarkets etc. on their intranet or internet, can be taken up part-time or full-time, with this tabulate script.

For eg. we can setup an information kiosk of their products, along with its details, in their own premises, for providing an easy search facility, for large amount of data, deployed and maintained from a centralized webserver location.

Please do share this news about tabulate script on your social media websites, to help struggling people, find new job opportunities.

The latest version of the tabulate script supports nearly all available and popular operating systems. So the below mentioned method may not be required.

To run the script, all you need is a small and free linux operating system like Puppy Linux ( http://puppylinux.com/ ) installed on to a small pen drive ( https://www.youtube.com/watch?v=QpzP2I8QPZE ) and boot from pen drive ( https://www.youtube.com/watch?v=xzd73vS9WXo and https://www.youtube.com/watch?v=IU0b48XpNsU ) on any desktop or a laptop computer. Also extract the files in tabulate.zip file to the pen drive, for using later. Start Puppy Linux, launch console/terminal on it, and copy/execute the tabulate.zip files on the data folder. The output html file generated must be right clicked on the file manager and choose Open with default viewer.

If your anti-virus scanner does not allow writing puppy linux files, then you can try my other project MultiBootUsb-Live, which can be downloaded here : https://sourceforge.net/projects/multibootusb-live/ and written to a DVD and boot through it. Watch https://www.youtube.com/watch?v=CNgVxKvOdxI for writing isofiles on to pen drives.

Section 8: Compatibility with internet browser softwares

The tabulate.sh script's html output is known to work with most browser softwares like firefox, chrome, opera, safari, palemoon etc. However, very very old browser softwares will have a problem with the integrated search functionality, created with javascript.
For those users, using such older browsers, are encouraged to install and use newer versions of browser softwares, like the ones listed above.

Section 9: Copy the search.png to wherever you copy the html file generated

The tabulate.sh script comes in the form of a zip file, with a search.png file, along with it. This search.png has to be copied to the locations, where you are copying the html generated by the script. This will help in displaying a nice search icon, on the left of the search input box.

Section 10: Character Set to be used for your input data

The html file generated carries a default charset setting, in its fourth line, as follows : charset="ISO-8859-1"
If your data is still not getting displayed properly, then change this charset, using a text editor, to suit your input data. List of charsets are
given in the following links :

http://www.madore.org/~david/computers/unicode/cstab.html
https://www.iana.org/assignments/character-sets/character-sets.xhtml

A good alternate choice could be : charset="UTF-8"

Also, you can edit the tabulate.sh script and search for and change the charset setting temporarily or permanently.

Another technique to identify charset for a particular language is to do Google Search for that language websites, open them one by one and right click them to View Page Source ( on Google Chrome Browser ) and search for charset setting used on that page.

Section 11: Miscellaneous

Special Characters List for complex delimiters are given in this wiki page here : [Special_Characters]

Section 12: Searching Records based on Numerical Comparision

To achieve this, you have to add new input column with rating of High, Medium or Low. For this, read the wiki page : [How to Use Comparison Operators with Awk in Linux]

Section 13: Dynamically alter your output html file or Dynamically alter your input data file before further processing

Before you write the final output.html file, you can do dynamic substitutions, by passing the output through a pipe, i.e. | and using tools like sed, awk, tr or grep commands, along with their options and pattern.

Also, you can modify your input data file dynamically too, using a pipe and these same tools. Eg. cat input.txt | sed or awk or tr or grep, along with their options and pattern.

For this, watch these easy youtube tutorial videos:

awk command:

https://www.youtube.com/watch?v=X-3AXAsU3Wo
https://www.youtube.com/watch?v=FbSpuZVb164
https://www.youtube.com/watch?v=9YOZmI-zWok

grep command:

https://www.youtube.com/watch?v=TvkBv1iUf80

Also, download the awk tutorial attachments from the top of this wiki page.

I have additionally collected and created wiki pages for these commands like :

[collection of linux one-liner commands]
[tr command in Unix-Linux with examples]
[AWK one-liner collection]
[grep command with examples]

Section 14: Visit the tickets or discussion webpages on this site from time to time

The tickets or discussion webpages on this website provide additional inputs. So, please view them from time to time.

Section 15: Printing Output of Commands along with the Header

Example:

df -h | { cat ; echo ; } | sed 's/Mounted on/Mounted_on/g' | grep -v "^map" | ./tabulate.sh -d " " -t "My System" -h "Disk Free as of $(date '+%Y/%m/%d %H:%M:%S')" > diskfree_iv.html

This prints the header along with the date and time :

Disk Free as of 2022/03/09 04:01:36

Google Search for appropriate commands like this : linux terminal print date and time command

and enclose the commands within the $( ) structure, like the example above.

Another Example:

Gathering and tabulating CPU Statistics:

top -a -n20 | awk 'FNR>=11 && FNR<=31{print $0};FNR==31{exit}' > cpustat.txt

cat cpustat.txt | { cat ; echo ; } | ./tabulate.sh -d " " -t "My System" -h "CPU STATS as of $(date '+%Y/%m/%d %H:%M:%S')" > cpustat.html

Section 16: LibreOffice Calc ( a spreadsheet program on all major operating systems )

LibreOffice Calc allows you to change field and string separator characters while saving as TEXT CSV type on Windows, Linux and MAC Operating Systems. It also supports wide range of file formats like Spreadsheets and Text files and can help in opening and converting from one to another format.

For this, watch the youtube video :

https://youtu.be/s8Uby1Rex_s for LibreOffice Calc File Save With Any Custom Delimiter

The official website:
LibreOffice is free to download for Linux, Windows and Mac.
https://www.libreoffice.org/download/

Google Search for :

  1. LibreOffice Portable Windows
  2. LibreOffice Portable Linux
  3. LibreOffice Portable Mac

to get the Portable Versions.

Search for scalc or soffice in all the subfolders
and launch these programs.

Also, for the Custom Puppy Linux Live CD mentioned at the top of this Wiki Page, LibreOffice.sfs files are available for download in the same location.

Note : Use field separators like | ( pipe ) or # ( hash ) etc. Verify that this character is not in your data already, with search on cell content.

To generate a CSV file from an Excel Sheet with Pipe Delimiter instead of Comma, please watch : https://www.youtube.com/watch?v=RM6L8sRIPpU

Section 17: Dynamically Sortable Table with Javascript

For introducing a Dynamically Sortable Table with Javascript, just download the Dynamically_Sort_Any_Table.zip
attachment, from the top of this Wiki Page, extract it, and then View the README_FIRST file for easy instructions.

Always, test your new features on different internet browsers on different operating systems, as well, to ensure that your
users get the best performance and features.

Section 18: Pre-Sorting your input data text files

If you need to pre-sort your input data text files, before passing on the tabulate command, then refer these articles:

  1. https://unix.stackexchange.com/questions/170600/sorting-a-csv-file-but-not-its-header

  2. https://ss64.com/bash/sort.html

  3. https://man7.org/linux/man-pages/man1/sort.1.html

For Windows Users : You can get the needed binaries like head, tail, sort etc. from my other project :
https://sourceforge.net/projects/portable-linux-binary-windows/


Related

Wiki: AWK one-liner collection
Wiki: How to Use Comparison Operators with Awk in Linux
Wiki: Special_Characters
Wiki: collection of linux one-liner commands
Wiki: grep command with examples
Wiki: tr command in Unix-Linux with examples

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.