Recent changes to Home

Home modified by rctdesigns

rctdesigns — Sun, 16 Jun 2013 20:25:18 -0000

--- v4
+++ v5
@@ -1,26 +1,37 @@
 From README.txt

-This pair of programs solves a problem I could not find a solution for.  It is two very simple pieces of Java code combined with the sqlite database.  Feel free to use it in any way you choose.  I place it in the public domain and I hope you find it useful.
+This pair of programs solves a problem I could not find a solution for.  It is two very simple pieces of Java code
+combined with the sqlite database.  Feel free to use it in any way you choose.  I place it in the public domain.

-Currently it is meant to be used on Linux, but can easily be modified to create Window's .bat files.  I may get around to that, but haven't needed it yet.
+It works on both Linux and Windows.  (use --win option for MakeScript)
+
+Requirements: sqlite jdbc and Java jre

 Use case:

-You have thousands of files (of any file type), many of which are duplicates.  They reside in multiple directories under a single base directory. You want to copy one of each unique file into a new set of directories, leaving the original files and directories unchanged.
+You have thousands of files (of any file type), many of which are duplicates.  They reside in multiple directories
+under a single base directory. You want to copy one of each unique file into a new set of directories, leaving the original files
+and directories unchanged.

 You run at the command line:

     java com.bizdash.main.Scanner --showpaths --createtable /basedir

-Scanner will scan through basedir and all directories under it, creating a MD5 hash for each file found.  The hash and the file's information are saved in an sqlite database for later processing.
+Scanner will scan through basedir and all directories under it, creating a MD5 hash for each file found.  The hash
+and the file's information are saved in an sqlite database for later processing.

 You can optionally limit what file types are processed, specify directories to be skipped, and specify multiple base directories.
+
+The scanner checks for and skips files that contain invalid file characters.  The skipped files are noted in the log as "RENAME====" 
+followed by the current file name.  If you want these files included in the database, rename them and run the scanner again.

 Once you have scanned the files of interest, you are ready to run:

     java com.bizdash.main.MakeScript --showfiles /basedir /newbasedir

-MakeScript will go through the database and find one copy of each unique file.  It will create a script file (sh or bash script) that has all of the commands to copy the unique set of files to /newbasedir.  /basedir needs to be the same as the /basedir you specified for Scanner.
+MakeScript will go through the database and find one copy of each unique file.  It will create a script file (sh or bash script)
+that has all of the commands to copy the unique set of files to /newbasedir.  /basedir needs to be the same as the /basedir
+you specified for Scanner.

 MakeScript also lets you filter by file type.  The following command will copy only JPG files:

@@ -29,3 +40,19 @@
 Or do multiple file types with:

    java com.bizdash.main.MakeScript --showfiles '.jpg .mpg' /basedir /newbasedir
+   
+Want a windows bat file?  Use:
+
+   java com.bizdash.main.MakeScript --win --showfiles '.jpg .mpg' \basedir \newbasedir
+   
+You can change the destination by either running MakeScript again with new arguments or you can edit two variables at the top of the
+script or bat file.
+
+After running the copy script, you can have MakeScript check the copy results, using either the --check or --hash option.
+--check checks to see if the file exists and reports it missing if not.
+--hash (forces --check) and hashes the file and compares it to the original file's hash
+
+Run either Scanner or MakeScript without any arguments to see their options.
+
+
+

Home modified by rctdesigns

rctdesigns — Thu, 13 Jun 2013 21:02:10 -0000

--- v3
+++ v4
@@ -1,23 +1,18 @@
 From README.txt

-This pair of programs solves a problem I could not find a solution for.  It is two very simple pieces of Java code
-combined with the sqlite database.  Feel free to use it in any way you choose.  I place it in the public domain and I hope you find it useful.
+This pair of programs solves a problem I could not find a solution for.  It is two very simple pieces of Java code combined with the sqlite database.  Feel free to use it in any way you choose.  I place it in the public domain and I hope you find it useful.

-Currently it is meant to be used on Linux, but can easily be modified to create Window's .bat files.  I may get
-around to that, but haven't needed it yet.
+Currently it is meant to be used on Linux, but can easily be modified to create Window's .bat files.  I may get around to that, but haven't needed it yet.

 Use case:

-You have thousands of files (of any file type), many of which are duplicates.  They reside in multiple directories
-under a single base directory. You want to copy one of each unique file into a new set of directories, leaving the original files
-and directories unchanged.
+You have thousands of files (of any file type), many of which are duplicates.  They reside in multiple directories under a single base directory. You want to copy one of each unique file into a new set of directories, leaving the original files and directories unchanged.

 You run at the command line:

     java com.bizdash.main.Scanner --showpaths --createtable /basedir

-Scanner will scan through basedir and all directories under it, creating a MD5 hash for each file found.  The hash
-and the file's information are saved in an sqlite database for later processing.
+Scanner will scan through basedir and all directories under it, creating a MD5 hash for each file found.  The hash and the file's information are saved in an sqlite database for later processing.

 You can optionally limit what file types are processed, specify directories to be skipped, and specify multiple base directories.

@@ -25,9 +20,7 @@

     java com.bizdash.main.MakeScript --showfiles /basedir /newbasedir

-MakeScript will go through the database and find one copy of each unique file.  It will create a script file (sh or bash script)
-that has all of the commands to copy the unique set of files to /newbasedir.  /basedir needs to be the same as the /basedir
-you specified for Scanner.
+MakeScript will go through the database and find one copy of each unique file.  It will create a script file (sh or bash script) that has all of the commands to copy the unique set of files to /newbasedir.  /basedir needs to be the same as the /basedir you specified for Scanner.

 MakeScript also lets you filter by file type.  The following command will copy only JPG files:

Home modified by rctdesigns

rctdesigns — Thu, 13 Jun 2013 20:41:02 -0000

--- v2
+++ v3
@@ -1,5 +1,7 @@
+From README.txt
+
 This pair of programs solves a problem I could not find a solution for.  It is two very simple pieces of Java code
-combined with the sqlite database.  Feel free to use it in any way you choose.  I place it in the public domain.
+combined with the sqlite database.  Feel free to use it in any way you choose.  I place it in the public domain and I hope you find it useful.

 Currently it is meant to be used on Linux, but can easily be modified to create Window's .bat files.  I may get
 around to that, but haven't needed it yet.

Home modified by rctdesigns

rctdesigns — Thu, 13 Jun 2013 20:39:30 -0000

--- v1
+++ v2
@@ -1,8 +1,36 @@
-Welcome to your wiki!
+This pair of programs solves a problem I could not find a solution for.  It is two very simple pieces of Java code
+combined with the sqlite database.  Feel free to use it in any way you choose.  I place it in the public domain.

-This is the default page, edit it as you see fit. To add a new page simply reference it within brackets, e.g.: [SamplePage].
+Currently it is meant to be used on Linux, but can easily be modified to create Window's .bat files.  I may get
+around to that, but haven't needed it yet.

-The wiki uses [Markdown](/p/dupless/wiki/markdown_syntax/) syntax.
+Use case:

-[[members limit=20]]
-[[download_button]]
+You have thousands of files (of any file type), many of which are duplicates.  They reside in multiple directories
+under a single base directory. You want to copy one of each unique file into a new set of directories, leaving the original files
+and directories unchanged.
+
+You run at the command line:
+
+    java com.bizdash.main.Scanner --showpaths --createtable /basedir
+    
+Scanner will scan through basedir and all directories under it, creating a MD5 hash for each file found.  The hash
+and the file's information are saved in an sqlite database for later processing.
+
+You can optionally limit what file types are processed, specify directories to be skipped, and specify multiple base directories.
+
+Once you have scanned the files of interest, you are ready to run:
+
+    java com.bizdash.main.MakeScript --showfiles /basedir /newbasedir
+    
+MakeScript will go through the database and find one copy of each unique file.  It will create a script file (sh or bash script)
+that has all of the commands to copy the unique set of files to /newbasedir.  /basedir needs to be the same as the /basedir
+you specified for Scanner.
+
+MakeScript also lets you filter by file type.  The following command will copy only JPG files:
+
+    java com.bizdash.main.MakeScript --showfiles '.jpg' /basedir /newbasedir
+    
+Or do multiple file types with:
+
+   java com.bizdash.main.MakeScript --showfiles '.jpg .mpg' /basedir /newbasedir

Discussion for Home page

rctdesigns — Thu, 13 Jun 2013 20:38:39 -0000

This pair of programs solves a problem I could not find a solution for. It is two very simple pieces of Java code
combined with the sqlite database. Feel free to use it in any way you choose. I place it in the public domain.

Currently it is meant to be used on Linux, but can easily be modified to create Window's .bat files. I may get
around to that, but haven't needed it yet.

Use case:

You have thousands of files (of any file type), many of which are duplicates. They reside in multiple directories
under a single base directory. You want to copy one of each unique file into a new set of directories, leaving the original files
and directories unchanged.

You run at the command line:

java com.bizdash.main.Scanner --showpaths --createtable /basedir

Scanner will scan through basedir and all directories under it, creating a MD5 hash for each file found. The hash
and the file's information are saved in an sqlite database for later processing.

You can optionally limit what file types are processed, specify directories to be skipped, and specify multiple base directories.

Once you have scanned the files of interest, you are ready to run:

java com.bizdash.main.MakeScript --showfiles /basedir /newbasedir

MakeScript will go through the database and find one copy of each unique file. It will create a script file (sh or bash script)
that has all of the commands to copy the unique set of files to /newbasedir. /basedir needs to be the same as the /basedir
you specified for Scanner.

MakeScript also lets you filter by file type. The following command will copy only JPG files:

java com.bizdash.main.MakeScript --showfiles '.jpg' /basedir /newbasedir

Or do multiple file types with:

java com.bizdash.main.MakeScript --showfiles '.jpg .mpg' /basedir /newbasedir

Home modified by rctdesigns

rctdesigns — Thu, 13 Jun 2013 19:51:34 -0000

Welcome to your wiki!

This is the default page, edit it as you see fit. To add a new page simply reference it within brackets, e.g.: [SamplePage].

The wiki uses Markdown syntax.

Project Members:

rctdesigns (admin)