HuntnGather – Diff between revs 31 and 37

Subversion Repositories:

?pathlinks?

Rev 31 – Rev 44 – Go to most recent revision – Details – Blame – Last modification – View Log

 Short:        File indexing and search utilities.
 Author:       Wizardry and Steamworks
-Uploader:     "Wizardry and Steamworks" <office@grimore.org>
+Uploader:     "Wizardry and Steamworks" <mail@grimore.org>
 Type:         util/dir
 Replaces:     util/dir/HuntnGather.lha
-Version:      1.7.3
+Version:      1.7.4
 Architecture: m68k-amigaos
              Hunt & Gather - File search and indexing utilities.
            Released under the MIT License, all rights reserved.
 -=:[ Changes ]:=-
+20211105:
+    * Use a single binary for all CPUs.
-TBA:
+	* Refocus the documentation to match the latest changes.
-    * Testing with MuForce.
+	* MuForce and MuGuardianAngel memory usage stress testing.
-    * Tested using large entire drives and eliminated some bugs.
+	* Large storage (cca. 8k directories / 80k files) stress testing.
-    * Use locale for string comparisons.
+	* Use locale for string comparisons on the Amiga.
-    * Gather now requires either of -a (add), -r (remove), -c (create).
+	* "Gather" now requires either of -a (add), -r (remove), -c (create).
-    * Allow specifying multiple paths when gathering.
+	* Allow multiple paths when issuing the "Gather" command.
 -=:[ Introduction ]:=-
-    Hunt and Gather are two utiltities for indexing and then searching
+    "Hunt" and "Gather" are two utiltities for indexing and then searching
     fileswithin a drive or directory designed to speed up searching files.
-    The Gather utility is meant to index any path and generate a search
+    The "Gather" utility is meant to index any path and generate a search
-    database. The Hunt utility will then open the database generated by
+    database. The "Hunt" utility will then open the database generated by
-    Gather and look for files matching the string provided to Hunt as
+    Gather and look for files matching the string provided to "Hunt" as
     parameter.
     The utility was designed to check large collections of icons but the
     indexing and the fast search results proved indispensable enough for a
     self-standing program.
--=:[ Design ]:=-
-    Hunt and Gather are designed with constant memory usage in order to be
+-=:[ Design ]:=-
     suitable for all Amiga models. Namely, the Gather utility will search
-    all files in a given path, sort the files in ascending order by using
+    "Hunt" and "Gather" are designed with constant memory usage in order
-    an external merge sort (tailored down to a 256KiB memory limit).
+    to be suitable for all Amigas. Namely, the "Gather" utility will
+    search all files in a given path, sort the files in ascending order by
+    using an external file-based merge sort.
-    Conversely, Hunt uses brute force to search for files but by reading
+    Conversely, "Hunt" searchs files by reading lines from the database
-    lines from the database without loading the entire database in RAM.
+    without loading the entire database in RAM or by searching files
-    Perhaps ulterior versions of Hunt might partition the database file
+    again.
-    just like Gather does and then build Tries in oder to speed up finding
+    The project adheres to the ANSI C standard and Amiga-centric semantics
-    files on the filesystem.
+    are compiled conditionally (in case the "___AmigaOS__" macro is
+    defined at compile time). Otherwise, "Hunt" and "Gather" should run
-    The project adheres to the ANSI C standard and Amiga-centric semantics
+    under any platform that benefits from an ANSI C compiler.
     are compiled conditionally (in case the "___AmigaOS__" macro is
+    The project is developed from scratch on a real Amiga using StormC.
-    defined at compile time).Otherwise, Hunt & Gather should run under any
+-=:[ Usage ]:=-
-    platform that benefits from an ANSI C compiler.
+    The "Gather" utility is used to index a path. The following command:
 -=:[ Usage ]:=-
+	Gather -c RAM:
+    will create a file in the S: directory named "gather.db". "Gather"is
+    verbose by default and will show the user what the utility is doing
+    but the behaviour can be changed with the "-q" (quiet) flag that will
+    make "Gather" print only errors.
+	In order to look for a file, the "Hunt" utility is then invoked with
+	an AmigaOS search pattern, for instance, the pattern "#?test#?:
+	Hunt #?test#?
+    "Hunt" will then search the database previously generated by the
+    "Gather" utility and will print out all the paths corresponding to the
+    files matching the supplied pattern.
+    In the previous example, in case any of the files previously indexed
+    by "Gather" contain the term "test", then the "Hunt" utility will
+    display the path to the file.
+    At some point you might decide to add some other path to the search
+    database as well. In that case, "Gather" would be invoked with the
+    "-a" option instead of "-c" in order to add the files:
+    Gather -a HDH0:Icons/
+    "Gather" will then index the additional directory and add the new
+    files to the database. Adding a path to the index database will
+    require that "Gather" sorts the database again such that after adding
+    the new files, "Gather" will proceed with sorting.
-    First the Gather utility is used to index a path:
+    Lastly, the "-r" parameter can be used with "Gather" to remove paths
+    that have been previously indexed. Let's say that you have indexed the
+    following paths with "Gather":
-	Gather RAM:
+    RAM:
+    HDH0:Icons/
+    but now you would like to remove the "RAM:" path and all the files
+    indexed below that path. In that case, you would issue a "Gather"
+    command with the "-r" parameter:
-    which will create a file in the S: directory named "gahter.db". While
+    Gather -r RAM:
+    and "Gather" will remove all files matching the "RAM:" path. Removing
+    a path with the "-r" parameter does not take a long time compared to
+    adding files to the database.
-    Gather is running, the utility will display the number of indexed
+-=:[ Gather ]:=-
+	"Gather" requires that one of the following parameters is specified:
+	  * -a (add files to an already existing database),
+	  * -r (remove files from an already existing database),
+	  * -c (delete the previous database file and create a new database).
     directories and files on the command line.
+	The "Gather" utility takes several paths as parameters representing
-	In order to look for a file, the Hunt utility is invoked with an
+	the paths to be indexed; for example, all the following paths are
-    Amiga search pattern:
+	valid:
-	Hunt #?test#?
+	RAM:
 	DH0:System/
-    in this case, "#?test#?", that will be compared to all the files
-    indexed previously by Gather. In case any of the files previously
+	When the "Gather" utility runs, a database is created at "S:gather.db"
+	containing all the found files.
+	"Gather" is also happy to work with a different database file other
+	than the default database at "S:gather.db" by passing the "-d"
+	parameter when "Gather" is invoked. For instance, the following
+	command invocation will create the database file at "T:gather.db" and
+	index the paths "RAM:" and "HDH0:Icons":
+	Gather -d T:gather.db RAM: HDH0:Icons
+	Conversely, the "Hunt" utility can then be used to search specified
+	database files:
+	Hunt -d T:gather.db #?test?#
+	The previous "Hunt" command will search a database file located at
+	"T:gather.db" for all files matching the pattern "#?test?".
+-=:[ Hunt ]:=-
+	"Hunt" is the counterpart to "Gather" and will search a given database
+	generated by the "Gather" utility for files matching the terms passed
+	to "Hunt" on the command line.
+    For instance:
+	Hunt #?test#?
+    will search all files in the "Gather" database "S:gather.db" for the
+    term "test". If any file within the database partially matches the
+    term "test", then "Hunt" will display the path on the command line.
-    indexed by Gather contain the term "test", then the Hunt utility will
+    "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS.
     display the path to the file.
 -=:[ Notes ]:=-
 -=:[ Gather ]:=-
     * The "Gather" utility will be slow and that is the intended
+      behaviour: slow indexing with "Gather", fast searching with "Hunt".
-	The Gather utility takes one single parameter representing the path
+    * Temporary files might end up created in the same location where the
-    to be indexed; for example, all the following paths are valid:
+      "Gather" utility is invoked. Traditionally the temporary directory
+      on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it
+      intends to index very large hierarchies. Fortunately, "Gather"will
-	RAM:
+      delete the temporary files once "Gather" is done indexing.
 	DH0:System/
       Nevertheless, in case you intend to index a large filesystem