HuntnGather

Subversion Repositories:
Compare Path: Rev
With Path: Rev
?path1? @ HEAD  →  ?path2? @ 1
/trunk/HuntnGather/HuntnGather.readme
@@ -1,9 +1,9 @@
Short: File indexing and search utilities.
Author: Wizardry and Steamworks
Uploader: "Wizardry and Steamworks" <mail@grimore.org>
Uploader: "Wizardry and Steamworks" <office@grimore.org>
Type: util/dir
Replaces: util/dir/HuntnGather.lha
Version: 1.7.7
Version: 1.0
Architecture: m68k-amigaos
 
Hunt & Gather - File search and indexing utilities.
@@ -13,20 +13,30 @@
 
Released under the MIT License, all rights reserved.
 
-=:[ Changes ]:=-
-=:[ ChangeLog ]:=-
 
20211206:
* Compile pattern only once before searching through files.
20210107:
 
* Add the quiet option (-q) to the Gather tool.
* Process arguments cannonically for both Hunt & Gather.
* Switch to a case-insensitive search for Hunt.
 
20210105:
 
* Ensure the database is sorted for future development.
* Insert CTRL+C breaking of both tools for AmigaOS.
* Make both tools more verbose.
* Dot the is, dash the ts...
 
-=:[ Introduction ]:=-
 
 
"Hunt" and "Gather" are two utiltities for indexing and then searching
Hunt and Gather are two utiltities for indexing and then searching
fileswithin a drive or directory designed to speed up searching files.
 
The "Gather" utility is meant to index any path and generate a search
database. The "Hunt" utility will then open the database generated by
Gather and look for files matching the string provided to "Hunt" as
The Gather utility is meant to index any path and generate a search
database. The Hunt utility will then open the database generated by
Gather and look for files matching the string provided to Hunt as
parameter.
 
The utility was designed to check large collections of icons but the
@@ -35,184 +45,86 @@
 
-=:[ Design ]:=-
 
"Hunt" and "Gather" are designed with constant memory usage in order
to be suitable for all Amigas. Namely, the "Gather" utility will
search all files in a given path, sort the files in ascending order by
using an external file-based merge sort.
Hunt and Gather are designed with constant memory usage in order to be
suitable for all Amiga models. Namely, the Gather utility will search
all files in a given path, sort the files in ascending order by using
an external merge sort (tailored down to a 256KiB memory limit).
 
Conversely, "Hunt" searchs files by reading lines from the database
without loading the entire database in RAM or by searching files
again.
Conversely, Hunt uses brute force to search for files but by reading
lines from the database without loading the entire database in RAM.
Perhaps ulterior versions of Hunt might partition the database file
just like Gather does and then build Tries in oder to speed up finding
files on the filesystem.
 
The project adheres to the ANSI C standard and Amiga-centric semantics
are compiled conditionally (in case the "___AmigaOS__" macro is
defined at compile time). Otherwise, "Hunt" and "Gather" should run
under any platform that benefits from an ANSI C compiler.
The project is developed from scratch on a real Amiga using StormC.
defined at compile time).Otherwise, Hunt & Gather should run under any
platform that benefits from an ANSI C compiler.
 
-=:[ Usage ]:=-
 
The "Gather" utility is used to index a path. The following command:
First the Gather utility is used to index a path:
 
Gather -c RAM:
 
will create a file in the S: directory named "gather.db". "Gather"is
verbose by default and will show the user what the utility is doing
but the behaviour can be changed with the "-q" (quiet) flag that will
make "Gather" print only errors.
Gather RAM:
 
In order to look for a file, the "Hunt" utility is then invoked with
an AmigaOS search pattern, for instance, the pattern "#?test#?:
 
Hunt #?test#?
which will create a file in the S: directory named "gahter.db". While
Gather is running, the utility will display the number of indexed
directories and files on the command line.
 
"Hunt" will then search the database previously generated by the
"Gather" utility and will print out all the paths corresponding to the
files matching the supplied pattern.
In the previous example, in case any of the files previously indexed
by "Gather" contain the term "test", then the "Hunt" utility will
display the path to the file.
At some point you might decide to add some other path to the search
database as well. In that case, "Gather" would be invoked with the
"-a" option instead of "-c" in order to add the files:
Gather -a HDH0:Icons/
"Gather" will then index the additional directory and add the new
files to the database. Adding a path to the index database will
require that "Gather" sorts the database again such that after adding
the new files, "Gather" will proceed with sorting.
Lastly, the "-r" parameter can be used with "Gather" to remove paths
that have been previously indexed. Let's say that you have indexed the
following paths with "Gather":
RAM:
HDH0:Icons/
but now you would like to remove the "RAM:" path and all the files
indexed below that path. In that case, you would issue a "Gather"
command with the "-r" parameter:
Gather -r RAM:
and "Gather" will remove all files matching the "RAM:" path. Removing
a path with the "-r" parameter does not take a long time compared to
adding files to the database.
In order to look for a file, the Hunt utility is invoked with a search
term:
 
 
Hunt test
 
 
in this case, "test", that will be compared to all the files indexed
previously by Gather. In case any of the files previously indexed by
Gather contain the term "test", then the Hunt utility will display
the path to the file.
 
-=:[ Gather ]:=-
 
"Gather" requires that one of the following parameters is specified:
* -a (add files to an already existing database),
* -r (remove files from an already existing database),
* -c (delete the previous database file and create a new database).
The Gather utility takes one single parameter representing the path
to be indexed; for example, all the following paths are valid:
 
The "Gather" utility takes several paths as parameters representing
the paths to be indexed; for example, all the following paths are
valid:
 
RAM:
DH0:System/
RAM:
DH0:System/
 
When the "Gather" utility runs, a database is created at "S:gather.db"
containing all the found files.
"Gather" is also happy to work with a different database file other
than the default database at "S:gather.db" by passing the "-d"
parameter when "Gather" is invoked. For instance, the following
command invocation will create the database file at "T:gather.db" and
index the paths "RAM:" and "HDH0:Icons":
Gather -d T:gather.db RAM: HDH0:Icons
Conversely, the "Hunt" utility can then be used to search specified
database files:
Hunt -d T:gather.db #?test?#
The previous "Hunt" command will search a database file located at
"T:gather.db" for all files matching the pattern "#?test?".
 
When Gather runs, a database is created at "S:gather.db" by
overwriting the previous database. For best results, Gather should
run periodically and should scan a path that is most frequently
searched for files.
 
-=:[ Hunt ]:=-
 
"Hunt" is the counterpart to "Gather" and will search a given database
generated by the "Gather" utility for files matching the terms passed
to "Hunt" on the command line.
Hunt is the counterpart to Gather and will search the database at
"S:gather.db" for files matching the terms passed to Hunt on the
command line.
 
For instance:
For instance:
 
Hunt #?test#?
Hunt test
 
will search all files in the "Gather" database "S:gather.db" for the
will search all files in the Gather database "S:gather.db" for the
term "test". If any file within the database partially matches the
term "test", then "Hunt" will display the path on the command line.
"Hunt" uses AmigaOS pattern for matching the file names on AmigaOS.
term "test", then Hunt will display the path on the command line.
 
-=:[ Notes ]:=-
-=:[ Mentions ]:=-
 
* The "Gather" utility will be slow and that is the intended
behaviour: slow indexing with "Gather", fast searching with "Hunt".
* Temporary files might end up created in the same location where the
"Gather" utility is invoked. Traditionally the temporary directory
on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it
intends to index very large hierarchies. Fortunately, "Gather"will
delete the temporary files once "Gather" is done indexing.
Nevertheless, in case you intend to index a large filesystem
hierarchy please make sure that you invoke "Gather" from a directory
that is able to hold large temporary files.
 
* The output of the "Hunt" utility can be combined with the pipe
operator (in newer AmigaOS releases) or the PIPE: handler on older
AmigaOS releases in order to to perform some action on the found
files. For example, using Thomas Radtke's "from" utility located at:
http://aminet.net/package/util/batch/from
and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5
hashes of all files indexed by "Gather" ending in "#?.library:
Hunt #?.library | from - md5sum $1
Or you could generate a list of versions of all libraries indexed
with the "Gahter" utility:
Hunt #?.library | from - version $1
-=:[ Source ]:=-
 
The project is open sourced and licensed under MIT. The source code
is included in the AmiNET release or can be checked out via subversion
from the Wizardry and Steamworks repository:
My boss spilled the data of 800+ employees.
I called him out and left.
svn co http://svn.grimore.org/HuntnGather
StormC was used as the developer environment.
-=:[ Mentions ]:=-
Supporting Romania is like giving a gun to a monkey.
 
The code includes a shim for "getopt" in order to process command line
parameters on AmigaOS just like one would on a POSIX sytem. The shim
is created by Daniel J. Barrett, barrett@cs.umass.edu and is
available on AmiNET:
http://aminet.net/package/dev/misc/GetOpt-1.3
-=:[ Contact ]:=-
 
E-Mail(tor):
 
office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion
 
E-Mail(tor): office@kaarvixjxfdy2wv2.onion
Website: https://grimore.org/amiga/hunt_and_gather
Website(tor):
 
http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/
 
Website(tor): http://kaarvixjxfdy2wv2.onion
Discord: https://discord.gg/k9kyDsa