HuntnGather – Blame information for rev 37

Subversion Repositories:
Rev:
Rev Author Line No. Line
1 office 1 Short: File indexing and search utilities.
2 Author: Wizardry and Steamworks
37 office 3 Uploader: "Wizardry and Steamworks" <mail@grimore.org>
1 office 4 Type: util/dir
5 Replaces: util/dir/HuntnGather.lha
37 office 6 Version: 1.7.4
1 office 7 Architecture: m68k-amigaos
8  
9 Hunt & Gather - File search and indexing utilities.
10 (or Hunt'n'Gather bro, ghetto style)
11  
12 © Copyright 2021 by Wizardry and Steamworks
13  
14 Released under the MIT License, all rights reserved.
15  
22 office 16 -=:[ Changes ]:=-
1 office 17  
37 office 18 20211105:
19 * Use a single binary for all CPUs.
20 * Refocus the documentation to match the latest changes.
21 * MuForce and MuGuardianAngel memory usage stress testing.
22 * Large storage (cca. 8k directories / 80k files) stress testing.
23 * Use locale for string comparisons on the Amiga.
24 * "Gather" now requires either of -a (add), -r (remove), -c (create).
25 * Allow multiple paths when issuing the "Gather" command.
20 office 26  
1 office 27 -=:[ Introduction ]:=-
28  
29  
37 office 30 "Hunt" and "Gather" are two utiltities for indexing and then searching
1 office 31 fileswithin a drive or directory designed to speed up searching files.
32  
37 office 33 The "Gather" utility is meant to index any path and generate a search
34 database. The "Hunt" utility will then open the database generated by
35 Gather and look for files matching the string provided to "Hunt" as
1 office 36 parameter.
37  
38 The utility was designed to check large collections of icons but the
39 indexing and the fast search results proved indispensable enough for a
40 self-standing program.
41  
42 -=:[ Design ]:=-
43  
37 office 44 "Hunt" and "Gather" are designed with constant memory usage in order
45 to be suitable for all Amigas. Namely, the "Gather" utility will
46 search all files in a given path, sort the files in ascending order by
47 using an external file-based merge sort.
1 office 48  
37 office 49 Conversely, "Hunt" searchs files by reading lines from the database
50 without loading the entire database in RAM or by searching files
51 again.
1 office 52  
53 The project adheres to the ANSI C standard and Amiga-centric semantics
54 are compiled conditionally (in case the "___AmigaOS__" macro is
37 office 55 defined at compile time). Otherwise, "Hunt" and "Gather" should run
56 under any platform that benefits from an ANSI C compiler.
57  
58 The project is developed from scratch on a real Amiga using StormC.
1 office 59  
60 -=:[ Usage ]:=-
61  
37 office 62 The "Gather" utility is used to index a path. The following command:
1 office 63  
64  
37 office 65 Gather -c RAM:
1 office 66  
67  
37 office 68 will create a file in the S: directory named "gather.db". "Gather"is
69 verbose by default and will show the user what the utility is doing
70 but the behaviour can be changed with the "-q" (quiet) flag that will
71 make "Gather" print only errors.
1 office 72  
37 office 73 In order to look for a file, the "Hunt" utility is then invoked with
74 an AmigaOS search pattern, for instance, the pattern "#?test#?:
1 office 75  
76  
3 office 77 Hunt #?test#?
1 office 78  
79  
37 office 80 "Hunt" will then search the database previously generated by the
81 "Gather" utility and will print out all the paths corresponding to the
82 files matching the supplied pattern.
83  
84 In the previous example, in case any of the files previously indexed
85 by "Gather" contain the term "test", then the "Hunt" utility will
22 office 86 display the path to the file.
37 office 87  
88 At some point you might decide to add some other path to the search
89 database as well. In that case, "Gather" would be invoked with the
90 "-a" option instead of "-c" in order to add the files:
91  
92 Gather -a HDH0:Icons/
93  
94 "Gather" will then index the additional directory and add the new
95 files to the database. Adding a path to the index database will
96 require that "Gather" sorts the database again such that after adding
97 the new files, "Gather" will proceed with sorting.
98  
99 Lastly, the "-r" parameter can be used with "Gather" to remove paths
100 that have been previously indexed. Let's say that you have indexed the
101 following paths with "Gather":
102  
103 RAM:
104 HDH0:Icons/
105  
106 but now you would like to remove the "RAM:" path and all the files
107 indexed below that path. In that case, you would issue a "Gather"
108 command with the "-r" parameter:
109  
110 Gather -r RAM:
111  
112 and "Gather" will remove all files matching the "RAM:" path. Removing
113 a path with the "-r" parameter does not take a long time compared to
114 adding files to the database.
1 office 115  
116 -=:[ Gather ]:=-
117  
37 office 118 "Gather" requires that one of the following parameters is specified:
119 * -a (add files to an already existing database),
120 * -r (remove files from an already existing database),
121 * -c (delete the previous database file and create a new database).
1 office 122  
37 office 123 The "Gather" utility takes several paths as parameters representing
124 the paths to be indexed; for example, all the following paths are
125 valid:
1 office 126  
37 office 127  
1 office 128 RAM:
129 DH0:System/
130  
131  
37 office 132 When the "Gather" utility runs, a database is created at "S:gather.db"
133 containing all the found files.
134  
135 "Gather" is also happy to work with a different database file other
136 than the default database at "S:gather.db" by passing the "-d"
137 parameter when "Gather" is invoked. For instance, the following
138 command invocation will create the database file at "T:gather.db" and
139 index the paths "RAM:" and "HDH0:Icons":
140  
141 Gather -d T:gather.db RAM: HDH0:Icons
142  
143 Conversely, the "Hunt" utility can then be used to search specified
144 database files:
145  
146 Hunt -d T:gather.db #?test?#
147  
148 The previous "Hunt" command will search a database file located at
149 "T:gather.db" for all files matching the pattern "#?test?".
1 office 150  
151 -=:[ Hunt ]:=-
152  
37 office 153 "Hunt" is the counterpart to "Gather" and will search a given database
154 generated by the "Gather" utility for files matching the terms passed
155 to "Hunt" on the command line.
1 office 156  
3 office 157 For instance:
1 office 158  
3 office 159 Hunt #?test#?
1 office 160  
37 office 161 will search all files in the "Gather" database "S:gather.db" for the
1 office 162 term "test". If any file within the database partially matches the
37 office 163 term "test", then "Hunt" will display the path on the command line.
3 office 164  
37 office 165 "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS.
166  
167 -=:[ Notes ]:=-
168  
169 * The "Gather" utility will be slow and that is the intended
170 behaviour: slow indexing with "Gather", fast searching with "Hunt".
171  
172 * Temporary files might end up created in the same location where the
173 "Gather" utility is invoked. Traditionally the temporary directory
174 on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it
175 intends to index very large hierarchies. Fortunately, "Gather"will
176 delete the temporary files once "Gather" is done indexing.
177  
178 Nevertheless, in case you intend to index a large filesystem
179 hierarchy please make sure that you invoke "Gather" from a directory
180 that is able to hold large temporary files.
181  
182 * The output of the "Hunt" utility can be combined with the pipe
183 operator (in newer AmigaOS releases) or the PIPE: handler on older
184 AmigaOS releases in order to to perform some action on the found
185 files. For example, using Thomas Radtke's "from" utility located at:
186  
187 http://aminet.net/package/util/batch/from
188  
189 and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5
190 hashes of all files indexed by "Gather" ending in "#?.library:
191  
192 Hunt #?.library | from - md5sum $1
193  
194 Or you could generate a list of versions of all libraries indexed
195 with the "Gahter" utility:
196  
197 Hunt #?.library | from - version $1
198  
3 office 199 -=:[ Source ]:=-
1 office 200  
3 office 201 The project is open sourced and licensed under MIT. The source code
202 is included in the AmiNET release or can be checked out via subversion
203 from the Wizardry and Steamworks repository:
1 office 204  
3 office 205 svn co http://svn.grimore.org/HuntnGather
206  
37 office 207 StormC was used as the developer environment.
208  
4 office 209 -=:[ Mentions ]:=-
210  
211 The code includes a shim for "getopt" in order to process command line
37 office 212 parameters on AmigaOS just like one would on a POSIX sytem. The shim
213 is created by Daniel J. Barrett, barrett@cs.umass.edu and is
214 available on AmiNET:
4 office 215  
216 http://aminet.net/package/dev/misc/GetOpt-1.3
217  
1 office 218 -=:[ Contact ]:=-
219  
22 office 220 E-Mail(tor):
221  
222 office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion
223  
1 office 224 Website: https://grimore.org/amiga/hunt_and_gather
22 office 225 Website(tor):
226  
227 http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/
228  
1 office 229 Discord: https://discord.gg/k9kyDsa
230