HuntnGather – Blame information for rev 44

Subversion Repositories:
Rev:
Rev Author Line No. Line
1 office 1 Short: File indexing and search utilities.
2 Author: Wizardry and Steamworks
37 office 3 Uploader: "Wizardry and Steamworks" <mail@grimore.org>
1 office 4 Type: util/dir
5 Replaces: util/dir/HuntnGather.lha
44 office 6 Version: 1.7.5
1 office 7 Architecture: m68k-amigaos
8  
9 Hunt & Gather - File search and indexing utilities.
10 (or Hunt'n'Gather bro, ghetto style)
11  
12 © Copyright 2021 by Wizardry and Steamworks
13  
14 Released under the MIT License, all rights reserved.
15  
22 office 16 -=:[ Changes ]:=-
1 office 17  
44 office 18 20211114:
20 office 19  
44 office 20 * Rewrite and preparations for pooling memory the Amiga way.
21  
1 office 22 -=:[ Introduction ]:=-
23  
24  
37 office 25 "Hunt" and "Gather" are two utiltities for indexing and then searching
1 office 26 fileswithin a drive or directory designed to speed up searching files.
27  
37 office 28 The "Gather" utility is meant to index any path and generate a search
29 database. The "Hunt" utility will then open the database generated by
30 Gather and look for files matching the string provided to "Hunt" as
1 office 31 parameter.
32  
33 The utility was designed to check large collections of icons but the
34 indexing and the fast search results proved indispensable enough for a
35 self-standing program.
36  
37 -=:[ Design ]:=-
38  
37 office 39 "Hunt" and "Gather" are designed with constant memory usage in order
40 to be suitable for all Amigas. Namely, the "Gather" utility will
41 search all files in a given path, sort the files in ascending order by
42 using an external file-based merge sort.
1 office 43  
37 office 44 Conversely, "Hunt" searchs files by reading lines from the database
45 without loading the entire database in RAM or by searching files
46 again.
1 office 47  
48 The project adheres to the ANSI C standard and Amiga-centric semantics
49 are compiled conditionally (in case the "___AmigaOS__" macro is
37 office 50 defined at compile time). Otherwise, "Hunt" and "Gather" should run
51 under any platform that benefits from an ANSI C compiler.
52  
53 The project is developed from scratch on a real Amiga using StormC.
1 office 54  
55 -=:[ Usage ]:=-
56  
37 office 57 The "Gather" utility is used to index a path. The following command:
1 office 58  
44 office 59 Gather -c RAM:
1 office 60  
37 office 61 will create a file in the S: directory named "gather.db". "Gather"is
62 verbose by default and will show the user what the utility is doing
63 but the behaviour can be changed with the "-q" (quiet) flag that will
64 make "Gather" print only errors.
1 office 65  
44 office 66 In order to look for a file, the "Hunt" utility is then invoked with
67 an AmigaOS search pattern, for instance, the pattern "#?test#?:
1 office 68  
44 office 69 Hunt #?test#?
1 office 70  
37 office 71 "Hunt" will then search the database previously generated by the
72 "Gather" utility and will print out all the paths corresponding to the
73 files matching the supplied pattern.
74  
75 In the previous example, in case any of the files previously indexed
76 by "Gather" contain the term "test", then the "Hunt" utility will
22 office 77 display the path to the file.
37 office 78  
79 At some point you might decide to add some other path to the search
80 database as well. In that case, "Gather" would be invoked with the
81 "-a" option instead of "-c" in order to add the files:
82  
83 Gather -a HDH0:Icons/
84  
85 "Gather" will then index the additional directory and add the new
86 files to the database. Adding a path to the index database will
87 require that "Gather" sorts the database again such that after adding
88 the new files, "Gather" will proceed with sorting.
89  
90 Lastly, the "-r" parameter can be used with "Gather" to remove paths
91 that have been previously indexed. Let's say that you have indexed the
92 following paths with "Gather":
93  
94 RAM:
95 HDH0:Icons/
96  
97 but now you would like to remove the "RAM:" path and all the files
98 indexed below that path. In that case, you would issue a "Gather"
99 command with the "-r" parameter:
100  
101 Gather -r RAM:
102  
103 and "Gather" will remove all files matching the "RAM:" path. Removing
104 a path with the "-r" parameter does not take a long time compared to
105 adding files to the database.
1 office 106  
107 -=:[ Gather ]:=-
108  
44 office 109 "Gather" requires that one of the following parameters is specified:
110 * -a (add files to an already existing database),
111 * -r (remove files from an already existing database),
112 * -c (delete the previous database file and create a new database).
1 office 113  
44 office 114 The "Gather" utility takes several paths as parameters representing
115 the paths to be indexed; for example, all the following paths are
116 valid:
1 office 117  
44 office 118 RAM:
119 DH0:System/
37 office 120  
44 office 121 When the "Gather" utility runs, a database is created at "S:gather.db"
122 containing all the found files.
37 office 123  
44 office 124 "Gather" is also happy to work with a different database file other
125 than the default database at "S:gather.db" by passing the "-d"
126 parameter when "Gather" is invoked. For instance, the following
127 command invocation will create the database file at "T:gather.db" and
128 index the paths "RAM:" and "HDH0:Icons":
37 office 129  
44 office 130 Gather -d T:gather.db RAM: HDH0:Icons
37 office 131  
44 office 132 Conversely, the "Hunt" utility can then be used to search specified
133 database files:
37 office 134  
44 office 135 Hunt -d T:gather.db #?test?#
37 office 136  
44 office 137 The previous "Hunt" command will search a database file located at
138 "T:gather.db" for all files matching the pattern "#?test?".
1 office 139  
140 -=:[ Hunt ]:=-
141  
44 office 142 "Hunt" is the counterpart to "Gather" and will search a given database
143 generated by the "Gather" utility for files matching the terms passed
144 to "Hunt" on the command line.
1 office 145  
3 office 146 For instance:
1 office 147  
3 office 148 Hunt #?test#?
1 office 149  
37 office 150 will search all files in the "Gather" database "S:gather.db" for the
1 office 151 term "test". If any file within the database partially matches the
37 office 152 term "test", then "Hunt" will display the path on the command line.
3 office 153  
37 office 154 "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS.
155  
156 -=:[ Notes ]:=-
157  
158 * The "Gather" utility will be slow and that is the intended
159 behaviour: slow indexing with "Gather", fast searching with "Hunt".
160  
161 * Temporary files might end up created in the same location where the
162 "Gather" utility is invoked. Traditionally the temporary directory
163 on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it
164 intends to index very large hierarchies. Fortunately, "Gather"will
165 delete the temporary files once "Gather" is done indexing.
166  
167 Nevertheless, in case you intend to index a large filesystem
168 hierarchy please make sure that you invoke "Gather" from a directory
169 that is able to hold large temporary files.
170  
171 * The output of the "Hunt" utility can be combined with the pipe
172 operator (in newer AmigaOS releases) or the PIPE: handler on older
173 AmigaOS releases in order to to perform some action on the found
174 files. For example, using Thomas Radtke's "from" utility located at:
175  
176 http://aminet.net/package/util/batch/from
177  
178 and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5
179 hashes of all files indexed by "Gather" ending in "#?.library:
180  
181 Hunt #?.library | from - md5sum $1
182  
183 Or you could generate a list of versions of all libraries indexed
184 with the "Gahter" utility:
185  
186 Hunt #?.library | from - version $1
187  
3 office 188 -=:[ Source ]:=-
1 office 189  
3 office 190 The project is open sourced and licensed under MIT. The source code
191 is included in the AmiNET release or can be checked out via subversion
192 from the Wizardry and Steamworks repository:
1 office 193  
3 office 194 svn co http://svn.grimore.org/HuntnGather
195  
37 office 196 StormC was used as the developer environment.
197  
4 office 198 -=:[ Mentions ]:=-
199  
200 The code includes a shim for "getopt" in order to process command line
37 office 201 parameters on AmigaOS just like one would on a POSIX sytem. The shim
202 is created by Daniel J. Barrett, barrett@cs.umass.edu and is
203 available on AmiNET:
4 office 204  
205 http://aminet.net/package/dev/misc/GetOpt-1.3
206  
1 office 207 -=:[ Contact ]:=-
208  
22 office 209 E-Mail(tor):
210  
211 office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion
212  
1 office 213 Website: https://grimore.org/amiga/hunt_and_gather
22 office 214 Website(tor):
215  
216 http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/
217  
1 office 218 Discord: https://discord.gg/k9kyDsa
219