HuntnGather – Blame information for rev 37
?pathlinks?
Rev | Author | Line No. | Line |
---|---|---|---|
1 | office | 1 | Short: File indexing and search utilities. |
2 | Author: Wizardry and Steamworks |
||
37 | office | 3 | Uploader: "Wizardry and Steamworks" <mail@grimore.org> |
1 | office | 4 | Type: util/dir |
5 | Replaces: util/dir/HuntnGather.lha |
||
37 | office | 6 | Version: 1.7.4 |
1 | office | 7 | Architecture: m68k-amigaos |
8 | |||
9 | Hunt & Gather - File search and indexing utilities. |
||
10 | (or Hunt'n'Gather bro, ghetto style) |
||
11 | |||
12 | © Copyright 2021 by Wizardry and Steamworks |
||
13 | |||
14 | Released under the MIT License, all rights reserved. |
||
15 | |||
22 | office | 16 | -=:[ Changes ]:=- |
1 | office | 17 | |
37 | office | 18 | 20211105: |
19 | * Use a single binary for all CPUs. |
||
20 | * Refocus the documentation to match the latest changes. |
||
21 | * MuForce and MuGuardianAngel memory usage stress testing. |
||
22 | * Large storage (cca. 8k directories / 80k files) stress testing. |
||
23 | * Use locale for string comparisons on the Amiga. |
||
24 | * "Gather" now requires either of -a (add), -r (remove), -c (create). |
||
25 | * Allow multiple paths when issuing the "Gather" command. |
||
20 | office | 26 | |
1 | office | 27 | -=:[ Introduction ]:=- |
28 | |||
29 | |||
37 | office | 30 | "Hunt" and "Gather" are two utiltities for indexing and then searching |
1 | office | 31 | fileswithin a drive or directory designed to speed up searching files. |
32 | |||
37 | office | 33 | The "Gather" utility is meant to index any path and generate a search |
34 | database. The "Hunt" utility will then open the database generated by |
||
35 | Gather and look for files matching the string provided to "Hunt" as |
||
1 | office | 36 | parameter. |
37 | |||
38 | The utility was designed to check large collections of icons but the |
||
39 | indexing and the fast search results proved indispensable enough for a |
||
40 | self-standing program. |
||
41 | |||
42 | -=:[ Design ]:=- |
||
43 | |||
37 | office | 44 | "Hunt" and "Gather" are designed with constant memory usage in order |
45 | to be suitable for all Amigas. Namely, the "Gather" utility will |
||
46 | search all files in a given path, sort the files in ascending order by |
||
47 | using an external file-based merge sort. |
||
1 | office | 48 | |
37 | office | 49 | Conversely, "Hunt" searchs files by reading lines from the database |
50 | without loading the entire database in RAM or by searching files |
||
51 | again. |
||
1 | office | 52 | |
53 | The project adheres to the ANSI C standard and Amiga-centric semantics |
||
54 | are compiled conditionally (in case the "___AmigaOS__" macro is |
||
37 | office | 55 | defined at compile time). Otherwise, "Hunt" and "Gather" should run |
56 | under any platform that benefits from an ANSI C compiler. |
||
57 | |||
58 | The project is developed from scratch on a real Amiga using StormC. |
||
1 | office | 59 | |
60 | -=:[ Usage ]:=- |
||
61 | |||
37 | office | 62 | The "Gather" utility is used to index a path. The following command: |
1 | office | 63 | |
64 | |||
37 | office | 65 | Gather -c RAM: |
1 | office | 66 | |
67 | |||
37 | office | 68 | will create a file in the S: directory named "gather.db". "Gather"is |
69 | verbose by default and will show the user what the utility is doing |
||
70 | but the behaviour can be changed with the "-q" (quiet) flag that will |
||
71 | make "Gather" print only errors. |
||
1 | office | 72 | |
37 | office | 73 | In order to look for a file, the "Hunt" utility is then invoked with |
74 | an AmigaOS search pattern, for instance, the pattern "#?test#?: |
||
1 | office | 75 | |
76 | |||
3 | office | 77 | Hunt #?test#? |
1 | office | 78 | |
79 | |||
37 | office | 80 | "Hunt" will then search the database previously generated by the |
81 | "Gather" utility and will print out all the paths corresponding to the |
||
82 | files matching the supplied pattern. |
||
83 | |||
84 | In the previous example, in case any of the files previously indexed |
||
85 | by "Gather" contain the term "test", then the "Hunt" utility will |
||
22 | office | 86 | display the path to the file. |
37 | office | 87 | |
88 | At some point you might decide to add some other path to the search |
||
89 | database as well. In that case, "Gather" would be invoked with the |
||
90 | "-a" option instead of "-c" in order to add the files: |
||
91 | |||
92 | Gather -a HDH0:Icons/ |
||
93 | |||
94 | "Gather" will then index the additional directory and add the new |
||
95 | files to the database. Adding a path to the index database will |
||
96 | require that "Gather" sorts the database again such that after adding |
||
97 | the new files, "Gather" will proceed with sorting. |
||
98 | |||
99 | Lastly, the "-r" parameter can be used with "Gather" to remove paths |
||
100 | that have been previously indexed. Let's say that you have indexed the |
||
101 | following paths with "Gather": |
||
102 | |||
103 | RAM: |
||
104 | HDH0:Icons/ |
||
105 | |||
106 | but now you would like to remove the "RAM:" path and all the files |
||
107 | indexed below that path. In that case, you would issue a "Gather" |
||
108 | command with the "-r" parameter: |
||
109 | |||
110 | Gather -r RAM: |
||
111 | |||
112 | and "Gather" will remove all files matching the "RAM:" path. Removing |
||
113 | a path with the "-r" parameter does not take a long time compared to |
||
114 | adding files to the database. |
||
1 | office | 115 | |
116 | -=:[ Gather ]:=- |
||
117 | |||
37 | office | 118 | "Gather" requires that one of the following parameters is specified: |
119 | * -a (add files to an already existing database), |
||
120 | * -r (remove files from an already existing database), |
||
121 | * -c (delete the previous database file and create a new database). |
||
1 | office | 122 | |
37 | office | 123 | The "Gather" utility takes several paths as parameters representing |
124 | the paths to be indexed; for example, all the following paths are |
||
125 | valid: |
||
1 | office | 126 | |
37 | office | 127 | |
1 | office | 128 | RAM: |
129 | DH0:System/ |
||
130 | |||
131 | |||
37 | office | 132 | When the "Gather" utility runs, a database is created at "S:gather.db" |
133 | containing all the found files. |
||
134 | |||
135 | "Gather" is also happy to work with a different database file other |
||
136 | than the default database at "S:gather.db" by passing the "-d" |
||
137 | parameter when "Gather" is invoked. For instance, the following |
||
138 | command invocation will create the database file at "T:gather.db" and |
||
139 | index the paths "RAM:" and "HDH0:Icons": |
||
140 | |||
141 | Gather -d T:gather.db RAM: HDH0:Icons |
||
142 | |||
143 | Conversely, the "Hunt" utility can then be used to search specified |
||
144 | database files: |
||
145 | |||
146 | Hunt -d T:gather.db #?test?# |
||
147 | |||
148 | The previous "Hunt" command will search a database file located at |
||
149 | "T:gather.db" for all files matching the pattern "#?test?". |
||
1 | office | 150 | |
151 | -=:[ Hunt ]:=- |
||
152 | |||
37 | office | 153 | "Hunt" is the counterpart to "Gather" and will search a given database |
154 | generated by the "Gather" utility for files matching the terms passed |
||
155 | to "Hunt" on the command line. |
||
1 | office | 156 | |
3 | office | 157 | For instance: |
1 | office | 158 | |
3 | office | 159 | Hunt #?test#? |
1 | office | 160 | |
37 | office | 161 | will search all files in the "Gather" database "S:gather.db" for the |
1 | office | 162 | term "test". If any file within the database partially matches the |
37 | office | 163 | term "test", then "Hunt" will display the path on the command line. |
3 | office | 164 | |
37 | office | 165 | "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS. |
166 | |||
167 | -=:[ Notes ]:=- |
||
168 | |||
169 | * The "Gather" utility will be slow and that is the intended |
||
170 | behaviour: slow indexing with "Gather", fast searching with "Hunt". |
||
171 | |||
172 | * Temporary files might end up created in the same location where the |
||
173 | "Gather" utility is invoked. Traditionally the temporary directory |
||
174 | on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it |
||
175 | intends to index very large hierarchies. Fortunately, "Gather"will |
||
176 | delete the temporary files once "Gather" is done indexing. |
||
177 | |||
178 | Nevertheless, in case you intend to index a large filesystem |
||
179 | hierarchy please make sure that you invoke "Gather" from a directory |
||
180 | that is able to hold large temporary files. |
||
181 | |||
182 | * The output of the "Hunt" utility can be combined with the pipe |
||
183 | operator (in newer AmigaOS releases) or the PIPE: handler on older |
||
184 | AmigaOS releases in order to to perform some action on the found |
||
185 | files. For example, using Thomas Radtke's "from" utility located at: |
||
186 | |||
187 | http://aminet.net/package/util/batch/from |
||
188 | |||
189 | and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5 |
||
190 | hashes of all files indexed by "Gather" ending in "#?.library: |
||
191 | |||
192 | Hunt #?.library | from - md5sum $1 |
||
193 | |||
194 | Or you could generate a list of versions of all libraries indexed |
||
195 | with the "Gahter" utility: |
||
196 | |||
197 | Hunt #?.library | from - version $1 |
||
198 | |||
3 | office | 199 | -=:[ Source ]:=- |
1 | office | 200 | |
3 | office | 201 | The project is open sourced and licensed under MIT. The source code |
202 | is included in the AmiNET release or can be checked out via subversion |
||
203 | from the Wizardry and Steamworks repository: |
||
1 | office | 204 | |
3 | office | 205 | svn co http://svn.grimore.org/HuntnGather |
206 | |||
37 | office | 207 | StormC was used as the developer environment. |
208 | |||
4 | office | 209 | -=:[ Mentions ]:=- |
210 | |||
211 | The code includes a shim for "getopt" in order to process command line |
||
37 | office | 212 | parameters on AmigaOS just like one would on a POSIX sytem. The shim |
213 | is created by Daniel J. Barrett, barrett@cs.umass.edu and is |
||
214 | available on AmiNET: |
||
4 | office | 215 | |
216 | http://aminet.net/package/dev/misc/GetOpt-1.3 |
||
217 | |||
1 | office | 218 | -=:[ Contact ]:=- |
219 | |||
22 | office | 220 | E-Mail(tor): |
221 | |||
222 | office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion |
||
223 | |||
1 | office | 224 | Website: https://grimore.org/amiga/hunt_and_gather |
22 | office | 225 | Website(tor): |
226 | |||
227 | http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/ |
||
228 | |||
1 | office | 229 | Discord: https://discord.gg/k9kyDsa |
230 |