HuntnGather – Blame information for rev 44
?pathlinks?
Rev | Author | Line No. | Line |
---|---|---|---|
1 | office | 1 | Short: File indexing and search utilities. |
2 | Author: Wizardry and Steamworks |
||
37 | office | 3 | Uploader: "Wizardry and Steamworks" <mail@grimore.org> |
1 | office | 4 | Type: util/dir |
5 | Replaces: util/dir/HuntnGather.lha |
||
44 | office | 6 | Version: 1.7.5 |
1 | office | 7 | Architecture: m68k-amigaos |
8 | |||
9 | Hunt & Gather - File search and indexing utilities. |
||
10 | (or Hunt'n'Gather bro, ghetto style) |
||
11 | |||
12 | © Copyright 2021 by Wizardry and Steamworks |
||
13 | |||
14 | Released under the MIT License, all rights reserved. |
||
15 | |||
22 | office | 16 | -=:[ Changes ]:=- |
1 | office | 17 | |
44 | office | 18 | 20211114: |
20 | office | 19 | |
44 | office | 20 | * Rewrite and preparations for pooling memory the Amiga way. |
21 | |||
1 | office | 22 | -=:[ Introduction ]:=- |
23 | |||
24 | |||
37 | office | 25 | "Hunt" and "Gather" are two utiltities for indexing and then searching |
1 | office | 26 | fileswithin a drive or directory designed to speed up searching files. |
27 | |||
37 | office | 28 | The "Gather" utility is meant to index any path and generate a search |
29 | database. The "Hunt" utility will then open the database generated by |
||
30 | Gather and look for files matching the string provided to "Hunt" as |
||
1 | office | 31 | parameter. |
32 | |||
33 | The utility was designed to check large collections of icons but the |
||
34 | indexing and the fast search results proved indispensable enough for a |
||
35 | self-standing program. |
||
36 | |||
37 | -=:[ Design ]:=- |
||
38 | |||
37 | office | 39 | "Hunt" and "Gather" are designed with constant memory usage in order |
40 | to be suitable for all Amigas. Namely, the "Gather" utility will |
||
41 | search all files in a given path, sort the files in ascending order by |
||
42 | using an external file-based merge sort. |
||
1 | office | 43 | |
37 | office | 44 | Conversely, "Hunt" searchs files by reading lines from the database |
45 | without loading the entire database in RAM or by searching files |
||
46 | again. |
||
1 | office | 47 | |
48 | The project adheres to the ANSI C standard and Amiga-centric semantics |
||
49 | are compiled conditionally (in case the "___AmigaOS__" macro is |
||
37 | office | 50 | defined at compile time). Otherwise, "Hunt" and "Gather" should run |
51 | under any platform that benefits from an ANSI C compiler. |
||
52 | |||
53 | The project is developed from scratch on a real Amiga using StormC. |
||
1 | office | 54 | |
55 | -=:[ Usage ]:=- |
||
56 | |||
37 | office | 57 | The "Gather" utility is used to index a path. The following command: |
1 | office | 58 | |
44 | office | 59 | Gather -c RAM: |
1 | office | 60 | |
37 | office | 61 | will create a file in the S: directory named "gather.db". "Gather"is |
62 | verbose by default and will show the user what the utility is doing |
||
63 | but the behaviour can be changed with the "-q" (quiet) flag that will |
||
64 | make "Gather" print only errors. |
||
1 | office | 65 | |
44 | office | 66 | In order to look for a file, the "Hunt" utility is then invoked with |
67 | an AmigaOS search pattern, for instance, the pattern "#?test#?: |
||
1 | office | 68 | |
44 | office | 69 | Hunt #?test#? |
1 | office | 70 | |
37 | office | 71 | "Hunt" will then search the database previously generated by the |
72 | "Gather" utility and will print out all the paths corresponding to the |
||
73 | files matching the supplied pattern. |
||
74 | |||
75 | In the previous example, in case any of the files previously indexed |
||
76 | by "Gather" contain the term "test", then the "Hunt" utility will |
||
22 | office | 77 | display the path to the file. |
37 | office | 78 | |
79 | At some point you might decide to add some other path to the search |
||
80 | database as well. In that case, "Gather" would be invoked with the |
||
81 | "-a" option instead of "-c" in order to add the files: |
||
82 | |||
83 | Gather -a HDH0:Icons/ |
||
84 | |||
85 | "Gather" will then index the additional directory and add the new |
||
86 | files to the database. Adding a path to the index database will |
||
87 | require that "Gather" sorts the database again such that after adding |
||
88 | the new files, "Gather" will proceed with sorting. |
||
89 | |||
90 | Lastly, the "-r" parameter can be used with "Gather" to remove paths |
||
91 | that have been previously indexed. Let's say that you have indexed the |
||
92 | following paths with "Gather": |
||
93 | |||
94 | RAM: |
||
95 | HDH0:Icons/ |
||
96 | |||
97 | but now you would like to remove the "RAM:" path and all the files |
||
98 | indexed below that path. In that case, you would issue a "Gather" |
||
99 | command with the "-r" parameter: |
||
100 | |||
101 | Gather -r RAM: |
||
102 | |||
103 | and "Gather" will remove all files matching the "RAM:" path. Removing |
||
104 | a path with the "-r" parameter does not take a long time compared to |
||
105 | adding files to the database. |
||
1 | office | 106 | |
107 | -=:[ Gather ]:=- |
||
108 | |||
44 | office | 109 | "Gather" requires that one of the following parameters is specified: |
110 | * -a (add files to an already existing database), |
||
111 | * -r (remove files from an already existing database), |
||
112 | * -c (delete the previous database file and create a new database). |
||
1 | office | 113 | |
44 | office | 114 | The "Gather" utility takes several paths as parameters representing |
115 | the paths to be indexed; for example, all the following paths are |
||
116 | valid: |
||
1 | office | 117 | |
44 | office | 118 | RAM: |
119 | DH0:System/ |
||
37 | office | 120 | |
44 | office | 121 | When the "Gather" utility runs, a database is created at "S:gather.db" |
122 | containing all the found files. |
||
37 | office | 123 | |
44 | office | 124 | "Gather" is also happy to work with a different database file other |
125 | than the default database at "S:gather.db" by passing the "-d" |
||
126 | parameter when "Gather" is invoked. For instance, the following |
||
127 | command invocation will create the database file at "T:gather.db" and |
||
128 | index the paths "RAM:" and "HDH0:Icons": |
||
37 | office | 129 | |
44 | office | 130 | Gather -d T:gather.db RAM: HDH0:Icons |
37 | office | 131 | |
44 | office | 132 | Conversely, the "Hunt" utility can then be used to search specified |
133 | database files: |
||
37 | office | 134 | |
44 | office | 135 | Hunt -d T:gather.db #?test?# |
37 | office | 136 | |
44 | office | 137 | The previous "Hunt" command will search a database file located at |
138 | "T:gather.db" for all files matching the pattern "#?test?". |
||
1 | office | 139 | |
140 | -=:[ Hunt ]:=- |
||
141 | |||
44 | office | 142 | "Hunt" is the counterpart to "Gather" and will search a given database |
143 | generated by the "Gather" utility for files matching the terms passed |
||
144 | to "Hunt" on the command line. |
||
1 | office | 145 | |
3 | office | 146 | For instance: |
1 | office | 147 | |
3 | office | 148 | Hunt #?test#? |
1 | office | 149 | |
37 | office | 150 | will search all files in the "Gather" database "S:gather.db" for the |
1 | office | 151 | term "test". If any file within the database partially matches the |
37 | office | 152 | term "test", then "Hunt" will display the path on the command line. |
3 | office | 153 | |
37 | office | 154 | "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS. |
155 | |||
156 | -=:[ Notes ]:=- |
||
157 | |||
158 | * The "Gather" utility will be slow and that is the intended |
||
159 | behaviour: slow indexing with "Gather", fast searching with "Hunt". |
||
160 | |||
161 | * Temporary files might end up created in the same location where the |
||
162 | "Gather" utility is invoked. Traditionally the temporary directory |
||
163 | on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it |
||
164 | intends to index very large hierarchies. Fortunately, "Gather"will |
||
165 | delete the temporary files once "Gather" is done indexing. |
||
166 | |||
167 | Nevertheless, in case you intend to index a large filesystem |
||
168 | hierarchy please make sure that you invoke "Gather" from a directory |
||
169 | that is able to hold large temporary files. |
||
170 | |||
171 | * The output of the "Hunt" utility can be combined with the pipe |
||
172 | operator (in newer AmigaOS releases) or the PIPE: handler on older |
||
173 | AmigaOS releases in order to to perform some action on the found |
||
174 | files. For example, using Thomas Radtke's "from" utility located at: |
||
175 | |||
176 | http://aminet.net/package/util/batch/from |
||
177 | |||
178 | and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5 |
||
179 | hashes of all files indexed by "Gather" ending in "#?.library: |
||
180 | |||
181 | Hunt #?.library | from - md5sum $1 |
||
182 | |||
183 | Or you could generate a list of versions of all libraries indexed |
||
184 | with the "Gahter" utility: |
||
185 | |||
186 | Hunt #?.library | from - version $1 |
||
187 | |||
3 | office | 188 | -=:[ Source ]:=- |
1 | office | 189 | |
3 | office | 190 | The project is open sourced and licensed under MIT. The source code |
191 | is included in the AmiNET release or can be checked out via subversion |
||
192 | from the Wizardry and Steamworks repository: |
||
1 | office | 193 | |
3 | office | 194 | svn co http://svn.grimore.org/HuntnGather |
195 | |||
37 | office | 196 | StormC was used as the developer environment. |
197 | |||
4 | office | 198 | -=:[ Mentions ]:=- |
199 | |||
200 | The code includes a shim for "getopt" in order to process command line |
||
37 | office | 201 | parameters on AmigaOS just like one would on a POSIX sytem. The shim |
202 | is created by Daniel J. Barrett, barrett@cs.umass.edu and is |
||
203 | available on AmiNET: |
||
4 | office | 204 | |
205 | http://aminet.net/package/dev/misc/GetOpt-1.3 |
||
206 | |||
1 | office | 207 | -=:[ Contact ]:=- |
208 | |||
22 | office | 209 | E-Mail(tor): |
210 | |||
211 | office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion |
||
212 | |||
1 | office | 213 | Website: https://grimore.org/amiga/hunt_and_gather |
22 | office | 214 | Website(tor): |
215 | |||
216 | http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/ |
||
217 | |||
1 | office | 218 | Discord: https://discord.gg/k9kyDsa |
219 |