HuntnGather – Blame information for rev 53
?pathlinks?
Rev | Author | Line No. | Line |
---|---|---|---|
1 | office | 1 | Short: File indexing and search utilities. |
2 | Author: Wizardry and Steamworks |
||
37 | office | 3 | Uploader: "Wizardry and Steamworks" <mail@grimore.org> |
1 | office | 4 | Type: util/dir |
5 | Replaces: util/dir/HuntnGather.lha |
||
53 | office | 6 | Version: 1.7.7 |
1 | office | 7 | Architecture: m68k-amigaos |
8 | |||
9 | Hunt & Gather - File search and indexing utilities. |
||
10 | (or Hunt'n'Gather bro, ghetto style) |
||
11 | |||
12 | © Copyright 2021 by Wizardry and Steamworks |
||
13 | |||
14 | Released under the MIT License, all rights reserved. |
||
15 | |||
22 | office | 16 | -=:[ Changes ]:=- |
1 | office | 17 | |
52 | office | 18 | 20211206: |
19 | * Compile pattern only once before searching through files. |
||
44 | office | 20 | |
1 | office | 21 | -=:[ Introduction ]:=- |
22 | |||
23 | |||
37 | office | 24 | "Hunt" and "Gather" are two utiltities for indexing and then searching |
1 | office | 25 | fileswithin a drive or directory designed to speed up searching files. |
26 | |||
37 | office | 27 | The "Gather" utility is meant to index any path and generate a search |
28 | database. The "Hunt" utility will then open the database generated by |
||
29 | Gather and look for files matching the string provided to "Hunt" as |
||
1 | office | 30 | parameter. |
31 | |||
32 | The utility was designed to check large collections of icons but the |
||
33 | indexing and the fast search results proved indispensable enough for a |
||
34 | self-standing program. |
||
35 | |||
36 | -=:[ Design ]:=- |
||
37 | |||
37 | office | 38 | "Hunt" and "Gather" are designed with constant memory usage in order |
39 | to be suitable for all Amigas. Namely, the "Gather" utility will |
||
40 | search all files in a given path, sort the files in ascending order by |
||
41 | using an external file-based merge sort. |
||
1 | office | 42 | |
37 | office | 43 | Conversely, "Hunt" searchs files by reading lines from the database |
44 | without loading the entire database in RAM or by searching files |
||
45 | again. |
||
1 | office | 46 | |
47 | The project adheres to the ANSI C standard and Amiga-centric semantics |
||
48 | are compiled conditionally (in case the "___AmigaOS__" macro is |
||
37 | office | 49 | defined at compile time). Otherwise, "Hunt" and "Gather" should run |
50 | under any platform that benefits from an ANSI C compiler. |
||
51 | |||
52 | The project is developed from scratch on a real Amiga using StormC. |
||
1 | office | 53 | |
54 | -=:[ Usage ]:=- |
||
55 | |||
37 | office | 56 | The "Gather" utility is used to index a path. The following command: |
1 | office | 57 | |
44 | office | 58 | Gather -c RAM: |
1 | office | 59 | |
37 | office | 60 | will create a file in the S: directory named "gather.db". "Gather"is |
61 | verbose by default and will show the user what the utility is doing |
||
62 | but the behaviour can be changed with the "-q" (quiet) flag that will |
||
63 | make "Gather" print only errors. |
||
1 | office | 64 | |
44 | office | 65 | In order to look for a file, the "Hunt" utility is then invoked with |
66 | an AmigaOS search pattern, for instance, the pattern "#?test#?: |
||
1 | office | 67 | |
44 | office | 68 | Hunt #?test#? |
1 | office | 69 | |
37 | office | 70 | "Hunt" will then search the database previously generated by the |
71 | "Gather" utility and will print out all the paths corresponding to the |
||
72 | files matching the supplied pattern. |
||
73 | |||
74 | In the previous example, in case any of the files previously indexed |
||
75 | by "Gather" contain the term "test", then the "Hunt" utility will |
||
22 | office | 76 | display the path to the file. |
37 | office | 77 | |
78 | At some point you might decide to add some other path to the search |
||
79 | database as well. In that case, "Gather" would be invoked with the |
||
80 | "-a" option instead of "-c" in order to add the files: |
||
81 | |||
82 | Gather -a HDH0:Icons/ |
||
83 | |||
84 | "Gather" will then index the additional directory and add the new |
||
85 | files to the database. Adding a path to the index database will |
||
86 | require that "Gather" sorts the database again such that after adding |
||
87 | the new files, "Gather" will proceed with sorting. |
||
88 | |||
89 | Lastly, the "-r" parameter can be used with "Gather" to remove paths |
||
90 | that have been previously indexed. Let's say that you have indexed the |
||
91 | following paths with "Gather": |
||
92 | |||
93 | RAM: |
||
94 | HDH0:Icons/ |
||
95 | |||
96 | but now you would like to remove the "RAM:" path and all the files |
||
97 | indexed below that path. In that case, you would issue a "Gather" |
||
98 | command with the "-r" parameter: |
||
99 | |||
100 | Gather -r RAM: |
||
101 | |||
102 | and "Gather" will remove all files matching the "RAM:" path. Removing |
||
103 | a path with the "-r" parameter does not take a long time compared to |
||
104 | adding files to the database. |
||
1 | office | 105 | |
106 | -=:[ Gather ]:=- |
||
107 | |||
44 | office | 108 | "Gather" requires that one of the following parameters is specified: |
109 | * -a (add files to an already existing database), |
||
110 | * -r (remove files from an already existing database), |
||
111 | * -c (delete the previous database file and create a new database). |
||
1 | office | 112 | |
44 | office | 113 | The "Gather" utility takes several paths as parameters representing |
114 | the paths to be indexed; for example, all the following paths are |
||
115 | valid: |
||
1 | office | 116 | |
44 | office | 117 | RAM: |
118 | DH0:System/ |
||
37 | office | 119 | |
44 | office | 120 | When the "Gather" utility runs, a database is created at "S:gather.db" |
121 | containing all the found files. |
||
37 | office | 122 | |
44 | office | 123 | "Gather" is also happy to work with a different database file other |
124 | than the default database at "S:gather.db" by passing the "-d" |
||
125 | parameter when "Gather" is invoked. For instance, the following |
||
126 | command invocation will create the database file at "T:gather.db" and |
||
127 | index the paths "RAM:" and "HDH0:Icons": |
||
37 | office | 128 | |
44 | office | 129 | Gather -d T:gather.db RAM: HDH0:Icons |
37 | office | 130 | |
44 | office | 131 | Conversely, the "Hunt" utility can then be used to search specified |
132 | database files: |
||
37 | office | 133 | |
44 | office | 134 | Hunt -d T:gather.db #?test?# |
37 | office | 135 | |
44 | office | 136 | The previous "Hunt" command will search a database file located at |
137 | "T:gather.db" for all files matching the pattern "#?test?". |
||
1 | office | 138 | |
139 | -=:[ Hunt ]:=- |
||
140 | |||
44 | office | 141 | "Hunt" is the counterpart to "Gather" and will search a given database |
142 | generated by the "Gather" utility for files matching the terms passed |
||
143 | to "Hunt" on the command line. |
||
1 | office | 144 | |
3 | office | 145 | For instance: |
1 | office | 146 | |
3 | office | 147 | Hunt #?test#? |
1 | office | 148 | |
37 | office | 149 | will search all files in the "Gather" database "S:gather.db" for the |
1 | office | 150 | term "test". If any file within the database partially matches the |
37 | office | 151 | term "test", then "Hunt" will display the path on the command line. |
3 | office | 152 | |
37 | office | 153 | "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS. |
154 | |||
155 | -=:[ Notes ]:=- |
||
156 | |||
157 | * The "Gather" utility will be slow and that is the intended |
||
158 | behaviour: slow indexing with "Gather", fast searching with "Hunt". |
||
159 | |||
160 | * Temporary files might end up created in the same location where the |
||
161 | "Gather" utility is invoked. Traditionally the temporary directory |
||
162 | on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it |
||
163 | intends to index very large hierarchies. Fortunately, "Gather"will |
||
164 | delete the temporary files once "Gather" is done indexing. |
||
165 | |||
166 | Nevertheless, in case you intend to index a large filesystem |
||
167 | hierarchy please make sure that you invoke "Gather" from a directory |
||
168 | that is able to hold large temporary files. |
||
169 | |||
170 | * The output of the "Hunt" utility can be combined with the pipe |
||
171 | operator (in newer AmigaOS releases) or the PIPE: handler on older |
||
172 | AmigaOS releases in order to to perform some action on the found |
||
173 | files. For example, using Thomas Radtke's "from" utility located at: |
||
174 | |||
175 | http://aminet.net/package/util/batch/from |
||
176 | |||
177 | and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5 |
||
178 | hashes of all files indexed by "Gather" ending in "#?.library: |
||
179 | |||
180 | Hunt #?.library | from - md5sum $1 |
||
181 | |||
182 | Or you could generate a list of versions of all libraries indexed |
||
183 | with the "Gahter" utility: |
||
184 | |||
185 | Hunt #?.library | from - version $1 |
||
186 | |||
3 | office | 187 | -=:[ Source ]:=- |
1 | office | 188 | |
3 | office | 189 | The project is open sourced and licensed under MIT. The source code |
190 | is included in the AmiNET release or can be checked out via subversion |
||
191 | from the Wizardry and Steamworks repository: |
||
1 | office | 192 | |
3 | office | 193 | svn co http://svn.grimore.org/HuntnGather |
194 | |||
37 | office | 195 | StormC was used as the developer environment. |
196 | |||
4 | office | 197 | -=:[ Mentions ]:=- |
198 | |||
199 | The code includes a shim for "getopt" in order to process command line |
||
37 | office | 200 | parameters on AmigaOS just like one would on a POSIX sytem. The shim |
201 | is created by Daniel J. Barrett, barrett@cs.umass.edu and is |
||
202 | available on AmiNET: |
||
4 | office | 203 | |
204 | http://aminet.net/package/dev/misc/GetOpt-1.3 |
||
205 | |||
1 | office | 206 | -=:[ Contact ]:=- |
207 | |||
22 | office | 208 | E-Mail(tor): |
209 | |||
210 | office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion |
||
211 | |||
1 | office | 212 | Website: https://grimore.org/amiga/hunt_and_gather |
22 | office | 213 | Website(tor): |
214 | |||
215 | http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/ |
||
216 | |||
1 | office | 217 | Discord: https://discord.gg/k9kyDsa |
218 |