HuntnGather – Diff between revs 52 and 53
?pathlinks?
Rev 52 | Rev 53 | |||
---|---|---|---|---|
1 | Short: File indexing and search utilities. |
1 | Short: File indexing and search utilities. |
|
2 | Author: Wizardry and Steamworks |
2 | Author: Wizardry and Steamworks |
|
3 | Uploader: "Wizardry and Steamworks" <mail@grimore.org> |
3 | Uploader: "Wizardry and Steamworks" <mail@grimore.org> |
|
4 | Type: util/dir |
4 | Type: util/dir |
|
5 | Replaces: util/dir/HuntnGather.lha |
5 | Replaces: util/dir/HuntnGather.lha |
|
6 | Version: 1.7.5 |
6 | Version: 1.7.7 |
|
7 | Architecture: m68k-amigaos |
7 | Architecture: m68k-amigaos |
|
8 | |
8 | |
|
9 | Hunt & Gather - File search and indexing utilities. |
9 | Hunt & Gather - File search and indexing utilities. |
|
10 | (or Hunt'n'Gather bro, ghetto style) |
10 | (or Hunt'n'Gather bro, ghetto style) |
|
11 | |
11 | |
|
12 | © Copyright 2021 by Wizardry and Steamworks |
12 | © Copyright 2021 by Wizardry and Steamworks |
|
13 | |
13 | |
|
14 | Released under the MIT License, all rights reserved. |
14 | Released under the MIT License, all rights reserved. |
|
15 | |
15 | |
|
16 | -=:[ Changes ]:=- |
16 | -=:[ Changes ]:=- |
|
17 | |
17 | |
|
18 | 20211206: |
18 | 20211206: |
|
19 | |
- | ||
20 | * Compile pattern only once before searching through files. |
19 | * Compile pattern only once before searching through files. |
|
21 | |
20 | |
|
22 | -=:[ Introduction ]:=- |
21 | -=:[ Introduction ]:=- |
|
23 | |
22 | |
|
24 | |
23 | |
|
25 | "Hunt" and "Gather" are two utiltities for indexing and then searching |
24 | "Hunt" and "Gather" are two utiltities for indexing and then searching |
|
26 | fileswithin a drive or directory designed to speed up searching files. |
25 | fileswithin a drive or directory designed to speed up searching files. |
|
27 | |
26 | |
|
28 | The "Gather" utility is meant to index any path and generate a search |
27 | The "Gather" utility is meant to index any path and generate a search |
|
29 | database. The "Hunt" utility will then open the database generated by |
28 | database. The "Hunt" utility will then open the database generated by |
|
30 | Gather and look for files matching the string provided to "Hunt" as |
29 | Gather and look for files matching the string provided to "Hunt" as |
|
31 | parameter. |
30 | parameter. |
|
32 | |
31 | |
|
33 | The utility was designed to check large collections of icons but the |
32 | The utility was designed to check large collections of icons but the |
|
34 | indexing and the fast search results proved indispensable enough for a |
33 | indexing and the fast search results proved indispensable enough for a |
|
35 | self-standing program. |
34 | self-standing program. |
|
36 | |
35 | |
|
37 | -=:[ Design ]:=- |
36 | -=:[ Design ]:=- |
|
38 | |
37 | |
|
39 | "Hunt" and "Gather" are designed with constant memory usage in order |
38 | "Hunt" and "Gather" are designed with constant memory usage in order |
|
40 | to be suitable for all Amigas. Namely, the "Gather" utility will |
39 | to be suitable for all Amigas. Namely, the "Gather" utility will |
|
41 | search all files in a given path, sort the files in ascending order by |
40 | search all files in a given path, sort the files in ascending order by |
|
42 | using an external file-based merge sort. |
41 | using an external file-based merge sort. |
|
43 | |
42 | |
|
44 | Conversely, "Hunt" searchs files by reading lines from the database |
43 | Conversely, "Hunt" searchs files by reading lines from the database |
|
45 | without loading the entire database in RAM or by searching files |
44 | without loading the entire database in RAM or by searching files |
|
46 | again. |
45 | again. |
|
47 | |
46 | |
|
48 | The project adheres to the ANSI C standard and Amiga-centric semantics |
47 | The project adheres to the ANSI C standard and Amiga-centric semantics |
|
49 | are compiled conditionally (in case the "___AmigaOS__" macro is |
48 | are compiled conditionally (in case the "___AmigaOS__" macro is |
|
50 | defined at compile time). Otherwise, "Hunt" and "Gather" should run |
49 | defined at compile time). Otherwise, "Hunt" and "Gather" should run |
|
51 | under any platform that benefits from an ANSI C compiler. |
50 | under any platform that benefits from an ANSI C compiler. |
|
52 | |
51 | |
|
53 | The project is developed from scratch on a real Amiga using StormC. |
52 | The project is developed from scratch on a real Amiga using StormC. |
|
54 | |
53 | |
|
55 | -=:[ Usage ]:=- |
54 | -=:[ Usage ]:=- |
|
56 | |
55 | |
|
57 | The "Gather" utility is used to index a path. The following command: |
56 | The "Gather" utility is used to index a path. The following command: |
|
58 | |
57 | |
|
59 | Gather -c RAM: |
58 | Gather -c RAM: |
|
60 | |
59 | |
|
61 | will create a file in the S: directory named "gather.db". "Gather"is |
60 | will create a file in the S: directory named "gather.db". "Gather"is |
|
62 | verbose by default and will show the user what the utility is doing |
61 | verbose by default and will show the user what the utility is doing |
|
63 | but the behaviour can be changed with the "-q" (quiet) flag that will |
62 | but the behaviour can be changed with the "-q" (quiet) flag that will |
|
64 | make "Gather" print only errors. |
63 | make "Gather" print only errors. |
|
65 | |
64 | |
|
66 | In order to look for a file, the "Hunt" utility is then invoked with |
65 | In order to look for a file, the "Hunt" utility is then invoked with |
|
67 | an AmigaOS search pattern, for instance, the pattern "#?test#?: |
66 | an AmigaOS search pattern, for instance, the pattern "#?test#?: |
|
68 | |
67 | |
|
69 | Hunt #?test#? |
68 | Hunt #?test#? |
|
70 | |
69 | |
|
71 | "Hunt" will then search the database previously generated by the |
70 | "Hunt" will then search the database previously generated by the |
|
72 | "Gather" utility and will print out all the paths corresponding to the |
71 | "Gather" utility and will print out all the paths corresponding to the |
|
73 | files matching the supplied pattern. |
72 | files matching the supplied pattern. |
|
74 | |
73 | |
|
75 | In the previous example, in case any of the files previously indexed |
74 | In the previous example, in case any of the files previously indexed |
|
76 | by "Gather" contain the term "test", then the "Hunt" utility will |
75 | by "Gather" contain the term "test", then the "Hunt" utility will |
|
77 | display the path to the file. |
76 | display the path to the file. |
|
78 | |
77 | |
|
79 | At some point you might decide to add some other path to the search |
78 | At some point you might decide to add some other path to the search |
|
80 | database as well. In that case, "Gather" would be invoked with the |
79 | database as well. In that case, "Gather" would be invoked with the |
|
81 | "-a" option instead of "-c" in order to add the files: |
80 | "-a" option instead of "-c" in order to add the files: |
|
82 | |
81 | |
|
83 | Gather -a HDH0:Icons/ |
82 | Gather -a HDH0:Icons/ |
|
84 | |
83 | |
|
85 | "Gather" will then index the additional directory and add the new |
84 | "Gather" will then index the additional directory and add the new |
|
86 | files to the database. Adding a path to the index database will |
85 | files to the database. Adding a path to the index database will |
|
87 | require that "Gather" sorts the database again such that after adding |
86 | require that "Gather" sorts the database again such that after adding |
|
88 | the new files, "Gather" will proceed with sorting. |
87 | the new files, "Gather" will proceed with sorting. |
|
89 | |
88 | |
|
90 | Lastly, the "-r" parameter can be used with "Gather" to remove paths |
89 | Lastly, the "-r" parameter can be used with "Gather" to remove paths |
|
91 | that have been previously indexed. Let's say that you have indexed the |
90 | that have been previously indexed. Let's say that you have indexed the |
|
92 | following paths with "Gather": |
91 | following paths with "Gather": |
|
93 | |
92 | |
|
94 | RAM: |
93 | RAM: |
|
95 | HDH0:Icons/ |
94 | HDH0:Icons/ |
|
96 | |
95 | |
|
97 | but now you would like to remove the "RAM:" path and all the files |
96 | but now you would like to remove the "RAM:" path and all the files |
|
98 | indexed below that path. In that case, you would issue a "Gather" |
97 | indexed below that path. In that case, you would issue a "Gather" |
|
99 | command with the "-r" parameter: |
98 | command with the "-r" parameter: |
|
100 | |
99 | |
|
101 | Gather -r RAM: |
100 | Gather -r RAM: |
|
102 | |
101 | |
|
103 | and "Gather" will remove all files matching the "RAM:" path. Removing |
102 | and "Gather" will remove all files matching the "RAM:" path. Removing |
|
104 | a path with the "-r" parameter does not take a long time compared to |
103 | a path with the "-r" parameter does not take a long time compared to |
|
105 | adding files to the database. |
104 | adding files to the database. |
|
106 | |
105 | |
|
107 | -=:[ Gather ]:=- |
106 | -=:[ Gather ]:=- |
|
108 | |
107 | |
|
109 | "Gather" requires that one of the following parameters is specified: |
108 | "Gather" requires that one of the following parameters is specified: |
|
110 | * -a (add files to an already existing database), |
109 | * -a (add files to an already existing database), |
|
111 | * -r (remove files from an already existing database), |
110 | * -r (remove files from an already existing database), |
|
112 | * -c (delete the previous database file and create a new database). |
111 | * -c (delete the previous database file and create a new database). |
|
113 | |
112 | |
|
114 | The "Gather" utility takes several paths as parameters representing |
113 | The "Gather" utility takes several paths as parameters representing |
|
115 | the paths to be indexed; for example, all the following paths are |
114 | the paths to be indexed; for example, all the following paths are |
|
116 | valid: |
115 | valid: |
|
117 | |
116 | |
|
118 | RAM: |
117 | RAM: |
|
119 | DH0:System/ |
118 | DH0:System/ |
|
120 | |
119 | |
|
121 | When the "Gather" utility runs, a database is created at "S:gather.db" |
120 | When the "Gather" utility runs, a database is created at "S:gather.db" |
|
122 | containing all the found files. |
121 | containing all the found files. |
|
123 | |
122 | |
|
124 | "Gather" is also happy to work with a different database file other |
123 | "Gather" is also happy to work with a different database file other |
|
125 | than the default database at "S:gather.db" by passing the "-d" |
124 | than the default database at "S:gather.db" by passing the "-d" |
|
126 | parameter when "Gather" is invoked. For instance, the following |
125 | parameter when "Gather" is invoked. For instance, the following |
|
127 | command invocation will create the database file at "T:gather.db" and |
126 | command invocation will create the database file at "T:gather.db" and |
|
128 | index the paths "RAM:" and "HDH0:Icons": |
127 | index the paths "RAM:" and "HDH0:Icons": |
|
129 | |
128 | |
|
130 | Gather -d T:gather.db RAM: HDH0:Icons |
129 | Gather -d T:gather.db RAM: HDH0:Icons |
|
131 | |
130 | |
|
132 | Conversely, the "Hunt" utility can then be used to search specified |
131 | Conversely, the "Hunt" utility can then be used to search specified |
|
133 | database files: |
132 | database files: |
|
134 | |
133 | |
|
135 | Hunt -d T:gather.db #?test?# |
134 | Hunt -d T:gather.db #?test?# |
|
136 | |
135 | |
|
137 | The previous "Hunt" command will search a database file located at |
136 | The previous "Hunt" command will search a database file located at |
|
138 | "T:gather.db" for all files matching the pattern "#?test?". |
137 | "T:gather.db" for all files matching the pattern "#?test?". |
|
139 | |
138 | |
|
140 | -=:[ Hunt ]:=- |
139 | -=:[ Hunt ]:=- |
|
141 | |
140 | |
|
142 | "Hunt" is the counterpart to "Gather" and will search a given database |
141 | "Hunt" is the counterpart to "Gather" and will search a given database |
|
143 | generated by the "Gather" utility for files matching the terms passed |
142 | generated by the "Gather" utility for files matching the terms passed |
|
144 | to "Hunt" on the command line. |
143 | to "Hunt" on the command line. |
|
145 | |
144 | |
|
146 | For instance: |
145 | For instance: |
|
147 | |
146 | |
|
148 | Hunt #?test#? |
147 | Hunt #?test#? |
|
149 | |
148 | |
|
150 | will search all files in the "Gather" database "S:gather.db" for the |
149 | will search all files in the "Gather" database "S:gather.db" for the |
|
151 | term "test". If any file within the database partially matches the |
150 | term "test". If any file within the database partially matches the |
|
152 | term "test", then "Hunt" will display the path on the command line. |
151 | term "test", then "Hunt" will display the path on the command line. |
|
153 | |
152 | |
|
154 | "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS. |
153 | "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS. |
|
155 | |
154 | |
|
156 | -=:[ Notes ]:=- |
155 | -=:[ Notes ]:=- |
|
157 | |
156 | |
|
158 | * The "Gather" utility will be slow and that is the intended |
157 | * The "Gather" utility will be slow and that is the intended |
|
159 | behaviour: slow indexing with "Gather", fast searching with "Hunt". |
158 | behaviour: slow indexing with "Gather", fast searching with "Hunt". |
|
160 | |
159 | |
|
161 | * Temporary files might end up created in the same location where the |
160 | * Temporary files might end up created in the same location where the |
|
162 | "Gather" utility is invoked. Traditionally the temporary directory |
161 | "Gather" utility is invoked. Traditionally the temporary directory |
|
163 | on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it |
162 | on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it |
|
164 | intends to index very large hierarchies. Fortunately, "Gather"will |
163 | intends to index very large hierarchies. Fortunately, "Gather"will |
|
165 | delete the temporary files once "Gather" is done indexing. |
164 | delete the temporary files once "Gather" is done indexing. |
|
166 | |
165 | |
|
167 | Nevertheless, in case you intend to index a large filesystem |
166 | Nevertheless, in case you intend to index a large filesystem |
|
168 | hierarchy please make sure that you invoke "Gather" from a directory |
167 | hierarchy please make sure that you invoke "Gather" from a directory |
|
169 | that is able to hold large temporary files. |
168 | that is able to hold large temporary files. |
|
170 | |
169 | |
|
171 | * The output of the "Hunt" utility can be combined with the pipe |
170 | * The output of the "Hunt" utility can be combined with the pipe |
|
172 | operator (in newer AmigaOS releases) or the PIPE: handler on older |
171 | operator (in newer AmigaOS releases) or the PIPE: handler on older |
|
173 | AmigaOS releases in order to to perform some action on the found |
172 | AmigaOS releases in order to to perform some action on the found |
|
174 | files. For example, using Thomas Radtke's "from" utility located at: |
173 | files. For example, using Thomas Radtke's "from" utility located at: |
|
175 | |
174 | |
|
176 | http://aminet.net/package/util/batch/from |
175 | http://aminet.net/package/util/batch/from |
|
177 | |
176 | |
|
178 | and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5 |
177 | and the Workbench 3.2 "MD5Sum" utility, you could print out the MD5 |
|
179 | hashes of all files indexed by "Gather" ending in "#?.library: |
178 | hashes of all files indexed by "Gather" ending in "#?.library: |
|
180 | |
179 | |
|
181 | Hunt #?.library | from - md5sum $1 |
180 | Hunt #?.library | from - md5sum $1 |
|
182 | |
181 | |
|
183 | Or you could generate a list of versions of all libraries indexed |
182 | Or you could generate a list of versions of all libraries indexed |
|
184 | with the "Gahter" utility: |
183 | with the "Gahter" utility: |
|
185 | |
184 | |
|
186 | Hunt #?.library | from - version $1 |
185 | Hunt #?.library | from - version $1 |
|
187 | |
186 | |
|
188 | -=:[ Source ]:=- |
187 | -=:[ Source ]:=- |
|
189 | |
188 | |
|
190 | The project is open sourced and licensed under MIT. The source code |
189 | The project is open sourced and licensed under MIT. The source code |
|
191 | is included in the AmiNET release or can be checked out via subversion |
190 | is included in the AmiNET release or can be checked out via subversion |
|
192 | from the Wizardry and Steamworks repository: |
191 | from the Wizardry and Steamworks repository: |
|
193 | |
192 | |
|
194 | svn co http://svn.grimore.org/HuntnGather |
193 | svn co http://svn.grimore.org/HuntnGather |
|
195 | |
194 | |
|
196 | StormC was used as the developer environment. |
195 | StormC was used as the developer environment. |
|
197 | |
196 | |
|
198 | -=:[ Mentions ]:=- |
197 | -=:[ Mentions ]:=- |
|
199 | |
198 | |
|
200 | The code includes a shim for "getopt" in order to process command line |
199 | The code includes a shim for "getopt" in order to process command line |
|
201 | parameters on AmigaOS just like one would on a POSIX sytem. The shim |
200 | parameters on AmigaOS just like one would on a POSIX sytem. The shim |
|
202 | is created by Daniel J. Barrett, barrett@cs.umass.edu and is |
201 | is created by Daniel J. Barrett, barrett@cs.umass.edu and is |
|
203 | available on AmiNET: |
202 | available on AmiNET: |
|
204 | |
203 | |
|
205 | http://aminet.net/package/dev/misc/GetOpt-1.3 |
204 | http://aminet.net/package/dev/misc/GetOpt-1.3 |
|
206 | |
205 | |
|
207 | -=:[ Contact ]:=- |
206 | -=:[ Contact ]:=- |
|
208 | |
207 | |
|
209 | E-Mail(tor): |
208 | E-Mail(tor): |
|
210 | |
209 | |
|
211 | office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion |
210 | office@3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion |
|
212 | |
211 | |
|
213 | Website: https://grimore.org/amiga/hunt_and_gather |
212 | Website: https://grimore.org/amiga/hunt_and_gather |
|
214 | Website(tor): |
213 | Website(tor): |
|
215 | |
214 | |
|
216 | http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/ |
215 | http://3wymlmcsvxiaqzmbepsdawqpk6o2qsk65jhms72qqjulk5u4bgmvs3qd.onion/ |
|
217 | |
216 | |
|
218 | Discord: https://discord.gg/k9kyDsa |
217 | Discord: https://discord.gg/k9kyDsa |
|
219 | |
218 | |
|
220 | |
219 | |