HuntnGather – Diff between revs 31 and 37
?pathlinks?
Rev 31 | Rev 37 | |||
---|---|---|---|---|
Line 1... | Line 1... | |||
1 | Short: File indexing and search utilities. |
1 | Short: File indexing and search utilities. |
|
2 | Author: Wizardry and Steamworks |
2 | Author: Wizardry and Steamworks |
|
3 | Uploader: "Wizardry and Steamworks" <office@grimore.org> |
3 | Uploader: "Wizardry and Steamworks" <mail@grimore.org> |
|
4 | Type: util/dir |
4 | Type: util/dir |
|
5 | Replaces: util/dir/HuntnGather.lha |
5 | Replaces: util/dir/HuntnGather.lha |
|
6 | Version: 1.7.3 |
6 | Version: 1.7.4 |
|
7 | Architecture: m68k-amigaos |
7 | Architecture: m68k-amigaos |
|
Line 8... | Line 8... | |||
8 | |
8 | |
|
9 | Hunt & Gather - File search and indexing utilities. |
9 | Hunt & Gather - File search and indexing utilities. |
|
Line 13... | Line 13... | |||
13 | |
13 | |
|
Line 14... | Line 14... | |||
14 | Released under the MIT License, all rights reserved. |
14 | Released under the MIT License, all rights reserved. |
|
Line 15... | Line 15... | |||
15 | |
15 | |
|
16 | -=:[ Changes ]:=- |
16 | -=:[ Changes ]:=- |
|
- | 17 | |
||
- | 18 | 20211105: |
||
17 | |
19 | * Use a single binary for all CPUs. |
|
18 | TBA: |
20 | * Refocus the documentation to match the latest changes. |
|
19 | * Testing with MuForce. |
21 | * MuForce and MuGuardianAngel memory usage stress testing. |
|
20 | * Tested using large entire drives and eliminated some bugs. |
22 | * Large storage (cca. 8k directories / 80k files) stress testing. |
|
Line 21... | Line 23... | |||
21 | * Use locale for string comparisons. |
23 | * Use locale for string comparisons on the Amiga. |
|
Line 22... | Line 24... | |||
22 | * Gather now requires either of -a (add), -r (remove), -c (create). |
24 | * "Gather" now requires either of -a (add), -r (remove), -c (create). |
|
23 | * Allow specifying multiple paths when gathering. |
25 | * Allow multiple paths when issuing the "Gather" command. |
|
Line 24... | Line 26... | |||
24 | |
26 | |
|
25 | -=:[ Introduction ]:=- |
27 | -=:[ Introduction ]:=- |
|
26 | |
28 | |
|
27 | |
29 | |
|
Line 28... | Line 30... | |||
28 | Hunt and Gather are two utiltities for indexing and then searching |
30 | "Hunt" and "Gather" are two utiltities for indexing and then searching |
|
29 | fileswithin a drive or directory designed to speed up searching files. |
31 | fileswithin a drive or directory designed to speed up searching files. |
|
30 | |
32 | |
|
Line 31... | Line 33... | |||
31 | The Gather utility is meant to index any path and generate a search |
33 | The "Gather" utility is meant to index any path and generate a search |
|
Line 32... | Line 34... | |||
32 | database. The Hunt utility will then open the database generated by |
34 | database. The "Hunt" utility will then open the database generated by |
|
33 | Gather and look for files matching the string provided to Hunt as |
35 | Gather and look for files matching the string provided to "Hunt" as |
|
34 | parameter. |
36 | parameter. |
|
35 | |
37 | |
|
36 | The utility was designed to check large collections of icons but the |
38 | The utility was designed to check large collections of icons but the |
|
37 | indexing and the fast search results proved indispensable enough for a |
39 | indexing and the fast search results proved indispensable enough for a |
|
38 | self-standing program. |
40 | self-standing program. |
|
39 | |
- | ||
40 | -=:[ Design ]:=- |
- | ||
41 | |
41 | |
|
Line 42... | Line 42... | |||
42 | Hunt and Gather are designed with constant memory usage in order to be |
42 | -=:[ Design ]:=- |
|
43 | suitable for all Amiga models. Namely, the Gather utility will search |
43 | |
|
44 | all files in a given path, sort the files in ascending order by using |
44 | "Hunt" and "Gather" are designed with constant memory usage in order |
|
45 | an external merge sort (tailored down to a 256KiB memory limit). |
45 | to be suitable for all Amigas. Namely, the "Gather" utility will |
|
- | 46 | search all files in a given path, sort the files in ascending order by |
||
- | 47 | using an external file-based merge sort. |
||
Line 46... | Line 48... | |||
46 | |
48 | |
|
Line 47... | Line 49... | |||
47 | Conversely, Hunt uses brute force to search for files but by reading |
49 | Conversely, "Hunt" searchs files by reading lines from the database |
|
Line 48... | Line 50... | |||
48 | lines from the database without loading the entire database in RAM. |
50 | without loading the entire database in RAM or by searching files |
|
Line 49... | Line 51... | |||
49 | Perhaps ulterior versions of Hunt might partition the database file |
51 | again. |
|
50 | just like Gather does and then build Tries in oder to speed up finding |
52 | |
|
- | 53 | The project adheres to the ANSI C standard and Amiga-centric semantics |
||
51 | files on the filesystem. |
54 | are compiled conditionally (in case the "___AmigaOS__" macro is |
|
Line 52... | Line 55... | |||
52 | |
55 | defined at compile time). Otherwise, "Hunt" and "Gather" should run |
|
53 | The project adheres to the ANSI C standard and Amiga-centric semantics |
56 | under any platform that benefits from an ANSI C compiler. |
|
Line 54... | Line 57... | |||
54 | are compiled conditionally (in case the "___AmigaOS__" macro is |
57 | |
|
Line -... | Line 58... | |||
- | 58 | The project is developed from scratch on a real Amiga using StormC. |
||
55 | defined at compile time).Otherwise, Hunt & Gather should run under any |
59 | |
|
- | 60 | -=:[ Usage ]:=- |
||
- | 61 | |
||
56 | platform that benefits from an ANSI C compiler. |
62 | The "Gather" utility is used to index a path. The following command: |
|
57 | |
63 | |
|
58 | -=:[ Usage ]:=- |
64 | |
|
- | 65 | Gather -c RAM: |
||
- | 66 | |
||
- | 67 | |
||
- | 68 | will create a file in the S: directory named "gather.db". "Gather"is |
||
- | 69 | verbose by default and will show the user what the utility is doing |
||
- | 70 | but the behaviour can be changed with the "-q" (quiet) flag that will |
||
- | 71 | make "Gather" print only errors. |
||
- | 72 | |
||
- | 73 | In order to look for a file, the "Hunt" utility is then invoked with |
||
- | 74 | an AmigaOS search pattern, for instance, the pattern "#?test#?: |
||
- | 75 | |
||
- | 76 | |
||
- | 77 | Hunt #?test#? |
||
- | 78 | |
||
- | 79 | |
||
- | 80 | "Hunt" will then search the database previously generated by the |
||
- | 81 | "Gather" utility and will print out all the paths corresponding to the |
||
- | 82 | files matching the supplied pattern. |
||
- | 83 | |
||
- | 84 | In the previous example, in case any of the files previously indexed |
||
- | 85 | by "Gather" contain the term "test", then the "Hunt" utility will |
||
- | 86 | display the path to the file. |
||
- | 87 | |
||
- | 88 | At some point you might decide to add some other path to the search |
||
- | 89 | database as well. In that case, "Gather" would be invoked with the |
||
- | 90 | "-a" option instead of "-c" in order to add the files: |
||
- | 91 | |
||
- | 92 | Gather -a HDH0:Icons/ |
||
Line 59... | Line 93... | |||
59 | |
93 | |
|
Line -... | Line 94... | |||
- | 94 | "Gather" will then index the additional directory and add the new |
||
- | 95 | files to the database. Adding a path to the index database will |
||
- | 96 | require that "Gather" sorts the database again such that after adding |
||
- | 97 | the new files, "Gather" will proceed with sorting. |
||
- | 98 | |
||
60 | First the Gather utility is used to index a path: |
99 | Lastly, the "-r" parameter can be used with "Gather" to remove paths |
|
61 | |
100 | that have been previously indexed. Let's say that you have indexed the |
|
- | 101 | following paths with "Gather": |
||
Line 62... | Line 102... | |||
62 | |
102 | |
|
63 | Gather RAM: |
103 | RAM: |
|
Line 64... | Line 104... | |||
64 | |
104 | HDH0:Icons/ |
|
- | 105 | |
||
- | 106 | but now you would like to remove the "RAM:" path and all the files |
||
- | 107 | indexed below that path. In that case, you would issue a "Gather" |
||
- | 108 | command with the "-r" parameter: |
||
65 | |
109 | |
|
66 | which will create a file in the S: directory named "gahter.db". While |
110 | Gather -r RAM: |
|
- | 111 | |
||
- | 112 | and "Gather" will remove all files matching the "RAM:" path. Removing |
||
- | 113 | a path with the "-r" parameter does not take a long time compared to |
||
- | 114 | adding files to the database. |
||
- | 115 | |
||
67 | Gather is running, the utility will display the number of indexed |
116 | -=:[ Gather ]:=- |
|
- | 117 | |
||
- | 118 | "Gather" requires that one of the following parameters is specified: |
||
- | 119 | * -a (add files to an already existing database), |
||
- | 120 | * -r (remove files from an already existing database), |
||
- | 121 | * -c (delete the previous database file and create a new database). |
||
Line 68... | Line 122... | |||
68 | directories and files on the command line. |
122 | |
|
Line 69... | Line 123... | |||
69 | |
123 | The "Gather" utility takes several paths as parameters representing |
|
70 | In order to look for a file, the Hunt utility is invoked with an |
124 | the paths to be indexed; for example, all the following paths are |
|
71 | Amiga search pattern: |
125 | valid: |
|
Line 72... | Line 126... | |||
72 | |
126 | |
|
Line 73... | Line 127... | |||
73 | |
127 | |
|
Line 74... | Line 128... | |||
74 | Hunt #?test#? |
128 | RAM: |
|
75 | |
129 | DH0:System/ |
|
76 | |
130 | |
|
77 | in this case, "#?test#?", that will be compared to all the files |
- | ||
78 | indexed previously by Gather. In case any of the files previously |
- | ||
Line -... | Line 131... | |||
- | 131 | |
||
- | 132 | When the "Gather" utility runs, a database is created at "S:gather.db" |
||
- | 133 | containing all the found files. |
||
- | 134 | |
||
- | 135 | "Gather" is also happy to work with a different database file other |
||
- | 136 | than the default database at "S:gather.db" by passing the "-d" |
||
- | 137 | parameter when "Gather" is invoked. For instance, the following |
||
- | 138 | command invocation will create the database file at "T:gather.db" and |
||
- | 139 | index the paths "RAM:" and "HDH0:Icons": |
||
- | 140 | |
||
- | 141 | Gather -d T:gather.db RAM: HDH0:Icons |
||
- | 142 | |
||
- | 143 | Conversely, the "Hunt" utility can then be used to search specified |
||
- | 144 | database files: |
||
- | 145 | |
||
- | 146 | Hunt -d T:gather.db #?test?# |
||
- | 147 | |
||
- | 148 | The previous "Hunt" command will search a database file located at |
||
- | 149 | "T:gather.db" for all files matching the pattern "#?test?". |
||
- | 150 | |
||
- | 151 | -=:[ Hunt ]:=- |
||
- | 152 | |
||
- | 153 | "Hunt" is the counterpart to "Gather" and will search a given database |
||
- | 154 | generated by the "Gather" utility for files matching the terms passed |
||
- | 155 | to "Hunt" on the command line. |
||
- | 156 | |
||
- | 157 | For instance: |
||
- | 158 | |
||
- | 159 | Hunt #?test#? |
||
- | 160 | |
||
- | 161 | will search all files in the "Gather" database "S:gather.db" for the |
||
- | 162 | term "test". If any file within the database partially matches the |
||
- | 163 | term "test", then "Hunt" will display the path on the command line. |
||
- | 164 | |
||
79 | indexed by Gather contain the term "test", then the Hunt utility will |
165 | "Hunt" uses AmigaOS pattern for matching the file names on AmigaOS. |
|
Line 80... | Line 166... | |||
80 | display the path to the file. |
166 | |
|
81 | |
167 | -=:[ Notes ]:=- |
|
82 | -=:[ Gather ]:=- |
168 | |
|
Line 83... | Line 169... | |||
83 | |
169 | * The "Gather" utility will be slow and that is the intended |
|
Line -... | Line 170... | |||
- | 170 | behaviour: slow indexing with "Gather", fast searching with "Hunt". |
||
- | 171 | |
||
84 | The Gather utility takes one single parameter representing the path |
172 | * Temporary files might end up created in the same location where the |
|
Line 85... | Line 173... | |||
85 | to be indexed; for example, all the following paths are valid: |
173 | "Gather" utility is invoked. Traditionally the temporary directory |
|
86 | |
174 | on AmigaOS is mainted in RAM but "Gather" cannot use RAM since it |
|
87 | |
175 | intends to index very large hierarchies. Fortunately, "Gather"will |
|
88 | RAM: |
176 | delete the temporary files once "Gather" is done indexing. |
|
Line 89... | Line 177... | |||
89 | DH0:System/ |
177 | |
|
Line 90... | Line 178... | |||
90 | |
178 | Nevertheless, in case you intend to index a large filesystem |