DEBUGGING RULES
FOR EFFICIENCY

Once sorting is complete you will be dropped into a submenu where you can find the Pareto curve analysis along with milestone breakpoints showing exactly how many rules are needed to cover a given percentage of the entire initial dataset.

Pareto milestones from concentrator — rules needed to reach each coverage percentage of the full 887K-rule corpus (21.4M total occurrences).

This makes it straightforward to cut the ruleset down to a practical size — for most cracking jobs the top 1–5% of rules by frequency covers the vast majority of real-world hits.

Using the Pareto milestones as cut points, four rule files were produced — one for each practical use case, from a large comprehensive set down to a tiny high-precision set:

$ wc -l *.rule
 500001 rulest_large.rule
 150001 rulest_medium.rule
  25001 rulest_small.rule
    251 rulest_tiny.rule
 675254 total

Results from the rules comparison spreadsheet. Each of the four output files is compared against the best-in-class rule set in its weight category, and against other existing rulest_* variants.

The existing rulest_* family in the benchmark includes rulest_rules_strip_1p7m (44.65%, 1.7M rules), rulest_rules_strip_1p1m (43.98%, 1.1M rules), rulest_rules_strip (38.10%, 332k rules), rulest_rules_ga (31.76%, 643k rules), and rulest_rules_no_ga (31.35%, 329k rules). The new rulest_large at 500k sits right in the middle of that family — better recovery than the strip and GA variants while being a fraction of the size of the 1M+ strip files. A good trade-off for most use cases.

The benchmark leaderboard has expanded significantly. New top entries include Fordyv4a (56.94%, 4M rules), Fordyv4b (56.28%, 6.9M rules), A11313M (53.97%, 3.2M rules), sapphire_v1 (52.06%, 1.27M rules), and CakeV1 (52.04%, 1M rules). These large rule sets push the ceiling considerably above the previous 48–49% range. The rulest_* output files remain competitive within their respective size categories, but the top of the overall leaderboard is now materially higher.

One thing worth highlighting: rulest runs comfortably on cards with less than 4 GB of VRAM. The entire ruleflow — rule generation, debug pass, concentrator analysis — can be run on modest hardware most people already own. You don't need a flagship GPU to produce a competitive rule set. The debug pass itself is the most time-consuming part, but even that is a one-time cost. The resulting files can then be reused across any number of jobs at no additional overhead.

Combined with the Pareto insight that a few thousand high-frequency rules cover a disproportionate share of real-world hashes, this means the barrier to having solid, personalized rule sets is much lower than it appears.

After a short break — and fixing a few bugs in rulest related to generation and validation — I decided to run a quick second round (tagged r1) using the rules already acquired, this time debugging against the hashmob.tiny.found dictionary (~2 MB) rather than a full-size corpus. The rule pool itself was also refined and capped at depth 1–8 instead of the previous 1–10.

Trimming the maximum chain length is a deliberate compromise, not a step back. Longer rules are disproportionately expensive to debug and score worse against a large hash database once the operator count climbs, so they yield fewer matches per unit of debug time. Capping at 8 keeps the rule space dense where it's actually worthwhile — this was somewhat confirmed by getting 6.75M rules out of seven rulest runs using comparatively small input files. That first approach was roughly hashmob.mini vs. rockyou in scale. For comparison, rulest on hashmob.micro vs. hashmob.small alone extracts 3.2M+ deduplicated rules in balanced RCR mode at the default depth of 6 — plenty to justify a debug pass as a first filtering step.

$ wc -l rulest2debug_r1.rule
6748072 rulest2debug_r1.rule

$ ls -lh hashmob.tiny.found
-rw-r--r-- 1 user user 2.0M hashmob.tiny.found

Output of the concentrator Pareto pass on the r1 debug run — 1,256,836 unique rules with 35,020,748 total occurrences. Unsurprisingly, the top ten are all classic insecure number-append patterns.

Same shape as before, just a bigger pool underneath: 579 rules already reach 10% of total occurrences, and half the value sits in under 46k rules out of 1.26M total — even denser at the head than the previous r0 pass.

The four r1 rule files are cut straight off the milestone breakpoints above — large ≈ 95%, medium ≈ 80%, small ≈ 40%, tiny ≈ 10% of cumulative coverage.

$ wc -l *_r1.rule
 513641 rulest_large_r1.rule
 179166 rulest_medium_r1.rule
  25730 rulest_small_r1.rule
    585 rulest_tiny_r1.rule
 719122 total

Real numbers from the rules comparison spreadsheet, run against all four r1 rule files.

For the second respin (tagged r2) I pushed the rule depth even lower, capping at depth 1–6 instead of the previous 1–8. The goal was to test whether a tighter, denser rule space — extracted and debugged against a substantially larger corpus — could match or exceed the r1 results while keeping the rule pool smaller and more focused.

The extraction phase used hashmob.mini.found as the base wordlist, extracted against hashmob.medium.found, SkullSecurityComp, and a small supplementary dictionary as targets. The resulting rule pool was then debugged against hashmob.small.found (2,341,652 words) run over 160,268,160 MD5 hashes. Corpus extraction took roughly 70 minutes, and the full debug pass across 2,103,114 rules completed in 4 hours and 40 minutes on the same RTX 3060 Ti.

Dropping from depth 8 to depth 6 is a significant constraint — it eliminates a large swath of multi-operator chains that rarely hit in practice but consume disproportionate debug time. If the coverage curve stays steep, it confirms that the highest-value rules are concentrated in short, predictable transforms rather than exotic long chains.

This respin also carries an update to rulest's Stage 3 (GA) — the genetic algorithm's fitness function was changed from a raw hit-count to a marginal coverage gain (submodular greedy) model. Individuals in each generation are now sorted by raw hits and each is credited only for the base words it covers that no higher-priority individual in that generation already covered — a chain that just duplicates coverage another individual already provides scores ~0 fitness regardless of its own hit count. This should push the GA toward genuinely complementary rule chains instead of rewarding near-duplicates of already-strong rules.

$ wc -l rulest2debug_r2.rule
2103114 rulest2debug_r2.rule

$ wc -l hashmob.small.found
2341652 hashmob.small.found

$ wc -l MD5_debug_hashes_r2.txt
160268160 MD5_debug_hashes_r2.txt

Output of the concentrator Pareto pass on the r2 debug run, run after functional minimization — 809,828 unique rules with 38,091,962 total occurrences. The top rule alone (zZx38) already accounts for 0.5% of all occurrences on its own, and the top 10 rules together cover 2.4% — a much flatter drop-off than r1's top 10, dominated here by o0-append variants and r^N^7^4^1-style reverse/truncate chains rather than single-operator rules.

The r2 curve is noticeably flatter than r1's — hitting 90% of cumulative value now takes 180,807 rules (22.3% of the pool) versus a much smaller slice in r1, and 50% coverage alone already needs 15,978 rules. With depth capped at 6, value is spread across far more individually-weaker chains instead of concentrating in a small set of dominant short rules, which is the opposite of what the tighter depth cap was expected to produce.

The four r2 rule files are cut straight off the milestone breakpoints above — but this round the actual cut points landed at large ≈ 99%, medium ≈ 90%, small ≈ 60%, tiny ≈ 10% of cumulative coverage (versus large ≈ 95%/medium ≈ 80%/small ≈ 40%/tiny ≈ 10% in r1), a direct consequence of the flatter r2 coverage curve above — hitting the same relative tier required reaching further out into the pool. Each file carries 5 lines of header comments.

$ wc -l *_r2.rule
 531174 rulest_large_r2.rule
 180813 rulest_medium_r2.rule
  29500 rulest_small_r2.rule
    101 rulest_tiny_r2.rule
 741588 total

Real numbers from the rules comparison spreadsheet, run against all four r2 rule files, benchmarked as usual against a hashlist of 111,449 MD5 hashes using the hashmob.medium.found dictionary.

A short clip from the debug session, left here as a curiosity — just a few seconds of the hashcat debug pass itself.