Please note, this is a STATIC archive of website hashcat.net from October 2020, cach3.com does not collect or store any user information, there is no "phishing" involved.
hashcat Forum
Official Best64 Challenge Thread - Printable Version

+- hashcat Forum (https://hashcat.net/forum)
+-- Forum: Misc (https://hashcat.net/forum/forum-15.html)
+--- Forum: Organisation and Events (https://hashcat.net/forum/forum-24.html)
+--- Thread: Official Best64 Challenge Thread (/thread-1002.html)

Pages: 1 2 3 4 5

RE: Official Best64 Challenge Thread - Hash-IT - 03-29-2012

Congratulations arex1337, well done.

I enjoyed reading your write up and thank you for putting so much work into making hashcat's rules even better.

It was also nice to see that you not only shared your new rules but also shared the method and code you used to make your best 64. You are very generous and I hope you enjoyed doing it.

Thank you for your contribution and I hope this is only the start of your "ideal rule" search !! Big Grin


RE: Official Best64 Challenge Thread - forumhero - 03-29-2012

i enjoyed arex1337's write up as well. very nicely done and congrats

RE: Official Best64 Challenge Thread - M@LIK - 03-29-2012

yea me too arex1337.. me too : )
THANKS!!

RE: Official Best64 Challenge Thread - arex1337 - 03-29-2012

I updated my write-up with some post-challenge details, plus a chart that shows everybody's progression! https://thepasswordproject.com/2012-03-28_my_participation_in_the_best64_challenge

RE: Official Best64 Challenge Thread - Hash-IT - 03-29-2012

Good grief !!!

What are you arex1337 ?? A scientist or a Jedi ? Big Grin

Great stuff, even though I don't pretend for a minute to understand it all !

Thanks for your contribution.

RE: Official Best64 Challenge Thread - atom - 03-30-2012

This is the write-up how I rebuild the new best64.rule out of all the submissions but for real-life use.

This new best64.rule will go into all hashcat distributions.

I took arex best 80's results posted here: https://thepasswordproject.com/2012-03-28_my_participation_in_the_best64_challenge#post-competition

1. Check how useful it really is

To do this i made a MD5 hashlist of rockyou.txt which is the ultimate real-life password resource we have.

Code:
$ cat /root/dict/untouched/rockyou.txt | perl /root/dict2hash.pl > rockyou.md5

The resulting hashlist has 14 million hashes. This is to much.

Code:
$ wc -l rockyou.md5

Quote:14344391 rockyou.md5

I reduced them a bit by randomly fetching out 1 million hashes. This is the hashlist we will work with.

Code:
$ sort -R rockyou.md5 | head -1000000 > rockyou.md5.1000000
$ wc -l rockyou.md5.1000000

Quote:1000000 rockyou.md5.1000000

Now run the top 10k dictionary and the best 80 rules from the contest against it.

Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80.rule

Quote:...
Recovered.: 17097

Here is the original best64.rule. We will use it later for comparison with the final result:

Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r rules/best64.rule

Quote:...
Recovered.: 12345

Funny number.. Well, but thats it for real Smile

2. Clean up arex's rules

There is a lot of useless stuff inside. Will do:
  • Deobfuscate rules, stuff like ^7 { which is actually just $7
  • Remove dupes that come out after deobfuscation.
  • Remove as much as possible hashcat specific functions to make resulting hashlist jtr/pwp compatible.
  • Remove useless functions to increase performance while rules are applied at runtime in rule-engine: Making it faster!

Code:
$2$4]]                  :
$4]]]                   ] ]
T0                      T0
$1$2i42@2               $1
i42@2T0u                u
}}D1{                   ]
$si42@2                 $s
]]]d                    ] ] ] d
$3$2]]]]]               ] ] ]
krKz3x13                r
i42@2$2                 $2
}}}}'4                  } } } } '4
:so0                    so0
$0$1i42@2               $0 $1
}]]{                    } ] ] {
$1$2$3*22               $1 $2 $3
i42@2$9Z1               $9 $9
lx31                    D3
$b]$7                   $7
dx28Y4'4d               d x28 Y4 '4 d
}}}}}'5                 } } } } } '5
z3x13$3                 $3
^5{                     $5
$6 $9                   $6 $9
f*A5'8x14               f *A5 '8 x14
si1                     si1
z3x13$4                 $4
$0$0$7}[                $0 $0
]]$a                    ] ] $a
]]]$m$a$n               ] ] ] $m $a $n
+5]}}}}'4               +5 ] } } } } '4
*05x03d'3p1             *05 x03 d '3 p1
]]]$1$2$3               ] ] ] $1 $2 $3
i42@2$1$1               $1 $1
$9li42@2                $9
$1 $2                   $1 $2
]]]$y                   ] ] ] $y
^b^b^p^h^p]]]]          ^b ^b ^p ^h ^p ] ] ] ]
$2$2'5$1                '5 $1
Z4'8x42                 Z4 '8 x42
$0                      $0
$6                      $6
i42@2$2$3               $2 $3
$2 $2                   $2 $2
^&x03{{{{{{             x02 { { { { { {
$1]]]$e$r               ] ] $e $r
se3                     se3
] ] $i $e               ] ] $i $e
]]l]$o                  ] ] ] $o
$8z3x13                 $8
i01i42@2                ^1
}}}}}}Y4'4d             } } } } } } Y4 '4 d
$2 $1                   $2 $1
^t@tD2D2                D2 D2
{{{{{{{{x31'4           '5 D3
p2'7p1x58               p2 '7 p1 x58
]]D1]                   ] ] D1 ]
*44D4                   D4
+0+0+0+0+0+0+0+0        +0 +0 +0 +0 +0 +0 +0 +0
*04+0'4                 *04 +0 '4
$8$8                    $8 $8
x14dp2'6                x14 d p2 '6
]]]l$p$h$p$b$b          ] ] ] $p $h $p $b $b
Z5*75'5{x02             Z5 *75 '5 { x02
$7$7                    $7 $7
x21i42@2                D2
^e^h^ti42@2             ^e ^h ^t
] ] ] $d $o $g          ] ] ] $d $o $g
}}-0x12                 } } -0 x12
o0to0b                  o0b
l[^d^t@t                [ ^d
$e                      $e
l]$sl]$a                ] $a
$.]]l]$s                ] ] $s
+0+0+0x12               +0 +0 +0 x12
:}}}                    } } }
$1$3                    $1 $3
o1ao0ml                 ^m ^a
Z5'6x31]p1              Z5 '6 x31 ] p1
$0$2                    $0 $2

Some rules could not get deobfuscated/optimized.

Some of the changes might have an influence on the cracked results since I removed lots of stuff, too.

To make sure the changes are not so bad, here is a backup run to see its not so far away from the 17097 from the original set.

Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80-de.rule

Quote:...
Recovered.: 17050

OK, thats fine Smile

2. Remove the both rules that are only efficient cracking the phpbb hashlist

Code:
^b ^b ^p ^h ^p ] ] ] ]
] ] ] $p $h $p $b $b

Now we need to run again to see what we lost. In theory, should be zero:

Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80-de.rule

Quote:...
Recovered.: 17050

Zero! Toda!

3. Regroup them

By doing this I mean I am doing no changes at all to the rules itself. But you will see, it gives a great new view on everything.

Code:
## nothing, reverse, case... base stuff
:
r
u
T0

## simple number append
$0
$1
$2
$3
$4
$5
$6
$7
$8
$9

## special number append
$0 $0
$0 $1
$0 $2
$1 $1
$1 $2
$1 $3
$2 $1
$2 $2
$2 $3
$6 $9
$7 $7
$8 $8
$9 $9
$1 $2 $3

## high frequency append
$e
$s

## high frequency overwrite at end
] $a
] ] $s
] ] $a
] ] $e $r
] ] $i $e
] ] ] $o
] ] ] $y
] ] ] $1 $2 $3
] ] ] $m $a $n
] ] ] $d $o $g

## high frequency prepend
^1
^e ^h ^t

## high frequency overwrite at start
o0b
o0d
o0m o1a

## leetify
so0
si1
se3

## simple extracts
D2
D2 D2
D3
D4

## undouble word
'5 D3
'5 $1

## removes suffixes from 'strongified' passwords in dict
]
] ]
] ] ]
] ] ] d
] ] D1 ]

## rotates
+5 ] } } } } '4
x02 { { { { { {
} ] ] {
} } -0 x12
} } }
} } } } '4
} } } } } '5
} } } } } } Y4 '4 d

## unknown
*04 +0 '4
*05 x03 d '3 p1
+0 +0 +0 +0 +0 +0 +0 +0
+0 +0 +0 x12
Z4 '8 x42
Z5 '6 x31 ] p1
Z5 *75 '5 { x02
d x28 Y4 '4 d
f *A5 '8 x14
p2 '7 p1 x58
x14 d p2 '6

Verify results:

Code:
$ ./hashcat-cli64.bin rockyou.md5.1000000 passwords_top10k.dict.txt -o y -r best80-de.rule

Quote:...
Recovered.: 17050

Perfect!

4. Reduce the 78 rules down to 64 by removing the rules that are less efficient

It is possible that two or more different rules can crack the same hash just because of the many words in our dictionary.
So what we focus in are all these hashes that have been cracked by only one single rule.
This will happend more or less often for each rule. We add a counter for each rule and with each hit we increase it.
When we are finished we simply sort out these rules that have the lowest number in their counter.

To do this, you can read arex way of doing it, see his writeup: https://thepasswordproject.com/2012-03-28_my_participation_in_the_best64_challenge#pick_64_rules_that_combined_will_match_the_largest_possible_number_of_hashes

5. Result

Here is our final new best64.rule:

Code:
## nothing, reverse, case... base stuff
:
r
u
T0

## simple number append
$0
$1
$2
$3
$4
$5
$6
$7
$8
$9

## special number append
$0 $0
$0 $1
$1 $1
$1 $2
$2 $1
$2 $2
$2 $3
$6 $9
$7 $7
$8 $8
$9 $9
$1 $2 $3

## high frequency append
$s

## high frequency overwrite at end
] ] $a
] ] $e $r
] ] $i $e
] ] ] $o
] ] ] $y
] ] ] $1 $2 $3
] ] ] $m $a $n

## high frequency prepend
^1
^e ^h ^t

## leetify
so0
si1
se3

## simple extracts
D2 D2
D3
D4

## undouble word
'5 $1

## removes suffixes from 'strongified' passwords in dict
]
] ]
] ] ]
] ] ] d
] ] D1 ]

## rotates
+5 ] } } } } '4
x02 { { { { { {
} ] ] {
} } -0 x12
} } } } '4
} } } } } '5
} } } } } } Y4 '4 d

## unknown
*04 +0 '4
*05 x03 d '3 p1
+0 +0 +0 +0 +0 +0 +0 +0
Z4 '8 x42
Z5 *75 '5 { x02
d x28 Y4 '4 d
f *A5 '8 x14
p2 '7 p1 x58
x14 d p2 '6

And our final value:

Quote:...
Recovered.: 15357

The original best64.rule was:

Quote:...
Recovered.: 12345

The original best64.rule was already very efficient on the random hashlist.

The new one is 24.39% more efficient.

Again, thanks to all participating in the contest.

RE: Official Best64 Challenge Thread - Hash-IT - 03-30-2012

Fantastic work there atom. you have clearly put a lot of effort into that, it is much appreciated.

I have just deleted my humble attempt to do this as it was junk in comparison.

Thank you.


RE: Official Best64 Challenge Thread - frekvent - 03-31-2012

I have experienced the same issue as described phish (but the original post seems to be deleted). I tried to simplify rules by doing some simple substitutions:
  • sXY@Y -> @X@Y
  • iNX@X -> @X
  • ^X@X -> @X
  • $X@X -> @X
But somehow this makes the rules less effective. For example, take the rules i42@2 and @2. They should be identical but they're not. When I test them separately against the phpbb hashes, i42@2 gives me 7520 hits but @2 gives me only 264.

Oh and thanks to atom for compiling the new best64.rule! I have tested it against some random hashes with some random wordlists and got some very good results! I did spot a minor simplification you could do. It turns out that "}}D1{" can be simplified to a single "]".

Anyway thanks a lot! It is truly amazing how much the original rules could be improved upon!

RE: Official Best64 Challenge Thread - atom - 03-31-2012

the post was not deleted, just splitted into a seperate thread, since this thread is about the challange, not about strange rule engine behaiviors. please read the https://hashcat.net/forum/thread-1027.html then you hopefully understand why it works how it works...

RE: Official Best64 Challenge Thread - atom - 03-31-2012

(03-31-2012, 12:10 AM)frekvent Wrote: It turns out that "}}D1{" can be simplified to a single "]".

This is cool. I will update the post and the best64.rule. Thanks!