hashcat Forum

As a continuation of this thread, I've got updates...

I'm now running:

Broker- 1U server, 12GB RAM, Mellanox ConnectX-3 QDR Infiniband HCA connected directly to other HCA (running opensm), oclhashcat 1.0, VCL1.22

Node- 7990, 6970, Mellanox ConnectX-3 QDR Infiniband HCA

I've updated both HCAs to the latest firmware; all modules are loaded, ping works, oclhashcat detects the 3 GPUs and works, but...

SLOW!!! I'm still only getting ~50% of what I get with the machine running standalone.

The HCAs are running in connected mode, and I've tried MTUs of 2044 and 65520.

Ideas? I'm very frustrated.

Thanks!

EDIT- forgot to mention that eth0 is on 192.168.10.x and ib0 is on 10.10.10.x...

VCL is broken and is no longer supported. It is officially considered a dead technology. Sorry.
The new version of disthc should be coming out soon. It will have limited features at first (most to come in subsequent updates) but that might be a better option for you.

What kind of performance could I expect with disthc running over Infiniband compared to standalone? Is it designed to work over a high speed/low latency link like VCL is, or is it much different?

Thanks!

Just for the record- I've managed to get ~20 000K/sec over gigabit using VCL, which is the speed I was expecting all along. The trick was ramping up the -n value way up- anything under 300 or 400 was way slow, 600 seems to be my sweet spot, 800 just hangs after enumerating the devices.

We'll see if I run out of RAM, now!

12GB lasted me ~60 hours, before running out, adding more soon.

(12-23-2013, 04:32 AM)davejcb Wrote: [ -> ]12GB lasted me ~60 hours, before running out, adding more soon.

Are you seeing memory leaks? I see them too but epixoip told my very clearly that if I see anything that resembles a memory leak I am dreaming so I havent reported them since. I'm still seeing them. It might well be 100% the fault of VCL but I sensed there was no room for discussion here.

If I watch top, the oclhashcat process just consumes more and more memory, starting at ~15%, and steadily consumes until all is exhausted. The rate is about 2%/hr, if I remember correctly.

Not sure if it's VCL related or oclhashcat related- either way, I'm throwing in more RAM and restarting every so often as a cure for now.

That is exactly what I am seeing. Unless people here get upset again I am willing to discuss it.