>>
>>I'm using md5 to calculate a hash of a password.
>>
>>I start off with the hash of the password. Then, I do a series of hashes where each new hash is done with the previous hash + a substring of the password
>>
>>The number of additional hashes depends on (1) the password length and (2) the byte value of each of the bytes modulo a number
>>
>>If you don't know how I hash, how can you crack it - I wonder
>
>One thought - if you create an account (or accounts) on a system you will be stealing the password file from, you will already know the password. So you'll be able to attempt all of the basic hashing schemes before realizing something else is happening. You could then create more accounts with various "basic" passwords to attempt to analyze the types of transformations that are occurring.
>
>Honestly, even just doing something slightly differently would probably weed out 99% of the hackers, leaving only people who are really interested in cryptology (or who like a challenge) to continue the attempt.
>
>If it were me, I'd probably decide that a direct approach isn't going to work. If I was able to compromise their password file/table, can I get a hold of the files the site actually runs on (ex. DLL's, assemblies, etc.) and then attempt to decompile or reverse-engineer it to determine what types of transforms are occurring? Seems like that would be much simpler.

I've only told part of the story. The password will be converted to a byte array. Each hash is a byte array

On the password byte array and on each digest, there may be an optional byte swapping, bit rotation (to the left or to the right) depending of the actual or previous values of the byte array. The whole process: number of bits rotated rotation/ the bytes to swap/ number of hashes depend on the password that has been entered

To find the algorithm without the code - or decompilation, makes it virtually impossible

The question then becomes - how to hide the assemblies