Ok, for those who've been following the thread, here's an update from my side.
NOTE: Throughout this explanation, ctx, ctx1, ctx2, ctx3 ... are md5 contexts, final and final2 are md5 data arrays (unsigned char[16]) the same variable name refers to the same variable always ie, scope of variables is this mail.
1. Yahoo uses the exact same algorithm as pam does to generate MD5 passwords.
You can get the source for this from the pam package. Look in modules/pam_unix/md5_crypt.c:MD5Name(crypt_md5)()
There is only one minor change that does not affect the outcome of the code, and that is the calls to MD5_Update with ctx1 happen before the calls to MD5_Update with ctx (if you see the code, it'll make more sense, or I can explain then).
2. There are four more calls to the MD5 library. The first does this:
MD5_Init(&ctx2); MD5_Update(&ctx2, passwd, strlen(passwd)); MD5_Final(final2, &ctx2);
MD5_Init(&ctx3); MD5_Update(&ctx3, str, strlen(str)); MD5_Final(final2, &ctx3);
str is obtained by appending/prepending something to the username. That something is 49 bytes in length.
The last two calls are:
MD5_Init(&ctx4); MD5_Update(&ctx4, str2, strlen(str2)); MD5_Final(final2, &ctx4);
MD5_Init(&ctx5); MD5_Update(&ctx5, str3, strlen(str3)); MD5_Final(final2, &ctx5);
str3 is also derived from the username, again with a 49 byte string appended/prepended to it.
str2 seems to be 34 characters in length always.
This data was obtained through statistical analysis of function calls. We still haven't determined what the actual contents of str, str2, str3 or the salt are. Once we have this, I believe we will have cracked it completely.
If anyone wants to figure it out, you have to somehow get the data that's being passed to MD5_Update.
The first call is the password, the second call is with str. You need to figure out how str is derived.
The next three calls are password, magic ($1$ I think), salt (8 chars). The next three are password, salt, password.
Try and figure out how salt is derived, and if magic is different from $1$, what is it? Is it constant across calls?
Then, look at the last two calls to MD5_Update - calls number 3536 and 3537. No 3536 seems to be constant, but someone will have to confirm this. No 3537 is derived from the username. Figure out how.
Philip