For this challenge, they gave us an address to netcat to and an executable. We viewed the executable in IDA and noticed that it requests for a registration code, runs the given string through a function, and then compares the returned number to 0xEF2E3558. If they are equal, it reads in the key from a file and sends it to the user. So we figured that we might need to reverse that function to create input that it would accept.
IDA has kindly reconstructed the function in C for us:
From this we could see that the function takes in a string. It takes the returned variable (starting at 1337) and multiplies it by 32 and adds the value of the current character in the string. It then adds this value to the return variable, and repeats the process until the next character is NULL.
One major problem to this was that v4 is a signed integer and because of that it would reset into negative values if it would increase pass the maximum signed integer value. We also realized that the length of the input influenced the output of the function much more than the value of characters in the string. So we wrote a C program that would run this function and return the value.
We then figured that would attempt to find a possible solution by finding the length of the string by inputting a string of the same character. Then we would modify these values from right to left, so that the returned hex value would begin to form 0xEF2E3558.
We began by feeding the program 'A', then 'AA', then 'AAA', and so on until we reached 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAA'. At this point the returned hex value was fairly close the needed hex value, that is 0xa9ef0df6 since a distance of 0x50000000 would be possible to obtain by modifying the other values in the string to obtain the needed value.
We then began to go through each character in the string and increment it by one until we found a string that would only increment the most significant byte of the hex by only 1 or maybe 2. Once we found that, we modified it to give us something like 0xEXXXXXXX.
The final inputed string that gave us 0xEF2E3558 was: 'AAAAAAAAAAA$AAAAAAAAAAA[PV=Qv'.
The key is: day 145: they still do not realize this software sucks