Looks like Zob tried to make his own encryption algorithm again; here's the program. Here, here, and here are two messages encrypted with the same key and the plaintext of one of the messages. Convince Zob to stop trying to make his own ciphers!
Technically, all the math you need for this problem you learned in Algebra 2.