Encryption and shared secret

[admwx7]

Need to come up and implement a form of encryption for the client-server communication, right now RSA is an option on the table as well as a keyfob (authenticator) setup where the ID is all we would know about any keyfob and what the current secret is, then we can do a shared secret exchange encryption based on the keyfob and use the shared secret for a session. Any other ideas are welcome, just need some sort of standard we can use. We can key off of API version or something along those lines to change our client-server encryption in the future if needed.

[schumannd]

We can use RSA with an 2048 bit key for the key exchange and then 256-AES. I have found an implementation of aes online we can use.
http://anh.cs.luc.edu/331/code/aes.py

[nb333]

@schumannd Sounds good to me. :-)

[boxtown]

Python has an RSA library just called rsa. Also, I think pyCrypto is the library Python recommends for AES

[admwx7]

Implementation specifics don't matter too much to me, just need to be agreed upon beforehand. I still like the idea of some auth standard (RSA) to negotiate a one time use key to do the rest of the session traffic in, seems like it's the most secure option.

[boxtown]

@admwx7 Right that's exactly what @schumannd is suggesting. We create a 1 time key to mask session traffic (256-AES symmetric key). We then have to share this key and to do so we mask it using RSA with a 2048 bit key.

[Sp3ctr3]

I have a sample implementation here: https://github.com/Sp3ctr3/CryptoEnc . We need to adapt it to HTTP.

[schumannd]

I ran and played around with the scripts a bit.
Now I ran into two difficulties:

I can modify the headers on server side easily. But as they are parsed and interpreted automatically by the urllib2 on client side we would need to hack that library to implement decryption there.

I am running into problems of modifying the content buffer on the server side. It might be possible to write an encryption module that gets a buffer and has the encrypted buffer as an output. But I have not played around with that type of buffer yet.

What are your suggestions guys?

Also I have a RSA implementation written from scratch which we could modify. (github.com/schumannd/manualRSA) But we can use the existing module as well. For AES We could use my source or work with the existing module.

If we get the above 2 problems solved it should be a matter of minutes to get AES working. Implementing the RSA key exchange I have run into synchronization issues in the past. But that should be possible to work out.

[schumannd]

Ok, got the buffer problem on server side solved. Was not a problem at all, buffers behave just like strings. On the client side I can also access the content in a variable, so decryption shouldn't be a problem either. Except if you want to use some library functionalities I am not aware of. For now I am only working with command line output.

Next tasks:

1. implement key exchange
2. Have a data structure that keeps track of connections and their respective keys. For now I haven't checked how the twisted library does this.

[schumannd]

Last update for the day:
I got AES working, kinda. For some reason the client only receives a part of the encrypted message. He decrypts successfully but then stops. The solution right now is a little bit hacky, thats why I don't want to push it. I'll make a branch. if you want to see how it works just check it out

[Sp3ctr3]

I'm checking the code now but the the client only receiving the part maybe because of the use of HandleResponsePart function? Try overloading the handleResponse and see if you get the same error. I'm working on a similar implementation. Will post soon on how it's going.

[boxtown]

You might want to try to change the implementation a little bit. It looks like your encrypting first with AES, prepending the key, and then encrypting with RSA. The problem is, RSA is not very good at encrypting large bodies of data. This might be why the client only receives a part of the encrypted message. Try encrypting with AES, and then just encrypting the AES key with RSA and prepending the encrypted key to the AES encrypted body.

I haven't been able to do much actual coding since I'm wrapping up Finals week as we speak. Let me know if this helps!

[schumannd]

@boxtown : There is no RSA encryption going on. I only used AES encryption for now, the shared secret being hardcoded on both sides. This was just a proof of concept. For the RSA key exchange I would have to implement a handshake first. But as I am running into transmission problems already I will postpone that.

@Sp3ctr3 the HandleResponsePart function in the twistedMatrix doc is described as "undocumented". Time to sift through source code...

[schumannd]

I have made some more interesting / annoying discoveries today:

1. the AES implementation I use is too slow. It takes several seconds. We will need to try the existing library and hope that it is faster
2. For some sites the system transimtted exactly half of my message. For some other sites it transmitted everything, but decryption broke on the way.

Maybe I'll have time to work on it this weekend

[boxtown]

@schumannd I just took a look at your AES implementation in server.py and testclient.py. It seems you are using pyCrypto but it also looks like what you're doing might be a little more complex than necessary. Take a look at the example in https://www.dlitz.net/software/pycrypto/api/current/, specifically in module Crypto.Cipher.AES. I don't know if that'll help but it's just a suggestion

[schumannd]

Sorry, that info was out of date. AES encryption works now with sufficient speed using PyCrypto. The Problem now is, that it still runs into errors and only transmits part of the requests. What gets transmitted is decrypted successfully though.

Before I continue I would like to discuss with you guys what exactly our interface is supposed to look like. What libraries do we want to use to transmit data (twisted?) and to display it (probably the browser itself?) Maybe start writing the browser plugin and testing with that? Twisted is giving me a lot of trouble as I have to continuously try to circumvent its features.

Your thoughts?

[Sp3ctr3]

Our client side will be a browser plugin being developed at openfaux/openfaux-client.

That is how the current plan is. If you check out the code I posted, it does the encryption part for now. The trick will be converting that to HTTP. And yes we'll have to overload a lot of functions in twisted.