• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Jeanne Boyarsky
  • Ron McLeod
Sheriffs:
  • Paul Clapham
  • Liutauras Vilda
  • Devaka Cooray
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

1st attempt: 'ENCRYPT then SERIALIZE'

 
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi, I am new to both, so there are probably many misunderstandings.
I have installed the Unlimited Strength files from Oracle, and also the BouncyCastle files (bcprov-ext-jdk15on-150.jar), and amended java security.
I have proven both are correctly in place:



Goal:
I have a simple class holding 3 integers and two enums that is serialization enabled including the enums.
I have a LinkedHashMap containing a large number of these that I wish to AES256 encrypt using SealedObject, then serialize.
I need to retrieve a single class object quickly and decrypt it.

Questions:
What is the sequence of events? is this correct:
1) add all the items unecrypted to the map
2) create the cipher
3) put the map into the SealedObject using the cipher
4) serialise the Map.

or do I do all of the above in a loop to each class object, then add the serialised objects to the map?
Can a LinkedHashMap hold two million items?
AES128 or 256: I cannot see a method to set the type to 256.

eg none of these objects in this code example, seem to have a property to set 128 or 256, do I have to use password & salt methods instead?
Another article suggested you create a key using SHA-256. But where do you then use the 256 bit key, as I cannot find a property to set?


I have spent many hours on this today and have more questions than answers!
many thanks
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
A List can definitely hold 2m objects, it's more of a question whether the 2m objects fit into memory. But 3 ints and 2 enums sounds fine.

AES strength: Call KeyGenerator.init(256) before generating the key. The resulting key should be 32 bytes large (8 * 32 = 256).
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the reply Ulf, I'll have another go in the morning.

Kind regards, Nigel
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
re: key.init(256);

Hi Ulf,

for key i have created an instance of SecretKey based on an example I found.
For SecretKey the init method is undefined (line 33). What class should I have used, can you point me to a better example?



many thanks,
 
Ulf Dittmer
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That method is in KeyGenerator. You'll need to break line 31 into two so that you have a reference to the KeyGenerator object.
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Ulf,

My key id was an instance of SecretKey, when I changed this to be an instance of KeyGenerator then I was able to split line 31 without errors.
However the cipher lines 15 & 16, no longer accept the key, it expects Key and not KeyGenerator.



not sure of the relationship from KeyGenerator back to Key to initialize the Cipher objects?

thanks again for your help
 
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The Javadoc for KeyGenerator is your friend ! It is the generator that has the init() method and not the key.
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Richard, key is an instance of KeyGenerator already, I've renamed it to be keyGen to make that clearer.



the SecretKey line is new, and this is now accepted in the cipher initialisation lines (13 & 14).
It does looks more promising, not run yet.

thanks!

 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


If you are going to comment this line, and I don't think it needs it, then please make sure the comment is correct! It returns a SecretKey and not a String!
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
will do; a learning curve at the moment!
thanks for your help
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
am I correct in saying the SecretKey is a system generated password in effect, so it must be stored for use in the decipher object at a later date?



thank you
 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Nigel Shrin wrote:
am I correct in saying the SecretKey is a system generated password in effect, so it must be stored for use in the decipher object at a later date?
thank you



In essence yes but passwords are normally strings but a SecretKey is most definitely not a string. You now have to find some way of securely shipping the key to the recipient of the encrypted data!
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
thanks Richard, much appreciated!
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
NPE retrieving SecretKey from sealed object

Hi,

I am having difficult tracing this error, the exact error is:



line 45 is a test call to the routine below from main():
public static void main(String[] args) {



Method:



I am pretty sure the error is in lines 3 or 9. The reason I created an object in line 3 was to have a return variable.
The getSkey() call on line 3, is a setter, to return the static class member, and this call works in itself.

Many thanks
 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
There is just so much about those code fragments that I don't understand.

1) I don't understand why "The getSkey() call on line 3, is a setter," . Surely it is a getter .
2) I don't understand why in line 3 of getStoreKey() you need to generate a key.
3) I don't understand "skUnsealed.equals((SecretKey)sealed2.getObject(dciTemp)); " . Since 'skUnsealed' is generated by a keyGen.generateKey() it probably returns a new random key so the chances of this new key being equal to any other key is just about zero.
4) I thought you were trying to place a collection in the sealed object and not a key.

From my point of view the main problem is that there is not enough code to allow me to see what is happening and the bit you have posted does not seem to make sense. What you need to post is an SSCCE (follow the link) that illustrates the problem.
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Richard

1) I don't understand why "The getSkey() call on line 3, is a setter," . Surely it is a getter .
2) I don't understand why in line 3 of getStoreKey() you need to generate a key.
3) I don't understand "skUnsealed.equals((SecretKey)sealed2.getObject(dciTemp)); " . Since 'skUnsealed' is generated by a keyGen.generateKey() it probably returns a new random key so the chances of this new key being equal to any other key is just about zero.
4) I thought you were trying to place a collection in the sealed object and not a key.

point 1 - my typo, it is a getter, apologies
point 2 - I didn't want another secret key, just a way to return the result (extracted key), looking at it again, perhaps I could have done this on line 9, at the end of the try block:

point 3 - explained by answer to point 2
point 4 - this is a test to return a saved SecretKey that is in a SealedObject and serialized so it can be reused, when I have this working, so I am working towards the collection.

I could work on a SSCCE if the above is still not clear enough.

thanks again for your help,
 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Nigel Shrin wrote:
I could work on a SSCCE if the above is still not clear enough.



Most definitely not clear enough for me. Since you are currently just playing with things I would have thought that at the moment your whole code would be an SSCCE.
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Richard,
I'll do that, it'll be tomorrow now, I've been playing with this most of today,
Kind regards,
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I have attempted to implement KeyStore but currently get an uninitialised keystore error.
The map of objects to encrypt then serialise is created, but no attempt has been made to do this yet.
The error has occurred from trying to create a KeyStore then retrieve the SecretKey for it.

Full error from line 86:



Full code:



object in map:



thank you!
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I just noticed that the alias was different on line 120, so I made it the same, but this made no difference to the error message:

line 120 (was: "privateKeyAlias") now reads:


thank you!
 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Your code is not an SSCE since it is incomplete (I can't see EnumInData2Store ). The code seams very very complicated and, since class KeyStore is very easy to use without needing either KeyStore.SecretKeyEntry or class KeyStore.ProtectionParameter , it is not obvious to me what you are doing using those internal classes.

On a general note, you seem to be trying to use the 'big bang' approach to and you should separate the concerns by keeping the map and the encryption separate. You can create an SSCCE without needing the map at all by encrypting any serializable object (a String being a good starting point).

I can see one potential problem, it may be another one of your bad comments, but

does not create an empty KeyStore. It tries to read the content of the file as a KeyStore. You create an empty KeyStore by initialising it using null as the stream and a char array as the password. You should check the Javadoc.




 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Richard,

here's the enum:



In the previous to last example I had problems trying to store the key, so I read some more and found keystore.
The current example is to a large extent based on the code from the javadoc, I use Eclipse and that forced me to put so many catch clauses in.
http://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html#getInstance(java.lang.String)

I'll try and modify it tonight to just use a string to encrypt.

Can you see where the exception is coming from?

Thanks again for your time,

 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Nigel Shrin wrote:
Can you see where the exception is coming from?



You have not initialised a KeyStore! The stack trace tells you exactly where the exception is coming from and therefore which KeyStore. A comment in your code indicates you know where the exception is coming from!

On a more general note you are going to run into a much bigger problem than finding where the exception is coming from. RSA is only really any good for encrypting small amounts of data since the basic algorithm limits one to a block of less than or equal to the modulus length. When one takes into account the use of PKCS1 padding (one is using that by default) one actually has 11 bytes less than that. There are ways to get round this by breaking the cleartext into blocks and encrypting each individually but then one has to make sure one knows exactly how long each block of ciphertext is (they are not guaranteed to be all the same). Some JCE implementations do this behind the scenes but last time I checked the Sun/Oracle JCE implementation did not. Also, compared to the speed of most block ciphers such as AES, RSA is very slow.

The normal approach to using RSA with large cleartext is to use a hybrid approach. In this one actually encrypts the data using a block cipher such as AES with a random ephemeral key and only encrypts the ephemeral key using RSA. Assuming AES is used for the block cipher, in essence the steps are -

1) Generate a random AES block cipher key.
2) RSA encrypt the random key.
3) Write the encrypted key to the ciphertext channel. This normally means first writing a representation of how long the encrypted random key is and then writing the bytes of encrypted random key. DataOutputStream makes this very easy.
4) Encrypt the cleartext using the random key with AES and write the associated ciphertext to the channel. CipherOutputStream is good for this.

To decrypt one first reads the RSA encrypted AES key and recovers the key. One then decrypts the AES encrypted ciphertext to generate the original cleartext.

Section 13.6 of "Practical Cryptography" by Ferguson and Schneier expands on this approach to improve it's security.
 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Having just spent some time looking in detail at your latest code I note that you generate a SecretKey and put it in a KeyStore as a SecretKey but then you try to extract it as a PrivateKey ! You won't like me saying this but you should sit back and decide how to break the code into more manageable and testable components/modules and then build and test them as individual components/modules. The most obvious component is a class to handle your RSA key(s) ; this can be written and tested without any reference to the rest of your code.

Now you could continue as you are trying to create this monolith but it will get increasingly difficult to build, test and debug and most people here will not be inclined to trawl though the mass of code trying to find errors that would be obvious to all, including you, if the program was modularised.
 
Nigel Shrin
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Richard,
I don't mind in the slightest, and I am very grateful for the time you have taken to explain some principles of encryption to me. I read your comments yesterday evening and have ordered your recommended book. I think Java is a great language, and would very much like to improve the quality of my coding.

Although I knew the line of the error, I didn't know how to fix it! I am not familiar with many of the terms & concepts at this point; PKCS1, modulus length, padding etc. I thought I was only using AES256 and not RSA as well. I did O-level maths, but did not study maths further. So please forgive my ignorance. My next attempt may not be until next weekend - the book should have arrived by then.

Thank you for taking the time & effort to further my understanding.
 
Richard Tookey
Bartender
Posts: 1166
17
Netbeans IDE Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Nigel Shrin wrote:I read your comments yesterday evening and have ordered your recommended book.


I did not mean you to go out an buy the book! The authors of the book are generally recognised as experts in modern cryptography and the book is a very good book (sits right in front of me on my bookshelf) and well worth buying if you intend to continue working with cryptography. I was just quoting it as an authority on an approach to RSA encryption of data longer than the RSA modulus. I'm not sure it will help you with this project as it is not a tutorial on encryption and assumes some significant background knowledge.


I think Java is a great language, and would very much like to improve the quality of my coding.

Although I knew the line of the error, I didn't know how to fix it! I am not familiar with many of the terms & concepts at this point; PKCS1, modulus length, padding etc. I thought I was only using AES256 and not RSA as well. I did O-level maths, but did not study maths further. So please forgive my ignorance.



You should do some background reading before the weekend to understand the difference between symmetric block encryption such as AES and asymmetric public key encryption such as RSA. Some maths is required to understand the detail but not to just understand the concepts.

 
passwords must contain 14 characters, a number, punctuation, a small bird, a bit of cheese and a tiny ad.
We need your help - Coderanch server fundraiser
https://coderanch.com/wiki/782867/Coderanch-server-fundraiser
reply
    Bookmark Topic Watch Topic
  • New Topic