Encrypted values all contain same final characters

[Travis Thomas]

This isn't a problem per se but I find the behavior odd and I would like to understand it.

I'm using the standard Java Cipher class to encrypt sensitive information. I've got it functional--it encrypts the user input, and when requested, successfully decrypts it, yielding the desired datum. However, if I look at the encrypted result of each of the inputs, there is a consistent tail on the data. That is, this string: "Lxõò±"àÐêð" is present at the end of every encrypted value. Since I'm using one key for all the encryptions, I assume that some other piece of information is being encrypted separately and the encrypted results pasted together. Any ideas what this extra information is?

Thanks.

[James Sabre]

So what are you encrypting and how? I hope you are not using ECB block mode since it leaks information, it allows splicing of ciphertext and is generally considered insecure. You should be using something like CBC block mode with a random IV.

[Pat Farrell]

Before we can help much (but @james has made a very good start) we need to know exactly which cipher you are using.

And it would be a lot easier to help if you convert the string to hex. Its unclear from the glyphs that you have posted how many octets are suspect.

[Travis Thomas]

Here is the code I am using to initialize the Cipher. I'm unsure which mode of encryption is the default.

byte[] results = null;
try {
	key = getKey();
	secretKeySpec = new SecretKeySpec(key, algorithm); //algorithm = "AES"
	cipher = Cipher.getInstance(algorithm);
	cipher.init(Cipher.ENCRYPT_MODE, secretKeySpec);
	results = cipher.doFinal(s);
} finally {
	cipher = null;
	secretKeySpec = null;
	key = null; // empty everything for GC
	if (results == null)
		results = FAIL; //FAIL is a constant
}
return results;

Here are 3 encrypted... 32 bytes, 16 of which are the same each timeHex values. Original Strings are 16 characters long:

D2 C4 C3 5D 4C B2 BE C9 A8 C7 5F F4 B9 C1 0C 51   20 CD 06 16 4C 68 F5 F2 15 B1 22 E0 D0 14 EA F0
F6 97 85 22 B6 F5 4A 88 23 10 12 D3 20 80 A5 FF   20 CD 06 16 4C 68 F5 F2 15 B1 22 E0 D0 14 EA F0
39 B4 9E 42 39 DF D6 01 4B 6D 49 CB 74 36 2E 59   20 CD 06 16 4C 68 F5 F2 15 B1 22 E0 D0 14 EA F0

[Travis Thomas]

Hm. I've run into a secondary bug--when I decrypt the data and re-save it, I get an exception stating the data is too long for the SQL column. Tracking it down...

Edit: resolved. Original issue remains.

[James Sabre]

Trevor Basden wrote:Here is the code I am using to initialize the Cipher. I'm unsure which mode of encryption is the default.

byte[] results = null;
try {
	key = getKey();
	secretKeySpec = new SecretKeySpec(key, algorithm); //algorithm = "AES"
	cipher = Cipher.getInstance(algorithm);
	cipher.init(Cipher.ENCRYPT_MODE, secretKeySpec);
	results = cipher.doFinal(s);
} finally {
	cipher = null;
	secretKeySpec = null;
	key = null; // empty everything for GC
	if (results == null)
		results = FAIL; //FAIL is a constant
}
return results;

Here are 3 encrypted... 32 bytes, 16 of which are the same each timeHex values. Original Strings are 16 characters long:

D2 C4 C3 5D 4C B2 BE C9 A8 C7 5F F4 B9 C1 0C 51   20 CD 06 16 4C 68 F5 F2 15 B1 22 E0 D0 14 EA F0
F6 97 85 22 B6 F5 4A 88 23 10 12 D3 20 80 A5 FF   20 CD 06 16 4C 68 F5 F2 15 B1 22 E0 D0 14 EA F0
39 B4 9E 42 39 DF D6 01 4B 6D 49 CB 74 36 2E 59   20 CD 06 16 4C 68 F5 F2 15 B1 22 E0 D0 14 EA F0

This code

	cipher = Cipher.getInstance("AES");

actually gives you

	cipher = Cipher.getInstance("AES/ECB/PKCS5Padding");

As I said in my previous reply, ECB is generally not a good idea since it allow the splicing of ciphertext to forge valid ciphertext. You need to use one of the block modes such as CBC with a random IV.

The problem with using a random IV is that it has to be stored with the ciphertext and is a block long. For AES this means 16 byte extra with every ciphertext on top of the padding that must be applied. Using AES with CBC and PKCS5 padding means that your ciphertext is therefore between 17 (there is always at least 1 byte of padding) and 32 bytes longer than the bytes of the cleartext. There is nothing secret about the random IV so it is normal to write it as a prefix to the ciphertext so that it can be used in the decryption process.

If you don't specify the IV when initialiing then using

	cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
	cipher.init(Cipher.ENCRYPT_MODE, secretKeySpec);

automatically generates a random IV for you which you can get from the Cipher using the method

byte[] 	getIV()

and you can prefix your doFinal() result with this.

When you come to decrypt then you must first read the IV and use that to initialise the decryption cipher. You can then decrypt the remainder using the doFinal() of the decryption cipher.

If you follow this approach you will get a different value for the ciphertext (see note) for a given cleartext every time you use it.

My standard example of forgery using ECB mode with DES (simple enough to modify to forge using AES as long as you have a long enough message)

import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;

public class ForgeryThroughSplicingUsingECB
{
    public static void main(String[] args) throws Exception
    {
        final String plaintext1 = "Pay Fred £100.00";
        final String plaintext2 = "Pay Bill £10.00";

        final SecretKey key = new SecretKeySpec(new byte[]
                {
                    1, 2, 3, 4, 5, 6, 7, 8
                }, "DES");
        Cipher cipher = Cipher.getInstance("DES/ECB/PKCS5Padding");
        cipher.init(Cipher.ENCRYPT_MODE, key);

        final byte[] ciphertext1 = cipher.doFinal(plaintext1.getBytes("UTF-8"));
        final byte[] ciphertext2 = cipher.doFinal(plaintext2.getBytes("UTF-8"));

        final byte[] splicedCiphertext = new byte[ciphertext2.length];
        for (int i = 0; i < 8; i++)
        {
            splicedCiphertext[i] = ciphertext2[i];
        }
        for (int i = 8; i < ciphertext1.length; i++)
        {
            splicedCiphertext[i] = ciphertext1[i];
        }
        cipher.init(Cipher.DECRYPT_MODE, key);
        final String splicedPlaintext = new String(cipher.doFinal(splicedCiphertext), "utf-8");
        System.out.println("Forgery = " + splicedPlaintext);
    }

}

Note - not strictly true since the secure random number generator could give the same result for two IVs but it is very much against the odds. You would need about 256 ^ 8 random IVs to have an even chance of getting a duplicate pair of IVs (the Birthday paradox).

[Pat Farrell]

James Sabre wrote:

        final SecretKey key = new SecretKeySpec(new byte[]
                {
                    1, 2, 3, 4, 5, 6, 7, 8
                }, "DES");

One nit on James' excellent explaination. The DES spec uses the high order 7 bits per octet (aka byte), resulting in the lowest bit value being ignored. So 2 and 3 are the same value, as are 4 and 5. Thus, James' key is really

        final SecretKey key = new SecretKeySpec(new byte[]
                {
                    0, 2, 2, 4, 4, 6, 6, 8
                }, "DES");

Now, no one should be using DES any more, its been obsolete for well over a decade. But keys in DES are not what they seem to the casual observer.

[James Sabre]

Pat Farrell wrote:
One nit on James' excellent explaination.

Sorry Pat but that 'nit' is irrelevant in the context of this thread and is just a distraction.

[Pat Farrell]

James Sabre wrote:Sorry Pat but that 'nit' is irrelevant in the context of this thread and is just a distraction.

Huh? Why this response?

Too many folks try to write crypto code from examples they pull from books or websites. They don't understand what they are doing, and don't understand any of the subtleties. Yet as this thread shows, its all about subtleties. So good example code should be correct. If you had used AES, the keys you used were OK, but they were wrong for DES. If you are claiming to be an expert, and complaining about irrelevant comments, then I think that your example code should actually be correct.

[James Sabre]

Pat Farrell wrote:

James Sabre wrote:Sorry Pat but that 'nit' is irrelevant in the context of this thread and is just a distraction.

Huh? Why this response?

Too many folks try to write crypto code from examples they pull from books or websites. They don't understand what they are doing, and don't understand any of the subtleties. Yet as this thread shows, its all about subtleties. So good example code should be correct. If you had used AES, the keys you used were OK, but they were wrong for DES. If you are claiming to be an expert, and complaining about irrelevant comments, then I think that your example code should actually be correct.

So by that token your correction to my key should be correct but it isn't . DES keys should have odd parity so your corrected key bytes should have been

{1, 2, 2, 4, 4, 7, 7, 8}

Many years ago when I wrote the example I deliberately kept it very very simple and linear. You could have criticised my exception handling and you could have criticised the fact that the key was stored in the code and I would have agreed with you but I would not have changed anything. To fix both of these would have taken the focus away from the heart of the code that shows that ECB block mode is unsafe because ciphertext can be spliced to create forgeries.

I still think your nit-pick is irrelevant to the original discussion. The OP was quite rightly worried about the fact that some of his ciphertext blocks in different messages were the same. I pointed out that this was a natural consequence of using ECB block mode and I provided a simple example that illustrated how his observation could be used to forge messages. To introduce the fact that I had not used the correct parity for the key bytes added nothing to the discussion and diverted attention away from ECB. I will let readers of this thread judge whether or not your diversion is relevant.

I have locked horns with you in the past and know that I don't have debating skills, the technical expertise, the will or the stamina to continue with this.

[Pat Farrell]

James Sabre wrote:So by that token your correction to my key should be correct but it isn't . DES keys should have odd parity so your corrected key bytes should have been

{1, 2, 2, 4, 4, 7, 7, 8}

Yet your correction is still wrong, since DES ignores the lowest order bit. Your "7" is the same as my "6" and your "1" is the same as "0".

Details matter. Stop being an a**

[James Sabre]

Pat Farrell wrote:

James Sabre wrote:So by that token your correction to my key should be correct but it isn't . DES keys should have odd parity so your corrected key bytes should have been

{1, 2, 2, 4, 4, 7, 7, 8}

Yet your correction is still wrong, since DES ignores the lowest order bit. Your "7" is the same as my "6" and your "1" is the same as "0".

Details matter. Stop being an a**

The pot calling the kettle black!

        byte[] keyBytes =
        {
            1, 2, 3, 4, 5, 6, 7, 8
        };
        final DESKeySpec keySpec = new DESKeySpec(keyBytes);
        SecretKeyFactory keyFactory = SecretKeyFactory.getInstance("DES");
        SecretKey key = keyFactory.generateSecret(keySpec);
        System.out.println(Arrays.toString(key.getEncoded()));

[Pat Farrell]

what part of DES ignores low order bits in the key do you not understand?

[Travis Thomas]

Thanks for the help. I have implemented AES/CBC encryption, storing the IV and initializing the decryption-time Cipher with it. Here are my code snippets for any comments and anyone else's future reference:

	public byte[] encryptData(byte[] s) throws GeneralSecurityException {
		System.out.println("Encrypting...");
		byte[] iv = null;
		byte[] tempResults = null;
		byte[] results = null;
		init();
		cipher.init(Cipher.ENCRYPT_MODE, secretKeySpec);
		tempResults = cipher.doFinal(s);
		iv = cipher.getIV();
		results = new byte[iv.length + tempResults.length];
		cipher = null;
		secretKeySpec = null;
		key = null; // empty everything for GC
		System.arraycopy(iv, 0, results, 0, iv.length);
		System.arraycopy(tempResults, 0, results, iv.length, tempResults.length);
		return results;
	}

	public byte[] decryptData(byte[] s) throws GeneralSecurityException {
		byte[] iv = Arrays.copyOfRange(s, 0, IV_LENGTH);
		byte[] data = Arrays.copyOfRange(s,IV_LENGTH,s.length);
		byte[] results;
		init();
		cipher.init(Cipher.DECRYPT_MODE, secretKeySpec, new IvParameterSpec(iv));
		results = cipher.doFinal(data);
		cipher = null;
		secretKeySpec = null;
		key = null; // empty everything for GC
		return results;
	}

and finally, my init() method:

	public void init() throws GeneralSecurityException {
		key = getKey();
		secretKeySpec = new SecretKeySpec(key, "AES");
		cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
	}

[greg stark]

Pat Farrell wrote:

        final SecretKey key = new SecretKeySpec(new byte[]
                {
                    0, 2, 2, 4, 4, 6, 6, 8
                }, "DES");

Since the low order bit is ignored, your key is equivalent to James', not better or more correct.

[Pat Farrell]

greg stark wrote:Since the low order bit is ignored, your key is equivalent to James', not better or more correct.

Yes, that was my intent, to show an equivalent set of octets to his as they are interpreted by any compliant DES cipher. For purposes of enciphering or deciphering using DES, James' and my keys are exactly the same.

I presented my version with the comment that it was a nit for a reason.

(Wikipedia: As nitpicking inherently requires fastidious, meticulous attention to detail, the term has become appropriated to describe the practice of meticulously searching for minor, even trivial errors in detail (often referred to as "nits" as well), and then criticising them (see nitpicking (pastime)).)

While part of the programs that we write have to be read by the compiler, I believe that most of the time spent "reading" source code is done by humans. So we have two audiences: the compiler and the future editor/programmer. And for this second audience, its safe to assume that they have never read the boring details of the DES spec, so they will not on casual read know that both James' and my keys cause exactly the same action.

His version look very natural but don't work the way a human would expect. Mine look strange, and work just like his. I believe that mine are therefor better for educating other programmers. Its OK if you don't agree with me.

[greg stark]

Pat Farrell wrote:

greg stark wrote:Since the low order bit is ignored, your key is equivalent to James', not better or more correct.

Yes, that was my intent, to show an equivalent set of octets to his as they are interpreted by any compliant DES cipher. For purposes of enciphering or deciphering using DES, James' and my keys are exactly the same.

I presented my version with the comment that it was a nit for a reason.

(Wikipedia: As nitpicking inherently requires fastidious, meticulous attention to detail, the term has become appropriated to describe the practice of meticulously searching for minor, even trivial errors in detail (often referred to as "nits" as well), and then criticising them (see nitpicking (pastime)).)

If you get to nitpick then so do I. James' version contains no errors whatsoever, not even minor ones. At best you expanded on James' explanation, not corrected it.

[Pat Farrell]

greg stark wrote:If you get to nitpick then so do I. James' version contains no errors whatsoever, not even minor ones. At best you expanded on James' explanation, not corrected it.

What is your problem? In my initial response, I did not say his code was "wrong" or needed "correction"

Go find a fight with someone who cares about what you are ranting about.

[greg stark]

Pat Farrell wrote:

What is your problem? In my initial response, I did not say his code was "wrong" or needed "correction"

Go find a fight with someone who cares about what you are ranting about.

Pat,

Peace, buddy. You do a great job providing excellent answers to many questions. This little spat does nothing to lessen my considerable respect for your knowledge and talents.