AEADCipher Related Problem

[WilburZjh]

<<grinder>>

[WilburZjh]

In NSS docs version 3.15, it is mentioned that for Cipher mode AES-GCM.

PK11_Decrypt - Performs decryption as a single PKCS#11 operation (eg: not multi-part). This is necessary for AES-GCM.
PK11_Encrypt - Performs encryption as a single PKCS#11 operation (eg: not multi-part). This is necessary for AES-GCM.

[WilburZjh]

Single-part

Modify the static int variable multiples from
private static int multiples = 3;
to
private static int multiples = 0;

Then modify the iterative for-loop at line 80 from

      /*
        * Iterate through various sizes to make sure that the code works with
        * internal temp buffer size 4096.
        */
       for (int t = 1; t <= multiples; t++) {
           int size = t * dataSize;

           System.out.println("\nTesting data size: " + size);

           try {
               decrypt(cipher, key, s, dataChunk, t,
                       ByteBuffer.allocate(dataSize),
                       ByteBuffer.allocate(size),
                       ByteBuffer.allocateDirect(dataSize),
                       ByteBuffer.allocateDirect(size));
           } catch (Exception e) {
               System.out.println("\tFailed with data size " + size);
               failedOnce = true;
               failedReason = e;
           }

to

        /*
         * Iterate through various sizes to make sure that the code works with
         * internal temp buffer size 4096.
         */
        for (int t = 0; t <= multiples; t++) {
            int size = t * dataSize; 

            System.out.println("\nTesting data size: " + size);

            try {
                decrypt(cipher, key, s, dataChunk, t,
                        ByteBuffer.allocate(dataSize),
                        ByteBuffer.allocate(size),
                        ByteBuffer.allocateDirect(dataSize),
                        ByteBuffer.allocateDirect(size));
            } catch (Exception e) {
                System.out.println("\tFailed with data size " + size);
                failedOnce = true;
                failedReason = e;
            }

The variable t will pass to the decrypt() function as one of the parameters which is named as multiples inside of decrypt() function.

private static void decrypt(Cipher cipher, SecretKey key,
            GCMParameterSpec s, byte[] dataChunk, int multiples,
            ByteBuffer heapIn, ByteBuffer heapOut, ByteBuffer directIn,
            ByteBuffer directOut) throws Exception {

Once the multiples is edited to 0, at line 149, there will be no cipher.update() operation. Instead, a cipher.doFinal(inBB, outBB) operation will be executed successfully.

            try {
                // Always re-init the Cipher object so cipher is in
                // a good state for future testing
                cipher.init(Cipher.DECRYPT_MODE, key, s);

                for (int i = 0; i < multiples; i++) {
                    inBB.flip();
                    cipher.update(inBB, outBB);
                    if (inBB.hasRemaining()) {
                        throw new Exception("buffer not empty");
                    }
                }
                // finish decryption and process all data buffered
                cipher.doFinal(inBB, outBB);
                throw new RuntimeException("Error: doFinal completed without exception");
            } catch (AEADBadTagException ex) {
                System.out.println("Expected AEADBadTagException thrown");
                continue;
            }

============================================================================================

Once the cipher.doFinal(inBB, outBB) operation is executed successfully, it will thrown a new RuntimeException

Testing SunPKCS11-NSS-FIPS version 1.8

Testing data size: 0
 HEAP_HEAP
	Failed with data size 0
Exception in thread "main" java.lang.RuntimeException: Error: doFinal completed without exception
	at TestGCMWithByteBuffer.decrypt(TestGCMWithByteBuffer.java:158)
	at TestGCMWithByteBuffer.test(TestGCMWithByteBuffer.java:86)
	at TestGCMWithByteBuffer.main(TestGCMWithByteBuffer.java:52)

It indicates that the doFinal() function is executed successfully without executing the cipher.update() method.

============================================================================================

Similarly, changing
private static int multiples = 3;
to
private static int multiples = 1;

And changing
i=0
to
i=1
at line 149

The cipher.update(inBB, outBB) function is not executed and only the cipher.doFinal(inBB, outBB) operation is executed. It is indicated that the the cipher.doFinal(inBB, outBB) operation is executed successfully without executing the cipher.update(inBB, outBB) method.

Testing SunPKCS11-NSS-FIPS version 1.8

Testing data size: 4096
 HEAP_HEAP
	Failed with data size 4096
Exception in thread "main" java.lang.RuntimeException: Error: doFinal completed without exception
	at TestGCMWithByteBuffer.decrypt(TestGCMWithByteBuffer.java:158)
	at TestGCMWithByteBuffer.test(TestGCMWithByteBuffer.java:86)
	at TestGCMWithByteBuffer.main(TestGCMWithByteBuffer.java:52)

[WilburZjh]

Multi-part

The traceStack from <<grinder>> is

javax.crypto.BadPaddingException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ENCRYPTED_DATA_INVALID
	at sun.security.pkcs11.P11AEADCipher.handleException(P11AEADCipher.java:749)
	at sun.security.pkcs11.P11AEADCipher.implDoFinal(P11AEADCipher.java:722)
	at sun.security.pkcs11.P11AEADCipher.engineDoFinal(P11AEADCipher.java:548)
	at javax.crypto.Cipher.doFinal(Cipher.java:2463)
	at TestGCMWithByteBuffer.decrypt(TestGCMWithByteBuffer.java:157)
	at TestGCMWithByteBuffer.test(TestGCMWithByteBuffer.java:86)
	at TestGCMWithByteBuffer.main(TestGCMWithByteBuffer.java:52)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
	at java.lang.Thread.run(Thread.java:826)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ENCRYPTED_DATA_INVALID
	at sun.security.pkcs11.wrapper.PKCS11.C_Decrypt(Native Method)
	at sun.security.pkcs11.P11AEADCipher.implDoFinal(P11AEADCipher.java:708)
	... 11 more

This happens because the AES-GCM-Nopadding cipher executes both cipher.update(inBB, outBB) and cipher.doFinal(inBB, outBB). Since in FIPS mode, we are currently utilizing NSS as DB. It only support single PKCS#11 operation for AES-GCM specially.