projects/linux·commit d47db9bf

Merge tag 'libcrypto-updates-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux

d47db9bf · Linus Torvalds <[email protected]> · 2026-08-17 19:16:42 -07:00

Pull crypto library updates from Eric Biggers:
 "Add library APIs for most AES encryption modes that are used in the
  kernel (ECB, CBC, CBC-CTS, CTR, XCTR, XTS, GCM, CCM).

  These AES modes have many in-kernel users that are currently using the
  crypto_skcipher or crypto_aead APIs. These existing APIs are difficult
  to use and inefficient. Until now, the lack of proper library support
  for these has been the main gap in the crypto library.

  This set of changes is the next stage of addressing it:

   - Implement the new APIs on top of the existing support for
     single-block AES in the library.

   - Fully document the new APIs.

   - Migrate the only user of the old AES-GCM library API to the new,
     more flexible API; then remove the old API and its implementation.

   - Wire up the new APIs to the traditional crypto API by adding
     crypto_skcipher and crypto_aead algorithms.

     This makes the new APIs be covered by the traditional crypto API's
     self-tests. It also makes them be already used for real on systems
     that don't have architecture-optimized code for these modes.

     But most importantly, this is a prerequisite for migrating the
     architecture-optimized code for these AES modes (i.e.
     arch/*/crypto/aes*) into the library, which as usual will eliminate
     a lot of redundant "glue" code.

  Note that unlike some of the other algorithms that have been migrated
  to the library, e.g. SHA-512, for these AES modes there was too much
  to get done in one cycle. Nor did it make sense to handle these modes
  one at a time, because they tend to be coupled together or depend on
  each other, especially in the architecture-optimized AES code.

  Thus, most of the benefits (reductions in lines of code, performance
  improvements, etc.) will follow in later cycles when
  architecture-optimized code is migrated into the library and users of
  crypto_skcipher and crypto_aead are updated to use the new APIs.

  The design of the new APIs was informed by writing proof-of-concept
  patches for many kernel subsystems currently accessing these same
  algorithms via crypto_skcipher or crypto_aead (patches 18-33 of
  https://lore.kernel.org/r/[email protected]/).

  While those patches will be resent for real later, the total diffstat
  for them was negative 1905 lines. So clearly the new APIs are quite a
  bit easier to use and align better with what users actually need.

  Besides the new AES encryption APIs, there are also a few changes for
  improved AES-CMAC key and context zeroization"

* tag 'libcrypto-updates-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux:
  mac80211: fils_aead: Use __cleanup() instead of memzero_explicit()
  Bluetooth: SMP: clear the aes_cmac_key when done
  smb: clear the aes_cmac_key and aes_cmac_ctx when done
  lib/crypto: aes-cmac: Add zeroization functions
  lib/crypto: aesgcm: Remove old AES-GCM library
  x86/sev: Remove obsolete virtual address check
  x86/sev: Use new AES-GCM library
  crypto: aes - Add CCM support using library
  crypto: aes - Add GCM support using library
  crypto: aes - Add XTS support using library
  crypto: aes - Add CTR and XCTR support using library
  crypto: aes - Add CBC and CBC-CTS support using library
  crypto: aes - Add ECB support using library
  lib/crypto: aes: Add CCM support
  lib/crypto: aes: Add GCM support
  lib/crypto: aes: Add XTS support
  lib/crypto: aes: Add CTR and XCTR support
  lib/crypto: aes: Add CBC and CBC-CTS support
  lib/crypto: aes: Add ECB support
  crypto: xts - Split out __xts_verify_key() helper
Commit
d47db9bf50d28647689e6743ee93c45cb755ffc3
Parents
1d7443e4 · ff6ac629

Merge commit. Changes below are compared with the first parent: 1d7443e4.

Changes

27 files changed · +3073 −799

Only the first 600 diff lines are shown. Download the complete patch.

Documentation/crypto/libcrypto-auth-encryption.rst

View file
diff --git a/Documentation/crypto/libcrypto-auth-encryption.rst b/Documentation/crypto/libcrypto-auth-encryption.rst
new file mode 100644
index 0000000..1e52768
--- /dev/null
+++ b/Documentation/crypto/libcrypto-auth-encryption.rst
@@ -0,0 +1,20 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Authenticated encryption
+========================
+
+These APIs provide support for authenticated encryption and decryption.
+
+AES-CCM
+-------
+
+This API provides support for AES in the CCM mode of operation.
+
+.. kernel-doc:: include/crypto/aes-ccm.h
+
+AES-GCM
+-------
+
+This API provides support for AES in the GCM mode of operation.
+
+.. kernel-doc:: include/crypto/aes-gcm.h

Documentation/crypto/libcrypto-unauth-encryption.rst

View file
diff --git a/Documentation/crypto/libcrypto-unauth-encryption.rst b/Documentation/crypto/libcrypto-unauth-encryption.rst
new file mode 100644
index 0000000..b4639b8
--- /dev/null
+++ b/Documentation/crypto/libcrypto-unauth-encryption.rst
@@ -0,0 +1,49 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Unauthenticated encryption
+==========================
+
+These APIs provide support for unauthenticated encryption and decryption,
+including bare stream ciphers and other length-preserving algorithms such as
+block ciphers in XTS mode. The legitimate use cases for these algorithms are:
+
+- Support for legacy protocols that really should have chosen an authenticated
+ mode (or even another primitive entirely) but didn't.
+
+- Internal components of authenticated modes. For example, AES-CTR is used by
+ AES-GCM and AES-CCM internally.
+
+- Storage encryption that cannot accommodate ciphertext expansion. Usually
+ AES-XTS is used for this.
+
+- Stream ciphers for key derivation and random number generation.
+
+Besides the above, these shouldn't be used.
+
+AES-CBC and AES-CBC-CTS
+-----------------------
+
+This API provides support for AES in the CBC and CBC-CTS modes of operation.
+
+.. kernel-doc:: include/crypto/aes-cbc.h
+
+AES-CTR and AES-XCTR
+--------------------
+
+This API provides support for AES in the CTR and XCTR modes of operation.
+
+.. kernel-doc:: include/crypto/aes-ctr.h
+
+AES-ECB
+-------
+
+This API provides support for AES in the ECB mode of operation.
+
+.. kernel-doc:: include/crypto/aes-ecb.h
+
+AES-XTS
+-------
+
+This API provides support for AES in the XTS mode of operation.
+
+.. kernel-doc:: include/crypto/aes-xts.h

Documentation/crypto/libcrypto.rst

View file
diff --git a/Documentation/crypto/libcrypto.rst b/Documentation/crypto/libcrypto.rst
index 0733e60..e911e05 100644
--- a/Documentation/crypto/libcrypto.rst
+++ b/Documentation/crypto/libcrypto.rst
@@ -159,8 +159,10 @@ API documentation
.. toctree::
:maxdepth: 2
+ libcrypto-auth-encryption
libcrypto-blockcipher
libcrypto-hash
libcrypto-signature
+ libcrypto-unauth-encryption
libcrypto-utils
sha3

arch/x86/Kconfig

View file
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index bdad90f..0b89641 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1495,7 +1495,7 @@ config AMD_MEM_ENCRYPT
select ARCH_HAS_CC_PLATFORM
select X86_MEM_ENCRYPT
select UNACCEPTED_MEMORY
- select CRYPTO_LIB_AESGCM
+ select CRYPTO_LIB_AES_GCM
help
Say yes to enable support for the encryption of system memory.
This requires an AMD processor that supports Secure Memory

arch/x86/coco/sev/core.c

View file
diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index ecd77d3..cc292d7 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -25,7 +25,7 @@
#include <linux/psp-sev.h>
#include <linux/dmi.h>
#include <uapi/linux/sev-guest.h>
-#include <crypto/gcm.h>
+#include <crypto/aes-gcm.h>
#include <asm/init.h>
#include <asm/cpu_entry_area.h>
@@ -1535,21 +1535,21 @@ static u8 *get_vmpck(int id, struct snp_secrets_page *secrets, u32 **seqno)
return key;
}
-static struct aesgcm_ctx *snp_init_crypto(u8 *key, size_t keylen)
+static struct aes_gcm_key *snp_init_crypto(const u8 *key, size_t keylen)
{
- struct aesgcm_ctx *ctx;
+ struct aes_gcm_key *gcm_key;
- ctx = kzalloc_obj(*ctx);
- if (!ctx)
+ gcm_key = kzalloc_obj(*gcm_key);
+ if (!gcm_key)
return NULL;
- if (aesgcm_expandkey(ctx, key, keylen, AUTHTAG_LEN)) {
- pr_err("Crypto context initialization failed\n");
- kfree(ctx);
+ if (aes_gcm_preparekey(gcm_key, key, keylen, AUTHTAG_LEN)) {
+ pr_err("AES-GCM key preparation failed\n");
+ kfree_sensitive(gcm_key);
return NULL;
}
- return ctx;
+ return gcm_key;
}
int snp_msg_init(struct snp_msg_desc *mdesc, int vmpck_id)
@@ -1572,8 +1572,8 @@ int snp_msg_init(struct snp_msg_desc *mdesc, int vmpck_id)
mdesc->vmpck_id = vmpck_id;
- mdesc->ctx = snp_init_crypto(mdesc->vmpck, VMPCK_KEY_LEN);
- if (!mdesc->ctx)
+ mdesc->gcm_key = snp_init_crypto(mdesc->vmpck, VMPCK_KEY_LEN);
+ if (!mdesc->gcm_key)
return -ENOMEM;
return 0;
@@ -1624,7 +1624,7 @@ void snp_msg_free(struct snp_msg_desc *mdesc)
if (!mdesc)
return;
- kfree(mdesc->ctx);
+ kfree_sensitive(mdesc->gcm_key);
free_shared_pages(mdesc->response, sizeof(struct snp_guest_msg));
free_shared_pages(mdesc->request, sizeof(struct snp_guest_msg));
iounmap((__force void __iomem *)mdesc->secrets);
@@ -1709,7 +1709,7 @@ static int verify_and_dec_payload(struct snp_msg_desc *mdesc, struct snp_guest_r
struct snp_guest_msg *req_msg = &mdesc->secret_request;
struct snp_guest_msg_hdr *req_msg_hdr = &req_msg->hdr;
struct snp_guest_msg_hdr *resp_msg_hdr = &resp_msg->hdr;
- struct aesgcm_ctx *ctx = mdesc->ctx;
+ struct aes_gcm_key *gcm_key = mdesc->gcm_key;
u8 iv[GCM_AES_IV_SIZE] = {};
pr_debug("response [seqno %lld type %d version %d sz %d]\n",
@@ -1732,23 +1732,21 @@ static int verify_and_dec_payload(struct snp_msg_desc *mdesc, struct snp_guest_r
* If the message size is greater than our buffer length then return
* an error.
*/
- if (unlikely((resp_msg_hdr->msg_sz + ctx->authsize) > req->resp_sz))
+ if (unlikely(resp_msg_hdr->msg_sz + AUTHTAG_LEN > req->resp_sz))
return -EBADMSG;
/* Decrypt the payload */
memcpy(iv, &resp_msg_hdr->msg_seqno, min(sizeof(iv), sizeof(resp_msg_hdr->msg_seqno)));
- if (!aesgcm_decrypt(ctx, req->resp_buf, resp_msg->payload, resp_msg_hdr->msg_sz,
- &resp_msg_hdr->algo, AAD_LEN, iv, resp_msg_hdr->authtag))
- return -EBADMSG;
-
- return 0;
+ return aes_gcm_decrypt(req->resp_buf, resp_msg->payload,
+ resp_msg_hdr->msg_sz, resp_msg_hdr->authtag,
+ &resp_msg_hdr->algo, AAD_LEN, iv, gcm_key);
}
static int enc_payload(struct snp_msg_desc *mdesc, u64 seqno, struct snp_guest_req *req)
{
struct snp_guest_msg *msg = &mdesc->secret_request;
struct snp_guest_msg_hdr *hdr = &msg->hdr;
- struct aesgcm_ctx *ctx = mdesc->ctx;
+ struct aes_gcm_key *gcm_key = mdesc->gcm_key;
u8 iv[GCM_AES_IV_SIZE] = {};
memset(msg, 0, sizeof(*msg));
@@ -1769,12 +1767,12 @@ static int enc_payload(struct snp_msg_desc *mdesc, u64 seqno, struct snp_guest_r
pr_debug("request [seqno %lld type %d version %d sz %d]\n",
hdr->msg_seqno, hdr->msg_type, hdr->msg_version, hdr->msg_sz);
- if (WARN_ON((req->req_sz + ctx->authsize) > sizeof(msg->payload)))
+ if (WARN_ON(req->req_sz + AUTHTAG_LEN > sizeof(msg->payload)))
return -EBADMSG;
memcpy(iv, &hdr->msg_seqno, min(sizeof(iv), sizeof(hdr->msg_seqno)));
- aesgcm_encrypt(ctx, msg->payload, req->req_buf, req->req_sz, &hdr->algo,
- AAD_LEN, iv, hdr->authtag);
+ aes_gcm_encrypt(msg->payload, req->req_buf, req->req_sz, hdr->authtag,
+ &hdr->algo, AAD_LEN, iv, gcm_key);
return 0;
}
@@ -1869,15 +1867,6 @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
u64 seqno;
int rc;
- /*
- * enc_payload() calls aesgcm_encrypt(), which can potentially offload to HW.
- * The offload's DMA SG list of data to encrypt has to be in linear mapping.
- */
- if (!virt_addr_valid(req->req_buf) || !virt_addr_valid(req->resp_buf)) {
- pr_warn("AES-GSM buffers must be in linear mapping");
- return -EINVAL;
- }
-
guard(mutex)(&snp_cmd_mutex);
/* Check if the VMPCK is not empty */

arch/x86/include/asm/sev.h

View file
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index 594cfa1..9e7a077 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -314,7 +314,7 @@ struct snp_msg_desc {
struct snp_secrets_page *secrets;
- struct aesgcm_ctx *ctx;
+ struct aes_gcm_key *gcm_key;
u32 *os_area_msg_seqno;
u8 *vmpck;

crypto/Kconfig

View file
diff --git a/crypto/Kconfig b/crypto/Kconfig
index b61401b..981d5dc 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -358,8 +358,19 @@ config CRYPTO_AES
tristate "AES (Advanced Encryption Standard)"
select CRYPTO_ALGAPI
select CRYPTO_LIB_AES
+ select CRYPTO_LIB_AES_CBC if CRYPTO_CBC != n || CRYPTO_CTS != n
select CRYPTO_LIB_AES_CBC_MACS if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
+ select CRYPTO_LIB_AES_CCM if CRYPTO_CCM != n
+ select CRYPTO_LIB_AES_CTR if CRYPTO_CTR != n || CRYPTO_XCTR != n
+ select CRYPTO_LIB_AES_ECB if CRYPTO_ECB != n
+ select CRYPTO_LIB_AES_GCM if CRYPTO_GCM != n
+ select CRYPTO_LIB_AES_XTS if CRYPTO_XTS != n
+ select CRYPTO_AEAD if CRYPTO_GCM != n || CRYPTO_CCM != n
select CRYPTO_HASH if CRYPTO_CMAC != n || CRYPTO_XCBC != n || CRYPTO_CCM != n
+ # CRYPTO_SKCIPHER should be selected only if a mode that needs it is
+ # enabled, but that doesn't work due to a recursive dependency caused by
+ # CRYPTO_SKCIPHER selecting CRYPTO_ECB. So just always select it.
+ select CRYPTO_SKCIPHER
help
AES cipher algorithms (Rijndael)(FIPS-197, ISO/IEC 18033-3)

crypto/aes.c

View file
diff --git a/crypto/aes.c b/crypto/aes.c
index 6bf23eb..94791f4 100644
--- a/crypto/aes.c
+++ b/crypto/aes.c
@@ -6,12 +6,22 @@
*/
#include <crypto/aes-cbc-macs.h>
+#include <crypto/aes-cbc.h>
+#include <crypto/aes-ccm.h>
+#include <crypto/aes-ctr.h>
+#include <crypto/aes-ecb.h>
+#include <crypto/aes-gcm.h>
+#include <crypto/aes-xts.h>
#include <crypto/aes.h>
#include <crypto/algapi.h>
+#include <crypto/internal/aead.h>
#include <crypto/internal/hash.h>
+#include <crypto/internal/skcipher.h>
+#include <crypto/scatterwalk.h>
#include <linux/module.h>
static_assert(__alignof__(struct aes_key) <= CRYPTO_MINALIGN);
+static_assert(__alignof__(struct aes_enckey) <= CRYPTO_MINALIGN);
static int crypto_aes_setkey(struct crypto_tfm *tfm, const u8 *in_key,
unsigned int key_len)
@@ -85,7 +95,6 @@ static int __maybe_unused crypto_aes_cmac_digest(struct shash_desc *desc,
return 0;
}
-static_assert(__alignof__(struct aes_enckey) <= CRYPTO_MINALIGN);
#define AES_CBCMAC_KEY(tfm) ((struct aes_enckey *)crypto_shash_ctx(tfm))
#define AES_CBCMAC_CTX(desc) ((struct aes_cbcmac_ctx *)shash_desc_ctx(desc))
@@ -200,6 +209,828 @@ static struct shash_alg mac_algs[] = {
#endif
};
+static __maybe_unused int
+crypto_aes_skcipher_setkey(struct crypto_skcipher *tfm, const u8 *in_key,
+ unsigned int key_len)
+{
+ struct aes_key *key = crypto_skcipher_ctx(tfm);
+
+ return aes_preparekey(key, in_key, key_len);
+}
+
+static __maybe_unused int
+crypto_aes_skcipher_setenckey(struct crypto_skcipher *tfm, const u8 *in_key,
+ unsigned int key_len)
+{
+ struct aes_enckey *key = crypto_skcipher_ctx(tfm);
+
+ return aes_prepareenckey(key, in_key, key_len);
+}
+
+/*
+ * Return true if the request uses only a single scatterlist element and high
+ * memory isn't enabled. This assumes that both scatterlists are non-NULL, i.e.
+ * the caller must have handled the cryptlen == 0 case already.
+ */
+static inline bool
+skcipher_request_is_linear_lowmem(const struct skcipher_request *req)
+{
+ return !IS_ENABLED(CONFIG_HIGHMEM) &&
+ req->dst->length >= req->cryptlen &&
+ req->src->length >= req->cryptlen;
+}
+
+/*
+ * Call crypt_func() (a function that operates on simple virtual addresses) zero
+ * or more times to en/decrypt 'cryptlen' bytes of data from the source
+ * scatterlist 'src' and write it into the destination scatterlist 'dst',
+ * starting at 'start_pos' bytes into both.
+ *
+ * This always calls crypt_func() with a length that's a multiple of
+ * AES_BLOCK_SIZE, except the last call which includes any remainder. This is
+ * implemented by using an on-stack bounce buffer when necessary. The current
+ * implementation also tries to prefer passing at least 4 blocks, so e.g.
+ * scatterlist entries [16,16,16,16] result in a single 64-byte call.
+ *
+ * The scatterlists must describe either entirely different memory
+ * (out-of-place) or entirely the same memory (in-place). In the latter case,
+ * crypt_func() is always called with the source and dest pointers the same.
+ */
+#define AES_CRYPT_SG(crypt_func, dst, src, cryptlen, start_pos, ...) \
+ ({ \
+ unsigned int remaining = (cryptlen); \
+ unsigned int spos = (start_pos); \
+ \
+ if (remaining != 0) { \
+ struct scatter_walk dst_walk, src_walk; \
+ u8 tmp[4 * AES_BLOCK_SIZE] __aligned( \
+ __alignof__(long)); \
+ \
+ scatterwalk_start_at_pos(&dst_walk, (dst), spos); \
+ scatterwalk_start_at_pos(&src_walk, (src), spos); \
+ do { \
+ unsigned int dst_avail = scatterwalk_clamp( \
+ &dst_walk, remaining); \
+ unsigned int src_avail = scatterwalk_clamp( \
+ &src_walk, remaining); \
+ unsigned int n = min(dst_avail, src_avail); \
+ u8 *dst_virt; \
+ const u8 *src_virt; \
+ \
+ if (n < remaining) { \
+ if (n < sizeof(tmp)) { \
+ n = min(remaining, \
+ sizeof(tmp)); \
+ memcpy_from_scatterwalk( \
+ tmp, &src_walk, n); \
+ crypt_func(tmp, tmp, n, \
+ ##__VA_ARGS__); \
+ memcpy_to_scatterwalk( \
+ &dst_walk, tmp, n); \
+ remaining -= n; \
+ continue; \
+ } \
+ n = round_down(n, AES_BLOCK_SIZE); \
+ } \
+ \
+ scatterwalk_map(&dst_walk); \
+ dst_virt = dst_walk.addr; \
+ if (IS_ENABLED(CONFIG_HIGHMEM) && \
+ offset_in_page(src_walk.offset) == \
+ offset_in_page(dst_walk.offset) && \
+ sg_page(src_walk.sg) + (src_walk.offset / \
+ PAGE_SIZE) == \
+ sg_page(dst_walk.sg) + \
+ (dst_walk.offset / \
+ PAGE_SIZE)) { \
+ src_virt = dst_virt; \
+ } else { \
+ scatterwalk_map(&src_walk); \
+ src_virt = src_walk.addr; \
+ } \
+ crypt_func(dst_virt, src_virt, n, \
+ ##__VA_ARGS__); \
+ if (src_virt != dst_virt) \
+ scatterwalk_unmap(&src_walk); \
+ scatterwalk_advance(&src_walk, n); \
+ scatterwalk_done_dst(&dst_walk, n); \
+ remaining -= n; \
+ } while (remaining); \
+ memzero_explicit(tmp, sizeof(tmp)); \
+ } \
+ })
+
+/*
+ * Call ad_func() as needed to process the associated data in the first
+ * 'assoclen' bytes of the scatterlist 'src'.
+ */
+#define AES_PROCESS_ASSOC_DATA(ad_func, src, assoclen, ctx) \
+ ({ \
+ unsigned int remaining = (assoclen); \
+ \
+ if (remaining != 0) { \
+ struct scatter_walk walk; \
+ \
+ scatterwalk_start(&walk, (src)); \
+ do { \
+ unsigned int n = \
+ scatterwalk_next(&walk, remaining); \
+ \
+ ad_func((ctx), walk.addr, n); \
+ scatterwalk_done_src(&walk, n); \
+ remaining -= n; \
+ } while (remaining); \
+ } \
+ })
+
+/* AES-ECB */
+
+static __maybe_unused int crypto_aes_ecb_encrypt(struct skcipher_request *req)
+{
+ const struct aes_key *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+
+ if (unlikely(req->cryptlen % AES_BLOCK_SIZE))
+ return -EINVAL;
+ AES_CRYPT_SG(aes_ecb_encrypt, req->dst, req->src, req->cryptlen, 0,
+ key);
+ return 0;
+}
+
+static __maybe_unused int crypto_aes_ecb_decrypt(struct skcipher_request *req)
+{
+ const struct aes_key *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+
+ if (unlikely(req->cryptlen % AES_BLOCK_SIZE))
+ return -EINVAL;
+ AES_CRYPT_SG(aes_ecb_decrypt, req->dst, req->src, req->cryptlen, 0,
+ key);
+ return 0;
+}
+
+/* AES-CBC */
+
+static void crypto_aes_cbc_encrypt_sg(struct skcipher_request *req,
+ unsigned int cryptlen,
+ const struct aes_key *key)
+{
+ AES_CRYPT_SG(aes_cbc_encrypt, req->dst, req->src, cryptlen, 0, req->iv,
+ key);
+}
+
+static void crypto_aes_cbc_decrypt_sg(struct skcipher_request *req,
+ unsigned int cryptlen,
+ const struct aes_key *key)
+{
+ AES_CRYPT_SG(aes_cbc_decrypt, req->dst, req->src, cryptlen, 0, req->iv,
+ key);
+}
+
+static __maybe_unused int crypto_aes_cbc_encrypt(struct skcipher_request *req)
+{
+ const struct aes_key *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+
+ if (unlikely(req->cryptlen % AES_BLOCK_SIZE))
+ return -EINVAL;
+ crypto_aes_cbc_encrypt_sg(req, req->cryptlen, key);
+ return 0;
+}
+
+static __maybe_unused int crypto_aes_cbc_decrypt(struct skcipher_request *req)
+{
+ const struct aes_key *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+
+ if (unlikely(req->cryptlen % AES_BLOCK_SIZE))
+ return -EINVAL;
+ crypto_aes_cbc_decrypt_sg(req, req->cryptlen, key);
+ return 0;
+}
+
+/* AES-CBC-CTS */
+
+/*
+ * This handles AES-CBC-CTS en/decryption requests that use a nonlinear
+ * scatterlist layout or where HIGHMEM is enabled. It is explicitly 'noinline'
+ * to keep the temporary buffer out of the stack frame of the fast path.
+ */
+static noinline int
+crypto_aes_cbc_cts_crypt_nonlinear(struct skcipher_request *req, bool enc)
+{
+ const struct aes_key *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+ unsigned int main_len = req->cryptlen;
+ unsigned int tail_len;
+ u8 tmp[2 * AES_BLOCK_SIZE] __aligned(__alignof__(long));
+
+ if (main_len == AES_BLOCK_SIZE) {
+ /* Single block is a special case that just does CBC. */
+ if (enc)
+ crypto_aes_cbc_encrypt_sg(req, main_len, key);
+ else
+ crypto_aes_cbc_decrypt_sg(req, main_len, key);
+ return 0;
+ }
+ /* Just do the last two blocks separately. */
+ tail_len = AES_BLOCK_SIZE + ((main_len - 1) % AES_BLOCK_SIZE) + 1;
+ main_len -= tail_len;
+ if (enc)
+ crypto_aes_cbc_encrypt_sg(req, main_len, key);
+ else
+ crypto_aes_cbc_decrypt_sg(req, main_len, key);
+ memcpy_from_sglist(tmp, req->src, main_len, tail_len);
+ if (enc)
+ aes_cbc_cts_encrypt(tmp, tmp, tail_len, req->iv, key);
+ else
+ aes_cbc_cts_decrypt(tmp, tmp, tail_len, req->iv, key);
+ memcpy_to_sglist(req->dst, main_len, tmp, tail_len);
+ memzero_explicit(tmp, sizeof(tmp));
+ return 0;
+}
+
+static __maybe_unused int
+crypto_aes_cbc_cts_encrypt(struct skcipher_request *req)
+{
+ const struct aes_key *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+
+ if (unlikely(req->cryptlen < AES_BLOCK_SIZE))
+ return -EINVAL;
+ if (likely(skcipher_request_is_linear_lowmem(req))) {
+ /* Fast path */
+ aes_cbc_cts_encrypt(sg_virt(req->dst), sg_virt(req->src),
+ req->cryptlen, req->iv, key);
+ return 0;
+ }
+ return crypto_aes_cbc_cts_crypt_nonlinear(req, /* enc= */ true);
+}
+
+static __maybe_unused int
+crypto_aes_cbc_cts_decrypt(struct skcipher_request *req)
+{
+ const struct aes_key *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+
+ if (unlikely(req->cryptlen < AES_BLOCK_SIZE))
+ return -EINVAL;
+ if (likely(skcipher_request_is_linear_lowmem(req))) {
+ /* Fast path */
+ aes_cbc_cts_decrypt(sg_virt(req->dst), sg_virt(req->src),
+ req->cryptlen, req->iv, key);
+ return 0;
+ }
+ return crypto_aes_cbc_cts_crypt_nonlinear(req, /* enc= */ false);
+}
+
+/* AES-CTR */
+
+static __maybe_unused int crypto_aes_ctr_crypt(struct skcipher_request *req)
+{
+ const struct aes_enckey *key =
+ crypto_skcipher_ctx(crypto_skcipher_reqtfm(req));
+

This diff is incomplete. Download the complete patch.

drivers/virt/coco/sev-guest/sev-guest.c

View file

This diff is incomplete. Download the complete patch.

fs/smb/client/smb2transport.c

View file

This diff is incomplete. Download the complete patch.

fs/smb/server/auth.c

View file

This diff is incomplete. Download the complete patch.

include/crypto/aes-cbc-macs.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/aes-cbc.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/aes-ccm.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/aes-ctr.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/aes-ecb.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/aes-gcm.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/aes-xts.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/gcm.h

View file

This diff is incomplete. Download the complete patch.

include/crypto/xts.h

View file

This diff is incomplete. Download the complete patch.

lib/crypto/Kconfig

View file

This diff is incomplete. Download the complete patch.

lib/crypto/Makefile

View file

This diff is incomplete. Download the complete patch.

lib/crypto/aes.c

View file

This diff is incomplete. Download the complete patch.

lib/crypto/aesgcm.c

View before deletion

This diff is incomplete. Download the complete patch.

lib/crypto/tests/Kconfig

View file

This diff is incomplete. Download the complete patch.

net/bluetooth/smp.c

View file

This diff is incomplete. Download the complete patch.

net/mac80211/fils_aead.c

View file

This diff is incomplete. Download the complete patch.