[openssl/openssl] 6ca0a1: remove ossl_method_store cache culling

  Branch: refs/heads/master
  Home:   https://github.com/openssl/openssl
  Commit: 6ca0a15d9822fd6741e59a1b5ce8b688b899606a
      https://github.com/openssl/openssl/commit/6ca0a15d9822fd6741e59a1b5ce8b688b899606a
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c
    M test/property_test.c

  Log Message:
  -----------
  remove ossl_method_store cache culling

Theres no point in thrashing the cache like this, it just gives us more
opportunities to dirty the cpu cache by taking the write lock

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:03 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: d31591fc0416f56203d809db1a4c4c90a7f4c5ee
      https://github.com/openssl/openssl/commit/d31591fc0416f56203d809db1a4c4c90a7f4c5ee
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c

  Log Message:
  -----------
  Add atomic list inserted to method store cache

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:05 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 4d80958db6be6fa1b1bb7352b25599a05b54ffc3
      https://github.com/openssl/openssl/commit/4d80958db6be6fa1b1bb7352b25599a05b54ffc3
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c

  Log Message:
  -----------
  add atomic list removal to property cache

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:08 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: b84faffb2776c725f0cf40a5ede727af04c7451b
      https://github.com/openssl/openssl/commit/b84faffb2776c725f0cf40a5ede727af04c7451b
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c

  Log Message:
  -----------
  fully replace hash table with linked list

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:10 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 79cd5c9bdc9a898c468bc3a9d5efa8fdb1549e35
      https://github.com/openssl/openssl/commit/79cd5c9bdc9a898c468bc3a9d5efa8fdb1549e35
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c
    M include/internal/threads_common.h
    M providers/fips/fipsprov.c

  Log Message:
  -----------
  remove read lock from method store cache lookup

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:12 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: e8b562dbc041d80bcc3157b4fcc4b32c5b957ea8
      https://github.com/openssl/openssl/commit/e8b562dbc041d80bcc3157b4fcc4b32c5b957ea8
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c

  Log Message:
  -----------
  clean out lru list and write lock

We don't need either anymore

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:14 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 8b19d36ed91faa3e1846f38e5b0379d5b139e25f
      https://github.com/openssl/openssl/commit/8b19d36ed91faa3e1846f38e5b0379d5b139e25f
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c

  Log Message:
  -----------
  improve sharding of cache_lists

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:17 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 8f9f0d2da0669a49fcb943eb433af1368461258a
      https://github.com/openssl/openssl/commit/8f9f0d2da0669a49fcb943eb433af1368461258a
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/threads_pthread.c

  Log Message:
  -----------
  Fix persniketyness in tsan

TSAN seems to be having a problem with atomic_load_ptr and
atomic_store_ptr.  Both are, by default, __ATOMIC_RELAXED operations.

According to the tsan docs, it flags these operations as a race because,
while they are indivisible, they create no happens-before constraint,
meaning they can be reordered.

An exemplar race that is reported is:

WARNING: ThreadSanitizer: data race (pid=2139404)
  Read of size 4 at 0x723400002308 by thread T39:
    #0 EVP_MD_up_ref crypto/evp/digest.c:995 (threadstest+0x45032d) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #1 evp_md_up_ref crypto/evp/digest.c:974 (threadstest+0x450242) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #2 ossl_method_up_ref crypto/property/property.c:201 (threadstest+0x4b7a55) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #3 ossl_method_store_cache_get_locked crypto/property/property.c:941 (threadstest+0x4b9922) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #4 ossl_method_store_cache_get crypto/property/property.c:974 (threadstest+0x4b9a47) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #5 inner_evp_generic_fetch crypto/evp/evp_fetch.c:314 (threadstest+0x458186) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #6 evp_generic_fetch crypto/evp/evp_fetch.c:404 (threadstest+0x4586dc) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #7 EVP_MD_fetch crypto/evp/digest.c:985 (threadstest+0x4502d7) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #8 derive_kdk crypto/rsa/rsa_ossl.c:472 (threadstest+0x4cf738) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #9 rsa_ossl_private_decrypt crypto/rsa/rsa_ossl.c:646 (threadstest+0x4d0174) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #10 RSA_private_decrypt crypto/rsa/rsa_crpt.c:48 (threadstest+0x4c6971) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #11 rsa_decrypt providers/implementations/asymciphers/rsa_enc.c:321 (threadstest+0x51cab7) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #12 EVP_PKEY_decrypt crypto/evp/asymcipher.c:280 (threadstest+0x44a9ca) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #13 thread_shared_evp_pkey test/threadstest.c:966 (threadstest+0x404be7) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #14 thread_run test/threadstest.h:67 (threadstest+0x40132d) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)

  Previous write of size 8 at 0x723400002308 by main thread (mutexes: write M0):
    #0 memset <null> (libtsan.so.2+0x4c1eb) (BuildId: 40906101a3a1e1f1ececafafda314aee009d688a)
    #1 CRYPTO_zalloc crypto/mem.c:228 (threadstest+0x48679d) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #2 evp_md_new crypto/evp/digest.c:758 (threadstest+0x44f35e) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #3 evp_md_from_algorithm crypto/evp/digest.c:839 (threadstest+0x44f885) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #4 construct_evp_method crypto/evp/evp_fetch.c:230 (threadstest+0x457ec9) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #5 ossl_method_construct_this crypto/core_fetch.c:110 (threadstest+0x4801bf) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #6 algorithm_do_map crypto/core_algorithm.c:77 (threadstest+0x47f7a3) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #7 algorithm_do_this crypto/core_algorithm.c:122 (threadstest+0x47f987) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #8 ossl_provider_doall_activated crypto/provider_core.c:1609 (threadstest+0x49a42a) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #9 ossl_algorithm_do_all crypto/core_algorithm.c:164 (threadstest+0x47fb14) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #10 ossl_method_construct crypto/core_fetch.c:157 (threadstest+0x4803d0) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #11 inner_evp_generic_fetch crypto/evp/evp_fetch.c:333 (threadstest+0x4583a2) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #12 evp_generic_fetch crypto/evp/evp_fetch.c:404 (threadstest+0x4586dc) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #13 EVP_MD_fetch crypto/evp/digest.c:985 (threadstest+0x4502d7) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #14 derive_kdk crypto/rsa/rsa_ossl.c:472 (threadstest+0x4cf738) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #15 rsa_ossl_private_decrypt crypto/rsa/rsa_ossl.c:646 (threadstest+0x4d0174) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #16 RSA_private_decrypt crypto/rsa/rsa_crpt.c:48 (threadstest+0x4c6971) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #17 rsa_decrypt providers/implementations/asymciphers/rsa_enc.c:321 (threadstest+0x51cab7) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)
    #18 EVP_PKEY_decrypt crypto/evp/asymcipher.c:280 (threadstest+0x44a9ca) (BuildId: f34377d95e3c1d13ab9aa3204d2f1f7840d1c84a)

What tsan is saying here is that the memset in evp_md_new may get
re-ordered such that the contents of the EVP_MD may still be getting
zeroed at the time we have (a) found the EVP_MD in the method store
cache, and (b) attempted to do an up_ref on it.

This is plainly impossible, especially given that, in order to reach the
method store cache, it must be places in the method store algorithm
sparse array, which still requires the taking of the method store write
lock.  But for some reason tsan fails to see the memory fence that
creates.

It seems the simplest solution to correct this is, if we are running
under tsan, use __ATOMIC_ACQUIRE and __ATOMIC_RELEASE on
CRYPTO_atomic_[load|store]_ptr to make sure tsan sees the proper memory
ordering.

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:19 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: c1ad1773f5b76dcc836a9744677449885d060c43
      https://github.com/openssl/openssl/commit/c1ad1773f5b76dcc836a9744677449885d060c43
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M test/property_test.c

  Log Message:
  -----------
  correct property_test

The duplicate property test has to change because we now archive QUERYs
instead of removing them immediately (i.e. we don't drop the ref count
until the store is freed).

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:21 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 431f786ee63818605d7f2fb2f5e1f312081b225d
      https://github.com/openssl/openssl/commit/431f786ee63818605d7f2fb2f5e1f312081b225d
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/evp/evp_fetch.c

  Log Message:
  -----------
  Fix caching of EVP methods when NO_CACHED_FETCH is asserted

noticed during the debugging of this that, even though we may have
no-cached-fetch configured, we still put things in the method store,
which is wrong.

Don't cache things when we say we're not caching things

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:24 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 6daa86082e2391125e4721175effaf563c11f95f
      https://github.com/openssl/openssl/commit/6daa86082e2391125e4721175effaf563c11f95f
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c

  Log Message:
  -----------
  Opportunistiaclly move archived items to a separate list

if a provider is unloaded and reloaded to a context, all its algorithms
get archived, making them un-look-up-able.  While this is good, having
all those effectively dead entries in the list slows down the linear
traversal in the lookup path, so periodically, while we have the write
lock held, migrate those entries to a separate archive list so that they
don't imact the normal hot path.

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:26 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 1c81e499a898e492130835dc326f33939ebfa705
      https://github.com/openssl/openssl/commit/1c81e499a898e492130835dc326f33939ebfa705
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M crypto/property/property.c

  Log Message:
  -----------
  clean up the code a bit

Remove some vestigual code from the property cache and name things
appropriately

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:28 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 4ed89160265b2e520258235f307ab812c3cd82b4
      https://github.com/openssl/openssl/commit/4ed89160265b2e520258235f307ab812c3cd82b4
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M pkcs11-provider

  Log Message:
  -----------
  update pkcs11 submodule

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:30 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


  Commit: 5191045371b4ae1383f0ae1a0f078117e9d9b1c4
      https://github.com/openssl/openssl/commit/5191045371b4ae1383f0ae1a0f078117e9d9b1c4
  Author: Neil Horman <[email protected]>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    R test/recipes/95-test_external_pkcs11_provider_data/patches/0001-Fix-direct-ASN1_STRING-access-in-encoder.c.patch
    A test/recipes/95-test_external_pkcs11_provider_data/patches/0001-temporarily-disable-tls-test-in-pkcs11.patch
    R test/recipes/95-test_external_pkcs11_provider_data/patches/0002-Fix-direct-ASN1-STRING-access-in-objects.c.patch

  Log Message:
  -----------
  Disable tls test in pkcs11 provider

The head of the tree is broken due to a combined inability for openssl
and the provider to allow for duplication of MD contexts on an in flight
session

Reviewed-by: Saša Nedvědický <[email protected]>
Reviewed-by: Bob Beck <[email protected]>
Reviewed-by: Nikola Pajkovsky <[email protected]>
MergeDate: Tue Jun  9 18:17:32 2026
(Merged from https://github.com/openssl/openssl/pull/31018)


Compare: https://github.com/openssl/openssl/compare/15abb53c06db...5191045371b4

To unsubscribe from these emails, change your notification settings at https://github.com/openssl/openssl/settings/notifications

-- 
You received this message because you are subscribed to the Google Groups "openssl-commits" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-commits/openssl/openssl/push/refs/heads/master/15abb5-519104%40github.com.