This post talks about OpenSSL and threads. In particular, using OpenSSL in multi-threaded applications. It traces through the history, explains what was changed for the 1.1.0 release, and will hopefully provide some guidance to developers.
While none of the behaviors have really changed, and therefore none of this should be new information, the documentation has not been as clear as it could, or should, be. Therefore, some readers might be surprised by what’s in this post.
In short, OpenSSL has always, and only, supported the concept of locking an object and sometimes it locks its internal objects. Read on for more details.
In OpenSSL 1.0.2 (and earlier), applications had to provide their own integration with locking and threads, as documented in the threads.pod file. This page starts with the following unfortunate text:
OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set, …
The problem is that the word safely was never actually defined. This led
many developers to think that OpenSSL’s objects could safely be used by
multiple threads at the same time. For example, using a single
object in two threads, one each for reading and writing.
The file crypto/th-lock.c in 1.0.2 had a “sample” implementation of the lock callbacks, for a number of platforms. The intent was that applications take that file, make it work on their platform and include it in their application. Many apparently managed to do that, although I wryly point out this comment from the Git history in early 2000:
At least, this compiles nicely on Linux using PTHREADS. I’ve done no other tests so far.
With 1.1.0, the major change was to move this from run-time to compile-time.
If threads support is enabled (still the same misleading name), then the
native threads package is used. On Windows, this means using the
“critical section” API for locks, as can be seen in
On all other platforms, the pthreads API is used,
In cases where the native threads facility isn’t known, or if explicitly
no-threads then dummy functions are used,
It is worth looking at at least one of these files. They are all very small, 120 to 170 lines, and can provide an overview of what OpenSSL actually needs and provides. The documentation is a bit better, but could still give the impression that concurrent use of objects in multiple threads is supported. We should fix that, soon.
So what is supported? One way to find out is to do
git grep CRYPTO_THREAD_.*lock
If you do this, you’ll see that there is some locking around internal lookup tables for engines, error codes, and some RSA and X.509 lookup operations, and also about SSL sessions. Almost none of this behavior is documented, and it’s risky to plan on future behavior from just looking at the current code, but in general you can think that either tables of things (errors, sessions) or low-level internal details (RSA blinding, RSA Montgomeray optimizations) is safe. And most of the time, you probably won’t even think of that at all, anyway.
The other place where threads support comes in, isn’t necessarily about
threads. It’s about maintaining reference counts on objects, and ensuring
that an object is not free’d until the count hits zero. This is implemented
using per-object locks, and the
CRYPTO_atomic_add API. You can see some
relatively simple examples of the
up_ref API in
which also shows one of the common use-cases: adding an object to a
container, but still “needing” that object for later, after the container
is removed. Most datatypes have their own
Internally, OpenSSL is moving to do more things using native facilities. For example, the auto-init is done using “run-once” facilities, and the error state is maintained using thread-local storage. I’m not going to provide pointers to the source, because I don’t want to encourage people to rely on undocumented behavior. But, what you can rely on is that if you natively use your system threads facility, OpenSSL will work seamlessly with it. But you still can’t concurrently use most objects in multiple threads.