OpenSSL Blog

Engine Building Lesson 2: An Example MD5 Engine

Posted by Richard Levitte ,

Coming back after a month and two weeks, it’s time to resume with the next engine lesson, this time building an engine implementing a digest.

It doesn’t matter much what digest algorithm we choose. Being lazy, I’ve chosen one with a well defined reference implementation, MD5 (reference implementation is found in RFC 1321)

In this example, I’ve extracted the three files global.h, md5.h and md5c.c into rfc1321/. According to the license, I need to identify them as “RSA Data Security, Inc. MD5 Message-Digest Algorithm”, hereby done. [^1]

From an engine point of view, there are three things that need to be done to implement a digest:

  1. Create an OpenSSL digest method structure with pointers to the functions that will be OpenSSL’s interface to the reference implementation.
  2. Create the interface functions.
  3. Create a digest selector function.

Let’s begin with the structure by example:

``` C The digest method #include <openssl/evp.h> #include “rfc1321/global.h” #include “rfc1321/md5.h”

static const EVP_MD digest_md5 = { NID_md5, /* The name ID for MD5 / 0, / IGNORED: MD5 with private key encryption NID / 16, / Size of MD5 result, in bytes / 0, / Flags / md5_init, / digest init / md5_update, / digest update / md5_final, / digest final / NULL, / digest copy / NULL, / digest cleanup / EVP_PKEY_NULL_method, / IGNORED: pkey methods / 64, / Internal blocksize, see rfc1321/md5.h / sizeof(MD5_CTX), NULL / IGNORED: control function */ };


*NOTE: the EVP_MD will become opaque in future OpenSSL versions,
starting with version 1.1, and the structure init above will have to
be replaced with a number of function calls to initialise the
different parts of the structure.  More on that later.*

A slightly complicating factor with this structure is that it also
involves private/public key ciphers.  I'm ignoring the associated
fields in this lesson, along with the control function (all marked
"IGNORED:") and will get back to them in a future lesson where I'll
get into public/private key algo implementations.

The numerical ID is an OpenSSL number that identifies the digest
algorithm, and is an index to the OID database.  
The flags are not really used for pure digests and can be left zero.  
The copy and cleanup functions are hooks to be used if there's more to
copying and cleanup `EVP_MD_CTX` for our implementation than OpenSSL
can handle on its own.  The `MD5_CTX` structure in the reference MD5
implementation has nothing magic about it, and there is therefore no
need to use the `copy` and `cleanup`hooks.  
The internal block size is present for optimal use of the algorithm.  

On to implementing the interface functions `md5_init`, `md5_update`
and `md5_final`:

``` C The interface functions
static int md5_init(EVP_MD_CTX *ctx)
{
  MD5Init(ctx->md_data);
  return 1;
}

static int md5_update(EVP_MD_CTX *ctx, const void *data, size_t count)
{
  MD5Update(ctx->md_data, data, count);
  return 1;
}

static int md5_final(EVP_MD_CTX *ctx, unsigned char *md)
{
  MD5Final(md, ctx->md_data);
  return 1;
}

With the reference implementation, things are very straight forward, no errors can happen, all we do is pass data back and forth. A note: ctx->md_data is preallocated by OpenSSL using the size given in the EVP_MD structure. In a more complex implementation, these functions are expected to return 0 on error, 1 on success.

The selector function deserves some special attention, as it really performs two separate functions. The prototype is as follows:

``` C Digest selector prototype int digest_selector(ENGINE *e, const EVP_MD **digest, const int **nids, int nid);


OpenSSL calls it in the following ways:

1. with `digest` being `NULL`.  In this case, `*nids` is expected to
   be assigned a zero-terminated array of NIDs and the call returns
   with the number of available NIDs.  OpenSSL uses this to determine
   what digests are supported by this engine.
2. with `digest` being non-`NULL`.  In this case, `*digest` is
   expected to be assigned the pointer to the `EVP_MD` structure
   corresponding to the NID given by `nid`.  The call returns with 1
   if the request NID was one supported by this engine, otherwise 0.

The implementation would be this for our little engine:

``` C Digest selector
static int digest_nids[] = { NID_md5, 0 };
static int digests(ENGINE *e, const EVP_MD **digest,
                   const int **nids, int nid)
{
  int ok = 1;
  if (!digest) {
    /* We are returning a list of supported nids */
    *nids = digest_nids;
    return (sizeof(digest_nids) - 1) / sizeof(digest_nids[0]);
  }

  /* We are being asked for a specific digest */
  switch (nid) {
  case NID_md5:
    *digest = &digest_md5;
    break;
  default:
    ok = 0;
    *digest = NULL;
    break;
  }
  return ok;
}

What remains to do is the following call as part of setting up this engine:

``` C Setting up if (!ENGINE_set_digests(e, digests)) { printf(“ENGINE_set_name failed\n”); return 0; }


Combine with the code from last lesson results in this:

<div class='bogus-wrapper'><notextile><figure class='code'><figcaption><span> (md5-engine.c)</span> <a href='/blog/downloads/code/engine-lesson-2/md5-engine.c'>download</a></figcaption>
<div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
<span class='line-number'>22</span>
<span class='line-number'>23</span>
<span class='line-number'>24</span>
<span class='line-number'>25</span>
<span class='line-number'>26</span>
<span class='line-number'>27</span>
<span class='line-number'>28</span>
<span class='line-number'>29</span>
<span class='line-number'>30</span>
<span class='line-number'>31</span>
<span class='line-number'>32</span>
<span class='line-number'>33</span>
<span class='line-number'>34</span>
<span class='line-number'>35</span>
<span class='line-number'>36</span>
<span class='line-number'>37</span>
<span class='line-number'>38</span>
<span class='line-number'>39</span>
<span class='line-number'>40</span>
<span class='line-number'>41</span>
<span class='line-number'>42</span>
<span class='line-number'>43</span>
<span class='line-number'>44</span>
<span class='line-number'>45</span>
<span class='line-number'>46</span>
<span class='line-number'>47</span>
<span class='line-number'>48</span>
<span class='line-number'>49</span>
<span class='line-number'>50</span>
<span class='line-number'>51</span>
<span class='line-number'>52</span>
<span class='line-number'>53</span>
<span class='line-number'>54</span>
<span class='line-number'>55</span>
<span class='line-number'>56</span>
<span class='line-number'>57</span>
<span class='line-number'>58</span>
<span class='line-number'>59</span>
<span class='line-number'>60</span>
<span class='line-number'>61</span>
<span class='line-number'>62</span>
<span class='line-number'>63</span>
<span class='line-number'>64</span>
<span class='line-number'>65</span>
<span class='line-number'>66</span>
<span class='line-number'>67</span>
<span class='line-number'>68</span>
<span class='line-number'>69</span>
<span class='line-number'>70</span>
<span class='line-number'>71</span>
<span class='line-number'>72</span>
<span class='line-number'>73</span>
<span class='line-number'>74</span>
<span class='line-number'>75</span>
<span class='line-number'>76</span>
<span class='line-number'>77</span>
<span class='line-number'>78</span>
<span class='line-number'>79</span>
<span class='line-number'>80</span>
<span class='line-number'>81</span>
<span class='line-number'>82</span>
<span class='line-number'>83</span>
<span class='line-number'>84</span>
<span class='line-number'>85</span>
<span class='line-number'>86</span>
<span class='line-number'>87</span>
<span class='line-number'>88</span>
<span class='line-number'>89</span>
<span class='line-number'>90</span>
<span class='line-number'>91</span>
<span class='line-number'>92</span>
<span class='line-number'>93</span>
</pre></td><td class='code'><pre><code class='c'><span class='line'><span class="cp">#include &lt;stdio.h&gt;</span>
</span><span class='line'><span class="cp">#include &lt;string.h&gt;</span>
</span><span class='line'>
</span><span class='line'><span class="cp">#include &lt;openssl/engine.h&gt;</span>
</span><span class='line'>
</span><span class='line'><span class="cp">#include &lt;openssl/evp.h&gt;</span>
</span><span class='line'><span class="cp">#include &quot;rfc1321/global.h&quot;</span>
</span><span class='line'><span class="cp">#include &quot;rfc1321/md5.h&quot;</span>
</span><span class='line'>
</span><span class='line'><span class="k">static</span> <span class="kt">int</span> <span class="nf">md5_init</span><span class="p">(</span><span class="n">EVP_MD_CTX</span> <span class="o">*</span><span class="n">ctx</span><span class="p">)</span>
</span><span class='line'><span class="p">{</span>
</span><span class='line'>  <span class="n">MD5Init</span><span class="p">(</span><span class="n">ctx</span><span class="o">-&gt;</span><span class="n">md_data</span><span class="p">);</span>
</span><span class='line'>  <span class="k">return</span> <span class="mi">1</span><span class="p">;</span>
</span><span class='line'><span class="p">}</span>
</span><span class='line'>
</span><span class='line'><span class="k">static</span> <span class="kt">int</span> <span class="nf">md5_update</span><span class="p">(</span><span class="n">EVP_MD_CTX</span> <span class="o">*</span><span class="n">ctx</span><span class="p">,</span> <span class="k">const</span> <span class="kt">void</span> <span class="o">*</span><span class="n">data</span><span class="p">,</span> <span class="kt">size_t</span> <span class="n">count</span><span class="p">)</span>
</span><span class='line'><span class="p">{</span>
</span><span class='line'>  <span class="n">MD5Update</span><span class="p">(</span><span class="n">ctx</span><span class="o">-&gt;</span><span class="n">md_data</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="n">count</span><span class="p">);</span>
</span><span class='line'>  <span class="k">return</span> <span class="mi">1</span><span class="p">;</span>
</span><span class='line'><span class="p">}</span>
</span><span class='line'>
</span><span class='line'><span class="k">static</span> <span class="kt">int</span> <span class="nf">md5_final</span><span class="p">(</span><span class="n">EVP_MD_CTX</span> <span class="o">*</span><span class="n">ctx</span><span class="p">,</span> <span class="kt">unsigned</span> <span class="kt">char</span> <span class="o">*</span><span class="n">md</span><span class="p">)</span>
</span><span class='line'><span class="p">{</span>
</span><span class='line'>  <span class="n">MD5Final</span><span class="p">(</span><span class="n">md</span><span class="p">,</span> <span class="n">ctx</span><span class="o">-&gt;</span><span class="n">md_data</span><span class="p">);</span>
</span><span class='line'>  <span class="k">return</span> <span class="mi">1</span><span class="p">;</span>
</span><span class='line'><span class="p">}</span>
</span><span class='line'>
</span><span class='line'><span class="k">static</span> <span class="k">const</span> <span class="n">EVP_MD</span> <span class="n">digest_md5</span> <span class="o">=</span> <span class="p">{</span>
</span><span class='line'>  <span class="n">NID_md5</span><span class="p">,</span>                      <span class="cm">/* The name ID for MD5 */</span>
</span><span class='line'>  <span class="mi">0</span><span class="p">,</span>                            <span class="cm">/* IGNORED: MD5 with private key encryption NID */</span>
</span><span class='line'>  <span class="mi">16</span><span class="p">,</span>                           <span class="cm">/* Size of MD5 result, in bytes */</span>
</span><span class='line'>  <span class="mi">0</span><span class="p">,</span>                            <span class="cm">/* Flags */</span>
</span><span class='line'>  <span class="n">md5_init</span><span class="p">,</span>                     <span class="cm">/* digest init */</span>
</span><span class='line'>  <span class="n">md5_update</span><span class="p">,</span>                   <span class="cm">/* digest update */</span>
</span><span class='line'>  <span class="n">md5_final</span><span class="p">,</span>                    <span class="cm">/* digest final */</span>
</span><span class='line'>  <span class="nb">NULL</span><span class="p">,</span>                         <span class="cm">/* digest copy */</span>
</span><span class='line'>  <span class="nb">NULL</span><span class="p">,</span>                         <span class="cm">/* digest cleanup */</span>
</span><span class='line'>  <span class="n">EVP_PKEY_NULL_method</span><span class="p">,</span>         <span class="cm">/* IGNORED: pkey methods */</span>
</span><span class='line'>  <span class="mi">64</span><span class="p">,</span>                           <span class="cm">/* Internal blocksize, see rfc1321/md5.h */</span>
</span><span class='line'>  <span class="k">sizeof</span><span class="p">(</span><span class="n">MD5_CTX</span><span class="p">),</span>
</span><span class='line'>  <span class="nb">NULL</span>                          <span class="cm">/* IGNORED: control function */</span>
</span><span class='line'><span class="p">};</span>
</span><span class='line'>
</span><span class='line'><span class="k">static</span> <span class="kt">int</span> <span class="n">digest_nids</span><span class="p">[]</span> <span class="o">=</span> <span class="p">{</span> <span class="n">NID_md5</span><span class="p">,</span> <span class="mi">0</span> <span class="p">};</span>
</span><span class='line'><span class="k">static</span> <span class="kt">int</span> <span class="nf">digests</span><span class="p">(</span><span class="n">ENGINE</span> <span class="o">*</span><span class="n">e</span><span class="p">,</span> <span class="k">const</span> <span class="n">EVP_MD</span> <span class="o">**</span><span class="n">digest</span><span class="p">,</span>
</span><span class='line'>                   <span class="k">const</span> <span class="kt">int</span> <span class="o">**</span><span class="n">nids</span><span class="p">,</span> <span class="kt">int</span> <span class="n">nid</span><span class="p">)</span>
</span><span class='line'><span class="p">{</span>
</span><span class='line'>  <span class="kt">int</span> <span class="n">ok</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span>
</span><span class='line'>  <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">digest</span><span class="p">)</span> <span class="p">{</span>
</span><span class='line'>    <span class="cm">/* We are returning a list of supported nids */</span>
</span><span class='line'>    <span class="o">*</span><span class="n">nids</span> <span class="o">=</span> <span class="n">digest_nids</span><span class="p">;</span>
</span><span class='line'>    <span class="k">return</span> <span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="n">digest_nids</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="o">/</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">digest_nids</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
</span><span class='line'>  <span class="p">}</span>
</span><span class='line'>
</span><span class='line'>  <span class="cm">/* We are being asked for a specific digest */</span>
</span><span class='line'>  <span class="k">switch</span> <span class="p">(</span><span class="n">nid</span><span class="p">)</span> <span class="p">{</span>
</span><span class='line'>  <span class="k">case</span> <span class="nl">NID_md5</span><span class="p">:</span>
</span><span class='line'>    <span class="o">*</span><span class="n">digest</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">digest_md5</span><span class="p">;</span>
</span><span class='line'>    <span class="k">break</span><span class="p">;</span>
</span><span class='line'>  <span class="k">default</span><span class="o">:</span>
</span><span class='line'>    <span class="n">ok</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
</span><span class='line'>    <span class="o">*</span><span class="n">digest</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
</span><span class='line'>    <span class="k">break</span><span class="p">;</span>
</span><span class='line'>  <span class="p">}</span>
</span><span class='line'>  <span class="k">return</span> <span class="n">ok</span><span class="p">;</span>
</span><span class='line'><span class="p">}</span>
</span><span class='line'>
</span><span class='line'><span class="k">static</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="n">engine_id</span> <span class="o">=</span> <span class="s">&quot;MD5&quot;</span><span class="p">;</span>
</span><span class='line'><span class="k">static</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="n">engine_name</span> <span class="o">=</span> <span class="s">&quot;A simple md5 engine for demonstration purposes&quot;</span><span class="p">;</span>
</span><span class='line'><span class="k">static</span> <span class="kt">int</span> <span class="nf">bind</span><span class="p">(</span><span class="n">ENGINE</span> <span class="o">*</span><span class="n">e</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="n">id</span><span class="p">)</span>
</span><span class='line'><span class="p">{</span>
</span><span class='line'>  <span class="kt">int</span> <span class="n">ret</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
</span><span class='line'>
</span><span class='line'>  <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">ENGINE_set_id</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">engine_id</span><span class="p">))</span> <span class="p">{</span>
</span><span class='line'>    <span class="n">fprintf</span><span class="p">(</span><span class="n">stderr</span><span class="p">,</span> <span class="s">&quot;ENGINE_set_id failed</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">);</span>
</span><span class='line'>    <span class="k">goto</span> <span class="n">end</span><span class="p">;</span>
</span><span class='line'>  <span class="p">}</span>
</span><span class='line'>  <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">ENGINE_set_name</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">engine_name</span><span class="p">))</span> <span class="p">{</span>
</span><span class='line'>    <span class="n">printf</span><span class="p">(</span><span class="s">&quot;ENGINE_set_name failed</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">);</span>
</span><span class='line'>    <span class="k">goto</span> <span class="n">end</span><span class="p">;</span>
</span><span class='line'>  <span class="p">}</span>
</span><span class='line'>  <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">ENGINE_set_digests</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">digests</span><span class="p">))</span> <span class="p">{</span>
</span><span class='line'>    <span class="n">printf</span><span class="p">(</span><span class="s">&quot;ENGINE_set_name failed</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">);</span>
</span><span class='line'>    <span class="k">goto</span> <span class="n">end</span><span class="p">;</span>
</span><span class='line'>  <span class="p">}</span>
</span><span class='line'>
</span><span class='line'>  <span class="n">ret</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span>
</span><span class='line'> <span class="nl">end</span><span class="p">:</span>
</span><span class='line'>  <span class="k">return</span> <span class="n">ret</span><span class="p">;</span>
</span><span class='line'><span class="p">}</span>
</span><span class='line'>
</span><span class='line'><span class="n">IMPLEMENT_DYNAMIC_BIND_FN</span><span class="p">(</span><span class="n">bind</span><span class="p">)</span>
</span><span class='line'><span class="n">IMPLEMENT_DYNAMIC_CHECK_FN</span><span class="p">()</span>
</span></code></pre></td></tr></table></div></figure></notextile></div>

Building is just as easy as last time:

``` sh
$ gcc -fPIC -o rfc1321/md5c.o -c rfc1321/md5c.c
$ gcc -fPIC -o md5-engine.o -c md5-engine.c
$ gcc -shared -o md5-engine.so -lcrypto md5-engine.o rfc1321/md5c.o

Let’s start with checking that OpenSSL loads it correctly. Remember what Lesson 1 taught us, that we can pass an absolute path for an engine? Let’s use that again.

$ openssl engine -t -c `pwd`/md5-engine.so
(/home/levitte/tmp/md5-engine/md5-engine.so) A simple md5 engine for demonstration purposes
Loaded: (MD5) A simple md5 engine for demonstration purposes
 [MD5]
     [ available ]

Finally, we can compare the result of a run with OpenSSL’s own implementation.

$ echo whatever | openssl dgst -engine `pwd`/md5-engine.so -md5
engine "MD5" set.
(stdin)= d8d77109f4a24efc3bd53d7cabb7ee35
$ echo whatever | openssl dgst -md5
(stdin)= d8d77109f4a24efc3bd53d7cabb7ee35

And there we have it, a functioning implementation of an MD5 engine.

Just as in the previous lesson, I made a slightly more elaborate variant of this engine, again using the GNU auto* tools. This variant also has a different way of initialising digest_md5 by taking a copy of the OpenSSL builtin structure and just replacing the appropriate fields. This allows it to be used for operations involving public/private keys as well.

Next lesson will be about adding a symmetric cipher implementation.

[^1] I needed to make a small change in rfc1321/global.h

``` diff rfc1321/global.h /* UINT2 defines a two byte word */ -typedef unsigned short int UINT2; +typedef uint16_t UINT2;

/* UINT4 defines a four byte word */ -typedef unsigned long int UINT4; +typedef uint32_t UINT4;

```

The reason is that I’m running on a 64-bit machine, so UINT4 ended up not being 32 bits, and the reference MD5 gave me faulty results.

Comments