The previous article shared WebAssembly concepts and basic usage, and provided an overview of WebAssembly through an analysis of two code examples. This article will share the practice of encrypting tools based on WebAssembly. We will use md5 and SHA1, the digest algorithms of OpenSSL, as an example to compile OpenSSL to WebAssembly on Mac.

The environment

  • Emscripten version 2.0.3
  • Openssl version 1.1.1 d
  • Browser Version 85.0.4183.121 (official) (64-bit)

An overview of the

  • Compile openSSL to WebAssembly on Mac
  • Problems encountered
  • conclusion

Build openSSL to WebAssembly on Mac

The process for compiling Openssl to WebAssembly is as follows: md5.c file – > Emscripten compile – >.wasm file – > combine with the WebAssembly JS API – > run in a browser.

1. The md5. C file

//md5.c
#include <emscripten.h>
#include <openssl/md5.h>
#include <openssl/sha.h>
#include <string.h>
#include <stdio.h>

EMSCRIPTEN_KEEPALIVE
void md5(char *str, char *result,int strlen) {
    MD5_CTX md5_ctx;
    int MD5_BYTES = 16;
    unsigned char md5sum[MD5_BYTES];
    MD5_Init(&md5_ctx);  
    MD5_Update(&md5_ctx, str,strlen);
    MD5_Final(md5sum, &md5_ctx);
    char temp[3] = {0};
    memset(result,0.sizeof(char) * 32);
    for (int i = 0; i < MD5_BYTES; i++) {
        sprintf(temp, "%02x", md5sum[i]);
        strcat(result, temp);
    }
    result[32] = '\ 0';
}

EMSCRIPTEN_KEEPALIVE
void sha1(char *str, char result[],int strlen) {
    unsigned char digest[SHA_DIGEST_LENGTH];
    SHA_CTX ctx;
    SHA1_Init(&ctx);
    SHA1_Update(&ctx, str, strlen);
    SHA1_Final(digest, &ctx);
    for (int i = 0; i < SHA_DIGEST_LENGTH; i++){
        sprintf(&result[i*2]."%02x", (unsigned int)digest[i]); }}Copy the code

The md5.c file contains two functions md5 and sha1, which will be used later to compile to WASM.

1. By default, Emscripten generates code that only calls the main() function, and any other functions are considered useless. Adding EMSCRIPTEN_KEEPALIVE before a function name prevents this from happening. You need to import the emscripten.h library to use EMSCRIPTEN_KEEPALIVE. 2. The internal implementation calls functions provided by OpenSSL, which can be directly called under simple encapsulation.Copy the code
2. Emscripten compilation
Download OpenSSL and generate the Makefile

I am using openSSL version 1.1.1d at github.com/openssl/ope… After decompressing, go to the openSSL-OpenSSL_1_1_1D folder. Compile to generate the Makefile file.

Emcmake./Configure darwin64-x86_64-cc-no-asm -- API =1.1.0Copy the code

Modify the generated Makefile. Otherwise, compilation errors may occur.

  • The CROSS_COMPILE = / usr/local/Cellar/emscripten 1.38.44 / libexec/em CROSS_COMPILE = instead
  • Change CNF_CFLAGS=-arch x86_64 to CNF_CFLAGS=
Compile the openssl
emmake make -j 12 build_generated libssl.a libcrypto.a
mkdir -p ~/resource/openssl/libs
cp -R include ~/resource/openssl/include
cp libcrypto.a libssl.a ~/Downloads/openssl/libs
Copy the code

An OpenSSL directory was created to reference the static library location in MD5.c. Libssl. a and libcrypto.

Compile wasm
emcc md5.c -I ~/resource/openssl/include -L ~/resource/openssl/libs -lcrypto -s EXTRA_EXPORTED_RUNTIME_METHODS='["cwrap", "ccall"]' -o md5.js
Copy the code

After the compilation is successful, md5.js and md5.wasm files are generated.

Since Emscripten is available in V1.38, ccall/cwrap helper functions are not exported by default, they need to be exported explicitly at compile time using -s "EXTRA_EXPORTED_RUNTIME_METHODS=['ccall', 'cwrap']".Copy the code
3. Invoke the WASM file

Wasm is invoked using the WebAssembly JS API. Md5 and SHA1 codes are put in MD5.html. They are used in the same way. Only md5-related codes are posted in this article. Code address: github.com/likai1130/s…

<div>
	<div>
        <input type="file" id="md5files" style="display: none" onchange="md5fileImport();">To calculate the md5<input type="button" id="md5fileImport" value="Import">
    </div>
</div>

<script src="Jquery - 3.5.1 track of. Min. Js"></script>
<script src="md5.js"></script>
<script type='text/javascript'>
    Module = {};
    const mallocByteBuffer = len= > {
        const ptr = Module._malloc(len)
        const heapBytes = new Uint8Array(Module.HEAPU8.buffer, ptr, len)
        return heapBytes
    }
    // Click the import button, cause files to trigger the click event, and then finish reading the file operation
    $("#md5fileImport").click(function() {$("#md5files").click();
    })
    function md5fileImport() {
        // Get the File object that reads my File
        var selectedFile = document.getElementById('md5files').files[0];
        var name = selectedFile.name; // Reads the filename of the selected file
        var size = selectedFile.size; // Reads the size of the selected file
        console.log("File name :" + name + "Size." + size);
        var reader = new FileReader(); // It does the reading.
        reader.readAsArrayBuffer(selectedFile)
        reader.onload = function() {
            // This function is called after reading, and the contents of the file are stored in result
            console.log(reader.result);
            const md5 = Module.cwrap('md5'.null['number'.'number'])		            const inBuffer = mallocByteBuffer(reader.result.byteLength)
            var ctx = new Uint8Array(reader.result)		            inBuffer.set(ctx)
            const outBuffer = mallocByteBuffer(32)
            md5(inBuffer.byteOffset,outBuffer.byteOffset,inBuffer.byteLength)
            console.log("Md5 value =".Array.from(outBuffer).map(v= > String.fromCharCode(v)).join(' ')) Module._free(inBuffer); Module._free(outBuffer); }}</script>
Copy the code
4. Run it in the browser

File a.out, md5 is a binary data: 0 d3c57ec65e81c7ff6da72472c68d95b sha1:9 ef00799a4472c71f2177fd7254faaaadedb0807

One is md5 and SHA1 calculated by the program, and the other is MD5 and SHA1 calculated by OpenSSL on the system, indicating that the practice of compiling OpenSSL for Webassembly is successful.

Second, problems encountered

The call chain is as follows:

Md5.js (glue code)<-----> md5.c <-----> OpenSSL APICopy the code
Data communication problem

In the whole process of practice, the most troublesome problem is the data communication problem. Passing complex data structures between C/C++ and JS is cumbersome and requires operating memory to implement.

  • Javascript exchanges data with C/C++

    Typescript #md5. Wasm parsing md5 function in wasm file code func $md5 (; 3) (export "md5") (param $var0 i32) (param $var1 i32) (param $var2 i32)Copy the code

Because wASM currently can only import and export C-language functional style API, and parameters only have four data types (i32, I64, F32, F64), are numbers, can be understood as a stark binary encoding, can not directly transfer complex types and data structures. So in browsers these high-level apis must be wrapped in JS, and a mechanism is needed to convert complex data structures across languages.

  • Module.buffer

    Regardless of whether the target is ASM.js or WASM, C/C++ code actually sees the memory space as the ArrayBuffer object provided by Emscripten: module. buffer. The C/C memory address corresponds to the subscript of the array Module.

function md5fileImport() {
   var selectedFile = 	document.getElementById('md5files').files[0];
   var name = selectedFile.name; // Reads the filename of the selected file
   var size = selectedFile.size; // Reads the size of the selected file
   console.log("File name :" + name + "Size." + size);
   var reader = new FileReader(); // This is the core that does the reading.
  
   reader.readAsArrayBuffer(selectedFile)
   .....
}
Copy the code

In this code we use reader.readasArrayBuffer () to read the file and return the ArrayBuffer array. However, it still cannot call C functions. It needs to create a typed array, such as Int8Array and UInt32Array, with its specific format as the view of this binary data for reading and writing operations.

All data that C/C++ code can access directly through addresses is in memory (including the runtime heap and runtime stack), and memory corresponds to the module. buffer object. The data that C/C code can access directly is actually limited to the Module.Copy the code

WebAssembly’s memory is also an ArrayBuffer, and Emscripten’s Module package provides various views such as module. HEAP8, module. HEAPU8, etc. The appended drawings:

  • Access C/C++ memory in JavaScript

The calculation of MD5 / SHA1 requires javascript to input a large amount of data into the C/C++ environment, while C/C++ cannot predict the size of the data block. In this case, you can allocate memory in javascript and load the data, then pass in the data pointer and call THE C function for processing.

The core reason this works is that Emscripten exports the malloc()/free() of C.Copy the code

I declare the method that allocates memory space as a public method.

Module = {};
const mallocByteBuffer = len= > {
    const ptr = Module._malloc(len)
    const heapBytes = new Uint8Array(Module.HEAPU8.buffer, ptr, len)
    return heapBytes
}

function md5fileImport() {
    // Get the File object that reads my File
    var selectedFile = document.getElementById('md5files').files[0]; .var reader = new FileReader(); // This is the core that does the reading.
    reader.readAsArrayBuffer(selectedFile)
    reader.onload = function() {
        // This function is called after reading, and the contents of the file are stored in result
        const md5 = Module.cwrap('md5'.null['number'.'number'])
        const inBuffer = mallocByteBuffer(reader.result.byteLength)
        var ctx = new Uint8Array(reader.result)
        inBuffer.set(ctx)
        const outBuffer = mallocByteBuffer(32)
        md5(inBuffer.byteOffset,outBuffer.byteOffset,inBuffer.byteLength)

        console.log("Md5 value =".Array.from(outBuffer).map(v= > String.fromCharCode(v)).join(' ')) Module._free(inBuffer); Module._free(outBuffer); }}Copy the code
Tips: C/C++ has no gc mechanism for memory. When the malloc() function is used in JavaScript, you need to use free() to free the memory.Copy the code

In addition, Emscripten provides a series of helper functions such as AsciiToString()/stringToAscii()/UTF8ArrayToString()/stringToUTF8Array() to handle the conversion of strings of various formats into various stored objects. Please refer to the glue code for more details.

Third, summary

Wasm openSSL:

The technical problems encountered in this practice are data communication problems, and another is the problem of thinking. I always thought that openSSL could be used by compiling as a whole into.wASM file, but it turns out that glue code is also needed to be used in the Web. The wASM file is essentially a binary file, so is there a tool that can run it directly? Take a look at WebAssembly package management tools in the next article.

The resources

  • Example code address:

    • Github.com/likai1130/s…

  • WebAssembly API (In Chinese, solve the problem of logical JS calling WASM) :

    • Developer.mozilla.org/zh-CN/docs/…

  • Emscripten syntax learning (solve C language call JS syntax problems) :

    • Emscripten.org/docs/api_re… _

  • Openssl compilation reference

    • Github.com/wapm-packag…

Netwarps is composed of a senior cloud computing and distributed technology development team in China, which has rich experience in the financial, power, communication and Internet industries. Netwarps has set up research and development centers in Shenzhen and Beijing, with a team size of 30+, most of which are technical personnel with more than 10 years of development experience, respectively from the Internet, finance, cloud computing, blockchain and scientific research institutions and other professional fields. Netwarps focuses on the development and application of secure storage technology products, the main products are decentralized file system (DFS), decentralized computing platform (DCP), is committed to providing distributed storage and distributed computing platform based on decentralized network technology, with high availability, low power consumption and low network technical characteristics. Applicable to scenarios such as the Internet of Things and industrial Internet. Public account: Netwarps