Update on rlbox wasm sandbox crashes

Hi guys.

So those who are familiar with this will know I dug into what was 
crashing Firefox on startup. Which was expat crashing inside a wasm 
compiled sandbox. An issue known about for years and worked around by 
disabling the sandboxing. An existing fix for s390x. Which fixed the 
Firefox build on Debian ppc/64 so it worked again. (Thanks Adrian.)

I pretty much threw myself into the deep end as I wasn't familiar with 
RLBox nor how it was integrated into Firefox but did know about wasm. So 
I had to learn what was crashing in GDB and undo my confusion as to what 
was causing it and why sources for symbols were missing. I've spent time 
since investigating it and learning about all the processes involving 
this sandboxing.

(TL;DR, or afraid to but still reading this line, it looks like wasm2c 
did it.)


There's three main components involved:

* rbox - The core sandbox.

* rlbox_wasm2c_sandbox - Sand boxing using wasm as a middle man.

* rlbox-book - Example code and tutorials for porting and testing 
sandboxing.

These come in different levels with their own sources. The first, rlbox 
is small. This easily compiled on ppc64 and passed all tests. The logic 
appears fine. Next, rlbox_wasm2c_sandbox, is more complicated. This is 
designed to compile the whole toolchain before you can even use it to 
compile and test any code for sandboxing. The last, rlbox-book, has code 
for a tutorial and examples for basic "no-op" false sandbox source 
testing and a real "wasm" sandbox source example. This relies on 
rlbox_wasm2c_sandbox but, the sources are obsolete or mismatched, as the 
example code expects sources generated from the build that don't exist. 
So you cannot build it all without the examples still breaking. Somewhat 
time wasting if you need to hunt down what it's looking for. Which I did.

RLBox and WASM have been available for some time now days so there 
should be no need to build the entire toolchain just to do some simple 
test compiles. And there are prebult packages. In particular there is 
wabt which provides wasm2c, wasi-sdk which provides wasm build tools, 
and clang actually can compile source to wasm code itself. Of these 
Debian ppc64 doesn't have wasi-sdk. But does have wabt and a wasi-libc. 
And the ppc64 clang build does support wasm. As expected there are no 
ppc/64 releases of wasi-sdk. However, tools are available here in ppc64 
for what building an rlbox wasm example needs, if you find where to 
look. :-)

The whole RLBox and WASM integration is one big convoluted monster. It 
will take a nice small source code and bloat it up into a big 
complicated mess. All for the purpose of a software VM to isolate code 
and protect the host. To simplify the process, a source needs to be 
compiled into wasm, then wasm code reverse engineered (or back compiled) 
into C, then finally wrapped in the rlbox API and compiled as C/C++ into 
the end binary. Sounds simple! I wanted to compile this on ppc64 since 
this process breaks binaries on ppc64 and I had a devil of a time doing 
so. But, I also wanted to avoid building the whole toolchain, since the 
tools do exist. First, I needed a generic clang to work for me, since 
the wasm clang build is custom.  A search revealed clang needed 
--target=wasm32-wasi so was a simple fix. Next I needed wasm2c to work 
normally. Unfortunately, the documentation is a bit sparse. For example, 
wasm2c from wabt kept generating source that caused clang (and gcc for 
that matter) to spew out errors everywhere. All the labels had been 
surrounded by 0x24 and 0x2e (dollar and dot if you know your ASCII hex) 
and it kept wrecking the whole build, which looked strange, as the 
sources it output were isolated to one C and header file. Turns out, 
with the wasm2c I was using, I needed to pass --no-debug-names. Looks 
simple, but this took me ages to find out. It also didn't help that some 
options give errors about being unimplemented as if was2c was still an 
unfinished beta. I looked on the net for info on wasm2c putting 0x24 and 
0x2e in labels and got no results at all! In the end I just added that 
debug option as a guess since it looked related and that was it. Well 
they could have mentioned it! How about the manual actually mentioning 
that by default this wasm2c will put 0x24 and 0x2e in labels? Or wreck 
it so it won't compile? :-facepalm.

After getting over all that I could finally do some real testing in gdb. 
At this point, rlbox and an rlbox wasm noop-hello-example from 
rlbox-book compiled and worked fine. Now I had the wasm-hello-example 
from rlbox-book compiled which was the real challenge. I ran the test 
binary and it crashed. Actually, unlike expat in Firefox, it didn't 
actually crash but aborted itself. Either way this was good as I now 
knew what code was broken and what built the code. I've now ran the code 
through gdb and also compared with running on x64. A few things I have 
ascertained:

* Code wasm2c generates is big endian aware. Despite lacking tier 1 
support the host code generated checks for big endian and runs alternate 
code to fix bytes. So regardless of host endian the output of wasm2c 
code is designed to and should work on big endian/ppc native. This is 
good as not only does the upstream code take big endian into account it 
is designed to run correctly on a big endian host. I checked and both 
x64 and ppc64 builds of wasm2c produced big endian aware sources.

* Related to my first point there is a quirky macro that fixes endian 
when ran from big. It actually will slow down the code so although it's 
designed to fix endian it should be fixed to be optimised. It's a load 
data routine that normally just does a mem copy. On big endian it does a 
mem copy and then, then reverses the code in place, almost neatly by 
reading a byte at one end and swapping with a byte at the opposite end 
with a half length array loop. Although it looks novel I wonder why they 
just didn't stick a reverse mem copy in? But that's just how I would do 
it. Any built in mem copy is rendered useless by the whole operation.

I have found some issues. The example code I compiled from the 
wasm-hello-example breaks on big endian. I don't mean to repeat the 
obvious, but relating to my first point, any code compiled on big endian 
will not include the macro. It relies on a WABT_BIG_ENDIAN define being 
set to 1 when the final wasm2c C code is compiled. On my system this was 
not set although I expected the upper headers included would test and 
set it. This could relate to my test build but is one to watch out for 
upstream. It is tested with from a CMake list in rlbox_wasm2c_sandbox. 
But the example just has a simple Makefile with no checks for configuring.

Next is loading data. The wasm2c code has data built in. There are both 
data tables and some function tables all built from byte arrays. Plus 
what looks like hex encoded ASCII string tables. Looking at the data it 
appears there are scalars embedded in little endian order. This would be 
consistent with wasm being little endian but will obviously cause 
trouble in big endian. Although the code should be aware of any byte 
positions. I suspect this could be what is crashing expat where just one 
offset among a few parameters to a function was byte swapped. I don't 
know how clean the code is, since it doesn't look clean from the onset, 
but if data load routines are doing tricks, such as reading the LSB or 
reading one byte from a bigger scalar it will badly break. There could 
be big trouble in little endian here! :-D

The macro has more issues. It is designed for scalars but accepts any 
value. I don't know if they intended it to be like that to me it looks 
like a bug. The routine is designed to load data but is abused/confused 
by a macro that reverses the loaded data. However, since it wasn't 
enabled in my build, this wasn't what was aborting the code. For fun I 
enabled the WABT_BIG_ENDIAN macro on x64. Which I'm also doing 
comparative testing with. It broke it and caused the binary to abort. 
Lol. But, on ppc64, enabling WABT_BIG_ENDIAN almost made it worse. The 
code doesn't crash or abort but ends up in an endless loop! :-?

After examining more code I can see this getting more complex. Due to it 
coding these tables in. This will result in mixed endian data throughout 
the code. There are fields code assigns and reads back in native endian 
from source. There's scalars read from data. But since they are both set 
and read in the same data table it's hard to see what endian will be 
what! Only wasm2c would know what the offset is. If it knows what an 
offset does in the first place. Then the code itself appears to be 
simulating a CPU. It assumes all these local variables it spends time 
reading and writing to, doing conditionals on with gotos, passing values 
around and doing what looks like the least optimised way of an 
operations. Doing operations atomically. It actually looks like a C 
version of RISC ASM. :-)

This is running deep. Of course, this is due to taking perfectly 
portable code, compiling it into little endian machine code, then 
reverse engineering that into an analogous C source. The result has 
ended up with introducing a false dependence of little endian. Be good 
if wasm2c could be told to generate big endian or portable code. But 
since the world is now running on a little endian web, this problem 
won't go away now.

After gathering the evidence so far I can only conclude the entire 
process going on here has made an already convoluted process more 
complicated. That becomes more worse on an endian it isn't designed for. 
I've already dealt with code like this before that, based on 
examination, was generated by reverse engineering x86 code into a non 
portable convoluted mess of C. It wasn't pretty. Yet I manged to find 
every byte out of place and fix to be portable. So now, I can see that 
disabling the sandboxing is for now likely the best fix, because if it 
worked the code would be even slower. But, the compromise is here is 
more security, so getting it working down the track is a good idea 
regardless and even for an "exotic endian" platform. In the mean time, 
I'll continue digging, now it's becoming clearer where to point the 
finger. And what I can use as an MVE.


Links:

https://github.com/PLSysSec/rlbox

https://github.com/PLSysSec/rlbox_wasm2c_sandbox

https://github.com/PLSysSec/rlbox-book


The wasm2c [excuse for a] man page example:

https://manpages.debian.org/bookworm/wabt/wasm2c.1.en.html


That quirky macro:

static inline void load_data(void* dest, const void* src, size_t n) {
   if (!n) {
     return;
   }
   size_t i = 0;
   u8* dest_chars = dest;
   wasm_rt_memcpy(dest, src, n);
   for (i = 0; i < (n >> 1); i++) {
     u8 cursor = dest_chars[i];
     dest_chars[i] = dest_chars[n - i - 1];
     dest_chars[n - i - 1] = cursor;
   }
}