Re: REST impact paper

> On Sep 8, 2017, at 10:10 PM, Sean Conner <[email protected]> wrote:
> 
>  I was playing around with the x86 SIMD instructions.  I have code I wrote
> in the early 90s (in x86 assembly) that matched (I thought) quote well to a
> vectorized version.  The core was an interative application of these two
> functions:
> 
> 	x1 = (A * y + B) * x * (1 - x)
> 	y1 = (C * x + D) * y * (1 - y)
> 
>  I wrote a version using the vector instructions of the x86.  I could
> easily beat clang (which uses LLVM), but try as I might, I could not beat
> GCC.  


These days GCC and clang are pretty similar at the macro level, each with things they do a little better than the other. For complex logic you can still beat either of them by rolling your own assembly or intrinsics but the skill threshold required to do that consistently is very high. It is easy to verify if the compiler is as clever as one assumes it is and I am often pleasantly surprised.

Even in C++, there are times where the compiler (surprisingly) misses obvious and material optimizations. Simply writing the same logic a slightly different way is often sufficient for the compiler to get it right. 

I tend to prefer LLVM for two reasons. First, there is a lot of dynamic/JIT compilation in applications these days, so might as well use the same tool chain for JIT as AOT. Second, some of the people on my team are LLVM contributors, so excellent support. :-)  


>  So what magic things were done to make JSON parsing faster with
> proprietary code than with open source code?  Or alternatively, what does
> open source do wrong?  I'm asking for specific examples and not a
> generalized "it sucks" argument.


It is just a difference in priorities. Operational cost is not even in the top three priorities for open source projects when building software, so when there is a conflict between priorities it loses out. These journeys always start out with using the high-performance open source library du jour but end up with proprietary implementations.

For JSON specifically, I needed to parse trillions of documents as quickly as possible, so performance was rarely satisfactory in any case. Because “open source”, we could inspect the architecture and implementation for ways to improve performance. Being software performance junkies, we find many opportunities to materially improve throughput, some algorithmic and some architectural. 

Open source projects will often only accept patches for a subset of algorithmic optimizations, rejecting some because they violate some other aesthetic or goal of the project (e.g. it is an anti-optimization for some obscure or archaic hardware). Architectural optimizations are effectively a non-starter for an existing open source project even when it would be hugely beneficial (e.g. PostgreSQL). The projects aren’t wrong, they just value many things above performance, and it is their project. 

We ended up implementing our own highly-optimized JSON parser written in C++11. It was much faster than the open source ones it replaced. Nothing fancy, just tight engineering that prioritized performance. We went back later and reimplemented certain hot paths in assembly using SSE4 for systems that supported it, for another 2.5x improvement in throughput as I recall. 


The lack of attention to operational cost in open source is no less of a problem than the lack of attention to user interface design. Users care about these things a lot even if programmers tend not to.

Andrew
_______________________________________________
FoRK mailing list
http://xent.com/mailman/listinfo/fork