Re: REST impact paper

On 9/8/17 10:10 PM, Sean Conner wrote:
> It was thus said that the Great J. Andrew Rogers once stated:
>> ...
>>
>>    At least with open source stuff we can *fix* the problems and not have to
>> wait for the vendor to provide a fix.
+1  I often debug through library and tool code, places I would be seriously stuck with proprietary solutions.  I do use proprietary 
packages, but often they provide source code for just that reason.

>> Anecdotally, the only open
>> source software my teams use that hasn’t been superseded by vastly better
>> closed source is LLVM.
>    I was playing around with the x86 SIMD instructions.  I have code I wrote
> in the early 90s (in x86 assembly) that matched (I thought) quote well to a
> vectorized version.  The core was an interative application of these two
> functions:
>
> 	x1 = (A * y + B) * x * (1 - x)
> 	y1 = (C * x + D) * y * (1 - y)
>
>    I wrote a version using the vector instructions of the x86.  I could
> easily beat clang (which uses LLVM), but try as I might, I could not beat
> GCC.

Using inline-vector instructions?  That can help a little, but to really get serious impact, you need to write the whole loop / 
function in vectorized assembly.  The proper interleaving of vector and non-vector instructions is where you can start getting 
serious multipliers.  I understand much of it, but have little experience accomplishing much with it.  I'm at the interview people, 
architect, and manage levels there.  I have friends, one friend in particular, who is expert at this.  If I had to write such code, 
I'd rather do it for ARM.  That's were it is likely needed these days anyway.


>> Everything from JSON parsing to computational
>> geometry to database engines, on the other hand, migrated to proprietary
>> implementations.
>    So what magic things were done to make JSON parsing faster with
> proprietary code than with open source code?  Or alternatively, what does
> open source do wrong?  I'm asking for specific examples and not a
> generalized "it sucks" argument.

For XML / JSON / similar: usually, creating and gc'ing/freeing a zillion memory buffers / objects, copies, function calls for what 
should be loops, etc.  Finally, people are now using in-place navigable buffers to avoid parsing more often.

My Java XML parser from 6+ years ago was far faster than anything else available on Android in Java, and it was only about 1000 
lines of code for DOM+SAX (with the core parsing loop <100).  When I looked at some of the popular XML packages, I was horrified by 
the gigantic amounts of code involved.

In a standards related discussion 8+ years ago, the IBM lead insisted that tuned XML parsers were very fast, obviating the need for 
binary formats.  I suspected tool divisions didn't want any disruption.  However, for restricted cases, you can implement very fast 
parsers that pick out needed data with very little overhead. But that isn't equivalent.  Once C++ XML parser, header only I think, 
would null terminate strings in-place, building a pointer tree to the data.  Clever, avoids allocations and other overhead, good for 
many cases.

Proprietary code that you can't get to, probably because you can't afford it, doesn't matter.  With the cost of commodity hardware, 
per-core style of pricing may not be competitive with only a 2-5x improvement.  More likely, a particular application doesn't need 
full capacity anyway.  But sometimes you do.

>    -spc (I have a friend that bitches all the time about "open sores" and how
> 	bad it is, but everytime he talks about working with proprietary
> 	code it sounds even worse than "open sores" ... )

sdw

_______________________________________________
FoRK mailing list
http://xent.com/mailman/listinfo/fork