Re: Matrix multiplication performance

On 22 Jan 2016, at 00:28, nasos <[email protected]> wrote:

> Michael,
> please see below
> 
> On 01/21/2016 05:23 PM, Michael Lehn wrote:
>> Hi Nasos,
>> 
>> first of all I don’t want to take wrong credits and want to point out that this is not my algorithm.  It is based on
>> 
>> 	http://www.cs.utexas.edu/users/flame/pubs/blis2_toms_rev3.pdf
>> 
>> 	https://github.com/flame/blis
>> 
>> For a few cores (4-8) it can easily made multithreaded.  For many-cores like Intel Xeon Phi this is a bit more
>> sophisticated but still not too hard.  
> Setting up Phis is indeed an issue, especially because they are "locked" with icpc. Openmp is working properly though.
> 
>> The demo I posted does not use micro kernels that exploit SSE, AVX or
>> FMA instructions.  With that the matrix product is on par with Intel MKL.  Just like BLIS. For my platforms I wrote
>> my own micro-kernels but the interface of function ugemm is compatible to BLIS.
>> 
> If you compile with -O3 I think you are getting  near optimal SSE vectorization. gcc is truly impressive and intel is even more.
>> Maybe you could help me to integrate your code in the benchmark example I posted above.
>> 
> I will try to find some time to spend on the code. 
>> About Blaze:  Do they have their own implementation of a matrix-matrix product?  It seems to require a
>> tuned BLAS implementation (“Otherwise you get only poor performance”) for the matrix-matrix product.
> I will check the benchmarks I run. I think I was using MKL with Blaze, but Blaze is taking it a step further (I am not sure how) and they are getting better performance than the underlying GEMM. Their benchmarks indicate that they are faster than MKL (https://bitbucket.org/blaze-lib/blaze/wiki/Benchmarks#!row-major-matrixmatrix-multiplication)

I started today with similar experiments on BLAZE and had closer look at their internal implementation.  By default
they are calling an external BLAS backend.  On my machine I used the Intel MKL.   But you are right, they also have
an internal implementation that can be used if no external BLAS is available.  I will publish the results on this page:

	http://www.mathematik.uni-ulm.de/~lehn/test_blaze/index.html

At the moment the benchmarks for the internal BLAZE implementation for the matrix-matrix product seem to look
poor.  I asked Klaus Iglberger (the author of BLAZE) to check the compiler flags that I have used.  So don’t take the
current results as-is.

Cheers,

Michael