Deciding on tensor parameters
Cem Bassoy via ublas <[email protected]> Thu, 13 Sep 2018 17:34:52 +0200
| Newsgroups | gmane.comp.lib.boost.ublas |
|---|---|
| Message-ID | <CADrR+Fsgi2KQw8PVpKeZ8FezWYn6GWKigJZEebkXExeoj3-AbA@mail.gmail.com> |
--===============8502985306503178363==
Content-Type: multipart/alternative; boundary="0000000000006a65d60575c27323"
--0000000000006a65d60575c27323
Content-Type: text/plain; charset="UTF-8"
The GSOC 2018 project with the title "Adding tensor support " has been
succefully completed. Boost.uBlas may support tensors in future. The code,
project and documentation can be found here
<https://github.com/BoostGSoC18/tensor> and here
<https://github.com/BoostGSoC18/tensor/wiki/Documentation>.
The tensor template class is parametrized in terms of data type, storage
format (first- or last-order), storage type (e.g. std::vector or
std::array):
template<class T, class F=first_order, class
A=std::vector<T,std::allocator<T>>>class tensor;
An instance of a tensor template class has dynamic rank (number of
dimensions) and dimensions using a shape class that holds the data. It is
a adaptor of std::vector where the rank is the size of it:
// {3,4,2} could be runtime variables of an integer type.
auto A = tensor<float>{make_shape(3,4,2)};
---------------------
---------------------
I am thinking to redesign the tensor template class where the rank is a
compile time parameter:
template<class T, std::size_t N, class F=first_order<N>, class
A=std::vector<T,std::allocator<T>>>class tensor;
An instance of a tensor template class could be generated as follows:
// {3,4,2} could be runtime variables of an integer type.
auto A = tensor<float,3>(make_shape(3,4,2));
This instantiation could be definitely improved. However, having a static
rank has the following advantages and disadvantages:
-------------
*Advantages*:
1. improving runtime behavior about 30% to 5 % of basic tensor
operations ( depends according to my findings on the length of the inner
most loop ).
2. ability to statically distinguish between different tensor types at
compile time. tensor<float,3> is a different type than tensor<float,4>. If
so, why not setting matrix as an alias:
template <class type, class format, class storage>
using matrix = tensor<type,2,format,storage>.
We would only need to specify and implement one data structure ' tensor '
and if needed provide optimized functions for matrices. This simplifies
the maintenance.
Also there might be advantages in terms of subtensor and iterator support.
However implementing them will be harder.
---------
*Disadvantages*:
1. The implementations become more complicated especially for tensor
multiplications and tensor reshaping.
2. With static rank the interfaces are harder to use (setting the rank
as a template parameter).
3. The number of contracted dimensions must be known at compile time.
Therefore, implementing some tensor algorithms would only be possible with
template specialization instead of simple for loops. Making algorithms
becomes more difficult.
Although Eigen and Boost.MultiArray decided for compile time, it might be
a critical point for uBLAS.
I am working on this right now and also I am trying to suppprt p! number of
linear storage formats as a compile time parameter if p is the rank of the
tensor. Actually unit-testing becomes very hard as I am not able to use
fixtures so easily. Supporting static and dynamic rank will be a
maintenance nightmare.
Cheers
C
--0000000000006a65d60575c27323
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div><font size=3D"2">The GSOC 2018 project with the title=
"Adding tensor support " has been succefully completed. <font si=
ze=3D"2"><font size=3D"2">Boost.uBlas may support tensors in future.</font>=
</font> The code, project and documentation can be found <a href=3D"https:/=
/github.com/BoostGSoC18/tensor">here</a> and <a href=3D"https://github.com/=
BoostGSoC18/tensor/wiki/Documentation">here</a>.<br></font></div><div><font=
size=3D"2"><br></font></div><div><font size=3D"2"></font></div><div><font =
size=3D"2">The tensor template class is parametrized in terms of d</font><f=
ont size=3D"2">ata type,</font><font size=3D"2"> storage format (first- or =
last-order)</font><font size=3D"2">, storage type (e.g. std::vector or std:=
:array):</font></div><div><font size=3D"2"><br></font></div><div><pre><span=
style=3D"font-family:monospace,monospace"><span class=3D"gmail-pl-k">templ=
ate</span><<span class=3D"gmail-pl-k">class</span> <span class=3D"gmail-=
pl-en">T</span>, <span class=3D"gmail-pl-k">class</span> <span class=3D"gma=
il-pl-en">F</span>=3Dfirst_order, <span class=3D"gmail-pl-k">class</span> <=
span class=3D"gmail-pl-en">A</span>=3Dstd::vector<T,std::allocator<T&=
gt;>>
<span class=3D"gmail-pl-k">class</span> <span class=3D"gmail-pl-en">tensor<=
/span>;</span></pre></div><div><font size=3D"2"><br></font></div><div><font=
size=3D"2">An instance of a tensor template class has </font><font size=3D=
"2">dynamic rank (number of dimensions) and=C2=A0 dimensions using a shape =
class that </font>holds the data. It is a adaptor of std::vector where the =
rank is the size of it: <br></div><div><br></div><div><pre><span style=3D"f=
ont-family:monospace,monospace"><font size=3D"2"><span class=3D"gmail-pl-k"=
>// {3,4,2} could be runtime variables of an integer type.<br></span></font=
></span></pre><pre><span style=3D"font-family:monospace,monospace"><font si=
ze=3D"2"><span class=3D"gmail-pl-k">auto</span> A =3D tensor<<span class=
=3D"gmail-pl-k">float</span>>{<span class=3D"gmail-pl-c1">make_shape(3</=
span>,<span class=3D"gmail-pl-c1">4</span>,<span class=3D"gmail-pl-c1">2</s=
pan>)};<br><br></font></span><span style=3D"font-family:arial,helvetica,san=
s-serif"><font size=3D"2">---------------------<br><font size=3D"2">-------=
--------------</font></font></span><span style=3D"font-family:monospace,mon=
ospace"><font size=3D"2"><br></font></span></pre></div><div><font size=3D"2=
">I am thinking to redesign the tensor template class where=C2=A0 the rank =
is a compile time parameter:<br></font></div><div><font size=3D"2"><br></fo=
nt></div><div><pre><span style=3D"font-family:monospace,monospace"><span cl=
ass=3D"gmail-pl-k">template</span><<span class=3D"gmail-pl-k">class</spa=
n> <span class=3D"gmail-pl-en">T</span>, std::size_t N, <span class=3D"gmai=
l-pl-k">class</span> <span class=3D"gmail-pl-en">F</span>=3Dfirst_order<=
N>, <span class=3D"gmail-pl-k">class</span> <span class=3D"gmail-pl-en">=
A</span>=3Dstd::vector<T,std::allocator<T>>>
<span class=3D"gmail-pl-k">class</span> <span class=3D"gmail-pl-en">tensor<=
/span>;<br><br></span></pre><pre><span style=3D"font-family:arial,helvetica=
,sans-serif">An instance of a tensor template class could be generated as f=
ollows:</span><span style=3D"font-family:monospace,monospace"><br></span></=
pre><pre><span style=3D"font-family:monospace,monospace"><span style=3D"fon=
t-family:monospace,monospace"><font size=3D"2"><span class=3D"gmail-pl-k">/=
/ {3,4,2} could be runtime variables of an integer type.<br></span></font><=
/span><span style=3D"font-family:monospace,monospace"><font size=3D"2"><spa=
n class=3D"gmail-pl-k">auto</span> A =3D tensor<<span class=3D"gmail-pl-=
k">float</span>,3>(make_shape(<span class=3D"gmail-pl-c1">3</span>,<span=
class=3D"gmail-pl-c1">4</span>,<span class=3D"gmail-pl-c1">2))</span>;</fo=
nt></span></span></pre></div><div dir=3D"auto"><font size=3D"2"><br></font>
</div>
<div><font size=3D"2">This instantiation could be definitely impr=
oved. However, having a static rank has the following advantages and disadv=
antages:<br></font></div><div><font size=3D"2"><br></font></div><div dir=3D=
"auto"><font size=3D"2">-------------</font></div>
<div dir=3D"auto"><font size=3D"2"><br></font>
</div>
<div dir=3D"auto"><font size=3D"2"><b>Advantages</b>:</font></div=
>
<div dir=3D"auto"><ol><li><font size=3D"2">improving runtime beha=
vior
about 30% to 5 % of basic tensor operations ( depends
according to my findings on the length of the inner most
loop ).</font></li><li><font size=3D"2">ability to statically
distinguish between different tensor types at compile
time. tensor<float,3> is a different type than
tensor<float,4>. If so, why not setting matrix as an
alias:</font>
</li></ol></div><div dir=3D"auto"><span style=3D"font-family:mono=
space,monospace"><font size=3D"2"><br></font></span>
</div><span style=3D"font-family:monospace,monospace">
</span><div dir=3D"auto"><span style=3D"font-family:monospace,mon=
ospace"><font size=3D"2">template <class type,
class format, class storage></font></span></div><span st=
yle=3D"font-family:monospace,monospace">
</span><div dir=3D"auto"><span style=3D"font-family:monospace,mon=
ospace"><font size=3D"2">using matrix =3D
tensor<type,2,format,storage>.</font></span></div>
<div dir=3D"auto"><font size=3D"2"><br></font>
</div>
<div dir=3D"auto"><font size=3D"2">We would only need to specify
and implement one data structure ' tensor ' and if ne=
eded
=C2=A0provide optimized functions for matrices. This simplifi=
es
the maintenance.=C2=A0</font></div>
<div dir=3D"auto"><font size=3D"2">Also there might be advantages
in terms of subtensor and iterator support. However
implementing them will be harder.=C2=A0</font></div>
<div dir=3D"auto"><font size=3D"2"><br></font>
</div>
<div dir=3D"auto"><font size=3D"2">---------</font></div>
<div dir=3D"auto"><font size=3D"2"><b>Disadvantages</b>:</font></=
div>
<div dir=3D"auto"><ol><li><font size=3D"2">The implementations
become more complicated especially for tensor
multiplications and tensor reshaping. <br></font></li><li><fo=
nt size=3D"2">With static
rank the interfaces are harder to use (setting the rank as a
template parameter). <br></font></li><li><font size=3D"2">The=
number of contracted dimensions must be
known at compile time. Therefore, implementing some tensor
algorithms would only be possible with template
specialization instead of simple for loops. Making algorithms=
becomes more difficult.<br></font></li></ol><div><font size=3D"2">Although=
Eigen
and Boost.MultiArray decided=C2=A0 for compile time, it might
be a critical point for uBLAS.</font><font size=3D"2"> <br></=
font></div><div><font size=3D"2"><br></font></div><div><font size=3D"2">I a=
m working on this right now
and also I am trying to suppprt p! number of linear
storage formats as a compile time parameter if p is the
rank of the tensor. Actually unit-testing becomes very
hard as I am not able to use fixtures so easily.
Supporting static and dynamic rank will be a maintenance
nightmare.</font></div><div><font size=3D"2"><br></font></div=
><div><font size=3D"2">Cheers</font></div><div><font size=3D"2">C<br></font=
></div></div></div>
--0000000000006a65d60575c27323--
--===============8502985306503178363==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline