FPGA optimization ...

Gunther <[email protected]> Mon, 4 Nov 2019 18:33:15 -0500
Newsgroups gmane.comp.db.postgresql.performance
Message-ID <[email protected]>
The time has come.

FPGA optimization is in the palm of our hands (literally a 2 TB 40 GB/s 
IO PostgreSQL server fits into less than a shoe box), and on Amazon AWS 
F1 instances.

Some demos are beginning to exist: https://github.com/Xilinx/data-analytics.
<https://github.com/Xilinx/data-analytics>

But a lot more could be done. How about linear sort performance at O(N)? 
https://hackaday.com/2016/01/20/a-linear-time-sorting-algorithm-for-fpgas/. 
And how about ​https://people.csail.mit.edu/wjun/papers/fccm2017.pdf, 
the following four sorting accelerators are used:

  * Tuple Sorter : Sorts an N-tuple using a sorting network.
  * Page Sorter : Sorts an 8KB (a flash page) chunk of sorted N-tuples
    in on-chip memory.
  * Super-Page Sorter : Sorts 16 8K-32MB sorted chunks in DRAM.
  * Storage-to-Storage Sorter: Sorts 16 512MB or larger sorted chunks in
    flash.

Order of magnitude speed improvements? Better than Hadoop clusters on a 
single chip? 40 GB/s I/O throughput massive full table scan, blazing 
fast sort-merge joins? Here it is. Anybody working more on that? Should 
be an ideal project for a student or a group of students.

Is there a PostgreSQL foundation I could donate to, 501(c)(3) tax 
exempt? I can donate and possibly find some people at Purdue University 
who might take this on. Interest?

regards,
-Gunther