Re: ipropd slow when pushing complete database

Hi, you need the commits from master listed below (which will be in
8.0), which disable sync writes whild doing bulk HDB loads.  It's the
per-record fsync()s that kill performance.

If you copied the HDB with scp and loaded it with kadmin -l, you'd find
it's still slow -- the network has nothing to do with it :/

Our HDB is larger than yours, so we needed this sooner...

commit 7d5f8bb051ca84592d1196bf5d5522da5a50f9d6
Author: Nicolas Williams <[email protected]>
Date:   Tue Oct 10 12:18:57 2017 -0500

    Disable sync during kadmin load

commit 305dc816525f461f9bfe640d87f671f53f0e0fc6
Author: Nicolas Williams <[email protected]>
Date:   Tue Oct 10 12:11:26 2017 -0500

    Disable sync during iprop receive_everything()

    Doing an fsync per-record when receiving the complete HDB is a performance
    disaster.  Among other things, if the HDB is very large, then one slave
    receving a full HDB can cause other slaves to timeout and, if HDB write
    activity is high enough to cause iprop log truncation, then also need full
    syncs, which leads to a cycle of full syncs for all slaves until HDB write
    activity drops.

    Allowing the iprop log to be larger helps, but improving receive_everything()
    performance helps even more.

commit 5bcbe2125b18160f6ad348b15f8036ffedc15770
Author: Nicolas Williams <[email protected]>
Date:   Tue Oct 10 13:06:21 2017 -0500

    Add hdb_set_sync() method