benchmarking i/o throughput

Andreas Schäfer <[email protected]> Mon, 20 Feb 2006 20:09:51 +0100

Newsgroups	gmane.linux.cluster.openmosix.general
Message-ID	<20060220190951.GA4357@wintermute>

Hi,

I was curious how good openMosix' 2.6 I/O bandwith is, so I wrote to
simple programs (see the attachment). The first one writes a
configurable amount of data to the stdout and measures how long this
takes. The second one simply reads data into a buffer.

When I connected both programs with a pipe and migrated only the
reading process to another node I got a througput of approx. 6.3
megabytes/s (on fast ethernet, both machine's CPUs loathing), which
seemed to me pretty much ok.

The weird thing was that when migrating only the writing process, the
throughput dropped to 90 kilobyte/s. I've also tried a much simpler C
program which just printf'ed it's data to the stdout, but still the
throughput was as low as before.

BTW: process migration causes the first sleep() in the writing
program to return immediately, is this an expected behavior?

Has anybody seen something similar? Any ideas how to debug this? 

Thanks!
-Andreas

writeTest.cpp (text/plain, 1.3 KB)

#include <iostream>
#include <sstream>
#include <string>
#include <sys/time.h>
#define BUF_SIZE 32768

void
dump(int megaBytes) {
    char buf[BUF_SIZE];
    for (int i = 0; i < BUF_SIZE; i++) {
        buf[i] = 'a';
    }

    for (int i = 0; i < megaBytes; i++) {
        for (int c = 0; c < 1024*1024/BUF_SIZE; c++) {
            std::cout.write(buf, BUF_SIZE);
        }
    }
    std::cout.flush();
}

int 
str2int(std::string str) {
    std::istringstream conv(str);
    int res;
    conv >> res;
    return res;
}

int 
main(int argc, char **argv) {
    if (argc < 3) {
        std::cerr << "usage: ./writeTest megaBytes sleepTime\n";
        return 1;
    }

    int megaBytes = str2int(argv[1]);
    int sleepTime = str2int(argv[2]);
    struct timeval tBegin;
    struct timeval tEnd;
    struct timezone tz;

    std::cerr << "sleeping " << sleepTime << " seconds\n";
    sleep(sleepTime);
    std::cerr << "sleeping " << sleepTime << " seconds\n";
    sleep(sleepTime);
    std::cerr << "dumping " << megaBytes << " mega bytes\n";

    gettimeofday(&tBegin,&tz);
    dump(megaBytes);
    gettimeofday(&tEnd,&tz);
    
    int milliSpan = (tEnd.tv_sec - tBegin.tv_sec) * 1000 + (tEnd.tv_usec - tBegin.tv_usec) / 1000;
    std::cerr << "write took " << milliSpan << " milli seconds\n";
    std::cerr << (megaBytes*1024.0/milliSpan) << " MB/s\n";
    return 0;
}

readTest.cpp (text/plain, 225 B)

#include <iostream>
#define BUF_SIZE 32768

void
slurp() {
    char buf[BUF_SIZE];

    while (!std::cin.eof()) {
        std::cin.read(buf, BUF_SIZE);
    }
}

int 
main(int argc, char **argv) {
    slurp();
    return 0;
}