[jetty-user] 回复： [jetty-user] why jetty7 is slower than jetty6?

thanks for reply.
I tried jetty-distribution-7.2.1.v20101111,  but it still had poor performance for solr's replication like 7.1.6.v20100715. I embed jetty 6.1.25(maven) to get great performance.I can give some solr's replication codes. For master's write, it's like:

public void write(final OutputStream out) throws IOException {
      final String fileName = params.get(FILE);
      final String cfileName = params.get(CONF_FILE_SHORT);
      final String sOffset = params.get(OFFSET);
      final String sLen = params.get(LEN);
      final String compress = params.get(COMPRESSION);
      final String sChecksum = params.get(CHECKSUM);
      final String sindexVersion = params.get(CMD_INDEX_VERSION);
      if (sindexVersion != null) {
        indexVersion = Long.parseLong(sindexVersion);
      }
      if (Boolean.parseBoolean(compress)) {
        fos = new FastOutputStream(new DeflaterOutputStream(out));
      } else {
        fos = new FastOutputStream(out);
      }
      FileInputStream inputStream = null;
      int packetsWritten = 0;
      try {
        long offset = -1;
        int len = -1;
        //check if checksum is requested
        final boolean useChecksum = Boolean.parseBoolean(sChecksum);
        if (sOffset != null) {
          offset = Long.parseLong(sOffset);
        }
        if (sLen != null) {
          len = Integer.parseInt(sLen);
        }
        if (fileName == null && cfileName == null) {
          //no filename do nothing
          writeNothing();
        }
        
        File file = null;
        if (cfileName != null) {
          //if if is a conf file read from config diectory
          file = new File(core.getResourceLoader().getConfigDir(), cfileName);
        } else {
          //else read from the indexdirectory
          file = new File(core.getIndexDir(), fileName);
        }
        if (file.exists() && file.canRead()) {
          inputStream = new FileInputStream(file);
          final FileChannel channel = inputStream.getChannel();
          //if offset is mentioned move the pointer to that point
          if (offset != -1) {
            channel.position(offset);
          }
          final byte[] buf = new byte[len == -1 || len > PACKET_SZ ? PACKET_SZ : len];
          Checksum checksum = null;
          if (useChecksum) {
            checksum = new Adler32();
          }
          final ByteBuffer bb = ByteBuffer.wrap(buf);
          
          while (true) {
            bb.clear();
            final long bytesRead = channel.read(bb);
            if (bytesRead <= 0) {
              writeNothing();
              fos.close();
              break;
            }
            fos.writeInt((int) bytesRead);
            if (useChecksum) {
              checksum.reset();
              checksum.update(buf, 0, (int) bytesRead);
              fos.writeLong(checksum.getValue());
            }
            //            final long startTime = System.currentTimeMillis();
            fos.write(buf, 0, (int) bytesRead);
            fos.flush();
            //            LOG.info("write "+bytesRead+" cost:"+(System.currentTimeMillis() - startTime));
            if (indexVersion != null && packetsWritten % 5 == 0) {
              //after every 5 packets reserve the commitpoint for some time
              delPolicy.setReserveDuration(indexVersion, reserveCommitDuration);
            }
            packetsWritten++;
          }
        } else {
          writeNothing();
        }
      } catch (final IOException e) {
        LOG.warn("Exception while writing response for params: " + params, e);
      } finally {
        IOUtils.closeQuietly(inputStream);
      }
    }

for slave's read,it's like:

 private int fetchPackets(final FastInputStream fis) throws Exception {
      final byte[] intbytes = new byte[4];
      final byte[] longbytes = new byte[8];
      try {
        while (true) {
          if (stop) {
            stop = false;
            aborted = true;
            throw new ReplicationHandlerException("User aborted replication");
          }
          long checkSumServer = -1;
          fis.readFully(intbytes);
          //read the size of the packet
          final int packetSize = readInt(intbytes);
          if (packetSize <= 0) {
            LOG.warn("No content recieved for file: " + currentFile);
            return NO_CONTENT;
          }
          if (buf.length < packetSize) {
            buf = new byte[packetSize];
          }
          if (checksum != null) {
            //read the checksum
            fis.readFully(longbytes);
            checkSumServer = readLong(longbytes);
          }
          //then read the packet of bytes
          final long startTime = System.currentTimeMillis();
          fis.readFully(buf, 0, packetSize);
          //          LOG.info("readFully"+packetSize+" cost " + (System.currentTimeMillis() - startTime));
          //compare the checksum as sent from the master
          if (includeChecksum) {
            checksum.reset();
            checksum.update(buf, 0, packetSize);
            final long checkSumClient = checksum.getValue();
            if (checkSumClient != checkSumServer) {
              LOG.error("Checksum not matched between client and server for: " + currentFile);
              //if checksum is wrong it is a problem return for retry
              return 1;
            }
          }
          //if everything is fine, write down the packet to the file
          fileChannel.write(ByteBuffer.wrap(buf, 0, packetSize));
          bytesDownloaded += packetSize;
          if (bytesDownloaded >= size) {
            return 0;
          }
          //errorcount is always set to zero after a successful packet
          errorCount = 0;
        }
      } catch (final ReplicationHandlerException e) {
        throw e;
      } catch (final Exception e) {
        LOG.warn("Error in fetching packets ", e);
        //for any failure , increment the error count
        errorCount++;
        //if it fails for the same pacaket for   MAX_RETRIES fail and come out
        if (errorCount > MAX_RETRIES) {
          throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
              "Fetch failed for file:" + fileName, e);
        }
        return ERR;
      }
    }

solr writes PACKET_SZ(1M default) every time.I find one time is too slow(more than 900ms) every 3 times.When I change  PACKET_SZ to 10M,it's about 3s every time.The effect is still like PACKET_SZ = 1M.


 
 
------------------ 原始邮件 ------------------
发件人: "Jesse McConnell"<[email protected]>;
发送时间: 2010年11月12日(星期五) 晚上9:53
收件人: "user"<[email protected]>; 

主题: Re: [jetty-user] why jetty7 is slower than jetty6?

 
which version of jetty?  we have resolved an issue that might apply to
your case were performance was reduced, its resolved in the staged
7.2.1.v20101111 build that we'll likely promote soon..

you can try it at:

https://oss.sonatype.org/content/repositories/jetty-024/org/eclipse/jetty/jetty-distribution/7.2.1.v20101111/

cheers,
jesse

--
jesse mcconnell
[email protected]



On Fri, Nov 12, 2010 at 05:13, kafka0102 <[email protected]> wrote:
> I use jetty as a web server for solr, and I find a problem. solr replicate
> data from master to slave, and master uses a long http connection to write
> data 1M every time.I print the time cost every 1M transit, and find
> default's jetty7 cost is like 996ms,5ms,6ms,998ms,4ms,8ms,thus one time cost
> about 1s every 3 times. If I change to jetty6,it's so fast, and every time
> just cost less then 1ms.For jetty6 and jetty7,I just use default's conf, and
> the problem consist in both embed and external jetty . Who can tell me what
> I'm wrong?
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email