svn commit: r52296 - head/en_US.ISO8859-1/articles/hubs
Benedict Reuschling <[email protected]>
| Newsgroups | gmane.os.freebsd.devel.cvs.doc |
|---|---|
| Message-ID | <[email protected]> |
Author: bcr
Date: Mon Sep 24 09:34:54 2018
New Revision: 52296
URL: https://svnweb.freebsd.org/changeset/doc/52296
Log:
Clean up the article from textproc/igor warnings such as:
- wrap long lines
- use tabs instead of spaces
- capitalization
- put content after <para> tags
- leave a blank line after <title> tags
- use two spaces at sentence start
Modified:
head/en_US.ISO8859-1/articles/hubs/article.xml
Modified: head/en_US.ISO8859-1/articles/hubs/article.xml
==============================================================================
--- head/en_US.ISO8859-1/articles/hubs/article.xml Sun Sep 23 18:41:53 2018 (r52295)
+++ head/en_US.ISO8859-1/articles/hubs/article.xml Mon Sep 24 09:34:54 2018 (r52296)
@@ -1,22 +1,57 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V5.0-Based Extension//EN"
"http://www.FreeBSD.org/XML/share/xml/freebsd50.dtd">
-<article xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en">
- <info><title>Mirroring FreeBSD</title>
-
+<article xmlns="http://docbook.org/ns/docbook"
+ xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
+ xml:lang="en">
+ <info>
+ <title>Mirroring FreeBSD</title>
+
<authorgroup>
- <author><personname><firstname>Jun</firstname><surname>Kuriyama</surname></personname><affiliation>
- <address><email>[email protected]</email></address>
- </affiliation></author>
- <author><personname><firstname>Valentino</firstname><surname>Vaschetto</surname></personname><affiliation>
- <address><email>[email protected]</email></address>
- </affiliation></author>
- <author><personname><firstname>Daniel</firstname><surname>Lang</surname></personname><affiliation>
- <address><email>[email protected]</email></address>
- </affiliation></author>
- <author><personname><firstname>Ken</firstname><surname>Smith</surname></personname><affiliation>
- <address><email>[email protected]</email></address>
- </affiliation></author>
+ <author>
+ <personname>
+ <firstname>Jun</firstname>
+ <surname>Kuriyama</surname>
+ </personname>
+ <affiliation>
+ <address>
+ <email>[email protected]</email>
+ </address>
+ </affiliation>
+ </author>
+ <author>
+ <personname>
+ <firstname>Valentino</firstname>
+ <surname>Vaschetto</surname>
+ </personname>
+ <affiliation>
+ <address>
+ <email>[email protected]</email>
+ </address>
+ </affiliation>
+ </author>
+ <author>
+ <personname>
+ <firstname>Daniel</firstname>
+ <surname>Lang</surname>
+ </personname>
+ <affiliation>
+ <address>
+ <email>[email protected]</email>
+ </address>
+ </affiliation>
+ </author>
+ <author>
+ <personname>
+ <firstname>Ken</firstname>
+ <surname>Smith</surname>
+ </personname>
+ <affiliation>
+ <address>
+ <email>[email protected]</email>
+ </address>
+ </affiliation>
+ </author>
</authorgroup>
<legalnotice xml:id="trademarks" role="trademarks">
@@ -30,7 +65,7 @@
<abstract>
<para>An in-progress article on how to mirror FreeBSD, aimed at
- hub administrators.</para>
+ hub administrators.</para>
</abstract>
</info>
@@ -47,662 +82,723 @@
</sect1>
<sect1 xml:id="mirror-requirements">
- <title>Requirements for FreeBSD mirrors</title>
+ <title>Requirements for FreeBSD Mirrors</title>
+
<sect2 xml:id="mirror-diskspace">
<title>Disk Space</title>
- <para>
- Disk space is one of the most important requirements.
- Depending on the set of releases, architectures,
- and degree of completeness you want to mirror, a huge
- amount of disk space may be consumed. Also keep in mind
- that <emphasis>official</emphasis> mirrors are probably required to be
- complete. The web pages should
- always be mirrored completely. Also note that the
- numbers stated here are reflecting the current
- state (at &rel2.current;-RELEASE/&rel.current;-RELEASE). Further development and
- releases will only increase the required amount.
- Also make sure to keep some (ca. 10-20%) extra space
- around just to be sure.
- Here are some approximate figures:
- </para>
+
+ <para>Disk space is one of the most important requirements.
+ Depending on the set of releases, architectures, and degree of
+ completeness you want to mirror, a huge amount of disk space
+ may be consumed. Also keep in mind that
+ <emphasis>official</emphasis> mirrors are probably required to
+ be complete. The web pages should always be mirrored
+ completely. Also note that the numbers stated here are
+ reflecting the current state (at
+ &rel2.current;-RELEASE/&rel.current;-RELEASE). Further
+ development and releases will only increase the required
+ amount. Also make sure to keep some (ca. 10-20%) extra space
+ around just to be sure. Here are some approximate
+ figures:</para>
+
<itemizedlist>
- <listitem><para>Full FTP Distribution: 1.4 TB</para></listitem>
- <listitem><para>CTM deltas: 10 GB</para></listitem>
- <listitem><para>Web pages: 1GB</para></listitem>
+ <listitem>
+ <para>Full FTP Distribution: 1.4 TB</para>
+ </listitem>
+ <listitem>
+ <para>CTM deltas: 10 GB</para>
+ </listitem>
+ <listitem>
+ <para>Web pages: 1GB</para>
+ </listitem>
</itemizedlist>
- <para>
- The current disk usage of FTP Distribution can be found at
- <link xlink:href="ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes">ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes</link>.
- </para>
+
+ <para>The current disk usage of FTP Distribution can be found at
+ <link
+ xlink:href="ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes">ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes</link>.</para>
</sect2>
<sect2 xml:id="mirror-bandwidth">
<title>Network Connection/Bandwidth</title>
- <para>
- Of course, you need to be connected to the Internet.
- The required bandwidth depends on your intended use
- of the mirror. If you just want to mirror some
- parts of FreeBSD for local use at your site/intranet,
- the demand may be much smaller than if you want to
- make the files publicly available. If you intend
- to become an official mirror, the bandwidth required will be even higher. We can only give rough
- estimates here:
- </para>
+
+ <para>Of course, you need to be connected to the Internet. The
+ required bandwidth depends on your intended use of the mirror.
+ If you just want to mirror some parts of FreeBSD for local use
+ at your site/intranet, the demand may be much smaller than if
+ you want to make the files publicly available. If you intend
+ to become an official mirror, the bandwidth required will be
+ even higher. We can only give rough estimates here:</para>
+
<itemizedlist>
- <listitem><para>Local site, no public access: basically no minimum,
- but < 2 Mbps could make syncing too slow.</para></listitem>
- <listitem><para>Unofficial public site: 34 Mbps is probably a good start.</para></listitem>
- <listitem><para>Official site: > 100 Mbps is recommended, and your host
- should be connected as close as possible to your border router.</para></listitem>
+ <listitem>
+ <para>Local site, no public access: basically no minimum,
+ but < 2 Mbps could make syncing too
+ slow.</para>
+ </listitem>
+
+ <listitem>
+ <para>Unofficial public site: 34 Mbps is probably a good
+ start.</para>
+ </listitem>
+
+ <listitem>
+ <para>Official site: > 100 Mbps is recommended, and your
+ host should be connected as close as possible to your
+ border router.</para>
+ </listitem>
</itemizedlist>
</sect2>
+
<sect2 xml:id="mirror-system">
<title>System Requirements, CPU, RAM</title>
- <para>
- One thing this depends on the expected number of clients,
- which is determined by the server's policy. It is
- also affected by the types of services you want to offer.
- Plain FTP or HTTP services may not require a huge
- amount of resources. Watch out if you provide
- rsync. This can have a huge
- impact on CPU and memory requirements as it is
- considered a memory hog.
- The following
- are just examples to give you a very rough hint.
- </para>
- <para>
- For a moderately visited site that offers
- <application>rsync</application>, you might
- consider a current CPU with around 800MHz - 1 GHz,
- and at least 512MB RAM. This is probably the
- minimum you want for an <emphasis>official</emphasis>
- site.
- </para>
- <para>
- For a frequently used site you definitely need
- more RAM (consider 2GB as a good start)
- and possibly more CPU, which could also mean
- that you need to go for a SMP system.
- </para>
- <para>
- You also want to consider a fast disk subsystem.
- Operations on the SVN repository require a fast
- disk subsystem (RAID is highly advised). A SCSI
- controller that has a cache of its own can also
- speed up things since most of these services incur a
- large number of small modifications to the disk.
- </para>
+
+ <para>One thing this depends on the expected number of clients,
+ which is determined by the server's policy. It is also
+ affected by the types of services you want to offer. Plain
+ FTP or HTTP services may not require a huge amount of
+ resources. Watch out if you provide rsync. This can have a
+ huge impact on CPU and memory requirements as it is considered
+ a memory hog. The following are just examples to give you a
+ very rough hint.</para>
+
+ <para>For a moderately visited site that offers
+ <application>rsync</application>, you might consider a current
+ CPU with around 800MHz - 1 GHz, and at least 512MB RAM. This
+ is probably the minimum you want for an
+ <emphasis>official</emphasis> site.</para>
+
+ <para>For a frequently used site you definitely need more RAM
+ (consider 2GB as a good start) and possibly more CPU, which
+ could also mean that you need to go for a SMP system.</para>
+
+ <para>You also want to consider a fast disk subsystem.
+ Operations on the SVN repository require a fast disk subsystem
+ (RAID is highly advised). A SCSI controller that has a cache
+ of its own can also speed up things since most of these
+ services incur a large number of small modifications to the
+ disk.</para>
</sect2>
+
<sect2 xml:id="mirror-services">
- <title>Services to offer</title>
- <para>
- Every mirror site is required to have a set of core services
- available. In addition to these required services, there are
- a number of optional services that
- server administrators may choose to offer. This section explains
- which services you can provide and how to go about implementing them.
- </para>
+ <title>Services to Offer</title>
+
+ <para>Every mirror site is required to have a set of core
+ services available. In addition to these required services,
+ there are a number of optional services that server
+ administrators may choose to offer. This section explains
+ which services you can provide and how to go about
+ implementing them.</para>
+
<sect3 xml:id="mirror-serv-ftp">
- <title>FTP (required for FTP fileset)</title>
- <para>
- This is one of the most basic services, and
- it is required for each mirror offering public
- FTP distributions. FTP access must be
- anonymous, and no upload/download ratios
- are allowed (a ridiculous thing anyway).
- Upload capability is not required (and <emphasis>must</emphasis>
- never be allowed for the FreeBSD file space).
- Also the FreeBSD archive should be available under
- the path <filename>/pub/FreeBSD</filename>.
- </para>
- <para>
- There is a lot of software available which
- can be set up to allow anonymous FTP
- (in alphabetical order).</para>
- <itemizedlist>
- <listitem><para><command>/usr/libexec/ftpd</command>: FreeBSD's own ftpd
- can be used. Be sure to read &man.ftpd.8;.</para>
- </listitem>
- <listitem>
- <para><package>ftp/ncftpd</package>: A commercial package,
- free for educational use.</para>
- </listitem>
- <listitem>
- <para><package>ftp/oftpd</package>: An ftpd designed with
- security as a main focus.</para>
- </listitem>
- <listitem>
- <para><package>ftp/proftpd</package>: A modular and very flexible ftpd.</para>
- </listitem>
- <listitem>
- <para><package>ftp/pure-ftpd</package>: Another ftpd developed with
- security in mind.</para>
- </listitem>
- <listitem><para><package>ftp/twoftpd</package>: As above.</para></listitem>
- <listitem><para><package>ftp/vsftpd</package>: The <quote>very secure</quote> ftpd.</para></listitem>
- </itemizedlist>
- <para>FreeBSD's <application>ftpd</application>, <application>proftpd</application>
- and maybe <application>ncftpd</application>
- are among the most commonly used FTPds.
- The others do not have a large userbase among mirror sites. One
- thing to consider is that you may need flexibility in limiting
- how many simultaneous connections are allowed, thus limiting how
- much network bandwidth and system resources are consumed.
- </para>
+ <title>FTP (required for FTP Fileset)</title>
+
+ <para>This is one of the most basic services, and it is
+ required for each mirror offering public FTP distributions.
+ FTP access must be anonymous, and no upload/download ratios
+ are allowed (a ridiculous thing anyway). Upload capability
+ is not required (and <emphasis>must</emphasis> never be
+ allowed for the FreeBSD file space). Also the FreeBSD
+ archive should be available under the path
+ <filename>/pub/FreeBSD</filename>.</para>
+
+ <para>There is a lot of software available which can be set up
+ to allow anonymous FTP (in alphabetical order).</para>
+
+ <itemizedlist>
+ <listitem>
+ <para><command>/usr/libexec/ftpd</command>: FreeBSD's own
+ ftpd can be used. Be sure to read &man.ftpd.8;.</para>
+ </listitem>
+
+ <listitem>
+ <para><package>ftp/ncftpd</package>: A commercial package,
+ free for educational use.</para>
+ </listitem>
+
+ <listitem>
+ <para><package>ftp/oftpd</package>: An ftpd designed with
+ security as a main focus.</para>
+ </listitem>
+
+ <listitem>
+ <para><package>ftp/proftpd</package>: A modular and very
+ flexible ftpd.</para>
+ </listitem>
+
+ <listitem>
+ <para><package>ftp/pure-ftpd</package>: Another ftpd
+ developed with security in mind.</para>
+ </listitem>
+
+ <listitem>
+ <para><package>ftp/twoftpd</package>: As
+ above.</para>
+ </listitem>
+
+ <listitem>
+ <para><package>ftp/vsftpd</package>: The <quote>very
+ secure</quote> ftpd.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>FreeBSD's <application>ftpd</application>,
+ <application>proftpd</application> and maybe
+ <application>ncftpd</application> are among the most
+ commonly used FTPds. The others do not have a large
+ userbase among mirror sites. One thing to consider is that
+ you may need flexibility in limiting how many simultaneous
+ connections are allowed, thus limiting how much network
+ bandwidth and system resources are consumed.</para>
</sect3>
+
<sect3 xml:id="mirror-serv-rsync">
- <title>Rsync (optional for FTP fileset)</title>
- <para>
- <application>Rsync</application> is often offered for access to the
- contents of the FTP area of FreeBSD, so other mirror sites can use your system as their source. The
- protocol is different from FTP in many ways.
- It is much more
- bandwidth friendly, as only differences between files
- are transferred instead of whole files when they change.
- <application>Rsync</application> does require a significant amount of memory for
- each instance. The size depends on the size of
- the synced module in terms of the number of directories and
- files. <application>Rsync</application> can use <command>rsh</command> and
- <command>ssh</command> (now default) as a transport,
- or use its own protocol for stand-alone access
- (this is the preferred method for public rsync servers).
- Authentication, connection limits, and other restrictions
- may be applied. There is just one software package
- available:</para>
- <itemizedlist>
- <listitem><para><package>net/rsync</package></para></listitem>
- </itemizedlist>
+ <title>Rsync (optional for FTP Fileset)</title>
+
+ <para><application>Rsync</application> is often offered for
+ access to the contents of the FTP area of FreeBSD, so other
+ mirror sites can use your system as their source. The
+ protocol is different from FTP in many ways. It is much
+ more bandwidth friendly, as only differences between files
+ are transferred instead of whole files when they change.
+ <application>Rsync</application> does require a significant
+ amount of memory for each instance. The size depends on the
+ size of the synced module in terms of the number of
+ directories and files. <application>Rsync</application> can
+ use <command>rsh</command> and <command>ssh</command> (now
+ default) as a transport, or use its own protocol for
+ stand-alone access (this is the preferred method for public
+ rsync servers). Authentication, connection limits, and
+ other restrictions may be applied. There is just one
+ software package available:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para><package>net/rsync</package></para>
+ </listitem>
+ </itemizedlist>
</sect3>
+
<sect3 xml:id="mirror-serv-http">
- <title>HTTP (required for web pages, optional for FTP fileset)</title>
- <para>
- If you want to offer the FreeBSD web pages, you will need
- to install a web server.
- You may optionally offer the FTP fileset via HTTP.
- The choice of web server software is left up to the mirror administrator.
- Some of the most popular choices are:</para>
+ <title>HTTP (required for Web Pages, Optional for FTP
+ Fileset)</title>
- <itemizedlist>
- <listitem>
- <para><package>www/apache24</package>:
- <application>Apache</application> is still one of the most widely
- deployed web servers on the Internet. It is used
- extensively by the FreeBSD Project.</para>
- </listitem>
+ <para>If you want to offer the FreeBSD web pages, you will
+ need to install a web server. You may optionally offer the
+ FTP fileset via HTTP. The choice of web server software is
+ left up to the mirror administrator. Some of the most
+ popular choices are:</para>
- <listitem>
- <para><package>www/boa</package>:
- <application>Boa</application> is a single-tasking HTTP server.
- Unlike traditional web servers, it does not fork for each incoming
- connection, nor does it fork many copies of itself to handle multiple
- connections. Although, it should provide considerably great
- performance for purely static content.</para>
- </listitem>
+ <itemizedlist>
+ <listitem>
+ <para><package>www/apache24</package>:
+ <application>Apache</application> is still one of the
+ most widely deployed web servers on the Internet. It is
+ used extensively by the FreeBSD Project.</para>
+ </listitem>
- <listitem>
- <para><package>www/cherokee</package>:
- <application>>Cherokee</application> is a very fast, flexible and
- easy to configure web server. It supports the widespread technologies
- nowadays: FastCGI, SCGI, PHP, CGI, SSL/TLS encrypted connections,
- vhosts, users authentication, on the fly encoding and load balancing.
- It also generates <application>Apache</application> compatible log
- files.</para>
- </listitem>
+ <listitem>
+ <para><package>www/boa</package>:
+ <application>Boa</application> is a single-tasking HTTP
+ server. Unlike traditional web servers, it does not
+ fork for each incoming connection, nor does it fork many
+ copies of itself to handle multiple connections.
+ Although, it should provide considerably great
+ performance for purely static content.</para>
+ </listitem>
- <listitem>
- <para><package>www/lighttpd</package>:
- <application>lighttpd</application> is a secure, fast, compliant and
- very flexible web server which has been optimized for high-performance
- environments. It has a very low memory footprint compared to other web
- servers and takes care of cpu-load.</para>
- </listitem>
+ <listitem>
+ <para><package>www/cherokee</package>:
+ <application>>Cherokee</application> is a very fast,
+ flexible and easy to configure web server. It supports
+ the widespread technologies nowadays: FastCGI, SCGI,
+ PHP, CGI, SSL/TLS encrypted connections, vhosts, users
+ authentication, on the fly encoding and load balancing.
+ It also generates <application>Apache</application>
+ compatible log files.</para>
+ </listitem>
- <listitem>
- <para><package>www/nginx</package>:
- <application>nginx</application> is a high performance edge web
- server with a low memory footprint and key features to build
- a modern and efficient web infrastructure. Features include
- a HTTP server, HTTP and mail reverse proxy, caching, load
- balancing, compression, request throttling, connection
- multiplexing and reuse, SSL offload and HTTP media
- streaming.</para>
- </listitem>
+ <listitem>
+ <para><package>www/lighttpd</package>:
+ <application>lighttpd</application> is a secure, fast,
+ compliant and very flexible web server which has been
+ optimized for high-performance environments. It has a
+ very low memory footprint compared to other web servers
+ and takes care of cpu-load.</para>
+ </listitem>
- <listitem>
- <para><package>www/thttpd</package>:
- If you are going to be serving a large amount of static content
- you may find that using an application such as
- <application>thttpd</application> is more efficient than others.
- It is also optimized for excellent performance on FreeBSD.</para>
- </listitem>
- </itemizedlist>
+ <listitem>
+ <para><package>www/nginx</package>:
+ <application>nginx</application> is a high performance
+ edge web server with a low memory footprint and key
+ features to build a modern and efficient web
+ infrastructure. Features include a HTTP server, HTTP
+ and mail reverse proxy, caching, load balancing,
+ compression, request throttling, connection multiplexing
+ and reuse, SSL offload and HTTP media streaming.</para>
+ </listitem>
+
+ <listitem>
+ <para><package>www/thttpd</package>: If you are going to
+ be serving a large amount of static content you may find
+ that using an application such as
+ <application>thttpd</application> is more efficient than
+ others. It is also optimized for excellent performance
+ on FreeBSD.</para>
+ </listitem>
+ </itemizedlist>
</sect3>
- </sect2>
+ </sect2>
</sect1>
+
<sect1 xml:id="mirror-howto">
<title>How to Mirror FreeBSD</title>
- <para>
- Ok, now you know the requirements and how to offer
- the services, but not how to get it. :-)
- This section explains how to actually mirror
- the various parts of FreeBSD, what tools to use,
- and where to mirror from.
- </para>
+
+ <para>Ok, now you know the requirements and how to offer the
+ services, but not how to get it. :-) This section explains how
+ to actually mirror the various parts of FreeBSD, what tools to
+ use, and where to mirror from.</para>
+
<sect2 xml:id="mirror-ftp-rsync">
- <title>Mirroring the FTP site</title>
- <para>
- The FTP area is the largest amount of data that
- needs to be mirrored. It includes the <emphasis>distribution
- sets</emphasis> required for network installation, the
- <emphasis>branches</emphasis> which are actually snapshots
- of checked-out source trees, the <emphasis>ISO Images</emphasis>
- to write CD-ROMs with the installation distribution,
- a live file system, and a snapshot of the ports tree. All of
- course for various FreeBSD versions, and various architectures.
- </para>
- <para>
- The best way to mirror the FTP area is <application>rsync</application>.
- You can install the port <package>net/rsync</package> and then use
- rsync to sync with your upstream host.
- <application>rsync</application> is already mentioned
- in <xref linkend="mirror-serv-rsync"/>.
- Since <application>rsync</application> access is not
- required, your preferred upstream site may not allow it.
- You may need to hunt around a little bit to find a site
- that allows <application>rsync</application> access.</para>
- <note>
- <para>
- Since the number of <application>rsync</application>
- clients will have a significant impact on the server
- machine, most admins impose limitations on their
- server. For a mirror, you should ask the site maintainer
- you are syncing from about their policy, and maybe
- an exception for your host (since you are a mirror).
- </para>
- </note>
- <para>A command line to mirror FreeBSD might look like:</para>
- <screen>&prompt.user; <userinput>rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/</userinput></screen>
- <para>Consult the documentation for <application>rsync</application>,
- which is also available at
- <link xlink:href="http://rsync.samba.org/">http://rsync.samba.org/</link>,
- about the various options to be used with rsync.
- If you sync the whole module (unlike subdirectories),
- be aware that the module-directory (here "FreeBSD")
- will not be created, so you cannot omit the target directory.
- Also you might
- want to set up a script framework that calls such a command
- via &man.cron.8;.
- </para>
+ <title>Mirroring the FTP Site</title>
+
+ <para>The FTP area is the largest amount of data that needs to
+ be mirrored. It includes the <emphasis>distribution
+ sets</emphasis> required for network installation, the
+ <emphasis>branches</emphasis> which are actually snapshots of
+ checked-out source trees, the <emphasis>ISO Images</emphasis>
+ to write CD-ROMs with the installation distribution, a live
+ file system, and a snapshot of the ports tree. All of course
+ for various FreeBSD versions, and various
+ architectures.</para>
+
+ <para>The best way to mirror the FTP area is
+ <application>rsync</application>. You can install the port
+ <package>net/rsync</package> and then use rsync to sync with
+ your upstream host. <application>rsync</application> is
+ already mentioned in <xref linkend="mirror-serv-rsync"/>.
+ Since <application>rsync</application> access is not required,
+ your preferred upstream site may not allow it. You may need
+ to hunt around a little bit to find a site that allows
+ <application>rsync</application> access.</para>
+
+ <note>
+ <para>Since the number of <application>rsync</application>
+ clients will have a significant impact on the server
+ machine, most admins impose limitations on their server.
+ For a mirror, you should ask the site maintainer you are
+ syncing from about their policy, and maybe an exception for
+ your host (since you are a mirror).</para>
+ </note>
+
+ <para>A command line to mirror FreeBSD might look like:</para>
+
+ <screen>&prompt.user; <userinput>rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/</userinput></screen>
+
+ <para>Consult the documentation for
+ <application>rsync</application>, which is also available at
+ <link
+ xlink:href="http://rsync.samba.org/">http://rsync.samba.org/</link>,
+ about the various options to be used with rsync. If you sync
+ the whole module (unlike subdirectories), be aware that the
+ module-directory (here "FreeBSD") will not be created, so you
+ cannot omit the target directory. Also you might want to set
+ up a script framework that calls such a command via
+ &man.cron.8;.</para>
</sect2>
+
<sect2 xml:id="mirror-www">
- <title>Mirroring the WWW pages</title>
- <para>
- The FreeBSD website should only be mirrored via
+ <title>Mirroring the WWW Pages</title>
+
+ <para>The FreeBSD website should only be mirrored via
<application>rsync</application>.</para>
- <para>A command line to mirror the FreeBSD web site might look like:</para>
+
+ <para>A command line to mirror the FreeBSD web site might look
+ like:</para>
+
<screen>&prompt.user; <userinput>rsync -vaHz --delete rsync://bit0.us-west.freebsd.org/FreeBSD-www-data/ /usr/local/www/</userinput></screen>
- </sect2>
- <sect2 xml:id="mirror-pkgs">
- <title>Mirroring Packages</title>
- <para>Due to very high requirements of bandwidth, storage and
- adminstration the &os; Project has decided not to allow public
- mirrors of packages. For sites with lots of machines, it might
- be advantagous to run a caching HTTP proxy for the &man.pkg.8;
- process. Alternatively specific packages and their dependencies
- can be fetched by running something like the following:</para>
+ </sect2>
- <screen>&prompt.user; <userinput>pkg fetch -d -o <replaceable>/usr/local/mirror</replaceable> <replaceable>vim</replaceable></userinput></screen>
+ <sect2 xml:id="mirror-pkgs">
+ <title>Mirroring Packages</title>
- <para>Once those packages have been fetched, the repository metadata must be generated by running:</para>
+ <para>Due to very high requirements of bandwidth, storage and
+ adminstration the &os; Project has decided not to allow public
+ mirrors of packages. For sites with lots of machines, it
+ might be advantagous to run a caching HTTP proxy for the
+ &man.pkg.8; process. Alternatively specific packages and
+ their dependencies can be fetched by running something like
+ the following:</para>
- <screen>&prompt.user; <userinput>pkg repo <replaceable>/usr/local/mirror</replaceable></userinput></screen>
+ <screen>&prompt.user; <userinput>pkg fetch -d -o <replaceable>/usr/local/mirror</replaceable> <replaceable>vim</replaceable></userinput></screen>
- <para>Once the packages have been fetched and the metadata for the
- repository has been generated, serve the packages up to the
- client machines via HTTP. For additional information see the
- man pages for &man.pkg.8;, specifically the &man.pkg-repo.8; page.
- </para>
- </sect2>
- <sect2 xml:id="mirror-how-often">
- <title>How often should I mirror?</title>
- <para>
- Every mirror should be updated at a minimum of once per day.
- Certainly a script with locking to prevent multiple runs
- happening at the same time will be needed to run from
- &man.cron.8;. Since nearly every admin does this in their own
- way, specific instructions cannot be provided. It could work
- something like this:
- </para>
- <procedure>
- <step>
- <para>
- Put the command to run your mirroring application
- in a script. Use of a plain <command>/bin/sh</command>
- script is recommended.
- </para>
- </step>
- <step>
- <para>
- Add some output redirections so diagnostic
- messages are logged to a file.
- </para>
- </step>
- <step>
- <para>
- Test if your script works. Check the logs.
- </para>
- </step>
- <step>
- <para>
- Use &man.crontab.1; to add the script to the
- appropriate user's &man.crontab.5;. This should be a
- different user than what your FTP daemon runs as so that
- if file permissions inside your FTP area are not
- world-readable those files can not be accessed by anonymous
- FTP. This is used to <quote>stage</quote> releases —
- making sure all of the official mirror sites have all of the
- necessary release files on release day.
- </para>
- </step>
- </procedure>
- <para>
- Here are some recommended schedules:</para>
- <itemizedlist>
- <listitem><para>FTP fileset: daily</para></listitem>
- <listitem><para>WWW pages: daily</para></listitem>
- </itemizedlist>
- </sect2>
+ <para>Once those packages have been fetched, the repository
+ metadata must be generated by running:</para>
+
+ <screen>&prompt.user; <userinput>pkg repo <replaceable>/usr/local/mirror</replaceable></userinput></screen>
+
+ <para>Once the packages have been fetched and the metadata for
+ the repository has been generated, serve the packages up to
+ the client machines via HTTP. For additional information see
+ the man pages for &man.pkg.8;, specifically the
+ &man.pkg-repo.8; page.</para>
+ </sect2>
+
+ <sect2 xml:id="mirror-how-often">
+ <title>How Often Should I Mirror?</title>
+
+ <para>Every mirror should be updated at a minimum of once per
+ day. Certainly a script with locking to prevent multiple runs
+ happening at the same time will be needed to run from
+ &man.cron.8;. Since nearly every admin does this in their own
+ way, specific instructions cannot be provided. It could work
+ something like this:</para>
+
+ <procedure>
+ <step>
+ <para>Put the command to run your mirroring application in a
+ script. Use of a plain <command>/bin/sh</command> script
+ is recommended.</para>
+ </step>
+
+ <step>
+ <para>Add some output redirections so diagnostic messages
+ are logged to a file.</para>
+ </step>
+
+ <step>
+ <para>Test if your script works. Check the logs.</para>
+ </step>
+
+ <step>
+ <para>Use &man.crontab.1; to add the script to the
+ appropriate user's &man.crontab.5;. This should be a
+ different user than what your FTP daemon runs as so that
+ if file permissions inside your FTP area are not
+ world-readable those files cannot be accessed by anonymous
+ FTP. This is used to <quote>stage</quote> releases
+ — making sure all of the official mirror sites have
+ all of the necessary release files on release day.</para>
+ </step>
+ </procedure>
+
+ <para>Here are some recommended schedules:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>FTP fileset: daily</para>
+ </listitem>
+ <listitem>
+ <para>WWW pages: daily</para>
+ </listitem>
+ </itemizedlist>
+ </sect2>
</sect1>
+
<sect1 xml:id="mirror-where">
- <title>Where to mirror from</title>
- <para>
- This is an important issue. So this section will
- spend some effort to explain the backgrounds. We will say this
- several times: under no circumstances should you mirror from
- <systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem>.
- </para>
+ <title>Where to Mirror From</title>
+
+ <para>This is an important issue. So this section will spend some
+ effort to explain the backgrounds. We will say this several
+ times: under no circumstances should you mirror from <systemitem
+ class="fqdomainname">ftp.FreeBSD.org</systemitem>.</para>
+
<sect2 xml:id="mirror-where-organization">
- <title>A few words about the organization</title>
- <para>
- Mirrors are organized by country. All
- official mirrors have a DNS entry of the form
- <systemitem class="fqdomainname">ftpN.CC.FreeBSD.org</systemitem>.
- <emphasis>CC</emphasis> (i.e. country code) is the
- <emphasis>top level domain</emphasis> (TLD)
- of the country where this mirror is located.
- <emphasis>N</emphasis> is a number,
- telling that the host would be the <emphasis>Nth</emphasis>
- mirror in that country.
- (Same applies to
- <systemitem>wwwN.CC.FreeBSD.org</systemitem>, etc.)
- There are mirrors with no <emphasis>CC</emphasis> part.
- These are the mirror sites that are very well connected and
- allow a large number of concurrent users.
- <systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem> is actually two machines, one currently
- located in Denmark and the other in the United States.
- It is <emphasis>NOT</emphasis> a master site and should never be
- used to mirror from. Lots of online documentation leads
- <quote>interactive</quote>users to
- <systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem> so automated mirroring
- systems should find a different machine to mirror from.
- </para>
- <para>
- Additionally there exists a hierarchy of mirrors, which
- is described in terms of <emphasis>tiers</emphasis>.
- The master sites are not referred to but can be
- described as <emphasis>Tier-0</emphasis>. Mirrors
- that mirror from these sites can be considered
- <emphasis>Tier-1</emphasis>, mirrors of <emphasis>Tier-1</emphasis>-mirrors,
- are <emphasis>Tier-2</emphasis>, etc.
- Official sites are encouraged to be of a low <emphasis>tier</emphasis>,
- but the lower the tier the higher the requirements in
- terms as described in <xref linkend="mirror-requirements"/>.
- Also access to low-tier-mirrors may be restricted, and
- access to master sites is definitely restricted.
- The <emphasis>tier</emphasis>-hierarchy is not reflected
- by DNS and generally not documented anywhere except
- for the master sites. However, official mirrors with low numbers
- like 1-4, are usually <emphasis>Tier-1</emphasis>
- (this is just a rough hint, and there is no rule).
- </para>
+ <title>A few Words About the Organization</title>
+
+ <para>Mirrors are organized by country. All official mirrors
+ have a DNS entry of the form <systemitem
+ class="fqdomainname">ftpN.CC.FreeBSD.org</systemitem>.
+ <emphasis>CC</emphasis> (i.e., country code) is the
+ <emphasis>top level domain</emphasis> (TLD) of the country
+ where this mirror is located. <emphasis>N</emphasis> is a
+ number, telling that the host would be the
+ <emphasis>Nth</emphasis> mirror in that country. (Same
+ applies to <systemitem>wwwN.CC.FreeBSD.org</systemitem>, etc.)
+ There are mirrors with no <emphasis>CC</emphasis> part. These
+ are the mirror sites that are very well connected and allow a
+ large number of concurrent users. <systemitem
+ class="fqdomainname">ftp.FreeBSD.org</systemitem> is
+ actually two machines, one currently located in Denmark and
+ the other in the United States. It is
+ <emphasis>NOT</emphasis> a master site and should never be
+ used to mirror from. Lots of online documentation leads
+ <quote>interactive</quote>users to <systemitem
+ class="fqdomainname">ftp.FreeBSD.org</systemitem> so
+ automated mirroring systems should find a different machine to
+ mirror from.</para>
+
+ <para>Additionally there exists a hierarchy of mirrors, which is
+ described in terms of <emphasis>tiers</emphasis>. The master
+ sites are not referred to but can be described as
+ <emphasis>Tier-0</emphasis>. Mirrors that mirror from these
+ sites can be considered <emphasis>Tier-1</emphasis>, mirrors
+ of <emphasis>Tier-1</emphasis>-mirrors, are
+ <emphasis>Tier-2</emphasis>, etc. Official sites are
+ encouraged to be of a low <emphasis>tier</emphasis>, but the
+ lower the tier the higher the requirements in terms as
+ described in <xref linkend="mirror-requirements"/>. Also
+ access to low-tier-mirrors may be restricted, and access to
+ master sites is definitely restricted. The
+ <emphasis>tier</emphasis>-hierarchy is not reflected by DNS
+ and generally not documented anywhere except for the master
+ sites. However, official mirrors with low numbers like 1-4,
+ are usually <emphasis>Tier-1</emphasis> (this is just a rough
+ hint, and there is no rule).</para>
</sect2>
+
<sect2 xml:id="mirror-where-where">
- <title>Ok, but where should I get the stuff now?</title>
- <para>
- Under no circumstances should you mirror from <systemitem class="fqdomainname">ftp.FreeBSD.org</systemitem>.
- The short answer is: from the
- site that is closest to you in Internet terms, or gives you
- the fastest access.
- </para>
+ <title>Ok, but Where Should I get the Stuff Now?</title>
+
+ <para>Under no circumstances should you mirror from <systemitem
+ class="fqdomainname">ftp.FreeBSD.org</systemitem>. The
+ short answer is: from the site that is closest to you in
+ Internet terms, or gives you the fastest access.</para>
+
<sect3 xml:id="mirror-where-simple">
- <title>I just want to mirror from somewhere!</title>
- <para>
- If you have no special intentions or
- requirements, the statement in <xref linkend="mirror-where-where"/>
- applies. This means:
- </para>
- <procedure>
- <step>
- <para>
- Check for those which provide fastest access
- (number of hops, round-trip-times)
- and offer the services you intend to
- use (like <application>rsync</application>).
- </para>
- </step>
- <step>
- <para>
- Contact the administrators of your chosen site stating your
- request, and asking about their terms and
- policies.
- </para>
- </step>
- <step>
- <para>
- Set up your mirror as described above.
- </para>
- </step>
- </procedure>
+ <title>I Just Want to Mirror from Somewhere!</title>
+
+ <para>If you have no special intentions or requirements, the
+ statement in <xref linkend="mirror-where-where"/> applies.
+ This means:</para>
+
+ <procedure>
+ <step>
+ <para>Check for those which provide fastest access (number
+ of hops, round-trip-times) and offer the services you
+ intend to use (like
+ <application>rsync</application>).</para>
+ </step>
+
+ <step>
+ <para>Contact the administrators of your chosen site
+ stating your request, and asking about their terms and
+ policies.</para>
+ </step>
+
+ <step>
+ <para>Set up your mirror as described above.</para>
+ </step>
+ </procedure>
</sect3>
+
<sect3 xml:id="mirror-where-official">
- <title>I am an official mirror, what is the right site for me?</title>
- <para>
*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/svn-doc-all
To unsubscribe, send any mail to "[email protected]"