[RFC/PATCH] checksum: add SHA-3 (FIPS 202) as an opt-in --checksum-choice
"D. Jain via rsync" <[email protected]> Mon, 11 May 2026 18:18:16 -0400
| Newsgroups | gmane.network.rsync.general |
|---|---|
| Message-ID | <[email protected]> |
This is a multipart message in MIME format.
--===============6746594979202382869==
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_151D_01DCE172.89A27900"
Content-Language: en-us
This is a multipart message in MIME format.
------=_NextPart_000_151D_01DCE172.89A27900
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Hi all,
Posting before opening a PR per the customary convention. I have a
working, tested implementation that adds SHA-3 (sha3-256, sha3-384,
sha3-512) to the algorithms accepted by --checksum-choice.
I want to lead with the question I expect first, because rsync already
ships a generous set of hashes:
Why add SHA-3 when rsync already has xxh128 / xxh3 / md5 / sha1 /
sha256 / sha512?
Five reasons. None of them is "SHA-2 is broken" -- it isn't, today.
1. Different cryptographic design family.
SHA-2 (sha256, sha512) uses the Merkle-Damgaard construction with
a Davies-Meyer compression function. SHA-3 uses the sponge
construction over the Keccak-f[1600] permutation. These share no
structural elements. A theoretical break in one would not imply a
break in the other. Cryptographic agility -- the ability to swap
to a structurally-independent primitive without redesigning the
surrounding system -- is a property worth having, especially in
tools whose output (rsync's --checksum verification, batch files,
--link-dest farms) can persist for years.
2. Length-extension immunity.
SHA-256 and SHA-512 are vulnerable to length-extension attacks:
given H(s || m), an attacker who does not know s can compute
H(s || m || pad || m') without knowing s. rsync does not directly
use a naive secret-prefix MAC, but operators occasionally do build
integrity-verification pipelines on top of "rsync checksum + extra
data". SHA-3's sponge construction is structurally immune to this
class of attack; SHA-2 requires HMAC-style wrapping to avoid it.
3. FIPS 202 / compliance.
SHA-3 is FIPS 202 (2015). A growing set of regulated environments
(FIPS 140-3 module validations, some federal procurement contexts,
defense-adjacent contracts) explicitly require or prefer SHA-3 for
new deployments. Operators in those environments today either
accept "rsync's strongest hash is SHA-2, that has to be good enough"
or wrap rsync in external integrity tooling. Native SHA-3 removes
the friction.
4. No new dependencies.
This is the boring but important one. SHA-3 has been in OpenSSL's
EVP since 1.1.1 (released September 2018) -- which means it is in
every libcrypto rsync already links against on every supported
distro. No new vendored crypto code, no autoconf detection, no new
build dependency. The implementation is exactly:
{ CSUM_SHA3_512, NNI_EVP, "sha3-512", NULL },
{ CSUM_SHA3_384, NNI_EVP, "sha3-384", NULL },
{ CSUM_SHA3_256, NNI_EVP, "sha3-256", NULL },
plus length/canonical case entries. The existing verify_digest()
probe handles older libcrypto cleanly: if the linked OpenSSL doesn't
expose sha3-*, the entry is demoted to CSUM_gone at init and a user
asking for --checksum-choice=sha3-256 gets the familiar "unknown
checksum name" error.
5. Hardware acceleration is starting to land.
ARMv8.2-A's SHA-3 extension (EOR3, RAX1, XAR, BCAX instructions) is
shipping in production silicon -- Apple M-series, AWS Graviton 3+,
recent Snapdragon and MediaTek parts. On those parts SHA-3 can
outperform software SHA-2. Intel/AMD don't have direct SHA-3
instructions today but OpenSSL's optimized software path is
competitive. No advantage on commodity x86 in 2026, but the
trajectory matters.
A few things SHA-3 is NOT being offered for:
- It is not a replacement for xxh3. xxh3 / xxh128 are
non-cryptographic and very fast; they are the right default for
"detect random corruption in a trusted-LAN backup." SHA-3 is for
cases where the threat model includes adversarial collision
construction or compliance.
- It is not being made the default. The "auto" negotiation path is
untouched. An unmodified peer keeps negotiating xxh128 / xxh3 /
md5 / MD4 as before. SHA-3 has to be requested explicitly via
--checksum-choice=sha3-* on both sides (or via the
RSYNC_CHECKSUM_LIST environment override).
- It is not a protocol bump. Negotiation in 3.2+ is already
capability-based via the name_num_obj table; adding entries to
that table is the entire mechanism that xxh3 / xxh128 use.
DESIGN
Wholly through OpenSSL EVP, no vendored hash code:
- 3 entries in valid_checksums_items[] with NNI_EVP, named
"sha3-256" / "sha3-384" / "sha3-512" so that EVP_get_digestbyname()
resolves them directly with no special-case in csum_evp_md().
- 3 new CSUM_SHA3_{256,384,512} constants in lib/md-defines.h, plus
SHA3_{256,384,512}_DIGEST_LEN literals. (OpenSSL never published
SHA3_*_DIGEST_LENGTH macros, so the literal lengths come from us.)
- Length cases in csum_len_for_type(), and -1 in canonical_checksum()
to match the other strong hashes' byte order.
- No new code paths in get_checksum2(), file_checksum(), or
sum_init/update/end. They all short-circuit to the EVP path when
xfer_sum_evp_md / file_sum_evp_md / cur_sum_evp_md is set, so SHA-3
just rides the existing rails -- same as sha256 and sha512.
VERIFIED
Built on Ubuntu 22.04 / OpenSSL 3.0.2 / gcc 11.4, against upstream
master at b9cc0c6 (3.4.2 + 12 commits, protocol 32).
* rsync --version "Checksum list" now reads:
xxh128 xxh3 xxh64 (xxhash) sha3-512 sha3-384 sha3-256
md5 md4 sha1 none
* FIPS 202 vector match:
SHA3-256("abc") = 3a985da7...1532 (correct)
* 1 GiB urandom file, --checksum-choice={sha3-256,sha3-384,sha3-512}:
Local copy: byte-identical, digest matches reference
openssl dgst output in all three.
rsync:// PULL: same. (daemon and client both built from this
patch, listening on 127.0.0.1:8730.)
rsync:// PUSH: same.
Delta-update path: corrupted 16 x 64 KiB blocks scattered through
a 1 GiB destination, resync with sha3-256 and
--no-whole-file. rsync sent 1.5 MiB literal +
1.7 MiB metadata, matched 1022 MiB of existing
blocks (555x speedup), and the assembled file
was byte-identical with matching sha3-256
digest. Confirms the rolling-block strong-
checksum path is wired correctly.
* With --debug=NSTR, auto negotiation still picks xxh128 by default;
SHA-3 is selected only when explicitly requested.
* `make check` passes the new testsuite/checksum-sha3.test. (The
pre-existing 'devices' failure on the test host is environmental
and reproduces on unmodified master -- it needs root to create
/dev/* nodes.)
DIFFSTAT
NEWS.md | 11 +++++
checksum.c | 20 +++++++++
lib/md-defines.h | 12 +++++
rsync.1.md | 7 +++
testsuite/checksum-sha3.test | 60 ++++++++++++++++++++++++
5 files changed, 110 insertions(+)
OPEN QUESTIONS FOR THE LIST
(a) Algorithm names. I used "sha3-256" / "sha3-384" / "sha3-512"
to mirror OpenSSL's EVP digest names directly so the lookup is
transparent. Happy to switch to "sha3_256" / etc. if the house
style prefers underscores.
(b) Single commit, or split implementation / docs / test into a
three-commit series? I've gone with one commit since the change
is small and tightly coupled, but I'm easy.
(c) Anything missing on the protocol / negotiation side I should
worry about? As far as I can see the name_num_obj negotiation
handles new entries automatically and no protocol-version bump
is required, but I'd rather have that confirmed than assumed.
If reception here is positive I'll open a PR against
RsyncProject/rsync and link this thread from it.
Patch inline below. Also happy to attach as a file or send via
git-send-email if anyone prefers that form.
Thanks,
Deepak Jain
AiNET Factory
[email protected]
---
NEWS.md | 11 +++++
checksum.c | 20 +++++++++
lib/md-defines.h | 12 +++++
rsync.1.md | 7 +++
testsuite/checksum-sha3.test | 60 ++++++++++++++++++++++++
5 files changed, 110 insertions(+)
create mode 100755 testsuite/checksum-sha3.test
diff --git a/NEWS.md b/NEWS.md
index c4a73d6..3c659ba 100644
--- a/NEWS.md
+++ b/NEWS.md
@@ -2,6 +2,17 @@
## Changes in this version:
+### ENHANCEMENTS:
+
+- Added SHA-3 (FIPS 202) as an opt-in `--checksum-choice` algorithm
+ family: `sha3-256`, `sha3-384`, and `sha3-512`. The implementation
+ uses OpenSSL EVP and so requires that rsync be built with OpenSSL
+ support and linked against libcrypto >= 1.1.1. Like `sha256` and
+ `sha512`, SHA-3 is never picked by `auto` negotiation; it must be
+ explicitly requested on both ends (e.g.
+ `--checksum-choice=sha3-256`) or made part of the
+ `RSYNC_CHECKSUM_LIST` environment override.
+
### BUG FIXES:
- Fixed a regression introduced by the 3.4.0 secure_relative_open()
diff --git a/checksum.c b/checksum.c
index 24e46bf..f357aa5 100644
--- a/checksum.c
+++ b/checksum.c
@@ -54,6 +54,15 @@ struct name_num_item valid_checksums_items[] = {
#ifdef SUPPORT_XXHASH
{ CSUM_XXH64, 0, "xxh64", NULL },
{ CSUM_XXH64, 0, "xxhash", NULL },
+#endif
+#ifdef USE_OPENSSL
+ /* SHA-3 is offered only via OpenSSL EVP (sha3-* digests,
OpenSSL >= 1.1.1).
+ * verify_digest() probes each entry at startup and disables any
that the
+ * linked libcrypto does not support, so listing them here is
safe even
+ * when the build environment is older. */
+ { CSUM_SHA3_512, NNI_EVP, "sha3-512", NULL },
+ { CSUM_SHA3_384, NNI_EVP, "sha3-384", NULL },
+ { CSUM_SHA3_256, NNI_EVP, "sha3-256", NULL },
#endif
{ CSUM_MD5, NNI_BUILTIN|NNI_EVP, "md5", NULL },
{ CSUM_MD4, NNI_BUILTIN|NNI_EVP, "md4", NULL },
@@ -237,6 +246,14 @@ int csum_len_for_type(int cst, BOOL flist_csum)
#ifdef SHA512_DIGEST_LENGTH
case CSUM_SHA512:
return SHA512_DIGEST_LENGTH;
+#endif
+#ifdef USE_OPENSSL
+ case CSUM_SHA3_256:
+ return SHA3_256_DIGEST_LEN;
+ case CSUM_SHA3_384:
+ return SHA3_384_DIGEST_LEN;
+ case CSUM_SHA3_512:
+ return SHA3_512_DIGEST_LEN;
#endif
case CSUM_XXH64:
case CSUM_XXH3_64:
@@ -266,6 +283,9 @@ int canonical_checksum(int csum_type)
case CSUM_SHA1:
case CSUM_SHA256:
case CSUM_SHA512:
+ case CSUM_SHA3_256:
+ case CSUM_SHA3_384:
+ case CSUM_SHA3_512:
return -1;
case CSUM_XXH64:
case CSUM_XXH3_64:
diff --git a/lib/md-defines.h b/lib/md-defines.h
index 6ef6a68..0b58c57 100644
--- a/lib/md-defines.h
+++ b/lib/md-defines.h
@@ -10,6 +10,15 @@
#define MD4_DIGEST_LEN 16
#define MD5_DIGEST_LEN 16
+
+/* SHA-3 (FIPS 202). OpenSSL exposes these via EVP only, with no
+ * SHA3_*_DIGEST_LENGTH macros, so define our own. SHA3-512 shares
+ * its 64-byte digest length with SHA-512, so MAX_DIGEST_LEN is
+ * unaffected. */
+#define SHA3_256_DIGEST_LEN 32
+#define SHA3_384_DIGEST_LEN 48
+#define SHA3_512_DIGEST_LEN 64
+
#if defined SHA512_DIGEST_LENGTH
#define MAX_DIGEST_LEN SHA512_DIGEST_LENGTH
#elif defined SHA256_DIGEST_LENGTH
@@ -35,3 +44,6 @@
#define CSUM_SHA1 9
#define CSUM_SHA256 10
#define CSUM_SHA512 11
+#define CSUM_SHA3_256 12
+#define CSUM_SHA3_384 13
+#define CSUM_SHA3_512 14
diff --git a/rsync.1.md b/rsync.1.md
index 2b4b750..0e51d44 100644
--- a/rsync.1.md
+++ b/rsync.1.md
@@ -1781,11 +1781,18 @@ expand it.
- `xxh128`
- `xxh3`
- `xxh64` (aka `xxhash`)
+ - `sha3-512`
+ - `sha3-384`
+ - `sha3-256`
- `md5`
- `md4`
- `sha1`
- `none`
+ The `sha3-*` choices require an rsync built with OpenSSL support and
+ a libcrypto that provides SHA-3 (OpenSSL 1.1.1 or newer). They are
+ never selected by `auto` negotiation and must be requested explicitly.
+
Run `rsync --version` to see the default checksum list compiled into
your
version (which may differ from the list above).
diff --git a/testsuite/checksum-sha3.test b/testsuite/checksum-sha3.test
new file mode 100755
index 0000000..37a4a12
--- /dev/null
+++ b/testsuite/checksum-sha3.test
@@ -0,0 +1,60 @@
+#!/bin/sh
+
+# Test SHA-3 checksum support (sha3-256, sha3-384, sha3-512).
+#
+# Requires rsync to be built with OpenSSL support and linked against a
+# libcrypto that exposes the SHA-3 family (OpenSSL 1.1.1 or newer).
+# The test probes for SHA-3 by asking rsync to use it on a no-op copy;
+# if the local rsync rejects the choice (e.g. no OpenSSL at build time,
+# or older libcrypto), the test is skipped rather than failed.
+
+. "$suitedir/rsync.fns"
+
+probe() {
+ $RSYNC --checksum-choice="$1" --help >/dev/null 2>&1 || return 1
+ # --help doesn't actually parse the choice; force a real parse.
+ mkdir -p "$tmpdir/probe.src"
+ : > "$tmpdir/probe.src/x"
+ rm -rf "$tmpdir/probe.dst"
+ $RSYNC -a --checksum-choice="$1" "$tmpdir/probe.src/"
"$tmpdir/probe.dst/" 2>"$outfile"
+ rc=$?
+ rm -rf "$tmpdir/probe.src" "$tmpdir/probe.dst"
+ if [ $rc -ne 0 ] && grep -q 'unknown checksum name' "$outfile"; then
+ return 1
+ fi
+ return $rc
+}
+
+if ! probe sha3-256; then
+ test_skipped "rsync was not built with OpenSSL SHA-3 support"
+fi
+
+mkdir "$fromdir"
+mkdir "$fromdir/sub"
+# Mix of empty, small, and chunk-spanning sizes (CHUNK_SIZE is 32k
internally).
+: > "$fromdir/empty"
+echo "hello sha-3" > "$fromdir/small"
+dd if=/dev/urandom of="$fromdir/big" bs=1024 count=70 2>/dev/null
+
+for algo in sha3-256 sha3-384 sha3-512; do
+ rm -rf "$todir"
+
+ # Whole-file path (transfer checksum only).
+ checkit "$RSYNC -a --checksum-choice=$algo '$fromdir/' '$todir/'"
"$fromdir" "$todir"
+
+ # Pre-transfer + transfer checksum path (-c forces file_checksum to
run).
+ rm -rf "$todir"
+ checkit "$RSYNC -ac --checksum-choice=$algo '$fromdir/' '$todir/'"
"$fromdir" "$todir"
+
+ # --checksum-choice with distinct xfer,file pair.
+ rm -rf "$todir"
+ checkit "$RSYNC -ac --checksum-choice=$algo,$algo '$fromdir/'
'$todir/'" "$fromdir" "$todir"
+done
+
+# Delta-update path: mutate the destination, re-sync with SHA-3, confirm
match.
+rm -rf "$todir"
+$RSYNC -a --checksum-choice=sha3-256 "$fromdir/" "$todir/" || test_fail
"initial sync failed"
+echo "modified locally" >> "$todir/small"
+checkit "$RSYNC -a --checksum-choice=sha3-256 '$fromdir/' '$todir/'"
"$fromdir" "$todir"
+
+exit 0
------=_NextPart_000_151D_01DCE172.89A27900
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:x=3D"urn:schemas-microsoft-com:office:excel" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40"><head><META =
HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii"><meta name=3DGenerator content=3D"Microsoft Word 15 =
(filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DEN-US =
link=3D"#0563C1" vlink=3D"#954F72"><div class=3DWordSection1><p =
class=3DMsoNormal>Hi all,<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Posting =
before opening a PR per the customary convention. I have =
a<o:p></o:p></p><p class=3DMsoNormal>working, tested implementation that =
adds SHA-3 (sha3-256, sha3-384,<o:p></o:p></p><p =
class=3DMsoNormal>sha3-512) to the algorithms accepted by =
--checksum-choice.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>I want to =
lead with the question I expect first, because rsync =
already<o:p></o:p></p><p class=3DMsoNormal>ships a generous set of =
hashes:<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> Why add SHA-3 when rsync already has =
xxh128 / xxh3 / md5 / sha1 /<o:p></o:p></p><p =
class=3DMsoNormal> sha256 / sha512?<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Five =
reasons. None of them is "SHA-2 is broken" -- it isn't, =
today.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>1. Different cryptographic design =
family.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> SHA-2 (sha256, sha512) uses the =
Merkle-Damgaard construction with<o:p></o:p></p><p =
class=3DMsoNormal> a Davies-Meyer compression =
function. SHA-3 uses the sponge<o:p></o:p></p><p =
class=3DMsoNormal> construction over the Keccak-f[1600] =
permutation. These share no<o:p></o:p></p><p =
class=3DMsoNormal> structural elements. A theoretical =
break in one would not imply a<o:p></o:p></p><p =
class=3DMsoNormal> break in the other. Cryptographic =
agility -- the ability to swap<o:p></o:p></p><p =
class=3DMsoNormal> to a structurally-independent primitive =
without redesigning the<o:p></o:p></p><p class=3DMsoNormal> =
surrounding system -- is a property worth having, especially =
in<o:p></o:p></p><p class=3DMsoNormal> tools whose output =
(rsync's --checksum verification, batch files,<o:p></o:p></p><p =
class=3DMsoNormal> --link-dest farms) can persist for =
years.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>2. Length-extension immunity.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> =
SHA-256 and SHA-512 are vulnerable to length-extension =
attacks:<o:p></o:p></p><p class=3DMsoNormal> given H(s || =
m), an attacker who does not know s can compute<o:p></o:p></p><p =
class=3DMsoNormal> H(s || m || pad || m') without knowing =
s. rsync does not directly<o:p></o:p></p><p =
class=3DMsoNormal> use a naive secret-prefix MAC, but =
operators occasionally do build<o:p></o:p></p><p =
class=3DMsoNormal> integrity-verification pipelines on top =
of "rsync checksum + extra<o:p></o:p></p><p =
class=3DMsoNormal> data". SHA-3's sponge =
construction is structurally immune to this<o:p></o:p></p><p =
class=3DMsoNormal> class of attack; SHA-2 requires =
HMAC-style wrapping to avoid it.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>3. FIPS 202 =
/ compliance.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> SHA-3 is FIPS 202 (2015). A growing =
set of regulated environments<o:p></o:p></p><p =
class=3DMsoNormal> (FIPS 140-3 module validations, some =
federal procurement contexts,<o:p></o:p></p><p =
class=3DMsoNormal> defense-adjacent contracts) explicitly =
require or prefer SHA-3 for<o:p></o:p></p><p =
class=3DMsoNormal> new deployments. Operators in those =
environments today either<o:p></o:p></p><p =
class=3DMsoNormal> accept "rsync's strongest hash is =
SHA-2, that has to be good enough"<o:p></o:p></p><p =
class=3DMsoNormal> or wrap rsync in external integrity =
tooling. Native SHA-3 removes<o:p></o:p></p><p =
class=3DMsoNormal> the friction.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>4. No new =
dependencies.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> This is the boring but important =
one. SHA-3 has been in OpenSSL's<o:p></o:p></p><p =
class=3DMsoNormal> EVP since 1.1.1 (released September 2018) =
-- which means it is in<o:p></o:p></p><p class=3DMsoNormal> =
every libcrypto rsync already links against on every =
supported<o:p></o:p></p><p class=3DMsoNormal> distro. =
No new vendored crypto code, no autoconf detection, no =
new<o:p></o:p></p><p class=3DMsoNormal> build =
dependency. The implementation is exactly:<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> { CSUM_SHA3_512, =
NNI_EVP, "sha3-512", NULL },<o:p></o:p></p><p =
class=3DMsoNormal> { CSUM_SHA3_384, =
NNI_EVP, "sha3-384", NULL },<o:p></o:p></p><p =
class=3DMsoNormal> { CSUM_SHA3_256, =
NNI_EVP, "sha3-256", NULL },<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> =
plus length/canonical case entries. The existing =
verify_digest()<o:p></o:p></p><p class=3DMsoNormal> probe =
handles older libcrypto cleanly: if the linked OpenSSL =
doesn't<o:p></o:p></p><p class=3DMsoNormal> expose sha3-*, =
the entry is demoted to CSUM_gone at init and a user<o:p></o:p></p><p =
class=3DMsoNormal> asking for --checksum-choice=3Dsha3-256 =
gets the familiar "unknown<o:p></o:p></p><p =
class=3DMsoNormal> checksum name" =
error.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>5. Hardware acceleration is starting to =
land.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> ARMv8.2-A's SHA-3 extension (EOR3, RAX1, =
XAR, BCAX instructions) is<o:p></o:p></p><p =
class=3DMsoNormal> shipping in production silicon -- Apple =
M-series, AWS Graviton 3+,<o:p></o:p></p><p =
class=3DMsoNormal> recent Snapdragon and MediaTek =
parts. On those parts SHA-3 can<o:p></o:p></p><p =
class=3DMsoNormal> outperform software SHA-2. =
Intel/AMD don't have direct SHA-3<o:p></o:p></p><p =
class=3DMsoNormal> instructions today but OpenSSL's =
optimized software path is<o:p></o:p></p><p =
class=3DMsoNormal> competitive. No advantage on =
commodity x86 in 2026, but the<o:p></o:p></p><p =
class=3DMsoNormal> trajectory matters.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>A few things =
SHA-3 is NOT being offered for:<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> - It =
is not a replacement for xxh3. xxh3 / xxh128 are<o:p></o:p></p><p =
class=3DMsoNormal> non-cryptographic and very fast; =
they are the right default for<o:p></o:p></p><p =
class=3DMsoNormal> "detect random corruption in a =
trusted-LAN backup." SHA-3 is for<o:p></o:p></p><p =
class=3DMsoNormal> cases where the threat model =
includes adversarial collision<o:p></o:p></p><p =
class=3DMsoNormal> construction or =
compliance.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> - It is not being made the default. The =
"auto" negotiation path is<o:p></o:p></p><p =
class=3DMsoNormal> untouched. An unmodified peer =
keeps negotiating xxh128 / xxh3 /<o:p></o:p></p><p =
class=3DMsoNormal> md5 / MD4 as before. SHA-3 =
has to be requested explicitly via<o:p></o:p></p><p =
class=3DMsoNormal> --checksum-choice=3Dsha3-* on both =
sides (or via the<o:p></o:p></p><p class=3DMsoNormal> =
RSYNC_CHECKSUM_LIST environment override).<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> - It =
is not a protocol bump. Negotiation in 3.2+ is =
already<o:p></o:p></p><p class=3DMsoNormal> =
capability-based via the name_num_obj table; adding entries =
to<o:p></o:p></p><p class=3DMsoNormal> that table is =
the entire mechanism that xxh3 / xxh128 use.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>DESIGN<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Wholly =
through OpenSSL EVP, no vendored hash code:<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> - 3 =
entries in valid_checksums_items[] with NNI_EVP, named<o:p></o:p></p><p =
class=3DMsoNormal> "sha3-256" / =
"sha3-384" / "sha3-512" so that =
EVP_get_digestbyname()<o:p></o:p></p><p =
class=3DMsoNormal> resolves them directly with no =
special-case in csum_evp_md().<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> - 3 =
new CSUM_SHA3_{256,384,512} constants in lib/md-defines.h, =
plus<o:p></o:p></p><p class=3DMsoNormal> =
SHA3_{256,384,512}_DIGEST_LEN literals. (OpenSSL never =
published<o:p></o:p></p><p class=3DMsoNormal> =
SHA3_*_DIGEST_LENGTH macros, so the literal lengths come from =
us.)<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> - Length cases in csum_len_for_type(), and -1 =
in canonical_checksum()<o:p></o:p></p><p =
class=3DMsoNormal> to match the other strong hashes' =
byte order.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> - No new code paths in get_checksum2(), =
file_checksum(), or<o:p></o:p></p><p =
class=3DMsoNormal> sum_init/update/end. They all =
short-circuit to the EVP path when<o:p></o:p></p><p =
class=3DMsoNormal> xfer_sum_evp_md / file_sum_evp_md / =
cur_sum_evp_md is set, so SHA-3<o:p></o:p></p><p =
class=3DMsoNormal> just rides the existing rails -- =
same as sha256 and sha512.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>VERIFIED<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>Built on =
Ubuntu 22.04 / OpenSSL 3.0.2 / gcc 11.4, against =
upstream<o:p></o:p></p><p class=3DMsoNormal>master at b9cc0c6 (3.4.2 + =
12 commits, protocol 32).<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> * =
rsync --version "Checksum list" now reads:<o:p></o:p></p><p =
class=3DMsoNormal> xxh128 xxh3 =
xxh64 (xxhash) sha3-512 sha3-384 sha3-256<o:p></o:p></p><p =
class=3DMsoNormal> md5 md4 =
sha1 none<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> * FIPS 202 vector match:<o:p></o:p></p><p =
class=3DMsoNormal> =
SHA3-256("abc") =3D 3a985da7...1532 =
(correct)<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> * 1 GiB urandom file, =
--checksum-choice=3D{sha3-256,sha3-384,sha3-512}:<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal> Local =
copy: byte-identical, digest =
matches reference<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; openssl dgst output in all three.<o:p></o:p></p><p =
class=3DMsoNormal> rsync:// =
PULL: same. (daemon and client both built =
from this<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; patch, listening on 127.0.0.1:8730.)<o:p></o:p></p><p =
class=3DMsoNormal> rsync:// =
PUSH: same.<o:p></o:p></p><p =
class=3DMsoNormal> Delta-update path: corrupted 16 x =
64 KiB blocks scattered through<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; a 1 GiB destination, resync with sha3-256 and<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; --no-whole-file. rsync sent 1.5 MiB literal =
+<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; 1.7 MiB metadata, matched 1022 MiB of existing<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; blocks (555x speedup), and the assembled file<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; was byte-identical with matching sha3-256<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; digest. Confirms the rolling-block strong-<o:p></o:p></p><p =
class=3DMsoNormal> &=
nbsp; &n=
bsp; checksum path is wired correctly.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> * =
With --debug=3DNSTR, auto negotiation still picks xxh128 by =
default;<o:p></o:p></p><p class=3DMsoNormal> SHA-3 is =
selected only when explicitly requested.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> * =
`make check` passes the new testsuite/checksum-sha3.test. =
(The<o:p></o:p></p><p class=3DMsoNormal> pre-existing =
'devices' failure on the test host is environmental<o:p></o:p></p><p =
class=3DMsoNormal> and reproduces on unmodified master =
-- it needs root to create<o:p></o:p></p><p =
class=3DMsoNormal> /dev/* nodes.)<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>DIFFSTAT<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> =
NEWS.md =
| 11 =
+++++<o:p></o:p></p><p class=3DMsoNormal> =
checksum.c &nb=
sp; | 20 =
+++++++++<o:p></o:p></p><p class=3DMsoNormal> =
lib/md-defines.h &nb=
sp; | 12 +++++<o:p></o:p></p><p class=3DMsoNormal> =
rsync.1.md &nb=
sp; | 7 =
+++<o:p></o:p></p><p class=3DMsoNormal> =
testsuite/checksum-sha3.test | 60 =
++++++++++++++++++++++++<o:p></o:p></p><p class=3DMsoNormal> 5 =
files changed, 110 insertions(+)<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>OPEN =
QUESTIONS FOR THE LIST<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> (a) =
Algorithm names. I used "sha3-256" / =
"sha3-384" / "sha3-512"<o:p></o:p></p><p =
class=3DMsoNormal> to mirror OpenSSL's EVP =
digest names directly so the lookup is<o:p></o:p></p><p =
class=3DMsoNormal> transparent. =
Happy to switch to "sha3_256" / etc. if the =
house<o:p></o:p></p><p class=3DMsoNormal> =
style prefers underscores.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> (b) =
Single commit, or split implementation / docs / test into =
a<o:p></o:p></p><p class=3DMsoNormal> =
three-commit series? I've gone with one commit since the =
change<o:p></o:p></p><p class=3DMsoNormal> =
is small and tightly coupled, but I'm easy.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal> (c) =
Anything missing on the protocol / negotiation side I =
should<o:p></o:p></p><p class=3DMsoNormal> =
worry about? As far as I can see the name_num_obj =
negotiation<o:p></o:p></p><p =
class=3DMsoNormal> handles new entries =
automatically and no protocol-version bump<o:p></o:p></p><p =
class=3DMsoNormal> is required, but I'd =
rather have that confirmed than assumed.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>If reception =
here is positive I'll open a PR against<o:p></o:p></p><p =
class=3DMsoNormal>RsyncProject/rsync and link this thread from =
it.<o:p></o:p></p><p class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>Patch inline below. Also happy to attach as a =
file or send via<o:p></o:p></p><p class=3DMsoNormal>git-send-email if =
anyone prefers that form.<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>Thanks,<o:p></o:p></p><p class=3DMsoNormal>Deepak =
Jain<o:p></o:p></p><p class=3DMsoNormal>AiNET Factory<o:p></o:p></p><p =
class=3DMsoNormal>[email protected]<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p =
class=3DMsoNormal>---<o:p></o:p></p><p class=3DMsoNormal> =
NEWS.md =
| 11 =
+++++<o:p></o:p></p><p class=3DMsoNormal> =
checksum.c =
&=
nbsp; | 20 +++++++++<o:p></o:p></p><p class=3DMsoNormal> =
lib/md-defines.h &nb=
sp; | 12 +++++<o:p></o:p></p><p class=3DMsoNormal> =
rsync.1.md &nb=
sp; | 7 =
+++<o:p></o:p></p><p class=3DMsoNormal> testsuite/checksum-sha3.test | =
60 ++++++++++++++++++++++++<o:p></o:p></p><p class=3DMsoNormal> 5 files =
changed, 110 insertions(+)<o:p></o:p></p><p class=3DMsoNormal> create =
mode 100755 testsuite/checksum-sha3.test<o:p></o:p></p><p =
class=3DMsoNormal><o:p> </o:p></p><p class=3DMsoNormal>diff --git =
a/NEWS.md b/NEWS.md<o:p></o:p></p><p class=3DMsoNormal>index =
c4a73d6..3c659ba 100644<o:p></o:p></p><p class=3DMsoNormal>--- =
a/NEWS.md<o:p></o:p></p><p class=3DMsoNormal>+++ =
b/NEWS.md<o:p></o:p></p><p class=3DMsoNormal>@@ -2,6 +2,17 =
@@<o:p></o:p></p><p class=3DMsoNormal> <o:p></o:p></p><p =
class=3DMsoNormal> ## Changes in this version:<o:p></o:p></p><p =
class=3DMsoNormal> <o:p></o:p></p><p class=3DMsoNormal>+### =
ENHANCEMENTS:<o:p></o:p></p><p class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal>+- Added SHA-3 (FIPS 202) as an opt-in =
`--checksum-choice` algorithm<o:p></o:p></p><p class=3DMsoNormal>+ =
family: `sha3-256`, `sha3-384`, and `sha3-512`. The =
implementation<o:p></o:p></p><p class=3DMsoNormal>+ uses OpenSSL =
EVP and so requires that rsync be built with OpenSSL<o:p></o:p></p><p =
class=3DMsoNormal>+ support and linked against libcrypto >=3D =
1.1.1. Like `sha256` and<o:p></o:p></p><p class=3DMsoNormal>+ =
`sha512`, SHA-3 is never picked by `auto` negotiation; it must =
be<o:p></o:p></p><p class=3DMsoNormal>+ explicitly requested on =
both ends (e.g.<o:p></o:p></p><p class=3DMsoNormal>+ =
`--checksum-choice=3Dsha3-256`) or made part of the<o:p></o:p></p><p =
class=3DMsoNormal>+ `RSYNC_CHECKSUM_LIST` environment =
override.<o:p></o:p></p><p class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal> ### BUG FIXES:<o:p></o:p></p><p class=3DMsoNormal> =
<o:p></o:p></p><p class=3DMsoNormal> - Fixed a regression =
introduced by the 3.4.0 secure_relative_open()<o:p></o:p></p><p =
class=3DMsoNormal>diff --git a/checksum.c b/checksum.c<o:p></o:p></p><p =
class=3DMsoNormal>index 24e46bf..f357aa5 100644<o:p></o:p></p><p =
class=3DMsoNormal>--- a/checksum.c<o:p></o:p></p><p =
class=3DMsoNormal>+++ b/checksum.c<o:p></o:p></p><p class=3DMsoNormal>@@ =
-54,6 +54,15 @@ struct name_num_item valid_checksums_items[] =3D =
{<o:p></o:p></p><p class=3DMsoNormal> #ifdef =
SUPPORT_XXHASH<o:p></o:p></p><p class=3DMsoNormal> =
=
{ CSUM_XXH64, 0, "xxh64", NULL },<o:p></o:p></p><p =
class=3DMsoNormal> =
=
{ CSUM_XXH64, 0, "xxhash", NULL },<o:p></o:p></p><p =
class=3DMsoNormal>+#endif<o:p></o:p></p><p class=3DMsoNormal>+#ifdef =
USE_OPENSSL<o:p></o:p></p><p =
class=3DMsoNormal>+ =
/* SHA-3 is offered only via OpenSSL EVP (sha3-* digests, =
OpenSSL >=3D 1.1.1).<o:p></o:p></p><p =
class=3DMsoNormal>+ =
* verify_digest() probes each entry at startup and =
disables any that the<o:p></o:p></p><p =
class=3DMsoNormal>+ =
* linked libcrypto does not support, so listing them here =
is safe even<o:p></o:p></p><p =
class=3DMsoNormal>+ =
* when the build environment is older. */<o:p></o:p></p><p =
class=3DMsoNormal>+ =
{ CSUM_SHA3_512, NNI_EVP, "sha3-512", NULL =
},<o:p></o:p></p><p =
class=3DMsoNormal>+ =
{ CSUM_SHA3_384, NNI_EVP, "sha3-384", NULL =
},<o:p></o:p></p><p =
class=3DMsoNormal>+ =
{ CSUM_SHA3_256, NNI_EVP, "sha3-256", NULL =
},<o:p></o:p></p><p class=3DMsoNormal> #endif<o:p></o:p></p><p =
class=3DMsoNormal> =
=
{ CSUM_MD5, NNI_BUILTIN|NNI_EVP, "md5", NULL =
},<o:p></o:p></p><p class=3DMsoNormal> =
=
{ CSUM_MD4, NNI_BUILTIN|NNI_EVP, "md4", NULL =
},<o:p></o:p></p><p class=3DMsoNormal>@@ -237,6 +246,14 @@ int =
csum_len_for_type(int cst, BOOL flist_csum)<o:p></o:p></p><p =
class=3DMsoNormal> #ifdef SHA512_DIGEST_LENGTH<o:p></o:p></p><p =
class=3DMsoNormal> =
=
case CSUM_SHA512:<o:p></o:p></p><p class=3DMsoNormal> =
&=
nbsp; &n=
bsp; return SHA512_DIGEST_LENGTH;<o:p></o:p></p><p =
class=3DMsoNormal>+#endif<o:p></o:p></p><p class=3DMsoNormal>+#ifdef =
USE_OPENSSL<o:p></o:p></p><p =
class=3DMsoNormal>+ =
case CSUM_SHA3_256:<o:p></o:p></p><p =
class=3DMsoNormal>+ =
=
&=
nbsp; return SHA3_256_DIGEST_LEN;<o:p></o:p></p><p =
class=3DMsoNormal>+ =
case CSUM_SHA3_384:<o:p></o:p></p><p =
class=3DMsoNormal>+ =
&=
nbsp; return =
SHA3_384_DIGEST_LEN;<o:p></o:p></p><p =
class=3DMsoNormal>+ =
case CSUM_SHA3_512:<o:p></o:p></p><p =
class=3DMsoNormal>+ =
&=
nbsp; return =
SHA3_512_DIGEST_LEN;<o:p></o:p></p><p class=3DMsoNormal> =
#endif<o:p></o:p></p><p class=3DMsoNormal> =
=
case CSUM_XXH64:<o:p></o:p></p><p class=3DMsoNormal> =
=
case CSUM_XXH3_64:<o:p></o:p></p><p class=3DMsoNormal>@@ =
-266,6 +283,9 @@ int canonical_checksum(int csum_type)<o:p></o:p></p><p =
class=3DMsoNormal> =
=
case CSUM_SHA1:<o:p></o:p></p><p class=3DMsoNormal> =
=
case CSUM_SHA256:<o:p></o:p></p><p class=3DMsoNormal> =
=
case CSUM_SHA512:<o:p></o:p></p><p =
class=3DMsoNormal>+ =
case CSUM_SHA3_256:<o:p></o:p></p><p =
class=3DMsoNormal>+ =
case CSUM_SHA3_384:<o:p></o:p></p><p =
class=3DMsoNormal>+ =
case CSUM_SHA3_512:<o:p></o:p></p><p =
class=3DMsoNormal> =
&=
nbsp; &n=
bsp; return -1;<o:p></o:p></p><p class=3DMsoNormal> =
=
case CSUM_XXH64:<o:p></o:p></p><p class=3DMsoNormal> =
=
case CSUM_XXH3_64:<o:p></o:p></p><p class=3DMsoNormal>diff =
--git a/lib/md-defines.h b/lib/md-defines.h<o:p></o:p></p><p =
class=3DMsoNormal>index 6ef6a68..0b58c57 100644<o:p></o:p></p><p =
class=3DMsoNormal>--- a/lib/md-defines.h<o:p></o:p></p><p =
class=3DMsoNormal>+++ b/lib/md-defines.h<o:p></o:p></p><p =
class=3DMsoNormal>@@ -10,6 +10,15 @@<o:p></o:p></p><p class=3DMsoNormal> =
<o:p></o:p></p><p class=3DMsoNormal> #define MD4_DIGEST_LEN =
16<o:p></o:p></p><p class=3DMsoNormal> #define MD5_DIGEST_LEN =
16<o:p></o:p></p><p class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal>+/* SHA-3 (FIPS 202). OpenSSL exposes these via EVP =
only, with no<o:p></o:p></p><p class=3DMsoNormal>+ * =
SHA3_*_DIGEST_LENGTH macros, so define our own. SHA3-512 =
shares<o:p></o:p></p><p class=3DMsoNormal>+ * its 64-byte digest length =
with SHA-512, so MAX_DIGEST_LEN is<o:p></o:p></p><p class=3DMsoNormal>+ =
* unaffected. */<o:p></o:p></p><p class=3DMsoNormal>+#define =
SHA3_256_DIGEST_LEN 32<o:p></o:p></p><p class=3DMsoNormal>+#define =
SHA3_384_DIGEST_LEN 48<o:p></o:p></p><p class=3DMsoNormal>+#define =
SHA3_512_DIGEST_LEN 64<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal> #if defined =
SHA512_DIGEST_LENGTH<o:p></o:p></p><p class=3DMsoNormal> #define =
MAX_DIGEST_LEN SHA512_DIGEST_LENGTH<o:p></o:p></p><p class=3DMsoNormal> =
#elif defined SHA256_DIGEST_LENGTH<o:p></o:p></p><p class=3DMsoNormal>@@ =
-35,3 +44,6 @@<o:p></o:p></p><p class=3DMsoNormal> #define CSUM_SHA1 =
9<o:p></o:p></p><p class=3DMsoNormal> #define CSUM_SHA256 =
10<o:p></o:p></p><p class=3DMsoNormal> #define CSUM_SHA512 =
11<o:p></o:p></p><p class=3DMsoNormal>+#define CSUM_SHA3_256 =
12<o:p></o:p></p><p class=3DMsoNormal>+#define CSUM_SHA3_384 =
13<o:p></o:p></p><p class=3DMsoNormal>+#define CSUM_SHA3_512 =
14<o:p></o:p></p><p class=3DMsoNormal>diff --git a/rsync.1.md =
b/rsync.1.md<o:p></o:p></p><p class=3DMsoNormal>index 2b4b750..0e51d44 =
100644<o:p></o:p></p><p class=3DMsoNormal>--- =
a/rsync.1.md<o:p></o:p></p><p class=3DMsoNormal>+++ =
b/rsync.1.md<o:p></o:p></p><p class=3DMsoNormal>@@ -1781,11 +1781,18 @@ =
expand it.<o:p></o:p></p><p class=3DMsoNormal> - =
`xxh128`<o:p></o:p></p><p class=3DMsoNormal> - =
`xxh3`<o:p></o:p></p><p class=3DMsoNormal> - =
`xxh64` (aka `xxhash`)<o:p></o:p></p><p =
class=3DMsoNormal>+ - `sha3-512`<o:p></o:p></p><p =
class=3DMsoNormal>+ - `sha3-384`<o:p></o:p></p><p =
class=3DMsoNormal>+ - `sha3-256`<o:p></o:p></p><p =
class=3DMsoNormal> - `md5`<o:p></o:p></p><p =
class=3DMsoNormal> - `md4`<o:p></o:p></p><p =
class=3DMsoNormal> - `sha1`<o:p></o:p></p><p =
class=3DMsoNormal> - `none`<o:p></o:p></p><p =
class=3DMsoNormal> <o:p></o:p></p><p =
class=3DMsoNormal>+ The `sha3-*` choices require an =
rsync built with OpenSSL support and<o:p></o:p></p><p =
class=3DMsoNormal>+ a libcrypto that provides SHA-3 =
(OpenSSL 1.1.1 or newer). They are<o:p></o:p></p><p =
class=3DMsoNormal>+ never selected by `auto` =
negotiation and must be requested explicitly.<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal> Run `rsync --version` to see =
the default checksum list compiled into your<o:p></o:p></p><p =
class=3DMsoNormal> version (which may differ =
from the list above).<o:p></o:p></p><p class=3DMsoNormal> =
<o:p></o:p></p><p class=3DMsoNormal>diff --git =
a/testsuite/checksum-sha3.test =
b/testsuite/checksum-sha3.test<o:p></o:p></p><p class=3DMsoNormal>new =
file mode 100755<o:p></o:p></p><p class=3DMsoNormal>index =
0000000..37a4a12<o:p></o:p></p><p class=3DMsoNormal>--- =
/dev/null<o:p></o:p></p><p class=3DMsoNormal>+++ =
b/testsuite/checksum-sha3.test<o:p></o:p></p><p class=3DMsoNormal>@@ =
-0,0 +1,60 @@<o:p></o:p></p><p =
class=3DMsoNormal>+#!/bin/sh<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal>+# Test SHA-3 =
checksum support (sha3-256, sha3-384, sha3-512).<o:p></o:p></p><p =
class=3DMsoNormal>+#<o:p></o:p></p><p class=3DMsoNormal>+# Requires =
rsync to be built with OpenSSL support and linked against =
a<o:p></o:p></p><p class=3DMsoNormal>+# libcrypto that exposes the SHA-3 =
family (OpenSSL 1.1.1 or newer).<o:p></o:p></p><p class=3DMsoNormal>+# =
The test probes for SHA-3 by asking rsync to use it on a no-op =
copy;<o:p></o:p></p><p class=3DMsoNormal>+# if the local rsync rejects =
the choice (e.g. no OpenSSL at build time,<o:p></o:p></p><p =
class=3DMsoNormal>+# or older libcrypto), the test is skipped rather =
than failed.<o:p></o:p></p><p class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal>+. "$suitedir/rsync.fns"<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal>+probe() =
{<o:p></o:p></p><p class=3DMsoNormal>+ $RSYNC =
--checksum-choice=3D"$1" --help >/dev/null 2>&1 || =
return 1<o:p></o:p></p><p class=3DMsoNormal>+ # --help =
doesn't actually parse the choice; force a real parse.<o:p></o:p></p><p =
class=3DMsoNormal>+ mkdir -p =
"$tmpdir/probe.src"<o:p></o:p></p><p =
class=3DMsoNormal>+ : > =
"$tmpdir/probe.src/x"<o:p></o:p></p><p =
class=3DMsoNormal>+ rm -rf =
"$tmpdir/probe.dst"<o:p></o:p></p><p =
class=3DMsoNormal>+ $RSYNC -a =
--checksum-choice=3D"$1" "$tmpdir/probe.src/" =
"$tmpdir/probe.dst/" =
2>"$outfile"<o:p></o:p></p><p =
class=3DMsoNormal>+ rc=3D$?<o:p></o:p></p><p =
class=3DMsoNormal>+ rm -rf =
"$tmpdir/probe.src" =
"$tmpdir/probe.dst"<o:p></o:p></p><p =
class=3DMsoNormal>+ if [ $rc -ne 0 ] && grep =
-q 'unknown checksum name' "$outfile"; then<o:p></o:p></p><p =
class=3DMsoNormal>+ return =
1<o:p></o:p></p><p class=3DMsoNormal>+ =
fi<o:p></o:p></p><p class=3DMsoNormal>+ return =
$rc<o:p></o:p></p><p class=3DMsoNormal>+}<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal>+if ! probe =
sha3-256; then<o:p></o:p></p><p class=3DMsoNormal>+ =
test_skipped "rsync was not built with OpenSSL SHA-3 =
support"<o:p></o:p></p><p class=3DMsoNormal>+fi<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal>+mkdir =
"$fromdir"<o:p></o:p></p><p class=3DMsoNormal>+mkdir =
"$fromdir/sub"<o:p></o:p></p><p class=3DMsoNormal>+# Mix of =
empty, small, and chunk-spanning sizes (CHUNK_SIZE is 32k =
internally).<o:p></o:p></p><p class=3DMsoNormal>+: > =
"$fromdir/empty"<o:p></o:p></p><p class=3DMsoNormal>+echo =
"hello sha-3" > "$fromdir/small"<o:p></o:p></p><p =
class=3DMsoNormal>+dd if=3D/dev/urandom of=3D"$fromdir/big" =
bs=3D1024 count=3D70 2>/dev/null<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal>+for algo in =
sha3-256 sha3-384 sha3-512; do<o:p></o:p></p><p =
class=3DMsoNormal>+ rm -rf =
"$todir"<o:p></o:p></p><p class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal>+ # Whole-file path (transfer =
checksum only).<o:p></o:p></p><p class=3DMsoNormal>+ =
checkit "$RSYNC -a --checksum-choice=3D$algo '$fromdir/' =
'$todir/'" "$fromdir" "$todir"<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal>+ # Pre-transfer + transfer checksum =
path (-c forces file_checksum to run).<o:p></o:p></p><p =
class=3DMsoNormal>+ rm -rf =
"$todir"<o:p></o:p></p><p =
class=3DMsoNormal>+ checkit "$RSYNC -ac =
--checksum-choice=3D$algo '$fromdir/' '$todir/'" =
"$fromdir" "$todir"<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p =
class=3DMsoNormal>+ # --checksum-choice with distinct =
xfer,file pair.<o:p></o:p></p><p class=3DMsoNormal>+ =
rm -rf "$todir"<o:p></o:p></p><p =
class=3DMsoNormal>+ checkit "$RSYNC -ac =
--checksum-choice=3D$algo,$algo '$fromdir/' '$todir/'" =
"$fromdir" "$todir"<o:p></o:p></p><p =
class=3DMsoNormal>+done<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal>+# Delta-update =
path: mutate the destination, re-sync with SHA-3, confirm =
match.<o:p></o:p></p><p class=3DMsoNormal>+rm -rf =
"$todir"<o:p></o:p></p><p class=3DMsoNormal>+$RSYNC -a =
--checksum-choice=3Dsha3-256 "$fromdir/" "$todir/" =
|| test_fail "initial sync failed"<o:p></o:p></p><p =
class=3DMsoNormal>+echo "modified locally" >> =
"$todir/small"<o:p></o:p></p><p class=3DMsoNormal>+checkit =
"$RSYNC -a --checksum-choice=3Dsha3-256 '$fromdir/' '$todir/'" =
"$fromdir" "$todir"<o:p></o:p></p><p =
class=3DMsoNormal>+<o:p></o:p></p><p class=3DMsoNormal>+exit =
0<o:p></o:p></p></div></body></html>
------=_NextPart_000_151D_01DCE172.89A27900--
--===============6746594979202382869==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--===============6746594979202382869==--