Re: [lustre-devel] RFC: Spill device for Lustre OSD

--===============0166530236277440844==
Content-Type: multipart/signed;
 boundary="Apple-Mail=_4B4369F7-3D61-49D0-AFC0-D1012D65ED04";
 protocol="application/pgp-signature"; micalg=pgp-sha256


--Apple-Mail=_4B4369F7-3D61-49D0-AFC0-D1012D65ED04
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

On Nov 4, 2025, at 4:42 PM, Jinshan Xiong wrote:
> On Nov 4, 2025, at 15:07, Andreas Dilger <[email protected]> wrote:
>> Timothy Day <[email protected]> wrote:
>>> Overall, I think the concept is interesting. It reminds me of how
>>> Bcachefs handle multi-device support. Each device can be
>>> designated as holding metadata or data replicas. And you
>>> can control the promotion and migration between different
>>> targets (all managed by a migration daemon). But this design is
>>> too limited, IMHO. If we're going to accept the additional =
complexity
>>> in the OSD, the solution has to be extensible. What if I want to
>>> replicate to multiple targets? What if I want more than two tiers?
>>> What if I want to transparently migrate data from one spill device =
to
>>> another? We don't need this for the initial implementation, sure.
>>> But these seem like natural extensions.
>=20
> It=E2=80=99s possible to extend the design to have multiple spill =
devices in the OSD; you could have two spill devices and mirror them, or =
raid0 to make a larger device. I don=E2=80=99t see the design would not =
allow you to do that.
>=20
>> This is essentially replicating Lustre file layouts in the end, which
>> was my original suggestion - to use FLR and/or PCC-RO foreign
>> mirror layouts for this, even if it is not directly accessible from
>> clients.  That avoids reimplementing tools/formats that already
>> exist in Lustre today for relatively little benefit.
>=20
> One of the goals is to not have a file system-level scanner, which is =
not good. Otherwise, we can just use FLR-based tiered storage.

I don't see how the layout xattr format is related to filesystem-level
scanning?  This is "just" the xattr format *STORED ON THE OST OBJECT*,
and it could be handled by direct device-level (OST) scan tools as well.

This would allow mirrors (FLR), multiple spill devices (multiple FLR
mirrors), concatenated devices (PFL), etc.  It just avoids adding new
formats that need to be parsed, printed, etc.

Cheers, Andreas






--Apple-Mail=_4B4369F7-3D61-49D0-AFC0-D1012D65ED04
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iQIzBAEBCAAdFiEEDb73u6ZejP5ZMprvcqXauRfMH+AFAmkKl7sACgkQcqXauRfM
H+B8yA/+IaycCKrzkPWcJp2mjTCuoCkvSe1y9dXnD4ujI4ot+8jCK5bOQQfVGzvL
xvUyOkifmUX4/eM3uZ61uzDeDEAEVUpXq0rSALIFWeldpP1bwLJlZo5kylL5g9mv
O0vpND7X1EhFaYgI+ePYFZOFN+MRtQyZj3g1YGCF7DjLjTufJlX0CUQFNNvWd6c+
f90zuLeOoHOju37cV6VlVCsQUMEVLUb3oDXhiFES/hU4Va5KIF3X57JAOwN7qr/H
dKNLq5mWoeGDnheMMKnJFhqUJ9Qbww6AOYN3qexWawNw3kYI2r3dh0rqAPJat2A9
weROmBfOgrpi4Ew/af/xgbtyJIIHKPx7iUHWtQtClJ0yhw7lGAxhC2g69Y15PvKg
TNXsr5iwXOVlv52s4wDNJOlcJjuMi0TQTM5WDtxjC2PpKYJnXtI5DgxKhImaAtAG
SdlCMle098FZga28v//AVaprWHLmHWyGhYdu/MTfWkJ5nISddNOyDhmz63pvq1dK
b8Vv2kUc6Dhi7AOws+Xfg+ibWLK2rkaRQ6ZuXUZpetmnvO1VzX43ySxgNQuQFtyi
/zZglU8yGmt3A6pIaOp+fbcw86u71NVtS6zYqAHCrqN+k4ywQ1OCCB4o4hw8wHh6
3AnkJnOfRPxbIsQoVt+WQViD7bjxO4bYSPIbFJpdTkYWCeyJnks=
=n50t
-----END PGP SIGNATURE-----

--Apple-Mail=_4B4369F7-3D61-49D0-AFC0-D1012D65ED04--

--===============0166530236277440844==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
lustre-devel mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

--===============0166530236277440844==--