Re: [lustre-devel] RFC: Spill device for Lustre OSD

--===============4353154605139319170==
Content-Type: multipart/alternative; boundary="00000000000096b23b0642ce6b8d"

--00000000000096b23b0642ce6b8d
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Nov 4, 2025 at 4:46=E2=80=AFPM Andreas Dilger <[email protected]> w=
rote:

> On Nov 4, 2025, at 5:28 PM, Jinshan Xiong <[email protected]> wrote:
> > On Tue, Nov 4, 2025 at 4:20=E2=80=AFPM Andreas Dilger wrote:
> >> I don't see how the layout xattr format is related to filesystem-level
> >> scanning?  This is "just" the xattr format *STORED ON THE OST OBJECT*,
> >> and it could be handled by direct device-level (OST) scan tools as wel=
l.
> >
> > I see. I initially thought you proposed the spill device would function
> as a mirror and that the existing software stack would handle it.
> >
> > Are we going to store the spilled object's status in the layout? It
> sounds difficult because you will have to initiate a layout change from t=
he
> OSTs. I think we still need to store such an xattr locally in the OST
> object.
>
> I'm exactly proposing that the layout xattr be stored on the OST object.
> It would have the same format as the existing Lustre layout stored on
> the MDT, but *maybe* a different name (e.g. "trusted.spl", "trusted.osd",
> or whatever).
>
> I'm not sure if using "trusted.lov" will confuse filesystem scanning
> tools or help them...  It probably makes sense to have a different
> xattr name, just in case we want MDT spilling (e.g. NVRAM->QLC) in
> the future.
>

I would prefer a different name, even though the client likely won't see it
directly. Using the same name for different things will only lead to
confusion. I would go for "trusted.spl".


>
> >> This would allow mirrors (FLR), multiple spill devices (multiple FLR
> >> mirrors), concatenated devices (PFL), etc.  It just avoids adding new
> >> formats that need to be parsed, printed, etc.
>
> Cheers, Andreas
>
>
>
>
>
>

--00000000000096b23b0642ce6b8d
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote g=
mail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Nov 4, =
2025 at 4:46=E2=80=AFPM Andreas Dilger &lt;<a href=3D"mailto:adilger@dilger=
.ca">[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_q=
uote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,2=
04);padding-left:1ex">On Nov 4, 2025, at 5:28 PM, Jinshan Xiong &lt;<a href=
=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>&g=
t; wrote:<br>
&gt; On Tue, Nov 4, 2025 at 4:20=E2=80=AFPM Andreas Dilger wrote:<br>
&gt;&gt; I don&#39;t see how the layout xattr format is related to filesyst=
em-level<br>
&gt;&gt; scanning?=C2=A0 This is &quot;just&quot; the xattr format *STORED =
ON THE OST OBJECT*,<br>
&gt;&gt; and it could be handled by direct device-level (OST) scan tools as=
 well.<br>
&gt; <br>
&gt; I see. I initially thought you proposed the spill device would functio=
n as a mirror and that the existing software stack would handle it.<br>
&gt; <br>
&gt; Are we going to store the spilled object&#39;s status in the layout? I=
t sounds difficult because you will have to initiate a layout change from t=
he OSTs. I think we still need to store such an xattr locally in the OST ob=
ject.<br>
<br>
I&#39;m exactly proposing that the layout xattr be stored on the OST object=
.<br>
It would have the same format as the existing Lustre layout stored on<br>
the MDT, but *maybe* a different name (e.g. &quot;trusted.spl&quot;, &quot;=
trusted.osd&quot;,<br>
or whatever).<br>
<br>
I&#39;m not sure if using &quot;trusted.lov&quot; will confuse filesystem s=
canning<br>
tools or help them...=C2=A0 It probably makes sense to have a different<br>
xattr name, just in case we want MDT spilling (e.g. NVRAM-&gt;QLC) in<br>
the future.<br></blockquote><div><br></div><div>I would prefer a different =
name, even though the client likely won&#39;t see it directly. Using the sa=
me name for different things will only lead to confusion. I would go for &q=
uot;trusted.spl&quot;.</div><div>=C2=A0</div><blockquote class=3D"gmail_quo=
te" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204=
);padding-left:1ex">
<br>
&gt;&gt; This would allow mirrors (FLR), multiple spill devices (multiple F=
LR<br>
&gt;&gt; mirrors), concatenated devices (PFL), etc.=C2=A0 It just avoids ad=
ding new<br>
&gt;&gt; formats that need to be parsed, printed, etc.<br>
<br>
Cheers, Andreas<br>
<br>
<br>
<br>
<br>
<br>
</blockquote></div></div>

--00000000000096b23b0642ce6b8d--

--===============4353154605139319170==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
lustre-devel mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

--===============4353154605139319170==--