Re: [lustre-devel] RFC: Spill device for Lustre OSD

--===============5067702325757491578==
Content-Type: multipart/alternative; boundary="000000000000c449b20642c83c59"

--000000000000c449b20642c83c59
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Nov 4, 2025 at 8:37=E2=80=AFAM Andreas Dilger <[email protected]> wro=
te:

> On Nov 3, 2025, at 18:58, Oleg Drokin via lustre-devel <
> [email protected]> wrote:
>
>
> On Mon, 2025-11-03 at 16:33 -0800, Jinshan Xiong wrote:
>
> I guess users won't have 1PB OSTs, will they?
>
>
> There probably are already? NASA has a known 0.5P OST configuration:
>
> https://www.nas.nasa.gov/hecc/support/kb/lustre-progressive-file-layout-(=
pfl)-with-ssd-and-hdd-pools_680.html#:~:text=3DThe%20available%20SSD%20spac=
e%20in%20each%20filesystem,decimal%20(far%20right)%20labels%20of%20each%20O=
ST
>
>
In that case, they don't need to use this feature.


>
>
> In order to maximize rebuild performance for declustered parity RAID,
> there are OSTs in production with 90x20TB HDDs =3D 1.4 PB today,
> and requests to have even larger OSTs.  We've done a bunch of work
> to improve huge ldiskfs OST performance, including the hybrid OST
> patches like https://review.whamcloud.com/51625 ("LU-16750 ldiskfs:
> optimize metadata allocation for hybrid LUNs"), but there could still
> be further improvements in supporting such large OSTs.
>
> Cheers, Andreas
> =E2=80=94
> Andreas Dilger
> Lustre Principal Architect
> Whamcloud/DDN
>
>
>
>
>

--000000000000c449b20642c83c59
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote g=
mail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Tue, Nov 4, =
2025 at 8:37=E2=80=AFAM Andreas Dilger &lt;<a href=3D"mailto:[email protected]=
m">[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote=
" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);=
padding-left:1ex">



<div>
On Nov 3, 2025, at 18:58, Oleg Drokin via lustre-devel &lt;<a href=3D"mailt=
o:[email protected]" target=3D"_blank">[email protected]=
e.org</a>&gt; wrote:<br>
<div>
<blockquote type=3D"cite"><br>
<div><span style=3D"font-family:Helvetica;font-size:14px;font-style:normal;=
font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:s=
tart;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0p=
x;text-decoration:none;float:none;display:inline">On
 Mon, 2025-11-03 at 16:33 -0800, Jinshan Xiong wrote:</span><br style=3D"fo=
nt-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:norm=
al;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;t=
ext-transform:none;white-space:normal;word-spacing:0px;text-decoration:none=
">
<blockquote type=3D"cite" style=3D"font-family:Helvetica;font-size:14px;fon=
t-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:norm=
al;text-align:start;text-indent:0px;text-transform:none;white-space:normal;=
word-spacing:0px;text-decoration:none">
I guess users won&#39;t have 1PB OSTs, will they?<br>
</blockquote>
<br style=3D"font-family:Helvetica;font-size:14px;font-style:normal;font-va=
riant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;te=
xt-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-=
decoration:none">
<span style=3D"font-family:Helvetica;font-size:14px;font-style:normal;font-=
variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;=
text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;tex=
t-decoration:none;float:none;display:inline">There
 probably are already? NASA has a known 0.5P OST configuration:</span><br s=
tyle=3D"font-family:Helvetica;font-size:14px;font-style:normal;font-variant=
-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-in=
dent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decor=
ation:none">
<a href=3D"https://www.nas.nasa.gov/hecc/support/kb/lustre-progressive-file=
-layout-(pfl)-with-ssd-and-hdd-pools_680.html#:~:text=3DThe%20available%20S=
SD%20space%20in%20each%20filesystem,decimal%20(far%20right)%20labels%20of%2=
0each%20OST" style=3D"font-family:Helvetica;font-size:14px;font-style:norma=
l;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align=
:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:=
0px" target=3D"_blank">https://www.nas.nasa.gov/hecc/support/kb/lustre-prog=
ressive-file-layout-(pfl)-with-ssd-and-hdd-pools_680.html#:~:text=3DThe%20a=
vailable%20SSD%20space%20in%20each%20filesystem,decimal%20(far%20right)%20l=
abels%20of%20each%20OST</a></div></blockquote></div></div></blockquote><div=
><br></div><div>In that case, they don&#39;t need to use this feature.</div=
><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px=
 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><d=
iv><blockquote type=3D"cite"><div><br style=3D"font-family:Helvetica;font-s=
ize:14px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-=
spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-s=
pace:normal;word-spacing:0px;text-decoration:none">
</div>
</blockquote>
</div>
<div><br>
</div>
In order to maximize rebuild performance for declustered parity RAID,
<div>there are OSTs in production with 90x20TB HDDs =3D 1.4 PB today,</div>
<div>and requests to have even larger OSTs.=C2=A0 We&#39;ve done a bunch of=
 work</div>
<div>to improve huge ldiskfs OST performance, including the hybrid OST</div=
>
<div>patches like=C2=A0<a href=3D"https://review.whamcloud.com/51625" targe=
t=3D"_blank">https://review.whamcloud.com/51625</a>=C2=A0(&quot;LU-16750 ld=
iskfs:</div>
<div>optimize metadata allocation for hybrid LUNs&quot;), but there could s=
till</div>
<div>be further improvements in supporting such large OSTs.<br>
<div><br>
<div>
<div dir=3D"auto" style=3D"color:rgb(0,0,0);letter-spacing:normal;text-alig=
n:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing=
:0px;text-decoration:none">
<div dir=3D"auto" style=3D"color:rgb(0,0,0);letter-spacing:normal;text-alig=
n:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing=
:0px;text-decoration:none">
<div>Cheers, Andreas</div>
<div>=E2=80=94</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud/DDN</div>
</div>
<br>
</div>
<br>
<br>
</div>
<br>
</div>
</div>
</div>

</blockquote></div></div>

--000000000000c449b20642c83c59--

--===============5067702325757491578==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
lustre-devel mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

--===============5067702325757491578==--