Re: [SPDK] question about env_spdk.cc
"Harris, James R" <[email protected]> Mon, 10 Apr 2023 22:37:14 +0000
| Newsgroups | dev.linux.lists.spdk |
|---|---|
| Message-ID | <BYAPR11MB33038C78632CD67D265C4CE7DE959@BYAPR11MB3303.namprd11.prod.outlook.com> |
--_000_BYAPR11MB33038C78632CD67D265C4CE7DE959BYAPR11MB3303namp_
Content-Type: text/plain; charset="gb2312"
Content-Transfer-Encoding: base64
KFJlLWFkZGluZyBTUERLIG1haWxpbmcgbGlzdCBvbiBteSByZXNwb25zZSkNCg0KSGksDQoNCldv
dWxkIHlvdSBsaWtlIHRvIHN1Ym1pdCBhIHBhdGNoIHRvIGZpeCBpdD8gIFBsZWFzZSBzZWUgaHR0
cHM6Ly9zcGRrLmlvL2RldmVsb3BtZW50IGZvciBob3cgdG8gcHVzaCBwYXRjaGVzIHRvIHRoZSBT
UERLIEdlcnJpdCBpbnN0YW5jZSBmb3IgQ0kgdGVzdGluZyBhbmQgcmV2aWV3Lg0KDQpUaGFua3Ms
DQoNCkppbQ0KDQoNCkZyb206ILqrueLR9CA8Z3Vhbmd5YW5nLmhhbkBmb3htYWlsLmNvbT4NCkRh
dGU6IFNhdHVyZGF5LCBBcHJpbCA4LCAyMDIzIGF0IDk6MzQgUE0NClRvOiBIYXJyaXMsIEphbWVz
IFIgPGphbWVzLnIuaGFycmlzQGludGVsLmNvbT4NClN1YmplY3Q6IFJlOiBbU1BES10gcXVlc3Rp
b24gYWJvdXQgZW52X3NwZGsuY2MNClRoYW5rcyBmb3IgeW91ciB0aW1lLg0KSW4gZmFjdCB0aGF0
IGhhcHBlbnMgc29tZXRpbWVzLg0KQmVmb3JlIGNhbGxpbmcgU3Bka1JhbmRvbUFjY2Vzc0ZpbGU6
OlJlYWQsIFJvY2tzZGIgd291bGQgZXh0ZW5kIHRoZSBiZWdpbiBhbmQgZW5kIHJlYWQgcG9pbnQg
dG8gU1NEIHBhZ2UgYm91bmRhcnksIHNvbWV0aW1lcyBpdCBnb2VzIHBhc3QgdGhlIGVuZCBvZiB0
aGUgZmlsZSBpZiBSb2Nrc2RiIHJlYWQgdGhlIGxhc3Rlc3QgcGFydCB0aGF0IGhhdmVuJ3QgZmls
bCB1cCBhIHBhZ2UuDQpXaGVuIHJjPG4sIEkgYWRkIGEgcHJpbnRmIHRvIG5vdGljZSB0aGlzLCB0
aGVuIEkgZmluZCBpdCBoYXBwZW5zIGZyZXF1ZW50bHkuDQpMdWNraWx5LCB0aGlzIGhhc24ndCBs
ZWFkIHRvIGEgYnVnIGJlY2F1c2Ugcm9ja3NkYiBub3QgdXNlIHRoZSByZXR1cm5lZCBTbGljZSAi
cmVzdWx0Iiwgcm9ja3NkYiB1c2UgdGhlIGRhdGEgaW4gYnVmICJzY3JhdGNoIiBkaXJlY3RseSwg
YnV0IEkgc3RpbGwgdGhpbmsgdGhpcyBzaG91bGQgYmUgbW9kaWZ5IHRvIHByZXZlbnQgZnV0dXJl
IG1pc3Rha2UuDQotLS1PcmlnaW5hbC0tLQ0KRnJvbTogIkhhcnJpcywgSmFtZXMgUiI8amFtZXMu
ci5oYXJyaXNAaW50ZWwuY29tPg0KRGF0ZTogMjAyMy80LzggMDI6MzI6MTkNClRvOiAiuqu54tH0
IjxndWFuZ3lhbmcuaGFuQGZveG1haWwuY29tPjsic3BkayI8c3Bka0BsaXN0cy5saW51eGZvdW5k
YXRpb24ub3JnPjsNClN1YmplY3Q6IFJlOiBbU1BES10gcXVlc3Rpb24gYWJvdXQgZW52X3NwZGsu
Y2MNCg0KVGhpcyBkb2VzIGxvb2sgbGlrZSBhIGJ1Zy4gIEJ1dCBvbmx5IHdheSBpdCBjYW4gdHJp
Z2dlciBpcyBpZiBTcGRrUmFuZG9tQWNjZXNzRmlsZTo6UmVhZCBpcyBjYWxsZWQgd2l0aCBsZW5n
dGggKyBvZmZzZXQgdGhhdCBnb2VzIHBhc3QgdGhlIGVuZCBvZiB0aGUgZmlsZS4gIFJvY2tzREIg
d29uoa90IGFjdHVhbGx5IGRvIHRoYXQgdGhvdWdoLg0KDQoNCg0KRnJvbTogU1BESyA8c3Bkay1i
b3VuY2VzQGxpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmc+IG9uIGJlaGFsZiBvZiC6q7ni0fQgPGd1
YW5neWFuZy5oYW5AZm94bWFpbC5jb20+DQpEYXRlOiBGcmlkYXksIEFwcmlsIDcsIDIwMjMgYXQg
MTA6MjMgQU0NClRvOiBzcGRrIDxzcGRrQGxpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmc+DQpTdWJq
ZWN0OiBbU1BES10gcXVlc3Rpb24gYWJvdXQgZW52X3NwZGsuY2MNClJlY2VudGx5ICBJIGFtIHRy
eWluZyB0byB0ZWFtIHNwZGsgd2l0aCBSb2Nrc2RiLCB0aGUgY29kZSBpbiBlbnZfc3Bkay5jYyBz
ZWVtcyBoYXZpbmcgYSBidWcuDQoNCkluIGZ1bmMgU3Bka1JhbmRvbUFjY2Vzc0ZpbGU6OlJlYWQs
IHRoZSBmdW5jIHNwZGtfZmlsZV9yZWFkIHJldHVybnMgdGhlIG51bWJlciBvZiBieXRlcyBzdWNj
ZXNzZnVsbHkgcmVhZCBmcm9tIGZpbGUsIHRoZW4gdGhlIGJ1ZmZlciBjb250YWlucyB0aG9zZSBi
eXRlcyBhcmUgdXNlZCB0byBpbml0aWFsaXplIHRoZSByZXN1bHQgU2xpY2UsIGhvd2V2ZXIsIHRo
ZSBzaXplIGlmICJuIiBpbnN0ZWFkIG9mIHJjLCBJIHRoaW5rIGl0IGlzIGNvbmZ1c2luZy4NCkFz
IGEgbWF0dGVyIG9mIGZhY3QsIGluIHRoZSBzYW1lIGZ1bmMgU3Bka1NlcXVlbnRpYWxGaWxlOjpS
ZWFkLCB0aGUgbGVudGggdG8gaW5pdGlhbGl6ZSByZXN1bHQgc2xpY2UgaXMgcmMsIG5vdCBuLg0K
DQpJIGhhdmUgdHJpZWQgdG8gbW9kaWZ5IHRoZSBuIHRvIHJjLCB0aGluZ3Mgc2VlbSB0byBoYXZl
IG5vIGNoYW5nZSwgd2hpY2ggaXMgbW9yZSBjb25mdXNpbmcuDQoNCkhvcGUgc29tZW9uZSBjYW4g
aGVscCBtZSwgdGhhbmtzIHlvdSENCg==
--_000_BYAPR11MB33038C78632CD67D265C4CE7DE959BYAPR11MB3303namp_
Content-Type: text/html; charset="gb2312"
Content-Transfer-Encoding: quoted-printable
<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" xmlns:w=3D"urn:sc=
hemas-microsoft-com:office:word" xmlns:m=3D"http://schemas.microsoft.com/of=
fice/2004/12/omml" xmlns=3D"http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dgb2312">
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"PingFang TC";
panose-1:2 11 4 0 0 0 0 0 0 0;}
@font-face
{font-family:"\@PingFang TC";}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72" style=3D"word-wrap:=
break-word">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><span style=3D"color:#1D1D1D">(Re-adding SPDK mailin=
g list on my response)<o:p></o:p></span></p>
<p class=3D"MsoNormal"><span style=3D"color:#1D1D1D"><o:p> </o:p></spa=
n></p>
<p class=3D"MsoNormal"><span style=3D"color:#1D1D1D">Hi,</span><span style=
=3D"color:#1D1D1D"><o:p></o:p></span></p>
<p class=3D"MsoNormal" style=3D"caret-color: rgb(29, 29, 29);font-variant-c=
aps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-a=
djust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style=3D"color:#1D1D1D"><br>
Would you like to submit a patch to fix it? Please see<span class=3D"=
apple-converted-space"> </span><a href=3D"https://spdk.io/development"=
title=3D"https://spdk.io/development"><span style=3D"color:#0058B9">https:=
//spdk.io/development</span></a><span class=3D"apple-converted-space"> =
;</span>for
how to push patches to the SPDK Gerrit instance for CI testing and review.=
<o:p></o:p></span></p>
<p class=3D"MsoNormal" style=3D"caret-color: rgb(29, 29, 29);font-variant-c=
aps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-a=
djust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style=3D"color:#1D1D1D"> <o:p></o:p></span></p>
<p class=3D"MsoNormal" style=3D"caret-color: rgb(29, 29, 29);font-variant-c=
aps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-a=
djust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style=3D"color:#1D1D1D">Thanks,<o:p></o:p></span></p>
<p class=3D"MsoNormal" style=3D"caret-color: rgb(29, 29, 29);font-variant-c=
aps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-a=
djust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style=3D"color:#1D1D1D"> <o:p></o:p></span></p>
<p class=3D"MsoNormal" style=3D"caret-color: rgb(29, 29, 29);font-variant-c=
aps: normal;orphans: auto;text-align:start;widows: auto;-webkit-text-size-a=
djust: auto;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style=3D"color:#1D1D1D">Jim<o:p></o:p></span></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;mar=
gin-bottom:12.0pt;margin-left:.5in">
<b><span style=3D"font-size:12.0pt;color:black">From: </span></b><span styl=
e=3D"font-size:12.0pt;font-family:"PingFang TC",sans-serif;color:=
black">=BA=AB</span><span style=3D"font-size:12.0pt;font-family:"MS Go=
thic";color:black">=B9=E2=D1=F4</span><span style=3D"font-size:12.0pt;=
color:black">
<[email protected]><br>
<b>Date: </b>Saturday, April 8, 2023 at 9:34 PM<br>
<b>To: </b>Harris, James R <[email protected]><br>
<b>Subject: </b>Re: [SPDK] question about env_spdk.cc<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in">Thanks for your time.<o:p=
></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in">In fact that happens some=
times. <o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in">Before calling SpdkRandom=
AccessFile::Read, Rocksdb would extend the begin and end read point to SSD&=
nbsp;page boundary, sometimes it goes past the end of the file if Rocksdb r=
ead the lastest part that haven't fill up
a page.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in">When rc<n, I add a pri=
ntf to notice this, then I find it happens frequently.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in">Luckily, this hasn't lead=
to a bug because rocksdb not use the returned Slice "result", ro=
cksdb use the data in buf "scratch" directly, but I still think t=
his should be modify to prevent future mistake.<o:p></o:p></p>
</div>
<div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in"><span style=3D"font-size:=
9.0pt">---Original---<o:p></o:p></span></p>
</div>
<div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in;background:#F0F0F0"><b><sp=
an style=3D"font-size:9.0pt;color:#212121">From:</span></b><span style=3D"f=
ont-size:9.0pt;color:#212121"> "Harris, James R"<james.r.harri=
[email protected]><o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in;background:#F0F0F0"><b><sp=
an style=3D"font-size:9.0pt;color:#212121">Date:</span></b><span style=3D"f=
ont-size:9.0pt;color:#212121"> 2023/4/8 02:32:19<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in;background:#F0F0F0"><b><sp=
an style=3D"font-size:9.0pt;color:#212121">To:</span></b><span style=3D"fon=
t-size:9.0pt;color:#212121"> "</span><span style=3D"font-size:9.0pt;fo=
nt-family:"PingFang TC",sans-serif;color:#212121">=BA=AB</span><s=
pan style=3D"font-size:9.0pt;font-family:"MS Gothic";color:#21212=
1">=B9=E2=D1=F4</span><span style=3D"font-size:9.0pt;color:#212121">"&=
lt;[email protected]>;"spdk"<[email protected]=
tion.org>;<o:p></o:p></span></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in;background:#F0F0F0"><b><sp=
an style=3D"font-size:9.0pt;color:#212121">Subject:</span></b><span style=
=3D"font-size:9.0pt;color:#212121"> Re: [SPDK] question about env_spdk.cc<o=
:p></o:p></span></p>
</div>
</div>
<p class=3D"MsoNormal" style=3D"margin-left:.5in"><o:p> </o:p></p>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:.5in">
This does look like a bug. But only way it can trigger is if SpdkRand=
omAccessFile::Read is called with length + offset that goes past the end of=
the file. RocksDB won=A1=AFt actually do that though.<o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:.5in">
<o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:.5in">
<o:p></o:p></p>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:.5in">
<o:p></o:p></p>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in =
0in 0in">
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:0in;margin-right:0in;mar=
gin-bottom:12.0pt;margin-left:1.0in">
<b>From: </b>SPDK <[email protected]> on behalf =
of <span style=3D"font-family:"PingFang TC",sans-serif">
=BA=AB</span><span style=3D"font-family:"MS Gothic"">=B9=E2=D1=F4=
</span> <[email protected]><br>
<b>Date: </b>Friday, April 7, 2023 at 10:23 AM<br>
<b>To: </b>spdk <[email protected]><br>
<b>Subject: </b>[SPDK] question about env_spdk.cc<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
Recently I am trying to team spdk with Rocksdb, the code in env_spdk.=
cc seems having a bug.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
In func SpdkRandomAccessFile::Read, the func spdk_file_read returns the num=
ber of bytes successfully read from file, then the buffer contains those by=
tes are used to initialize the result Slice, however, the size if "n&q=
uot; instead of rc, I think it is confusing.
<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
As a matter of fact, in the same func SpdkSequentialFile::Read, the lenth t=
o initialize result slice is rc, not n.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
I have tried to modify the n to rc, things seem to have no change, which is=
more confusing.<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
<o:p></o:p></p>
</div>
<div>
<p class=3D"MsoNormal" style=3D"mso-margin-top-alt:auto;mso-margin-bottom-a=
lt:auto;margin-left:1.0in">
Hope someone can help me, thanks you!<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>
--_000_BYAPR11MB33038C78632CD67D265C4CE7DE959BYAPR11MB3303namp_--