Re: [PATCH v2] ceph: revalidate ki_pos for O_APPEND writes after cap acquisition

On Tue, 21 Jul 2026 at 01:38, Viacheslav Dubeyko <[email protected]> wrote:
>
> On Sat, 2026-07-18 at 10:29 +0800, Xiubo Li via B4 Relay wrote:
> > From: Xiubo Li <[email protected]>
> >
> > For O_APPEND writes, ki_pos is set to the current EOF via
> > generic_write_checks() after fetching i_size from the MDS.  However,
> > ceph_get_caps() may need to wait for Fwx exclusive caps if the write
> > extends the file (endoff > i_max_size).  While waiting for Fwx, the
> > previous Fwx holder (another client) may have already extended the
> > file.  When the MDS grants us Fwx, the cap grant message updates the
> > local i_size, but ki_pos remains at the old EOF, causing the append
> > write to land at a stale offset and overwrite data from the other
> > client.
> >
> > Fix by re-reading i_size_read(inode) after ceph_get_caps() returns.
> > At this point we hold Fwx exclusive caps, no other client can modify
> > the file, and i_size reflects the true EOF from the MDS cap grant.
> > No extra MDS round-trip is needed.  Only adjust ki_pos when the EOF
> > has actually changed.
> >
> > After adjusting ki_pos forward, the write range [pos, pos+count) may
> > now exceed the i_max_size that was validated by ceph_get_caps() for
> > the old range.  Re-check against i_max_size and truncate the write
> > if necessary to stay within the MDS-granted limit.
> >
> > Fixes: 8e4473bb50a1 ("ceph: do not execute direct write in parallel
> > if O_APPEND is specified")
> > Link: https://tracker.ceph.com/issues/7333
> > Signed-off-by: Xiubo Li <[email protected]>
> > ---
> > Changes in v2:
> > - Re-check write range against i_max_size after adjusting ki_pos
> > forward, and truncate if it exceeds the MDS-granted limit.(Viacheslav
> > Dubeyko)
> > - Link to v1:
> > https://patch.msgid.link/[email protected]
> > ---
> >  fs/ceph/file.c | 42 ++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 42 insertions(+)
> >
> > diff --git a/fs/ceph/file.c b/fs/ceph/file.c
> > index 9d89d7fc1095..a2b4d261e23c 100644
> > --- a/fs/ceph/file.c
> > +++ b/fs/ceph/file.c
> > @@ -2491,6 +2491,48 @@ static ssize_t ceph_write_iter(struct kiocb
> > *iocb, struct iov_iter *from)
> >       if (err < 0)
> >               goto out;
> >
> > +     /*
> > +      * For O_APPEND writes we may have waited for Fwx exclusive
> > caps
> > +      * while the previous Fwx holder (another client) extended
> > the
> > +      * file.  i_size has been updated via the cap grant message
> > from
> > +      * the MDS, but ki_pos is still the old EOF.  Re-read i_size
> > here
> > +      * (no extra MDS round-trip needed) and adjust ki_pos to the
> > true
> > +      * EOF.  Since we hold Fwx, no other client can change the
> > file.
> > +      */
> > +     if (iocb->ki_flags & IOCB_APPEND) {
> > +             loff_t cur_eof = i_size_read(inode);
> > +
> > +             if (cur_eof != pos) {
> > +                     doutc(cl,
> > +                           "%p %llx.%llx O_APPEND: pos adjusted
> > %lld -> %lld\n",
> > +                           inode, ceph_vinop(inode), pos,
> > cur_eof);
> > +                     iocb->ki_pos = cur_eof;
> > +                     pos = cur_eof;
> > +                     if (pos >= limit) {
> > +                             err = -EFBIG;
> > +                             goto out_caps;
> > +                     }
> > +                     iov_iter_truncate(from, limit - pos);
> > +                     count = iov_iter_count(from);
> > +
> > +                     /*
> > +                      * ceph_get_caps() validated the old endoff
> > +                      * against i_max_size; adjusting ki_pos
> > forward
> > +                      * may have shifted the write range beyond
> > the
> > +                      * granted max_size.  Re-check and truncate
> > if
> > +                      * necessary.
> > +                      */
> > +                     if (pos + count > (loff_t)ci->i_max_size) {
>
> If I am not wrong, every access of i_max_size is done under ci-
> >i_ceph_lock. Should we use this lock here?
>

Hi Slava,

Good question. Actually this is a benign race: after ceph_get_caps()
returns with Fwx, the MDS won't decrease our i_max_size without first
revoking Fwx (which can't happen while we hold a cap reference). So
i_max_size can only increase or stay the same. The worst case: we read
a stale smaller value and unnecessarily truncate the write. No data
corruption.

To maintain consistency with other locations, I will improve this anyway.

Thanks
- Xiubo


> Thanks,
> Slava.
>
> > +                             if (pos >= (loff_t)ci->i_max_size) {
> > +                                     err = -EFBIG;
> > +                                     goto out_caps;
> > +                             }
> > +                             iov_iter_truncate(from, ci-
> > >i_max_size - pos);
> > +                             count = iov_iter_count(from);
> > +                     }
> > +             }
> > +     }
> > +
> >       err = file_update_time(file);
> >       if (err)
> >               goto out_caps;
> >
> > ---
> > base-commit: df47791ab9f12cd9144aca477ddae195f68072a5
> > change-id: 20260717-ceph-append-fix-9820e3b1a78c
> >
> > Best regards,
> > --
> > Xiubo Li <[email protected]>
> >