Re: [PATCH] ovl: Allow changing default fsync_mode

On 2026/6/23 17:59, Yafang Shao wrote:
> On Tue, Jun 23, 2026 at 5:49 PM Gao Xiang <[email protected]> wrote:
>>
>>
>>
>> On 2026/6/23 17:34, Yafang Shao wrote:
>>> On Tue, Jun 23, 2026 at 5:25 PM Gao Xiang <[email protected]> wrote:
>>>>
>>>>
>>>>
>>>> On 2026/6/23 17:15, Yafang Shao wrote:
>>>>> On Tue, Jun 23, 2026 at 5:00 PM Gao Xiang <[email protected]> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2026/6/23 16:43, Yafang Shao wrote:
>>>>>>> We have enabled "volatile" fsync_mode on our Kubernetes production
>>>>>>> environment to prevent container exit from being blocked when there
>>>>>>> are many dirty pages to flush. This has worked well without introducing
>>>>>>> any issues.
>>>>>>>
>>>>>>> However, on some of our production servers, upgrading the container
>>>>>>> runtime to support the "volatile" mount option is not straightforward [0].
>>>>>>> To address this, we want to enable it by default within the kernel.
>>>>>>
>>>>>> Just a side note: "upgrade the container runtime is not
>>>>>> straightforward", how? it seems that issue is already resolved and
>>>>>> there is no more discussion.
>>>>>
>>>>> We still have many production servers running Docker, while the
>>>>> "volatile" mount option is only supported by containerd. Upgrading
>>>>> from Docker to containerd is a difficult process.
>>>>
>>>> But docker can be patched too: if upgrading the userspace is
>>>> hard, why upgrading the linux kernel is easy?
>>>
>>> It is quite easy since the kernel can be livepatched without
>>> rebooting. My employer is a heavy livepatch user. [1]
>>>
>>> [1]. https://lore.kernel.org/live-patching/
>>
>> It's just a generic opinion, in general, docker can be live
>> upgraded without pausing the containers, and upgrading
>> userspace is easier / safer than patching the kernel.
> 
> Are you sure? How would one change an already mounted overlayfs that
> was not enabled with "volatile"?
> 
>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> Not quite sure applying a default volatile policy is quite feasible,
>>>>>> especially the issue documented in
>>>>>> https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33
>>>>>>
>>>>>> then userspace cannot drop `volatile` option as a somewhat
>>>>>> workaround now.
>>>>>
>>>>> OS vendors can still set "auto" as the default config, while customers
>>>>> can override it dynamically via sysfs. We have been running with
>>>>> "volatile" on many production servers across different workloads for
>>>>> over a year, and it has worked as expected without any issues.
>>>>
>>>> but sysfs setting still applies as system-wide, and there
>>>> are some edge cases that we cannot apply volatile as
>>>> default, that is my one concern.
>>>
>>> It is unclear whether there are mixed workloads on the same server
>>> that require both "volatile" and "strict" modes, but we have not
>>> encountered such use cases across our large fleet of servers.
>>
>> At least containerd needs to strip out `volatile` in some use cases,
>> again see:
>> https://github.com/containerd/containerd/pull/9555
>> https://github.com/containerd/containerd/pull/10274/files#diff-9239161e2af83fd84df5792f9fe64701c517fe4598eae60d4d245d039955f46cR33
>>
>> So set `volatile` as default will break userspace (containerd),
>> and containerd needs to add another mount option to avoid the
>> default `volatile` behavior, which is messy and makes the
>> userspace more harder.
> 
> However, our userspace is easier with it ;)

It depends on how you define "our", if your docker can be
upgraded, why bothering kernel?

As I said, common existing applications (like containerd) will
break since they already assume "OVL_FSYNC_AUTO" is the default
behavior, and explicitly use mount options to override the
default "OVL_FSYNC_AUTO" behavior.

But with the patch, it just breaks the application assumption
and userspace application needs to change again.

> 
>>
>>>
>>>>
>>>> The other concern is that since `volatile` omits fsync, so
>>>> it's a posix violation (even that makes sense for container
>>>> writable layers), not sure if we have to use it as the
>>>> system-wide default configuration too.
>>>
>>> We have use cases for setting this as a system-wide configuration. It
>>> is unclear whether others have similar needs.
>>
>> While I cannot speak out of overlayfs, but really it depends
>> on if the use cases is generic.
>>
>> Usually filesystems need to obey posix semantics as much as
>> possible, and using specific mount option to relax (violate)
>> some restriction, but it shouldn't be a system-wide stuff
>> since otherwise user applications cannot know if they really
>> live in the posix world.
> 
> AFAICS, the Linux kernel does not strictly follow POSIX in a number of areas.
> 
> BTW,
> 1. The "volatile" is not the default option.
> 2. Even when the system-wide default is set to "volatile", users can
> still change it per mount.

Here, I'm strongly against this, again: especially it
just breaks application which implies "OVL_FSYNC_AUTO" is
the default, and we shouldn't ask userspace to handle this
to deal with the fact that the "volatile" could become
the default again and again.

Thanks,
Gao Xiang

>