Re: [BUG REPORT] missing memory counter introduced by xfs
Dave Chinner <[email protected]>
| Newsgroups | gmane.comp.file-systems.xfs.general |
|---|---|
| Message-ID | <20160907212206.GP30056@dastard> |
On Wed, Sep 07, 2016 at 06:36:19PM +0800, Lin Feng wrote: > Hi all nice xfs folks, > > I'm a rookie and really fresh new in xfs and currently I ran into an > issue same as the following link described: > http://oss.sgi.com/archives/xfs/2014-04/msg00058.html > > In my box(running cephfs osd using xfs kernel 2.6.32-358) and I sum > all possible memory counter can be find but it seems that nearlly > 26GB memory has gone and they are back after I echo 2 > > /proc/sys/vm/drop_caches, so seems these memory can be reclaimed by > slab. It isn't "reclaimed by slab". The XFS metadata buffer cache is reclaimed by a memory shrinker, which are for reclaiming objects from caches that aren't the page cache. "echo 2 > /proc/sys/vm/drop_caches" runs the memory shrinkers rather than page cache reclaim. Many slab caches are backed by memory shrinkers, which is why it is thought that "2" is "slab reclaim".... > And according to what David said replying in the list: .. > That's where your memory is - in metadata buffers. The xfs_buf slab > entries are just the handles - the metadata pages in the buffers > usually take much more space and it's not accounted to the slab > cache nor the page cache. That's exactly the case. > Minimum / Average / Maximum Object : 0.02K / 0.33K / 4096.00K > > OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME > 4383036 4383014 99% 1.00K 1095759 4 4383036K xfs_inode > 5394610 5394544 99% 0.38K 539461 10 2157844K xfs_buf So, you have *5.4 million* active metadata buffers. Each buffer will hold 1 or 2 4k pages on your kernel, so simple math says 4M * 4k + 1.4M * 8k = 26G. There's no missing counter here.... Obviously your workload is doing something extremely metadata intensive to have a cache footprint like this - you have more cached buffers than inodes, dentries, etc. That in itself is very unusual - can you describe what is stored on that filesystem and how large the attributes being stored in each inode are? Cheers, Dave. -- Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs