[PATCH] Ake Sandgren: Fix RLIMIT_RSS madvise calculation bug

ChangeSet 1.1578, 2005/01/27 08:06:04-02:00, [email protected]

	[PATCH] Ake Sandgren: Fix RLIMIT_RSS madvise calculation bug
	
	On Wed, Jan 26, 2005 at 12:07:50PM +0100, Ake wrote:
	> Use of rlim[RLIMIT_RSS] in mm/filemap.c is wrong.
	> It is passed down to kernel as a number of bytes but is being used as a
	> number of pages.
	>
	> There is also a misinformative comment in fs/proc/array.c
	> in proc_pid_stat where it says
	> mm ? mm->rss : 0, /* you might want to shift this left 3 */
	> the number 3 should probably be PAGE_SHIFT-10.
	
	Amazing that this has never been noticed before - I bet not many people use RSS
	limits with madvise().
	
	This transform the rlimit in pages before the comparison, can you please test
	it.



 filemap.c |    3 +++
 1 files changed, 3 insertions(+)


diff -Nru a/mm/filemap.c b/mm/filemap.c
--- a/mm/filemap.c	2005-01-30 15:36:41 -08:00
+++ b/mm/filemap.c	2005-01-30 15:36:41 -08:00
@@ -2609,6 +2609,9 @@
 	error = -EIO;
 	rlim_rss = current->rlim ?  current->rlim[RLIMIT_RSS].rlim_cur :
 				LONG_MAX; /* default: see resource.h */
+
+	rlim_rss = (rlim_rss & PAGE_MASK) >> PAGE_SHIFT;
+
 	if ((vma->vm_mm->rss + (end - start)) > rlim_rss)
 		return error;