NetBSD Project: Auto create swap on memory pressure

Soumyajyoti Sarkar <[email protected]>
Newsgroups gmane.os.netbsd.devel.kernel
Message-ID <CAMBT70U=OAGiucXnb0WwS1F6rJDxGOHn_-nE1XunVM_pvd+Jtg@mail.gmail.com>
Dear Tech-Kern,

I was interested in participating in GSOC 2025 under the NetBSD project. I
have been using NetBSD for quite some time now, and I am actively involved
in the NetBSD community. Over the course of my experience, I have developed
a keen understanding of the system and have been experimenting with various
improvements and scripts that enhance its functionality.

One such initiative I have been working on is an automated swap file
management system for NetBSD, which I have successfully implemented and
tested on my system. The idea behind the script is to create and manage
swap files dynamically based on memory usage, while also considering disk
space constraints. The script includes features like memory usage
thresholds, disk space checks, swap file encryption, and logging for system
monitoring.

*I have attached a prototype of this script for your review. *I believe
that this project aligns well with the goals of GSOC 2025 and could
significantly contribute to improving memory management in NetBSD. In
addition to the current functionality, there are several enhancements I am
considering, including:

   - *Advanced Swap File Optimization*: Allowing users to set a more
   dynamic configuration for swap file creation (e.g., adjusting the swap file
   size based on system load or available resources).
   - *Integration with NetBSD's Resource Management Framework*:
   Investigating how the script could be integrated with NetBSD’s existing
   resource management mechanisms for a more seamless experience.
   - *Support for Multiple Swap Locations*: Enabling the script to handle
   swap files on multiple disks or partitions to improve performance and fault
   tolerance.
   - *Swap File Monitoring and Alerts*: Introducing a more comprehensive
   monitoring system to alert users if swap file usage exceeds certain
   thresholds or if there are issues with disk space or memory.

I have been testing the script extensively on my own system, and it has
shown promising results so far. However, I am looking for further feedback
and suggestions to refine the implementation.

Some specific questions I have regarding the implementation are:

   1.

   *Integration with NetBSD's Swap System*: Are there any preferred or
   recommended methods for integrating the swap file management directly into
   NetBSD’s existing swap system? I want to make sure it fits well with
   NetBSD’s resource management architecture.
   2.

   *Error Handling*: The script currently handles disk space and memory
   usage checks, but I would like to explore further how we can make the error
   handling more robust. Would it be helpful to include additional logging or
   notifications for specific errors, such as swap file creation failures or
   insufficient disk space?
   3.

   *Security Considerations*: I’ve included an option for swap file
   encryption in the script. Would it be valuable to include additional
   encryption options, or are there specific encryption standards preferred
   within NetBSD?
   4.

   *Performance Concerns*: Since the script dynamically creates and deletes
   swap files based on memory usage, how can we ensure that this process
   doesn’t cause significant performance overhead or thrashing? I would
   appreciate your insights on optimizing this.

I would love to hear your thoughts on the prototype, as well as any
feedback you may have on further improving the script. I am eager to
contribute to this project during GSOC 2025 and would be grateful for any
guidance or suggestions you can provide.

I am looking forward to your feedback and the possibility of working on
this project.

Best regards,

Soumyajyoti Sarkar.
swap_manager.sh (application/octet-stream, 4 KB)
#!/bin/bash

# Configurable parameters
MAX_SWAP_SIZE=80                # Max space usage as percentage of disk
MIN_FREE_SPACE=10               # Minimum free space percentage on disk
MEMORY_THRESHOLD=80             # Memory usage threshold for swap creation
SWAP_FILE="/swapfile"           # Swap file location
LOG_FILE="/var/log/swap_manager.log"  # Log file location
SWAP_SIZE_INCREMENT=1024        # Size increment for swap file (in MB)
THROTTLE_TIMEOUT=300            # Timeout in seconds between swap creations (5 minutes)

# Log function
log_message() {
    local message=$1
    local current_time=$(date '+%Y-%m-%d %H:%M:%S')
    echo "$current_time - $message" >> $LOG_FILE
}

# Monitor and create swap file if needed
create_swap() {
    log_message "Checking disk and memory usage..."

    # Get disk space usage and free memory
    FREE_SPACE=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')
    FREE_MEMORY=$(vmstat -s | grep "free memory" | awk '{print $1}')
    TOTAL_MEMORY=$(sysctl -n hw.physmem)
    AVAILABLE_MEMORY=$(($TOTAL_MEMORY - $FREE_MEMORY))

    # Check if enough disk space is available
    if [ "$FREE_SPACE" -ge "$MAX_SWAP_SIZE" ]; then
        log_message "Disk space is too low. Swap creation aborted."
        return 1
    fi

    # Check if memory usage exceeds the threshold
    if [ $(($AVAILABLE_MEMORY * 100 / $TOTAL_MEMORY)) -ge "$MEMORY_THRESHOLD" ]; then
        log_message "Memory usage is above threshold, creating swap file..."
        
        # Dynamically calculate swap file size based on memory usage (1 GB per 10% memory usage above threshold)
        SWAP_SIZE=$((($AVAILABLE_MEMORY * 100 / $TOTAL_MEMORY - $MEMORY_THRESHOLD) * $SWAP_SIZE_INCREMENT / 10))
        if [ "$SWAP_SIZE" -lt "$SWAP_SIZE_INCREMENT" ]; then
            SWAP_SIZE=$SWAP_SIZE_INCREMENT
        fi

        # Check if a swap file already exists, remove it
        if [ -f $SWAP_FILE ]; then
            log_message "Swap file $SWAP_FILE exists. Removing existing swap file..."
            swapoff $SWAP_FILE
            rm -f $SWAP_FILE
        fi
        
        # Create swap file and set up swap
        dd if=/dev/zero of=$SWAP_FILE bs=1M count=$SWAP_SIZE status=progress
        mkswap $SWAP_FILE
        swapon $SWAP_FILE
        log_message "Created swap file of size $SWAP_SIZE MB at $SWAP_FILE."
    else
        log_message "Memory usage below threshold, no swap file needed."
    fi
}

# Remove swap file if memory usage is low
remove_swap() {
    # Check if swap file is in use
    SWAP_USAGE=$(swapon -s | grep $SWAP_FILE)
    if [ -z "$SWAP_USAGE" ]; then
        log_message "No swap file in use. Cleanup skipped."
    else
        log_message "Memory usage has decreased. Removing swap file..."
        swapoff $SWAP_FILE
        rm -f $SWAP_FILE
        log_message "Removed swap file $SWAP_FILE."
    fi
}

# Check disk space before proceeding
check_disk_space() {
    DISK_USAGE=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')
    if [ "$DISK_USAGE" -ge "$MAX_SWAP_SIZE" ]; then
        log_message "Disk usage too high ($DISK_USAGE%). Aborting swap file creation."
        return 1
    fi
    return 0
}

# Rate limiting for swap file creation to avoid thrashing
throttle_check() {
    last_creation=$(stat -c %Y $LOG_FILE)
    current_time=$(date +%s)
    elapsed_time=$((current_time - last_creation))
    if [ "$elapsed_time" -lt "$THROTTLE_TIMEOUT" ]; then
        log_message "Throttling swap creation. Elapsed time: $elapsed_time seconds."
        return 1
    fi
    return 0
}

# Main control loop for swap management
manage_swap() {
    log_message "Starting swap management..."

    # Check disk space and memory, and create swap if necessary
    if check_disk_space && throttle_check; then
        create_swap
    else
        log_message "Swap creation skipped due to insufficient disk space or throttling."
    fi

    # Optionally, clean up swap if memory usage is back to normal
    remove_swap
}

# Run the script
manage_swap
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.