NetBSD Project: Auto create swap on memory pressure
Soumyajyoti Sarkar <[email protected]>
| Newsgroups | gmane.os.netbsd.devel.kernel |
|---|---|
| Message-ID | <CAMBT70U=OAGiucXnb0WwS1F6rJDxGOHn_-nE1XunVM_pvd+Jtg@mail.gmail.com> |
Dear Tech-Kern, I was interested in participating in GSOC 2025 under the NetBSD project. I have been using NetBSD for quite some time now, and I am actively involved in the NetBSD community. Over the course of my experience, I have developed a keen understanding of the system and have been experimenting with various improvements and scripts that enhance its functionality. One such initiative I have been working on is an automated swap file management system for NetBSD, which I have successfully implemented and tested on my system. The idea behind the script is to create and manage swap files dynamically based on memory usage, while also considering disk space constraints. The script includes features like memory usage thresholds, disk space checks, swap file encryption, and logging for system monitoring. *I have attached a prototype of this script for your review. *I believe that this project aligns well with the goals of GSOC 2025 and could significantly contribute to improving memory management in NetBSD. In addition to the current functionality, there are several enhancements I am considering, including: - *Advanced Swap File Optimization*: Allowing users to set a more dynamic configuration for swap file creation (e.g., adjusting the swap file size based on system load or available resources). - *Integration with NetBSD's Resource Management Framework*: Investigating how the script could be integrated with NetBSD’s existing resource management mechanisms for a more seamless experience. - *Support for Multiple Swap Locations*: Enabling the script to handle swap files on multiple disks or partitions to improve performance and fault tolerance. - *Swap File Monitoring and Alerts*: Introducing a more comprehensive monitoring system to alert users if swap file usage exceeds certain thresholds or if there are issues with disk space or memory. I have been testing the script extensively on my own system, and it has shown promising results so far. However, I am looking for further feedback and suggestions to refine the implementation. Some specific questions I have regarding the implementation are: 1. *Integration with NetBSD's Swap System*: Are there any preferred or recommended methods for integrating the swap file management directly into NetBSD’s existing swap system? I want to make sure it fits well with NetBSD’s resource management architecture. 2. *Error Handling*: The script currently handles disk space and memory usage checks, but I would like to explore further how we can make the error handling more robust. Would it be helpful to include additional logging or notifications for specific errors, such as swap file creation failures or insufficient disk space? 3. *Security Considerations*: I’ve included an option for swap file encryption in the script. Would it be valuable to include additional encryption options, or are there specific encryption standards preferred within NetBSD? 4. *Performance Concerns*: Since the script dynamically creates and deletes swap files based on memory usage, how can we ensure that this process doesn’t cause significant performance overhead or thrashing? I would appreciate your insights on optimizing this. I would love to hear your thoughts on the prototype, as well as any feedback you may have on further improving the script. I am eager to contribute to this project during GSOC 2025 and would be grateful for any guidance or suggestions you can provide. I am looking forward to your feedback and the possibility of working on this project. Best regards, Soumyajyoti Sarkar.
swap_manager.sh
(application/octet-stream, 4 KB)
#!/bin/bash
# Configurable parameters
MAX_SWAP_SIZE=80 # Max space usage as percentage of disk
MIN_FREE_SPACE=10 # Minimum free space percentage on disk
MEMORY_THRESHOLD=80 # Memory usage threshold for swap creation
SWAP_FILE="/swapfile" # Swap file location
LOG_FILE="/var/log/swap_manager.log" # Log file location
SWAP_SIZE_INCREMENT=1024 # Size increment for swap file (in MB)
THROTTLE_TIMEOUT=300 # Timeout in seconds between swap creations (5 minutes)
# Log function
log_message() {
local message=$1
local current_time=$(date '+%Y-%m-%d %H:%M:%S')
echo "$current_time - $message" >> $LOG_FILE
}
# Monitor and create swap file if needed
create_swap() {
log_message "Checking disk and memory usage..."
# Get disk space usage and free memory
FREE_SPACE=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')
FREE_MEMORY=$(vmstat -s | grep "free memory" | awk '{print $1}')
TOTAL_MEMORY=$(sysctl -n hw.physmem)
AVAILABLE_MEMORY=$(($TOTAL_MEMORY - $FREE_MEMORY))
# Check if enough disk space is available
if [ "$FREE_SPACE" -ge "$MAX_SWAP_SIZE" ]; then
log_message "Disk space is too low. Swap creation aborted."
return 1
fi
# Check if memory usage exceeds the threshold
if [ $(($AVAILABLE_MEMORY * 100 / $TOTAL_MEMORY)) -ge "$MEMORY_THRESHOLD" ]; then
log_message "Memory usage is above threshold, creating swap file..."
# Dynamically calculate swap file size based on memory usage (1 GB per 10% memory usage above threshold)
SWAP_SIZE=$((($AVAILABLE_MEMORY * 100 / $TOTAL_MEMORY - $MEMORY_THRESHOLD) * $SWAP_SIZE_INCREMENT / 10))
if [ "$SWAP_SIZE" -lt "$SWAP_SIZE_INCREMENT" ]; then
SWAP_SIZE=$SWAP_SIZE_INCREMENT
fi
# Check if a swap file already exists, remove it
if [ -f $SWAP_FILE ]; then
log_message "Swap file $SWAP_FILE exists. Removing existing swap file..."
swapoff $SWAP_FILE
rm -f $SWAP_FILE
fi
# Create swap file and set up swap
dd if=/dev/zero of=$SWAP_FILE bs=1M count=$SWAP_SIZE status=progress
mkswap $SWAP_FILE
swapon $SWAP_FILE
log_message "Created swap file of size $SWAP_SIZE MB at $SWAP_FILE."
else
log_message "Memory usage below threshold, no swap file needed."
fi
}
# Remove swap file if memory usage is low
remove_swap() {
# Check if swap file is in use
SWAP_USAGE=$(swapon -s | grep $SWAP_FILE)
if [ -z "$SWAP_USAGE" ]; then
log_message "No swap file in use. Cleanup skipped."
else
log_message "Memory usage has decreased. Removing swap file..."
swapoff $SWAP_FILE
rm -f $SWAP_FILE
log_message "Removed swap file $SWAP_FILE."
fi
}
# Check disk space before proceeding
check_disk_space() {
DISK_USAGE=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')
if [ "$DISK_USAGE" -ge "$MAX_SWAP_SIZE" ]; then
log_message "Disk usage too high ($DISK_USAGE%). Aborting swap file creation."
return 1
fi
return 0
}
# Rate limiting for swap file creation to avoid thrashing
throttle_check() {
last_creation=$(stat -c %Y $LOG_FILE)
current_time=$(date +%s)
elapsed_time=$((current_time - last_creation))
if [ "$elapsed_time" -lt "$THROTTLE_TIMEOUT" ]; then
log_message "Throttling swap creation. Elapsed time: $elapsed_time seconds."
return 1
fi
return 0
}
# Main control loop for swap management
manage_swap() {
log_message "Starting swap management..."
# Check disk space and memory, and create swap if necessary
if check_disk_space && throttle_check; then
create_swap
else
log_message "Swap creation skipped due to insufficient disk space or throttling."
fi
# Optionally, clean up swap if memory usage is back to normal
remove_swap
}
# Run the script
manage_swap