Re: System lockups in linux prevent productive work.

Hi Greg,
	The real question is why is your system behaving that way. Without
answering that question it's hard to tell you what to do to fix it. I
can hazard a guess.

	The behavior you describe sounds like something is using a lot of
memory and you're swapping.

Your system has a fixed amount of RAM. That has to be used by the
kernel, all the applications you're running, and any libraries those
applications use. Some of it can be shared. For example if two
applications use the same library Linux is smart enough to only load it
once and both share it.

If your system needs more memory than you have RAM, it uses "virtual
memory." It takes a chunk of your disk (referred to as swap) and adds
that to your RAM. First it uses all the RAM. Then when you start another
application that needs RAM, it picks some of the RAM and copies that to
disk to make room. If it later needs to use that memory it moves
something else to disk and copies that chunk back in to memory. That
process of copying memory between RAM and disk is called swapping.

That's great, because it lets you keep working even when you're out of
RAM, but the problem is that disk is about 100 times slower than RAM. So
if you're copying stuff back and forth it's really slow.

Linux will also try to use any extra RAM to it's advantage. So if
there's extra RAM that applications aren't using it'll keep the files
you used recently and store those in RAM. That way the next time you
need to access them, they'll already be in RAM and be very fast. That's
why you'll rarely see much "free" memory in a Linux system.

I've simplified this description quite a bit, but it gives you the right
idea.

What's probably happening is that the render over night is using a lot
of RAM. Since you're not using all those other applications, they get
"swapped out" to disk. Then when you come in the next morning all those
applications have to be copied back from disk to RAM before they can
run. That takes a while and you notice everything is sluggish. If the
render is finished (and all the RAM it was using is released) then the
system would eventually come back to normal (without a reboot). But that
takes a while.

So, the first question I would ask is: When you come back to the system
in the morning and start working do you see the disk drive light
flashing a bunch? That would match up with swapping.

The next more detailed thing to check is to bring up a shell and run
"top" or see if you have gtop as an application. You can run that before
you start your render and leave it running over night or try to start it
during the morning.

Top monitors your system and tells you about CPU usage, memory usage,
and what applications are running. The first few lines look something
like this:

top - 09:53:07 up 17:00,  7 users,  load average: 0.48, 0.20, 0.11
Tasks: 119 total,   1 running, 117 sleeping,   0 stopped,   1 zombie
Cpu(s): 3.0% us, 1.0% sy, 0.0% ni, 96.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 709800k total, 702056k used, 7744k free, 84032k buffers
Swap: 917496k total, 8k used, 917488k free, 212308k cached

PID  USER   PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
2122 daryll 15   0 84072  14m 9320 S  2.3  2.1   0:03.44 gnome-terminal
1999 root   15   0  127m  45m  10m S  1.3  6.6   5:57.22 Xorg
   1 root   16   0  2000  680  584 S  0.0  0.1   0:00.63 init

Let's go through the entries:

09:53:07 up 17:00 -- The current time and my laptop has been up 17 hours
7 users -- Not really users, but how many login sessions. Ignore this
load average -- This is how many processes are trying to use your CPU.
The first number is your current load and the others are averages over
longer periods of time. This first number is important. If it is high,
it means your CPU is busy.
Tasks -- That whole line tells you how many processes are running and
what they are trying to do. I've got one active one and bunch that
aren't doing anything at the moment.
Cpu(s) -- This is a summary of what your CPUs are doing. If the us
(user) and sy (system) numbers are high your system is busy. If the id
(idle) number is high your system is idle. If the wa (wait) number is
high your system is probably reading/writing disks.
Mem -- Your system memory usage. This and swap is important for your
case.
  I have a total 768MB in my laptop.
  I'm using 700MB of it right now. 
  7MB is free (unused) at the moment.
  84MB is being used for buffers.
Swap -- Is about the disk space used as memory
  I have 917MB total swap space
  I'm using 8k of it. 
  917MB is free
  212MB is cached

Finally we get a report of the processes that are using the most CPU.
  PID -- The process ID number
  USER -- The user running the process
  PR -- The priority of the process (influences how much CPU Linux gives
it)
  NI -- The nice value. It changes the priority. You use it to tell
Linux how much priority to give it.
  VIRT -- The total amount of memory the application is using. This
includes libraries and such so it's sort of a complicated number.
  RES -- This is the resident set size. This is how much memory the
application needs right now. Think of it as the memory usage.
  SHR -- Shared memory. How much memory is shared with other
applications on the sytem.
  S -- The state the application is in. (R)unning, (S)leeping, (D)isk,
etc.
  CPU -- This is how much of the total CPU the application is using. 
  MEM -- This is how much of the total Memory the application is using.
  TIME -- How long it is running.
  COMMAND -- Finally the command itself.

Whew that's a lot of details!!! It's almost everything you need to know
about what your system is doing.

So if I'm right and your system is swapping you will see that a your CPU
is spending a large percentage of it's time in wa (wait) state. You'll
see a lot of your swap space is in use. If you look at the application
lists, you might see several programs with big numbers in their RES
(resident set size) field.

In that case, the solution to your problem is easy. Buy more RAM. 

How much? If you press the '>' key, top will sort the list of
applications by memory usage instead of CPU usage. Then you need to
start running your blender renders and then add up the RES values for
all the items on your top display. That's the minimum amount of RAM you
need. You'd really like more than that. Probably twice that much to be
safe. Remember Linux will use the extra RAM to make things faster, so
it's always good to have lots of RAM.

This is really long, but I figured it might be useful to other people to
understand what's going on in their systems.

						- |Daryll

-- 
Daryll Strauss                  phone: 310-733-3001 x201
Digital Ordnance                email: [email protected]
1724 Rockefeller Lane           web: http://www.digitalordnance.com
Redondo Beach, CA 90278




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642