8:49am  up 1 day, 18:34,  2 users,  load average: 3.37, 3.55, 3.45
143 processes: 133 sleeping, 6 running, 4 zombie, 0 stopped
CPU states: 53.8% user, 36.0% system, 10.2% nice,  0.0% idle
Mem:   514152K av,  507520K used,    6632K free,       0K shrd,   26136K buff
Swap:  264952K av,  145344K used,  119608K free                  366428K cached
It's busy !
Try 
top -i
to show just the processes that are actually running at the moment, that will give a clue as to what your server is doing (and perhaps why it is going slow).