Your Android phone can store information in two places : memory (RAM) and persistent storage (an SD card for instance).
SWAP is, in short, virtual RAM.
With swap, a small portion of the hard drive is set aside and used like RAM. The computer will attempt to keep as much information as possible in RAM until the RAM is full. At that point, the computer will begin moving inactive blocks of memory (called pages) to the hard disk, freeing up RAM for active processes.
If one of the pages on the hard disk needs to be accessed again, it will be moved back into RAM, and a different inactive page in RAM will be moved onto the hard disk ('swapped'). The trade off is disks and SD cards are considerably slower than physical RAM, so when something needs to be swapped, there is a noticeable performance hit.
Unlike traditional swap, Android's Memory Manager kills inactive processes to free up memory. Android signals to the process, then the process will usually write out a small bit of specific information about its state (for example, Google Maps may write out the map view coordinates; Browser might write the URL of the page being viewed) and then the process exits. When you next access that application, it is restarted: the application is loaded from storage, and retrieves the state information that it saved when it last closed.
In some applications, this makes it seem as if the application never closed at all. This is not much different from traditional swap, except that Android apps are specially programmed to write out very specific information, making Android's Memory Manager more efficient that swap..