Purge Old Uploads?

Is it OK to purge the "Uploads" folder periodically?

When you upload something and add it to an album, does it get copied into the "Albums" directory or does it stay in "Uploads"

I'm trying to clean up my server a bit. I have over 14 GB in "Albums", but I noticed "Uploads" has over 2 GB in it. I'd like to recover that space if possible.