Click the "Edit" button in the lower right hand panel (which is a square icon with a north-west pointing white arrow besides base zoom buttons). You can move and rotate all of your structures, but only Improvements can be permanently deleted once they have been built.
If you find yourself running out of room, you may expand your Base by recruiting new friends to play or by spending some Diamonds. You may also use the edit function to upgrade your defenses to higher levels.
How do I modify or remove existing buildings?