Prime just what needs it.  If moisture is a problem use an acrylic primer suitable for  the substrate your painting.  For masonry surfaces, concrete block or stucco, there are special moisture blocking primers available at your local paint store.