No, just seal up the spots that need it. The primed areas shouldn't be noticeable once painted as long as you didn't apply too much primer too thick. Always try to use the same method to apply the primer as was used to paint the wall, example- with a roller, and feather out the primer and paint.