"I was thinking that maybe a high build primer would fill what gap there still was, and save a couple of ounces and a little hassle."

I wouldn't count on it. Paint tends to make flaws show up even more. Also, the CLC book "The Kayak Shop" says (if I remember right) high-hide paints are bulked up with the equivalent of really fine talcum powder, which makes them lack durability in a big way.