Oak has very coarse grain, and very open grain. The stain was probably not a good idea, but assuming that the stain is not a problem it's going to take you several coats with sanding in between to fill the grain on the oak before you have a smooth surface. If you run your hand across the same plywood, it will feel rough and pitted in the grain sections, probably like it does now.

m