LR44 type button cells (a.k.a. AG13, etc.) are ubiquitous and dirt-cheap, but for some reason, it's hard to find ready-made holders for them.

This holds 3 cells in series to provide a nominal 4.5 Volts. For PCB mounting the lead spacing is 0.8 inches.

To form the battery contacts (do this on each side):
Take a piece of bare copper wire or a scrap component lead and stick one end through the top hole. Bend it down/back and loop it into the bottom hole, then fold the excess down into the channel. Fold the other end down over it, and pull out any slack. Trim off the excess from the folded-under end. Leave enough length on the folded-over end to go through the PCB.