We've done both.
As soon as she had her social security number, we went to the bank that we bank with and opened a savings account. Birthday money plus an automatic (small) payment go in there every month. With my older children, they have piggy banks at home where they put their allowance and once a quarter we take the contents, roll the coins, and take it to the bank. I make a big deal out of the fact that they are saving for their future; they make the deposit, and then when we get home I make a matched deposit to their accounts from our account online, which they get to watch. I also let them see their interest accrue.
Which is all well and good, but it's not exactly high interest. So we've also opened all of them a 529 college savings fund which we deposit into automatically every month.
I'm not a savvy invester, but I figure the more we save anywhere, the better our chances are of having the money we need to put all three through college. Ooh, saying that made me feel a little dizzy... ;)
Congratulations on your baby!