It means that children need guidance (discipline). It doesn't mean they need to be hit with a rod.
The term "rod" does indeed refer to those used by shepherds, who guided their sheep in the direction they wanted them to go by placing the rod on one side of the sheep & nudging it or pushing it.
Like sheep, children need guidance (discipline). There are many other passages in the bible advising fathers (parents) to be kind to their children.
Train up a child in the way he should go, Even when he is old he will not depart from it. Proverbs 22:6
Fathers, do not provoke your children to anger, but bring them up in the discipline and instruction of the Lord. Ephesians 6:4
Fathers, provoke not your children, that they be not discouraged. Colossians 3:21
The great theologian John Calvin wrote on Ephesians 6:4 “Parents are exhorted not to irritate their children by unreasonable severity. This would excite hatred, & lead them to throw off the yoke altogether. Accordingly, in writing to the Colossians, he adds, “lest they be discouraged.” Kind & liberal treatment has rather a tendency to cherish reverence for their parents, & to increase the cheerfulness & activity of their obedience, while a harsh & unkind manner rouses them to obstinacy, & destroys the natural affections. But Paul goes on to say, “let them be fondly cherished;” for which unquestionably conveys the idea of gentleness & forbearance.”
Hope this is helpful.