The cool thing about a catering biz is you can start with really low budget and do it "on the side" while you keep your regular job (and paycheck and insurance)
The trick is getting your first few customers and doing such a great job that they are great references for you.
Think about what kind of events you'd like to cater. What kind of experience do you have? How big of a deal could you handle (solo? hiring help that you don't know? friends?)
Start talking to friends and acquaintances, tell everyone what you're thinking of doing, ask if they know anyone who has family parties or anniversaries or office parties or low-key weddings, and let them know you'd be willing to do it for just the food cost if they'll be a reference for you (but only if you do a great job, of course!)
Remember you can rent a lot of the stuff you need from party supply stores. Maybe take a part time job with one of them, or at a big hotel so you can figure out the logistics of catering a big party.
Just don't go blowing your savings and quitting your job on useless stuff like brochures and biz cards and trucks and an office... save your cash and learn as you go, you're far more likely to succeed if your mistakes are small.
p.s. also check out these resources:
your local library