I've heard of doctors in England (one in NYC, an ophthalmologist) leaving their profession and going into photography. I've also heard of a few attorneys, one was a woman who received a used Nikon F100 as a gift, and they've turned their stressful lives inside out and gone off into successful photography careers... while they were still learning along the way.... HOWEVER, the exception to the rule does not a rule make! There are exceptions to everything, right?
In photography, the more you know the more your chances of success and a successful career. That's why some go out of their way to go to colleges that specialize in photography. They KNOW that knowledge is earning power; the more you know about different aspects of photography, the more your chances of success... but, many don't go that route!
Depending on what you intend to do in photography. Let me try to explain that so that we're both on the same "wave length."
If you're going to be a Fashion Photographer or a Glamor Photographer or perhaps a Forensics Photographer, you may expect to know something of the field you're getting into. There ARE technical sides to it than just "takin' good pictures." The competition is a little fierce in these areas and today, most of the photographers have contacts and means of networking and teams of assistants and managers/agents, etc. Also, in Forensics, you'd have to really know the knitty-gritty of the nuts and bolts to stand up to a grueling cross-examination in a court of law if a defense attorney so chooses to do... (I've been there, and was misjudged by my "hippie" demeanor and my apparent youthfulness back in the early 70s; I made royal fools of a few over-zealous attorneys representing slumlords).
However, if you're interested in portraits, weddings, graduations, pet and/or children's photography... then, it's a matter of know how to deal with individuals, knowing how to pose people (to minimize physical flaws, for example) and knowing how to deal with difficult subjects (cats don't like to stay put... or dogs may feel intimidated and growl or bite... children whining and crying...) plus knowing the business end of the business (overhead, taxes, insurance, pricing to include a profit margin, etc.) A lot of very talented, skilled and gifted photographers go bankrupt every single day for lack of business sense. So, as I like to tell my friends and acquaintances, talent and skills alone ain't gonna help you much unless you have the PASSION to persevere the obstacles and difficulties you're sure to encounter along the way! It helps if you know how to use posing to your advantage and the clients' advantage, too. Also, you should have a working idea of design, colors, composition, etc. It really helps.
Take a few business courses at night at a local college. Learn about running a small business. Taxes. The need for a good accountant/bookkeeper. Too many talented, skilled and gifted photographers lose their businesses because they have no business sense whatsoever... don't you fall into that trap!
So, what I would suggest is that you gain a bit of experience in the field before you venture out on your own. Work for an established professional, do an apprenticeship for a year or two and get to know the shortcuts, how to deal with the unexpected emergencies, how to deal with clients, advertising, promoting the business, etc. Let the professional give you advice on the images you take, and don't take ill-criticism the wrong way; have a thick skin and bear it. You can work on weekends and keep your present job and if you find a less stressful job at a photo-lab, go for it and learn about the different processes used. In photography, knowledge is EARNING power. The more you know and understand what goes on in the lab, the more you'll be able to stretch the limits of your camera and gear.
Learn all you can about lighting (artificial and natural, how to use reflectors, different kinds of lighting, etc). Lighting is one of the most important aspects in photography, as you may very well know already. The use of the different kinds of filters to enhance and/or avoid certain effects due to lighting.
Unless you give it a try, you'll never know if you would've been good enough... right? Don't live with regrets... "I should've...." "I might've...." "I couldn've...." "Maybe if...." Those are the worse kinds of regrets to live with... I know.
Never lose the desire to learn and stay current with new innovations in the field. Occasionally go to some of the photography conventions and shows. READ as much as you possibly can, it's the greatest source of learning. And, remember, be PASSIONATE about photography if you want to succeed; you will have to make sacrifices along the way if you really want to succeed. Good luck and very best wishes.
Semi-retired, with 43 years experience (most of them very good)