Take him to a new vet. The cat should be much better, unless he was run over by a car!
Go to a new vet TODAY. If he is lethargic or not eating, you should consider taking him to an emergency vet because this is not good to see in a cat.
In coming out of surgery, the anesthetic might affect him for a day or two after, but not much beyond that. He needs to be seen or on other antibiotics if the first ones did not work. Additionally, if he got a shot of antibiotic, then you need to be careful in how you proceed with other meds. He probably should not have a high fever for more than a day or two with antibiotics - or it sounds like they aren't working very well. Always take the records of what the cat has been given if you go to a different vet, in the event they prescribe other medications.
Maybe, he was senstive to the drain? Or the meds? Try using iodine mixed with water to cleanse the wound. But, the cat should be feeling much better unless there were other injuries, or if the infection is not cleared from his system.