Go see your doctor, and ask them to test your thyroid.
There are hormones in your body that dictate how your food is stored and used by your body, and how all your functions use up energy. It's very possible to have a thyroid issue that keeps you heavy.
What you eat also makes a huge difference- *not* just how much of it. If you eat bread, cake, crackers, pasta, it is very possible your body is reacting badly to it. There are some people who cannot eat *any*, and if they do, they just get huge on even a little. You might have to try eating a low carb diet with no grains, to see if you can improve.
Go see your doctor first, and don't let them make you feel badly. Just tell them you want to figure out how this is happening so you can fix it. Our genes do make a big difference, but your hormones and diet can be managed. There are food sensitivities that can do this, even when you don't overeat.