Basically, you are correct. A plant flowers, and then a “fruit” develops that contain the seeds. Some plants produce fruit but no seeds (e.g. seedless grapes).
Some people feel that “fruits” are sweet but many are not. Limes, lemons, grapefruit, chilies, peppers, cucumbers, squash, avocado and sloes are all fruit.