For one thing Mexico is a country, just as The United States is a country.
America has 50 states, 48 contiguously. The District of Columbia is simply that a District (there is a long history). There are also Territories such as Puerto Rico.
New Mexico is a State within the 48 contiguous States.
Countries like Canada, designate boundaries as Provinces and Territories.
Australia uses States the same way as America does, and also has Territories.
Then we have Mexico which has the official name of 'The United Mexican States'. These are a federation of thirty-one free and sovereign states (sometimes referred to as Federative States), like 'Baja' or the 'Yucatan'.
There is a city in Southern California known as Baja, California. It's names does not mean that it is part of Mexico, nor does the naming of the Sovereign State of Baja, Mexico, mean that it is part of California.
To properly understand how the names originated you would need to study the etymology of each name. There is a story to be told with each.
Finally, I will add that common place names, such as London, in England, also have other places named the same way. There is a London in Australia, this city is found in the State of Queensland. There also are at least 10 London's in the United States, including London, Texas and London, California, there even is a 'New London' in the State of Wisconsin.