Castle construction techniques reached their acme in the 15th century. After that, they were no longer effective because improvements in siege weapons and tactics made it impossible for large numbers of people to fortify themselves behind walls for great periods of time. The "toughest" castles were not those that had the thickest walls or the biggest guns, but those that used geography to their advantage. That's why castles were built in strategic locations such as along the bends of a river or high atop a mountain or hilltop, on an outcropping, on an island, etc. The most impregnable fortresses are those that can withstand the greatest onslaught and can take whatever the enemy throws at them, but they also have to be livable, reliable and sustainable for the people they were built to protect. Early fortresses were built to serve as places of refuge. One can still see the remains of round towers across Ireland. They are all that remain of the ring-forts of which they were the central component. They were not built for habitation, but to be used only when needed.
It was during the feudal era when castles really began to take off on a grand scale, and some were just too large to sustain. Without an effective means of ensuring that the people inside will have a constant supply of fresh water and food, you can construct a castle with walls a mile thick and it won't matter - so every castle had to be built in a place where there was a sizable fresh water supply or fresh water source as well as enough arable land (or enough arable land close by), so that crops and livestock could be raised.
Because they were all built for the same purpose and built according to the same standards and with the same techniques, they were all pretty much on par with one another. Perhaps the quality of the craftsmanship was better here and there, but it was really more about where the castle was built that would determine its strength. Of course design did enter the equation to a certain extent, but the building materials varied from one place to another depending on what type of stone could be quarried in that particular area. But of course, once cannons entered the equation, none of that mattered. A relentless barrage of cannonballs would go through limestone eventually just as it would pass through sandstone or granite or anything else.