I grew up in N.B. and am quite familiar with the area. How do I think the whole "Pygmy Village" thing got started? Well, there was one house that had a half-size(or so) storm/screen door on it. It was about half the scale of a standard 80" door. It was not the main, or even secondary, entrance to the house. It appeared to be an entrance to what, I think, was a crawl space or storage area under what was a slightly elevated home. There were also a few stairs leading up to the normal, full-sized, entrance door to the home.
Anyone who remembers the small aluminum "floor display sample" doors from the size constricted hardware stores(long before Home Depot)know exactly what I'm talking about. It was a, full-featured, "pygmy" size version of a full-scale aluminum storm door. A homeowner probably got the brilliant idea of using a discarded sample door to use as a covering to an access portal to his plumbing and, boom, the legend of "Pygmy Village" was born. Anyone who saw it would, no doubt, say "Who the heck uses that??" Someone probably thought, "A person the size of a pygmy". Just my theory. I first saw it 35+ years ago.