The provinces were formed when Canada became a country. Back when Canada was first born in 1867, we had just the provinces, of Ontario, Quebec, New Brunswick, and Nova Scotia. These were three distinct colonies that had their own governing bodies, and a high population. Not long afterward, British Columbia, Manitoba, and Prince Edward Island joined as provinces too. Eventually, it became ten provinces and three territories that we have today.
Provinces are similar to states, except that provinces are mainly administrative unit. Canada's provinces don't really control anything, other than perhaps the local education and health requirements of the citizens in that province. There are provincial taxes, much like state taxes, which help pay for things like the government, infrastructure, and in the case of Quebec, cultural issues related to the French language.
The main reason that I see Canada has some territories and not just all provinces is because of the population of the three territories that we have. In 2016, the combined population of the Yukon, Nunavut, and Northwest Territories were less than the smallest province in Canada, Prince Edward Island. It does not warrant having an entire level of government in three territories, each responsible for 40,000 people or less. So instead, the federal government gets elected officials to manage the area, and they get one seat (one voting representative) in our parliamentary government.
Our territories are sort of the way the Brazilian territory of Fernando de Noronha was treated. It was a territory, and then the federal government chose to dissolve that territory and have it administered by the state of Pernambuco.