This town has a pretty bizarre backstory. Back in 1914, the federal government opened up a prison here that was actually an industrial farm, which focused on reform instead of punishment for the inmates, says the Toronto Public Library (TPL). A town of 600 to 1,000 people lived in the area and worked at the prison. The buildings were demolished when the prison closed in 1975, but you can explore the abandoned ruins for a small fee.

This once-thriving town in the early 1900s met tragedy after tragedy, which left it almost completely destroyed. The area was rich in silver and copper, and was at one time one of the world’s leading producers of silver. However, several fires ripped through the area throughout the 1900s, until a final blaze in 1977 burnt most of it to the ground. A small population of just over 1,000 people still lives there today, according to TPL.

Back in the early 1800s, southern Simcoe County was home to a thriving town called Ballycroy, reports Barrie Today. The town is named after the place in Ireland where many of the settlers who lived here had arrived from. One of the original buildings still stands today and it’s a general store and post office. The little town has a fascinating history, which you can read about in the Orangeville Citizen.