It has more than a 500-year history as a trading post, a leper colony and an island had also been used to house banished religious leaders.