As a general rule, with some exceptions, we live on a natural feature such as an island, hill, mountain, cliff, coast or (as in this case) cape. We live in a settlement (such as a town or city), a political subdivision (such as a county, state, province or country), or a continent (such as Asia). The main "natural feature" exceptions are we live in valleys, and we can live by rivers as well as on them, if we live beside them, and on an island in a river.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.