On that note - is usually a figurative expression, using "note" to mean something like an emotion, a mood, an attitude of mind.
If someone is (for example) very critical about the choice of a particular post, the may continue with "On the same note, I found the absence of capitals, context and punctuation in your post to be very annoying."