He is speaking figuratively of course. "Fair quarter" means "enough room" in this context. In other words, he wants to say that he is rather tolerant of vanity.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.