) All of them - rather, quite, pretty, rather - are the same part of speech (adverbs). Therefore structurally, as adverbs, they can modify adjectives, adverbs or verbs. In fact, they can be used interchangeably in some cases: I know him pretty well.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.