I can't imagine that anything of any importance hinges on naming these as one or the other. It seems to me that you sense intuitively that there are good arguments for either interpretation. In fact, you might even be able to argue that one of them is an adjective and the other is an adverb.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.