Generally, spoken dialogue and thoughts are given in new paragraphs, but it is not a rule. Your style and your editor will have more to say on the matter. If they appear in separate paragraphs, they sometimes have no enclosing marks at all; they are simply preceded by a dash.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.