They mean roughly the same thing. I would normally use 'make yourself at home' or 'feel at home' to a person who might be staying a day or two. 'Make yourself comfortable' is often used when the host has to go to another room, perhaps to do something in the kitchen for dinner.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
TomJwhen somebody visits somebody's house as a guestMy first choice would be "Make yourself at home" in this context, though I'm embarrassed to say that in my case, this is followed by a long list of house rules!