It isn't the fear that's after all, it's the reader's consideration of your claim, so put "after all" first. There are two slightly different things that you might mean. The first is that it's no good to live fearfully, which might be your problem or the world's or both.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.