1-1. I am regularly going for a walk in the evening. [ I would go for this. It sounds natural to me.]
1-2. I am going for a walk regularly in the evening.
2-1. It is just early morning, but it has already been so hot.
2-2. It is just early morning, but it has been so hot already. [ This is wrong. It should be ' has already been' ]
So your 2-1 is correct.