So we use " get off the point " as the opposite of " get to the point" , is it correct? Yes, generally speaking. Although the verb can vary. We probably more often use 'go off the point'.
and " get out of point" is wrong? Yes, it is.
There are a number of other expressions we commonly use, eg