I feel the best way to say this is the first one "You threw it away yesterday". It is simple and conveys the main message.
You are right in saying that 'yesterday' or other words that state definite time period (e.g. last night, 2 days ago) can't be used with 'have done'. So, if you write "You have thrown it away yesterday", the sentence won't be correct. However, you
In addition to the previous post, here is a reference that may help you.
Whenever we use reference of time passed, like last week, and yesterday, a simple past tense will do just fine, and also, Words of past time is never to be used along with present perfect tense.
I have talked to John about the project yesterday - this is wrong grammar.