There really isn't any difference and people use them interchangable. The only difference I could suggest is a connotation where thinking of an idea to fix the problem is thinking of a solution from his memory, thinking up a solution is inventing one. Very slight difference and I only trhink of the difference because you ask for one.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.