This is a really interesting question because I can't answer it! To me, only hardly strongly triggers inversion, yet I don't have any objections when barely or scarcely are used with inversion. I would say: Hardly had I suggested ...
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.