It could mean either, but since we don't know where exactly the items swept from the table landed (some pieces could have landed on a chair or some other item of furniture), you're safer using the second expression than the last. Naturally, given the context here, any sweeping done was probably with a free arm rather than with a brush. ;-)
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.