Googling around the net I had discovered 3M Citrus Adhesive Remover - I couldn't get that! The man at Bunnings (hardware store) suggested this stuff "oomph". So armed with that, some small wire and fibre brushes and a bag of old socks, I set to work.

This is one of the pictures I took with the old carpets in the car so we knew what the layout was. Note the old adhesive on the panel under the door.
Half cleaned. It took a LOT of rubbing.
Drivers side done. I haven't decided whether to clean the doors or not - I won't be using glue so I don't need them clean (unless I glue the waterproof plastic sheeting on which this Vogue was curiously missing) - we'll see how I go.
I finished around 5:30PM - it took the whole day. I did remove, clean, and reinstall the headlight dip switch and cleaned adhesive from inside the boot too. I got through about half the socks.