Chosen Solution

Is it ok to wear latex gloves to stop oil from my fingers getting onto the contacts during teardown?

Yes, latex gloves are perfectly fine. Do be aware that you can still build quite the static charge with latex. (Yes balloons are rubber but latex has similar properties)