Yeah Diana55 is right about doubling the time. I read the can, see the minutes, and my brain just says "close enough" and I wreck it. Did the same thing with a table last summer, put a glass down way too soon. That wait feels like forever but it's always smarter to just walk away and find something else to do. Your fence probably looked great for those twenty minutes though.
Lol that drying time gets everyone. Read somewhere that our brains are terrible at guessing how long finishes take to cure. They said to always double whatever time you think it needs. Patience is a muscle you gotta work out, but man it's the worst workout ever. Your poor fence feels your pain.