I never pegged them to be skittish. The wild ones we have here, you can walk right up to one and scoop it up. You can find them out by the trash and they won't even hop away, atleast not right away.
Usually most animals become skittish from stress or are naturally inclined towards stress. I can't say for your toads, but the ones here act like they're lions..