What kind of earthworms are we talking about? Think you could find Canadian Night Crawlers in there, maybe in a fish and tackle store (make sure they are not dyed).

If soil is not exposed to chemicals like insecticides, it should be OK from that viewpoint. Still, I would keep them in a bin or similar and culture them rather than collecting from wild all the time. Good luck !