A man convicted on drugs charges has lost his post as a care worker.