drift back
To move toward someone or something slowly, especially while floating on water. The innertubes were all scattered across the pool, but now they're drifting back to the steps.
drift back (to someone or something)
to move back to someone or something slowly, on the surface of water. The canoe drifted back to shore. My little boat finally drifted back to me.