How do you really “abandon” your Self? When we refer to “Self”, we mean innate Awareness, That which is always there. Abandoning the body and mind happens at the time of death. During deep meditation, we temporarily abandon identification with the body-mind ego. You are not “abandoning your Self” (with cap S) as That is your true Existence. The point is to diminish the influence of the (false) ego. The ego exists because of the body, senses, mind. But your true Self is beyond these, it is the Awareness of these.
You cannot truly “abandon” your Self. You can only forget it (for a while). And your Self never abandons you!