In yoga there is a lot of talk about "unity" and oneness, but there is a lot of misunderstanding about what exactly that means. It's definitely not an easy topic but here is the basics from my point of view. In Samkyha "yoga" philosophy there are two undeniably separate entities which are mistakingly mixed together (yoga sutra 2:23-24) through deficiencies in our perception, leading to and/or resulting from ignorance. The practice of the eight limbs of yoga (yoga sutra 2:28) reveals the distinctness of the purusha and prakriti and thus leads to the ability to pierce through avidya (ignorance) and realize the self, or "atman", as it were.
There is a lot of talk in yoga about yoga sutra 1:2 that "yoga is the cessation of fluctuations of consciousness", but the practical aspect of yoga in removing these fluctuations is entirely tied to the notion that without these fluctuations (or disturbances if you want to call them that) you would exist in a more authentic form, and the self would be revealed to you. So it is one and the same thing, and in layman terms we're just saying that our minds impede our ability to perceive the truth, so we must find a way "behind" that type of misperception, if you will, and that method is called yoga.
So there is a lot of talk about how all human beings are actually parts of a larger whole, and that is backed up by the Upanishads, at least to some degree, that the concept of an egoless self exists. Obviously if you were not a human being then you would take some other form; the question then is wether or not that form is unique, or wether or not it is separated from "other" such forms. There would be little argument within the yoga world that the purusha without a body would not be able to perceive itself, at the very least not as we do as human beings. The bigger argument comes when people try to interpret togetherness and separateness.
Brahman is the "one without a second" concept talked about in yoga philosophy, and it is constant in the Upanishads. The Samkhya philosophy shows clearly that the Brahman, like explained in the Upanishads, exists in a realm that is somewhat beyond our perception, so trying to define it is extremely mundane and a waste of time. In a tree of Samkhya (shown to the right) we can see that Prakrit manifests through the "matter" or physical realm, the more quantifiable aspects of our reality. The purusha remains distinct from the prakriti and sort of "hovers" there. It is sometimes regarded as being superior to prakriti, but that is, if anything, only implied in the sutras and never actually stated. The purusha tends to be difficult to perceive in our daily lives due to the constant attention we give to the grosser, more mundane aspects of ourselves, at the same time the two don't necessarily exist in a hierarchy.
In my opinion we are separate beings all the way up the ladder to Brahman, but I'm open to suggestions there. I see the purusha as something that carries karma along with prakriti as long as the two need each other, which could be indefinitely or permanently. Assuming that they're permanently intertwined then its logical to assess that we are not "all one" until we consider it from the perspective of Brahman, or unless there is a way to remove yourself from the cycle of birth and death, and thus no longer "having" a purusha/prakriti, and then defining that existence as our more validated form, so then it would be true to say that we are not separate, simply because the form in which we are separate is an illusory form, or isn't the form which defines who or what we are.