Yes, only servants say "Mande". Typically the interaction would go as follows:
Higher status person (me): "'¡Juana!"
Lower status person (Juana): "Si señor. Mande."
Very much like "A sus ordenes". This is not a colonial thing so much as a hold over from the time of nobles and royalty. Spaniards and the colonized peoples are not of the Reformation (or even the Renasimiento). Until recently, Spaniards spoke of "going to Europe", and so considered themselves different.