When you rub the balloon on the sweater, it collects negative charges (electrons) from the sweater, which makes the balloon negatively charged. When you bring the balloon close to the wall, something interesting happens: the wall (which is usually neutral) gets affected by the balloon's negative charge. The negative charges in the wall are pushed away from the balloon, while the positive charges in the wall are attracted towards the balloon.
This means that the positive charges in the wall move closer to the balloon while the negative charges move away. This happens because opposite charges attract each other (positive and negative), while like charges repel each other (negative and negative). So, the balloon ends up being able to "stick" to the wall because the attraction between the balloon's negative charges and the wall's positive charges is stronger than the repulsion of the negative charges in the wall.
Overall, the movement of negative charges is different from the movement of positive charges because of these attraction and repulsion forces!