The best explanation for the observation is:
The bus moves forward, while inertia keeps you in place.
This explains that as the bus starts to move, your body tends to remain at rest due to inertia, which is the tendency of an object to resist changes to its state of motion. Therefore, relative to the bus, you appear to move backward as it accelerates forward.