If someone has asthma and is affected by dust or allergens while using the dirt road, the situation may be concerning, but it wouldn't be fair to say it's your fault solely for using that road. Asthma is a complex condition influenced by various factors, including environmental triggers.
If you are aware that a specific person with asthma is particularly affected by the dust and you have the ability to change your route or take steps to mitigate the dust (like driving slowly to minimize dust clouds), that could be a considerate action. However, the responsibility for managing asthma largely lies with the individual and their healthcare provider.
If you're worried about the impact, you could consider discussing the situation with the person affected to see if there's a way to address their concerns together.