The correct phrase that signals the story has moved backward in time is "in the days of his grandfather."
This phrase indicates a reference to a period that predates the current events of the story, illustrating the historical context of the feud between the two families. It helps establish that the conflict has deep roots, going back to earlier generations, rather than being a recent development.