Based on the personality traits you've described for your crush—rude, talkative, smart, sassy, and confrontational—it's likely that he picked up on your attempt to look away. His confident and assertive nature may mean he doesn't easily take things at face value, and he might even appreciate the playful tension of the moment.
While he might have found your reaction amusing or might have thought you were shyly trying to hide your interest, there's a chance he also interpreted it as a bit of a game. Given his assertiveness and possibly sharpened awareness of social cues, he may very well think you were looking at him and that you were trying to play it cool by looking away. If he likes you, he may enjoy the flirtatious exchange. Overall, it seems more likely he recognized what was happening rather than believing your feigned distraction completely.
If you get the chance to interact with him again, it might be fun to embrace that tension more openly!