The best word to complete the sentence is "defiant."
The sentence would read: "Annika tried to get her little brother to bed, but he was defiant."
In this context, "defiant" describes the brother's behavior of resisting or refusing to follow instructions, which fits well with the action of trying to get him to bed.
"Defiantly" could also work, but it is an adverb, while "defiant" is an adjective that directly describes the brother. "Defy" is a verb, "defiance" is a noun, and neither would be grammatically correct in the given sentence structure.