Only 4 works well.
I'll do some rephrasing:
He walked away, holding an golden apple up high.
He walked away with a golden apple held up high.
He walked away after raising a golden apple in his hand.
[wordier] He walked away, having raised a golden apple in his hand.
Thank you for your help. [situation] The old man has raised a golden apple and started walking away.
1. He walked away with a golden apple up in his hand.
2. He walked away with a golden apple up.
3. He walked away with a golden apple raised by him.
4. He walked away, holding a golden apple up in his hand.
[Can we use these sentences? Are they sutitable for the situation? Do you have some more appropriate sentences?]
2 answers
1 and 4 are okay.
You could reverse 4.
Holding a golden apple up in his hand, he walked away.
You could reverse 4.
Holding a golden apple up in his hand, he walked away.