when will the resultant of the two vectors be at maximum when two forces of 5n and 7n act on an object?

Answer 1

When they are both in the same direction and therefore add.
|R| = |A| |B| cos theta
|cos theta| </= 1 for all theta and is max of 1 when theta = 0
5+7 = 12
If theta is the angle between vectors.

Answer 2

yes