I have no idea how to approach this problem, if someone knows just how to relate h, r with H,R, that would be extremely helpful and I can workout the rest! Thank you in advance.

Given a right circular cone, you put an upside-down cone inside it so that its vertex is at the
center of the base of the larger cone, and its base is parallel to the base of the larger cone. If
you choose the upside-down cone to have the largest possible volume, what fraction of the
volume of the larger cone does it occupy? (Let H and R be the height and radius of the large
cone, let h and r be the height and radius of the small cone. The formula for the volume of a cone is V= 1/3pi(r^2)h

1 answer

http://mathhelpforum.com/calculus/232420-maximize-cone-inside-cone.html

From his smaller volume V, take the derivative, set to zero, solve for r (for max volume.

Then determine the ratio