Let the joint density function of X and Ybe given by:

f(x)={kxy^2 for 0<x<y<1
(0 otherwise
what is the value of k?

Can someone please help me with this?
The formula I used was:
))kxy^2 dx dy (where )) is double integration both from 0 to 1).

I worked it out (I think) and got 15. I used integration by parts where i had u=x, du=dx, dv=y^2 dy, and v=(y^3)/3. If anyone can tell me if this is right, or I'm on the right track, let me know... PLEASE!!!

I'm not sure your integration is correct. This looks like an iterated integral from 0 to y, then from 0 to 1, where you have ))f(x,y)dxdy in that order.
There might be other ways to set this up to, but I'm a little short on time to look at it right now.

Here's the work I did... Sorry if its hard to follow

))kxy^2 dx dy
k))xy^2 dx dy <--Since k is a constant, I can pull it outside the integration sign, right?
k)uv-)v du 0|1 <--that's evaluated from 0 to 1
k)[(xy^3)/3]-[(xy^4)/12] 0|1 <--Only one integration sign, right after the k before the bracket
k)y^3/3-y^4/12
k/3)y^3-y^4/4 dy
k/3(y^4/4-y^5/20) 0|1
k/12- k/60=1
k/15=1
k=15

I really don't understand your integration by parts. What you have is
f(x,y)=kxy²
Let's use the uppercase S for the integral, so
SS_D f(x,y)dA would be the integral, where D is the area in the x-y plane we're integrating over.
According to this:
"f(x)={kxy^2 for 0<x<y<1
(0 otherwise
what is the value of k? "
we should integrate from x=0 to x=y.
I also think your f should be a function of both x and y.
Yes, you can pull the k out, so the integral should (if I've read the problem correctly) look like
kSS xy²dxdy where the limits look like
S¹₀S^y₀
When you do the first integral you should have
(1/2)x²y² evalulated from 0 to y for x. This gives
(1/2)y⁴ if you now evaluate this integral you should get
(1/10)y⁵ evaluated from 0 to 1 and end up with
k*(1/10)=1 so k=10
Hopefully I've read the problem correctly. Be sure to check your text for a worked example, this is a very common problem and you should have no problem finding an example to compare the problem with.