1. This is the desk. That belongs to Bill.

Question

= This is the desk that belongs to Bill.

2. This is the window. Bill broke it yesterday.
= This is the window that Bill broke yesterday.

3. This is the book. Its cover is red.
= This is the book whose cover is red.

4. This is the book. The cover of it is red.
= This is the book the cover of which is red.
= This is the book of which the cover is red.

(Are they all grammatical? Are there any wrong expressions?)

Answer 1

4, for the first one seems iffy. You might wanna use a comma after book.

=This is the book, the cover of which is red.

If you get so desperate just type your sentences in to microsoft word and spell check

Answer 2

The first three sentences are correct.

For number 4, it would be better to say

This is the red book.

Or

This is the book with the red cover.