Correct Sentence:

The handle of the cup is broken.

Explanation:
“of” shows possession or relationship. It tells us what the handle belongs to — in this case, the cup.

Example: The door of the house, The color of the sky