The author means that popular sovereignty "undermined the Missouri Compromise" by indicating that the earlier division of the country into free and slave states, established by the Missouri Compromise, no longer applied. This suggests that the boundary set by the Missouri Compromise was no longer a fixed line, allowing for the possibility of slavery in areas that were previously designated as free.
Therefore, the correct response is:
It reopened debate over the border between free and slave states.