The reason for the limited increase in urban development despite population growth can be attributed to multiple factors, but the most straightforward and relevant response is:
Habitable land is already being used for livestock and crops.
This answer highlights the competition for land use between urban development and agricultural needs, which limits the amount of land available for building new urban spaces.