The correct response is: it will increase.
When Elston places a book in the box, he is increasing the total weight of the box. To pull the box across the floor with the same distance and effort, the force exerted by the spring scale must increase to overcome both the additional weight of the book and any friction between the box and the floor. Therefore, the force measured by the spring scale will increase.