“When they were given permits to build all these building, there had to be a reasonable understanding that they were going to increase families.”