The historic buildings in the way of One Broadway Plaza were doomed long before the vote on Measure A