Three Supreme Court justices tell why they opposed removing ban