A boycott of a conference on racism was growing amid concerns that the Iranian President, Mahmoud Ahmadinejad, could use the meeting to attack Israel.