Mr Zuma told those who had fled the violence that it went against South African values and that he would bring it to an end.
He was jeered by some in the crowd who accused him of acting too slowly.
At least six people have died in xenophobic attacks in Durban, with violence spreading to other areas.
Migrants, mostly from other African states and Asia, have moved to South Africa in large numbers since white-minority rule ended in 1994.
Many South Africans accuse them of taking jobs in a country where the unemployment rate is 24%.
In a statement, Mr Zuma said: “These attacks go against everything we believe in. The majority of South Africans love peace and good relations with their brothers and sisters in the continent.”