"This harassment has included police raids, arbitrary arrests, ill-treatment and extortion, as well as destruction of informal housing," the London-based human rights group said in a statement.
"The requirements under the 'FIFA by-laws' which create extensive exclusion zones for informal economic activity are seen as particularly prejudicial," Amnesty said.
Police spokesman Vish Naidoo defended the action by police and local authorities, saying Amnesty was trying to link the World Cup to a perennial problem in South Africa, where millions live in grinding poverty and shanty towns.
"Just because the World Cup is happening, must we ignore laws and by-laws that are being broken?" he said.
Amnesty also said the deployment of huge numbers of police at World Cup-related sites would mean fewer officers in poorer neighborhoods, where the largest part of South Africa's notoriously high levels of crime occur.
It added that it was worried about police getting trigger-happy in their zeal to protect the 350,000 foreign visitors who are expected for the month-long tournament, which kicks off on June 11.
(Reporting by Ed Cropley)
(Editing by Jon Bramley)
Fox News First
Get all the stories you need-to-know from the most powerful name in news delivered first thing every morning to your inbox.
By entering your email and clicking the Subscribe button, you agree to the Fox News Privacy Policy and Terms of Use, and agree to receive content and promotional communications from Fox News. You understand that you can opt-out at any time.