Explain to me why we’re so very worried about genetically modified food

I read the other day (and am too lazy to hunt down a link), that some genetically modified rice got mixed into the American rice crop. It’s significant only because Europe has now closed the door on all American rice:

The European Commission will begin testing U.S. long-grain rice to ensure it does not contain an unauthorized genetically modified strain.

Known as LL Rice 601, it was developed by a German company to be resistant to a weed killer known as glufosinate, the EU Observer reported Thursday.

“We have strict legislation in place in the (European Union) to ensure that any GM product put on the European market has undergone a thorough authorization procedure based on scientific assessment,” said the EU’s health and consumer commissioner.

Last year, the EU imported 198,000 metric tons of long grain rice from the United States. The EU’s move follows a similar action by Japan.

I know I’m being a complete doofus, but I’m unclear on why genetically modified food is viewed with such fear. Haven’t humans been genetically modifying food since the dawn of time? Admittedly, we didn’t do it in the lab at the microscopic level, but the growth of human kind is tied to the fact that humans were able to take common grass crops and turn them into the broad variety of grains now available. The same goes, of course, for fruits. Farmers and gardeners have always grafted and changed foods, and with these changes humans have become increasingly able to avoid famines. (And can anyone imagine a world without those constant influx of newly created roses?) Can someone who is more informed than I am explain to me why genetic modification is so significantly different?

Talking to Technorati: Genetically modified foods