I believe racism is more likely to be based around culture than physical appearance or skin colour.

I actually believe the opposite is true and the trigger is physical appearance. Example. If someone tells a joke mocking the French, its often considered funny and they are fair game. By definition though, it's racism but an 'acceptable' form of racism.
If the same joke was changed to involve someone who did not have white skin, and no mention of nationality or culture, its is quite rightly treated with disgust and considered as racism, even though by definition, its not.
We also need to examine how this would work in different countries to get a fuller picture and not throw personal geographic bias into the research!

This one would be a long discussion over many beers!
Best wishes :-)