I don't think it's Racism but rather Xenophobia. Racism would imply being Racist towards particular races and also very direct (getting verbal and physically abused, which doesn't seem to happen alot in Japan from what iv heard/read). Xenophobia i think would be more a fear or direct misunderstanding of another.
Japan in regards foreign relations is still only a young country having only opened its borders to the world since Mid 19th Century. Also about 98% of the population are native Japanese. So i think its normal, particularly for more conversative older people to feel more Xenophobic regarding foreigners than the newer generations of people (because of the Influence of outside Media and also the fact Immigration has/is on the rise)
Even if you take Ireland, my home country, talk to some of the more eldery people and they would still have conservative views regarding foreigners and particularly the English. But also Ireland has a huge multi-cultural society now, mainly Chinese and Polish and have been for a good number of years now so Racism isn't neccessarily a huge issue here (although i think it would be ignorant to say that there would be no discrimination happening in some parts).
I don't know if im putting my point across well here, but im simply trying to say that i think that many misunderstand Japanese 'Racism' when it is infact a form of Xenophobia. You don't hear of Japanese people forming Lynch mobs and going after 'dem foreigners!!11!'. Give it time people, Japanese people are human just like the rest of us and it's uncool to tar an entire race with such a generalising brush.
P.S The above statement is not directed at anyone, i mean it generally