Just came back from Japan. Public washrooms in train stations and the such are usually clean but some do not have toilet paper and most do not have soap. my suggestion is to grab a tissue pack that are being handed out on the street and keep it in your pocket to use in the public restrooms as toilet paper or hand towels to dry your hands if you do not own a handkerchief (invest is one you'll use it a lot to dry your hands). If you really have to go I would suggest running into a convenience store(711, Lawsons, etc.) or a Mcdonald's/ BK or malls they have western style washrooms with soap and toilet paper. Buy something while you're the food is actually edible in Japan. I find that I was always thirsty in Japan.

Be careful of public washrooms in parks though, they are usually dirty and not western style, so it'll be a squat style (not sure about the Ladies washroom though). Bring hand sanitizer if you you feel you need it.