If you plan on going for only 2-5 weeks you will be fine with not alot of knowledge of the language. It is possible you will at some points find difficult than others but generally you wont. Although knowing some of the language is obviously a plus!
I myself and my friends went back packing around Japan for 5 weeks during the summer. Which included rural area's where no one knew English at all. Depsite everywhere we went we made it through and enjoyed our time immensely, even though we didn't know alot of the language ourselves.
Food: There will be times, depending on where you eat that if you don't know the language you could end up ordering somthing you unintentionally wanted to try eat (what we called 'Suprise Food' while we stayed
) as sometimes resturant menu's might not have pictures or English. Although we thought that was half the fun of it and one time i ordered a dish that turned out to be one of the most delicious things i had ever tasted. Some resturants or 'beef bowls' require you to purchase a ticket from a machine before giving it to the chef, sometimes the machines have pictures, sometimes they dont.
Directions: If you are around the cities, you would be fine usually. If you plan on making your way out to arkward or out-of-the-way places it may be a bit more difficult and so some problems may arise.
Accomodation: if you pre-book places to stay or book online using Hostelworld.com or similar you should be ok. They have huge ranges of Hostels/Hotels that can provide you with accomodation with your stay. You don't have to book every single place before your trip, we didn't and sometimes even booked a few days before arriving at our next hostel/hotel, but i do recommend that you do book at least a few days in advance. I think Hostelworld.com may have Ryokan's on their list but if not and you want to try one then google is your friend. I also recommend staying in a Capsule hotel at least once!
Transport: We had no problem getting normal Rail Tickets (you use machines anyway) and no problems getting our Shinkansen tickets. At least someone behind the counter will have some English or if not once you say where you want to go and what day they pretty much understand and it isn't a problem.
Those 4 above is what you mainly will interact with most on your trip and as i said, for the most part you will not require alot of knowledge of the language to get around but again, if you do know some of the language it will be a plus!
