The whole hobbits & Gandalf plotline is pointless and affects nothing outside of it. The only thing that would be noticeable in the others if it was cut altogether would be the lack of relief when the show jumps away from Rhun. The whole plotline, over both seasons is not a story, it's an agglomeration of meta elements they thought they needed:
- Hobbits
- Gandalf
- Mystery boxes (Payne & McKay are former Bad Robot writers)
- Additional red herrings for the Who is Sauron? mystery
- More spunky female leads who are too adventurous for their society
- Magic stuff they can pull out their asses without having to work it in to the rest of the setting

The plotline is clearly just treading water. There was enough story for maybe two episodes in Season 2.