I burned up all my non-physics courses early on because I started at a branch campus mostly setup for 2 year degrees on account of my age. When I turned 18 and the family was okay with me moving away I literally had 5 classes left that weren't physics, and one of those was Partial Diff-Eq math, which might as well be. That left me intro chem 1 & 2, which was also damn nearly physics, and German 1 & 2 and I suck at languages. Not fun. 4-5 physics courses each semester all in the same two near-identical adjacent rooms with the same people who I also studied with in the evenings, though we did usually have fun, followed by two more years of class work in grad school in those same two rooms because I stuck at the same school. Thank God a lot of my non-physics friends were still undergrads that first year of grad school or I'd have gone totally nuts, and I began deliberately cultivating friends like they were electives. It really is important, IMO, to keep up regular contact with other fields and studiers of it or you start turning into one of those field isolationists.

Long and short of it being I'd actually suggest auditing a medium difficult, like soph/junior, level course every semester or at least every year, or making event and socials that stick non-Computer people with brains in a room with you, for the sake of sanity and a well-rounded mind, you tend to get good ideas that way too, someone will suggest some application relevant to your own field you just wouldn't have seen otherwise..