Not that you can't deviate from it at all, but if you do, it had better be for a good reason - and in way that is coherent and clearly thought through. A good reference, though not sure if anybody here is familiar with it, is the Lord of the Rings Online MMO RPG game. They have hundreds of characters, locations, minor storylines that expand on Middle Earth around the time of the LotR main story and intertwine with it. And like in most MMOs, you can create male or female characters with no distinction (though as for racial diversity, that's rather more lacking). But it's very rare that any of their own inventions cause controversy, because their writers clearly have great respect and love for the books, the setting and the history of Middle Earth.

Regarding your other reply above, I'm in the apparently not very common position of someone relatively 'woke' and certainly strongly feminist, who nevertheless still struggles with the way the RoP and WoT show writers have handled diversity. I absolutely wanted gender balance and increased racial diversity in the shows, but I wanted it in a way that was coherent and thought through.

Look at Bridgerton, House of the Dragon, Sandman, the Harry Potter and the Cursed Child play. All genre shows that also chose to increase racial diversity by making one or more key characters PoC while they hadn't been (or at least weren't said to be) in the original works. But in all those cases, they either provided an in-universe explanation that was sufficiently plausible, or at least successfully showed that making this character a PoC did not in any way contradict the canon or internal logic of the original setting. Hence, while there was in each case still some controversy about that choice, it was much less widespread (Anonymous2000 and other people on this board have, as I recall, expressed their approval of the HotD changes).

Whereas RoP and WoT have just gone for 'color-blind casting' and in so doing created societies that, while they may accurately reflect the real-world viewing audience, certainly don't accurately reflect the universe being adapted. In the case of WoT, it's not even that there is any shortage of PoC in the books, but they are, as in the real world, geographically distributed in a very different way. You don't have to be a bigot to have a problem with that - merely a fan who attaches more importance to coherent choices when adapting any work.