UK, Ireland and the Channel Islands:

The Common Travel Area is an open borders area comprising the United Kingdom of Great Britain and Northern Ireland, Ireland, the Isle of Man, and the Channel Islands. The British Overseas Territories are not included. Based on agreements that are not legally binding, the internal borders of the Common Travel Area (CTA) are subject to minimal controls if at all, and can normally be crossed by British and Irish citizens with minimal identity documents, with certain exceptions. Moving to the Isle of Man from another country in the CTA is therefore very easy and simple.

EU / EEA & Switzerland:

Under Isle of Man law EEA and Swiss Nationals can enter and live in the Isle of Man to exercise EU treaty rights. A non-EEA National family member can also join their EEA National sponsor who is exercising EU treaty rights.

Further information is available on the Isle of Man Government website EEA Immigration page