Answer:
French overseas territories' phone numbers can be dialled from mainland France without a country code, but they do actually have their own separate country codes and prefixes.
In order to send SMS to these Countries via Nexmo SMS API, it is necessary to use their individual country code. Using France's +33 prefix will result in rejections and failed SMS deliveries.
For example, for a Réunion number that from France can be reached with 33692xxxxx, your Nexmo API calls should actually use the +262 Réunion prefix, ie. 262692xxxxx
Examples of valid number formats to use in your Nexmo API calls below:
Guadeloupe, Saint-Barthélemy and Saint-Martin
Outside France: Mobile phone line: +590 690 xx xx xx
French Guiana
Outside France: Mobile phone line: +594 694 xx xx xx
Martinique
Outside France: Mobile phone line: +596 696 xx xx xx
Réunion
Outside France: Mobile phone line: +262 692 xx xx xx or +262 693 xx xx xx