This library contains one OmniMark external source function and one OmniMark external output function implementation, as follows:
reader
is an external source function that reads a value string source, its argument, and returns the text of that file converted from a UTF-16 encoding to a UTF-8 encoding. That is, the provided source is in UTF-16, but the program sees UTF-8.
Any malformed input data is read as a Unicode NOT-A-CHARACTER character (0xFFFD). The only malformed case recognized is if only half of a surrogate pair is found.
Read-in UTF-16 data is assumed by default to be big-endian, but leading and embedded Byte Order Marks (BOM) in the data are recognized and acted upon. A leading BOM is removed from the input, but embedded ones are left in.
writer
is an external output function that accepts UTF-8 encoded data and writes that data to a value string sink, its first argument, converted from a UTF-8 encoding to a UTF-16 encoding. That is, the program writes UTF-8, but the provided output receives UTF-16.
writer has two further switch-valued arguments, placed ahead of the output argument. true is used as a default value in both cases. The two arguments are:
true if a Byte Order Mark (BOM) is to be written as the first character in the output. false if not.
true if the output is to be written big-endian.
Any malformed output data is written as a Unicode NOT-A-CHARACTER character (0xFFFD). The only malformed cases recognized are characters too large to be encodable as UTF-16 (i.e. larger than 0xFFFF), and characters whose UTF-16 encodings would be the value of half of a surrogate pair.
A good place to find information on the details of UTF-16 encoding is the UNICODE Consortium.
To use OMFFUTF16, you must import it into your program using a statement like this:
import "omffutf16.xmd" prefixed by utf16.
(Please see the import topic
for more on importing.)