utf32.reader

function

Library: UTF-32 (OMFFUTF32)
Import : omffutf32.xmd

Returns: a scannable input source for streaming data

Declaration

export string source function 
   reader   in value encoding-type encoding   optional 
          from value string source input-data

Purpose

Use utf32.reader to read a string source and return the text of that source converted from a UTF-32 encoding to a UTF-8 encoding. So, although the provided source is in a UTF-32 encoding, the program sees UTF-8.

The argument encoding can be used to specify a byte ordering. The utf32.reader function can determine the byte ordering from the input:

if an ordering is provided to the utf32.reader function, it is used, otherwise
if the input being read by the utf32.reader function begins with a byte-order-mark, the byte-order-mark is used to determine the byte ordering, otherwise
the input is assumed to be UTF-32BE.

This conforms to Paragraph D101 of the UTF-32 specification. In addition, utf32.reader will always discard a byte-order-mark that appears at the beginning of its input if it is the byte-order-mark corresponding to the encoding being processed, regardless of how the encoding was selected.

Example

The following example converts a file from UTF-32LE to UTF-8 for further processing by find rules:

  import "omffutf32.xmd" prefixed by utf32.
  
  process
     using group "process input"
        submit utf32.reader in utf32.encoding-utf-32le from file #args[1]
  
  group "process input"
     ; ...

Usage Note

To use utf32.omffutf32-version, you must import OMFFUTF32 into your program using an import declaration such as:

  import "omffutf32.xmd" prefixed by utf32.

Other Library Functions