org.bouncycastle.util.encoders

Class UTF8

java.lang.Object

org.bouncycastle.util.encoders.UTF8

public class UTF8
extends java.lang.Object

Utilities for working with UTF-8 encodings.

Decoding of UTF-8 is based on a presentation by Bob Steagall at CppCon2018 (see https://github.com/BobSteagall/CppCon2018). It uses a Deterministic Finite Automaton (DFA) to recognize and decode multi-byte code points.

Constructor Summary

Constructors

Constructor and Description
UTF8()

Method Summary

Modifier and Type	Method and Description
static int	transcodeToUTF16(byte[] utf8, char[] utf16) Transcode a UTF-8 encoding into a UTF-16 representation.
static int	transcodeToUTF16(byte[] utf8, int utf8Off, int utf8Length, char[] utf16) Transcode a UTF-8 encoding into a UTF-16 representation.
static int	transcodeToUTF16(byte[] utf8, int utf8Off, int utf8Length, char[] utf16, boolean lenient) Transcode a UTF-8 encoding into a UTF-16 representation, optionally replacing malformed input rather than failing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UTF8

public UTF8()

Method Detail

transcodeToUTF16

public static int transcodeToUTF16(byte[] utf8,
                                   char[] utf16)

Transcode a UTF-8 encoding into a UTF-16 representation. In the general case the output utf16 array should be at least as long as the input utf8 one to handle arbitrary inputs. The number of output UTF-16 code units is returned, or -1 if any errors are encountered (in which case an arbitrary amount of data may have been written into the output array). Errors that will be detected are malformed UTF-8, including incomplete, truncated or "overlong" encodings, and unmappable code points. In particular, no unmatched surrogates will be produced. An error will also result if utf16 is found to be too small to store the complete output.

Parameters:

utf8 - A non-null array containing a well-formed UTF-8 encoding.

utf16 - A non-null array, at least as long as the utf8 array in order to ensure the output will fit.

Returns:

The number of UTF-16 code units written to utf16 (beginning from index 0), or else -1 if the input was either malformed or encoded any unmappable characters, or if the utf16 is too small.

transcodeToUTF16

public static int transcodeToUTF16(byte[] utf8,
                                   int utf8Off,
                                   int utf8Length,
                                   char[] utf16)

Transcode a UTF-8 encoding into a UTF-16 representation. In the general case the output utf16 array should be at least as long as the input length from utf8 to handle arbitrary inputs. The number of output UTF-16 code units is returned, or -1 if any errors are encountered (in which case an arbitrary amount of data may have been written into the output array). Errors that will be detected are malformed UTF-8, including incomplete, truncated or "overlong" encodings, and unmappable code points. In particular, no unmatched surrogates will be produced. An error will also result if utf16 is found to be too small to store the complete output.

Parameters:

utf8 - A non-null array containing a well-formed UTF-8 encoding.

utf8Off - start position in the array for the well-formed encoding.

utf8Length - length in bytes of the well-formed encoding.

utf16 - A non-null array, at least as long as the utf8 array in order to ensure the output will fit.

Returns:

The number of UTF-16 code units written to utf16 (beginning from index 0), or else -1 if the input was either malformed or encoded any unmappable characters, or if the utf16 is too small.

transcodeToUTF16

public static int transcodeToUTF16(byte[] utf8,
                                   int utf8Off,
                                   int utf8Length,
                                   char[] utf16,
                                   boolean lenient)

Transcode a UTF-8 encoding into a UTF-16 representation, optionally replacing malformed input rather than failing.

With lenient false this behaves exactly as transcodeToUTF16(byte[], int, int, char[]). With lenient true, each maximal subpart of an ill-formed sequence is replaced by a single U+FFFD REPLACEMENT CHARACTER, as recommended in the Unicode Standard, ch. 3, "U+FFFD Substitution of Maximal Subparts" (the practice the WHATWG Encoding Standard also requires): a byte that cannot begin a sequence is replaced on its own, a sequence broken by an unexpected byte is replaced up to but not including that byte, which is then decoded afresh, and a sequence truncated by the end of the input is replaced. Well-formed input is decoded identically in either mode, and no unmatched surrogates are produced. Each replacement accounts for at least one input byte, so an output array at least as long as the input is still sufficient; -1 is then returned only if utf16 is too small.

Parameters:

utf8 - A non-null array containing a UTF-8 encoding.

utf8Off - start position in the array for the encoding.

utf8Length - length in bytes of the encoding.

utf16 - A non-null array, at least as long as utf8Length in order to ensure the output will fit.

lenient - true to replace malformed input with U+FFFD, false to fail on it.

Returns:

The number of UTF-16 code units written to utf16 (beginning from index 0), or else -1 if the utf16 is too small, or, when not lenient, if the input was malformed or encoded any unmappable characters.