Generate a string from the provided Unicode codepoints.

String codepointsToString(List<int> codepoints) {
  // TODO _is16BitCodeUnit() is used to work around a bug with frog/dartc
  // (http://code.google.com/p/dart/issues/detail?id=1357). Consider
  // removing after this issue is resolved.
  if (_is16BitCodeUnit()) {
    return new String.fromCharCodes(
        _codepointsToUtf16CodeUnits(codepoints));
  } else {
    return new String.fromCharCodes(codepoints);
  }
}

Encode code points as UTF-8 code units.

List<int> codepointsToUtf8(
    List<int> codepoints, [int offset = 0, int length]) {
  _ListRange source = new _ListRange(codepoints, offset, length);

  int encodedLength = 0;
  for (int value in source) {
    if (value < 0 || value > UNICODE_VALID_RANGE_MAX) {
      encodedLength += 3;
    } else if (value <= _UTF8_ONE_BYTE_MAX) {
      encodedLength++;
    } else if (value <= _UTF8_TWO_BYTE_MAX) {
      encodedLength += 2;
    } else if (value <= _UTF8_THREE_BYTE_MAX) {
      encodedLength += 3;
    } else if (value <= UNICODE_VALID_RANGE_MAX) {
      encodedLength += 4;
    }
  }

  List<int> encoded = new List<int>(encodedLength);
  int insertAt = 0;
  for (int value in source) {
    if (value < 0 || value > UNICODE_VALID_RANGE_MAX) {
      encoded.setRange(insertAt, 3, [0xef, 0xbf, 0xbd]);
      insertAt += 3;
    } else if (value <= _UTF8_ONE_BYTE_MAX) {
      encoded[insertAt] = value;
      insertAt++;
    } else if (value <= _UTF8_TWO_BYTE_MAX) {
      encoded[insertAt] = _UTF8_FIRST_BYTE_OF_TWO_BASE | (
          _UTF8_FIRST_BYTE_OF_TWO_MASK &
          _addToEncoding(insertAt, 1, value, encoded));
      insertAt += 2;
    } else if (value <= _UTF8_THREE_BYTE_MAX) {
      encoded[insertAt] = _UTF8_FIRST_BYTE_OF_THREE_BASE | (
          _UTF8_FIRST_BYTE_OF_THREE_MASK &
          _addToEncoding(insertAt, 2, value, encoded));
      insertAt += 3;
    } else if (value <= UNICODE_VALID_RANGE_MAX) {
      encoded[insertAt] = _UTF8_FIRST_BYTE_OF_FOUR_BASE | (
          _UTF8_FIRST_BYTE_OF_FOUR_MASK &
          _addToEncoding(insertAt, 3, value, encoded));
      insertAt += 4;
    }
  }
  return encoded;
}

Produce a String from a sequence of UTF-16 encoded bytes. This method always strips a leading BOM. Set the replacementCodepoint to null to throw an IllegalArgumentException rather than replace the bad value. The default value for the replacementCodepoint is U+FFFD.

String decodeUtf16(List<int> bytes, [int offset = 0, int length,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  Utf16BytesToCodeUnitsDecoder decoder = new Utf16BytesToCodeUnitsDecoder(bytes,
      offset, length, replacementCodepoint);
  List<int> codeunits = decoder.decodeRest();
  // TODO is16BitCodeUnit() is used to work around a bug with frog/dartc
  // (http://code.google.com/p/dart/issues/detail?id=1357). Consider
  // removing after this issue is resolved.
  if (_is16BitCodeUnit()) {
    return new String.fromCharCodes(codeunits);
  } else {
    return new String.fromCharCodes(
        _utf16CodeUnitsToCodepoints(codeunits, 0, null, replacementCodepoint));
  }
}

Decodes the UTF-16 bytes as an iterable. Thus, the consumer can only convert as much of the input as needed. Determines the byte order from the BOM, or uses big-endian as a default. This method always strips a leading BOM. Set the replacementCodepoint to null to throw an IllegalArgumentException rather than replace the bad value. The default value for replacementCodepoint is U+FFFD.

IterableUtf16Decoder decodeUtf16AsIterable(List<int> bytes, [int offset = 0,
    int length, int replacementCodepoint =
    UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new IterableUtf16Decoder._(
      () => new Utf16BytesToCodeUnitsDecoder(bytes, offset, length,
      replacementCodepoint), replacementCodepoint);
}

Produce a String from a sequence of UTF-16BE encoded bytes. This method strips a leading BOM by default, but can be overridden by setting the optional parameter stripBom to false. Set the replacementCodepoint to null to throw an IllegalArgumentException rather than replace the bad value. The default value for the replacementCodepoint is U+FFFD.

String decodeUtf16be(List<int> bytes, [int offset = 0, int length,
    bool stripBom = true,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  List<int> codeunits = (new Utf16beBytesToCodeUnitsDecoder(bytes, offset,
      length, stripBom, replacementCodepoint)).decodeRest();
  // TODO is16BitCodeUnit() is used to work around a bug with frog/dartc
  // (http://code.google.com/p/dart/issues/detail?id=1357). Consider
  // removing after this issue is resolved.
  if (_is16BitCodeUnit()) {
    return new String.fromCharCodes(codeunits);
  } else {
    return new String.fromCharCodes(
        _utf16CodeUnitsToCodepoints(codeunits, 0, null, replacementCodepoint));
  }
}

Decodes the UTF-16BE bytes as an iterable. Thus, the consumer can only convert as much of the input as needed. This method strips a leading BOM by default, but can be overridden by setting the optional parameter stripBom to false. Set the replacementCodepoint to null to throw an IllegalArgumentException rather than replace the bad value. The default value for the replacementCodepoint is U+FFFD.

IterableUtf16Decoder decodeUtf16beAsIterable(List<int> bytes, [int offset = 0,
    int length, bool stripBom = true, int replacementCodepoint =
    UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new IterableUtf16Decoder._(
      () => new Utf16beBytesToCodeUnitsDecoder(bytes, offset, length, stripBom,
      replacementCodepoint), replacementCodepoint);
}

Produce a String from a sequence of UTF-16LE encoded bytes. This method strips a leading BOM by default, but can be overridden by setting the optional parameter stripBom to false. Set the replacementCodepoint to null to throw an IllegalArgumentException rather than replace the bad value. The default value for the replacementCodepoint is U+FFFD.

String decodeUtf16le(List<int> bytes, [int offset = 0, int length,
    bool stripBom = true,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  List<int> codeunits = (new Utf16leBytesToCodeUnitsDecoder(bytes, offset,
      length, stripBom, replacementCodepoint)).decodeRest();
  // TODO is16BitCodeUnit() is used to work around a bug with frog/dartc
  // (http://code.google.com/p/dart/issues/detail?id=1357). Consider
  // removing after this issue is resolved.
  if (_is16BitCodeUnit()) {
    return new String.fromCharCodes(codeunits);
  } else {
    return new String.fromCharCodes(
        _utf16CodeUnitsToCodepoints(codeunits, 0, null, replacementCodepoint));
  }
}

Decodes the UTF-16LE bytes as an iterable. Thus, the consumer can only convert as much of the input as needed. This method strips a leading BOM by default, but can be overridden by setting the optional parameter stripBom to false. Set the replacementCodepoint to null to throw an IllegalArgumentException rather than replace the bad value. The default value for the replacementCodepoint is U+FFFD.

IterableUtf16Decoder decodeUtf16leAsIterable(List<int> bytes, [int offset = 0,
    int length, bool stripBom = true, int replacementCodepoint =
    UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new IterableUtf16Decoder._(
      () => new Utf16leBytesToCodeUnitsDecoder(bytes, offset, length, stripBom,
      replacementCodepoint), replacementCodepoint);
}

Produce a String from a sequence of UTF-32 encoded bytes. The parameters allow an offset into a list of bytes (as int), limiting the length of the values be decoded and the ability of override the default Unicode replacement character. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

String decodeUtf32(List<int> bytes, [int offset = 0, int length,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return codepointsToString((new Utf32BytesDecoder(bytes, offset, length,
      replacementCodepoint)).decodeRest());
}

Decodes the UTF-32 bytes as an iterable. Thus, the consumer can only convert as much of the input as needed. Determines the byte order from the BOM, or uses big-endian as a default. This method always strips a leading BOM. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

IterableUtf32Decoder decodeUtf32AsIterable(List<int> bytes, [
    int offset = 0, int length,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new IterableUtf32Decoder._(
      () => new Utf32BytesDecoder(bytes, offset, length, replacementCodepoint));
}

Produce a String from a sequence of UTF-32BE encoded bytes. The parameters allow an offset into a list of bytes (as int), limiting the length of the values be decoded and the ability of override the default Unicode replacement character. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

String decodeUtf32be(
    List<int> bytes, [int offset = 0, int length, bool stripBom = true,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) =>
  codepointsToString((new Utf32beBytesDecoder(bytes, offset, length, stripBom,
      replacementCodepoint)).decodeRest());

Decodes the UTF-32BE bytes as an iterable. Thus, the consumer can only convert as much of the input as needed. This method strips a leading BOM by default, but can be overridden by setting the optional parameter stripBom to false. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

IterableUtf32Decoder decodeUtf32beAsIterable(List<int> bytes, [
    int offset = 0, int length, bool stripBom = true,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new IterableUtf32Decoder._(
      () => new Utf32beBytesDecoder(bytes, offset, length, stripBom,
          replacementCodepoint));
}

Produce a String from a sequence of UTF-32LE encoded bytes. The parameters allow an offset into a list of bytes (as int), limiting the length of the values be decoded and the ability of override the default Unicode replacement character. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

String decodeUtf32le(
    List<int> bytes, [int offset = 0, int length, bool stripBom = true,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) =>
    codepointsToString((new Utf32leBytesDecoder(bytes, offset, length, stripBom,
        replacementCodepoint)).decodeRest());

Decodes the UTF-32LE bytes as an iterable. Thus, the consumer can only convert as much of the input as needed. This method strips a leading BOM by default, but can be overridden by setting the optional parameter stripBom to false. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

IterableUtf32Decoder decodeUtf32leAsIterable(List<int> bytes, [
    int offset = 0, int length, bool stripBom = true,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new IterableUtf32Decoder._(
      () => new Utf32leBytesDecoder(bytes, offset, length, stripBom,
          replacementCodepoint));
}

Produce a String from a List of UTF-8 encoded bytes. The parameters can set an offset into a list of bytes (as int), limit the length of the values to be decoded, and override the default Unicode replacement character. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

String decodeUtf8(List<int> bytes, [int offset = 0, int length,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return codepointsToString(
      (new Utf8Decoder(bytes, offset, length, replacementCodepoint))
      .decodeRest());
}

Decodes the UTF-8 bytes as an iterable. Thus, the consumer can only convert as much of the input as needed. Set the replacementCharacter to null to throw an IllegalArgumentException rather than replace the bad value.

IterableUtf8Decoder decodeUtf8AsIterable(List<int> bytes, [int offset = 0,
    int length,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new IterableUtf8Decoder(bytes, offset, length, replacementCodepoint);
}

Produce a list of UTF-16 encoded bytes. This method prefixes the resulting bytes with a big-endian byte-order-marker.

List<int> encodeUtf16(String str) =>
    encodeUtf16be(str, true);

Produce a list of UTF-16BE encoded bytes. By default, this method produces UTF-16BE bytes with no BOM.

List<int> encodeUtf16be(String str, [bool writeBOM = false]) {
  List<int> utf16CodeUnits = _stringToUtf16CodeUnits(str);
  List<int> encoding =
      new List<int>(2 * utf16CodeUnits.length + (writeBOM ? 2 : 0));
  int i = 0;
  if (writeBOM) {
    encoding[i++] = UNICODE_UTF_BOM_HI;
    encoding[i++] = UNICODE_UTF_BOM_LO;
  }
  for (int unit in utf16CodeUnits) {
    encoding[i++] = (unit & UNICODE_BYTE_ONE_MASK) >> 8;
    encoding[i++] = unit & UNICODE_BYTE_ZERO_MASK;
  }
  return encoding;
}

Produce a list of UTF-16LE encoded bytes. By default, this method produces UTF-16LE bytes with no BOM.

List<int> encodeUtf16le(String str, [bool writeBOM = false]) {
  List<int> utf16CodeUnits = _stringToUtf16CodeUnits(str);
  List<int> encoding =
      new List<int>(2 * utf16CodeUnits.length + (writeBOM ? 2 : 0));
  int i = 0;
  if (writeBOM) {
    encoding[i++] = UNICODE_UTF_BOM_LO;
    encoding[i++] = UNICODE_UTF_BOM_HI;
  }
  for (int unit in utf16CodeUnits) {
    encoding[i++] = unit & UNICODE_BYTE_ZERO_MASK;
    encoding[i++] = (unit & UNICODE_BYTE_ONE_MASK) >> 8;
  }
  return encoding;
}

Produce a list of UTF-32 encoded bytes. This method prefixes the resulting bytes with a big-endian byte-order-marker.

List<int> encodeUtf32(String str) =>
    encodeUtf32be(str, true);

Produce a list of UTF-32BE encoded bytes. By default, this method produces UTF-32BE bytes with no BOM.

List<int> encodeUtf32be(String str, [bool writeBOM = false]) {
  List<int> utf32CodeUnits = stringToCodepoints(str);
  List<int> encoding = new List<int>(4 * utf32CodeUnits.length +
      (writeBOM ? 4 : 0));
  int i = 0;
  if (writeBOM) {
    encoding[i++] = 0;
    encoding[i++] = 0;
    encoding[i++] = UNICODE_UTF_BOM_HI;
    encoding[i++] = UNICODE_UTF_BOM_LO;
  }
  for (int unit in utf32CodeUnits) {
    encoding[i++] = (unit >> 24) & UNICODE_BYTE_ZERO_MASK;
    encoding[i++] = (unit >> 16) & UNICODE_BYTE_ZERO_MASK;
    encoding[i++] = (unit >> 8) & UNICODE_BYTE_ZERO_MASK;
    encoding[i++] = unit & UNICODE_BYTE_ZERO_MASK;
  }
  return encoding;
}

Produce a list of UTF-32LE encoded bytes. By default, this method produces UTF-32BE bytes with no BOM.

List<int> encodeUtf32le(String str, [bool writeBOM = false]) {
  List<int> utf32CodeUnits = stringToCodepoints(str);
  List<int> encoding = new List<int>(4 * utf32CodeUnits.length +
      (writeBOM ? 4 : 0));
  int i = 0;
  if (writeBOM) {
    encoding[i++] = UNICODE_UTF_BOM_LO;
    encoding[i++] = UNICODE_UTF_BOM_HI;
    encoding[i++] = 0;
    encoding[i++] = 0;
  }
  for (int unit in utf32CodeUnits) {
    encoding[i++] = unit & UNICODE_BYTE_ZERO_MASK;
    encoding[i++] = (unit >> 8) & UNICODE_BYTE_ZERO_MASK;
    encoding[i++] = (unit >> 16) & UNICODE_BYTE_ZERO_MASK;
    encoding[i++] = (unit >> 24) & UNICODE_BYTE_ZERO_MASK;
  }
  return encoding;
}

Produce a sequence of UTF-8 encoded bytes from the provided string.

List<int> encodeUtf8(String str) =>
  codepointsToUtf8(stringToCodepoints(str));

Identifies whether a List of bytes starts (based on offset) with a byte-order marker (BOM).

bool hasUtf16Bom(List<int> utf32EncodedBytes, [int offset = 0, int length]) {
  return hasUtf16beBom(utf32EncodedBytes, offset, length) ||
      hasUtf16leBom(utf32EncodedBytes, offset, length);
}

Identifies whether a List of bytes starts (based on offset) with a big-endian byte-order marker (BOM).

bool hasUtf16beBom(List<int> utf16EncodedBytes, [int offset = 0, int length]) {
  int end = length != null ? offset + length : utf16EncodedBytes.length;
  return (offset + 2) <= end &&
      utf16EncodedBytes[offset] == UNICODE_UTF_BOM_HI &&
      utf16EncodedBytes[offset + 1] == UNICODE_UTF_BOM_LO;
}

Identifies whether a List of bytes starts (based on offset) with a little-endian byte-order marker (BOM).

bool hasUtf16leBom(List<int> utf16EncodedBytes, [int offset = 0, int length]) {
  int end = length != null ? offset + length : utf16EncodedBytes.length;
  return (offset + 2) <= end &&
      utf16EncodedBytes[offset] == UNICODE_UTF_BOM_LO &&
      utf16EncodedBytes[offset + 1] == UNICODE_UTF_BOM_HI;
}

Identifies whether a List of bytes starts (based on offset) with a byte-order marker (BOM).

bool hasUtf32Bom(
    List<int> utf32EncodedBytes, [int offset = 0, int length]) {
  return hasUtf32beBom(utf32EncodedBytes, offset, length) ||
      hasUtf32leBom(utf32EncodedBytes, offset, length);
}

Identifies whether a List of bytes starts (based on offset) with a big-endian byte-order marker (BOM).

bool hasUtf32beBom(List<int> utf32EncodedBytes, [int offset = 0, int length]) {
  int end = length != null ? offset + length : utf32EncodedBytes.length;
  return (offset + 4) <= end &&
      utf32EncodedBytes[offset] == 0 && utf32EncodedBytes[offset + 1] == 0 &&
      utf32EncodedBytes[offset + 2] == UNICODE_UTF_BOM_HI &&
      utf32EncodedBytes[offset + 3] == UNICODE_UTF_BOM_LO;
}

Identifies whether a List of bytes starts (based on offset) with a little-endian byte-order marker (BOM).

bool hasUtf32leBom(List<int> utf32EncodedBytes, [int offset = 0, int length]) {
  int end = length != null ? offset + length : utf32EncodedBytes.length;
  return (offset + 4) <= end &&
      utf32EncodedBytes[offset] == UNICODE_UTF_BOM_LO &&
      utf32EncodedBytes[offset + 1] == UNICODE_UTF_BOM_HI &&
      utf32EncodedBytes[offset + 2] == 0 && utf32EncodedBytes[offset + 3] == 0;
}

Provide a list of Unicode codepoints for a given string.

List<int> stringToCodepoints(String str) {
  List<int> codepoints;
  // TODO _is16BitCodeUnit() is used to work around a bug with frog/dartc
  // (http://code.google.com/p/dart/issues/detail?id=1357). Consider
  // removing after this issue is resolved.
  if (_is16BitCodeUnit()) {
    codepoints = _utf16CodeUnitsToCodepoints(str.charCodes());
  } else {
    codepoints = str.charCodes();
  }
  return codepoints;
}

List<int> utf8ToCodepoints(
    List<int> utf8EncodedBytes, [int offset = 0, int length,
    int replacementCodepoint = UNICODE_REPLACEMENT_CHARACTER_CODEPOINT]) {
  return new Utf8Decoder(utf8EncodedBytes, offset, length,
      replacementCodepoint).decodeRest();
}