wp-includes/compat-utf8.php:47Finds spans of valid and invalid UTF-8 bytes in a given string.
ISO-8859-1/latin1/Windows-1252/cp1252 is 0xF1.
"Pi\xF1a" === $pineapple = mb_convert_encoding( "Piña", 'Windows-1252', 'UTF-8' );
$at = $invalid_length = 0;
// The first step finds the invalid 0xF1 byte.
2 === _wp_scan_utf8( $pineapple, $at, $invalid_length );
$at === 2; $invalid_length === 1;
// The second step continues to the end of the string.
1 === _wp_scan_utf8( $pineapple, $at, $invalid_length );
$at === 4; $invalid_length === 0; Note! While passing an options array here might be convenient from a calling-code standpoint, this function is intended to serve as a very low-level foundation upon which to build higher level functionality. For the sake of keeping costs explicit all arguments are passed directly.$bytesstring$atint$invalid_lengthint$max_bytesint|nulloptionalnull$max_code_pointsint|nulloptionalnull$has_noncharactersbool|nulloptionalnullintfunction _wp_scan_utf8( string $bytes, int &$at, int &$invalid_length, ?int $max_bytes = null, ?int $max_code_points = null, ?bool &$has_noncharacters = null ): int { $byte_length = strlen( $bytes ); $end = min( $byte_length, $at + ( $max_bytes ?? PHP_INT_MAX ) ); $invalid_length = 0; $count = 0; $max_count = $max_code_points ?? PHP_INT_MAX; $has_noncharacters = false; for ( $i = $at; $i < $end && $count <= $max_count; $i++ ) { /* * Quickly skip past US-ASCII bytes, all of which are valid UTF-8. * * This optimization step improves the speed from 10x to 100x * depending on whether the JIT has optimized the function. */ $ascii_byte_count = strspn( $bytes, "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" . "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" . " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f", $i, $end - $i ); if ( $count + $ascii_byte_count >= $max_count ) { $at = $i + ( $max_count - $count ); $count = $max_count; return $count; } $count += $ascii_byte_count; $i += $ascii_byte_count; if ( $i >= $end ) { $at = $end; return $count; } /** * The above fast-track handled all single-byte UTF-8 characters. What * follows MUST be a multibyte sequence otherwise there’s invalid UTF-8. * * Therefore everything past here is checking those multibyte sequences. * * It may look like there’s a need to check against the max bytes here, * but since each match of a single character returns, this functions will * bail already if crossing the max-bytes threshold. This function SHALL * NOT return in the middle of a multi-byte character, so if a character * falls on each side of the max bytes, the entire character will be scanned. * * Because it’s possible that there are truncated characters, the use of * the null-coalescing operator with "\xC0" is a convenience for skipping * length checks on every continuation bytes. This works because 0xC0 is * always invalid in a UTF-8 string, meaning that if the string has been * truncated, it will find 0xC0 and reject as invalid UTF-8. * * > [The following table] lists all of the byte sequences that are well-formed * > in UTF-8. A range of byte values such as A0..BF indicates that any byte * > from A0 to BF (inclusive) is well-formed in that position. Any byte value * > outside of the ranges listed is ill-formed. * * > Table 3-7. Well-Formed UTF-8 Byte Sequences * ╭─────────────────────┬────────────┬──────────────┬─────────────┬──────────────╮ * │ Code Points │ First Byte │ Second Byte │ Third Byte │ Fourth Byte │ * ├─────────────────────┼────────────┼──────────────┼─────────────┼──────────────┤ * │ U+0000..U+007F │ 00..7F │ │ │ │ * │ U+0080..U+07FF │ C2..DF │ 80..BF │ │ │ * │ U+0800..U+0FFF │ E0 │ A0..BF │ 80..BF │ │ * │ U+1000..U+CFFF │ E1..EC │ 80..BF │ 80..BF │ │ * │ U+D000..U+D7FF │ ED │ 80..9F │ 80..BF │ │ * │ U+E000..U+FFFF │ EE..EF │ 80..BF │ 80..BF │ │ * │ U+10000..U+3FFFF │ F0 │ 90..BF │ 80..BF │ 80..BF │ * │ U+40000..U+FFFFF │ F1..F3 │ 80..BF │ 80..BF │ 80..BF │ * │ U+100000..U+10FFFF │ F4 │ 80..8F │ 80..BF │ 80..BF │ * ╰─────────────────────┴────────────┴──────────────┴─────────────┴──────────────╯ * * @see https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G27506 */ // Valid two-byte code points.Introduced in 6.9.0. Unchanged from 6.9.7 through 7.1.0.
Signature, return type and hooks compared across 3 parsed releases.
src/wp-includes/compat-utf8.php, and regenerated for each WordPress release so it tracks the code rather than a snapshot of it.