_wp_scan_utf8( string $bytes, int $at, int $invalid_length, int|null $max_bytes = null, int|null $max_code_points = null, bool|null $has_noncharacters = null ): int

Description

// ñ is U+F1, which in ISO-8859-1/latin1/Windows-1252/cp1252 is 0xF1.
"Pi\xF1a" === $pineapple = mb_convert_encoding( "Piña", 'Windows-1252', 'UTF-8' );
$at = $invalid_length = 0;

// The first step finds the invalid 0xF1 byte.
2 === _wp_scan_utf8( $pineapple, $at, $invalid_length );
$at === 2; $invalid_length === 1;

// The second step continues to the end of the string.
1 === _wp_scan_utf8( $pineapple, $at, $invalid_length );
$at === 4; $invalid_length === 0;

Compatibility

Present in 3 of the 5 tracked releases, added in 6.9.0, and compiles on PHP 7.4 through 8.6-dev.

Parameters

$bytesstring

UTF-8 encoded string which might include invalid spans of bytes.

$atint

Where to start scanning.

$invalid_lengthint

Will be set to how many bytes are to be ignored after $at.

$max_bytesint|nulloptional

Stop scanning after this many bytes have been seen.Default: null

$max_code_pointsint|nulloptional

Stop scanning after this many code points have been seen.Default: null

$has_noncharactersbool|nulloptional

Set to indicate if scanned string contained noncharacters.Default: null

Return value

int

How many code points were successfully scanned.

Performance profile

How much work a call to _wp_scan_utf8() does, and what it touches: the algorithmic scaling, the Zend instruction count per call across PHP versions, the hooks it hands control to, and the core code that calls it. Measured from the compiled opcodes, not a stopwatch, so every number is identical on any machine running the same PHP version, and every function in core is ranked by cost.

Cost class

Trivial

Touches nothing outside its own arguments.

Scaling

Constant

No loop in the body: the same number of instructions runs whatever you pass in.

Instructions

22–121

Executed per call on PHP 8.5, depending on the branch taken. The body compiles to 278.

Plugin surface

None

Nothing here hands control to plugin code.

Called by

7

7 places in core call this, so the cost is paid more often than your own code shows.

What one call costs · 3 distinct outcomes

One number would be a lie: the work depends on which branch runs. These are every distinct cost _wp_scan_utf8() can have, taken from its control-flow graph on PHP 8.5.

This body has more branch combinations than are worth enumerating, so the table covers the outcomes found first rather than every one that exists.

Used by · 7

• _mb_ord()Internal compat function to mimic mb_ord().
• _wp_is_valid_utf8_fallback()Fallback mechanism for safely validating UTF-8 bytes.
• _wp_scrub_utf8_fallback()Fallback mechanism for replacing invalid spans of UTF-8 bytes.
• _wp_utf8_codepoint_count()Returns how many code points are found in the given UTF-8 string.
• _wp_utf8_codepoint_span()Given a starting offset within a string and a maximum number of code points, return how many bytes are occupied by the span of characters.
• _wp_utf8_decode_fallback()Converts a string from UTF-8 to ISO-8859-1, maintaining backwards compatibility with the deprecated function from the PHP standard library.
• antispambot()Obscures email addresses in HTML to prevent spam bots from harvesting them.

Source code

View on Trac ↗View on GitHub ↗

47function _wp_scan_utf8( string $bytes, int &$at, int &$invalid_length, ?int $max_bytes = null, ?int $max_code_points = null, ?bool &$has_noncharacters = null ): int {48	$byte_length       = strlen( $bytes );49	$end               = min( $byte_length, $at + ( $max_bytes ?? PHP_INT_MAX ) );50	$invalid_length    = 0;51	$count             = 0;52	$max_count         = $max_code_points ?? PHP_INT_MAX;53	$has_noncharacters = false;54 55	for ( $i = $at; $i < $end && $count <= $max_count; $i++ ) {56		/*57		 * Quickly skip past US-ASCII bytes, all of which are valid UTF-8.58		 *59		 * This optimization step improves the speed from 10x to 100x60		 * depending on whether the JIT has optimized the function.61		 */62		$ascii_byte_count = strspn(63			$bytes,64			"\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" .65			"\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" .66			" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\x7f",67			$i,68			min( $end - $i, $max_count - $count )69		);70 71		if ( $count + $ascii_byte_count >= $max_count ) {72			$at    = $i + ( $max_count - $count );73			$count = $max_count;74			return $count;75		}76 77		$count += $ascii_byte_count;78		$i     += $ascii_byte_count;79 80		if ( $i >= $end ) {81			$at = $end;82			return $count;83		}84 85		/**86		 * The above fast-track handled all single-byte UTF-8 characters. What87		 * follows MUST be a multibyte sequence otherwise there’s invalid UTF-8.88		 *89		 * Therefore everything past here is checking those multibyte sequences.90		 *91		 * It may look like there’s a need to check against the max bytes here,92		 * but since each match of a single character returns, this functions will93		 * bail already if crossing the max-bytes threshold. This function SHALL94		 * NOT return in the middle of a multi-byte character, so if a character95		 * falls on each side of the max bytes, the entire character will be scanned.96		 *97		 * Because it’s possible that there are truncated characters, the use of98		 * the null-coalescing operator with "\xC0" is a convenience for skipping99		 * length checks on every continuation bytes. This works because 0xC0 is100		 * always invalid in a UTF-8 string, meaning that if the string has been101		 * truncated, it will find 0xC0 and reject as invalid UTF-8.102		 *103		 * > [The following table] lists all of the byte sequences that are well-formed104		 * > in UTF-8. A range of byte values such as A0..BF indicates that any byte105		 * > from A0 to BF (inclusive) is well-formed in that position. Any byte value106		 * > outside of the ranges listed is ill-formed.107		 *108		 * > Table 3-7. Well-Formed UTF-8 Byte Sequences109		 *  ╭─────────────────────┬────────────┬──────────────┬─────────────┬──────────────╮110		 *  │ Code Points         │ First Byte │ Second Byte  │ Third Byte  │ Fourth Byte  │111		 *  ├─────────────────────┼────────────┼──────────────┼─────────────┼──────────────┤112		 *  │ U+0000..U+007F      │ 00..7F     │              │             │              │113		 *  │ U+0080..U+07FF      │ C2..DF     │ 80..BF       │             │              │114		 *  │ U+0800..U+0FFF      │ E0         │ A0..BF       │ 80..BF      │              │115		 *  │ U+1000..U+CFFF      │ E1..EC     │ 80..BF       │ 80..BF      │              │116		 *  │ U+D000..U+D7FF      │ ED         │ 80..9F       │ 80..BF      │              │117		 *  │ U+E000..U+FFFF      │ EE..EF     │ 80..BF       │ 80..BF      │              │118		 *  │ U+10000..U+3FFFF    │ F0         │ 90..BF       │ 80..BF      │ 80..BF       │119		 *  │ U+40000..U+FFFFF    │ F1..F3     │ 80..BF       │ 80..BF      │ 80..BF       │120		 *  │ U+100000..U+10FFFF  │ F4         │ 80..8F       │ 80..BF      │ 80..BF       │121		 *  ╰─────────────────────┴────────────┴──────────────┴─────────────┴──────────────╯122		 *123		 * @see https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-3/#G27506124		 */125 126		// Valid two-byte code points.

Changelog

Introduced in 6.9.0. Unchanged from 6.9.7 through 7.1.0.

1. 6.9.7
2. 7.0.4
3. 7.1.0

Signature, return type and hooks compared across 3 parsed releases.

About this page

Parsed data

Generated from the wordpress-develop 7.1.0 tag, from src/wp-includes/compat-utf8.php, and regenerated for each WordPress release so it tracks the code rather than a snapshot of it.

Corrections

Something wrong on this page? Report it and it gets fixed in the next regeneration.