Code reference detail
text.rs
reqlan rq/extension/index-host/crates/reqlan-index/src/text.rs
| Idea | Status | Tags | Summary | Katz |
|---|---|---|---|---|
| text_prefix_skip | done | Before any index content read, inspect at most the first 8 KiB. Skip the file when that prefix has a NUL, a UTF-16 BOM ( `FF FE` or `FE FF` ), or invalid UTF-8. A trailing incomplete UTF-8 code unit is not a skip: the rest of the file may finish the character. Skip when the file is larger than 1 MiB. A rejected prefix is not followed by a read of the remainder. `.rq` files and allowlisted code files use the same check. This is in addition to [binary_ignore] extension globs. | 0.092 |
Reference labels
text_prefix_skip