vibed/pages/.beans/pages-r4ob--upload-dispatch-foundation-magic-byte-sniffing-tar.md

title	status	type	priority	created_at	updated_at	parent
Upload dispatch foundation: magic-byte sniffing + tar pipeline	completed	task	high	2026-07-21T04:15:03Z	2026-07-21T04:33:38Z	pages-w7ow

Shared groundwork all three compressed-tar formats depend on. Do this FIRST.

• Replace Content-Type dispatch in routes::api::validate_upload (api.rs ~100-119) with magic-byte sniffing on the buffered body. Magic: gzip 1f 8b, xz fd 37 7a 58 5a 00, zstd 28 b5 2f fd, zip 50 4b. Content-Type is a fallback hint only.
• Extend UploadKind in core/upload.rs to carry compressed-tar variants (suggest a Compression enum: Gzip/Zstd/Xz + a Tar kind).
• Add build_tar_manifest(bytes, compression, limits) mirroring build_zip_manifest: reuse check_entry_path, the seen/DuplicateEntry dedup, read_entry_limited, the root-level index.html requirement, max_entries, and all size limits. Skip tar directory/non-regular entries.
• Add a LimitedWriter (impl std::io::Write) that errors once cumulative bytes exceed max_uncompressed_total — used by the one-shot xz path.
• Add UploadError variants: InvalidTar and per-format decompression errors (or one DecompressionFailed { format }).
• tar reading: try the tar 0.4 crate first (Archive::new(reader).entries(), read each entry as a Read, no fs access). If it will not compile on wasm32-unknown-unknown (filetime/xattr), hand-roll a minimal ustar parser instead.

Preserve the zip-bomb guarantee: never trust declared sizes; count streamed decompressed bytes and abort on limit crossing.

Summary of Changes

Added detect_kind(bytes, content_type_hint) -> UploadKind (magic-byte sniffing: PK/gzip/xz/zstd, else Html) and wired routes::api::manifest_from_request to it (dropped the Content-Type mapping and the 'unsupported content type' 400). Added Compression enum, UploadKind::Tar(Compression), build_tar_manifest, read_tar_entries (mirrors build_zip_manifest: check_entry_path, seen-dedup, read_entry_limited, max_entries, root index.html), LimitedWriter, and UploadError::{InvalidTar, DecompressionFailed}. The tar 0.4 crate compiled cleanly for wasm32-unknown-unknown, so no hand-rolled ustar parser was needed in the production path.