You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
25 lines
2.2 KiB
Markdown
25 lines
2.2 KiB
Markdown
---
|
|
# pages-r4ob
|
|
title: 'Upload dispatch foundation: magic-byte sniffing + tar pipeline'
|
|
status: completed
|
|
type: task
|
|
priority: high
|
|
created_at: 2026-07-21T04:15:03Z
|
|
updated_at: 2026-07-21T04:33:38Z
|
|
parent: pages-w7ow
|
|
---
|
|
|
|
Shared groundwork all three compressed-tar formats depend on. Do this FIRST.
|
|
|
|
- Replace Content-Type dispatch in `routes::api::validate_upload` (api.rs ~100-119) with magic-byte sniffing on the buffered body. Magic: gzip `1f 8b`, xz `fd 37 7a 58 5a 00`, zstd `28 b5 2f fd`, zip `50 4b`. Content-Type is a fallback hint only.
|
|
- Extend `UploadKind` in `core/upload.rs` to carry compressed-tar variants (suggest a `Compression` enum: Gzip/Zstd/Xz + a `Tar` kind).
|
|
- Add `build_tar_manifest(bytes, compression, limits)` mirroring `build_zip_manifest`: reuse `check_entry_path`, the `seen`/DuplicateEntry dedup, `read_entry_limited`, the root-level `index.html` requirement, `max_entries`, and all size limits. Skip tar directory/non-regular entries.
|
|
- Add a `LimitedWriter` (impl std::io::Write) that errors once cumulative bytes exceed `max_uncompressed_total` — used by the one-shot xz path.
|
|
- Add `UploadError` variants: `InvalidTar` and per-format decompression errors (or one `DecompressionFailed { format }`).
|
|
- tar reading: try the `tar` 0.4 crate first (`Archive::new(reader).entries()`, read each entry as a Read, no fs access). If it will not compile on wasm32-unknown-unknown (filetime/xattr), hand-roll a minimal ustar parser instead.
|
|
|
|
Preserve the zip-bomb guarantee: never trust declared sizes; count streamed decompressed bytes and abort on limit crossing.
|
|
|
|
## Summary of Changes
|
|
Added `detect_kind(bytes, content_type_hint) -> UploadKind` (magic-byte sniffing: PK/gzip/xz/zstd, else Html) and wired `routes::api::manifest_from_request` to it (dropped the Content-Type mapping and the 'unsupported content type' 400). Added `Compression` enum, `UploadKind::Tar(Compression)`, `build_tar_manifest`, `read_tar_entries` (mirrors build_zip_manifest: check_entry_path, seen-dedup, read_entry_limited, max_entries, root index.html), `LimitedWriter`, and `UploadError::{InvalidTar, DecompressionFailed}`. The `tar` 0.4 crate compiled cleanly for wasm32-unknown-unknown, so no hand-rolled ustar parser was needed in the production path.
|