Serializer redesign doc

doc/design/serializer_redesign.md

@@ -0,0 +1,305 @@
+# Serializer redesign
+
+## Summary
+
+This document describes a redesign of the library's message serializer. As in the existing design, the serializer uses a single block of memory allocated at construction time and never exceeds it. That memory is reused across messages. The serializer remains a strictly sans-I/O component, while stream handling moves to a separate `message_writer`. Body octets supplied by the caller are framed by reference, so a single gathered write can cover the header, chunk framing, and caller-owned memory without copying; small pieces are coalesced in a staging buffer that the caller can also write into directly. Because the serialized header is held back until output is actually flushed, the framing can still be improved after `start`: a chunked or close-delimited body whose total size becomes known before transfer begins is sent with an explicit `Content-Length` instead. Chunked trailers can now be sent, content codings are applied by an encoder the serializer selects from the message's `Content-Encoding` through encode services installed in the system context, and the accounting remains exact when an operation is cancelled mid-write, so an interrupted message can be resumed.
+
+The design is not speculative: it is implemented and tested in the Burl project (<https://github.com/cppalliance/burl>).
+
+## The serializer interface
+
+```cpp
+/// Per-coding settings for the content
+/// encoders.
+struct encoder_config
+{
+    /// level, window_bits, mem_level
+    struct zlib_settings;
+
+    /// quality, lgwin, lgblock, mode
+    struct brotli_settings;
+
+    /// level, window_log, strategy
+    struct zstd_settings;
+
+    zlib_settings zlib = {};
+    brotli_settings brotli = {};
+    zstd_settings zstd = {};
+};
+
+class serializer
+{
+public:
+    /// Settings that apply for the lifetime of
+    /// the serializer.
+    struct config
+    {
+        /// The space reserved for staging
+        /// body octets.
+        std::size_t stage_buffer = 64 * 1024;
+
+        /// The free staging capacity below
+        /// which should_drain reports true.
+        std::size_t min_prepare = 4 * 1024;
+
+        /// Supplied body data at least this
+        /// large is framed by reference,
+        /// without copying.
+        std::size_t min_direct = 2 * 1024;
+
+        /// The space reserved for the encoder
+        /// input stage; unused when no encoder
+        /// is installed.
+        std::size_t enc_buffer = 8 * 1024;
+
+        /// Bodies smaller than this may skip
+        /// encoding entirely.
+        std::size_t enc_threshold = 4 * 1024;
+
+        /// The content encoder settings; when
+        /// null, no body is encoded.
+        std::shared_ptr<encoder_config const> encoder;
+    };
+
+    /// Constructor; performs the single
+    /// allocation.
+    explicit
+    serializer(config const& cfg);
+
+    /// Return true if every octet of the
+    /// message, including any trailer, has
+    /// been consumed.
+    bool
+    is_done() const noexcept;
+
+    /// Return true if every octet of the
+    /// serialized header has been consumed.
+    bool
+    is_header_done() const noexcept;
+
+    /// Return true if staged octets should
+    /// be drained.
+    bool
+    should_drain() const noexcept;
+
+    /// Start a message; the framing is
+    /// selected from msg->payload(). `head`
+    /// serializes the response to a HEAD
+    /// request, emitting only the header.
+    void
+    start(
+        message_head_base* msg,
+        bool head = false);
+
+    /// Set the trailer fields, serialized
+    /// after the final chunk.
+    void
+    set_trailer(fields_base const* t) noexcept;
+
+    /// Return a buffer for writing body
+    /// octets into the staging area.
+    std::span<capy::mutable_buffer>
+    prepare(std::span<capy::mutable_buffer> dest);
+
+    /// Make octets written into the region
+    /// returned by prepare part of the body.
+    void
+    commit(std::size_t n) noexcept;
+
+    /// Assign `dest` with descriptors for the
+    /// octets to transfer next, framing the
+    /// supplied body octets.
+    template<capy::ConstBufferSequence CB>
+    system::result<
+        std::span<capy::const_buffer const>,
+        std::error_code>
+    frame(
+        std::span<capy::const_buffer> dest,
+        CB const& buffers,
+        bool more);
+
+    /// Equivalent to frame with an empty
+    /// buffer sequence.
+    system::result<
+        std::span<capy::const_buffer const>,
+        std::error_code>
+    frame(
+        std::span<capy::const_buffer> dest,
+        bool more);
+
+    /// Report transferred octets; return how
+    /// many supplied body octets were accepted.
+    std::size_t
+    consume(std::size_t n) noexcept;
+};
+```
+
+## Zero-copy framing of user-provided buffers
+
+The new serializer interface allows the user to provide their buffers during the framing step. Because serialization and framing happen in the same call, the serializer can use the supplied buffers directly as part of the destination buffer sequence that is expected to be written to the stream. This simple change provides a streaming interface for writing body data from an external source without copying it into the serializer's internal buffer.
+
+The following is a possible implementation of a `write_some` algorithm using this interface:
+
+```cpp
+template<
+    capy::WriteStream S,
+    capy::ConstBufferSequence CB>
+capy::io_task<std::size_t>
+write_some(
+    S& stream,
+    serializer& sr,
+    CB buffers,
+    bool more)
+{
+    capy::consuming_buffers cb(buffers);
+    capy::const_buffer dest[16];
+    std::size_t total = 0;
+    for(;;)
+    {
+        auto const bufs = sr.frame(dest, cb.data(), more);
+        if(bufs.has_error())
+            co_return { bufs.error(), total };
+        auto [ec, n] = co_await stream.write_some(*bufs);
+        auto const k = sr.consume(n);
+        cb.consume(k);
+        total += k;
+        if(ec || n == 0)
+            co_return { ec, total };
+    }
+}
+```
+
+## Optimization of I/O-layer write operations
+
+There are two new configuration parameters that allow optimization of the number of write operations at the I/O layer:
+
+- `config::min_direct` sets a threshold for buffer sizes that the serializer frames directly in the destination buffers for writing. Buffers smaller than this threshold are copied into the serializer's internal buffer, where they wait for more data or for the end of the message.
+
+- `config::min_prepare` determines the minimum amount of internal buffer space that the serializer provides to the user in calls to `prepare`. As long as the serializer can satisfy that requirement, it does not hint to the I/O layer that it should flush, thereby reducing the number of write operations.
+
+## Late framing decisions
+
+Because the serializer does not request a flush until there is a reason to do so (e.g. enough body data has accumulated, the end of the body has been declared, etc.), it may still alter the framing-related fields of the header:
+
+- When the complete body arrives before the first write, chunked or close-delimited framing is replaced by an explicit `Content-Length`, and `chunked` is removed from the `Transfer-Encoding` field.
+
+- When a content encoder is selected but the complete body is smaller than `config::enc_threshold`, the serializer may skip encoding entirely, remove the `Content-Encoding` field, and serialize the body unencoded.
+
+Both rewrites require that no header octet has been consumed. Setting a trailer suppresses the `Content-Length` rewrite because trailers require chunked coding. Once transfer of the header has begun, the message head is never modified.
+
+In practice, this means that the common pattern of handing the whole body to a single `write_eof` produces a `Content-Length`-framed message even when the caller never computed the size, while true streaming falls back to chunked framing automatically. For example, in the following scenario, the proper value for the `Content-Length` field is set automatically:
+
+```CPP
+response_head res;
+res.set_chunked(true); // total size unknown up front
+
+serializer sr(cfg);
+message_writer writer(&stream, &sr);
+sr.start(&res);
+
+std::string body = make_response();
+
+// The complete body arrives before the first write,
+// so the message goes out with an explicit
+// Content-Length instead of chunked framing.
+auto [ec, n] = co_await writer.write_eof(
+    capy::make_buffer(body));
+```
+
+## Trailer fields can now be sent
+
+The existing serializer has no way to emit trailer fields. In the new design, `set_trailer` installs a caller-owned field container whose wire image is serialized after the final chunk of a chunked body; with any other framing, the trailer is ignored.
+
+## Content encoders are selected through services
+
+When `config::encoder` is set, the serializer selects the content coding from the message passed to `start`: a body whose `Content-Encoding` names `gzip`, `deflate`, `br`, or `zstd` is encoded as it is serialized, using the encode service installed for that coding in the system context and the per-coding settings from `encoder_config`, and the encoder's output is framed in place of the body. A coding without an installed service, or one the serializer does not know, leaves the body as supplied. As described in the late-framing section above, the serializer may skip encoding and remove the `Content-Encoding` field entirely; a user can set `config::enc_threshold` to zero to force encoding for all body sizes.
+
+## No need for a serializer configuration service
+
+The serializer reads its configuration only during construction, at which point the single buffer allocation is performed and the tuning values are stored as members. Since the configuration is never consulted again, there is no need for a shared reference or a configuration service.
+
+## No need for storing user-provided buffer sequences
+
+The `frame` interface receives the user-provided buffer sequence and in the same call frames it into the destination buffer span. Any octets that are not accepted remain with the caller, which re-offers them on the next call, so the serializer never needs to flatten the sequence into an internal array of descriptors or keep it alive between calls.
+
+## Error handling
+
+All errors surface from `frame`: a call yields either descriptors to write or an error, never both, so nothing framed in a failed call ever reaches the wire. When `frame` reports an error, the message is failed and serialization cannot proceed; the only valid operations are `start` and destruction. Error codes are split according to where the contract was broken:
+
+- A body that disagrees with a size declared in the message, a `Content-Length` mismatch, or body octets supplied for a bodiless message or a HEAD response results in `error::body_size_mismatch`.
+
+- Breaking the interface contract under chunked or close-delimited framing (for example, supplying more octets after the end of the body has been declared) results in `std::errc::invalid_argument`.
+
+- An error returned by the encoder propagates as-is, and subsequent calls on the failed message return `std::errc::state_not_recoverable`.
+
+## Cancellation and resumption
+
+The serializer's accounting is designed to remain faithful to the wire when an operation is cancelled mid-write. Completion counts already reported by the stream cover exactly the consumed octets, `consume` translates them into accepted body octets, and no per-operation counters exist that could be lost with a cancelled coroutine frame. A caller resumes an interrupted message either by re-offering the unconsumed remainder of its buffers or by committing that remainder into the staging buffer and draining it; both approaches continue the message from precisely where the wire stopped.
+
+## Quality of the implementation
+
+### One gathered write covers the whole message
+
+`frame` returns a single span of descriptors containing the remaining header octets, the current chunk prefix merged with the staged octets, the supplied body by reference, the chunk epilogue, and the trailer headers. The chunk prefix is written backwards into a small margin directly ahead of the staging region, so the prefix and the staged data form one contiguous descriptor instead of two. A small message therefore goes out in a single `write_some`, and a large zero-copy body adds exactly one gathered write per buffer.
+
+## The `message_writer` interface
+
+```cpp
+template<capy::WriteStream S>
+class message_writer
+{
+public:
+    /// Constructor; the stream and serializer
+    /// must outlive the writer.
+    message_writer(S* stream, serializer* sr) noexcept;
+
+    /// Return writable staging memory.
+    std::span<capy::mutable_buffer>
+    prepare(std::span<capy::mutable_buffer> dest);
+
+    /// Commit staged octets, flushing the
+    /// staging buffer when it runs low.
+    capy::io_task<>
+    commit(std::size_t n);
+
+    /// Commit final octets and end the body.
+    capy::io_task<>
+    commit_eof(std::size_t n);
+
+    /// End the body with no more octets.
+    capy::io_task<>
+    write_eof();
+
+    /// Write the header and any staged data,
+    /// without ending the body.
+    capy::io_task<>
+    write_header();
+
+    /// Write at least one octet of `buffers`;
+    /// small inputs are coalesced without I/O.
+    template<capy::ConstBufferSequence CB>
+    capy::io_task<std::size_t>
+    write_some(CB buffers);
+
+    /// Write until `buffers` is fully consumed.
+    template<capy::ConstBufferSequence CB>
+    capy::io_task<std::size_t>
+    write(CB buffers);
+
+    /// Write final octets and end the body.
+    template<capy::ConstBufferSequence CB>
+    capy::io_task<std::size_t>
+    write_eof(CB buffers);
+};
+```
+
+The rationale for `message_writer` is to satisfy the `capy::WriteStream`, `capy::WriteSink` and `capy::BufferSink` concepts, so it composes with generic stream algorithms.
+
+`message_writer::write_header` flushes pending output without ending the body. This is the building block for `Expect: 100-continue` support in higher-level libraries, where the server must receive the header before the body is generated. Once it returns, the framing and encoding are fixed, and the late-framing rewrites no longer apply.
+
+## Related links
+
+Reference documentation for the Burl implementation:
+
+- [`serializer`](https://develop.burl.cpp.al/burl/reference/boost/burl/serializer.html)
+- [`message_writer`](https://develop.burl.cpp.al/burl/reference/boost/burl/message_writer.html)