Problem:
To support Server-Side Events (SSE, aka JS EventSource) it is
necessary for the server to keep open an HTTP request and dribble out
data (the event stream) as it is generated.
Currently, Rocket handles this poorly. It likes to fill in complete
chunks. Also there is no way to force a flush of the underlying
stream: in particular, there is a BufWriter underneath hyper. hyper
would honour a flush request, but there is no way to send one.
Options:
Ideally the code which is producing the data would be able to
explicitly designate when a flush should occur. Certainly it would
not be acceptable to flush all the time for all readers.
1. Invent a new kind of Body (UnbufferedChunked) which translates the
data from each Read::read() call into a single call to the stream
write, and which always flushes. This would be a seriously invasive
change. And it would mean that SSE systems with fast event streams
might work poorly.
2. Invent a new kind of Body which doesn't use Read at all, and
instead has a more sophisticated API. This would be super-invasive
and heavyweight.
3. Find a way to encode the necessary information in the Read trait
protocol.
Chosen solution:
It turns out that option 3 is quite easy. The read() call can return
an io::Error. There are at least some errors that clearly ought not
to occur here. An obvious one is ErrorKind::WouldBlock.
Rocket expects the reader to block. WouldBlock is only applicable to
nonblocking objects. And indeed the reader will generally want to
return it (once) when it is about to block.
We have the Stream treat io::Error with ErrorKind::WouldBlock, from
its reader, as a request to flush. There are two effects: we stop
trying to accumulate a full chunk, and we issue a flush call to the
underlying writer (which, eventually, makes it all the way down into
hyper and BufWriter).
Implementation:
We provide a method ReadExt::read_max_wfs which is like read_max but
which handles the WouldBlock case specially. It tells its caller
whether a flush was wanted.
This is implemented by adding a new code to read_max_internal. with a
boolean to control it. This seemed better than inventing a trait or
something. (The other read_max call site is reading http headers in
data.rs, and I think it wants to tread WouldBlock as an error.)
Risks and downsides:
Obviously this ad-hoc extension to the Read protocol is not
particularly pretty. At least, people who aren't doing SSE (or
similar) won't need it and can ignore it.
If for some reason the data source is actually nonblocking, this new
arrangement would spin, rather than calling the situation a fatal
error. This possibility seems fairly remote, in production settings
at least. To migitate this it might be possible for the loop in
Rocket::issue_response to bomb out if it notices it is sending lots of
consecutive empty chunks.
It is possible that async Rocket will want to take a different
approach entirely. But it will definitely need to solve this problem
somehow, and naively it seems like the obvious transformation to eg
the Tokio read trait would have the same API limitation and admit the
same solution. (Having a flush occur every time the body stream
future returns Pending would not be good for performance, I think.)
This commits also implement the query reform from #608. It also consists
of many, many breaking changes. Among them are:
* Query parts in route paths use new query reform syntax.
* Routing for queries is now lenient.
- Default ranking has changed to reflect query reform.
* Format routing matching has been fixed.
- Routes with formats matching "accept" will always collide.
- Routes with formats matching "content-type" require requests to
have an equivalent content-type header to match.
- Requests with imprecise content-types are treated as not having a
content-type.
* Generated routes and catchers respect visibility modifiers.
* Raw getter methods from request were renamed and retooled.
- In particular, the index parameter is based on segments in the
route path, not dynamic parameters.
* The method-based attributes no longer accept a keyed 'path'.
* The 'rocket_codegen' crate is gone and will no longer be public.
* The 'FormItems' iterator emits values of type 'FormItem'.
- The internal form items' string can no longer be retrieved.
* In general, routes are more strictly validated.
* Logging from codegen now funnels through logging infrastructure.
* Routing has been optimized by caching routing metadata.
Resolves#93.
Resolves#608.
Resolves#693.
Resolves#476.
The new 'FromData' trait allows an implementor to instruct the caller to
maintain state on its stack and later pass a borrow for processing.
Among other things, it greatly simplifies the 'Form' type, removing a
use of unsafe, and allows references in deserialized data guards.
This completes the migration of custom derives to proc-macros, removing
the need for the `custom_derive` feature in consumer code. This commit
also includes documentation, unit tests, and compile UI tests for each
of the derives.
Additionally, this commit improves the existing `FromForm` and
`FromFormValue` derives. The generated code for `FromForm` now returns
an error value indicating the error condition. The `FromFormValue`
derive now accepts a `form` attribute on variants for specifying the
exact value string to match against.
Closes#590.
Closes#670.
This is fairly large commit with several entangled logical changes.
The primary change in this commit is to completely overhaul how URI
handling in Rocket works. Prior to this commit, the `Uri` type acted as
an origin API. Its parser was minimal and lenient, allowing URIs that
were invalid according to RFC 7230. By contrast, the new `Uri` type
brings with it a strict RFC 7230 compliant parser. The `Uri` type now
represents any kind of valid URI, not simply `Origin` types. Three new
URI types were introduced:
* `Origin` - represents valid origin URIs
* `Absolute` - represents valid absolute URIs
* `Authority` - represents valid authority URIs
The `Origin` type replaces `Uri` in many cases:
* As fields and method inputs of `Route`
* The `&Uri` request guard is now `&Origin`
* The `uri!` macro produces an `Origin` instead of a `Uri`
The strict nature of URI parsing cascaded into the following changes:
* Several `Route` methods now `panic!` on invalid URIs
* The `Rocket::mount()` method is (correctly) stricter with URIs
* The `Redirect` constructors take a `TryInto<Uri>` type
* Dispatching of a `LocalRequest` correctly validates URIs
Overall, URIs are now properly and uniformly handled throughout Rocket's
codebase, resulting in a more reliable and correct system.
In addition to these URI changes, the following changes are also part of
this commit:
* The `LocalRequest::cloned_dispatch()` method was removed in favor of
chaining `.clone().dispatch()`.
* The entire Rocket codebase uses `crate` instead of `pub(crate)` as a
visibility modifier.
* Rocket uses the `crate_visibility_modifier` and `try_from` features.
A note on unsafety: this commit introduces many uses of `unsafe` in the
URI parser. All of these uses are a result of unsafely transforming byte
slices (`&[u8]` or similar) into strings (`&str`). The parser ensures
that these casts are safe, but of course, we must label their use
`unsafe`. The parser was written to be as generic and efficient as
possible and thus can parse directly from byte sources. Rocket, however,
does not make use of this fact and so would be able to remove all uses
of `unsafe` by parsing from an existing `&str`. This should be
considered in the future.
Fixes#443.
Resolves#263.
Sergio Benitez
Ian Jackson
jeb
Linus Unnebäck
Tal
Christophe Courtaut
Eric Dattore
Sean Stangl