#TIL the query component of a URI is actually completely opaque.
-
@trwnh @AT1ST @oblomov well one massive negative to exposing your search as POST only is they your edge network can’t cache similar responses.
The query normalization doesn’t matter because the web app will ensure queries are in a consistent order.
As an example if you do /api/user/search?name=a* (such as in a mention lookup) then we can cache this response on our edge.
Now technically you could also cache a POST, but most edge networks can’t cache by body.
-
Alexander The 1streplied to Adam Charron on last edited by
@Charrondev @trwnh @oblomov As I understand it, part of the problem is that the order of query parameters means that "search?query=foo&option=bar" caches differently than "search?option=bar&query=foo" - because if you don't do query normalization before caching, they "look" different to the cache.
-
Alexander The 1streplied to infinite love ⴳ on last edited by
-
infinite love ⴳreplied to Alexander The 1st on last edited by
@AT1ST @oblomov right, html forms + cgi-bin popularized it massively
at least for my purposes though (personal services, protocol design) it is enough to say that we should avoid query components in identifiers or otherwise we must define a consistent way to normalize/canonize them.
more generally, identifiers SHOULD be expressed in normal form where possible to avoid issues like domain.example being different from domain.example/
-
Adam Charronreplied to Alexander The 1st on last edited by
-
Erin 💽✨ 🔜 38C3replied to infinite love ⴳ on last edited by
-
@Charrondev @AT1ST @oblomov there’s no standard for this, so it has to be defined at a protocol level
-
infinite love ⴳreplied to Erin 💽✨ 🔜 38C3 on last edited by
@erincandescent @jbqueru right, i was trying to figure out what the uri and http specs actually said. that’s how i found out that query params aren’t real and that query component normalization is a per-app/per-protocol thing
luckily nothing in activitypub depends on parsing query components, but we still need to worry about producing such ids in the first place (each server needs to be consistent if they’re going to do it)
-
@trwnh @AT1ST @oblomov there’s no fixed standards for bodies either though or paths. Data can be encoded in paths in arbitrary ways (including query parameters).
The client and server must agree on a serialization format in both cases. For bodies I typically use JSON. For paths the format is documented and generated by the application when possible.
-
@trwnh @AT1ST @oblomov For example, I might have GET /api/discussions which lists resources. I likely want to support a mechanism for that paginating, sorting, and filtering. For this I document query parameters for the purpose.
The resources themself will have a canonical URL generated by the server. In my case /discussions/3131/some-arbitrary-slug/p1
This time the pagination is encoded in the URL as well as arbitrary data.
-
@Charrondev @AT1ST @oblomov right, but arbitrary paths/etc generally don’t break the opaqueness
let me put it this way. the expectation some (many?) people might have is that the query string can be delimited by & and reordered without changing the response. but it’s wrong to say that it doesn’t change the *identity*. it’s as wrong as expecting /foo/bar to be the same as /bar/foo — there is a common understanding that the path component is opaque, but we can’t say the same for a query component
-
@Charrondev @AT1ST @oblomov right, i get this. now imagine that instead of writing a single app/server you are trying to paginate, sort, and filter ActivityPub Collections across multiple non-cooperating server implementations. you need to define normalization and canonization rules and also just plain syntax for parsing parameters out of an opaque string. if you don’t define this, you have nothing to go off of. the surprising bit to a lot of people is that uri/http rfcs don’t include this.
-
infinite love ⴳreplied to infinite love ⴳ on last edited by
@Charrondev @AT1ST @oblomov now imagine this is all exposed prominently to users, some of whom know how to edit an address bar.
this is why i said it’s akin to url hacking upthread. the url hacking only works if the query string behaves as you expect it to.
more to the point of fedi: impls need to be aware that they might GET one id but the response contains a different (canonized) id. unfortunately this might very well break existing fedi implementations who are exceptionally brittle…
-
infinite love ⴳreplied to infinite love ⴳ on last edited by
@Charrondev @AT1ST @oblomov now arguably the “correct” thing to do is have the server redirect to the canonicalized resource instead of just serving the request as-is, but again, i don’t exactly have faith in 100 random devs doing the “correct” thing independently of each other.
-
@trwnh @AT1ST @oblomov the challenge is definitely much greater in a federated system.
In our case the caches aren’t the end all be-all and we have quite mature caching at multiple layers. The edge and end user caching is more a UX improvement (particularly for reducing latency) rather than to protect our servers. Expensive things should have their own backend caching and invalidation logic.
I see what you’re saying though.
-
@Charrondev @AT1ST @oblomov so i found out this kinda exists: https://httpwg.org/http-extensions/draft-ietf-httpbis-safe-method-w-body.html
> This specification defines a new HTTP method, QUERY, as a safe, idempotent request method that can carry request content
> The QUERY method provides a solution that spans the gap between the use of GET and POST
> Unlike POST, however, the method is explicitly safe and idempotent, allowing functions like caching and automatic retries to operate.
