Making the case for richer HTML in ActivityPub

Angus McLeod

@jupiter_rowland I'm curious if you actually posted twice or if your post appearing twice is a bug in one of our implementations.

julian:

We can do this, we can send richer HTML across the protocol in such a way that all those things you two mentioned — in-line images, embedded videos, tables, etc. — can all show up as intended by the sender.

I agree with you Julian. I think we'll need to tackle this piece by piece, but I'm on-board with the vision. I know this is what the users of Discourse want (they've been telling me!).

I wonder whether the most extensible way to do this is by sending markdown in addition to the HTML we already send. Markdown

has canonical (word of the day) markup for media objects (e.g. we'll all send images as ![image](https://imageurl.com))
is more secure than HTML
is (mostly?) already supported by the platforms who'd want to federate with each other for this richer content.

If we put an additional markdown representation which included the richer content we'd also avoid the HTML being rejected or subject to parsing we don't control on the grounds it contained unsupported tags or overly-complex structures.

silverpill

@julian @mike @johnonolan @mike As other commenters have noted, embedding whole articles is not really new, they are just not rendered properly by Mastodon. However, I don't think anyone have tried to embed an interactive application:

Get Started - Webxdc

(webxdc.org)

[email protected]

angus:

@jupiter_rowland I’m curious if you actually posted twice or if your post appearing twice is a bug in one of our implementations.

I think it's a bug on the Discourse side.

julian

@[email protected] said in Making the case for richer HTML in ActivityPub:

I wonder whether the most extensible way to do this is by sending markdown in addition to the HTML we already send.

NodeBB already sends markdown through, although I'm not sure whether sending different markdown from the HTML in content is the right approach (something something two wrong don't make a right)

@[email protected] is right about one thing, which is that if a critical mass expanded support, then other implementers would likely follow suit. It's always been this way, but usually it's Mastodon that has gained, and everybody else who's has to adapt.

I see two paths forward:

Compromise (@[email protected] suggested this years ago) — as:Note contains a limited subset of HTML (and images are attached), as:Article contains the richer set.
- Downsides: it's a technically contrived distinction as opposed to one based on the kind of content relative to others (e.g. content is an article because it contains an image, vs content is an article because it is considered a standalone work)
Ignore — send as:Article if we want to, and send the richer HTML set because we can.
- Downside: End users don't care, they just see that their post content doesn't make it through to Mastodon and complain

As much as I dislike the "best viewed in" phrase, Mastodon would ideally need to show an info label instructing users to go to the original site, or render richer HTML outside of their feed (like in a modal).

Mike Macgirvin 🖥️

Evan had a Farmville clone for ActivityPub's predecessor pump.io. We've dabbled in some other server-side apps, like putting chess games into your stream - but the killer is letting strangers execute javascript in your stream. We ran some experimental stuff in sandboxed iframes, but finally decided it was a bad look for the free web and said yeah,nah.

Laurens Hof

@julian @johnonolan @mike

Also summarised as "federation happens in the client"

Lutin Discret

@laurenshof @julian @johnonolan @mike I really hope that by "rich content", you mean "structured content" (headings, images, lists) not "formatted content" (font size, font colors, alignments). While I would accept the firsts, we should reject the seconds in both the standards and the apps: i don't want to live with people poor choice about fonts and colors (theming and accessibility will be issues).

Mike McCue

@jupiter_rowland For the record my perception of the fediverse is that it is the open web of all people, content, apps, services and websites that have federated or bridged with ActivityPub.

Mike McCue

@julian @johnonolan I’d love to learn more about this!

Mike McCue

@mikedev @silverpill yeah JavaScript in your stream would be problematic.

It would be interesting to figure out a safe and standard way to provide limited interactivity on the level of things like polls. Farcaster seems to be experimenting with ideas like this though i have not looked at it closely.

Evan Prodromou

@mikedev @silverpill it might be time to dust off the OpenSocial spec and consider how this could work in stream. I don't think arbitrary JS is a good idea, but it might be interesting to have user-installable widgets in a sandbox.

julian

@[email protected] I think even @[email protected] could agree that Mastodon has done the bare minimum of support. They used to only support straight text (and even now, composing in Mastodon is plaintext only), but have since allowed a heavily locked-down subset of HTML.

Allowing arbitrary JavaScript execution is a bad idea, but relaxing html sanitisation to the minimum would allow for much more expressive content. Inline images and tabular data are two tags I'd like to see allowed, but herein lies the problem... What subset of HTML do you feel comfortable with? I wager that different implementers would answer differently, and Mastodon eschewing certain tags in order to prioritize the textual microblogging style is their choice, too!

The fact of the matter is Mastodon needn't support all the tags under the sun, but they do need to acknowledge that received content is not faithfully represented in all cases, and so a prompt to view the content on the original site would be a good intermediate step.

There already exists an ActivityStreams object type for this kind of content: as:Article. It gives other implementers a chance to explicitly say "hey, this might be better viewed natively if you don't elect to show everything we send you". I'd love to make this a reality.

Evan Prodromou

@mike @mikedev @silverpill the design we built into AP is to make external apps that use OAuth 2.0 and the ActivityPub API to interact with your account.

https://evanp.me/2024/04/22/cross-server-interactions-in-activitypub/

Richard MacManus

@evan @mikedev @silverpill Federated start pages? Getting Netvibes vibes here

Kevin Marks

@julian @johnonolan @mike the origins of ActivityPub is Atom, which had the name/summary/content split, and we preserved that through the various Activity Streams iterations, from Atom to JSON to the current JSON plus LD.
Mastodon undermines this by ignoring post names and treating summary as a Content Warning.

ShadSterling

@julian @johnonolan @mike it still baffles me that the content encoding standards aren’t the same as email. The spec is already there, why not use it?

Osma A

@julian @johnonolan @mike
Would be remiss to not mention that Hometown, which is a "light" Mastodon fork supports the Article post type and has no issues rendering long form.
https://github.com/hometown-fork/hometown?tab=readme-ov-file#reading-more-content-types

[email protected]

First, it’s disturbing to find a “threadiverse working group” hosted inside a centralised exclusive walled garden (community.nodebb.org is a Cloudflare site that restricts access and undermines transparency). A threadiverse working group should be aligned with the core fediverse principle of decentralisation. It signals unfitness to lead the threadiverse. I will not be subscribing to this exclusive community or following it.

As for the topic at hand, we must separate the good ideas from the bad.

The good:

Embedding the whole article in the post – great idea for many reasons:
- the web has become mostly exclusive (Cloudflare, paywalls, tor hostility, loss of netneutrality), thus most links being shared are to places that restrict some demographics of people.
- enshitification of the web means everyone visiting the link will have to fight with dark patterns, tracking, cookie walls, popups, and countless other shitty downgrades on everyone’s attempt to consume content. By copying the content to an accessible enshitification-free post, everyone commenting can enjoy equal access to the content

The bad:

Enrichening the HTML risks bringing enshitification into the threadiverse. Of course there would be tooling that makes it easy to mirror a webpage into the post in an automated way. The garbage on the webpage that downgrades everyone’s UX will be replicated. Users are lazy as fuck. Instead of cleaning up the garbage they will just let it ride and burden everyone else.
Try looking at websites in lynx. The web has become a shitshow of chasing the shiny and breaking our basic need for reading text. It’s inherently exclusive because those on capped internet connections cannot afford the bandwidth for eye candy. A text-based client should be able to function well and interact with all the content. It should be accessible from an a11y standpoint.

eeeee

First, it’s disturbing to find a “threadiverse working group” hosted inside a centralised exclusive walled garden (community.nodebb.org is a Cloudflare site that restricts access and undermines transparency). A threadiverse working group should be aligned with the core fediverse principle of decentralisation. It signals unfitness to lead the threadiverse. I will not be subscribing to this exclusive community or following it.

Hi DiyRebel,
Just so I can follow, isnt Cloudfare the domain registrar? Why does it reflect badly, in your opinion, on a project?
Im not necessarily disagreeing with you, just wondering why having that company involved in the domain, 'tars' whatever is hosted via them?

julian

@eeeee don't engage, some people will always be unhappy unless they get everything for free.