Making the case for richer HTML in ActivityPub

Jupiter Rowland

It really seems like @Mike McCue's perception of the Fediverse is Flipboard, Mastodon and nothing else.

Posting long-form articles in the Fediverse with almost the full HTML feature set from text formatting to tables to any number of embedded in-line images is completely, utterly inconceivable to many Mastodon users. But this is nothing that is only just being introduced right now. It has been done in the Fediverse since 2010, almost six years before Mastodon was launched, when Rochko was still a school kid, and Friendica which introduced this was still Mistpark.

However, Mastodon users don't notice this happening. They can't. Mastodon flat-out refuses to support full HTML formatting, in-line images etc. simply because that's not what a Twitter clone should do. Thus, Article-type objects are reduced to links to the original which Mastodon users don't even perceive as still within the Fediverse because it doesn't directly show up in their timelines.

And Note-type objects are butchered by Mastodon's "sanitiser" almost beyond recognition. Even images are still fully being stripped out, which is why e.g. Pleroma, Akkoma, Friendica, Hubzilla and (streams) resort to converting embedded in-line images into file attachments. And then Mastodon only keeps a maximum of four of these and throws the rest away.

This wouldn't matter much if it weren't for Mastodon having a market share of 65% which feels more like 95%. It's absolutely possible for Mastodon users to have been around since late October, 2022 and still "know" that the Fediverse is only Mastodon.

Mike Macgirvin 🖥️

but is somehow completely new, which is weird...

Laughs. Not weird at all because it isn't new. We've always had rich multi-media HTML of unlimited length in the fediverse. Years before Mastodon was created. And we still have it today.

Somebody is literally stuck in a filter bubble.

Angus McLeod

@jupiter_rowland I'm curious if you actually posted twice or if your post appearing twice is a bug in one of our implementations.

julian:

We can do this, we can send richer HTML across the protocol in such a way that all those things you two mentioned — in-line images, embedded videos, tables, etc. — can all show up as intended by the sender.

I agree with you Julian. I think we'll need to tackle this piece by piece, but I'm on-board with the vision. I know this is what the users of Discourse want (they've been telling me!).

I wonder whether the most extensible way to do this is by sending markdown in addition to the HTML we already send. Markdown

has canonical (word of the day) markup for media objects (e.g. we'll all send images as ![image](https://imageurl.com))
is more secure than HTML
is (mostly?) already supported by the platforms who'd want to federate with each other for this richer content.

If we put an additional markdown representation which included the richer content we'd also avoid the HTML being rejected or subject to parsing we don't control on the grounds it contained unsupported tags or overly-complex structures.

silverpill

@julian @mike @johnonolan @mike As other commenters have noted, embedding whole articles is not really new, they are just not rendered properly by Mastodon. However, I don't think anyone have tried to embed an interactive application:

Get Started - Webxdc

(webxdc.org)

[email protected]

angus:

@jupiter_rowland I’m curious if you actually posted twice or if your post appearing twice is a bug in one of our implementations.

I think it's a bug on the Discourse side.

julian

@[email protected] said in Making the case for richer HTML in ActivityPub:

I wonder whether the most extensible way to do this is by sending markdown in addition to the HTML we already send.

NodeBB already sends markdown through, although I'm not sure whether sending different markdown from the HTML in content is the right approach (something something two wrong don't make a right)

@[email protected] is right about one thing, which is that if a critical mass expanded support, then other implementers would likely follow suit. It's always been this way, but usually it's Mastodon that has gained, and everybody else who's has to adapt.

I see two paths forward:

Compromise (@[email protected] suggested this years ago) — as:Note contains a limited subset of HTML (and images are attached), as:Article contains the richer set.
- Downsides: it's a technically contrived distinction as opposed to one based on the kind of content relative to others (e.g. content is an article because it contains an image, vs content is an article because it is considered a standalone work)
Ignore — send as:Article if we want to, and send the richer HTML set because we can.
- Downside: End users don't care, they just see that their post content doesn't make it through to Mastodon and complain

As much as I dislike the "best viewed in" phrase, Mastodon would ideally need to show an info label instructing users to go to the original site, or render richer HTML outside of their feed (like in a modal).

Mike Macgirvin 🖥️

Evan had a Farmville clone for ActivityPub's predecessor pump.io. We've dabbled in some other server-side apps, like putting chess games into your stream - but the killer is letting strangers execute javascript in your stream. We ran some experimental stuff in sandboxed iframes, but finally decided it was a bad look for the free web and said yeah,nah.

Laurens Hof

@julian @johnonolan @mike

Also summarised as "federation happens in the client"

Lutin Discret

@laurenshof @julian @johnonolan @mike I really hope that by "rich content", you mean "structured content" (headings, images, lists) not "formatted content" (font size, font colors, alignments). While I would accept the firsts, we should reject the seconds in both the standards and the apps: i don't want to live with people poor choice about fonts and colors (theming and accessibility will be issues).

Mike McCue

@jupiter_rowland For the record my perception of the fediverse is that it is the open web of all people, content, apps, services and websites that have federated or bridged with ActivityPub.

Mike McCue

@julian @johnonolan I’d love to learn more about this!

Mike McCue

@mikedev @silverpill yeah JavaScript in your stream would be problematic.

It would be interesting to figure out a safe and standard way to provide limited interactivity on the level of things like polls. Farcaster seems to be experimenting with ideas like this though i have not looked at it closely.

Evan Prodromou

@mikedev @silverpill it might be time to dust off the OpenSocial spec and consider how this could work in stream. I don't think arbitrary JS is a good idea, but it might be interesting to have user-installable widgets in a sandbox.

julian

@[email protected] I think even @[email protected] could agree that Mastodon has done the bare minimum of support. They used to only support straight text (and even now, composing in Mastodon is plaintext only), but have since allowed a heavily locked-down subset of HTML.

Allowing arbitrary JavaScript execution is a bad idea, but relaxing html sanitisation to the minimum would allow for much more expressive content. Inline images and tabular data are two tags I'd like to see allowed, but herein lies the problem... What subset of HTML do you feel comfortable with? I wager that different implementers would answer differently, and Mastodon eschewing certain tags in order to prioritize the textual microblogging style is their choice, too!

The fact of the matter is Mastodon needn't support all the tags under the sun, but they do need to acknowledge that received content is not faithfully represented in all cases, and so a prompt to view the content on the original site would be a good intermediate step.

There already exists an ActivityStreams object type for this kind of content: as:Article. It gives other implementers a chance to explicitly say "hey, this might be better viewed natively if you don't elect to show everything we send you". I'd love to make this a reality.

Evan Prodromou

@mike @mikedev @silverpill the design we built into AP is to make external apps that use OAuth 2.0 and the ActivityPub API to interact with your account.

https://evanp.me/2024/04/22/cross-server-interactions-in-activitypub/

Richard MacManus

@evan @mikedev @silverpill Federated start pages? Getting Netvibes vibes here

Kevin Marks

@julian @johnonolan @mike the origins of ActivityPub is Atom, which had the name/summary/content split, and we preserved that through the various Activity Streams iterations, from Atom to JSON to the current JSON plus LD.
Mastodon undermines this by ignoring post names and treating summary as a Content Warning.

ShadSterling

@julian @johnonolan @mike it still baffles me that the content encoding standards aren’t the same as email. The spec is already there, why not use it?

Osma A

@julian @johnonolan @mike
Would be remiss to not mention that Hometown, which is a "light" Mastodon fork supports the Article post type and has no issues rendering long form.
https://github.com/hometown-fork/hometown?tab=readme-ov-file#reading-more-content-types

[email protected]

First, it’s disturbing to find a “threadiverse working group” hosted inside a centralised exclusive walled garden (community.nodebb.org is a Cloudflare site that restricts access and undermines transparency). A threadiverse working group should be aligned with the core fediverse principle of decentralisation. It signals unfitness to lead the threadiverse. I will not be subscribing to this exclusive community or following it.

As for the topic at hand, we must separate the good ideas from the bad.

The good:

Embedding the whole article in the post – great idea for many reasons:
- the web has become mostly exclusive (Cloudflare, paywalls, tor hostility, loss of netneutrality), thus most links being shared are to places that restrict some demographics of people.
- enshitification of the web means everyone visiting the link will have to fight with dark patterns, tracking, cookie walls, popups, and countless other shitty downgrades on everyone’s attempt to consume content. By copying the content to an accessible enshitification-free post, everyone commenting can enjoy equal access to the content

The bad:

Enrichening the HTML risks bringing enshitification into the threadiverse. Of course there would be tooling that makes it easy to mirror a webpage into the post in an automated way. The garbage on the webpage that downgrades everyone’s UX will be replicated. Users are lazy as fuck. Instead of cleaning up the garbage they will just let it ride and burden everyone else.
Try looking at websites in lynx. The web has become a shitshow of chasing the shiny and breaking our basic need for reading text. It’s inherently exclusive because those on capped internet connections cannot afford the bandwidth for eye candy. A text-based client should be able to function well and interact with all the content. It should be accessible from an a11y standpoint.