Article vs. Note vs. Page

Renaud Chaput

@silverpill @julian @[email protected] @[email protected] @nutomic I am not very knowledgable about ActivityPub so I cant discuss the specifics, but we would like to improve how non-Note objects are processed/displayed in Mastodon.
If there are discussions about this, or a WG working on the topic, please send me the links so I can reference them for us to read when we start working on this

One of the ideas would be to support an embedded "view more" modal, that somehow displays the remote content.

silverpill

@renchap @pfefferle @julian @pfefferle @nutomic

I know that @dev attempted to start a group for working on this issue: https://mediaformat.org/wordress-activitypub/article-working-group/

cc @liaizon

Jeff Sikes

@pfefferle @simonwood @julian @nodebb Curious how you set the “priority on image/video” in ActivityPub? I didn’t know that was possible!

a

julian: as:Article: Represents any kind of multi-paragraph written work. as:Note: Represents a short written work typically less than a single paragraph in length. as:Page: Represents a Web Page. these are the formal definitions, yes, but implementation practice has diverged somewhat. i think "paragraph length" is a bit of a red herring, as there are Notes with multiple

tags in the activitystreams/pub examples. there are also Notes with names (titles). i go into this more here: https://socialhub.activitypub.rocks/t/federating-the-content-of-posts-note-articles-and-character-limits/4087/7 julian: Mastodon only treats as:Note (and as:Question) as a first-class object and relegates anything else to a fallback handler that takes a short snippet of the content, and shows a link back to the original source, thus losing any in-app benefits (boosts, replies, etc.) Whether this is actually true or not, I do not know. https://github.com/mastodon/mastodon/blob/12ed2d793b1b4823b0df047a47677bb0667bf43d/app/lib/activitypub/activity.rb#L9 https://github.com/mastodon/mastodon/blob/65093c619fdd1b18a4cf0c288051d8c524d5f434/app/lib/activitypub/activity/create.rb#L375 julian: Lemmy, perhaps out of principle, sends out an as:Page for new generated content, and only the replies federate out as as:Note. It has unfortunately led to some assertions that Lemmy's federation is "broken", even though it is arguably not the case. to be honest i'm not sure why they do this. it's not particularly "wrong", it's just a little weird. the thing that's a bigger issue for federation is using Announce to wrap activities that ought to be forwarded. most other fediverse implementations operate at an object level, so Announce means sharing the object. Announcing activities as a substitute for proper inbox forwarding means that some implementations may interpret all of these Announces as separate "posts". This is also more of an issue when you deal with the "activity stream" directly, i.e. if you were Facebook -- there, an Announce Create Page" would be equivalent to saying in english "John reshared Sally's Create" rather than "Sally created a web page" or "John reshared Sally's web page". bringing it back to object types though, the expectation of Page is not "a submission to a link aggregating service". i would expect it to be reserved for actual web pages, like how Tumblr lets you create custom web pages in your blog theme editor. so i'd say that it breaks down more like this: Article = formally published text, like Wordpress, typically has a title and permalink (which may be used for "conversion" logic as mastodon does currently), may be read in a feed reader or syndicated to other publishers Note = informal text, like a status update or chat message or forum post or a comment in a comments section, etc. Typically viewed in context of something else (e.g. some collection of Notes, like a profile page, replies collection, context collection, or so on) Page = structured content meant to be displayed in a web browser, e.g. Neocities, or Tumblr's "custom pages". julian: that ought to change. The question is how, but this WG is not at the point where we start throwing around decrees and making up standards. What's important to me right now is what the landscape looks like right now, and why that is the case. i don't think it's "making up standards". i think it's just standards. you're not decreeing anything, you're just describing how things ought to be done. and on an ecosystem level, you get buy-in from implementers of those standards -- of which there should at least be NodeBB, Discourse, and Flarum, right? the process of standards alignment can be slow and painful, but it should be done nonetheless. the landscape right now looks like what it looks like because there hasn't been any real protocol stewardship, and Mastodon emerged as a de facto steward of its own protocol. how we change that is by developing actual standards via FEP, WG, etc. -- and then advocating other implementers use those standards for better interoperability. put another way, it's a question of trust and authority. in the absence of a trusted authority, implementers will do whatever they want. part of the argument for activitypub as a standard is that it received w3c approval as an official Technical Recommendation. at worst, it's a rubber stamp by one of the highest authorities of web standards. at best, it's a foundation for sharing activity streams on the linked data web. but it takes a lot of work from implementers and alignment initiatives to make it reach that point. "if no one implements it, all you have is a spec". strategically, another part of "how we change that" is by making it easier for implementers doing the "wrong* thing to transition to doing the "right" thing. providing migration paths. maybe even sending them PRs, instead of waiting for them to do it. it's outreach.

infinite love ⴳ

@pfefferle @simonwood @julian @nodebb why "note with priority on inage/video attachments"? do wordpress post formats allow Image/Video posts with multiple attachments?

julian

@[email protected] said in Article vs. Note vs. Page:

For Mastodon, it's keepin' it real and sticking to purist microblogging. For Hubzilla, it's a way to spite them and their silly text formatting and image embedding antics. Hubzilla still holds a grudge against Mastodon for this.

Thanks for sharing this history. I'm certain Eugen would have a different view of how things went down, but nonetheless we have ended up at this impasse where a Note can have more than 500 characters, and some rich formatting, but not a full set (e.g. no block/inline images, horizontal rules, etc.)

@[email protected] said in Article vs. Note vs. Page:

And I'm not even sure if a vast alliance of Fediverse devs could change it.

Certainly that is one pessimistic way of looking at it. I am new to the fediverse developer scene and perhaps am still naive enough into thinking we can make a difference (<-- Mastodon users can't see it, but I put a smirking emoji there)

@[email protected] said in Article vs. Note vs. Page:

The conclusion I came to: Mastodon is probably more-or-less doing the right thing by refusing to render articles, but perhaps it would be nice to have a pop-out view for reading them.

Not every software needs to handle Article/Note differently. From a forum perspective, or at least in NodeBB, Articles and Notes are both just added to topics and the posts that make up topics can be of any length.

But additional handling could be introduced, like an optional "reader mode" for longer form content. Who knows.

@[email protected] said in Article vs. Note vs. Page:

I don't think that's the case, but maybe you meant something more specific than what you wrote? Several implementations support content-related AS2 content types (both base types and extensions) that do not fallback to as:Note or as:Question. For example, Funkwhale has "Libraries", "Tracks", "Audio" (etc.), PeerTube has "PlayLists" and "Video", PixelFed has "Story", Mobilizon has "Event", and so on.

You're right, and that's why I worded it like I did, because I personally don't know about the other softwares around and how they handle this particular use case. That's the main question I wanted answered, and you've helped me out there!

So lots of other federating software send out AS2 content types, that at least does reinforce the need for some sort of "generic" handler for unknown types.

At least from a the WG perspective we'd be concerned with Notes and higher order collections of Notes, but there's no telling where this may go!

@[email protected] said in Article vs. Note vs. Page:

on an ecosystem level, you get buy-in from implementers of those standards – of which there should at least be NodeBB, Discourse, and Flarum, right? the process of standards alignment can be slow and painful, but it should be done nonetheless. the landscape right now looks like what it looks like because there hasn’t been any real protocol stewardship, and Mastodon emerged as a de facto steward of its own protocol. how we change that is by developing actual standards via FEP, WG, etc. – and then advocating other implementers use those standards for better interoperability.

Part of the reason I started this topic was to confirm my suspicion as to why WordPress sends out Notes. Mastodon being a micro-blogging application means they have rationale for resisting attempts to add support for rich-text Articles and such (as @[email protected] implied), but it should never have gotten to the point where an implementor like @[email protected] would choose to use as:Note over as:Article.

Alignment from other implementors would apply some additional pressure for Mastodon to re-consider its handling of as:Article.

Matthias Pfefferle

@trwnh @simonwood @julian @nodebb WordPress does not have the concept of attachments. So we are parsing inline-media and add them as attachments before federating them. So yes, you can add as much media to your post as you want!

infinite love ⴳ

@pfefferle interesting. is there a concept of mixed-media formats, then? the reason I ask is because my first assumption would have been that Image maps to Image, in particular when it's a single Image. likewise, that Video maps to a single Video where possible. but i suppose that wouldn't completely work with multiple media files or mixed media, would it?

it may be worth thinking about alternative approaches like perhaps an "Album" extension type that is a Collection of Image/Video items?

Evan Prodromou

@trwnh @pfefferle I mean, that's what a Collection *is*

Matthias Pfefferle

@trwnh you can add as much media as you want with mixed media formats! So for normal notes and articles we take the media files in the order they are added to the text. For video, we search for videos first and fall back to others if there are no videos. Same for audio and image posts.

Matthias Pfefferle

@evan @trwnh and yes the plugin uses collections (adds multiple files if available)

Matthias Pfefferle

@evan @trwnh see this gallery post for example https://notiz.blog/2019/10/16/10-hwc-karlsruhe/

Here is the JSON version https://notiz.blog/2019/10/16/10-hwc-karlsruhe/?activitypub

infinite love ⴳ

@evan @pfefferle well, not quite, a Collection is a generic presentation mechanism and can contain anything, like ldp containers or a filesystem directory/folder... whereas a MediaAlbum type would be specifically a Collection containing Image/Video, kind of like media albums on Facebook. i suppose it could also be done as a “range” property of type:vocab, but having a dedicated type allows some impls to treat it as a “post” if desired

infinite love ⴳ

@pfefferle @evan the thing i’d be interested in thinking through is Create Collection (or a subtype) instead of Create Note, because the Collection itself may have relevant metadata. or more abstractly, defining what a wordpress Image or wordpress Video really means

Evan Prodromou

@trwnh you don't need a separate type for that. Creating, and managing collections is an intrinsic part of ActivityPub.

infinite love ⴳ

@evan the separate type would be specifically for signalling additional semantic meaning, namely "this is a media album", i.e. "this Collection contains only Image and Video items"

Mike Macgirvin 🖥️

@julian @Matthias Pfefferle @pfefferle @nutomic @silverpill @Renaud Chaput

Here's the reason Article became a second class citizen...

https://github.com/mastodon/mastodon/issues/5022

In this issue I raised against Mastodon in 2017 (on a now defunct github account), Mastodon at the time treated Note and Article identically. In particular, it removed all the HTML except for 'a' tags - even from Article. This made federation with the elephant impossible for us. At this time the ActivityPub fediverse consisted of Hubzilla and Mastodon. Period. The specification wasn't even final yet. Hubzilla provides long-form multi-media content, just like a blog. This content was completely destroyed by Mastodon's HTML sanitizer, especially blockquotes, which displayed everything we quoted as original text and mis-attributed.

My proposal to the Mastodon team (which was basically Eugen) was to relax the input sanitisation on the Article type a bit , and Mastodon could have their plaintext Note and we could have our multi-media and the fediverse be one happy family. Regardless of the fact that HTML is specified as the default content-type for all content in ActivityPub.

The response from Eugen was to turn Article into a link, meaning our content wouldn't be shown inline at all - and closing the issue. I believe this is the last time I ever communicated with Eugen and I will never, ever file another issue against Mastodon.

We started using Note instead, so that our messages would federate at all and knowing that Article would have been the most sensible choice.

We also need to strip all the images out of our perfectly renderable content and add them back in as attachments - otherwise they won't be displayed on Mastodon. As it turns out, Mastodon only adds back 4 images and reverses the order. This is less than satisfactory because the source content lets us position text around each image, and it forces anybody with multi-media content to not only perform this unnecessary step, but also to check every attachment on import and see if it was already included in the HTML - or it will be displayed twice.

As far as I'm concerned, Mastodon should be taken to the mountain-top and cast into the volcano. But it appears we're stuck with the infernal thing.

silverpill

@mikedev @nutomic @pfefferle @renchap @julian @pfefferle @nutomic That was a different time. Since then WordPress has joined the Fediverse, and now Ghost is about to join as well. I think the ability to display rich content may soon become a serious competitive advantage.

Jupiter Rowland

@julian

Not every software needs to handle Article/Note differently. From a forum perspective, or at least in NodeBB, Articles and Notes are both just added to topics and the posts that make up topics can be of any length.

Right. If your software has full support for long-form, formatted posts, especially if that includes creating them, and if your users are used to them, there's no need in handling them differently. See Friendica and its descendants.

Lewis Cowles

@julian @pfefferle @nutomic

I Read part of this as "Is Matthias lying". And although we've never spoke directly, I thought "What an absurd proposition".

It likely makes sense for things to be Note when mastodon wants to read them. I Wonder if there is a possibility of content negotiation, so that when mastodon asks (presumably with mastodon user-agent) it gets a note, but future improved software could have a different type.

It's an interesting problem, as you can never truly own all clients