Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

NodeBB

  1. Home
  2. Technical Discussion
  3. FEP-f228: Backfilling conversations has been updated:https://codeberg.org/fediverse/fep/pulls/853

FEP-f228: Backfilling conversations has been updated:https://codeberg.org/fediverse/fep/pulls/853

Scheduled Pinned Locked Moved Technical Discussion
fedidevfepfepf228
33 Posts 6 Posters 0 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • julian@activitypub.spaceJ This user is from outside of this forum
    julian@activitypub.spaceJ This user is from outside of this forum
    julian@activitypub.space
    wrote last edited by
    #11

    @mike@macgirvin.com collection of objects, if that makes sense.

    Just a collection of Notes, Articles, Pages, whatnot

    1 Reply Last reply
    0
    • mike@macgirvin.comM This user is from outside of this forum
      mike@macgirvin.comM This user is from outside of this forum
      mike@macgirvin.com
      wrote last edited by
      #12
      So no likes/reactions, geolocation activities, invites, etc.. No audience restrictions beyond "public" and "unlisted". And lots of new code with issues that can never be resolved, but that our software needs to implement in order to comply with these new requirements -- and that nobody will ever use.  

      Got it.  Don't know how that qualifies as conversation completion in even an abstract sense, but it's not my circus and those certainly aren't my monkeys.
      1 Reply Last reply
      0
      • julian@activitypub.spaceJ This user is from outside of this forum
        julian@activitypub.spaceJ This user is from outside of this forum
        julian@activitypub.space
        wrote last edited by
        #13

        > @mike@macgirvin.com said:
        >
        > but that our software needs to implement in order to comply with these new requirements -- and that nobody will ever use.

        The wonderful thing about FEPs is you don't have to implement them, but you also don't get to decide who does.

        1 Reply Last reply
        0
        • mike@macgirvin.comM This user is from outside of this forum
          mike@macgirvin.comM This user is from outside of this forum
          mike@macgirvin.com
          wrote last edited by
          #14
          Certainly not. But all good. Thanks. Sorry for interrupting last call. Cheers.
          1 Reply Last reply
          0
          • silverpill@mitra.socialS This user is from outside of this forum
            silverpill@mitra.socialS This user is from outside of this forum
            silverpill@mitra.social
            wrote last edited by
            #15

            @mike

            Which is it? If it's both, how does that work?

            Yes, it's both. The backfilling algorithm described in the FEP should be used on a top-level post. When this post has a contextHistory property, it will resolve to a collection of activities. When it has a context property, it will resolve to a collection of posts.

            wtf is a "collection of posts"

            Alright, I will add a definition to the FEP.

            "post" is any kind of attributed object. Usually it's a Note or an Article, but it could be an Event too, or something else entirely. What's important is that "post" is not an activity.

            And if "contextHistory" is an equal player with "context", every.single.fediverse.project now has to examine every context they copy to their own objects and activities, just to ensure they get the right one in each case. And if both are present, copy both - after checking that "context" is or isn't the one they really want.

            It shouldn't be a problem because with the proposed backfilling algorithm the collection item type is always known beforehand.

            If it's a problem, you can use duck typing on collection items. That's what I do when I fetch an object whose class is not known.

            julian@activitypub.spaceJ trwnh@mastodon.socialT 2 Replies Last reply
            1
            • silverpill@mitra.socialS silverpill@mitra.social

              @mike

              Which is it? If it's both, how does that work?

              Yes, it's both. The backfilling algorithm described in the FEP should be used on a top-level post. When this post has a contextHistory property, it will resolve to a collection of activities. When it has a context property, it will resolve to a collection of posts.

              wtf is a "collection of posts"

              Alright, I will add a definition to the FEP.

              "post" is any kind of attributed object. Usually it's a Note or an Article, but it could be an Event too, or something else entirely. What's important is that "post" is not an activity.

              And if "contextHistory" is an equal player with "context", every.single.fediverse.project now has to examine every context they copy to their own objects and activities, just to ensure they get the right one in each case. And if both are present, copy both - after checking that "context" is or isn't the one they really want.

              It shouldn't be a problem because with the proposed backfilling algorithm the collection item type is always known beforehand.

              If it's a problem, you can use duck typing on collection items. That's what I do when I fetch an object whose class is not known.

              julian@activitypub.spaceJ This user is from outside of this forum
              julian@activitypub.spaceJ This user is from outside of this forum
              julian@activitypub.space
              wrote last edited by
              #16

              @mike@macgirvin.com for what it's worth (and I say this not knowing how streams and forte work), processing activities and objects can be done in a way where code paths are reused. However I think that's easier when you're object-first (as NodeBB is), and if you rely on activities to provide context around the object, then a collection of objects makes little sense.

              In that case, not supporting the collection of objects is probably also okay.

              1 Reply Last reply
              0
              • mike@macgirvin.comM This user is from outside of this forum
                mike@macgirvin.comM This user is from outside of this forum
                mike@macgirvin.com
                wrote last edited by
                #17
                Nah, we'll implement a posts context collection. Then everybody (including threadiverse apps and also including Mastodon) will be able to see with their own eyes that it's a broken concept.
                1 Reply Last reply
                1
                • mike@macgirvin.comM This user is from outside of this forum
                  mike@macgirvin.comM This user is from outside of this forum
                  mike@macgirvin.com
                  wrote last edited by
                  #18
                  It looks like others can expect an overall backfill effectiveness of about 8-12% from our software using objects.

                  Wahoo!

                  It's certainly better than the 0.0% we get trying to backfill from the same objects, because by definition these objects all belong to the un-moderated cesspool. Using contextHistory and object signatures, both apps should be able to successfully backfill 100% of the moderated conversation, minus anything they rejected due to their own content policies.

                  #Works for me.
                  1 Reply Last reply
                  0
                  • mike@macgirvin.comM This user is from outside of this forum
                    mike@macgirvin.comM This user is from outside of this forum
                    mike@macgirvin.com
                    wrote last edited by
                    #19
                    I believe you can put forte in the implementation list now. Should tick all the boxes. I've updated the Conversations document to compare target and context without fragments or query params. Conversation Containers may require a similar update to make sure that's still valid; as the identifier and presentation format now needs to change depending on the placement of the context attribute.
                    1 Reply Last reply
                    0
                    • silverpill@mitra.socialS This user is from outside of this forum
                      silverpill@mitra.socialS This user is from outside of this forum
                      silverpill@mitra.social
                      wrote last edited by
                      #20

                      @mike Forte is already listed as an implementer of "collection of activities". Does it also provide "collection of posts"?

                      @julian

                      mike@macgirvin.comM 1 Reply Last reply
                      0
                      • silverpill@mitra.socialS silverpill@mitra.social

                        @mike Forte is already listed as an implementer of "collection of activities". Does it also provide "collection of posts"?

                        @julian

                        mike@macgirvin.comM This user is from outside of this forum
                        mike@macgirvin.comM This user is from outside of this forum
                        mike@macgirvin.com
                        wrote last edited by
                        #21
                        Yes,  whenever the context element is referenced from a naked object per the current FEP-f228. The definition of "posts" was very useful, as this is about the 4th time I've done this - but trying to still provide reactions since our software considers them to be an integral part of the conversation.

                        No reactions.

                        Got it.

                        Don't worry that's it's completely flawed as a conversation filling mechanism and won't work well for anybody in the current fediverse until they implement this FEP fully ---  a situation which I tried so hard to avoid.

                        Got it.  

                        So it'll work like shit for the next five years, or maybe forever in the case of Mastodon, but it should be fully compliant with FEP-f228
                        silverpill@mitra.socialS 1 Reply Last reply
                        0
                        • mike@macgirvin.comM mike@macgirvin.com
                          Yes,  whenever the context element is referenced from a naked object per the current FEP-f228. The definition of "posts" was very useful, as this is about the 4th time I've done this - but trying to still provide reactions since our software considers them to be an integral part of the conversation.

                          No reactions.

                          Got it.

                          Don't worry that's it's completely flawed as a conversation filling mechanism and won't work well for anybody in the current fediverse until they implement this FEP fully ---  a situation which I tried so hard to avoid.

                          Got it.  

                          So it'll work like shit for the next five years, or maybe forever in the case of Mastodon, but it should be fully compliant with FEP-f228
                          silverpill@mitra.socialS This user is from outside of this forum
                          silverpill@mitra.socialS This user is from outside of this forum
                          silverpill@mitra.social
                          wrote last edited by
                          #22

                          @mike

                          Yes, whenever the context element is referenced from a naked object per the current FEP-f228.

                          I tried to fetch context from one of your (private) posts, but the collection contains activities:

                          https://macgirvin.com/.well-known/apgateway/did:key:z6MkhPXNfiHDh2qSNjFzZ9yY27C1iHnHVbb1eaxuoiEe4tjk/conversation/bf59f105-711d-4c71-96a3-923e90f76e18?posts=true
                          

                          but trying to still provide reactions since our software considers them to be an integral part of the conversation

                          One way to provide reactions when only "collection of posts" is available is to use likes and emojiReactions collections, but I doubt that any implementation would actually resolve them.

                          mike@macgirvin.comM 1 Reply Last reply
                          0
                          • silverpill@mitra.socialS silverpill@mitra.social

                            @mike

                            Yes, whenever the context element is referenced from a naked object per the current FEP-f228.

                            I tried to fetch context from one of your (private) posts, but the collection contains activities:

                            https://macgirvin.com/.well-known/apgateway/did:key:z6MkhPXNfiHDh2qSNjFzZ9yY27C1iHnHVbb1eaxuoiEe4tjk/conversation/bf59f105-711d-4c71-96a3-923e90f76e18?posts=true
                            

                            but trying to still provide reactions since our software considers them to be an integral part of the conversation

                            One way to provide reactions when only "collection of posts" is available is to use likes and emojiReactions collections, but I doubt that any implementation would actually resolve them.

                            mike@macgirvin.comM This user is from outside of this forum
                            mike@macgirvin.comM This user is from outside of this forum
                            mike@macgirvin.com
                            wrote last edited by
                            #23
                            I get exactly one "post", but the collection metadata was incorrect and said it was a collection of 3, and a collectionOf 'activity'. Are you seeing actual activities, or just incorrect metadata? Might need to DM a snippet if the former. If the latter, that should be resolved  now.
                            silverpill@mitra.socialS 1 Reply Last reply
                            0
                            • mike@macgirvin.comM mike@macgirvin.com
                              I get exactly one "post", but the collection metadata was incorrect and said it was a collection of 3, and a collectionOf 'activity'. Are you seeing actual activities, or just incorrect metadata? Might need to DM a snippet if the former. If the latter, that should be resolved  now.
                              silverpill@mitra.socialS This user is from outside of this forum
                              silverpill@mitra.socialS This user is from outside of this forum
                              silverpill@mitra.social
                              wrote last edited by
                              #24

                              @mike I see activities, but I figured out why. The URL is normalized before fetching and ?posts=true parameter is removed during the normalization.

                              This is because query parameters are not significant in 'ap' URIs:

                              https://codeberg.org/fediverse/fep/src/branch/main/fep/ef61/fep-ef61.md#comparing-ap-uris

                              Not sure how to deal with this. I think collections shouldn't have query parameters in their IDs, but query parameters should be allowed in collection views (when filtering is applied).

                              mike@macgirvin.comM 1 Reply Last reply
                              0
                              • silverpill@mitra.socialS silverpill@mitra.social

                                @mike I see activities, but I figured out why. The URL is normalized before fetching and ?posts=true parameter is removed during the normalization.

                                This is because query parameters are not significant in 'ap' URIs:

                                https://codeberg.org/fediverse/fep/src/branch/main/fep/ef61/fep-ef61.md#comparing-ap-uris

                                Not sure how to deal with this. I think collections shouldn't have query parameters in their IDs, but query parameters should be allowed in collection views (when filtering is applied).

                                mike@macgirvin.comM This user is from outside of this forum
                                mike@macgirvin.comM This user is from outside of this forum
                                mike@macgirvin.com
                                wrote last edited by
                                #25
                                I might argue that what this represents is quite literally a "filtered view"...

                                But I could use a path instead.

                                My normalised-comparison function doesn't alter the original. Will review the portable objects implementation shortly and make certain I got that part right.
                                mike@macgirvin.comM 1 Reply Last reply
                                0
                                • mike@macgirvin.comM mike@macgirvin.com
                                  I might argue that what this represents is quite literally a "filtered view"...

                                  But I could use a path instead.

                                  My normalised-comparison function doesn't alter the original. Will review the portable objects implementation shortly and make certain I got that part right.
                                  mike@macgirvin.comM This user is from outside of this forum
                                  mike@macgirvin.comM This user is from outside of this forum
                                  mike@macgirvin.com
                                  wrote last edited by
                                  #26
                                  But I could use a path instead.


                                  Actually it appears that no,  I cannot. Then it is a different object in the eyes of conversation containers. (Because it is a filtered view). I don't see a lot of ways out of this dilemma that doesn't involve breaking something..

                                  [Edit: maybe being silly, but I could send back an array of collections, with the preferred presentation first. What we called multipart/alternative back in the old days. Most software will probably not handle this well, and I'm not even certain my own would. But I would be happy to implement it if it would allow us to all co-exist in the same fediverse.]
                                  silverpill@mitra.socialS 1 Reply Last reply
                                  0
                                  • mike@macgirvin.comM mike@macgirvin.com
                                    But I could use a path instead.


                                    Actually it appears that no,  I cannot. Then it is a different object in the eyes of conversation containers. (Because it is a filtered view). I don't see a lot of ways out of this dilemma that doesn't involve breaking something..

                                    [Edit: maybe being silly, but I could send back an array of collections, with the preferred presentation first. What we called multipart/alternative back in the old days. Most software will probably not handle this well, and I'm not even certain my own would. But I would be happy to implement it if it would allow us to all co-exist in the same fediverse.]
                                    silverpill@mitra.socialS This user is from outside of this forum
                                    silverpill@mitra.socialS This user is from outside of this forum
                                    silverpill@mitra.social
                                    wrote last edited by
                                    #27

                                    @mike

                                    I might argue that what this represents is quite literally a "filtered view"...

                                    That's fine, I fixed it on my side.

                                    However, now I'm getting 403 responses when ?posts=true is present. At the same time, unfiltered collection can be retrieved without issues.

                                    My normalised-comparison function doesn't alter the original. Will review the portable objects implementation shortly and make certain I got that part right.

                                    The normalization algorithm recommended in FEP-ef61 removes query parameters because there is a magic query parameter gateways, which is used for location hints:

                                    ap://did:key:z6MkrJVnaZkeFzdQyMZu1cgjg7k1pZZ6pvBQ7XJPt4swbTQ2/actor?gateways=https%3A%2F%2Fserver1.example,https%3A%2F%2Fserver2.example
                                    

                                    I thought that there will be more magic parameters and decided to "reserve" the entire query component (quoting section 'ap' URIs😞

                                    The query is OPTIONAL. To avoid future conflicts, implementers SHOULD NOT use parameter names that are not defined in this proposal.

                                    But of course, we need them to filter collections, so this is not a hard requirement (SHOULD, not MUST)...

                                    But I would be happy to implement it if it would allow us to all co-exist in the same fediverse.

                                    I think a filtered view is fine, but you could also choose to not publish this collection - it is not required by FEP-f228. Other implementations should be able to backfill conversation using the contextHistory property.

                                    mike@macgirvin.comM 1 Reply Last reply
                                    0
                                    • silverpill@mitra.socialS silverpill@mitra.social

                                      @mike

                                      Which is it? If it's both, how does that work?

                                      Yes, it's both. The backfilling algorithm described in the FEP should be used on a top-level post. When this post has a contextHistory property, it will resolve to a collection of activities. When it has a context property, it will resolve to a collection of posts.

                                      wtf is a "collection of posts"

                                      Alright, I will add a definition to the FEP.

                                      "post" is any kind of attributed object. Usually it's a Note or an Article, but it could be an Event too, or something else entirely. What's important is that "post" is not an activity.

                                      And if "contextHistory" is an equal player with "context", every.single.fediverse.project now has to examine every context they copy to their own objects and activities, just to ensure they get the right one in each case. And if both are present, copy both - after checking that "context" is or isn't the one they really want.

                                      It shouldn't be a problem because with the proposed backfilling algorithm the collection item type is always known beforehand.

                                      If it's a problem, you can use duck typing on collection items. That's what I do when I fetch an object whose class is not known.

                                      trwnh@mastodon.socialT This user is from outside of this forum
                                      trwnh@mastodon.socialT This user is from outside of this forum
                                      trwnh@mastodon.social
                                      wrote last edited by
                                      #28

                                      @silverpill @mike

                                      > What's important is that "post" is not an activity.

                                      I don't see why this is important. In my view activities are posts. If Mastodon et al disagree, that's not my problem.

                                      Also, the context can be anything. I don't see why a 2nd "contextHistory" property is necessary, nor am I entirely clear what it's supposed to mean semantically...

                                      1 Reply Last reply
                                      0
                                      • silverpill@mitra.socialS silverpill@mitra.social

                                        @mike

                                        I might argue that what this represents is quite literally a "filtered view"...

                                        That's fine, I fixed it on my side.

                                        However, now I'm getting 403 responses when ?posts=true is present. At the same time, unfiltered collection can be retrieved without issues.

                                        My normalised-comparison function doesn't alter the original. Will review the portable objects implementation shortly and make certain I got that part right.

                                        The normalization algorithm recommended in FEP-ef61 removes query parameters because there is a magic query parameter gateways, which is used for location hints:

                                        ap://did:key:z6MkrJVnaZkeFzdQyMZu1cgjg7k1pZZ6pvBQ7XJPt4swbTQ2/actor?gateways=https%3A%2F%2Fserver1.example,https%3A%2F%2Fserver2.example
                                        

                                        I thought that there will be more magic parameters and decided to "reserve" the entire query component (quoting section 'ap' URIs😞

                                        The query is OPTIONAL. To avoid future conflicts, implementers SHOULD NOT use parameter names that are not defined in this proposal.

                                        But of course, we need them to filter collections, so this is not a hard requirement (SHOULD, not MUST)...

                                        But I would be happy to implement it if it would allow us to all co-exist in the same fediverse.

                                        I think a filtered view is fine, but you could also choose to not publish this collection - it is not required by FEP-f228. Other implementations should be able to backfill conversation using the contextHistory property.

                                        mike@macgirvin.comM This user is from outside of this forum
                                        mike@macgirvin.comM This user is from outside of this forum
                                        mike@macgirvin.com
                                        wrote last edited by
                                        #29
                                        Conversation owner can add any activity to the conversation. However, if a context property is present on the activity, its value SHOULD be identical to the ID of a conversation container.


                                        Let's just remove this line from FEP-171b. Then I think I can make most everything else work.

                                        Tying the conversation target to the context in any way, shape, or form appears to be un-workable. That includes tying it to contextHistory instead. Thy are separate concepts, although implementations MAY choose to use the same identifiers for both.

                                        About the 403 - was it fetched by your site actor perchance? They aren't one of my followers. I'm not seeing any permission issues currently, though I'll keep investigating.
                                        1 Reply Last reply
                                        0
                                        • silverpill@mitra.socialS silverpill@mitra.social

                                          FEP-f228: Backfilling conversations has been updated:
                                          https://codeberg.org/fediverse/fep/pulls/853

                                          I added tootik and Lemmy to the implementation list and did a little cleanup. This FEP feels complete, so I am requesting final comments.

                                          Full text:

                                          https://fediverse.codeberg.page/fep/fep/f228/

                                          #fep_f228 #fep #fedidev

                                          M This user is from outside of this forum
                                          M This user is from outside of this forum
                                          mayel@activitypub.space
                                          wrote last edited by
                                          #30

                                          @silverpill@mitra.social @julian I've posted a proposal for following threads as an alternative to backfilling (and as an extension to how audience is defined in FEP-171b): https://codeberg.org/fediverse/fep/issues/449#issuecomment-17397479

                                          It is inspired by what @holossocial@mastodon.social is proposing: https://tom79.dev/posts/follow-a-note/

                                          1 Reply Last reply
                                          0

                                          Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                          Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                          With your input, this post could be even better 💗

                                          Register Login
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          Powered by NodeBB Contributors
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups