Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

NodeBB

  1. Home
  2. Technical Discussion
  3. FEP-f228: Backfilling conversations has been updated:https://codeberg.org/fediverse/fep/pulls/853

FEP-f228: Backfilling conversations has been updated:https://codeberg.org/fediverse/fep/pulls/853

Scheduled Pinned Locked Moved Technical Discussion
fedidevfepfepf228
33 Posts 6 Posters 0 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • silverpill@mitra.socialS This user is from outside of this forum
    silverpill@mitra.socialS This user is from outside of this forum
    silverpill@mitra.social
    wrote last edited by
    #1

    FEP-f228: Backfilling conversations has been updated:
    https://codeberg.org/fediverse/fep/pulls/853

    I added tootik and Lemmy to the implementation list and did a little cleanup. This FEP feels complete, so I am requesting final comments.

    Full text:

    https://fediverse.codeberg.page/fep/fep/f228/

    #fep_f228 #fep #fedidev

    silverpill@mitra.socialS jupiter_rowland@hub.netzgemeinde.euJ mike@macgirvin.comM M 4 Replies Last reply
    0
    • silverpill@mitra.socialS silverpill@mitra.social

      FEP-f228: Backfilling conversations has been updated:
      https://codeberg.org/fediverse/fep/pulls/853

      I added tootik and Lemmy to the implementation list and did a little cleanup. This FEP feels complete, so I am requesting final comments.

      Full text:

      https://fediverse.codeberg.page/fep/fep/f228/

      #fep_f228 #fep #fedidev

      silverpill@mitra.socialS This user is from outside of this forum
      silverpill@mitra.socialS This user is from outside of this forum
      silverpill@mitra.social
      wrote last edited by
      #2

      @julian do you approve?

      julian@activitypub.spaceJ 1 Reply Last reply
      0
      • silverpill@mitra.socialS silverpill@mitra.social

        @julian do you approve?

        julian@activitypub.spaceJ This user is from outside of this forum
        julian@activitypub.spaceJ This user is from outside of this forum
        julian@activitypub.space
        wrote last edited by
        #3

        @silverpill@mitra.social looks fine to me, thanks <img class="not-responsive emoji" src="https://activitypub.space/assets/plugins/nodebb-plugin-emoji/emoji/android/1f44d.png?v=0e52d811655" title="👍" /> would be great to see it finalized.

        1 Reply Last reply
        0
        • silverpill@mitra.socialS silverpill@mitra.social

          FEP-f228: Backfilling conversations has been updated:
          https://codeberg.org/fediverse/fep/pulls/853

          I added tootik and Lemmy to the implementation list and did a little cleanup. This FEP feels complete, so I am requesting final comments.

          Full text:

          https://fediverse.codeberg.page/fep/fep/f228/

          #fep_f228 #fep #fedidev

          jupiter_rowland@hub.netzgemeinde.euJ This user is from outside of this forum
          jupiter_rowland@hub.netzgemeinde.euJ This user is from outside of this forum
          jupiter_rowland@hub.netzgemeinde.eu
          wrote last edited by
          #4
          @silverpill Generally, if (streams) has something implemented, ActivityPub-wise, then so does Forte. And vice versa.

          This means that Forte should be mentioned under Hubzilla and (streams).

          #FediMeta #FediverseMeta #CWFediMeta #CWFediverseMeta #ActivityPub #Streams #(streams) #Forte
          1 Reply Last reply
          0
          • silverpill@mitra.socialS silverpill@mitra.social

            FEP-f228: Backfilling conversations has been updated:
            https://codeberg.org/fediverse/fep/pulls/853

            I added tootik and Lemmy to the implementation list and did a little cleanup. This FEP feels complete, so I am requesting final comments.

            Full text:

            https://fediverse.codeberg.page/fep/fep/f228/

            #fep_f228 #fep #fedidev

            mike@macgirvin.comM This user is from outside of this forum
            mike@macgirvin.comM This user is from outside of this forum
            mike@macgirvin.com
            wrote last edited by
            #5
            I might recommend adding a pointer to a formal definition of "post" (as used here) for the benefit of newcomers.
            silverpill@mitra.socialS 1 Reply Last reply
            0
            • mike@macgirvin.comM mike@macgirvin.com
              I might recommend adding a pointer to a formal definition of "post" (as used here) for the benefit of newcomers.
              silverpill@mitra.socialS This user is from outside of this forum
              silverpill@mitra.socialS This user is from outside of this forum
              silverpill@mitra.social
              wrote last edited by
              #6

              @mike The formal definition of a 'thread' should suffice :]

              mike@macgirvin.comM 1 Reply Last reply
              0
              • silverpill@mitra.socialS silverpill@mitra.social

                @mike The formal definition of a 'thread' should suffice :]

                mike@macgirvin.comM This user is from outside of this forum
                mike@macgirvin.comM This user is from outside of this forum
                mike@macgirvin.com
                wrote last edited by
                #7
                OK, I need to push back this time. If your context collection contains enough information to rebuild a conversation from scratch, I don't care about the presentation. My context collection is legal ActivityStreams, as is a "posts" collection. It differs from a "posts" collection only in that it provides enough information to rebuild a restricted access conversation from scratch. A "posts" collection doesn't work well in that environment. In my experience it doesn't work at all, but I'll take any improvement in backfilling it might provide. If restricted access conversations matter to you, you might prefer to use signed activities in your presentation. If they don't, use any presentation that works for you.

                I believe that a "posts" collection has a limitation that backfilling contains no "likes" and not sure how you might rebuild the conversation accurately without them. This is why I'm looking for a clarification. I just want clarification that if I load your context that I still need to go out and hunt down reaction activities, or if you're going to provide them somehow (how?).  

                Otherwise, the precise presentation of the context element shouldn't be a deal-breaker issue for anybody at all.
                mike@macgirvin.comM 1 Reply Last reply
                0
                • mike@macgirvin.comM mike@macgirvin.com
                  OK, I need to push back this time. If your context collection contains enough information to rebuild a conversation from scratch, I don't care about the presentation. My context collection is legal ActivityStreams, as is a "posts" collection. It differs from a "posts" collection only in that it provides enough information to rebuild a restricted access conversation from scratch. A "posts" collection doesn't work well in that environment. In my experience it doesn't work at all, but I'll take any improvement in backfilling it might provide. If restricted access conversations matter to you, you might prefer to use signed activities in your presentation. If they don't, use any presentation that works for you.

                  I believe that a "posts" collection has a limitation that backfilling contains no "likes" and not sure how you might rebuild the conversation accurately without them. This is why I'm looking for a clarification. I just want clarification that if I load your context that I still need to go out and hunt down reaction activities, or if you're going to provide them somehow (how?).  

                  Otherwise, the precise presentation of the context element shouldn't be a deal-breaker issue for anybody at all.
                  mike@macgirvin.comM This user is from outside of this forum
                  mike@macgirvin.comM This user is from outside of this forum
                  mike@macgirvin.com
                  wrote last edited by
                  #8
                  A secondary issue is consistency. It manifests itself as the same cache problem we discussed recently with partial objects. Now we have two identifiers of the same name which provide different content depending on their "containing object instanceOf".

                  Please repeat after me: the context is the primary source of truth.

                  I really don't want multiple primary sources of truth that differ from each other. I doubt anybody else does either. If you can't parse my version of the truth, you probably won't be federating with me, and vice versa. That doesn't make my presentation invalid. If I can't parse and verify your version of the truth, I  won't be federating with you either. I really don't like "contextHistory" and propose that we simply let you display your version of the true conversation in a manner which allows participants to reconstruct it. Nothing more. When you involve restricted audiences, the requirements might be different than for public conversations. However, the "activities" presentation works well for both public and private content. The "posts" presentation is more suited to public content and doesn't work well with restricted audience communications. Again, this is an implementation concern -- I really don't think we need a formal protocol divide between such trivial implementation concerns. It kind of goes against everything I believe in.
                  silverpill@mitra.socialS 1 Reply Last reply
                  0
                  • mike@macgirvin.comM mike@macgirvin.com
                    A secondary issue is consistency. It manifests itself as the same cache problem we discussed recently with partial objects. Now we have two identifiers of the same name which provide different content depending on their "containing object instanceOf".

                    Please repeat after me: the context is the primary source of truth.

                    I really don't want multiple primary sources of truth that differ from each other. I doubt anybody else does either. If you can't parse my version of the truth, you probably won't be federating with me, and vice versa. That doesn't make my presentation invalid. If I can't parse and verify your version of the truth, I  won't be federating with you either. I really don't like "contextHistory" and propose that we simply let you display your version of the true conversation in a manner which allows participants to reconstruct it. Nothing more. When you involve restricted audiences, the requirements might be different than for public conversations. However, the "activities" presentation works well for both public and private content. The "posts" presentation is more suited to public content and doesn't work well with restricted audience communications. Again, this is an implementation concern -- I really don't think we need a formal protocol divide between such trivial implementation concerns. It kind of goes against everything I believe in.
                    silverpill@mitra.socialS This user is from outside of this forum
                    silverpill@mitra.socialS This user is from outside of this forum
                    silverpill@mitra.social
                    wrote last edited by
                    #9

                    @mike I agree that collection of activities is a better representation, because it is more complete, but this is not the subject of this FEP. The FEP is about backfilling, and its purpose is to document the current practices.

                    Implementers of this FEP are not required to publish any collections. However, they are advised to use a backfilling algorithm that supports all three conversation models (containers, threads and reply trees): https://fediverse.codeberg.page/fep/fep/f228/#reading-collections. The algorithm gives precedence to contextHistory, so collection of activities is used as a primary source when it is present.

                    mike@macgirvin.comM 1 Reply Last reply
                    1
                    • silverpill@mitra.socialS silverpill@mitra.social

                      @mike I agree that collection of activities is a better representation, because it is more complete, but this is not the subject of this FEP. The FEP is about backfilling, and its purpose is to document the current practices.

                      Implementers of this FEP are not required to publish any collections. However, they are advised to use a backfilling algorithm that supports all three conversation models (containers, threads and reply trees): https://fediverse.codeberg.page/fep/fep/f228/#reading-collections. The algorithm gives precedence to contextHistory, so collection of activities is used as a primary source when it is present.

                      mike@macgirvin.comM This user is from outside of this forum
                      mike@macgirvin.comM This user is from outside of this forum
                      mike@macgirvin.com
                      wrote last edited by
                      #10
                      When context property is present on a post, it MUST resolve to a collection of posts.

                      ...
                      When context property is present on an activity, it MUST resolve to a collection of activities.


                      In one place, we are asked to distinguish what type of thing we are fetching based on the name (context vs. contextHistory), and in another we are asked to distinguish based on the subclass of the element which contains "context".

                      Which is it? If it's both, how does that work? And since this is a MUST, once again, wtf is a "collection of posts"? I can't implement something if everybody refuses to define it.

                      And if "contextHistory" is an equal player with "context", every.single.fediverse.project now has to examine every context they copy to their own objects and activities, just to ensure they  get the right one in each case. And if both are present, copy both - after checking that "context" is or isn't the one they really want.  

                      And nobody is pushing back on this but me?
                      1 Reply Last reply
                      0
                      • julian@activitypub.spaceJ This user is from outside of this forum
                        julian@activitypub.spaceJ This user is from outside of this forum
                        julian@activitypub.space
                        wrote last edited by
                        #11

                        @mike@macgirvin.com collection of objects, if that makes sense.

                        Just a collection of Notes, Articles, Pages, whatnot

                        1 Reply Last reply
                        0
                        • mike@macgirvin.comM This user is from outside of this forum
                          mike@macgirvin.comM This user is from outside of this forum
                          mike@macgirvin.com
                          wrote last edited by
                          #12
                          So no likes/reactions, geolocation activities, invites, etc.. No audience restrictions beyond "public" and "unlisted". And lots of new code with issues that can never be resolved, but that our software needs to implement in order to comply with these new requirements -- and that nobody will ever use.  

                          Got it.  Don't know how that qualifies as conversation completion in even an abstract sense, but it's not my circus and those certainly aren't my monkeys.
                          1 Reply Last reply
                          0
                          • julian@activitypub.spaceJ This user is from outside of this forum
                            julian@activitypub.spaceJ This user is from outside of this forum
                            julian@activitypub.space
                            wrote last edited by
                            #13

                            > @mike@macgirvin.com said:
                            >
                            > but that our software needs to implement in order to comply with these new requirements -- and that nobody will ever use.

                            The wonderful thing about FEPs is you don't have to implement them, but you also don't get to decide who does.

                            1 Reply Last reply
                            0
                            • mike@macgirvin.comM This user is from outside of this forum
                              mike@macgirvin.comM This user is from outside of this forum
                              mike@macgirvin.com
                              wrote last edited by
                              #14
                              Certainly not. But all good. Thanks. Sorry for interrupting last call. Cheers.
                              1 Reply Last reply
                              0
                              • silverpill@mitra.socialS This user is from outside of this forum
                                silverpill@mitra.socialS This user is from outside of this forum
                                silverpill@mitra.social
                                wrote last edited by
                                #15

                                @mike

                                Which is it? If it's both, how does that work?

                                Yes, it's both. The backfilling algorithm described in the FEP should be used on a top-level post. When this post has a contextHistory property, it will resolve to a collection of activities. When it has a context property, it will resolve to a collection of posts.

                                wtf is a "collection of posts"

                                Alright, I will add a definition to the FEP.

                                "post" is any kind of attributed object. Usually it's a Note or an Article, but it could be an Event too, or something else entirely. What's important is that "post" is not an activity.

                                And if "contextHistory" is an equal player with "context", every.single.fediverse.project now has to examine every context they copy to their own objects and activities, just to ensure they get the right one in each case. And if both are present, copy both - after checking that "context" is or isn't the one they really want.

                                It shouldn't be a problem because with the proposed backfilling algorithm the collection item type is always known beforehand.

                                If it's a problem, you can use duck typing on collection items. That's what I do when I fetch an object whose class is not known.

                                julian@activitypub.spaceJ trwnh@mastodon.socialT 2 Replies Last reply
                                1
                                • silverpill@mitra.socialS silverpill@mitra.social

                                  @mike

                                  Which is it? If it's both, how does that work?

                                  Yes, it's both. The backfilling algorithm described in the FEP should be used on a top-level post. When this post has a contextHistory property, it will resolve to a collection of activities. When it has a context property, it will resolve to a collection of posts.

                                  wtf is a "collection of posts"

                                  Alright, I will add a definition to the FEP.

                                  "post" is any kind of attributed object. Usually it's a Note or an Article, but it could be an Event too, or something else entirely. What's important is that "post" is not an activity.

                                  And if "contextHistory" is an equal player with "context", every.single.fediverse.project now has to examine every context they copy to their own objects and activities, just to ensure they get the right one in each case. And if both are present, copy both - after checking that "context" is or isn't the one they really want.

                                  It shouldn't be a problem because with the proposed backfilling algorithm the collection item type is always known beforehand.

                                  If it's a problem, you can use duck typing on collection items. That's what I do when I fetch an object whose class is not known.

                                  julian@activitypub.spaceJ This user is from outside of this forum
                                  julian@activitypub.spaceJ This user is from outside of this forum
                                  julian@activitypub.space
                                  wrote last edited by
                                  #16

                                  @mike@macgirvin.com for what it's worth (and I say this not knowing how streams and forte work), processing activities and objects can be done in a way where code paths are reused. However I think that's easier when you're object-first (as NodeBB is), and if you rely on activities to provide context around the object, then a collection of objects makes little sense.

                                  In that case, not supporting the collection of objects is probably also okay.

                                  1 Reply Last reply
                                  0
                                  • mike@macgirvin.comM This user is from outside of this forum
                                    mike@macgirvin.comM This user is from outside of this forum
                                    mike@macgirvin.com
                                    wrote last edited by
                                    #17
                                    Nah, we'll implement a posts context collection. Then everybody (including threadiverse apps and also including Mastodon) will be able to see with their own eyes that it's a broken concept.
                                    1 Reply Last reply
                                    1
                                    • mike@macgirvin.comM This user is from outside of this forum
                                      mike@macgirvin.comM This user is from outside of this forum
                                      mike@macgirvin.com
                                      wrote last edited by
                                      #18
                                      It looks like others can expect an overall backfill effectiveness of about 8-12% from our software using objects.

                                      Wahoo!

                                      It's certainly better than the 0.0% we get trying to backfill from the same objects, because by definition these objects all belong to the un-moderated cesspool. Using contextHistory and object signatures, both apps should be able to successfully backfill 100% of the moderated conversation, minus anything they rejected due to their own content policies.

                                      #Works for me.
                                      1 Reply Last reply
                                      0
                                      • mike@macgirvin.comM This user is from outside of this forum
                                        mike@macgirvin.comM This user is from outside of this forum
                                        mike@macgirvin.com
                                        wrote last edited by
                                        #19
                                        I believe you can put forte in the implementation list now. Should tick all the boxes. I've updated the Conversations document to compare target and context without fragments or query params. Conversation Containers may require a similar update to make sure that's still valid; as the identifier and presentation format now needs to change depending on the placement of the context attribute.
                                        1 Reply Last reply
                                        0
                                        • silverpill@mitra.socialS This user is from outside of this forum
                                          silverpill@mitra.socialS This user is from outside of this forum
                                          silverpill@mitra.social
                                          wrote last edited by
                                          #20

                                          @mike Forte is already listed as an implementer of "collection of activities". Does it also provide "collection of posts"?

                                          @julian

                                          mike@macgirvin.comM 1 Reply Last reply
                                          0

                                          Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                          Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                          With your input, this post could be even better 💗

                                          Register Login
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          Powered by NodeBB Contributors
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups