Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

NodeBB

  1. Home
  2. General Discussion
  3. With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts.

With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts.

Scheduled Pinned Locked Moved General Discussion
holosdiscovernobotnoindexactivitypub
30 Posts 18 Posters 0 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • apps@toot.fedilab.appA apps@toot.fedilab.app

    With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
    Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
    We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

    raphaelmorgan@disabled.socialR This user is from outside of this forum
    raphaelmorgan@disabled.socialR This user is from outside of this forum
    raphaelmorgan@disabled.social
    wrote last edited by
    #5

    @apps I see no problem with this approach 🤷 generally opt-in is better than opt-out, so if there's a way to make it opt-in even for people on servers that allow indexing by default that would probably decrease the backlash, but like... If we don't want our posts indexed, we can turn indexing off and it seems y'all were respecting that so I don't mind

    1 Reply Last reply
    0
    • apps@toot.fedilab.appA apps@toot.fedilab.app

      With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
      Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
      We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

      apps@toot.fedilab.appA This user is from outside of this forum
      apps@toot.fedilab.appA This user is from outside of this forum
      apps@toot.fedilab.app
      wrote last edited by
      #6

      We were caught off guard by the reactions and acted quickly. But we want to give the community a voice. We'll follow the result of this poll: if it should have stayed up, we'll reopen the source code and bring the service back.

      lexinova@cyberplace.socialL catsalad@infosec.exchangeC 2 Replies Last reply
      0
      • apps@toot.fedilab.appA apps@toot.fedilab.app

        With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
        Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
        We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

        lutindiscret@mastodon.libre-entreprise.comL This user is from outside of this forum
        lutindiscret@mastodon.libre-entreprise.comL This user is from outside of this forum
        lutindiscret@mastodon.libre-entreprise.com
        wrote last edited by
        #7

        @apps if an account is marked as indexable, just index it. If the default is wrong, that's the instance admin fault, nit yours.

        If Google can index it you can too.

        If people are unhappy with the setting of their profile, they can change it or blame the software or the admin and not people of good faith like you.

        Public toots are talking to the world in a megaphone by design, expecting privacy is absurd

        apps@toot.fedilab.appA 1 Reply Last reply
        0
        • apps@toot.fedilab.appA apps@toot.fedilab.app

          @lexinova
          The debate goes beyond that. The real question is why developers chose to enable "indexable" by default when creating an account. A default opt-in is effectively an opt-out. That's where the conversation should start.

          lutindiscret@mastodon.libre-entreprise.comL This user is from outside of this forum
          lutindiscret@mastodon.libre-entreprise.comL This user is from outside of this forum
          lutindiscret@mastodon.libre-entreprise.com
          wrote last edited by
          #8

          @apps this 100℅ the wrong people in being blamed here

          1 Reply Last reply
          0
          • lutindiscret@mastodon.libre-entreprise.comL lutindiscret@mastodon.libre-entreprise.com

            @apps if an account is marked as indexable, just index it. If the default is wrong, that's the instance admin fault, nit yours.

            If Google can index it you can too.

            If people are unhappy with the setting of their profile, they can change it or blame the software or the admin and not people of good faith like you.

            Public toots are talking to the world in a megaphone by design, expecting privacy is absurd

            apps@toot.fedilab.appA This user is from outside of this forum
            apps@toot.fedilab.appA This user is from outside of this forum
            apps@toot.fedilab.app
            wrote last edited by
            #9

            @lutindiscret
            We panicked quickly because things escalated beyond what we expected. But we still believe our approach was far more respectful than what's already happening: the same data is being exploited by others without users even knowing. At worst, we raised awareness.

            iinavpov@mastodon.onlineI deutrino@mstdn.ioD 2 Replies Last reply
            0
            • apps@toot.fedilab.appA apps@toot.fedilab.app

              With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
              Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
              We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

              scriptfanix@maly.ioS This user is from outside of this forum
              scriptfanix@maly.ioS This user is from outside of this forum
              scriptfanix@maly.io
              wrote last edited by
              #10

              @apps Tough one: I feel like you did set it up respectfully, so it should have stayed up. BUT taking it down when people voice concerns and objected was also the right thing to do, both morally and for the reputation of the Holos project.

              1 Reply Last reply
              0
              • apps@toot.fedilab.appA apps@toot.fedilab.app

                We were caught off guard by the reactions and acted quickly. But we want to give the community a voice. We'll follow the result of this poll: if it should have stayed up, we'll reopen the source code and bring the service back.

                lexinova@cyberplace.socialL This user is from outside of this forum
                lexinova@cyberplace.socialL This user is from outside of this forum
                lexinova@cyberplace.social
                wrote last edited by
                #11

                @apps maybe make a post for the most used (mastodon), on witch settings change for people that do not want to allow this.

                As they may have missed this or misunderstood what the switch did

                And since one of the two is recommemded, it may make them allow thing they actually don't want 🙂

                some may misunderstand both settings:

                - Include public posts in search results
                - Feature profile and posts in discovery algorithms

                apps@toot.fedilab.appA 1 Reply Last reply
                0
                • apps@toot.fedilab.appA apps@toot.fedilab.app

                  We were caught off guard by the reactions and acted quickly. But we want to give the community a voice. We'll follow the result of this poll: if it should have stayed up, we'll reopen the source code and bring the service back.

                  catsalad@infosec.exchangeC This user is from outside of this forum
                  catsalad@infosec.exchangeC This user is from outside of this forum
                  catsalad@infosec.exchange
                  wrote last edited by
                  #12

                  @apps Fediverse search engines are a contentious subject with a bad history, so I undestand you shutting it down.

                  While I'm fine with it, many people are very against things like that.

                  1 Reply Last reply
                  0
                  • apps@toot.fedilab.appA apps@toot.fedilab.app

                    With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
                    Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
                    We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

                    stomata@procial.tchncs.deS This user is from outside of this forum
                    stomata@procial.tchncs.deS This user is from outside of this forum
                    stomata@procial.tchncs.de
                    wrote last edited by
                    #13

                    @apps@toot.fedilab.app accidentally voted wrong. I think it should stay. Search on small instances suck.

                    1 Reply Last reply
                    0
                    • apps@toot.fedilab.appA apps@toot.fedilab.app

                      With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
                      Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
                      We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

                      tati@eldritch.cafeT This user is from outside of this forum
                      tati@eldritch.cafeT This user is from outside of this forum
                      tati@eldritch.cafe
                      wrote last edited by
                      #14

                      @apps the things that concerned me at the time were, in the initial announcement, 'we follow every signal' but only in this post did we know what signals you were following; and language of 'people may have misunderstood the defaults' which unfortunately rhymes with what less conscientious organisations say to justify hoovering all the things

                      if i had known the list of things you were looking at, i would have simply checked them and slept well. i didn't, so i blocked the bot, which now proves to have been unnecessary

                      andi@snac.sonnenmulde.atA 1 Reply Last reply
                      0
                      • lexinova@cyberplace.socialL lexinova@cyberplace.social

                        @apps maybe make a post for the most used (mastodon), on witch settings change for people that do not want to allow this.

                        As they may have missed this or misunderstood what the switch did

                        And since one of the two is recommemded, it may make them allow thing they actually don't want 🙂

                        some may misunderstand both settings:

                        - Include public posts in search results
                        - Feature profile and posts in discovery algorithms

                        apps@toot.fedilab.appA This user is from outside of this forum
                        apps@toot.fedilab.appA This user is from outside of this forum
                        apps@toot.fedilab.app
                        wrote last edited by
                        #15

                        @lexinova
                        We can't change how Mastodon presents those settings, but we had a page explaining how the service works and how to opt out per platform. We'll communicate even more clearly if the service comes back.

                        Worth noting that this opt-in by default setting also allows Google to index fediverse profiles, except they don't check for consent at all. This is a much bigger issue that goes way beyond our project, which actually tries to do things the right way.

                        smallcircles@social.coopS 1 Reply Last reply
                        0
                        • ember@blobfox.coffeeE This user is from outside of this forum
                          ember@blobfox.coffeeE This user is from outside of this forum
                          ember@blobfox.coffee
                          wrote last edited by
                          #16

                          @lutindiscret@mastodon.libre-entreprise.com @apps i can see a case where someone is fine with other fedi instances indexing known posts, but not bots or external services

                          1 Reply Last reply
                          0
                          • apps@toot.fedilab.appA apps@toot.fedilab.app

                            @lutindiscret
                            We panicked quickly because things escalated beyond what we expected. But we still believe our approach was far more respectful than what's already happening: the same data is being exploited by others without users even knowing. At worst, we raised awareness.

                            iinavpov@mastodon.onlineI This user is from outside of this forum
                            iinavpov@mastodon.onlineI This user is from outside of this forum
                            iinavpov@mastodon.online
                            wrote last edited by
                            #17

                            @apps
                            You're right, and you're good people.

                            However, people posting on social media expecting their postings to be private are Just Wrong.

                            And I will add that they're the ones making mastodon toxic at times, and I'm sorry you were a victim of them.
                            @lutindiscret

                            1 Reply Last reply
                            0
                            • tati@eldritch.cafeT tati@eldritch.cafe

                              @apps the things that concerned me at the time were, in the initial announcement, 'we follow every signal' but only in this post did we know what signals you were following; and language of 'people may have misunderstood the defaults' which unfortunately rhymes with what less conscientious organisations say to justify hoovering all the things

                              if i had known the list of things you were looking at, i would have simply checked them and slept well. i didn't, so i blocked the bot, which now proves to have been unnecessary

                              andi@snac.sonnenmulde.atA This user is from outside of this forum
                              andi@snac.sonnenmulde.atA This user is from outside of this forum
                              andi@snac.sonnenmulde.at
                              wrote last edited by
                              #18
                              In the initial post, that sparked the discussion and led to the shutdown, there was a link to an article with an even deeper explanation. Maybe you didn't get it 'because federation' but it has been explained in detail from the start.

                              CC: @apps@toot.fedilab.app
                              1 Reply Last reply
                              0
                              • apps@toot.fedilab.appA apps@toot.fedilab.app

                                @lexinova
                                We can't change how Mastodon presents those settings, but we had a page explaining how the service works and how to opt out per platform. We'll communicate even more clearly if the service comes back.

                                Worth noting that this opt-in by default setting also allows Google to index fediverse profiles, except they don't check for consent at all. This is a much bigger issue that goes way beyond our project, which actually tries to do things the right way.

                                smallcircles@social.coopS This user is from outside of this forum
                                smallcircles@social.coopS This user is from outside of this forum
                                smallcircles@social.coop
                                wrote last edited by
                                #19

                                @apps @lexinova

                                The fediverse is built app-centric, where people build their own particular app functionality and use cases and then try to get a reasonable amount of interoperability with other apps (or only with themself in some cases).

                                I think the general issue is that in building a consent mechanism an app should not depend on consent mechanisms of other apps.

                                The fact that Mastodon does something with Indexable flag, and provides config settings in the UI is pure app-specific and not protocol functionality, not a native capability of the social network that can be relied upon.

                                1 Reply Last reply
                                0
                                • apps@toot.fedilab.appA apps@toot.fedilab.app

                                  With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
                                  Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
                                  We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

                                  D This user is from outside of this forum
                                  D This user is from outside of this forum
                                  dummi@norden.social
                                  wrote last edited by
                                  #20

                                  @apps
                                  To be honest, I didn't understand any of what you wrote. Is there a page explaining the issue?

                                  1 Reply Last reply
                                  0
                                  • apps@toot.fedilab.appA apps@toot.fedilab.app

                                    With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts. Every deletion, edit or block was processed instantly via #ActivityPub.
                                    Google uses that same "indexable" flag but ignores everything else, keeps deleted content cached for weeks.
                                    We shut it down after pushback. Was that the right call? Don't hesitate to share, this concerns the whole Fediverse.

                                    finchhaven@sfba.socialF This user is from outside of this forum
                                    finchhaven@sfba.socialF This user is from outside of this forum
                                    finchhaven@sfba.social
                                    wrote last edited by
                                    #21

                                    @apps

                                    So this dead horse flinched again

                                    "With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts."

                                    I'll copy-pasta with only slight edits:

                                    My entire point (all the noise notwithstanding) focused on

                                    Default opt-in versus default opt-out

                                    This is an agent --> recipient transaction

                                    Default opt-in: the recipient is opted into (and receives) the agent's action whether the recipient --> knows of <-- the action or not

                                    Default opt-out: the recipient is opted out of (and cannot receive) the action whether the recipient --> knows of <-- the action or not

                                    Neither default opt-in nor default opt-out have any logical meaning if

                                    --> THE RECIPIENT DOES NOT KNOW OF THE AGENT <--

                                    in advance

                                    There was no mechanism for prior notification *before* indexing

                                    People would have had to stumble on what you're doing, by how, exactly?

                                    How dose the recipient learn of what you've done *before* you do it?

                                    shadowwwind@fosstodon.orgS 1 Reply Last reply
                                    0
                                    • finchhaven@sfba.socialF finchhaven@sfba.social

                                      @apps

                                      So this dead horse flinched again

                                      "With #HolosDiscover we checked multiple criteria before indexing: "indexable" enabled, account not locked, no #nobot or #noindex in bio, not in opted-out list, only public posts."

                                      I'll copy-pasta with only slight edits:

                                      My entire point (all the noise notwithstanding) focused on

                                      Default opt-in versus default opt-out

                                      This is an agent --> recipient transaction

                                      Default opt-in: the recipient is opted into (and receives) the agent's action whether the recipient --> knows of <-- the action or not

                                      Default opt-out: the recipient is opted out of (and cannot receive) the action whether the recipient --> knows of <-- the action or not

                                      Neither default opt-in nor default opt-out have any logical meaning if

                                      --> THE RECIPIENT DOES NOT KNOW OF THE AGENT <--

                                      in advance

                                      There was no mechanism for prior notification *before* indexing

                                      People would have had to stumble on what you're doing, by how, exactly?

                                      How dose the recipient learn of what you've done *before* you do it?

                                      shadowwwind@fosstodon.orgS This user is from outside of this forum
                                      shadowwwind@fosstodon.orgS This user is from outside of this forum
                                      shadowwwind@fosstodon.org
                                      wrote last edited by
                                      #22

                                      @FinchHaven @apps so you would like to be asked everytime a search engine wants to index your profile? That's like the cookie popups, I will just stop reading them at some point.

                                      finchhaven@sfba.socialF 1 Reply Last reply
                                      0
                                      • shadowwwind@fosstodon.orgS shadowwwind@fosstodon.org

                                        @FinchHaven @apps so you would like to be asked everytime a search engine wants to index your profile? That's like the cookie popups, I will just stop reading them at some point.

                                        finchhaven@sfba.socialF This user is from outside of this forum
                                        finchhaven@sfba.socialF This user is from outside of this forum
                                        finchhaven@sfba.social
                                        wrote last edited by
                                        #23

                                        @shadowwwind

                                        This is the old, worn out Fediverse "Well anybody can search the Internet any time they want to so why do you care?"

                                        Which 1) most people cannot, lacking the technical expertise, and 2) most people don't want to have happen to them anyway

                                        <snip>

                                        cc @apps

                                        shadowwwind@fosstodon.orgS 1 Reply Last reply
                                        0
                                        • finchhaven@sfba.socialF finchhaven@sfba.social

                                          @shadowwwind

                                          This is the old, worn out Fediverse "Well anybody can search the Internet any time they want to so why do you care?"

                                          Which 1) most people cannot, lacking the technical expertise, and 2) most people don't want to have happen to them anyway

                                          <snip>

                                          cc @apps

                                          shadowwwind@fosstodon.orgS This user is from outside of this forum
                                          shadowwwind@fosstodon.orgS This user is from outside of this forum
                                          shadowwwind@fosstodon.org
                                          wrote last edited by
                                          #24

                                          @FinchHaven @apps huh, that's not what I meant

                                          The way I understood you, is that opt-in and opt-out don't work because the user doesn't know what service wants to do what with their profile.
                                          Which to me sounded like you want to get a notification and decide manually everytime a service wants to index your profile. Did I get that wrong?

                                          finchhaven@sfba.socialF 1 Reply Last reply
                                          0

                                          Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                          Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                          With your input, this post could be even better 💗

                                          Register Login
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          Powered by NodeBB Contributors
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups