Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

NodeBB

  1. Home
  2. General Discussion
  3. I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.

I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.

Scheduled Pinned Locked Moved General Discussion
fedifyjsonldfedidevactivitypub
106 Posts 24 Posters 3 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • cwebber@social.coopC cwebber@social.coop

    @evan @patmikemid @kopper @hongminhee I'm sorry hold on Evan I'm sorry but it's NOT very low risk. That's a COMPLETE misunderstanding of the information landscape we are currently in.

    Trust THEN verify?!?! Trust AND THEN verify?!?!!?!?!?!?

    "A random several minutes" until we know whether or not the content delivered authentically is from said actor...

    Even ONE minute is enough for someone to read, and believe, something false, and to reply, or to *take action*. Or to boost a post, which is then distributed across the fediverse, and then seen by a bunch of other nodes which also have not yet verified?

    Trust AND THEN verify doesn't make sense!!!

    AAAAAA I am losing my marbles over this one

    cwebber@social.coopC This user is from outside of this forum
    cwebber@social.coopC This user is from outside of this forum
    cwebber@social.coop
    wrote last edited by
    #88

    @evan @patmikemid @kopper @hongminhee Okay, sorry for blowing up in public, this is a heated issue for me, and something I strongly regret us not just shipping an answer for, and something I have been troubled by for what's now, well, a decade. But I should have taken this to DMs rather than blowing up in public. Mea culpa.

    1 Reply Last reply
    0
    • evan@cosocial.caE evan@cosocial.ca

      @aeva the thundering herd?

      @cwebber @patmikemid @kopper @hongminhee

      aeva@mastodon.gamedev.placeA This user is from outside of this forum
      aeva@mastodon.gamedev.placeA This user is from outside of this forum
      aeva@mastodon.gamedev.place
      wrote last edited by
      #89

      @evan @cwebber @patmikemid @kopper @hongminhee the json ld thing

      evan@cosocial.caE virtuous_sloth@cosocial.caV 2 Replies Last reply
      0
      • evan@cosocial.caE evan@cosocial.ca

        @cwebber it's ok if you don't get it. You don't have to use it. There are other ways to handle the thundering herd, like shared caches.

        julian@activitypub.spaceJ This user is from outside of this forum
        julian@activitypub.spaceJ This user is from outside of this forum
        julian@activitypub.space
        wrote last edited by
        #90

        @evan@cosocial.ca on the topic of shared caches as thundering herd mitigation... I spent a little bit of time trying to guard against it, and most of the load is just AP GETs.

        Nginx cache was all I needed to handle the thundering herd, although I'll admit I haven't tested it against a good evanp or tchambers boost.

        I'll write it all up as a separate topic another day.

        @cwebber@social.coop

        1 Reply Last reply
        0
        • aeva@mastodon.gamedev.placeA aeva@mastodon.gamedev.place

          @evan @cwebber @patmikemid @kopper @hongminhee the json ld thing

          evan@cosocial.caE This user is from outside of this forum
          evan@cosocial.caE This user is from outside of this forum
          evan@cosocial.ca
          wrote last edited by
          #91

          @aeva what

          @cwebber @patmikemid @kopper @hongminhee

          1 Reply Last reply
          0
          • aeva@mastodon.gamedev.placeA aeva@mastodon.gamedev.place

            @evan @cwebber @patmikemid @kopper @hongminhee the json ld thing

            virtuous_sloth@cosocial.caV This user is from outside of this forum
            virtuous_sloth@cosocial.caV This user is from outside of this forum
            virtuous_sloth@cosocial.ca
            wrote last edited by
            #92

            @aeva
            Premium pot-stirring there

            @evan @cwebber @patmikemid @kopper @hongminhee

            1 Reply Last reply
            0
            • evan@cosocial.caE evan@cosocial.ca

              @trwnh i was replying to a post that wanted all expanded terms.

              @gugurumbe @cwebber @kopper @hongminhee

              gugurumbe@mastouille.frG This user is from outside of this forum
              gugurumbe@mastouille.frG This user is from outside of this forum
              gugurumbe@mastouille.fr
              wrote last edited by
              #93

              @evan @trwnh @cwebber @kopper @hongminhee I think it would be great to have everything expanded besides the required as2 context.
              The results of the compaction algorithm would change if new things migrate into schema.org, so technically a document could become invalid or break without being modified, but this would be a lot better otherwise I guess.

              trwnh@mastodon.socialT 1 Reply Last reply
              0
              • gugurumbe@mastouille.frG gugurumbe@mastouille.fr

                @evan @trwnh @cwebber @kopper @hongminhee I think it would be great to have everything expanded besides the required as2 context.
                The results of the compaction algorithm would change if new things migrate into schema.org, so technically a document could become invalid or break without being modified, but this would be a lot better otherwise I guess.

                trwnh@mastodon.socialT This user is from outside of this forum
                trwnh@mastodon.socialT This user is from outside of this forum
                trwnh@mastodon.social
                wrote last edited by
                #94

                @gugurumbe @evan @cwebber @kopper @hongminhee yup, using full IRIs also has the advantage that ld-unaware processors only need to recognize 1 form instead of infinitely many.

                the thing is, we have semantics imported from the content type (activity+json) which can also change. which is why i think versioning the context document is also important -- it freezes the semantics at the time of publishing, like pinning your dependencies.

                without that, we might well have a simpler profile...

                gugurumbe@mastouille.frG 1 Reply Last reply
                0
                • trwnh@mastodon.socialT trwnh@mastodon.social

                  @gugurumbe @evan @cwebber @kopper @hongminhee yup, using full IRIs also has the advantage that ld-unaware processors only need to recognize 1 form instead of infinitely many.

                  the thing is, we have semantics imported from the content type (activity+json) which can also change. which is why i think versioning the context document is also important -- it freezes the semantics at the time of publishing, like pinning your dependencies.

                  without that, we might well have a simpler profile...

                  gugurumbe@mastouille.frG This user is from outside of this forum
                  gugurumbe@mastouille.frG This user is from outside of this forum
                  gugurumbe@mastouille.fr
                  wrote last edited by
                  #95

                  @trwnh as a paranoid person, I sometimes wonder what would happen if schema.org received a court order to partially censor its schema in certain regions of the world. Or inject a backdoor key. If it prevents people from sending memes across the geofence, it’s bad.

                  1 Reply Last reply
                  0
                  • cwebber@social.coopC cwebber@social.coop

                    @evan @kopper @hongminhee The problem is that signing json-ld is extremely hard, because effectively you have to turn to the RDF graph normalization algorithm, which has extremely expensive compute times. The lack of signatures means that when I boost peoples' posts, it takes down their instance, since effectively *every* distributed post on the network doesn't actually get accepted as-is, users dial-back to check its contents.

                    Which, at that point, we might as well not distribute the contents at all when we post to inboxes! We could just publish with the object of the activity being the object's id uri

                    rigo@mamot.frR This user is from outside of this forum
                    rigo@mamot.frR This user is from outside of this forum
                    rigo@mamot.fr
                    wrote last edited by
                    #96

                    @cwebber @evan @kopper @hongminhee For the purpose of "message" wouldn't it then be easier to just sign an entire file that is transmitted? Or is this stream processing.

                    1 Reply Last reply
                    0
                    • cwebber@social.coopC cwebber@social.coop

                      @kopper @hongminhee As the person probably most responsible for making sure json-ld stayed in the spec (two reasons: because it was the only extensibility answer we had, and because we were trying hard to retain interoperability with the linked data people, which ultimately did not matter), I agree with you. I do ultimately regret not having a simpler solution than json-ld, especially because it greatly hurt our ability to sign messages, which has considerable effect on the ecosystem.

                      Mea culpa 😕

                      I do think it's fixable. I'd be interested in joining a conversation about how to fix it.

                      rigo@mamot.frR This user is from outside of this forum
                      rigo@mamot.frR This user is from outside of this forum
                      rigo@mamot.fr
                      wrote last edited by
                      #97

                      @cwebber @kopper @hongminhee If we consider data exchange just in one application context (mastodon), then JSON-LD is overhead because data structures are fixed.

                      But as soon as we go out of that application context, JSON-LD will make a lot of sense. IMHO, the use will grow over time as this allows to add permissions to post on the data level using ODRL e.g. Or to have privacy considerations.

                      On a very short term, JSON-LD may be an overhead. But it is an investment into the future.

                      kopper@not-brain.d.on-t.workK 1 Reply Last reply
                      0
                      • rigo@mamot.frR rigo@mamot.fr

                        @cwebber @kopper @hongminhee If we consider data exchange just in one application context (mastodon), then JSON-LD is overhead because data structures are fixed.

                        But as soon as we go out of that application context, JSON-LD will make a lot of sense. IMHO, the use will grow over time as this allows to add permissions to post on the data level using ODRL e.g. Or to have privacy considerations.

                        On a very short term, JSON-LD may be an overhead. But it is an investment into the future.

                        kopper@not-brain.d.on-t.workK This user is from outside of this forum
                        kopper@not-brain.d.on-t.workK This user is from outside of this forum
                        kopper@not-brain.d.on-t.work
                        wrote last edited by
                        #98
                        @rigo @hongminhee @cwebber
                        But as soon as we go out of that application context, JSON-LD will make a lot of sense.
                        i can't see it. you can indeed add new fields to json and even namespace them (just expand the namespaces! it even compresses better! (open in your own instance, my instance doesn't show reply trees logged out yet)), without requiring json-ld processing. almost all json parsers will either drop unknown fields by default or have an option to turn that functionality on. json-ld, seriously, brings nothing to the table
                        the use will grow over time as this allows to add permissions to post on the data level using ODRL
                        i seriously doubt this will happen. for permissions the community (not w3c! the implementations, because implementations are who give standards their value!) is more or less converging on docs.gotosocial.org/en/latest/federation/interaction_controls/ (including mastodon's quote post approvals, which is built upon the same framework)
                        But it is an investment into the future.
                        i don't know about that. i can't really find anyone other than people who have drunk the w3c kool-aid (including you from your bio) who thinks json-ld as allowing any new form of extensibility that Just Namespaces can't accomplish.

                        no wonder why atproto decided to work with ietf on their standardization efforts. the strictness and validation capability of their lexicons actually make developers lives easy, letting them auto-generate the boring bits, and even with bluesky's own, infamously (?) strict lexicon that doesn't let any microblog >300 characters from being made,
                        there's nothing stopping extensions from being made
                        rigo@mamot.frR 2 Replies Last reply
                        0
                        • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                          @rigo @hongminhee @cwebber
                          But as soon as we go out of that application context, JSON-LD will make a lot of sense.
                          i can't see it. you can indeed add new fields to json and even namespace them (just expand the namespaces! it even compresses better! (open in your own instance, my instance doesn't show reply trees logged out yet)), without requiring json-ld processing. almost all json parsers will either drop unknown fields by default or have an option to turn that functionality on. json-ld, seriously, brings nothing to the table
                          the use will grow over time as this allows to add permissions to post on the data level using ODRL
                          i seriously doubt this will happen. for permissions the community (not w3c! the implementations, because implementations are who give standards their value!) is more or less converging on docs.gotosocial.org/en/latest/federation/interaction_controls/ (including mastodon's quote post approvals, which is built upon the same framework)
                          But it is an investment into the future.
                          i don't know about that. i can't really find anyone other than people who have drunk the w3c kool-aid (including you from your bio) who thinks json-ld as allowing any new form of extensibility that Just Namespaces can't accomplish.

                          no wonder why atproto decided to work with ietf on their standardization efforts. the strictness and validation capability of their lexicons actually make developers lives easy, letting them auto-generate the boring bits, and even with bluesky's own, infamously (?) strict lexicon that doesn't let any microblog >300 characters from being made,
                          there's nothing stopping extensions from being made
                          rigo@mamot.frR This user is from outside of this forum
                          rigo@mamot.frR This user is from outside of this forum
                          rigo@mamot.fr
                          wrote last edited by
                          #99

                          @kopper @hongminhee @cwebber but the JSON parser will not recognize naming conflicts. We are talking about data flows beyond the application context. I see you arguing in a social-media distributed twitter context and not a millimeter beyond. That's ok

                          Inside that context, JSON-LD remains valid JSON and your implementation can ignore the context. But as soon as you want to annotate stuff and NOT hardcode it into your app, this context becomes important.

                          kopper@not-brain.d.on-t.workK 1 Reply Last reply
                          0
                          • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                            @rigo @hongminhee @cwebber
                            But as soon as we go out of that application context, JSON-LD will make a lot of sense.
                            i can't see it. you can indeed add new fields to json and even namespace them (just expand the namespaces! it even compresses better! (open in your own instance, my instance doesn't show reply trees logged out yet)), without requiring json-ld processing. almost all json parsers will either drop unknown fields by default or have an option to turn that functionality on. json-ld, seriously, brings nothing to the table
                            the use will grow over time as this allows to add permissions to post on the data level using ODRL
                            i seriously doubt this will happen. for permissions the community (not w3c! the implementations, because implementations are who give standards their value!) is more or less converging on docs.gotosocial.org/en/latest/federation/interaction_controls/ (including mastodon's quote post approvals, which is built upon the same framework)
                            But it is an investment into the future.
                            i don't know about that. i can't really find anyone other than people who have drunk the w3c kool-aid (including you from your bio) who thinks json-ld as allowing any new form of extensibility that Just Namespaces can't accomplish.

                            no wonder why atproto decided to work with ietf on their standardization efforts. the strictness and validation capability of their lexicons actually make developers lives easy, letting them auto-generate the boring bits, and even with bluesky's own, infamously (?) strict lexicon that doesn't let any microblog >300 characters from being made,
                            there's nothing stopping extensions from being made
                            rigo@mamot.frR This user is from outside of this forum
                            rigo@mamot.frR This user is from outside of this forum
                            rigo@mamot.fr
                            wrote last edited by
                            #100

                            @kopper @hongminhee @cwebber and to know other people being really happy with JSON-LD and really needing this functionality, you need to get out of your own application context and into data flows. E.g. Trade transparency. And they may also use activity pub at some point:
                            https://uncefact.unece.org/spaces/uncefactpublic/pages/246317069/Verifiable+Credentials+for+Trade

                            Talking of kool-aid, note that I'm only the lawyer. 😁

                            kopper@not-brain.d.on-t.workK 1 Reply Last reply
                            0
                            • rigo@mamot.frR rigo@mamot.fr

                              @kopper @hongminhee @cwebber and to know other people being really happy with JSON-LD and really needing this functionality, you need to get out of your own application context and into data flows. E.g. Trade transparency. And they may also use activity pub at some point:
                              https://uncefact.unece.org/spaces/uncefactpublic/pages/246317069/Verifiable+Credentials+for+Trade

                              Talking of kool-aid, note that I'm only the lawyer. 😁

                              kopper@not-brain.d.on-t.workK This user is from outside of this forum
                              kopper@not-brain.d.on-t.workK This user is from outside of this forum
                              kopper@not-brain.d.on-t.work
                              wrote last edited by
                              #101
                              @rigo @hongminhee @cwebber i'm sorry but this conversation is drifting towards a very unrealistic "why aren't game items NFTs so you can move your items between games" view of the world that ignores a significant chunk of real world forces, including user experience (do people even want what you're after?), in order to promote a specific technology

                              i can't see how this discussion can produce anything useful when that's the lens you're looking at things from.
                              1 Reply Last reply
                              0
                              • rigo@mamot.frR rigo@mamot.fr

                                @kopper @hongminhee @cwebber but the JSON parser will not recognize naming conflicts. We are talking about data flows beyond the application context. I see you arguing in a social-media distributed twitter context and not a millimeter beyond. That's ok

                                Inside that context, JSON-LD remains valid JSON and your implementation can ignore the context. But as soon as you want to annotate stuff and NOT hardcode it into your app, this context becomes important.

                                kopper@not-brain.d.on-t.workK This user is from outside of this forum
                                kopper@not-brain.d.on-t.workK This user is from outside of this forum
                                kopper@not-brain.d.on-t.work
                                wrote last edited by
                                #102
                                @rigo @hongminhee @cwebber
                                but the JSON parser will not recognize naming conflicts
                                namespaces! explicit namespaces! the property name https://w.on-t.work/ns#content does not conflict in any way with https://www.w3.org/ns/activitystreams#content! please stop talking in circles, we're all tired i think
                                rigo@mamot.frR 1 Reply Last reply
                                0
                                • kopper@not-brain.d.on-t.workK kopper@not-brain.d.on-t.work
                                  @rigo @hongminhee @cwebber
                                  but the JSON parser will not recognize naming conflicts
                                  namespaces! explicit namespaces! the property name https://w.on-t.work/ns#content does not conflict in any way with https://www.w3.org/ns/activitystreams#content! please stop talking in circles, we're all tired i think
                                  rigo@mamot.frR This user is from outside of this forum
                                  rigo@mamot.frR This user is from outside of this forum
                                  rigo@mamot.fr
                                  wrote last edited by
                                  #103

                                  @kopper @hongminhee @cwebber Oh, I see what you mean. We had that (very heated) discussion in 2004 between the RDF folks and the XML folks. You're using JSON like the XML argumentation back then. And yes, it is possible to re-invent RDF functionality using JSON or XML. This was the point of the 2004 dispute. JSON-LD is only ONE way to solve those possible conflicts. It just happens to be an interoperability specification we call "recommendation".

                                  Sorry for my misunderstanding on namespaces.

                                  1 Reply Last reply
                                  0
                                  • evan@cosocial.caE evan@cosocial.ca

                                    @julia @cwebber @hongminhee @kopper

                                    Hi! 👋🏼 Nice to meet you. I'm well aware of `sharedInbox` and helped design it.

                                    Realtime is an illusion. You can make it pretty convincing.

                                    Your users are mostly not online. Remote users are mostly not online. Tracking the last time remote and local users were seen can help you prioritize local and remote delivery.

                                    It's a lot better to deliver to the tiny percent of users currently online first rather than delivering to the user named `aaaaaaaaamng` first.

                                    panos@ibe.socialP This user is from outside of this forum
                                    panos@ibe.socialP This user is from outside of this forum
                                    panos@ibe.social
                                    wrote last edited by
                                    #104

                                    @evan@cosocial.ca Can I say something, as someone who is not technical but cares deeply about the fediverse and its future, and who read the whole conversation with great interest?

                                    You might be familiar with the term "desire path". It's when a place, like a park, has pre-designed pathways, but people end up using their own, because they are more convenient. This is not user error - it's a design failure.

                                    I've been hearing complaints about JSON-LD since I was in Firefish. When so many people -including even people who co-wrote ActivityPub!- tell you that it's problematic, that it has ended up not being used, that most projects find their own shortcuts (or desire paths) and we need to find a better way, telling them "you should just use it as it was designed" comes off as arrogant and short-sighted. And these are not people who work for BlueSky or people who want to see fedi fail - they are people who care so much about ActivityPub that they have studied it and built whole applications around it. It's not nice to dismiss them like that.

                                    You have co-written and are largely responsible for something that was meant to be used by people and developers. Clearly, its core is great, or it wouldn't have been relatively successful. But if parts of it were too complicated and have not been adopted, then it's a design problem with these parts. A good protocol should be easy to work with and build on.

                                    I don't think you should be scared of breaking backwards compatibility. This is sometimes an essential part of progress - and if something doesn't progress, it usually dies. For how long will people keep building on a protocol that stays stagnant to not break backwards compatibility? Ten years? Twenty? Yes, you don't want to make big changes very often. But
                                    never making big changes also doesn't make much sense. Especially regarding parts that are not used as intended in practice, therefore have proven to have failed. Accept it and find collective ways to improve ActivityPub, or blame everyone else for not following to the letter a spec that obviously doesn't fit well with their needs.
                                    @julia@eepy.moe @cwebber@social.coop @hongminhee@hollo.social @kopper@not-brain.d.on-t.work

                                    1 Reply Last reply
                                    0
                                    • evan@cosocial.caE evan@cosocial.ca

                                      @kopper @gugurumbe

                                      Anyway, to me, a backwards-incompatible change is absolutely the worst possible choice we could make for the Fediverse. It splits the network, possibly permanently. We have about 100 implementations of ActivityPub, and they can't all upgrade at the same time.

                                      smallcircles@social.coopS This user is from outside of this forum
                                      smallcircles@social.coopS This user is from outside of this forum
                                      smallcircles@social.coop
                                      wrote last edited by
                                      #105

                                      @evan @kopper @gugurumbe

                                      I do note that you argue for backwards "Fediverse-compatibility" here, and I wonder how that equates with backwards #ActivityPub compatibiity.

                                      1 Reply Last reply
                                      1
                                      0
                                      • tag-activitypub@relay.fedi.buzzT tag-activitypub@relay.fedi.buzz shared this topic
                                      • sl007@digitalcourage.socialS sl007@digitalcourage.social

                                        @kopper

                                        ah, no - that is a misunderstanding!

                                        Anyone can feel free to represent the texts only and the user at least "knows" it.
                                        But the thing for Public Broadcasters means 47mio. users in DE alone and given the unified codebase for the 5 projects _these_ softwares will interpret it.
                                        It does JSON-LD you could just check by asking for any JSON-LD e.g. Q1055 (Hamburg) - it is content-negotiation.
                                        The taxiteam software is funded by the German yellow cabs - the official ones (!) the codename is FCKUBR 😉 and I have no doubt about adoption fortunately.

                                        Maybe we can work out better examples …

                                        @hongminhee @julian

                                        sl007@digitalcourage.socialS This user is from outside of this forum
                                        sl007@digitalcourage.socialS This user is from outside of this forum
                                        sl007@digitalcourage.social
                                        wrote last edited by
                                        #106

                                        @kopper @hongminhee @julian

                                        addendum;
                                        updated the hierarchical UNESCO World Heritage Collection with their 1247 Entry-Collections having 8000 Places.

                                        Apart from `Content-Negotiation`
                                        Noticed that you can also get the jsonld in the browser just by adding ".json" e.g.

                                        https://www.wikidata.org/wiki/Special:EntityData/Q122842228.json

                                        1 Reply Last reply
                                        0

                                        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                        With your input, this post could be even better 💗

                                        Register Login
                                        Reply
                                        • Reply as topic
                                        Log in to reply
                                        • Oldest to Newest
                                        • Newest to Oldest
                                        • Most Votes


                                        • Login

                                        • Don't have an account? Register

                                        • Login or register to search.
                                        Powered by NodeBB Contributors
                                        • First post
                                          Last post
                                        0
                                        • Categories
                                        • Recent
                                        • Tags
                                        • Popular
                                        • World
                                        • Users
                                        • Groups