If I visit a community from Feddit and then another instance, the comments often differ. I understand there are some issues with defederation between some instances, but this doesn’t seem related to that.

For example, here’s the post-race discussion on the Formula 1 community on Feddit: https://feddit.uk/post/320484

And then the same discussion on lemmy.world: https://lemmy.world/post/950479

I don’t understand why a significant number of comments aren’t visible on Feddit.

  • db0@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    7
    ·
    1 year ago

    Sometimes instances are overloaded which cause federation events to timeout and be dropped. Since federation is a push event when this happens, it cannot be recovered

    • ToastedYoyo@feddit.uk
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      Is that not a pretty big weakness to Lemmy? I know it’s never going to be perfect, but if Lemmy is to appeal to more people I feel issues like this add friction and makes Lemmy appear to be a little quiet/confusing as a result.

      For instance, I have two Lemmy profiles - one on Feddit.uk and one on FMHY - both have different comments on the same post in !gaming@beehaw.org

      • db0@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        1
        ·
        1 year ago

        I say, stop worrying and enjoy the chaos! So you miss a few comments here and there, so what? There’s plenty more around and as the traffic increases more and more, we can’t keep up with all this content anyway.

    • burningmatches@feddit.ukOP
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      I see. I’ll assume this is a problem related to the surge in users and will eventually resolve itself.

      • body_by_make@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        2
        ·
        edit-2
        1 year ago

        Lemmy.world has been struggling hard core the last couple days, sometimes not even loading. I think this is an issue with a free service that requires actual intense server hardware when you scale like this. Hopefully they can find a sustainable way to deal with their increased usage.

    • blackn1ght@feddit.uk
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Seems like the events should get pushed onto a queue so that if the instance is overloaded or down, the events aren’t lost.

      • db0@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        The problem is in the way ActivityPub works. The events are pushed into the queue, but if the receiving system takes too long to process it, there’s a datetime “expiry”

        • blackn1ght@feddit.uk
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          Ah, I’m totally unfamiliar with ActivityPub (never even heard of it until now!). Is the expiry time of the messages totally uncontrollable by Lemmy then?

            • spacedogroy@feddit.uk
              link
              fedilink
              English
              arrow-up
              1
              ·
              1 year ago

              I had a look at the ActivityPub W3C and I noticed the following paragraph:

              For federated servers performing delivery to a third party server, delivery SHOULD be performed asynchronously, and SHOULD additionally retry delivery to recipients if it fails due to network error.

              In this context is 3rd party server a federated server? I couldn’t see where expiry came in but presumably if the pushes are async then the server sending the payload won’t know if it succeeded or not (unless there’s some other mechanism to verify message acknowledgment).

              • db0@lemmy.dbzer0.com
                link
                fedilink
                English
                arrow-up
                3
                ·
                edit-2
                1 year ago

                The async doesn’t mean if the sending server doesn’t know. It’s not using UDP lol :) It will retry until it gets a RC 200 or 201. Then stop sending. The asynchronous part is that the receiving instance doesn’t have to import the event into the DB immediately. The received event sits in a special “to import” queue until processed. Each event has a timestamp attached to it. If it takes too long to process the event in the queue, because the receiving server is overloaded, then its timestamp may be considered expired.

                E.g. from https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/

                Mind that there is only a ±30 seconds time window when that signature would be considered valid, which is a big reason why it’s quite difficult to do manually.

  • burningmatches@feddit.ukOP
    link
    fedilink
    English
    arrow-up
    5
    arrow-down
    1
    ·
    1 year ago

    I should add that I thought being on a smaller instance like Feddit would avoid some of the defederation problems compared to being on lemmy.world or beehaw, so it’s weird to me that there are fewer comments on Feddit than on lemmy.world.

    Looking closer at the comments on the F1 post, it seems that the missing comments are from lemmy.world users. Has Feddit defedrated from lemmy.world? Did I miss that?

  • ᴇᴍᴘᴇʀᴏʀ 帝@feddit.uk
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    With Reddit shutting down the APIs this weekend and further unexpected tomfoolery from Elon Musk it has led to rapid growth of the Fediverse, as can be seen here. The bulk of the growth on Lemmy will be at lemmy.world, which was already twice the size of the other ten largest combined, so exerts a large amount of gravitational pull (which may or may not be healthy).

    As the folks behind lemmy.world are, I believe, also running mastodon.world and calckey.world, I imagine they’ve been having a fun weekend. With the systems overloaded some things are going to get a bit messy, I’m afraid.

    I personally still find it remarkable that all these even functions at all, but it does.