• General_Effort@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    3
    ·
    10 months ago

    It is obvious that you do not know what either “mathematical proof” or “emergence” mean. Unfortunately, you are misrepresenting the facts.

    I don’t mean to criticize your religious (or philosophical) convictions. There is a reason people mostly try to keep faith and science separate.

    • huginn@feddit.it
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      2
      ·
      10 months ago

      Here’s a white paper explicitly proving:

      No emergent properties (illusory due to bad measures)

      Predictable linear progress with model size

      https://arxiv.org/abs/2304.15004

      The field changes fast, I understand it is hard to keep up

        • huginn@feddit.it
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          1
          ·
          10 months ago
          1. Emergence is the whole being greater than the sum of its parts. That’s the original meaning of emergent properties, which is laid out in the first paragraph of the article. It’s the scholarly usage as well, and what the claims of observed emergence are using as the base of their claim.

          2. The article very explicitly demonstrated that only about 10% of any of the measures for LLMs displayed any emergence and that illusory emergence was the result of overly rigid metrics. Swapping to edit distance as an approximately close metric causes the sharp spikes to disappear for obvious reasons: no longer having a sharp yes/no allows for linear progression to reappear. It was always there, merely masked by flawed statistics.

          If you can’t be bothered to read here’s a very easy to understand video by one of the authors: https://www.youtube.com/watch?v=ypKwNrmuuPM