• otp@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      0
      ·
      17 days ago

      This sounds like a perfect example of news. Companies fucking up is news. Why wouldn’t it be?

    • Rothe@piefed.social
      link
      fedilink
      English
      arrow-up
      0
      ·
      17 days ago

      Why would this not be news? Is it perhaps because it conveys a message which you are opposed to?

      • homes@piefed.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        17 days ago

        It’s not news because it is a regular occurrence.

        And of course I’m against that. Everyone should be. Aren’t you?

    • pinball_wizard@lemmy.zip
      link
      fedilink
      English
      arrow-up
      0
      ·
      17 days ago

      That’s the beauty of it - the $500 million they spent on AI already made up the difference. Their products and processes have ascended to perfection. As soon as the damn customers get their acts together and start buying in, everything will be fine. ( /Sarcasm and/or accurate portrayal of a CEO presentation to their shareholders happening right now.)

  • wonderingwanderer@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    0
    ·
    16 days ago

    > Be a corporate executive

    > Tell your employees to use more AI in their worlflows

    > Punish employees who don’t use enough AI, while rewarding those who use it the most, irrespective of actual outcomes

    > Be shocked when your company blows through an absurd amount of tokens in one month

        • sureshot0@discuss.online
          link
          fedilink
          English
          arrow-up
          0
          ·
          16 days ago

          I’m really jealous of these types of guys’ ability to lie without feeling anything. If I lied like that, I’d be embarrassed because my words sounded like bullshit. How do they do it?

            • sureshot0@discuss.online
              link
              fedilink
              English
              arrow-up
              0
              ·
              16 days ago

              See, I don’t know! I used to think that too! But then I actually met some people like this, and a lot of them absolutely do not believe the shit they say! They’re just really good at convincing other people that they do.

                • sureshot0@discuss.online
                  link
                  fedilink
                  English
                  arrow-up
                  0
                  ·
                  16 days ago

                  I once knew a local “activist” who frequently weaponized leftist language in order to police or punish other people. Ie, accusing people of racism for ordering an Asian Chicken Salad at a fast food place, or for trying to learn Spanish at the community college, and she would phrase her criticisms in such a way that any complaint would make the other person seem wrong. I legitimately thought she was just mentally ill.

                  I found out later that she was trying to blackmail and defraud a local charity, and that’s when our association ended. She has been banned form some community organizing spaces because she inserts herself into every single one of them.

      • Spice Hoarder@lemmy.zip
        link
        fedilink
        English
        arrow-up
        0
        ·
        16 days ago

        The paradox of promotions based on performance in a previous roll is that you end up with incompetent managers unable to move upwards anymore.

  • Damarus@feddit.org
    link
    fedilink
    English
    arrow-up
    0
    ·
    17 days ago

    Sure it’s hella expensive if you put zero thought into what you’re actually doing.

  • x00z@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    16 days ago

    I think a lot of SaaS companies love it when people accidentally overuse their services.

  • UnderpantsWeevil@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    17 days ago

    When you owe Claude half a million, you’ve got a problem.

    When you owe Claude half a billion, Anthropic has a problem

  • teft@piefed.social
    link
    fedilink
    English
    arrow-up
    0
    ·
    16 days ago

    Most companies can’t eat a half billion dollar loss so who ends up paying this? AI queries burn actual energy so the AI company would have to charge I would think.

  • fistac0rpse@fedia.io
    link
    fedilink
    arrow-up
    0
    ·
    17 days ago

    Microsoft recently said that they were revoking Claude Code access for employees in favour of GitHub Copilot CLI, wonder if it was them lol

  • BlackLaZoR@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    16 days ago

    Just to make things clear: API access to most models is charged per input tokens + output tokens. It means that the longer your conversation is, the more you pay for every new answer. Single prompt with no context and 100 tokens of answer is cheap. Single prompt with 100k tokens of context and 100 tokens of answer is NOT cheap.

    Extremely long conversations with most expensive top of the line models can absolutely demolish your budget.

    • perviouslyiner@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      16 days ago

      does it give the full history to the LLM each time?

      Last time I tried implementing something like this, it suggested to have a rolling window of history so that it takes into account your last X messages but not the entire conversation.

      (I guess this is what ollama calls “context length”?)

      • Sabata@ani.social
        link
        fedilink
        English
        arrow-up
        0
        ·
        16 days ago

        You send the entire history for that conversation every time and likely more if its getting info from tools. If its not in the context the model dose not see it unless you have a memory system that dose something like feeding in summaries of past conversations that also takes up tokens and context. Rolling drops old messages to not reach context limits but you can lose important info or get odd results. If the history gets bigger than the context things break or slow way down.

        • perviouslyiner@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          16 days ago

          presumably this is why Claude periodically writes its conclusions so far into a text file that it can read later instead of having to remember everything. Sounds like an interesting approach.

      • BlackLaZoR@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        16 days ago

        does it give the full history to the LLM each time?

        It’s limited to the context size supported by given model. You can give the model 100k tokens of history but if it’s configured for less, it will just truncate it before processing (usually by removing oldest tokens first)

        • boonhet@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          0
          ·
          15 days ago

          Good news for tokenmaxxers: frontier models now have 1M context

          Easy way to blow through ridiculous amounts of tokens.