• 0 Posts
  • 18 Comments
Joined 2 years ago
cake
Cake day: June 12th, 2023

help-circle

  • Well, I work at an AI hyperscaler. I can tell you how much my facility uses, and how much each rack uses, but don’t have any way to determine what the customer is doing on that server. Or even which servers a given customer is using. Is it being used heavily for queries? How many? Of what kind? We don’t know. Only what the rack/row/pod/hall is consuming.

    Also, does the network gear overhead count? How do you apportion that?

    We have no visibility into the customer workload. Some of our customers use our systems for scientific research. Drugs, etc. How do you tally that?

    I’m not saying that it is impossible, just that if the customer won’t pay for that report, we’re not going to spend money to build the systems to produce it.

    Do I agree? No. But I’m just a grunt.







  • A major bottleneck is power capacity. Is is very difficult to find 50Mwatts+ (sometime hundreds) of capacity available at any site. It has to be built out. That involves a lot of red tape, government contracts, large transformers, contractors, etc. the current backlog on new transformers at that scale is years. Even Google and Microsoft can’t build, so they come to my company for infrastructure - as we already have 400MW in use and triple that already on contract. Further, Nvidia only makes so many chips a month. You can’t install them faster than they make them.