• grayclouds@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    6 hours ago

    50TB isn’t even that expensive on a HDD! Couple hundred to a thousand bucks! How much were they paying for that much storage?

  • BarneyPiccolo@lemmy.today
    link
    fedilink
    English
    arrow-up
    0
    ·
    12 hours ago

    The first time I heard about cloud storage, I wondered why anyone would store their stuff with a third party, for reasons EXACTLY like this. This is the exact scenario I thought of.

    And while we’re talking about this specific case, presumably all the other customers of that cloud storage company list all their stuff, too.

    • rmrf@lemmy.ml
      link
      fedilink
      English
      arrow-up
      0
      ·
      10 hours ago

      Using cloud is not the problem here and never has been an issue from a technical design standpoint, it’s having the data only with one cloud provider that allows this to happen.

      The only customers of that cloud provider that lost their stuff are the customers that would have lost their stuff using any other method they’d implement, because the only way you can lose your data this way is through poor design.

  • remon@ani.social
    link
    fedilink
    English
    arrow-up
    0
    ·
    14 hours ago

    50TB seems very little for 70 years worth of data … I have more than that it it didn’t even take me 4.

    • ZILtoid1991@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      13 hours ago

      “But optical discs are an old technology, everyone uses SSDs! Tape drives are even older technology! We need to SSD everything! SSD good! Oh no, why is that external SSD I haven’t powered up for five years and kept important file on it for fast access suddenly died? It was supposed to be the optical CDs that flake off and oxidize in a year or two!”

  • Aceticon@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    0
    ·
    15 hours ago

    Risk analysis in IT Management nowadays is spectacularly bad.

    If you become dependent on a 3rd supplier then their survival as a company is now a risk for you. It doesn’t matter how good the reliability you have contracted with them - if they go under whatever penalities in the contract for failing to deliver said reliability will never be paid.

    You might even be at risk of them simply chosing to close that division - small company signing a service supply contract with a big company: good luck getting a contract where the parent company (rather than whatever paper company they spun up for that business area) actually has to pay you stiff penalties if they fuck you over.

    As soon as mission critical anything is outside your company, a lot of otherwise irrelevant things become operational risks for your company.

    This is why you don’t put your mission critical anything on an external Cloud supplier.

    You know, stuff like the only copy of 70 years of TV history that’s actually just 50TB and would’ve fit a bunch of Hard Disks or Backup tapes.

    • northernlights@lemmy.today
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 hours ago

      Yeah I did risk analysis for 9 years and the job gradually devolved in useless box check-in with people checking nothing, until all of us were laid off and replaced by an AI and a few people in India checking the output. It’s all for show.

    • Test_Tickles@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      14 hours ago

      This is something I kept pointing out to management at my last company, but they were always so hellbent on putting everything in the cloud. Then construction across the street cut some fiber or something going to our building. When I told management that we had no Internet and I was sending everyone home, they lost their shit. I mollified them a bit by pointing out that at least some people would be able to work from home. I then waited until the meeting the next morning to point out that only 1 QA/support person and 1 dev had laptops (because of course management didn’t see any reason to spend the extra money for laptops over desktops). It was also nice to be able to make the CEO and VP to drive all the way home and back to get their wireless data pucks they used as their personal Internet connections.
      My overwhelming joy at all the petty little ways I was able to rub their bad decisions in their faces was a big part of what made me actually start hunting for a new job rather than just thinking about it.

      • Aceticon@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        0
        ·
        13 hours ago

        Yeah, even having learned to explain this shit in the IT Consultancy language of “supply contracts” and “mission critical”, it’s still not guaranteed that I’ll convince a manager to avoid or at least account for the higher exposure to outside risks when subcontracting certain things.

        Nowadays, with 3 decades of experience in the Industry, I honestly think that the majority of managers, no matter how high in a company, are either incompetent as fuck or simply manage with as priority maximizing their next bonus in the full knowledge that they’re leaving traps for the next one to solve and if they do blow up during their own time, they’ll still personally end up better off by going for bonus-today-problem-tomorrow than refraining from doing so.

        • BarneyPiccolo@lemmy.today
          link
          fedilink
          English
          arrow-up
          0
          ·
          12 hours ago

          The Peter Principle states that a manager will get promoted to his level of incompetency, then get stuck there. As time goes on, more and more managers reach their level of incompetency, until the entire management level is populated by people who are bad at their jobs.

          Say a guy is a great salesman, the best in the company, beats all his targets. He’s just great at it, and all his customers love him. So he gets promoted to sales manager, with a nice office, and a big raise.

          But just because he’s a talented salesman, doesn’t mean he knows how to manage sales people. Also, the guy who replaced him isn’t pulling in near the revenue, so the department he’s managing is now WORSE than before he took over. He becomes just another mediocre manager, in over his head, and he never gets another promotion again.

          Or maybe he moves up a few more levels, but eventually he hits the point that he can’t do well, and that’s where he remains, being mediocre. After a while, ALL the managers are people who were good at their old jobs, but mediocre at this one, and here they stay, until they are swept out in their 50s.

  • Cursed_Fig@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 day ago

    read the article, you losers

    Logo ✕ Programs/Content People Development System/Policy Tech Jobs Sponsored Posts Webinars Job Description Depot Copyright 2025. Current. All rights reserved. Logo

    Rescission Layoffs AI and Pubmedia Rising Stars History of Public Broadcasting Timeline Nine PBS sues Iron Mountain over blocked access to archival data By Francisco RodriguezAugust 11, 2026

    By Billy Hathorn - Own work, CC BY-SA 3.0

    Nine PBS in St. Louis filed a lawsuit against information management corporation Iron Mountain Data Centers July 28, seeking to recover over 50 terabytes of archival materials stored in one of the company’s Denver-based data centers.

    The lawsuit filed in Denver District Court alleges that the station’s cloud-storage vendor, Open Source Storage, abruptly cut off access to Nine PBS’ data earlier this year without warning. It states OSS, which had a separate relationship with Iron Mountain to provide data storage, went “defunct,” leaving Nine PBS’ archives in a data center operated by Iron Mountain.

    Iron Mountain has refused to return the materials to the station because its client, OSS, technically owned “the physical services housing the data” within Iron Mountain, according to the complaint.

    The station requested temporary and preliminary relief that would prevent Iron Mountain from deleting, modifying or overwriting its materials in the suit. A district judge granted the motion and set a hearing for Wednesday.

    In a statement to Current, Nine PBS VP and CCO Leah Freeman confirmed the station’s lawsuit against Iron Mountain and its dedication to retrieving the archival materials and programming, which she says span over “70 years of our organization’s history.”

    “We are committed to ensuring we can recover and restore full access to this valuable content, which Nine PBS rightfully owns, as it holds significant historical importance for St. Louis.”

    According to the lawsuit, the blocked materials include historical items such as Nine PBS’ coverage on the history of East St. Louis, the COVID-19 pandemic and the Great Flood of 1993, which ravaged along the Mississippi and Missouri rivers.

    ‘No choice but to file’ Nine PBS entered a relationship with a company described in the complaint as “OSS’ predecessor” in 2019. This unidentified vendor provided “hardware, software and cloud-storage services” for storing the public broadcaster’s archival materials and other data.

    Nine PBS renewed its contracts with the data services vendor and subsequently OSS annually, the complaint states.

    When Nine PBS attempted to schedule a meeting with OSS in February to discuss renewing for 2026, OSS didn’t respond or indicate “any intention not to renew the agreement.” The agreement was set to expire on March 6.

    The contract provided 30 days for Nine PBS to retrieve its data from OSS’ storage “upon termination of services.” But on March 6, OSS cut off the station’s access without warning, according to the complaint. When Nine PBS attempted to contact OSS to sort out the problem, it discovered that OSS’ website was defunct and the company had delinquency status with the Colorado Secretary of State.

    To ensure the station’s data remained secure, Nine PBS investigated further and discovered that OSS had a relationship with Iron Mountain, according to the complaint.

    Nine PBS sent a demand letter March 13, demanding that Iron Mountain preserve and return its data and offering “to pay any reasonable costs associated with its demand.” Iron Mountain neither confirmed nor denied the data was in its possession, the lawsuit states.

    Nine PBS filed a lawsuit against OSS and its “purported” president Charles Wells in the St. Louis Circuit Court April 16, the complaint states. The broadcaster later paused the litigation after James Tramel, a “managing partner of the group that officially acquired” OSS’ assets, confirmed that Nine PBS’ data was secure within Iron Mountain’s Denver data center.

    After communicating with Nine PBS for about a month, Tramel stopped responding. Weeks later, an automatic reply email from his account stated that he was no longer affiliated with OSS. Tramel revealed in a subsequent phone call that he “had been defrauded” into purchasing OSS, according to the complaint. At this point, the company’s previous owners, including Wells, Ben Nicholson, and Justine Ririe, resumed control of OSS’ operations.

    After Nine PBS attempted to contact OSS leadership without success, the station returned to the St. Louis Circuit Court and obtained a default judgment against OSS. The court’s ruling stated that the station both owned its data and had an “immediate right to possess the data,” according to the complaint. The judgment also ordered OSS to return the data to Nine PBS “and/or facilitate its transfer to a new vendor.”

    An attorney representing Nine PBS contacted Iron Mountain about returning the data, and noted that “litigation was imminent in Colorado,” and the company admitted that it possessed the data, the complaint states. Iron Mountain initially indicated that it wanted to avoid litigation and comply with Nine PBS’ request, but later refused to do so, citing OSS’ ownership of the infrastructure that houses the data.

    Nine PBS’ complaint states that, following multiple unsuccessful attempts to retrieve its digital property, it “had no choice but to file this lawsuit to force Iron Mountain to protect and ultimately provide” the data.

    Iron Mountain did not respond to a request for comment.

  • HugeNerd@lemmy.ca
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 day ago

    Jesus Christ, what happened to P2P? Just have the files in constant circulation through the world’s computers. And 50TB, while a lot, isn’t beyond the reach of a well-heeled private individual to host on a NAS.

    • elucubra@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      0
      ·
      18 hours ago

      50TB is not a lot, it’s actually a capacity that some homelabers and data hoarders have or surpass.

      • grrgyle@slrpnk.net
        link
        fedilink
        English
        arrow-up
        0
        ·
        12 hours ago

        It’s a lot if you were trying to source the hardware with today’s process, but 5 years ago that would be nothing to an organisation.

        • RamRabbit@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          10 hours ago

          It’s a 4-digit number, even today, to build a 50tb local backup box. Absolutely nothing to a company. They were simply lazy and didn’t do the bare minimum.

  • bad1080@piefed.social
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 day ago

    it sucks but they didn’t follow the basic 3-2-1 rule of back-ups (3 copies, 2 types of media, 1 off-site)

    • JingoBingo@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 day ago

      You can blame the storage provider but ultimately it’s their fault. Anyone who takes archiving serious knows that you need at least 3 backups of your archive in different locations for cases like this.

    • GutterRat42@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 day ago

      They are still around. This specific incident happened to a local PBS broadcaster, and it is only the local programming that was lost. Still sad, though

  • Ghostalmedia@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 day ago

    This looks like this is isolated to KETC’s locally produced shows for Missouri. They appear to be a smaller affiliate and they don’t make a ton of stuff that contributes to national and international broadcasts.

    Since they have a smaller audience I imagine it’s going to be harder for KETC to find community-created backups of their stuff.

    I also imagine that if Missouri gave more of a shit about public broadcasting, this affiliate would be operating with better funded / more experienced IT staff. These little affiliates really rely on a lot of volunteers and green IT folks who do the work for the experience.

    • LordCrom@lemmy.world
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 day ago

      I do volunteer work for nonprofits. Its amazing that normal people think we are swimming in cash… newsflash, we are not.

      The IT budget is nothing so alot of what i do is setup open source stuff to meet needs…think libreoffice erc.

      External storage for large backups is not on the table. And no one wants the reaponsibilty of shuffling external backup drives to and from.

      • Ghostalmedia@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 day ago

        And IT staff is often a lot of young folks who are building their resume and make really dumb rookie decisions. I was once that rookie.

        • Dave.@aussie.zone
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 day ago

          All the people in the thread dissing the station and suggesting all they needed is “X many Y terabyte drives for $2.50 each and give them to people to take off site and store and rotate somewhere and all this wouldn’t be a drama”… Those people have yet to discover that manual processes using fallible humans does not work long term.

          The station’s IT staff no doubt looked at all of this and decided that off-site backups via a third party was the simplest and most cost effective route. I have no doubt the middle-company advertised that all the actual storage was in Backblaze, a quite reputable storage company. The only thing that has bitten them in the ass is that the middle-company has folded unexpectedly, requiring them to scoot to court to get Backblaze to give them access to their data.

          After all of this, it’s quite possible that IT will now get enough resources to do the backup via Backblaze directly. If Backblaze wants to put a bit of PR spin on it, they’ll task a few engineers to set this up for them. And if Backblaze was smart, they’d go looking for alllll the other little PBS affiliates in the same boat and offer them the same setup.

    • TheOctonaut@piefed.zip
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 day ago

      That’s kind of how bankruptcy works. You can’t give notice that you’re going to be bankrupt. The moment you’d give notice of bankruptcy, everything would collapse anyway as lenders call in debts and customers cancel orders that are now in jeopardy.

      Other statuses that exist are examinership (an outside appointee trying to save the company by any means) and receivership (probably the closest to what you’d like - the company is entrusted to a third party to try to pay its debts before it is wound up, usually trying to determine which debts to pay for the greater good, and which ones to default on).

  • Greg Clarke@lemmy.ca
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 day ago

    I’ll store PBS’s content on my Plex server if they want. I’m sure I’ve got 50Tb free