I think stories like these highlight the need for clearer (not necessarily more) regulations around contractor/subcontractor/client relationships and what happens when one of them goes tits up.
There were rampant issues in the fintech world that exploded when Synapse, a banking as a service provider, went bankrupt and their ledger didn't match what partner banks had in their accounts. End users were told "your deposits are FDIC insured", but in bankruptcy court the judge was sort of at a loss over how to rectify things - the banks weren't insolvent, and the FDIC (rightfully) said "hey, this isn't our problem, our regulated entities are in compliance". Looks like a similar situation happened here, where the contractors are both doing the "not it" thing.
I feel like a lot of tech innovation and "business process innovation" over the past 15 years was just ignoring regulations that were built up over decades, only discovering the reasons for those regulations when the tide went out and seeing that lots of companies had been pantsless the whole time.
Whoever claims "your deposits are FDIC insured" needs to be prosecuted as fraud and scam artists. “Your” the company’s deposits in the bank is FDIC insured. My deposit with you the company is not. When the bank goes belly up, your deposit is FDIC secured up to the account limit which is tiny in the scale of things. When your company goes belly up, my deposit is gone.
My understanding is that they split the accounts to keep up with the limit. Otherwise though yeah, FDIC will only step in for the extreme minority scenario of failure by the underlying bank. You absolutely should not be allowed to advertise FDIC insurance unless it goes all the way to the consumer.
> When the bank goes belly up, your deposit is FDIC secured up to the account limit which is tiny in the scale of things.
The FDIC is meant to protect individual people from loosing all of their money from the collapse of a bank, currently at $250k. If you have more wealth than that yet have it all as cash in a single account, then, you're pretty much an ID10T. For regular mere mortals, that's a helluva lot better than a bank telling you to pound sand when they collapse. If you're a business thinking the gov't is meant to protect you, then you are also delusional.
If you talk to a lawyer (or, worse, a legislator, many of whom are also lawyers and all of whom are surrounded by them), they will insist up and down that this is a pretty standard custodial arrangement dispute and that the system worked as intended. And I would, very begrudgingly agree with that.
The legal system is perfectly capable of recognizing stolen property no matter how many layers of abstraction you put it through. The problem is always in the fact that the dispute resolution process is too expensive[0] to be useful. If you are defrauded for $10,000; but the legal fees for your representation will exceed that; then that juice ain't worth the squeeze. See also: Bricks and Minifigs.
In the Nine PBS case the judge correctly recognized Iron Mountain as a constructive bailee of Nine PBS's property and created a framework to retrieve their data. The problem is that this took years of legal work to get to the obvious outcome to make Nine PBS whole.
In Synapse's case, the problem is slightly different, because Synapse is not a bank, they are a reseller of banking services. That's the whole idea behind "fintech[1]" - that we can sell banking services while dodging all the regulatory compliance designed specifically to stop these kinds of issues so long as a real bank is involved. Saying their deposits are FDIC insured is like saying you have auto insurance because you happen to be riding a taxi. Technically correct but misleading and fraudulent. FDIC insurance doesn't cascade into your customers' accounts, because if it did, you'd be a bank.
[0] There's a similar problem with Bitcoin, where only a certain number of transactions can ever be processed per hour and thus it bottlenecks any higher-layer process that intends to use the Bitcoin blockchain as a settlement or dispute resolution system.
[1] "Fintech" in particular is meaningless as all banks are tech companies. They were one of the first adopters of electronic computers, online transaction processing, and a whole load of other things that seem utterly quaint now.
Open Source Storage (OSS), the vendor in question, was around for two decades before going out of business last year. The first and last archived versions of its web site:
I'd still argue that I.M. should have anticipated this situation, and have some process in place to address it. "See a court" might well be part of that process.
Sometimes these things can go better for companies like Iron Mountain when there's a court order/decision in place to cover them. Nine PBS, in order to access their own data, could end up accessing data belonging to other people and that may not be properly covered under existing contracts and policies. It's a risk for Iron Mountain if this happens. Having a court decision and court set procedures that essentially force them to participate and also establishes third party review of the data will give them some cover if one of those other clients of the now defunct OSS discovers that Nine PBS accessed or inadvertently retained their data.
A court is needed for cover since it is possible that sometimes else's data will be seen or even corrupted in the process. This way they can say they were doing what the court asked for. That is legal cover for everyone to do what they want. Also legal limits on what they can do.
It’s likely that Iron Mountain needed a court order in order to do this with legal cover.
It’s just a data warehouse and OSS likely had multiple clients data and backups. Iron Mountain can’t let one third party go searching through a defunct customers stuff that has a bunch of additional third parties property intermingled with it. Too many unknowns and potentially litigious third parties.
They couldn't really - I doubt Iron Mountain actually objected to giving the data, but in the end it probably wasn't in a position to know what data belonged to PBS and what belonged to other clients of OSS and you'd get very worried if a data storage company gave a companies data to someone else without authorisation.
If Iron Mountain made a deal with PBS-affiliate, they'd be breaking the terms of the contract with their customer (the fact it is defunct is just an asterisk). If you were an Iron Mountain client and one of your customers made an end run around you to go directly to Iron Mountain because they refused to pay your bill, you'd be pissed at Iron Mountain.
By solving this as they have done, Iron Mountain can assure other clients they will not just let a third person circumvent their clients. They can now say they only did it by a court order even if they were more than willing to accept the asterisk and do it on principle. Everyone is happy. Everyone is whole.
Thats my impression too. Iron mountain is doing exactly what I would want if I had data with them.
If you are wharehousing data, you dont just let your customer's customer come in, look around, and take whatever they want. Even if they have a sob story.
Also some articles and people on social media were unclear that it was just a single PBS station involved, rather than all PBS content. I saw comments that thought that all the archives for Sesame Street were lost.
Apparently PBS doesn't have some kind of massive conglomerate backup or archives of things that its member stations produce. It's up to every station to archive or back up their own stuff. No standardization as to storage formats, NASes, tapes, or anything.
Sometimes things are transmitted in painful ways like rebroadcast in off hours over microwave links between members so they can re-record what they lost.
Plus the endless game of "hey does anyone have that one obscure episode of this one program we made?" to your colleagues...
What is unfair? It seems completely normal and what courts are for sorting out. Iron mountain is acting exactly how I would want them to if they were storing data for me.
If you are wharehousing data, you dont just let your customer's customer come in, look around, and take whatever they want. Even if they have a sob story.
> These archives represent an important part of our region’s history, and we look forward to ensuring their preservation and protection through the Court-approved process.
Apparently you didn't find them important enough to have backups. Hundreds of thousands of dollars a year in net income and they couldn't afford a cheap NAS with 4 disks. Incredible.
They did, they contracted with a vendor to store and backup the data. If that is insufficient then just about every customer of AWS, Azure, GCP, OCP, and every other managed storage provider is guilty of the same sin.
I have to agree with GP here, this is pretty incredible. They contracted with a vendor to store their data, not to back it up. It's not a backup if you can lose it in the same instant you lose the data.
You're right, they are. If your data can't survive a single storage host vanishing from existence, you don't have a backup. If your data can't survive one predictable or regularly occurring catastrophic act of nature, you don't have a backup. If your data can't survive a piece of malware -- with all of the credentials you have -- erasing it, you don't have a backup.
EDIT: Also, if you don't regularly test that your backups actually work, you probably don't have a backup. Lots of companies learn that one the hard way.
I think stories like these highlight the need for clearer (not necessarily more) regulations around contractor/subcontractor/client relationships and what happens when one of them goes tits up.
There were rampant issues in the fintech world that exploded when Synapse, a banking as a service provider, went bankrupt and their ledger didn't match what partner banks had in their accounts. End users were told "your deposits are FDIC insured", but in bankruptcy court the judge was sort of at a loss over how to rectify things - the banks weren't insolvent, and the FDIC (rightfully) said "hey, this isn't our problem, our regulated entities are in compliance". Looks like a similar situation happened here, where the contractors are both doing the "not it" thing.
I feel like a lot of tech innovation and "business process innovation" over the past 15 years was just ignoring regulations that were built up over decades, only discovering the reasons for those regulations when the tide went out and seeing that lots of companies had been pantsless the whole time.
> … was just ignoring regulations that were built up over decades, only discovering the reasons for those regulations …
yeah it’s unreal to me how many people who imagine themselves intelligent are just now discovering the equivalent to why we make wheels round.
they never think to ask “why does regulation x exist?”
its absolutely crazypants.
Whoever claims "your deposits are FDIC insured" needs to be prosecuted as fraud and scam artists. “Your” the company’s deposits in the bank is FDIC insured. My deposit with you the company is not. When the bank goes belly up, your deposit is FDIC secured up to the account limit which is tiny in the scale of things. When your company goes belly up, my deposit is gone.
My understanding is that they split the accounts to keep up with the limit. Otherwise though yeah, FDIC will only step in for the extreme minority scenario of failure by the underlying bank. You absolutely should not be allowed to advertise FDIC insurance unless it goes all the way to the consumer.
> When the bank goes belly up, your deposit is FDIC secured up to the account limit which is tiny in the scale of things.
The FDIC is meant to protect individual people from loosing all of their money from the collapse of a bank, currently at $250k. If you have more wealth than that yet have it all as cash in a single account, then, you're pretty much an ID10T. For regular mere mortals, that's a helluva lot better than a bank telling you to pound sand when they collapse. If you're a business thinking the gov't is meant to protect you, then you are also delusional.
If you talk to a lawyer (or, worse, a legislator, many of whom are also lawyers and all of whom are surrounded by them), they will insist up and down that this is a pretty standard custodial arrangement dispute and that the system worked as intended. And I would, very begrudgingly agree with that.
The legal system is perfectly capable of recognizing stolen property no matter how many layers of abstraction you put it through. The problem is always in the fact that the dispute resolution process is too expensive[0] to be useful. If you are defrauded for $10,000; but the legal fees for your representation will exceed that; then that juice ain't worth the squeeze. See also: Bricks and Minifigs.
In the Nine PBS case the judge correctly recognized Iron Mountain as a constructive bailee of Nine PBS's property and created a framework to retrieve their data. The problem is that this took years of legal work to get to the obvious outcome to make Nine PBS whole.
In Synapse's case, the problem is slightly different, because Synapse is not a bank, they are a reseller of banking services. That's the whole idea behind "fintech[1]" - that we can sell banking services while dodging all the regulatory compliance designed specifically to stop these kinds of issues so long as a real bank is involved. Saying their deposits are FDIC insured is like saying you have auto insurance because you happen to be riding a taxi. Technically correct but misleading and fraudulent. FDIC insurance doesn't cascade into your customers' accounts, because if it did, you'd be a bank.
[0] There's a similar problem with Bitcoin, where only a certain number of transactions can ever be processed per hour and thus it bottlenecks any higher-layer process that intends to use the Bitcoin blockchain as a settlement or dispute resolution system.
[1] "Fintech" in particular is meaningless as all banks are tech companies. They were one of the first adopters of electronic computers, online transaction processing, and a whole load of other things that seem utterly quaint now.
Open Source Storage (OSS), the vendor in question, was around for two decades before going out of business last year. The first and last archived versions of its web site:
https://web.archive.org/web/20040628023451/https://www.ossto...
https://web.archive.org/web/20250329140721/https://www.ossto...
(The first version isn't too exciting. It's a broken Flash site.)
Earlier coverage (St. Louis KETC / Nine PBS sues Iron Mountain for data access):
"Nine PBS sues Iron Mountain over blocked access to archival data" <https://news.ycombinator.com/item?id=49285418>
This is working out largely as I'd suggested it should, albeit with a court's intervention. See: <https://news.ycombinator.com/item?id=49293058>.
I'd still argue that I.M. should have anticipated this situation, and have some process in place to address it. "See a court" might well be part of that process.
The ruling seems fair and reasonable, and I'm surprised they couldn't come to the same conclusion without a court.
Sometimes these things can go better for companies like Iron Mountain when there's a court order/decision in place to cover them. Nine PBS, in order to access their own data, could end up accessing data belonging to other people and that may not be properly covered under existing contracts and policies. It's a risk for Iron Mountain if this happens. Having a court decision and court set procedures that essentially force them to participate and also establishes third party review of the data will give them some cover if one of those other clients of the now defunct OSS discovers that Nine PBS accessed or inadvertently retained their data.
A court is needed for cover since it is possible that sometimes else's data will be seen or even corrupted in the process. This way they can say they were doing what the court asked for. That is legal cover for everyone to do what they want. Also legal limits on what they can do.
That is how I interpret the article anyway
It’s likely that Iron Mountain needed a court order in order to do this with legal cover.
It’s just a data warehouse and OSS likely had multiple clients data and backups. Iron Mountain can’t let one third party go searching through a defunct customers stuff that has a bunch of additional third parties property intermingled with it. Too many unknowns and potentially litigious third parties.
They couldn't really - I doubt Iron Mountain actually objected to giving the data, but in the end it probably wasn't in a position to know what data belonged to PBS and what belonged to other clients of OSS and you'd get very worried if a data storage company gave a companies data to someone else without authorisation.
Im curious why this topic has been popping up in the news. It seems like a bog standard contractor/subcontractor obligation case.
If Iron Mountain made a deal with PBS-affiliate, they'd be breaking the terms of the contract with their customer (the fact it is defunct is just an asterisk). If you were an Iron Mountain client and one of your customers made an end run around you to go directly to Iron Mountain because they refused to pay your bill, you'd be pissed at Iron Mountain.
By solving this as they have done, Iron Mountain can assure other clients they will not just let a third person circumvent their clients. They can now say they only did it by a court order even if they were more than willing to accept the asterisk and do it on principle. Everyone is happy. Everyone is whole.
Thats my impression too. Iron mountain is doing exactly what I would want if I had data with them.
If you are wharehousing data, you dont just let your customer's customer come in, look around, and take whatever they want. Even if they have a sob story.
You ask for a court order with specifics.
The initial reporting implied it was some catastrophic data loss, like https://en.wikipedia.org/wiki/2008_Universal_Studios_fire. The truth, while concerning, is a lot more boring.
Also some articles and people on social media were unclear that it was just a single PBS station involved, rather than all PBS content. I saw comments that thought that all the archives for Sesame Street were lost.
Apparently PBS doesn't have some kind of massive conglomerate backup or archives of things that its member stations produce. It's up to every station to archive or back up their own stuff. No standardization as to storage formats, NASes, tapes, or anything.
Sometimes things are transmitted in painful ways like rebroadcast in off hours over microwave links between members so they can re-record what they lost.
Plus the endless game of "hey does anyone have that one obscure episode of this one program we made?" to your colleagues...
It's the original torrent network
Archival/secure storage is big business with big liability headaches if you make a bad decisiion.
A non-profit in a deeply unfair situation tugs at the heartstrings.
What is unfair? It seems completely normal and what courts are for sorting out. Iron mountain is acting exactly how I would want them to if they were storing data for me.
If you are wharehousing data, you dont just let your customer's customer come in, look around, and take whatever they want. Even if they have a sob story.
You ask for a court order with specifics.
This audience thinks data center operators are Jedi.
> These archives represent an important part of our region’s history, and we look forward to ensuring their preservation and protection through the Court-approved process.
Apparently you didn't find them important enough to have backups. Hundreds of thousands of dollars a year in net income and they couldn't afford a cheap NAS with 4 disks. Incredible.
They did, they contracted with a vendor to store and backup the data. If that is insufficient then just about every customer of AWS, Azure, GCP, OCP, and every other managed storage provider is guilty of the same sin.
I have to agree with GP here, this is pretty incredible. They contracted with a vendor to store their data, not to back it up. It's not a backup if you can lose it in the same instant you lose the data.
You're right, they are. If your data can't survive a single storage host vanishing from existence, you don't have a backup. If your data can't survive one predictable or regularly occurring catastrophic act of nature, you don't have a backup. If your data can't survive a piece of malware -- with all of the credentials you have -- erasing it, you don't have a backup.
EDIT: Also, if you don't regularly test that your backups actually work, you probably don't have a backup. Lots of companies learn that one the hard way.
If you do not have at least one, but usually two copies of a backup in your physical possession, you do not have a backup. |o-o|
Then that is just the 1 in 3-2-1 backups, and neglecting the 3 & 2 is a sin (wrt backup strategies).
I wonder the proportion of companies who store backups with a minimum of two separate vendors. Low?