• Jo Miran@lemmy.ml
    link
    fedilink
    arrow-up
    0
    ·
    edit-2
    22 days ago

    The reason the problem I am talking about exists is because it is terribly boring and mundane. It is also 100% a cost center, meaning that it provides only cost and no possibility of profit. Things that explode or can explode are very high profile and people notice them. Mundane problems go unchecked until after the shit has hit the fan and politicians are looking for a scapegoat.

    I deal with information security. Initially when I type that people instantly think “hackers”. True, information security does deal with a lot of “keep out the baddies”, but more than that we also make sure that data reaches its intended destination when it is supposed to reach its intended destination. For example, you might want your fire suppression system to trigger as soon as a fire is ignited and not after everyone in the building is burned alive or dead from smoke inhalation.

    Right now I have a situation where everything is working well but I know that if something happens to this one thing, a very mundane system is going to collapse and literally nobody can fix it adequately. For the past five years we have done everything within our power to add redundancy but as I mentioned before, this is a mundane cost center. Nobody wants to spend money to fix something that works. So, when the thing no longer works, service will be tremendously degraded, people will figure out that it cannot be fixed, and the search for a replacement will begin. Eventually they will succeed but in the meantime things are going to suck and some people might die.

    “Greed is good” – Gordon Geko

    " Greef is self-defeating " – JoMiran

    • Monument@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      0
      ·
      22 days ago

      It’s the final decommissioning of 2g/3g and how it’ll affect monitoring tools!
      (I don’t know, now I’m just being silly.)

      But, yeah. I get that. The profit motive/cost to build in redundancy, or a failure of imagination for future technologies so certain kinds of redundancy or features aren’t included is problematic.

      Not saying it is your issue, but the 3g issue is on my mind because it was a big deal in my neck of the woods last year when local carriers shut it down, and it is again because a utility failed to update their meters in a timely manner, causing them to estimate on customer utility bills for longer than is allowable, resulting in statutory violations and customer overcharges. They got a pretty hefty fine and an order to refund the overcharged customers. I could see how similar foundational technology issues could completely fuck a utility for… whatever impacted systems are involved. (Also, going to be editing out this part of my comment in a few hours, because I don’t like how much specific info is in there, even though everything I just described is public record.)

      • Jo Miran@lemmy.ml
        link
        fedilink
        arrow-up
        0
        ·
        edit-2
        22 days ago

        I’ll also delete this purely hypothetical, definitely did not happen, scenario in a few minutes.

        Imagine a smart gas meter. Regulations state that said gas meter shutdown and alerts crews if a leak is detected within two minutes of fault. The automated system might have missed the requirement by almost 45 minutes before issuing the command. People died…hypothetically.

        People don’t understand how flimsy everything we depend on is. It’s easier to imagine a 13 year old bullshitting and trying to be cool than a man, tired of sounding the alarm, venting on Lemmy.

    • invertedspear@lemm.ee
      link
      fedilink
      arrow-up
      0
      ·
      22 days ago

      When everything works: “What do we even pay IT for?”

      When everything’s broke: “What do we even pay IT for?”

      “When you do your job right it’s as if you didn’t do anything at all”

      • God to bender in Futurama

      When they start looking for a scapegoat, I hope you find yourself far away from the eye of Sauron there.