Company underbosses outsource key infrastructure to outsource reponsibility w/inevitable worldwide failure as result

Sometimes outsourced computer infrastructure fails but then at lest the companies can blame 3'rd parties and say they can do nothing about it. /As if that is a positive.

The truth is that it is still their responsibility and top bosses shouldn't accept such feeble excuses from their management team. The different departments made thier choice and when it fails, even if its a third party fail, they need to still own it. when you outource the running and key parts of your infrastructure to sample Microsoft Azure it will still be you customers problem when it fails, and therefore yours. 

Large serverfarms are often remote from key personell. And the more of it there is the less personell there are per server. Do you trust them to test upgrades thoroughly before mass rollout. Do you thrust them to only mass rollout to a part of the network so that those can be isolated and other parts still function if mistakes are done. Did you check that this is the case, always.

Are you sure your setup are really multisite for redundancy or just physically multisite but on 2 or more closely connected sites that actually all get upgrades at the same time.

The worldwide affected and multiple types of service like airlines, airports, banks, retailers and pharmacies that was affected today by the Crowdstrike update points to a cloud service. And since it was Microsoft servers you can be pretty sure it was the sky service Microsoft Azure that was hit. And that Microsoft did a rapid worldwide rollout without a time between sites to check that all was till running smoothly. A shortcut probably implemented by some cost saving individual/team/boss without enough experience beetween them to realise that sometimes things goes south and that there need to be procedures and safeguards to catch that.


Some critical parts of the infrastructure should still be under direct control by the inidividual companies. At least they can then more diriectly control what level of redundancy and risk they are willing to have/take. 

CEO's need to stop letting their underbosses outsource their responsibilities to non-inhouse setup. Remind them that they can outsource some of the work but never the responsiblity. 

Comments

Popular posts from this blog

Questionmarks over how Boeing saved costs when using plug instead of door

When is a bolt not a bolt

Has SAS management done enough deals for the airline to ever again make any profit