On Fear and Incompetence.

When the angels met the shepherds to announce the birth of Jesus, the first thing they said was 'Be not afraid".

That seems to crop up quiet a bit in the Bible, actually. "Do not be afraid".

Fear really does seem to be the mindkiller. I remember when I realized that, in American history, the slave states were afraid the northern states would outlaw slavery... and there's quite a bit of history leading up to the civil war that was centered around that. It seems ironic, though, that that fear led the south to try to secede - and ultimately brought their fears into reality. (Slavery is evil, and it seems the north wasn't going to free the slaves until pushed into it because of the war, so perhaps that was for the best. Then again, would we still have the lingering issues we do today if it had happened less abruptly? Yet can you really be okay with letting slaves suffer a minute longer than necessary... ? But if it led to less suffering now...  Since we don't have time travel that's mostly just hypothetical, though worth thinking about as we consider strategies for bringing about change.)

I was reminded of that while reading Adults in the Room, because in his preface Varoufakis mentioned Oedipus (and the self-fulfilling prophecy at his birth), and claimed that "Fearing that Greece's undeclared bankruptcy might cause them to lose control over the West, they imposed policies on Greece that gradually undermined their political control, not just over Greece but over... the West."

I think this is a recurring pattern, I've seen it with managers. 

In one of our courses we asked whether it was better to have a good plan with poor execution, or a poor plan with good execution. I think it's the latter... because any good plan of execution will include feedback. You have to assess whether your plan is actually achieving your goals (as well as whether it's happening on schedule, and a bunch of other things.)

If your plan isn't working as is, good execution means adjusting your plan as needed, and if you do that enough then eventually your poor plan will become a good plan. 

The trouble, though, is that feedback sometimes is scary. If you worked hard and think you're doing a great job, hearing criticism sounds like an attack. Like they're saying you suck, you failed, you aren't really good at all.

That's when criticism and feedback starts seeming like a threat to your authority, and people often respond poorly.

And so they do things that ultimately do make them fail. (This is part of why getting out of your bubble, valuing people who have the courage to speak up - especially when it's something you don't want to hear, and various other things are so important.)

We see the same dynamic with saying like 'it's not the mistake, it's the cover up.' 

People can be very forgiving if they know you were trying your best, and that you changed when you saw it was necessary. Take John F. Kennedy - the discussions on groupthink often talk about the disastrous Bay of Pigs invasion. Then they go on to talk about how JFK took the lessons learned from that failure and used it to improve his team's decision making, best shown during the Cuban Missile Crisis.

Anyways, Varoufakis mentioned two rather disturbing things (if you take him at his word, and for the purposes of this post I will). 

One was a call where someone basically threatened his son. The other was the way the 'troika', as he calls some of the established powers in Europe that were pressuring Greece during their crisis, basically tried to engineer a bank run when he and the party that made him Finance Minister came to power.

Both are the kinds of things that are hard to get attribution for, much like cyber attacks, which is why I said I'd take him at his word for this post. You can have a pretty good idea of why something happened without necessarily having the sort of proof you could take to court, and with all the conspiracy theories going around I generally lean towards 'you need to put up some proof for me to take it seriously', but that's more about needing some sort of standard then because people's theories are always wrong. (Though theories that require over 60 different judges of different political affiliations to be in on it don't really pass the sniff test.)

Anyways. As for the threatening phone call, I have to wonder about the people behind it. I'm assuming the caller worked for someone (and why didn't they question who they were working for, that they accepted an assignment like that?) Was that someone one person? Ten people? Did they have some sort of misguided justification that led them to think they were doing the right thing? Were they so focused on power that they knew it was wrong and just didn't care? Seriously, if you're sending threatening calls to people doing things you don't like, you probably ought to think long and hard about how you got there.

As for the bank run... I've come across indications that there can be some pretty shady things going on in economics/finance. (Iirc, there was something funky that helped trigger the Lehman Brothers bankruptcy, too).

I don't feel knowledgeable enough to speak out about that, other than to say that economics seems about half con artistry. 

Hmmm. That's not quite right. It's more like this - stock prices are supposed to reflect the company's ability to return value when they pay their stock holders. Some analysts to really in depth analysis of a company - their financials, the market, their plans for the next year - and make some judgment about just how much the stock is worth.

Others - are more about reading what everyone else is doing. Like in poker, which is as much about interpreting the body language of the other players as it is about knowing how good your hand is and the odds of getting what you need on the next flop.

Between people trying to buy low and sell high based on their assessment of what everyone else is doing, and programs that will buy or sell based on fluctuations in price, we often see severe swing that don't seem to have anything whatsoever to do with the basic fundamentals of the company. Con artistry. (Okay, not always, but there's plenty of room for it.)

Anyways, let's say there's the consensus of the IMF, ECB, etc. And their consensus is that austerity measures are required. 

A person focused on serving the people will evaluate the results and adjust as needed if it doesn't seem like their plan is working... whereas someone who is insecure will react as though it's a threat that needs put down.

The bank run sounds like the latter. Like, if your consensus is correct, if your policies are right - won't that become obvious when people try something different? (Same thing goes for Argentina, and China for that matter. They didn't act in keeping with the consensus - if the consensus is correct the failures will be harder and harder to hide. Unless... unless you do something that allows them to blame the failure on something else. Like a bank run.)

It really sounds more like bullying and control issues, rather than knowledgeable people acting on behalf of the public.

And that's the problem, isn't it?

Just like Trump's obsession with the optics of covid meant he didn't make policies focused on protecting American lives.

In short - it's incompetence, and you will lose power if you can't fix it.

Musings

I have been feeling a bit bereft lately...

I go through periods where suddenly fiction just doesn't appeal, which is when I often switch to non-fiction...

But lately I haven't wanted to read things that are too grimdark. Call it a reaction to the perpetual bad news. Maybe we really are all doomed, but I'd rather focus on what we can, rather than keep complaining about how much things suck. (and boy, do they!)

And yet I also don't want to read things irrelevant to the issues of the day.

So I started looking through the books I've saved on my 'want to read' list on Goodreads, where I throw pretty much any book rec I come across that sounds the slightest bit interesting. 

Fascinating though the history of a siege on Malta or the Silk Road may be, they didn't feel right. Same for all the computer and programming related material. Well, I did hear good things about Clean Code, so I may get to that when I'm ready for a break from the one I finally chose:

Adults in the Room, by a Greek finance minister back when Europe was pressuring Greece to do austerity measures a few years ago. I kind of remember the time period, though as an American I was only peripherally aware. 

Anyways, in the very first chapter we have this quote:

"‘There are two kinds of politicians,’ he said: ‘insiders and outsiders. The outsiders prioritize their freedom to speak their version of the truth. The price of their freedom is that they are ignored by the insiders, who make the important decisions. The insiders, for their part, follow a sacrosanct rule: never turn against other insiders and never talk to outsiders about what insiders say or do. Their reward? Access to inside information and a chance, though no guarantee, of influencing powerful people and outcomes.’ "

That's rather depressing, really. Either speak your truth and remain powerless, or keep your mouth shut and basically become part of the problem?

Yeah, not cool.

I wouldn't mind 'speak truth in private and keep a united front in public', that's not too different from what I'd expect in the military (though speaking a contrary opinion behind closed doors also has a time limit, once the decision is made. I've seen that that doesn't hold true for all orgs though, as ideas that weren't chosen keep getting brought up.)

Heck, I'd probably even be okay with something like my views on the 'thin blue line' and cops protecting each other (namely that I don't really care if you ticket a cop, so much as that you address the bad behavior - if not formally then informally. If you don't want to give your buddy a ticket for drunk driving, make sure you tell them when they're sober that they can't do that again. Whether they need to seek treatment or call you next time or whatever you come up with... Whatever you choose, you need to fix it. That's also why racist cops are rotting the whole police force. Why aren't they pressuring their fellow officers to knock it off, just like they harass the ones who break that unwritten law about protecting each other? If they can't fix it privately, then they shouldn't get upset when their messes become public.)

Anyways, that rule is also why it's so hard to tell what's really going on. The insiders aren't saying, after all.

Given what a mess those insiders create (and I selected this book partly because it's not straight up bashing them for that), I can't help wanting a peek behind the curtain.

Network Effects

More details on why social media gains (and loses) power with network effects:

https://doctorow.medium.com/social-quitting-1ce85b67b456

On Twitter, DevOps, and My Work Experience

 Given how everyone is discussing when Twitter is going to die, I figured I'd talk a little bit about my work experiences - mostly from my previous job.

I'm going to try to make this as non-technical as I can. Since I came to tech late in life, I remember well what it was like before I understood pipelines, configs, scripting, etc... I'll probably use less technical terms (like sure, I know what a vm, host, machine, etc is, but in this post I'm just going to call them all 'computers'.)

I think the first thing to understand is scripting. Or perhaps the command line interface (CLI). Most people use Windows if they use a computer at all... and so we're used to having menus that let us know whether we want to copy files, or we click a mouse and choose to delete a file. It's a graphical user interface (GUI) that lets us work without memorizing the commands to do them all from the CLI.

You can do pretty much all of the same stuff from a CLI, so long as you know the words. On windows you can look for 'cmd' to get a simple screen where you can enter the commands, or you can use Powershell, their newer CLI. 

The reason I bring that up is that if you know the commands, you can also jot them down in a file (Use Notepad preferably, since Word tends to add some invisible characters that can screw things up. In Windows you have to give the file the proper extension in order to run it, and make sure it's executable.)

So if you regularly do the exact same thing, every day, you write a script with those commands. If you want to create a backup of your important files, you can manually go and copy it over every day, but that gets tedious. And some days you might forget to do it.

Instead you save the commands (in plain English, you might say 'check this folder for any file created today, and copy it over to this other location').

You can then run that script whenever you want to take a backup. 

Even better, you can then schedule it so that the computer runs it for you. That way you don't even have to think about it. You don't have to remember to make a copy, you don't have to run the script... it's all automated.

In my last job, the team had spent quite a bit of time automating these sorts of repetitive tasks. Some of them hadn't been changed for over a decade... and as long as the process is the same, it works great. 

Makes my job easier... 

The problem is that it doesn't always work. Or things change, and the scripts have to be modified. I joked that my job was generally about dealing with things when they go wrong.

Perhaps the most dramatic example (though thankfully not common) came about when a computer became obsolete.

We all generally by new computers on a regular basis, so we don't have to deal with it as much... but a large company has so many computers that it gets hard to keep track of them. Hard to know what they do, and who is responsible. 

And unfortunately you can't just let them sit and run their thing. One of the biggest reasons is security - as malware evolves and adapts, so too does software, and older computers sometimes stop being supported and updated and have vulnerabilities that you just can't fix. So the solution is to upgrade it to something newer.

So we'd had a couple of computers hosting some internal web pages for our work. We were supporting testing in non-production, and every time the developers made a change to the code, that code had to be pushed to the test environments. We called that a build, and so naturally pushing it was called a build push.

We also sometimes needed to reboot the application, sort of like how powering your computer on and off can fix weird issues. Except applications are a bit more complex than your computer... they may actually be running on two or three computers, or more. You may have one that dealt with your application logic, and another one that dealt with the website itself. They also generally will talk to a database, since that's where user information and other things are stored. So to reboot the application you have to 'bounce' a couple of different things, often in a specific order.

So we had an internal page where the testers could push a build and bounce our applications (of which we had many... customer service, payment processing, billing, etc).

Most of this was automated, again due to the hard work of people a good decade ago, and so most of my time was spent dealing with situations where the automation didn't work.

Maybe something interrupted the computer while it was copying some files, and the files were incomplete. Maybe someone deleted a key file. Maybe the operating system needed upgraded, and we had to change our configurations to reflect that change. Maybe a firewall was preventing a computer from talking to another computer. Maybe something was missing in a file and the computer didn't know how to reach another computer. We also used what's called 'third party software', like the databases and web hosting. Which also sometimes needed upgraded (especially if there's a vulnerability. That log4j issue a couple of years back caused a lot of work in that regard, and it's not enough to make sure our own applications were updated. We had to patch or upgrade some of that third party software, too.)

The issues were plentiful, though it wasn't necessarily consistent. One day could be very quiet, another day might have five or six different issues in as many different environments and/or applications.

Everything is highly interconnected, and every change may have an impact on other things. Applications...

Before I studied computers, I mostly thought of them as a single file. You know, you download an app from the app store. or a *.exe file for some software you want on your computer... then you run it and hey, presto! You have the app.

But really, at least in a business like ours, they're not so simple. And I'm not just talking about microservices. (Microservices make the application more flexible and modular, because you can update a small portion of it without worrying that it will impact other parts of the application. Basically if you go to your cell phone providers site, you can do a variety of things. Create a new account and purchase a plan, pay a bill, check your usage, etc. You can break all of those things down into separate parts. Like 'add a new subscriber', and have a developer focus on just that part of the application.)

If the developers and/or testers want to make changes, they don't necessarily want to have to update the code everywhere... so a lot of times what they do is they make a configuration (config) file. The code will query the config file to get the value it needs, and you can update that config file without having to change the code.

This was also something I spent a lot of time on... sometimes for simple reasons, like they needed to change the url for something. Or use a simulator. 

Sometimes it was because new variables were needed for the changes that were done, and we'd have a call with the developer where they'd have us try updating the config file in different ways until they figured out the right settings to make the application work.

One time there was an issue I helped resolve... the application needed to copy some files from one computer to another, and it kept asking for a password whenever it connected. We were asked to configure it so that it could connect without a password (generally you can connect from one computer to another with ssh, and it normally will ask for your username and password when you do. However, there are ssh keys you can store so that it verifies the connection without a password.)

We configured that, and computer A was able to talk to computer B, but they were still having the same problem.

I had to dig into the process in order to figure out that there was actually a third computer involved. So computer A could talk to computer B, and it could also talk to computer C... but computer C was asking for a password every time it talked to computer B. The issue wasn't with the two computers they told us about, but with an entirely different one that they didn't even mention.

I think I posted some things before about the struggle we had teaching new people this job, because while with experience you can learn how to deal with the most common issues, there are plenty of times where we'd get asked to fix something... and had no real idea what was wrong or where to go. So you had to learn how to figure it out.

To get back to scripting... most of those were written a decade ago, and the people who wrote it have moved on to other positions. Luckily, when I started we had a wealth of institutional knowledge... most of my co-workers had been there a decade or more. That meant that I had people I could go to when I needed to understand how something worked.

But there were plenty of times where I basically had to teach myself what I needed to know. By which I mean, read through the script that manages the build push... and it may be calling other scripts, or querying a database, so then I have to figure out what that other script is doing. Or (if it was our database, since we had one to store a lot of our configuration details) go to the database and figure out what was configured incorrectly or missing.

It is a lot harder to read code than it is to write it... (although scripts aren't really considered code, I think this statement applies to them as well)

When you write it, you know exactly what the commands do. When you read it, you may constantly have to look up the commands yourself... unless you're already quite familiar with them. And if it's a really long script, you may have to spend quite a bit of time working through each line to figure that out.

Tbh, given time constraints, I was more likely to skim the code and/or search for key terms in order to figure out just the portion related to the issue at hand.

I emphasize scripting because, to me at least, they seem to be the workhorses of what we do. All those fancy pipelines to run a complicated process? Like a build push? 

A lot of times they're just calling scripts in a set order. (With redundancies built in, like retrying a script if it fails the first time. Or picking up where it left off if something went wrong, so you don't have to do everything all over again.)

The tools used in DevOps are useful, and automation is nice... but someone, somewhere, has to understand what it's doing. 

Because sometimes you need to change them, or fix something that went wrong. When it breaks, you have to have someone who understands how to fix it.

And that is most definitely a 'when'. Maybe it'll be a good decade before it breaks, maybe it'll be tomorrow. 

If the person who designed it has moved on, then the people who got hired afterwards have to understand it... and if that information isn't passed down, then they'll have to teach themselves.

Anyways, to bring this back to Twitter...

They just lost a LOT of institutional knowledge.

A lot of people seem to expect it to break at any moment, and I'm not sure it will. Businesses put a lot of effort into making sure the code in production is reliable (hence all those test environments in non-production that I worked on).

Sure, we've all seen things go wrong when software gets buggy. Most of the time that's because certain issues don't show themselves until you're dealing with real world volumes. (The code may work fine when only a fraction of the users are using it). But businesses know that they lose money when things don't work. Customers get frustrated and switch to a competitor, people planning to buy your product suddenly decide they don't really need it... buggy code loses money.

The existing production code is probably as reliable as they can make it... the problems generally start when something new is introduced. (Or when the hardware wears out. That's also a thing, of course. Most people don't wear out their hard drive before they're ready to buy a new computer, but the wear and tear of a major enterprise is a whole other story.)

Anyways, that loss of institutional knowledge, to me, means that every time someone is trying to troubleshoot something they're going to have a heckuva time finding someone who can let them know where to look, or what is supposed to be happening. 

They're probably going to have to teach themselves everything, see if they can figure it out by poring through the code. And scripts.

Oh, I almost forgot - in addition to software updates, computer upgrades, and config changes, another common task is dealing with certifications. That is, many applications have certs that help ensure secure connections... and they expire and need to be renewed. 

Figuring out the process for renewing certs (who provides them? What commands help you get the start and end dates when the certs are often encrypted? where are they stored?) can be a bit of a pain, too. 

Anyways, the poor people left behind are now going to have to figure everything out themselves. If they're lucky, maybe they're still friends with their former colleagues and can call someone up. If they're unlucky, they may spend days troubleshooting something that the people who knew the system could have fixed in minutes.

I don't know when or how it will impact production, it depends on what goes wrong, and when. I'm fairly confident, though, that it'll take longer than it would have, that issues may pile on since they'll still be troubleshooting one thing when another thing pops, and that it will be extremely stressful.

Twitter Related

I had been thinking about layers of abstraction, ostensibly to make things easier but often just adding another realm of complexity. Since discussions on who's been fired at Twitter has become a thing, this sort of morphed into a 'maybe I should talk a bit about my work experience'.

I still might, but I stumbled across this thread which is done by someone with more experience, across a broader range, and while it doesn't fit the abstraction theme it does highlight some of the complexity -

https://twitter.com/MosquitoCapital/status/1593541177965678592?t=ZZ1U_w9pb4NgbSWF1jzBBA&s=19

An Excellent Article On What It Takes To Be A Genius

https://www.bbc.com/future/article/20221028-why-theres-more-to-being-smart-than-intelligence

A Eulogy for Twitter

 All this stuff going on with Twitter has really got me thinking of social media - of how I use it, how it works, and so on.

I guess, first... the real strength of a social media site is it's network. Facebook, for example, has it's issues. And I probably would have left it years ago if it wasn't the only place where I can stay in touch with a rather farflung group of friends and family. 

Where else can I get updates on my old grade school friends? Ones I hadn't seen in years, since we moved away when I started high school? Where else can I learn what's going on with various aunts, uncles, cousins, step-brothers and step-sisters? (Not so much the immediate family, since I generally call them and talk to them directly.)

Leaving Facebook means leaving my only connection to these people, since it's practically impossible to convince all of them to jump ship at the same time. (Though as that post on the thermocline shows, if things get bad enough on that site it IS possible. It's just that it'd have to get pretty bad first.)

So social networks strength comes from it's members. The more people on the network, the more people you can connect with, the more powerful it is. Which translates into possible strength in other ways, since you now have a large potential audience for anything on that site... 

And each network fills a role. Facebook for friends and family. LinkedIn for work and business (some may use it on a regular basis, but I mostly just have an account there so that when I'm job hunting any of the HR people doing their due diligence can see that yes, I do have a presence. And it lists the same jobs I have in my resume... For the most part I don't actually care for or want to socialize on there. Though, again, I do have some contacts there - former colleagues and coworkers - that I don't have elsewhere.)

Tumblr is pretty much for fandom. I only got an account there because I followed my friend - who was in a number of fandoms - to that site. I only later realized that I'd been following her through most of the fandom migrations, since she also was the one that got me on LiveJournal. And migrated to Tumblr as part of the fandom movement. (She died last year, and I miss her. 😢)

I'm not sure if Discord should be listed as a social media site. I mean, it is? But it's more like the tech equivalent of salons... you generally have to get an invite to join a discord server, so it's more isolated and less public. Started as a way for gamers to talk while playing together, but obviously used for a lot more than that now. 

Anyways. Twitter. 

I started on Twitter mostly for infosec. I mean, I seem to have fallen into DevOps and I'm not sure if or when I'll ever change over, but I find the topic interesting and I really like the community. They're knowledgeable about a wide range of things. 

And given my interests, I suppose it's no surprise that once I started using it regularly I also started following various other topics of interest. Politics, of course. History. News. Random accounts for everything from the history of Khorasan to geology and cool looking rocks. Also was good for diversifying my feed, and getting the perspectives of all sorts of people.

In so doing, I found that Twitter filled a much different role than my other accounts. If Facebook is for friends and family, and LinkedIn for professional contacts, Twitter was very much like a free-for-all. You could engage complete strangers on almost any topic imaginable. 

I've heard people talk about it as the town square, or the equivalent of the water cooler at work. That fits very well, except that the town square is a public place and Twitter is a private company. (This is probably part of the problem that led to the current situation.)

I discovered that I often heard the news on Twitter first, though you have to be careful since initial takes are often wrong. There's also a lot of bots, and misinformation, so generally you have to confirm anything you hear there elsewhere.

Now that it seems to be dying (which I'm not sure is true, since everyone said the same about Tumblr and Tumblr is still there in all it's wacky glory... but then again, Elon Musk seems to have lost quite a bit of money and is talking about Twitter going bankrupt, so maybe it really will) I've had to consider where and how I'll find anything that will give me the same experience. Maybe Mastodon... maybe not. We'll see.

It's crazy how poorly Elon Musk seems to understand the business. Like - I'm not going to pay $8 for a blue check mark no matter how much you want me to. 

I might pay $8 for an ad-free tier, like some applications do, but really if he ever insists people pay for an account it'll be the death knell. Or rather, you might have some people paying to be on it - like Patreon, I assume - but that will also limit the size of the network, because a lot of other people won't. 

I suppose it's a bit like toll roads vs regular highways. Anyone and everyone can get on a public road, so they get a lot of use. Toll roads limit who uses them to who is willing to pay. 

If you want a business like a toll road, that's fine... you won't get everyone, but you can probably still make a profit out of it? Maybe? 

It won't be Twitter, though. Or rather, it won't have the characteristics that made Twitter the powerhouse it was, or possibly could still be.

It has been darkly amusing to see what a madhouse it's become at the moment. The fake accounts using their paid for blue check marks have been a riot... though I never expected it to have a real impact on stock prices. (Surely that's only temporary?)

I'm not sure what the future holds, or whether I'll find anything that fills that role...

I think I'll miss it. 

I kind of wish I had a billion dollars myself, so I could invest in building my own version. It seems there are a lot of former Twitter employees looking for a job, so it ought to be easy to find people with experience.

Alas, I am not in a position to do such a thing. Perhaps someone else is, and will.