This year, something a little different. No opinion-piece on the blog, just 50 notes of hard-earned wisdom. Garnered from funny or painful experiences spanning two decades in InfoTech. Enjoy.
- It’s always DNS.
- RTFM.
- Always ask: which tech solution, or method, requires the Least Amount Of Administrative Effort. You can accomplish a whole lot more, when you have less to do.
- Read-only Friday — Never deploy to prod or make changes on Fridays.
- If it can go wrong, it will go wrong.
- If given enough time, most tickets solve themselves.
- When in doubt, blame the security configs or your predecessor.
- Backups don’t really exist unless you have multiple copies (3-2-1 rule). Backups always work, restores not so often. Just because the backup said “success” doesn’t mean it backed anything up. Trust, but verify. A backup is not a backup unless you have recently performed a restore from it, better yet do automated restore tests with reports frequently. Always test your backups - otherwise you like playing Russian roulette with 5 bullets in the gun.
- Backups will fail, see rule # 8.
- RAID is not backup.
- Document all the things. If your documentation takes longer to read than to fix the problem, you’re doing it wrong.
- Automate everything you possibly can (…that you’ve had to do > 2 times).
- Don’t make yourself a single point of failure.
- Don’t let other people make you a single point of failure.
- Always check the logs.
- Google is your friend. So is ChatGPT.
- Test, but verify. It’s not just for backups.
- Never stop learning.
- A lot of end user issues are the direct result of IT focusing on technical solutions and ignoring user experience.
- A lot of user experience issues are the direct result of the C-Suite not being invovled in setting IT Priorities & Policies.
- IT departments need to remember we’re there to solve business problems: if your solution is amazing technically, but doesn’t serve the business, you didn’t do you job.
- Know where the food is located.
- Never EVER let Perfect be the enemy of Good, or prevent forward-progress.
- The Infrastructure is not your baby https://github.com/dbeta/RulesofIT/blob/main/Rules/19.md
- If you make something idiot proof, the world will make a better idiot.
- Nothing is user-proof, either.
- There’s no crying in IT. Hyperventilating and panic, yes…but definitely no crying.
- If it’s not in the ticket, it didn’t happen.
- When it comes to network engineering, coding, or systems management: Design is everything.
- Technology does not fix bad design.
- If you are the smartest person in the room, you are in the wrong room.
- Always mispronounce with authority.
- Temporary is permanent: “this bandaid fix on the server is just temporary…” —somebody 5 years ago.
- There are no policies, only levels of resistance.
- The end user will lie.
- Don’t acknowledge or provide technical info about an active outage event to users: deflect until the problem is fixed. Nobody gets explanations until after your team analyzes the root cause, after uptime is restored. That’s what post incident-reports are for. Don’t allow a VIP to bait you into an explanation death-spiral while still troubleshooting: they don’t care, and are onto blame-assignment. Reason and patience only return after services get restored.
- Don’t do special favors because people will expect it everytime.
- Redundancy is key: two is one, one is none. Redundancy is key: two is one, one is none.
- Get that request in writing.
- When estimating time, add at least 50%.
- Good, Cheap, or Fast. Pick 1.
- Don’t chain people on emails where they have to read a week-long thread, to even understand the conversation or context.
- Design for uptime and redundancy where you can: if I can’t randomly unplug something in your datacenter, without a production outage, it’s a disaster waiting to happen.
- https://xyproblem.info/
- https://nohello.net/en/
- Remember that we work in a service industry.
- Always keep your resume updated. 😉
- Work life balance - Set standards with how approachable you are outside of work hours.
- Dev is dev and prod is prod. NEVER put anything that impacts prod in your dev environment.
- AI won’t take your job, someone using AI will take your job.
Bonus. If you’re in charge, and you have to tell people you’re in charge…you’re not in charge.