This is the first in what I think will be a regular thing on Ops n’ Lops, a look at the toolset that I use! This post is going to feature the basic tools that I use every single day to access servers.
Opsen are a fickle group when it comes to our tools, and I am no exception!
Every professional has a well-worn and loved set of tools they’ve acquired and worked with over their career. Just as carpenters have beloved hand-tools, and musicians have well worn and crafted instruments, operation engineers all have a set of configurations and workflow items that we’ve molded and shaped to fit our fancy.
My day-to-day workflow looks like this:
I work locally off a Mac, connected to our remote environment over a VPN, accessing my “home base” machine within our environment over mosh.
I do all my work off of the remote VM, and not on my local machine. I try to keep as little confidential data on my local machine as possible in case it’s ever stolen. (I commute via public transportation, so this is actually a fairly large concern!) Basically my local workstation is nothing more than a battery powered dumb terminal with an amazing screen and great keyboard.
At work each engineer (developer, ops, or anyone that wants one, really) is given their own VM to work in. It’s ours and we can do with it what we want. (Truth be told, I have four VMs at the moment, but that’s because I work with machine images and need places to test them!)
A common question we debate endlessly at work is: “Where is the best place to run your ssh-agent?” There’s good arguments on each side, but I’m firmly on the side of “keep as little confidential data locally as possible,” and an SSH key is no exception. (It can be a moot point, mostly, because SSH assumes that your servers have a way to share public keys, which is not trivial when running at scale!)
So why mosh and not SSH? Mosh is designed to work on high-latency, lossy connections. It maintains state locally so I don’t have to wait for keystrokes to echo back, which is great when using cellular-based connections. But the best part of mosh is that it can handle connections changing over time. I can be connected at the office, and close the lid on my laptop when it’s time go, and leave without a care. When I get home, the session just resumes as I’d left it, even if my IP address changed!
Technical explanation: it uses SSH to establish the remote connection (it starts a process on the remote host that maintains the session state), but then drops the SSH connection and transmits display updates over UDP instead. The authors claim it’s as just as secure (if not better) as SSH, but I run it over a VPN just to play it safe.
A Closer Look at Some of my Tools
Now I’ll talk a little bit about each of the tools I use, and why.
My local machine is a MacBook Air. I use it mostly because that’s what I was handed on my first day at work, but also because it’s a good machine. Apple laptops tend to have great displays, and keyboards that are good to work with all day. (Many hours of battery life is nice, too!)
I adore OSX because it’s basically UNIX with a very nice and workable UI. When I open up a command line I’m instantly at home, and the GUI is very nice to work with all day and into the night.
The folks on my team are mostly split between OSX and Linux for their local workstations. Both work well! I actually gave Linux a shot for a few months before going back to OSX. The reasons I stuck with OSX had nothing to do with the tools of my job… it was the little things that the apps just do better on OSX. (The OS itself is great.)
What about Windows?
I suppose it’s possible to use Windows in ops, but I don’t know anyone that does. It’s missing some pretty basic functionality that I depend on – like a good, workable (aka POSIX) command line. It also is lacking a lot of things that I take for granted on the UNIX side, like SSH. (I mean the entire suite of tools – including things like scp.) Windows is sorta the land that ops forgot… go look at the mosh download page for a Windows binary. You won’t find it.
I use iTerm 2 as my local terminal for a shockingly simple reason – it allows me to set up key macros. I have a few built that make interacting with tmux much faster… like Cmd-<LeftArrow> and Cmd-<RightArrow> to change between windows. I’m so use to having those macros built that I get really grumpy when I’m working on a machine without them. 🙂
There are a lot of strong feelings about font your terminal uses. I am a HUGE fan of Anonymous Pro. It’s one of the very first things I install when I’m setting up a new machine. If you’re looking for a good font, try it! Magic happens around size 15pt when the OSX anti-aliasing kicks in just right. O looks different than 0, and l looks different than 1.
I use a long-running tmux session as my main workspace. I picked tmux over screen because I’ve had better luck with it over time, but some of my co-workers swear by screen. To each their own. 🙂
Putting it all Together
Here’s a sample screenshot of what I look at all day, every day:
That’s bash, running in tmux, running in iTerm 2 with my font set to Anonymous Pro (15pt) on MacOS X. 🙂
I hope this has been an interesting look at the the tools I use all day, every day. Thanks for reading, and I think I’ll be doing more “Tools of the Trade” features in the future!
One response to “Tools of the Trade: The Basics”
Ooh, I’m going to have to check out mosh, for the same reasons you’ve outlined – cellular connections are usually fine, but.. not always. And being able to pick up again seamlessly after shifting to an entirely different connection would certainly be a pleasant bonus.