Tuesday, July 23, 2013

For Loop for Remote SSH Command Execution

This is something new I'm trying. Instead of trying to only post about large projects, difficult tasks, or troubleshooting (which often means I have a lot of posts that are sitting around waiting to be completed), I'm going to also start including quick snippets of knowledge that I would otherwise deem to small to dedicate to a whole post. So, without further ado... 

We're working on rolling out a CMS (Config Management Service) called Salt Stack. Our work with Salt is one of those aforementioned posts that sits in editing limbo. I needed to restart the salt minion on several servers because the hostname of the Salt master had changed. I knew I could do this with a simple for loop. I created a file called servers.txt, and added all of the servers I needed to reach into it. I then called the file with this command:

for $i in `cat servers.txt`; do ssh user@$i service salt-minion restart; done

I tested this first by having the remote command be "date" instead to confirm that I had the right syntax. It worked, so I moved on to actually restarting the service. The first box worked fine:

user@server1's password:
Stopping salt-minion daemon: [  OK  ]
Starting salt-minion daemon: [  OK  ]
^CKilled by signal 2. 

It would stop and start the daemon, but not complete and move on to the next host in the list. I had to Ctrl+C to get it to ssh to the next box.

I found that I needed to use "-t" with the ssh command.

for $i in `cat servers.txt`; do ssh -t user@$i service salt-minion restart; done

What does the -t flag do? Apparently, when you log in with ssh it uses psuedo-tty. This is not the case if you use ssh to send a remote command. If you execute a remote command that only has plaintext output (like the "date" command), no problem. However, if you use -t it forces the allocation of a psuedo tty session.

Lesson learned.