Thursday, March 5, 2015

Wget Mystery

This is just a quick snippet of some one-off weirdness. I was trying to deploy the Community Edition of Aerospike via Chef. I created a recipe and tested it out with Vagrant successfully before pushing it up to our Chef server. I then attempted to deploy the recipe to our 8-node Aerospike cluster and got error messages on the extract step. The recipe looks like this:

aerospike_tar_directory = "#{Chef::Config[:file_cache_path]}"
aerospike_tar = "#{aerospike_tar_directory}/aerospike-server-community-3.5.3-el6.tgz"
aerospike_temp_dir = Dir.mktmpdir

remote_file aerospike_tar do
  source node[:aerospike][:url]
  mode 00644
end

directory aerospike_temp_dir do
  owner 'root'
  group 'root'
  mode '0755'
end

execute 'extract_aerospike' do
  command "tar zxf #{aerospike_tar}"
  cwd aerospike_temp_dir
  action :run
end

user 'aerospike' do
end

directory node[:aerospike][:dir] do
  owner 'aerospike'
  mode '0755'
  recursive true
end

script 'move_aerospike' do
  action :run
  cwd aerospike_temp_dir
  interpreter 'bash'
  code <<-EOH
  mv #{aerospike_temp_dir}/aerospike-server-community-*/* #{node[:aerospike][:dir]}
  EOH
end

execute "install_aerospike" do
command "cd #{node[:aerospike][:dir]}; ./asinstall; chown -R aerospike:aerospike #{node[:aerospike][:dir]}; rm #{node[:aerospike][:dir]}/*.rpm"
end

service "aerospike" do
action [ :start, :enable ]
end


The error message I was receiving indicated that the downloaded file, aerospike.tgz, was not in fact a tarball. Sure enough I tested it and it was a text file with HTML enclosed. The HTML said the resource wasn't found. I tested it directly from my local machine and confirmed that hitting that same url on my Mac got me a properly formatted .tgz file. I ran wget on the server and was confused that it did not in fact download the file I was expecting. It instead downloaded a file with a different name all together. I repeated this process a couple of times to verify that I wasn't hallucinating.

A little research led me to the flag --trust-server-name. This flag is apparently used with wget when there are multiple redirects on the resource (which there were in this case). If there are redirects wget will download the file as the name of the last component in the redirection url, as opposed to the original resource name. Thus, if I did wget http://aerospike.com/download/server/latest/artifact/el6, without that flag wget would throw a file named el6 in my directory instead of the expected aerospike-server-community-*-.tgz file.

My Chef recipe still isn't exactly working, but I've solved one small problem at least.