Home > Mobile >  Getting the last item in an array of hashes in my for loop in a Chef custom resource
Getting the last item in an array of hashes in my for loop in a Chef custom resource

Time:01-21

I'm kind of stumped on this one... I am quite aware of the concept between the Chef run stage and compile stage but I still cannot seem to get what I need to accomplish figured out.

I think the solution listed in How to store array in node object chef? may have helped in a normal recipe but I am trying to create a custom resource that basically reads a databag that has the following structure:

  {                                                                                                                                                                    
  "id": "local_files",                                                                                                                                               
  "development":                                                                                                                                                     
    [                                                                                                                                                                
      {                                                                                                                                                              
        "source": "https://sourcecode.mydomain.com/projects/ASX-NBJG/repos/sapops/raw/config/lama/Z_LVM_CheckBackup.conf?at=refs/heads/feature/ADVR-575-as-a-basis- 
user-i-want-to-be-able-to-use-chef-to-copy-files-locally-to-servers",                                                                                                
        "target": "/tmp/some_folder/Z_LVM_CheckBackup.conf",                                                                                                         
        "owner": "root",                                                                                                                                             
        "group": "root",                                                                                                                                             
        "mode": "0777",                                                                                                                                              
        "action": "create",                                                                                                                                          
        "run_command" : "echo 'hello my beautiful world' > /tmp/helloworld.txt"                                                                                      
      },                                                                                                                                                             
      {                                                                                                                                                              
        "source": "https://sourcecode.mydomain.com/projects/ABCD/repos/build/raw/copylocal/sudoerspolicy?at=refs/heads/feature/ADVR575CopyLocal",   
        "target": "/etc/sudoers.d/mysudoers",                                                                                                                       
        "owner": "root",                                                                                                                                             
        "group": "root",                                                                                                                                             
        "mode": "0440",                                                                                                                                              
        "action": "create",                                                                                                                                          
        "run_command" : "echo \"sudoers file updated on `date`\" |tee -a /tmp/sudoers_updated.log"                                                                   
      },                                                                                                                                                             
      {                                                                                                                                                              
        "source": "file:///erpsoftware/auto/copylocal/testtar.tar",                                                                                               
        "target": "/tmp/testtar.tar",                                                                                                                                
        "owner": "root",                                                                                                                                             
        "group": "root",                                                                                                                                             
        "mode": "0440",                                                                                                                                              
        "action": "create",                                                                                                                                          
        "run_command" : "mkdir -p /tmp/letsSeeHowItGoes && tar -xvf /tmp/testtar.tar -C /tmp/letsSeeHowItGoes"                                                       
      },   
 ],       
"qa":                                                                                                                                                     
    [                                                                                                                                                                
      {                                                                                                                                                              
        "source": "https://sourcecode.mydomain.com/projects/ASX/repos/ops/raw/config/stuff/Z_LVM_CheckBackup.conf?at=refs/heads/feature/ADVR-575-as-a-basis- 
user-i-want-to-be-able-to-use-chef-to-copy-files-locally-to-servers",                                                                                                
        "target": "/tmp/some_folder/Z_LVM_CheckBackup.conf",                                                                                                         
        "owner": "root",                                                                                                                                             
        "group": "root",                                                                                                                                             
        "mode": "0777",                                                                                                                                              
        "action": "create",                                                                                                                                          
        "run_command" : "echo 'hello my beautiful world' > /tmp/helloworld.txt"                                                                                      
      }
   ]

The resource call:

copy_local 'Ensure all files that need to be copied locally are handled...' do
  databag_itemid node['srv']['copylocal_databag_id']
  action :from_databag
end

The problem I have is, even if I delete the first file, the "run_command" that my code chooses is always the last item in the array of hashes.

I have a simple loop like this:

resource_name :copy_local

property :databag_itemid, String
property :databag_auth_required, [true, false], default: true
property :databag_name, String, default: node['srv']['databag_name']
property :databag_env, String, default: node['scm_appbranch'].downcase
property :secret_databag_itemid, String, default: 'sa'
property :secret_keysrc, String, default: node['srv']['sa_sec_key_src']
property :secret_key, String, default: node['srv']['sa_secret_key']
property :service_acctname, String, default: 'service_account'
property :service_acctpwname, String, default: 'service_account_pw'
property :files, Hash
property :files_env, String, default: node['scm_appbranch'].downcase
property :debug, [true, false], default: false

  action :from_databag do
  
  # load variables
  skey      = new_resource.secret_key
  skeysrc   = new_resource.secret_keysrc
  sdbid     = new_resource.secret_databag_itemid
  dbenv     = new_resource.databag_env
  dbid      = new_resource.databag_itemid
  dbname    = new_resource.databag_name
  dbauthreq = new_resource.databag_auth_required
  saname    = new_resource.service_acctname
  sapwname  = new_resource.service_acctpwname
  dbg       = new_resource.debug

  # DL the secret
  remote_file skey do
    source skeysrc
    action :nothing
    sensitive true
  end.run_action(:create)

  # Load the secret and try to decrypt
  secret = Chef::EncryptedDataBagItem.load_secret(skey)
  begin
    credentials = data_bag_item(dbname, sdbid, secret)
    user        = credentials[saname]
    password    = credentials[sapwname]
  rescue StandardError => msg
    puts 'ERROR :: Could not get credentials from encrypted databag!!!'
    raise msg
  end

  # load the data bag item for copy local functionality
  all_local_files = data_bag_item(dbname, dbid)

  # load the hash
  my_files = missing?(all_local_files[dbenv]) ? all_local_files : all_local_files[dbenv]

  # now loop through files and begin local copy via the remote_file resource
  # This loop does not work, need to find a way to loop through and not only get the last item of the array
  auth = "Basic #{Base64.encode64("#{user}:#{password}")}"
  my_files.each do |file_obj|
    Array.wrap(file_obj).each do |file|
      # check all flavors for each file object
      checks = check_all_flavors(file)
      unless checks.has_value?(false)
        
        # Debug
        puts " :: D E B U G :: ==> file source     : #{file['source']}" if dbg
        puts " :: D E B U G :: ==> file target     : #{file['target']}" if dbg
        puts " :: D E B U G :: ==> file mode       : #{file['mode']}" if dbg
        puts " :: D E B U G :: ==> file owner      : #{file['owner']}" if dbg
        puts " :: D E B U G :: ==> file group      : #{file['group']}" if dbg
        puts " :: D E B U G :: ==> file action     : #{file['action']}" if dbg
        puts " :: D E B U G :: ==> file run_command: #{file['run_command']}" if dbg

        # Create the directory for the parent folder of the file['target'] if it doesn't exist
        directory dir_name(file['target']) do
          recursive true
          mode   file['mode']
          owner  file['owner']
          group  file['group']
          action :create
          not_if { dir_exists?(dir_name(file['target'])) }
        end

        # use remote_file resource to copy the file locally
        remote_file file['target'] do
          source file['source']
          mode   file['mode']
          owner  file['owner']
          group  file['group']
          headers('Authorization' => auth) if dbauthreq
          action file['action']
          notifies :run, 'execute[run-command-for-copy-local-databag]', :immediately unless missing?(file['run_command'])
        end

        # resource to execute any command specified in copy local attributes
        # allows us to "copy local and execute a command"
        execute 'run-command-for-copy-local-databag' do
          command     file['run_command']
          environment file['run_env']
          creates     file['run_creates']
          cwd         file['run_cwd']
          group       file['run_group']
          user        file['run_user']
          action      :nothing
        end
      end
    end
  end

  # # delete secret file once data bag is decrypted successfully
  # file skey do
  #   action :delete
  # end

end

But as I mentioned, instead of the affiliated "run_command" executing, it is always the last "run_command" in the list. To me, this seems like a pretty straight forward loop, and things are working for owner, group and permissions, but my "run_command" execute resource block seems to always be picking up the last item in the databag. Meaning, even if I purposely delete the /tmp/some_folder/Z_LVM_CheckBackup.conf file, when it should be triggering the affiliated echo "hello my beautiful world" command, it is running the last command in the list for the rum_command key: "run_command" : "mkdir -p /tmp/letsSeeHowItGoes && tar -xvf /tmp/testtar.tar -C /tmp/letsSeeHowItGoes"

I tried to implement the strategy laid out in How to store array in node object chef? with using the node.run_state but this didn't get me any luck either. Pretty stumpted at this point and would greatly appreciate any help.

Thanks! Steve

CodePudding user response:

So I was able to fix this by making my execute block "unique." I did this:

  my_files.each_with_index do |file_obj, index|                   
    Array.wrap(file_obj).each do |file|                           
      # check all flavors for each file object                    
      checks = check_all_flavors(file)                            
      unless checks.has_value?(false)                             
                                                                  
        # set cmd file for correct execution of commands          
        cmd_filename = "file #{index}: \"#{file['target']}\"".to_s

And then called my execute resource like this:

    # use remote_file resource to copy the file locally                                                                                  
    remote_file file['target'] do                                                                                                        
      source file['source']                                                                                                              
      mode   file['mode']                                                                                                                
      owner  file['owner']                                                                                                               
      group  file['group']                                                                                                               
      headers('Authorization' => auth) if dbauthreq                                                                                      
      action file['action']                                                                                                              
      notifies :run, "execute[run-command-for-copy-local-databag for #{cmd_filename}]", :immediately unless missing?(file['run_command'])
    end          

    # resource to execute any command specified in copy local attributes
    # allows us to "copy local and execute a command"                   
    execute "run-command-for-copy-local-databag for #{cmd_filename}" do 
      command     file['run_command']                                   
      environment file['run_env']                                       
      creates     file['run_creates']                                   
      cwd         file['run_cwd']                                       
      group       file['run_group']                                     
      user        file['run_user']                                      
      action      :nothing                                              
    end                                                                                                                                                                                         

Problem is now resolved. The reason the execute block was running the last item in the array was because the name of the execute resource was static. Once I added a "each_with_index" and then concatenated that with the file name, and renamed the execute block in the same manner, it fixed my issue and is now working consistently. I hope this helps someone else out!!

  • Related