ansible: with_items key and value

- name: Tuning sysctl
    name: "{{ item.key }}"
    value: "{{ item.value }}"
    state: present
    - { key: net.core.somaxconn, value: 65536 }

AWS S3 list keys containing a string

I am using python in AWS Lambda function to list keys in a s3 bucket that contains a specific id

for object in mybucket.objects.all():
            file_name = os.path.basename(object.key)
            match_id = file_name.split('_', 1)[0]

The problem is if a s3 bucket has several thousand files the iteration is very inefficient and sometimes lambda function times out

Here is an example file name

i want to only iterate objects that contains “012345” in the key name
Any good suggestion on how i can accomplish that


Here is how you need to solve it.

S3 stores everything as objects and there is no folder or filename. It is all for user convenience.

aws s3 ls s3://bucket/folder1/folder2/filenamepart –recursive

will get all s3 objects name that matches to that name.

import boto3
s3 = boto3.resource('s3')
my_bucket = s3.Bucket('bucketname')
for obj in my_bucket.objects.filter(Prefix='012345'):

To speed up the list you can run multiple scripts parallelly.

Hope it helps.

Saltstack and Vault integration

First install and configure vault using this tutorial:

Use the latest version of vault.

Then install salt using the steps given here:

If you face any issues then refer these links:

Now let’s integrate vault and salt so that we can access vault secrets from inside salt state.

    1. First let’s add some key values into our vault.
      vault write secret/ssh/user1 password=”abc123″
      Then you can check it by reading: vault read secret/ssh/user1
    2. To allow salt to access your secrets you must firstly create a policy as follows:

      path "secret/*" {
        capabilities = ["read", "list"]
      path "auth/*" {
        capabilities = ["read", "list","sudo","create","update","delete"]

      You can also point to your secret like secret/ssh/*
      We have added auth/* so that our token can create other tokens.

    3. Then create a new policy with the following command:
      vault policy-write salt-policy salt-policy.hcl
    4. Then we will create a token from the new salt-policy
      vault token-create -policy=salt-policy
      Save the token created.
    5. Then in the salt-master create a file:
      /etc/salt/master.d/vault.conf with the follwoing contents:

          method: token
          token: xxxxxx48-xxxx-xxxx-xxxx-xxxx1xxxx<span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>c4a
            - salt-policy

      Then create a file /etc/salt/master.d//peer_run.conf

              - vault.generate_token

      Then restart the salt-master with service salt-master restart

    6. Then execute the following command to access the secret stored in vault:
      salt ‘*’ vault.read_secret “secret/ssh/user1”
    7. To access the secret from inside jinja:
      my-secret: {{ salt[‘vault’].read_secret(‘secret/ssh/user1’, ‘password’) }}
      {% set supersecret = salt[‘vault’].read_secret(‘secret/ssh/user1’) %}
          my_secret: {{ supersecret.password }}
    8. If you want to access the secret as pillar then add the following in salt master configuration:
       – vault: sdb_vault path=secret/ssh/user1
      Restart the salt-master and salt-minion
      Then access the data with the following command:
      salt ‘*’ pillar.get ‘password’
      Then refresh the pillar data with: salt ‘*’ saltutil.refresh_pillar
    9. If your vault policy is not configured correctly you might get an error as:
      ERROR: {‘error’: ‘Forbidden’}
      2017-09-21 06:51:39,320 [][ERROR ][26333] Failed to get token from master! An error was returned: Forbidden
      2017-09-21 06:51:39,350 [salt.pillar ][ERROR ][26333] Execption caught loading ext_pillar ‘vault’:
      File “/usr/lib/python2.7/site-packages/salt/pillar/”, line 822, in ext_pillar
      File “/usr/lib/python2.7/site-packages/salt/pillar/”, line 765, in _external_pillar_data
      File “/usr/lib/python2.7/site-packages/salt/pillar/”, line 91, in ext_pillar
      response = __utils__[‘vault.make_request’](‘GET’, url)
      File “/usr/lib/python2.7/site-packages/salt/utils/”, line 124, in make_request
      connection = _get_vault_connection()
      File “/usr/lib/python2.7/site-packages/salt/utils/”, line 113, in _get_vault_connection
      return _get_token_and_url_from_master()
      File “/usr/lib/python2.7/site-packages/salt/utils/”, line 89, in _get_token_and_url_from_master
      raise salt.exceptions.CommandExecutionError(result)2017-09-21 06:51:39,351 [salt.pillar ][CRITICAL][26333] Pillar render error: Failed to load ext_pillar vault: {‘error’: ‘Forbidden’}

      Make sure you have added auth/* in the policy.

    10. If you get the following error:
      Failed to get token from master! No result returned – is the peer publish configuration correct?
      ERROR: {}
      Then make sure you have peer_run.conf created and configured.
    11. You can also access your secret with command:
      salt-call sdb.get ‘sdb://vault/secret/ssh/user1?password’