Parse yaml list and save it to variables in python

dejanmarich

I have yaml file, which I would like to iterate and save everything from the list as a new variable with the same name.

- app1:
   name: example1
   host: example1
   run: myscript1.sh
   tags: 
    - "user"  
    - "age"  
    - "gender"             
 
- app2:
   name: example2
   host: example4
   tags: 
    - "user"  
    - "age"  
    - "height"   

Code:

for i in range(0, len(data)):
    for key, value in data[i].items():
        print(value.get('tags'))
        for n in tags:
            print(n)

output:

['user', 'age', 'gender']
user
age
user
['user', 'age', 'height']
user
age
height

Would like in new iteration to save these values as variables,

user = 'user'
age = 'age'
height = 'height'

and in next iteration:

  user = 'user'
  age = 'age'
  gender = 'gender'

So I can pass these variables to some other stuff in the code.

EDIT

yaml variable (e.g. host) will be populated with the values from Elasticsearch. If we define in tags "age", I'll search Elasticsearch for "age" and grab value from document, so it becomes "age": "20". After that, this "age" tag will be called in sh script, something like:

for doc1 in resp['hits']['hits']:
            command = f"{value.get('run')} --age {age} #and other tags"
            p = subprocess.Popen(command.split(), shell=False, stdout=subprocess.PIPE)
2e0byo

I'm not quite certain what you want, but I think you're after something like this:

# test.yml
- app1:
   name: example1
   host: example1
   tags: 
    - "user"  
    - "age"  
    - "gender"             
 
- app2:
   name: example2
   host: example4
   tags: 
    - "user"  
    - "age"  
    - "height"   
import yaml
from pathlib import Path

with Path("test.yml").open() as f:
    data = yaml.load(f)

out = []

for entry in data:
    app, *_ = entry.keys()
    tags = {k: k for k in entry[app]["tags"]}
    out.append(tags)

print(out)

You could easily save these by the app name instead, by making out a dict.

Note that I don't know why you are after redundant key/value pairs, and if tags is actually supposed to contain data (unlike in your example) you will have to do something like:

tags = {k:v for k, v in entry[app]["tags"].items()}

Lastly you can of course dump back to yaml with yaml.dump(out).

** Unpacking

It occurs to me you might want to unpack this dict into a function. You can do so like this:

def my_fn(user=None, age=None, gender=None, height=None):
    print("user", user, "age", age, "gender", gender, "height", height)

out = [{'user': 'user', 'age': 'age', 'gender': 'gender'},
 {'user': 'user', 'age': 'age', 'height': 'height'}]

for row in out:
    my_fn(**row)

Thus ** unpacks the dict into keyword arguments, which might be what you were looking for. (At any rate it makes more sense to me than generating variables with programmatic names). Note the app, *_ which unpacked entry.keys() so we could get the first item without having to cast it to an indexable type. dict.keys() isn't a generator, so you can't use next(dict.keys()). This particular unpacking is a recent (and very useful) addition to python.

EDIT: using these variables

yaml variable (e.g. host) will be populated with the values from Elasticsearch. If we define in tags "age", I'll search Elasticsearch for "age" and grab value from document, so it becomes "age": "20".

Right. So you want to see what entries are in tags, and then populate them with the right values. So you want to do something like this:

for entry in data:
    tags = {}
    app, *_ = entry.keys()
    for tag in in entry[app]["tags"]:
        val = get_tag_value_from_elasticsearch(tag)
        tags[tag] = val

After that, this "age" tag will be called in sh script

So---still in the same loop for simplicity, but you can always append tags to a list and then iterate that list later:

    cmd = ["command"]
    for param, val in tags.items():
        cmd += [f"--{param}", str(val)]
    res = subprocess.run(cmd, capture_output=True)
    print(res.stdout)

Note a few things here: I've avoided the use of .get() to lookup a value in a dict when I can just use [] access; I've used subprocess.run instead of Popen (if you want to background you'll have to go back to Popen), and I've build the command up directly in a list, rather than building a string and splitting it.

At this point anything else you want to do with it is I think another question, so I'm going to leave this answer here.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related