Ad

Merge YAML Files With Overriding Values In List Elements

- 1 answer

I would like to merge two YAML files that contain list elements. (A) and (B) merged into a new file (C).

I would like to override existing attribute values of the list entries in (A) if they are also defined in (B).

I would like to add new attributes to list entries if they are not defined in (A) but defined in (B).

I would also like to add new list entries of (B) as well if not present in (A).

YAML file A:

list:
  - id: 1
    name: "name-from-A"
  - id: 2
    name: "name-from-A"

YAML file B:

list:
  - id: 1
    name: "name-from-B"
  - id: 2
    title: "title-from-B"
  - id: 3
    name: "name-from-B"
    title: "title-from-B"

The merged YAML file (C), I would like to produce:

list:
  - id: 1
    name: "name-from-B"
  - id: 2
    name: "name-from-A"
    title: "title-from-B"
  - id: 3
    name: "name-from-B"
    title: "title-from-B"

I need this functionality in a Bash script but I can require Python in the environment.

Is there any standalone YAML processor (like yq) that can do this?

How would I implement something like this in a Python script?

Ad

Answer

You can use ruamel.yaml python package to do it.

if you have python already installed, run following command in terminal :

pip install ruamel.yaml

python code adapted from here. (tested, and works fine) :

import ruamel.yaml
yaml = ruamel.yaml.YAML()

#Load the yaml files
with open('/test1.yaml') as fp:
    data = yaml.load(fp)
with open('/test2.yaml') as fp:
    data1 = yaml.load(fp)
# dict to contain merged ids
merged = dict()

#Add the 'list' from test1.yaml to test2.yaml 'list'
for i in data1['list']:
    for j in data['list']:
        # if same 'id'
        if i['id'] == j['id']:
            i.update(j)
            merged[i['id']] = True

# add new ids if there is some
for j in data['list']:
    if not merged.get(j['id'], False):
        data1['list'].append(j)

#create a new file with merged yaml
with open('/merged.yaml', 'w') as yaml_file:
    yaml.dump(data1, yaml_file)
Ad
source: stackoverflow.com
Ad