Introduction

Couple words on how it all works.

There are two main parts in this project: YANG model navigator and XML parser. Model navigator helps to create the XPath which is then runs against the selected configurations using XML parser.

YANG model navigator

As we all hoped, to simplify human-to-router communication YANG models initially supposed to be universal. Well, as we can see now — not really. Each vendor has its own data model structure, OpenConfig models cover only small part of the feature set, so as a result — we have yet another SNMP MIBs.

Ok, not SNMP, of course, but still, it's quite disappointing.

Here I've focused on the Nokia YANG models. To visualize YANG model in any convenient way, first it has to be translated into a hierarchy of Python objects. There are several libraries to do this job, but robust and full convertion of Nokia models I could make only with Yangson. Yangson requires to build an inventory file first. It's not that trivial for a large models, so I've automated this task with Yangson inventory builder

To get root container "configuration" as a list of objects:

yangson_data_model: DataModel = DataModel.from_file(name=inventory_request.target_inventory_file_name.__str__(), mod_path=(inventory_request.yang_directory.__str__(),))
configuration_container: list = yangson_data_model.schema.children[0].children

Next we need to store this hierarchical data in the DB. I've used MPTT Django app for that. Each YANG node is processed recursively, but it all starts with the root container — "configuration".

import yangson.schemanode as yangson_node
from mptt.models import MPTTModel, TreeForeignKey

class YangNode(MPTTModel):

    name = models.CharField(max_length=50, unique=False, verbose_name='YANG node name')
    type = models.CharField(max_length=50, unique=False, verbose_name='YANG node type')
    description = models.CharField(max_length=50, unique=False, verbose_name='YANG node description', null=True)
    model_name = models.CharField(max_length=50, unique=False, verbose_name='Name of the YANG Model node belongs to')
    parent = TreeForeignKey('self', on_delete=models.CASCADE, null=True, blank=True, related_name='children', db_index=True, verbose_name='Parent of the YANG node')

    class MPTTMeta:
        order_insertion_by = ['name']

    class Meta:
        db_table = 'yang_models'

    def __str__(self):
        return self.name

@dataclass()
class Ynode:
    yang_son_node: yangson_node
    parent_node: YangNode | None
    model_name: str

Each YANG node is processed recursively, but it all starts with the root container — "configuration".

for config_item in configuration_container:
    child_node = Ynode(
        yang_son_node=config_item,
        parent_node=None,
        model_name=model_name
    )
    process_node(child_node)

def process_node(yang_node: Ynode):
    match yang_node.yang_son_node:
        case yangson_node.ContainerNode():
            process_container_node(yang_node)
        case yangson_node.LeafListNode():
            process_leaf_list_node(yang_node)
        case yangson_node.ListNode():
            proces_list_node(yang_node)
        case yangson_node.LeafNode():
            process_leaf_node(yang_node)
        case yangson_node.ChoiceNode():
            process_choice_node(yang_node)
        case yangson_node.CaseNode():
            process_case_node(yang_node)
        case _:
            logger.error(f'Unsupported node type: {yang_node.yang_son_node.name}')

For example, LEAF node:

def process_leaf_node(yang_node: Ynode):
    YangNode.objects.create(
        name=yang_node.yang_son_node.name,
        type=YnodeType.LEAF.value,
        description=yang_node.yang_son_node.description,
        model_name=yang_node.model_name,
        parent=yang_node.parent_node
    )

XML Parser

To make XQueries user-friendly, there is a Xpath constructor, which allows to build a Xpath using GUI. Under the hood Xpath request is transformed into JSON, and XML parser receives something like:

{"device_list": ["r2.xml", "r1.xml"],
 "xpath": [{"name": "card"},
           {"name": "mda",
                "filters": [
                            {"filter_path": "mda-slot", "regexp": ""},
                            {"filter_path": "mda-type", "regexp": "10g"}
                ]
            }]}

Which is then translated into standard XPath and the query is executed using lxml library:

configuration/card/mda[re:match(mda-slot, ".*")][re:match(mda-type, "10g")]

XML Parser project is stored here.