[CCAMP][TEAS][NETMOD] Efficiency issues with YANG model data in integration

Hello Friends,

I have a question about YANG data model efficiency issues discovered during integration with OSS.
This issue is also mentioned in section2.2 of draft-yg3bp-ccamp-network-inventory-yang that we are planning to present at the CCAMP WG during IETF 113.

YANG data models define schema trees such as:

module: example
   +--rw root
      +--rw leaf-a?   string
      +--rw list-b* [leaf-c]
      |  +--rw leaf-c    string
      |  +--rw list-d* [leaf-e]
      |     +--rw leaf-e    string
      |     +--rw list-f* [leaf-g]
      |        +--rw leaf-g    string
      |        +--rw leaf-h?   string
      +--rw list-i* [leaf-j]
         +--rw leaf-j    string
         +--rw leaf-k?   string

From the point of view of data model definition, it is convenient to define objects and their relationship using a tree structure.

However, from a practical perspective, relational databases, in which these objects are saved in different tables, are widely used by network controllers and OSS/BSS. In this type of implementations, the model data need to be cut into several pieces of small object data for management and this is causing some efficiency issues with the integration between systems.

For example, in full data synchronization scenario, the YANG server needs to assemble the data of each small object into a tree structure based on the YANG model defined for the communication with its YANG client(s). The YANG client then disassembles the data into its own storage structure. This assembly-disassembly process is actually time consuming, especially when the storage structures between the upper and lower system are consistent, they don’t need this assembly-disassembly process at all. This efficiency issue is more prominent in large scale networks.

RESTCONF can also support querying small objects. For example, in the above YANG model, the YANG server can provide the list of list-f instances ({{restconf}}/data/example:root/list-b={leaf-c}/list-d={leaf-e}/list-f) to a YANG client, with no need to perform any assembly-disassembly process.

However, since list-f is a child list under list-d and list-b, if you want to get access the list-f list, the list-b’s identifier (leaf-c) and the list-d’s identifier (leaf-e) shall be specified in the URI. Therefore, in the full data synchronization scenario, a loop is required to get all the elements of list-f. If there are 100 list-b instances and there’re 100 list-d instances under each list-b instance, then 100*100 requests are needed to get all of the list-f instances.
Assume that there are only 10 instances by average in each list-d instance, there are totally 100,000 list-f instances. If, instead, it would possible to retrieve all the list-f instances without specifying the list-f’s parent and grandparent’s identifier, but using RESTFUL’s pagination, each RESTFUL request, can provide far more than 10 list-f instances, regardless of whether they belong to a same list-d or list-b instance or not. If RESTFUL interface can return 100 list-f instances per request, only 1000 requests are required, one tenth of RESTCONF interface. This approach will reduce the time taken by the integration.

So my question is whether, to address this efficiency issue, the only viable option is to define the YANG model with a more flat structure, as shown below:

module: example
   +--rw root
      +--rw root-summay-information
      |  +--rw leaf-a?             string
      |  +--rw contained-list-b*   string
      |  +--rw contianed-list-i*   string
      +--rw list-b* [leaf-c]
      |  +--rw leaf-c              string
      |  +--rw contained-list-d*   string
      +--rw list-d* [leaf-c leaf-e]
      |  +--rw leaf-c              leafref  (path "/exmp:root/exmp:list-b/exmp:leaf-c")
      |  +--rw leaf-e              string
      |  +--rw contained-list-f*   string
      +--rw list-f* [leaf-c leaf-e leaf-g]
      |  +--rw leaf-c    leafref  (path "/exmp:root/exmp:list-b/exmp:leaf-c")
      |  +--rw leaf-e    leafref  (path "/exmp:root/exmp:list-d/exmp:leaf-e")
      |  +--rw leaf-g    string
      |  +--rw leaf-h?   string
      +--rw list-i* [leaf-j]
         +--rw leaf-j    string
         +--rw leaf-k?   string

The leaf-list object contained-list-* is a list of identifier reference to list-* instance. With this kind of modeling, all the list-f instances in the whole network can be retrieved without the need to specify its parent’s and grandparent’s identifiers in URI.

We would appreciate receiving your comments and suggestions on whether there are other ways to resolve these integration efficiency issues.

Thank you!

B.R.
Chaode

玉朝德  Tel: +86-15002728675
华为技术有限公司
[Huawei]