Please review: The updation of the draft about hierarchy of PE in BGP/MPLS VPN

lidefeng <[email protected]> Tue, 27 May 2003 15:56:32 +0800
Newsgroups gmane.ietf.ppvpn
Message-ID <[email protected]>
Hi,all,

The latest version of draft about hierarchy of PE in BGP/MPLS
VPN(draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt) is attached,and the
updation from last version is as following:

   a. Add the interoperability statement of HoPE and [2547bis];
   b. Add the section to resolve the scalability of BGP/MPLS VPN in
      access the customer in the lower layer network.
   c. Correct some typo error.

The HoPE solution and the HoPE based solution can resolve the scalability
problem in access
the lower layer customer to BGP/MPLS VPN.

This draft is presented on the 56th IETF PPVPN WG meeting,which was not
carefully reviewed,I think,however,access to different layer customer is a
very important problem for Service Provider and customer,which can't be
resolved by normal 2547bis VPN.

Regards

Defeng Li
draft-libin-hierarchy-pe-bgp-mpls-vpn-02.txt (text/plain, 36.8 KB)
Network Working Group                                            Li Bin
Internet Draft                                               Dong Weisi
Expires: November 2003                              Huawei Technologies
                                                           
                                                           Chen Yunqing
                               China Telecom Beijing Research Institute
                               				   
                               				      Li Defeng		              
                               			    Huawei Technologies	      
                                                           MAY 27, 2003

            Hierarchy of Provider Edge Device in  BGP/MPLS VPN
                         
                <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>


Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

  Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

Abstract

   In the deployment of BGP/MPLS VPN,the PE(Provider Edge)Device should
   maintain all the VPN routes of the VPNs which it belong to.If there 
   are many VPNs attaches to a PE,and the capacity of PE is somewhat
   limited,then the bottleneck will be encountered.Another problem is 
   that the current BGP/MPLS VPN model is something of a "Plane Modle"
   where the demand of the performance of the PE device are all the 
   same no matter which layer the PE device is belongs to.However,the 
   typical network is "Core-Convergence-Access(Edge)" layered model,and
   the performance of the device is high in Core Layer and low in 
   Access Layer,and the scale of network is large in Access Layer and 
   small in Core Layer,the routes are converged in every layer,so in 
   current "Plane Modle",when PE device extend to the edge layer,it has 
   to maintain more VPN routes,which makes it difficult to extend the PE
   device to edge layer.This document defines an model of Hierarchy of 
   Provider Edge Device in BGP/MPLS VPN,where Hierarchy of Provider 
   Edge Device can be composed of several PE and each PE plays the 
   different part,partake the function of the normal RFC 2547 PE,we 
   call this model "Hierarchy Model",In this model,the demand of 
   performance in Routing and Switching is strict to the PE device 
   in higher layer,loose to the PE device in edge layer.
  
   One HoPE can be composed of an SPE and UPES connected with the SPE, 
   or be composed of a high-level SPE and HoPEs connected and build up 
   a new HoPE.This build is called nesting of HoPE,and this kind of 
   nesting can be done for many times.Thus the former HoPE connects 
   with the high-level SPE act as a role of UPE,and the new HoPE can 
   connect a single UPE too.



Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 1]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002

   
Table of Contents

   1. Introduction ...................................................3
   2. Working Principle ..............................................4
   2.1 VPN routes ....................................................4
   2.2 Control Flow(Route Advertising and Label Distribution).........5
   2.3 Data Flow(Label Operation and Packet Forwarding) ..............6
   3. Interface between UPE and SPE...................................7
   4. Nesting of HoPE.................................................7
   5. Multi-Homing UPE................................................9
   6. Backdoor link between UPEs......................................9
   7. UPE/CE routing protocol ........................................9
   7.1 Sham-link in HoPE..............................................9
   8. The Forwarding Procedure in Some Special Cases..................9
   9. Layered BGP/MPLS VPN with HoPE.................................10
   10. Interoperability..............................................11
   11. Security Consideration........................................11
   12. Updation from last version....................................11
   10. Acknowledge...................................................12
   11. References....................................................12
   12. Authors' Addresses............................................12
   Full Copyright Statement .........................................12

















































Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 2]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002


1. Introduction

   This document defines an model of Hierarchy of Provider Edge Device 
   in BGP/MPLS VPN,where Hierarchy of Provider Edge Device can be 
   composed of several PE device and every PE play the different 
   part,partake the function of the normal concentrative PE,we call 
   this model "Hierarchy Model",In this model the demand of 
   performance in Routing and Switching is strict to the PE device in 
   higher layer,loose to the PE device in edge layer.
   
   PE device can connect with not only the Customer Edge(CE) device,but 
   also a PE device,or even more generally an IP/MPLS VPN network,and 
   the connected PE devices formed the "Hierarchy of PE",and the PE 
   device which replace the former position of CE device in "Plane 
   Model" is called Under-layer PE(UPE),and the PE device which UPE is 
   connected with is called Superstratum PE(SPE),this architiecture is 
   called Hierarchy of PE(HoPE).and the architecture figure is as 
   follows(figure 1):
   
+----------+  +----+               +---+
|VPN1 Site1|--|    |---------------|   |
+----------+  |    | +----------+  |   | +-------+
+----------+  |UPE1| |VPN1 Site4|--|   | |       |  +--+  +----------+
|VPN2 Site1|--|    | +----------+  |   | |       |--|PE|--|VPN1 Site3|
+----------+  +----+ +----------+  |   |-| MPLS  |  +--+  +----------+
                     |VPN1 Site4|--|SPE| |NETWORK|
+----------+  +----+ +----------+  |   | |       |  +--+  +----------+
|VPN1 Site2|--|    |  +-------+    |   | |       |--|PE|--|VPN2 Site3|
+----------+  |    |--| MPLS  |----|   | |       |  +--+  +----------+
+----------+  |UPE2|  |NETWORK|    |   | +-------+
|VPN2 Site2|--|    |  +-------+    +---+ 
+----------+  +----+   
   
   	         figure 1:  Hierarchy of PE Architecture
   
   Several UPE and SPE formed the Hierarchy of PE,which provide the 
   conventional function of PE in "Plane Model"(rfc 2547),and their 
   respective functions are as follows:
   
   UPE maintains only the routes of the VPN sites which is directly 
   connected with UPE,it doesn't maintain the specific routes of the 
   remote VPN sites or only maintain the aggregate routes,SPE 
   maintains the routes of all the VPNs attached to the SPE or the UPEs 
   connected with the SPE.UPE distributes the inner MPLS labels for the 
   routes in the sites directly connected with it,and advertise the 
   labels concomitant with the VPN routes through MP-BGP to SPE ,SPE 
   doesn't advertise the routes in the remote VPN sites to UPE,it only 
   advertises the VRF default route or aggregate routes to UPE,and the 
   routes is concomitant with the MPLS labels.
   
   MP-IBGP or MP-EBGP can be applied between UPE and SPE,while MP-IBGP
   is applied,SPE should act as the Route Reflector(RR) for all the 
   UPEs collected to it,and UPE plays the role of Client of this RR,but 
   SPE doesn't act as the RR for other PEs. If MP-EBGP is applied 
   between MP-EBGP,the AS number of the UPEs should be the private AS
   number(64512~65535).In fact,the Hierarchy of PE can be handles with
   the rules of Confederation,in which case every confederation AS is 
   composed of only one BGP Speaker, that is the UPE collected with the 
   SPE.
   
   The packet forwarding between SPE and UPE is based on the labels,so
   only one interface is needed between them,this interface can be a 
   physical interface,or sub-interface,such as VLAN,PVC,tunnel such as 
   GRE or LSP.When tunnel is applied between UPE and SPE,an IP/MPLS 
   network can be deployed between them.


Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 3]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002


   
   Hierarchy of PE plays all the functions of the normal PE,
   there is no difference between them when looked outside,so this 
   "special" PE can coexist with other PEs in the MPLS network to build
   BGP/MPLS VPN.
   
2. Working Principle

   This section specifies the working principle of HoPE including the 
   maintaining and advertising of VPN routes,distribution of MPLS 
   labels,and packet forwarding.
   
2.1 VPN routes

   SPE can establish MP-BGP neighborship with UPE,if they are 
   administered by the same service provider(SP),they can be MP-IBGP
   neighborship,otherwise should establish the MP-EBGP neighorship.  
     
   In the MP-IBGP case,if there exists several UPEs collected with 
   an SPE belong to the same VPN,SPE acts as the Route Reflector
   for the connected UPEs,otherwise SPE act as the convergent PE for 
   all the UPEs collected with it.Route-Target lists are used to select 
   the right VPN routes from other PEs as the normal "Plane Modle" 
   BGP/MPLS VPN(RFC 2547bis),while only SPE will exchange the route 
   information with other PEs,UPE should send the import route 
   target list to SPE,SPE converges the route target lists and derives 
   the HoPE-wide import route target list,with this HoPE-wide import 
   route target,SPE can select the right VPN routes which belong to the 
   VPNs which have the sites attached to the SPE directly or through 
   UPE.
   
   This HoPE-wide import route target list can be configured manually
   or derived dynamically between SPE and UPEs.The dynamic mechanism is
   as follows:

   UPE advertises the ORF(Outbound Route Filter)[BGP-ORF] to SPE 
   through Route Refresh(RFC 2918) message,and an extended community 
   list is included in the ORF item,the content of the extended 
   community list is the aggregation of the import route target lists 
   of all the VRFs in the UPE,and SPE converges all the import route 
   target lists received from the UPEs connected with SPE and derive 
   the HoPE-wide import route target list.
   
   In MP-EBGP case,SPE should derive the HoPE-wide import route target
   list all the same with the mechanism as above.In general,UPE should
   adopt the private AS number in VPN routes advertised to SPE.When SPE
   advertised the routes to other PEs,it should omit the private AS.
 
   The scheme in which SPE connected with part of UPEs through MP-IBGP,
   and to the other part UPEs through MP-EBGP is permitted,then SPE 
   acts as RR and convergent PE at the same time.   
   
   UPE selects its own VPN routes by match its own import route target
   list with the export route target list attached with VPN routes
   respectively. 


Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 4]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002


   
   SPE advertises the VRF default route or aggregate routes(according 
   to which UPE transfers the VPN packet to SPE) to UPE,this default 
   VRF route can be formed dynamically or configured manually.When 
   formed dynamically,it can be filtered by the ORF mechanism mentioned
   above.
    
2.2 Control Flow(Route Advertising and Label Distribution)
  
   This section specifies the mechanism the network distributing the  
   labels for the routes in the VPN sites.The following figure
   (Figure 2) signifies the control flow between VPN1 site1 and VPN1 
   SITE2,the control flows(route and label distribution) in the two 
   directions are different,there are four steps in every direction,
   labeled (1) through (4) in the direction from VPN1 SITE2 to VPN1 
   site1,and labeled (5) through (8) in the direction from VPN1 site1 
   to VPN1 SITE2.
              
              |  (4)   |  (3)  |      (2)      | (1)  |
              |<-------|<------|<--------------|<-----|
              v        v       v               v      v
+----------+ +---+   +----+   +---+   +---+   +--+  +---+ +----------+		
|VPN1 Site1|-|CE1|---|UPE |---|SPE|---| P |---|PE|--|CE2|-|VPN1 SITE2|
+----------+ +---+   +----+   +---+   +---+   +--+  +---+ +----------+
              ^        ^       ^               ^      ^
              |  (5)   |  (6)  |     (7)       | (8 ) |
              |------->|------>|-------------->|----->|

        Figure 2: The control flow(route and label distribution)
  In Figure 2,the meanings of (1) through (8) are as follows:
  Label (1) through (4) specifies the procedure in the direction from
  VPN1 SITE2 to VPN1 Site1:
  
  (1) CE2 advertises a route in the VPN1 SITE2 to PE;
  
  (2) PE distributes an inner MPLS label for this route;PE advertises 
  this route concomitant with the inner label to SPE through MP-BGP ,
  and the relevant export route target list attached with the route;
  
  (3.a) SPE matches the export route target list in the received route 
  with the HoPE-wide import route target list to decide whether or not 
  import the VPN route,in this case,they can be matched,then SPE will 
  import the received VPN route.
  
  (3.b)At the same time SPE will match the 
  export route target list in the received route with the import route 
  target list advertised by VRF in UPEs connected with SPE,in this case,
  the import route target list of the VRF in UPE corresponding to VPN1 
  Site1(called VRF1) will match,then SPE will advertise the default 
  VRF1 route to UPE,with the inner MPLS label distributed by PE 
  attached;
    
  (4) UPE advertise this route to CE1 through the route protocol(RIP,
  OSPF,BGP,or default route) between CE1 and UPE.
     
  Label (5) through (8) specifies the procedure in the direction from
  VPN1 SITE2 to VPN1 Site1:
  
  (5) CE1 advertises a route in the VPN1 site1 to UPE; 


Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 5]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002


      
  (6) UPE distributes an inner label for this route;UPE advertises 
  this route to SPE through MP-BGP with the inner label;
  
  (7) SPE replaces the inner label distributed by UPE with another 
  inner label distributed by SPE,then advertises this route to PE 
  through MP-BGP with the new inner label attached;
  
  (8) PE distributes the route to CE2 with no label attached.
  
  Of course,the LSP should be established between SPE and PE in two 
  directions respectively,This mechanism is the same as RFC 2547bis.
  The procedure of forwarding VPN packet with the two labels detailed 
  in section 2.3.
  
2.3 Data Flow(Label Operation and Packet Forwarding)  

   This section specifies the mechanism of forwarding the VPN packets 
   in the network with the route and label information derived by 
   section 2.2. The following figure(Figure 3) signifies the data flows 
   between VPN1 site1 and VPN1 SITE2,the data flows(Label Operation and 
   Packet Forwarding) in the two directions are different,there are 
   five steps in every direction,labeled (1) through (5) in the 
   direction from VPN1 SITE2 to VPN1 site1,and labeled (6) through (10) 
   in the direction from VPN1 site1 to VPN1 SITE2.
                 
               |  (10)  |  (9)  |   (8)   |  (7) | (6)  |
               |<-------|<------|<--------|<-----|<-----|
               v        v       v         v      v      v
 +----------+ +---+   +----+  +---+   +---+   +--+  +---+ +----------+		
 |VPN1 Site1|-|CE1|---|UPE |--|SPE|---| P |---|PE|--|CE2|-|VPN1 SITE2|
 +----------+ +---+   +----+  +---+   +---+   +--+  +---+ +----------+
               ^        ^       ^        ^      ^      ^
               |  (1)   |  (2)  |  (3)   | (4)  | (5)  |
               |------->|------>|--------|----->|----->|
 
         Figure 3: Data Flow(Label Operation and Packet Forwarding)  

  In Figure 3,the meanings of (1) through (10) are as follows:
  Label (1) through (5) specifies the forwarding procedure in the 
  direction of VPN1 Site1 visit VPN1 SITE2:
  
  (1) When the VPN packet of VPN1 Site1 visit VPN1 SITE2 arrived to 
  CE1,CE1 forward the packet to UPE based on the default route or the 
  route derive from dynamic route protocol between CE1 and UPE
  specified in the (4) of section 2.2.
  
  (2) UPE push the inner label based on the default VRF route,forward
  the VPN packet to SPE,the inner label and the default VRF route are
  specified in the (3) of section 2.2.
  
  (3) SPE POP the inner label pushed by UPE specified in (2) above,and
  look up the VRF route,push the new inner label which distributed by 
  PE specified by (2) of section 2.2,then push the outer label 
  distributed by P and forward the VPN packet to P,in backbone network,
  all the P router in the path of LSP from SPE to PE swap the outer 
  label and forward the VPN packet through this LSP until the packet 
  arrived to PE,or optionally the P router pen-ultimate-hop pop the 
  label.


Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 6]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002


     
  (4) The P router pen-ultimate hop to PE POP the outer label,forward 
  the VPN packet to PE.
  
  (5) PE POPs the inner label,and forwards the VPN packet to CE2,then 
  CE2 forwards the VPN packet to the VPN1 SITE2. 
   
  Label (6) through (10) specifies the forwarding procedure in the 
  direction of VPN1 SITE2 visit VPN1 Site1:
  
  (6) When the VPN packet of VPN1 SITE2 visiting VPN1 Site1 arrives to 
  CE2, CE2 forwards the packet to PE based on the default route or the 
  route derived from dynamic route protocol between CE2 and PE 
  specified in the (8) of section 2.2.
  
  (7) PE PUSHes the inner label distributed by SPE specified in (7) of 
  section 2.2 based on the VRF route,then PUSHes the outer label 
  distributed by P and forwards the VPN packet to P,in backbone 
  network,all the P router in the path of LSP from PE to SPE swaps the 
  outer label and forwards the VPN packet through this LSP until the P 
  router pen-ultimate hop to SPE.
  
  (8) The P router optioanlly pen-ultimate hop to SPE POP the outer 
  label,forward the VPN packet to SPE.
  
  (9) SPE swaps the inner label in the VPN packet with the inner 
  label distributed by SPE with the new inner label distributed by UPE
  specified in (6) of section 2.2,then forward the VPN packet to UPE.
  
  (10) UPE POPs the new inner label,forwards the VPN packet to CE1,then 
  CE1 forwards the VPN packet to the VPN1 Site1.

3. Interface between UPE and SPE

   UPE can connect with SPE with any type of interface and 
   sub-interface,even with tunnel interface,in this case,UPE can 
   connect with SPE through an IP/MPLS network,and because SPE and UPE 
   are MP-BGP peers,the routes can be advertised directly through TCP 
   connection.In MP-EBGP case,UPE and SPE can set up EBGP peers across 
   the IP network or MPLS network by Multi-hop EBGP,when UPE or SPE 
   forwarding the VPN packets with the label,they must pass a tunnel,
   if the tunnel is GRE,MPLS encapsulation must be supported;If the 
   tunnel is LSP,the network between UPE ang SPE must be MPLS network,
   LDP/CR-LDP or RSVP-TE must be supported in UPE and SPE.

4. Nesting of HoPE

   One HoPE can be composed of a SPE and UPEs connected with the SPE, 
   or be composed of a high-level SPE and HoPEs connected with it and 
   build up a new HoPE.This build is called nesting of HoPE,and this 
   kind of nesting can be done for many times.Thus the former HoPE 
   connect with the high-level SPE acts as a role of UPE,and the 
   new HoPE can connect with a single UPE too.Figure 4 in the following 
   signifies a three-layer HoPE,and call the PE in the middle layer as 
   MPE(Middle-level PE).











Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 7]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002
   
 +----------+  +----+               +---+
 |VPN1 Site1|--|    |---------------|   |
 +----------+  |    | +----------+  |   | +-------+
 +----------+  |UPE1| |VPN1 Site4|--|   | |       |  +--+  +----------+
 |VPN2 Site1|--|    | +----------+  |   | |       |--|PE|--|VPN1 Site3|
 +----------+  +----+ +----------+  |   |-| MPLS  |  +--+  +----------+
                      |VPN1 Site4|--|SPE| |NETWORK|
 +----------+  +----+ +----------+  |   | |       |  +--+  +----------+
 |VPN1 Site2|--|    |  +-------+    |   | |       |--|PE|--|VPN2 Site3|
 +----------+  |    |--| MPLS  |----|   | |       |  +--+  +----------+
 +----------+  |UPE2|  |NETWORK|    |   | +-------+
 |VPN2 Site2|--|    |  +-------+    |   |
 +----------+  +----+               |   |
                                    |   |
  +----------+  +----+  +----+      |   |
  |VPN1 Site4|--|UPE3|--|MPE |------|   |
  +----------+  +----+  +----+      +---+                           
         (Nesting of HoPE)      
               	         figure 4:  Nesting of HoPE
   
   Between SPE and MPE,and between MPE and UPE run MP-BGP,if run 
   MP-IBGP,then SPE worked as the route reflector for all the MPEs,and
   MPEs worked as the route reflector for all the UPEs which connected 
   with it,and MP-BGP advertises all the VPN routes of the underlayer 
   PEs to the upperlayer PE,and advertise the default VRF route or the 
   aggregate route of the upperlayer to the underlayer PE. So SPE 
   maintains all the VPN routes of VPNs attached to the whole HoPE,and 
   the MPE maintains the VPN routes of UPEs connected with this MPE.UPE 
   maintains the VPN routes of sites connected with this UPE.
   
   SPE advertises the default VRF routes with the label attached to 
   MPE,MPE replaces this label with the new label,and advertises this 
   route with the new label attached to UPE.
   
   The upperlayer PE should create a HoPE-wide global import route 
   target list,filter the VPN routes that don't belong to the VPNs
   connected with this upperlayer PE.MPE converge all the import route 
   target lists of the UPEs connected with this MPE,and SPE converge 
   all the converged import route target lists of the MPEs connected 
   with this SPE.And this convergence can be configured statically or 
   derived dynamically,in the latter case,MPE should forward the import
   route target list of MPE to SPE through ORF.

   When the VPN packet from the local site of HoPE is forwarded to UPE,
   UPE look up the relevant default VRF route,push the label and 
   forward the packet to MPE,MPE POP the label,look up the default VRF 
   route or aggregate route to SPE,PUSH a new label forward the packet 
   to SPE,SPE POP this label,look up the VRF forwarding table,PUSH the 
   inner label,and the outer label the P router distribute for this 
   route,then follow the RFC 2547bis forwarding procedure.
   
   Because MPE has distributes the inner label for the destination 
   address of the VPN packet from the remote site,when the remote VPN
   packet is forwarded to SPE through the MPLS network following the 
   RFC 2547bis forwarding procedure. SPE swap the inner label,forward 
   this packet to MPE following the similar procedure,MPE swaps the 
   inner label and forwards this packet to UPE,then UPE POP the inner 
   label and forwards this packet to the local site.


Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN         [Page 8]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002

      
5. Multi-Homing UPE

   One UPE can connects with several SPEs,in this case UPE is called
   Multi-Homing UPE,all the SPEs advertise the default VRF routes to 
   this UPE,and UPE selects the best one or treat them as ECMP(Equal
   Cost Multi-Path) and share the load among them.UPE advertised the
   VPN routes to all the SPEs,it can advertise all the VPN routes to
   all the SPEs,or it can advertise one part of VPN routes to one SPE,
   the other part to the other and share the load among the SPEs.
   
6. Backdoor link between UPEs

   A kind of backdoor link can be setup between UPEs,so that the sites 
   connected with these two UPEs can communicate with each other 
   directly without passing through the SPE.And these UPEs can be those 
   connect to the same SPE,or can be those connect with the different 
   SPE,these UPEs advertise the VPN routes to each other through 
   MP-BGP,The control flow and data flow are the same with RFC 2547bis 
   Procedures,and even they can cross a ip/mpls network,and the packet 
   can pass through a tunnel such as GRE or LSP.
   
7. UPE/CE routing protocol
	   
   As specified in RFC 2547bis,RIP/OSPF/ISIS/EBGP can be routing 
   protocol between UPE/CE.If RIP/OSPF/ISIS runs between UPE and CEs,
   UPE should run each routing instance for every site.Especially,if 
   OSPF/ISIS runs between UPE/CE,the procedure following [VPN-OSPF] 
   except that the sham-link end-point is different.

7.1 Sham-link in HoPE 

   If OSPF/ISIS runs between UPE and CE,and the backdoor link exists 
   between a VPN site attached to HoPE and a VPN site attached to the 
   remote PE,and OSPF runs on the backdoor link,the above mentioned two
   sites belong to the same OSPF area,then Sham Link shoud be created 
   between the UPE in the HoPE and the remote PE.The site attached to 
   HoPE can access the routes in the remote sites without traverse the 
   SP backbone,if it can only get the default routes or aggregated 
   routes through UPE,the routes through the backdoor link is 
   preferred to the routes through UPE,and at most time it is not 
   desirable.To address the problem,you can select one of the 
   following two alternatives:
   
   --Aggregate the routes throught the backdoor at the same 
   granularity,and make the CE-UPE link to be preferred by configuring 
   the metric.
   
   --Remote PE distribute the routes in the remote sites to UPE through 
   sham-link,and configure the larger metric to the backdoor link,then 
   the CE-UPE link will be preferred.    

8. The Forwarding Procedure in Some Special Cases

   In one case that SPE connects with two UPEs called UPE1 and UPE2,and 
   these two UPEs connect with two sites called site1 and site2 
   respectively,and site1 and site2 belong to the same VPN and visit to 
   each other through SPE.The forwarding procedure is as follows:When 


Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN       [Page 9]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002


   the packet sent from site1 arrived to UPE,UPE pushes the label based 
   on the default route and forwards it to SPE,SPE POPs this label,then 
   looks up the VRF route table,PUSHes the label distributed by UPE2 
   and forwards it to UPE2,and UPE2 POPs the label,forwards it to 
   site2,and in the opposite direction,vice versa.
   
   In another case that SPE connects with one UPE and one CE,CE connect 
   with site1,UPE connect with site2,site1 and site2 belong to the same
   VPN.The forwarding procedure is as follows:
   
   (1)When the packet with no label sent from site1 arrives to SPE 
   through CE, SPE looks up VRF route table,and pushes the label 
   distributed by UPE,forwards it to UPE,UPE POPs this label,forwards 
   it to Site2.
   
   (2)When the packet originated from Site2 arrives to UPE,UPE forwards
   the packet to SPE on the basis of default or aggregate route,SPE 
   then looks up the VRF route table,POPs the label,forwards it to 
   Site1 as an IP packet.
    
9. Layered BGP/MPLS VPN with HoPE
   
   In [2547bis],all the PEs of the whole BGP/MPLS VPN is in the same 
   layer(though in the Route Reflector situation,the VPN structure is 
   somewhat layered,but it is hierarchy of RR and PE,not that of PEs),
   which is a plane structure.This VPN structure is difficult to attach
   to the VPN customer in the different layer of network,If a customer 
   would like to access the existing VPN with a large amount of VPN 
   routes,his CE router must be attached to the PE which can accomodate
   the whole VPN routes,if this customer is situated in the lowest 
   layer of network,where is faraway from the nearest PE,then maybe the 
   only way to attach this customer to PE is laying the long leased 
   line between the CE and PE.If there are many different customer who 
   want to access this VPN,then such long leased line to connect the 
   CEs and the PEs,not to speak the expensive cost,these long leased 
   lines make the network structure more complicated and scalability 
   difficult.
    
   In investigation of the BGP/MPLS VPN deployment situation,the desire 
   of the lower layer network customer to access in the BGP/MPLS VPN is 
   universal,such as subbranch offices of the different Government 
   department of nationwide corporation.To address this problem,it is 
   desirable to organize the whole VPN with the different sub-VPN at 
   different domain or different layer,the PEs in the different 
   sub-VPN only maintain the VPN routes of that sub-VPN,with the 
   exception in the PE connects to the higher layer sub-VPN,by which PE 
   the lower layer sub-VPN connect to the higher layer sub-VPN with the 
   aggregated routes and advertise the default VRF routes to the higher 
   layer sub-VPN,and the higher layer sub-VPN gets to the lower layer 
   sub-VPN with such default VRF routes.Then the different sub-VPN can 
   attach the customer at different network layer to the nearest PE 
   which only needs to maintain the VPN routes of that sub-VPN,and only 
   one of such PEs connect to the higher layer sub-VPN with aggregated 
   routes,and this particular PE takes on the part of Route Reflector 
   for the whole PEs in the sub-VPN it belongs to.This can be 
   illustrated as following figure(Figure 5).



Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN       [Page 10]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002
   
                       ____     ___       ____          
                     _/    \___/   \    _/    \__       
                    /               \__/         \_     
                   /                               \    
                  /  BGP/MPLS VPN (core Network)    |
                  \        suv-VPN                 /
                   \   ___      ___     __      _/                    
                    \_/   \____/   \___/  \____/
                    |PE|                  |PE|
                    |__|                  |__|
        ________     \                      \      ________
      _/        \__  _\_                     \____/        \__    
     /suv-VPN      \_|PE|(RR for sub-VPN)     |PE| suv-VPN    \____  
    /                |__|                     |__|                 \
   /BGP/MPLS VPN      \\                     ^||/(Convergence Layer)| 
   \(Convergence Layer)|\      Aggregated    |||\ BGP/MPLS VPN   __/  
    \   ___       ____/  \      routes       ||| \   ___       _/   
     \_/   \____/         \                  |||  \_/   \____/      
                           \__               |||Default VRF route
                   ________|PE|             __|V   ______         
                 _/        |__|             |PE| _/      \__       
                /             \__           |__|/           \    
               / suv-VPN          \         /   suv-VPN      \  
              /(Access Layer)     |        /(Access Layer)    |  
              \ BGP/MPLS VPN   __/         \ BGP/MPLS VPN   __/   
               \   ___       _/             \   ___       _/      
                \_/   \____/                 \_/   \____/         
                                                              
	  Figure 5. Layered BGP/MPLS VPN with nesting of HoPE
	  			
   The requirement of performance and capacity for the PEs belong to 
   the core network sub-VPN can be different from that for the  PEs 
   belong to the convergence layer sub-VPN and that for the PEs belong 
   to the access layer sub-VPN,so this solution can address the 
   bottleneck of scalability of BGP/MPLS VPN in the lower layer 
   network,and this problem is not touched at all in [2547BIS].
 
   The work principle of control plane and data plane between sub-VPNs 
   follows the procedure detailed in section 2 through section 8,with 
   the nesting of HoPE involved.
  
10. Interoperability
    
   This solution doesn't introduce any backward compatibility problem 
   with [2547BIS],the only difference in the label processing in SPE
   is that when it receive the labeled packet from UPE or MPE(in 
   nesting of HoPE situation),SPE will swap the outer label with the 
   label the remote PE advertised to it,so the solution can operate
   with [2547BIS] without any difficulty.You can deploy some PEs of 
   the whole BGP/MPLS VPN with HoPE,and the other PEs with [2547BIS].
   This deployment is verified as efficiently in the Service Provider.
    
11. Security Consideration
    
   The level of security provided by this architecture is identical to 
   that provided by the RFC 2547bis,there is no security problem 
   introduced by HoPE.

12. Updation from last version
   
   a. Add the interoperability statement of HoPE and [2547bis];
   b. Add the section to resolve the scalability of BGP/MPLS VPN in 
      access the customer in the lower layer network.
   c. Correct some typo error.



Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN       [Page 11]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002
   

13. Acknowledgements

   The authors would like to thank Li Hejun, Cao Xuegui,Chang Wenjun, 
   We are very appreciated  for their support

14. References

   [RFC2026]   Bradner, S., "The Internet Standards Process--Revision 
               3", BCP 9, RFC 2026, October 1996.
   [BGP-ORF]  Enke Chen,Yakov Rekhter,"Cooperative Route Filtering 
              Capability for BGP-4",draft-ietf-idr-route-filter-06.txt.
   [BGP-RR] Chen, E., "Route Refresh Capability for BGP-4", RFC2918,
   September 2000
   [RFC2547]   E. Rosen, Y. Rekhter, BGP/MPLS VPNs, RFC 2547,March 
               1999.  
   [2547bis]   Rosen, E., Rekhter, Y. et al., "BGP/MPLS VPNs", work in 
               progress. 
   [VPN-OSPF] Rosen, Psenak and Pillay-Esnault, "OSPF as the PE/CE
   Protocol in BGP/MPLS VPNs", February 2003, draft-rosen-vpns-ospf-bgp-
   mpls-06.txt 

15. Author's Address

    Li Bin  
    D201 ,HuaWei Bld. No3 Xinxi Rd.
    Shang-Di Information Industry Base,
    Hai-Dian District BeiJing P.R.China
    Zip : 100085
    Email : [email protected]
    
    Dong Weisi  
    C401 ,HuaWei Bld. No.3 Xinxi Rd.
    Shang-Di Information Industry Base,
    Hai-Dian District BeiJing P.R.China
    Zip : 100085
    Email : [email protected]
    
    Chen Yunqing
    China Telecom Beijing Research Institute
    No.52,Hua Yuan North Road,
    Haidian District,Beijing,100083,China
    Tel:+86-10-62304554
    Fax:+86-10-62304157
    E-mail:[email protected]
    
    Li Defeng
    D201 ,HuaWei Bld. No.3 Xinxi Rd.
    Shang-Di Information Industry Base,
    Hai-Dian District BeiJing P.R.China
    Zip : 100085
    Email : [email protected]
    

Full Copyright Statement

   Copyright (C) The Internet Society (2001).  All Rights Reserved.



Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN       [Page 12]

Draft     <draft-libin-Hierarchy-pe-bgp-mpls-vpn-02.txt>  November 2002
   

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


















Libin, et al.  Hierarchy of PE Device in  BGP/MPLS VPN      [Page 13]