Comware

Hi all

I'm having some issues with policy based routing on comware 7 switches.

I've just gone through hpe guide and I was hoping that someone with more experience could confirm this for me

If I get this right the below statements will achieve the same results

policy-based-route test permit node 10
if-match acl 3235

# traffic matching acl 3235 will be forwarded according to the routing table 

VS

policy-based-route test permit deny 10
if-match acl 3235

#as above, send traffic matching the acl according to the routing table 

Hi @pattap !

Yes, that is correct, I'd also expect same result, but for different reasons.

Case 1:

policy-based-route test permit node 10
 if-match acl 3235

PBR policy node matches traffic according ACL 3235, it is ready to apply action like 'apply next-hop' (since the node's match mode is 'permit') but the 'apply' clause is not configured, therefore the matched traffic will be routed according to the routing table because PBR simply doesn't know what to do with the traffic.

Case 2:

policy-based-route test deny node 10
 if-match acl 3235

 PBR policy node 10 match mode is 'deny' which means it matches traffic according ACL 3235 and routes it according to the routing table.

Hope this helps!

ok so we got that confirmed, thanks 

I'm experiencing strange behaviour. It appears that despite my PBR configured and applied to vlan interface traffic is being forwarded based on the routing table 

The below is the config of the policy, it also have been applied to my vlan interface. Next hop is an IP address of GRE tunnel in front of my internet firewall which acts as default gateway. GRE terminates on HPE5900 in front of the firewall, the switch has a default route pointing to the FW.

The problem is that the traffic is not hitting the firewall at all, instead it hits another internet firewall with no GRE configured in front of it. This indicates that the traffic scoped by acl 3236 is being routed based on routing table rather than controlled by apply  next hot action within the policy.

policy-based-route WIFI permit node 20
if-match acl 3236
apply next-hop <ip address>

I'm losing my head over this. Any ideas will be much appreciated.

EDIT: also just realised there's no any hits on the switch with PBR configured on, it's hpe5710

acl number 3236 name Allow_WIFI_via_GRE
rule 10 permit ip source x.x.x.x x.x.x.x logging counting

I've just ran a debug ip packet acl 3236

got this result

*Mar 17 09:54:20:035 2021 C3_L3 ACL/7/Match: No match for source address in advanced rule 10.
*Mar 17 09:54:19:114 2021 C3_L3 ACL/7/Match: -Slot=2; No match for source address in advanced rule 10.
*Mar 17 09:54:20:103 2021 C3_L3 ACL/7/Match: No match for source address in advanced rule 10.
*Mar 17 09:54:20:103 2021 C3_L3 ACL/7/Match: No match for source address in advanced rule 10.

Are you sure the wildcard mask in your ACL is correct?

yes it's /22 net and wild card in the acl is 0.0.3.255

edit: not sure if it's a bug on the switch but upgrading it to R2702P01 from R2702, can't think of anything else at that stage 

Hmm, that's weird.. Configuration-wise you need only correct ACL, proper PBR node that matches ACL and sets next-hop and then you just apply the PBR to the Vlan-interface... So you say the PBR doesn't work on two different switches - 5900 and 5710? If so, I'd say it's configuration issue, because chance you are hitting one bug on two different platforms (different by chipsets and software) has very low probability...

BTW, could you do one test:
- Login to the switch and enable logging to terminal with 'terminal monitor' command
- Enter the Vlan-interface where you applied the PBR policy and remove it using 'undo ip policy-based-route' command
- Re-apply the PBR policy. Look for any errors. If any errors, post them here.

If no errors, please, send me the output from 'display ip policy-based-route interface' command as well as 'display acl <acl_used_in_PBR>' You can obfuscate IP addresses if you use real public IPs outside RFC1918 private ranges.

As for the upgrade of 5710 from 2702 to 2702P01 - I couldn't find any bugfix for anything related to PBR in the P01, so the chance the upgrade will fix it is not so high.

Hi Ivan

I wasn't clear on the two switches, what I meant was that the GRE is between 5710 and 5900, policy is on 5710.

We have PBR configured in the same way for the same purpose on other models, 5510's and 5800's and they all have their  GRE established with the same 5900 switch.

Tried what you said, seen no logs during re-applying the policy

Also what is very strange, I have generated some traffic from witin net sepcified in acl 3236 from another switch connected to this 5710 and I'm not seeing it in deb ip pack acl 3236 - yes ter monitor and ter deb are on, vlan is tagged across the uplink

Sorry if my questions will look stupid, redundant and unneeded, but I do not have full 'display diag' from the switch, so let's verify a couple of very basic things that are currently unclear to me:

1. Is the next-hop IP address set by the 'apply next-hop' reachable from the routing table perspective? If the next-hop is not reachable, PBR won't work.
2. You don't use VRFs (vpn-instances), do you? If Vlan-interface224 is in separate vpn-instance or GRE tunnel is inside vpn-instance, that may require modification of ACL 3236 as well as 'apply next-hop' clause.
3. Are you sure that traffic from the network matched by the ACL 3236 comes on Vlan-interface224 of the switch? Of course this question makes sense if that rule matches different subnet, not the one of the Vlan-interface224 itself.
4. If that rule 10 matches the same subnet which Vlan-interface224 has, please, verify that your hosts use Vlan-interface224 IP address as default gateway, otherwise traffic won't be hitting the switch and it may explain why PBR doesn't work.
5. 'Matched' counter of the policy shows zero, so may be switch doesn't "like" the ACL 3236 due to certain, currently unknown reasons. Could you change it to match one single test IP (wildcard mask 0.0.0.0) and don't use any additional options like 'logging' and 'counting'?
6. Since you've mentioned you used debugging, let's try to debug not only ACL, but the PBR module itself:

terminal monitor
terminal debug
debugging ip policy-based-route

Check what messages you will get, maybe that output will point us to the right direction.

Hi Ivan 

No question is stupid and thanks for looking at this for me.

1. Yes next hop is applied by "apply next-hop" command. It is also reachable from the switch. It appears as direct route:

sorry about heavy redacting, . x.x.52.72/30 is the range for the GRE in question and 52.74 address is remote end of the tunnel.

2. No 

3. Yes, end devices are in vlan 224, this vlan is stretched across  to L2 switches, vlan tagged across the uplinks.

4. Yes devices in vlan 224 use int vlan 224 address as their default gateway

5 and 6 I need to be on site to generate traffic in that vlan, will do that later on today. I'll edit this response once I have the answers

A bit late but here are the results of points 5 and 6

5. I changed it to a single IP address and made sure it was the test device in vlan 224 I was using. I saw the ACL being matched based on the debug but traffic still wasn't sent via tunnel. I'm expecting traffic to go via tun24

6. with "IP policy based routing debugging switch is on" not a single entry popped in the terminal 

Also just on a side note. I just realised today that there is exactly same setup on one of other sites I have visited today also with 5710 as L3 and this is not working there either. 

Hmmm., to be honest I am starting to suspect some limitations int 5710 concerning next-hops over tunnel interfaces.

PBR and GRE tunnels in switches work concurrently and in some platfors there is such order of operations that if one module intercepts the packet (PBR for example), then another module (GRE for example) can't process the packet. For example in 5900 switches there was a requirement to reserve ASIC resources of one port for GRE tunnels and it was creating an internal loopback in the ASIC so when PBR finished processing the packet it was "sending" packet over that reserved interface and since there was a loopback configured there, the packet was entering the processing queue again, this time for GRE encapsulation. Some platforms do not require such trick, but I do not exclude the possibility that lack of such requirement to have ASIC loopback for GRE tunnel may lead to inability of certain modules to work together. BTW, the predecessor of your 5710, 5700 was exactly from such switches and it had certain limitations for GRE and even for NLB in multicast mode. 5710 is very different from 5700, but still it might have similar limitations...

Unfortunately I do not have that model switch to test, but I suggest you to do the following:

1. Test with next-hops over normal Vlan-interface
2. Open a support case and tell the support engineer depending on Step 1 result that either PBR doesn't work with next-hop over GRE tunnels or it doesn't work at all.

Configuration-wise I don't see any errors so it must be either a bug or limitation that is not described in the documents available to us.

I have just tried creating a dedicated int lo0 specifically for this tunnel, still no results. 

A question though. I have used one gre in two PBR's on that switch. One of them works as expected. Could this be somehow causing issues? Case with HPE opened

That service loopback is very different from 'interface Loopback', nothing in common really. If one time you will end up configuring 5900 and GRE tunnels there, check https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=a00037018en_us , page page 290. But again, it's not required and not available for 5710, that was just a guess trying to imagine all possible root causes for the issue.

I've never heard about such limitation that you can't use same GRE tunnel as next-hop outgoing interface in different PBR policies, so it's either a bug or is undocumented limitation. So let's see what Support will say.

ah ok I've misunderstood what you said about the loopback.

Ive read the part of the document you have mentioned. I'm not sure how related is this to my issues but I was just wondering what is the benefit of having multiple physical interfaces assigned to a single service loopback. Document states that the members of service loopback group are load balanced.

Just to emphasize again - it is not applicable to 5710. I want to be sure you understand that mentioning 5900 and service-loopback was just an analogy I made trying to imagine what could cause GRE and PBR to not play well. 

As for the 5900s and service-loopbacks there, to be honest I've never seen more than one reserved port per IRF slot. Of course if amount of looped traffic is more than reserved port's capacity, then probably there is a reason to reserve more ports, like when you dedicate 1G port for service-loopback group, but amount of traffic in tunnels and multiport flows is 2-3G. But I've never seen this "in the wild", so have no real experience with such setups.