Comware

This will be long-winded. Our network started before the internet was incorporated, and so the internal ip structure didn't matter. The people at that time chose 199.199.199.x/24 as the structure. As time passed and more devices came into play, they changed to 199.199.x.x/16.
We got a decent router several years ago, though the same layer two switches. With internet router in place, it became possible for me to try to change the network structure to a private ip range, 172.29.x.x/16. I have slowly been switching, but most the network remains on 199.199.x.x.
Late last year we were able to upgrade our network, so now I have an 8212zl as a core switch with a few 2910al switches as distribution switches. It had to be a quick switch, so I pretty much retained the default setup of the 8212zl default_vlan 1 and just plugged all my devices in. So now with the new switches I have a mix of 199.199.x.x and 172.29.x.x on my network, still routing through my internet router (sonicwall 4060).
Now I need to invoke IP Routing on my 8212 switch. If you were me, knowing you could not take the network down for long as we are a hospital, how would you proceed? Some docs I've read indicate the two networks should be in separate vlans, which would mean going back and figuring out each device on each port. I could do that, but am hoping for something simplier.
The end result I am trying to acheive is to have my internal network IP routing happen on my switches, mostly the 8212, and not in my router.
What would the people here suggest?
Thank you for your help
Ron

Yes, best practice is to separate ip-subnets using vlan's and connect those vlan's using a router.

but i understand at this time you allready have two subnets on the same lan ?
this is called a "multinet".

No worries, but try to keep it temporary as it does not match current network desing.
you allready are migrating away from the 199.199.x.x., so it dies eventually.

you can configure the vlan-interface of your switch with a primary and secondary address, and enable routing.
the switch will route between the two subnets on the same interface.

but as you don't mention dividing the 172.29.x.x subnet in smaller net's, what routing do you really need?
do you eventually want each 2910-switch to have it's own subnet?

Thank you Pieter. I really don't envision having the 2910's on their own subnets, I'm really only figuring on just a single subnet throughout. We are a small facility, though just large enough to go over the 254 possible addresses in a 24 bit subnet. Rather than messing with figuring a smaller subnet, at the time the people that did it figured it was just easier to go to 16. Sure, a tremendous number of addresses we would never use, but simplier. Since that was done, the only things I've really done is to just separate certain types of equipment into mentally different sections of the address range. I did not want, nor at the time was I able to, actually create true subnets as at the time we didn't have a router that could route between the subnets. Our entire network has to always see everything else, there is no need here for a logical separation of departments, for example.

That will change, and when it does I will have to subnet smaller. I'll likely shrink the 172.29.x.x subnet that I hope to finish with to a much smaller range, as I'm really only using 10 different ranges in the third octet, and no where near that many addresses. But when that time comes, the devices I'll be adding won't need to be routed to this range, it will likely be IP phones and possibly some medical equipment that should never be part of my normal data network.

Anyway, back to what you had said. If I read you right, I can configure the same vlan-interface - let's say default_vlan_1, with both a primary and secondary address on the switch, and then no matter what port I have a particular device plugged into it will route properly?
Say port A1 has a device with a 199.199.197.1 address, and port A2 has a device with a 172.29.199.50 address. By having both a primary and secondary address on the vlan of the switch itself, with IP routing enabled, these will route correctly? If so, that appears rather easy.
Then I'll just have to make sure the 2910al switches also retain such ability. Although it is my understanding the 2910al switches are not themselves Layer 3 switches, as long as I make sure they can route back to the 8212zl I should be fine. Is that a correct take on your statement?
Again, thank you for your help.

Ron

almost richt.
the 2910's have no l3 functionality needed.
Only the 8212 does L3 function.

I assume all is now configured for a single vlan (the default-vlan vlan-1).
the 2910's have no nowledge if a packet belongs to the 199 or the 172 network,
They just forward both packets based on l2 information (mac-address).

The client sends an ARP request to determine the mac-adress of the destination.
- the ARP-request is forwarded to all ports in the vlan
- the correct host (8212) responds
- the switch fills its mac-adress table
- the client now knows
- subsequent communication is based on mac-adress and is only forwarded to the a single port.

for communication between subnets the client must be configured with a gateway..
her to the 8212 switches ip-adress (prim/sec) in the corresponding subnet.
- the client sends a packet to the gateway
- it uses ARP to determine the mac-adress of the 8212
- the 2910 and the 8212 learn the mac adresses involved and associate them to the ports used.
- The 2910 forwards the packet on L2 to the 8212.
- the 8212 routes the packet (l3) to the other subnet using the same interface!
- her again ARP is used to determine what port to send it out.
- it may be forwarded to the same or another 2810 on L2.
- and sent out to the destination port.

Thanks, Pieter, you are helping more than you know.

For all intents and purposes I do have it configured just as a single vlan. I do have other vlans, but that is only for the MSM765zl controller, and a couple vlans to basically completely separate a public internet that we allow to pass through our network. These vlans are not routed internally at all, they are completely separate.

Now I need to complicate matters a bit.

Currently I have my Sonicwall acting as my router. Therefore, the gateway addresses for both networks are on certain interfaces of the Sonicwall.

My gateway addresses are 199.199.199.254 and 172.29.199.254.

Currently, all my clients in the network have the gateway assignment of the appropriate IP range, so a client with an IP address of 199.19.196.10 has the gateway of 199.199.199.254, and the client with an IP address of 172.29.197.30 has the gateway of 172.29.199.254.

In both cases these go to the Sonicwall router, which is then routing all the internal network. However, the Sonicwall is also our Internet router. So if the request is outside these networks, the request goes to the appropriate place, whether it be to the Internet itself or to one of my VPN's. (we have a few outer offices on VPN, all are simple 192.168.x.x networks, 24 bit)

Now, since my Sonicwall is currently my internal and Internet gateway, and all my clients already have a gateway address assigned to them, I think I would have to find a way to assign the existing gateway addresses to the 8212zl, in order to not have to get to each client to change it. Then I would have to assign a different IP address to my Sonicwall, and have my 8212zl use that address as the Internet gateway, which would also do my VPN's.

First question is, do I have that scenario correct, and second question is, what is the best and quickest way to proceed? I can't have my network internally down hardly at all, other than a few minutes. Externally we can be down a short time on a weekend, but not for too long, a couple hours at most.

Thank you, I believe I am getting a lot figured out from you.

Ron

do your clients use DHCP to get ip-adres/gateway etc?

if so i would suggest :
- to leave the sonicwall as it is.
- configuring new adresses on the 8212.
- on the 8212 configure the sonicwall as gateway
this procedure requires more work, but is more predictable than moving the sonicwalls' address to the 8212 and can migration to the new gateway can be done in phases.

next,
- manually configure a client for the new gatewayadress of the 8212
- test routing using the 8212.

- finally modify the DHCP scope with the new gateway adress
- wait for the lease timers of dhcp to expire
or manually ipconfig/refresh on the clients
or reboot clients
- reconfigure all manual ip-config's (hosts not using dhp).
(no reboots should be necessary).

this can be different steps for 199 and 172 nets.

Yes, Pieter, many of my clients do use dhcp. The dhcp server currently has an address of 199.199.199.247 and distributes dhcp addresses in the 199.199.0.x range. Just the 254 possible addresses for now.

The 172.29.x.x range is all manual at this point. Someday I plan on switching the dhcp to the 172.29 range, but haven't gotten that far yet. While I've moved several printers and clients to the 172.29 manual range, my main servers are still 199.199.x.x, and I'm trying to be cautious.

So, looking at your suggestion, it looks like I should set a different IP on the 8212zl than what the gateway of my Sonicwall is, and set the Sonicwall as the gateway for the 8212, in both IP ranges.
At this point, my clients would still have the gateway of the Sonicwall, which should work exactly as it does now. Then I can test a single client with the 8212 as the gateway for the client.
Then set the dhcp gateway change. Then, as my clients start pulling new dhcp information, or in those cases where I have to manually change the gateway, the transition will be so smooth that no one will notice.
All this time my 172.29 range could remain exactly as it is, except for putting the address on the 8212. Then as I go to each client to change gateway, it just switches from one to the other without any disruption.

Hopefully I restated your suggestion correctly so that I fully understand it. Looks like my next step is to examine the command line, which I struggle with, so I can see about implimenting this.

Thank you for your help. Any further suggestions, including confirmation that I have the thoughts right, are appreciated.

Ron

your description of the steps seems to cover my suggestions.

>>> Someday I plan on switching the dhcp to the 172.29 range <<<
you can configure the server to issue 172.29.x.x adresses allready even if the server itself still has a 199.199.x.x adress!
that may put your migration a giant step ahead?

It's been a while since i've seen this on a multinet, so the configuration for this is not direcly popping up.
steps to look at are :
- At least you must create a second scope.
- you may need to combine both scoped into a "superscope"
- your dhcp-server may need a secondary address
- you may need to configure dhcp-relay/ip-helper on the router.

Superb! I hadn't actually realized I could retain the server IP address yet distribute different range dhcp addresses. I will have to look into that. And you're right, it would definately put me several steps ahead.

Thank you for the confirmation. I'll likely be processing the 8212 changes in the next weekend or so, and I'll let you know the results.
Thank you for all your help.

Ron

http://technet.microsoft.com/en-us/library/dd759168.aspx
example-2 describes a superscope setup

ofcourse you do not use 192.168.1.0 and 192.168.2.0
but your scopes for your own networks 199.199.x.x and 172.29.x.x

Pieter, I've slowly started working on setting the IP addresses in my default_vlan. My original one was 172.29.190.100. I've added 199.199.190.100 as a secondary. For now I have not turned on IP routing, want to make sure I have things correct.
If I show ip it shows both address with correct subnet mask as manual for default_vlan. Currently, my original address of 172.29.190.100 shows No under Proxy and No under ARP. My new address of 199.199.190.100 does not show anything in these columns, and I'm wondering if it is supposed to.
I have added a static ip route of 0.0.0.0/0 172.29.199.254. It is my understanding that my current default gateway only functions when the switch is in Layer 2 mode, and will not work when in Layer 3. Therefore, I am reading that I had to put this static route in before I turne on IP routing.
This static route is supposed to take anything that does not match the two subnets we are planning on routing in the switch and sends them to this address which is one of the LAN addresses on my Sonicwall. I'm wondering if this is the only static entry I need to make.

I just want to be sure I have everything right, so at the moment I have found that as soon as I added the 199.199.190.100 address to the switch, I lost my telnet connection to the switch, and cannot get it back from my current machine. I experimented both with telnet and a browser in the following fashion.
The switch original IP is 172.29.190.100, the IP address of my computer is 199.199.0.8, with a gateway of 199.199.199.254, my sonicwall router in this subnet. If I try to attach to the switch with address 199.199.190.100, it works perfectly.
To further experiment, I used another machine, and set it to a static address of 172.29.63.100, just somewhere in the 172.29.x.x address range. With a gateway of 172.29.199.254, the address of my sonicwall router. If I attach to 172.29.190.100 it connects to the switch perfectly. But does not at all to 199.199.190.100. However, if I change the gateway to 172.29.190.100, the switch address, then I can connect to 199.199.190.100, even though IP routing is not yet enabled. However, I do not then connect to the internet, which I suspect is correct.

My question here, is this supposed to work this way as I currently am without IP routing enabled? I'm just taking it one step at a time. I want to be sure I am correct leading up to turning on IP routing.

thanks for all your help.
Ron

>>> I have added a static ip route of 0.0.0.0/0 172.29.199.254. It is my understanding that my current default gateway only functions when the switch is in Layer 2 mode, and will not work when in Layer 3. Therefore, I am reading that I had to put this static route in before I turn on IP routing <<<

Yes the statement "ip default-gateway" just has no function with "ip routing" enabled.
you must use the "ip route 0.0.0.0 ..." statement instead (to reach the same â default-gatewayâ ).
The default-gateway statement was ment to reach the management ip-adress of the switch from other subnets, not for any routing functionality on other hostst in the network!

==========
>>> I'm wondering if this is the only static entry I need to make. <<<
Yes this time it is the only static needed.
The other two subnets used are â directly connectedâ to the switch with two ip-adresses and the switch will not use a gateway for this.

==========
>>> I just want to be sure I have everything right, so at the moment I have found that as soon as I added the 199.199.190.100 address to the switch, I lost my telnet connection to the switch, and cannot get it back from my current machine. <<<

When adding the 199.199.190.100 address to the switch, the switch and your workstation should be in the same subnet and should be able to communicate directly without any router. Check if both devices use the same subnet mask.
Of course the current session was set up using the router and you must set up a new connection to access the switch directly. Maybe you need to clear the arp-cache of your workstation.
try "tracert

" to check if the direct path to the switch is chosen.

==========
>>> To further experiment, I used another machine, and set it to a static address of 172.29.63.100, just somewhere in the 172.29.x.x address range. With a gateway of 172.29.199.254, the address of my sonicwall router. If I attach to 172.29.190.100 it connects to the switch perfectly. But does not at all to 199.199.190.100. However, if I change the gateway to 172.29.190.100, the switch address, then I can connect to 199.199.190.100, even though IP routing is not yet enabled. However, I do not then connect to the internet, which I suspect is correct.<<<

I assume the sonicwall is not just a router but is a firewall?, you may need to adjust some rules to enable access?

==========
>>> My question here, is this supposed to work this way as I currently am without IP routing enabled? I'm just taking it one step at a time. I want to be sure I am correct leading up to turning on IP routing<<<

Enabling ip routing on the switch only has effect on hosts that have the switch configured as gateway, and for management access to the switch.
The switch will still forward traffic adressed to the sonicwall (as router) on L2.
So this will not interfere with other communication that uses the sonicwall as a router.

regards,
Pieter

Thank you Pieter. I believe that clears up any concerns I had expressed. One more question before I actually turn on IP routing, which I think you have almost answered.

I have a couple other Vlans on this switch, which do not have IP addresses.

One Vlan is the MSM765zl management Vlan, which is how the controller manages all the MSM422 access points. No IP address on this vlan.

Two other vlans I have are meant to direct traffic specifically to one of two paths to the internet, and not touch any of my network at all.

One vlan is tied by ports to all the MSM422 units, for a specific ssid that gets directly to my secondary ISP, an untagged port on the switch for this vlan only. The other vlan ties to the ports and AP's that are in a specific area where we lease the space to another party. This vlan goes to a specific port on my Sonicwall, and then to the internet, it is not routed back to my network, even by my firewall.
These vlans do not have IP addresses.

Do you see any affect of turning IP routing on, with this information, on these other vlans?

If not, then I'll be turning IP routing on very soon.

Thank you for your help.

Ron

If the switch has no interface (ip-adress) in a vlan, it willnot/cannot route traffic from/to other vlans.
Only L2 forwarding within this vlan will be done. Just as this vlan is a separate network.
So a vlan where the switch has no interface will behave as before ip routing is enabled.

Pieter, I have enabled IP routing on my switch. As you said, it didn't affect anything adversely.
However, at this point, it doesn't seem to be working properly. If I take a test client, and give it a static ip within either range, and then the appropriate gateway address, the client can ping anywhere on both subnets. However, it is unable to run or find anything else.

For example, my 8212 main ip address is 172.29.190.100, which is within the 172.29.x.x/16 bit subnet. My second ip address on the 8212 is 199.199.190.100 which is in the 199.199.x.x/16 subnet.

If I set a client for 172.29.63.10, for example, with the gateway of 172.29.190.100, I can ping pretty much anywhere on both subnets.
However, if I want to run an application, or browse, I get nothing. As an example, I have a citrix server on 199.199.50.101. From the above configured client, even though I can ping that server, I cannot connect to citrix. I have a sharepoint services server on 199.199.199.248. Again, I cannot access sharepoint. Even a simple file share on a server in the other subnet is inaccessible.

Also, the internet is inaccessible, even though my default gateway on the switch for vlan 1 points to the ip address of the sonicwall router that is within this same subnet.
That is, my static ip route of 0.0.0.0/0 points to 172.29.199.254, which is my sonicwall router, which should take it to the internet. Get nothing.

So even though I have enabled IP routing, I have not been successful in getting traffic other than ping to cross between the two subnets, nor to the internet.

As you stated, all the clients are still pointing thier gateways to the sonicwall router, so no one is noticing. So if you have a suggestion for getting the two subnets to cross and get to the internet through my sonicwall router, I'd appreciate it.
Thanks for all your help so far.

Ron

Pieter, I've managed to get my test machine to get to the internet, though not the other subnet of my multinetted vlan.
I looked through the docs, and turned on RIP, turned on ICMP. Not sure if that helped.

In looking through the docs, there is a piece that makes me wonder if I shouldn't be adding a couple static routes to the switch. In the docs, it tells us that when we turn on IP routing, that the default gateway of the switch stops working, so we should add a static ip route to the switch.

The only static IP route I added to the switch was 0.0.0.0/0 172.29.199.254, which was to say that anything not on my subnets route to this address on my sonicwall router.
If I show ip route, I currently also see 199.199.0.0 with a gateway of default_vlan, and 172.29.0.0 with a gateway of default_vlan.

Should I make a static route of 172.29.0.0/16 172.29.190.100, and a route of 199.199.0.0/16 199.199.190.100?

These would have the networks point to a gateway of thier own ip address of the vlan. Or would it then suddenly develop an infinite loop and crash the switch?

Not sure what I need to do to have both subnets on the default_vlan see each other totally.

Ron

look at my post Oct 6, 2010 07:12:34 GMT Unassigned

--------------------------------------------------------------------------------
Yes the statement "ip default-gateway" just has no function with "ip routing" enabled.
you must use the "ip route 0.0.0.0 ..." statement instead (to reach the same default-gateway).
The default-gateway statement was ment to reach the management ip-adress of the switch from other subnets, not for any routing functionality on other hostst in the network!
<<<
in the same post :
>>>
The other two subnets used are "directly connected" to the switch with two ip-adresses and the switch will not need/use a gateway for this.

remeber the sonicwall does more than just routing!

what i think happens is:
imagine host-1 (172 with gateway to switch)
and host-2 (199 with gateway to sonicwall)

host-1 sends packets to host-2
it does this by using it's default-gateway (the switch)
the switch forwards packets to host-2
host-2 sends packets back
but does this using the sonicwall as gateway!

the sonicwall does not know there was no session active from host-1 to host-2 (using the switch)
and (as its a firewall) won't forward packets back from host-2 to host-1
=> check logfiles of sonicwall for dropped traffic to confirm this

That makes perfect sense. I'm going to experiment with a second client configured on the other subnet, with the switch as the gateway.
It looks like if I want to migrate slowly, I'll have to figure out how to make the Sonicwall "see" the other traffic as valid, and forward it back.

Thank you for all your help. I'm going to continue on, and see where this takes me.

Ron

Ron,
Good luck with the project
thanks for the points,
Pieter

Pieter, I'm hoping you can help me figure out a related issue in my 8212zl IP routing setup.
Ignore for the moment most of what we talked about with my multinetted default_vlan.

Due to some medical equipment coming in, they required a Layer 3 switch in the mix, which is why I needed to turn IP routing on in the 8212. I had started out trying with my multinet.
However, now with the medical equipment here, we just can't seem to get things to work.
These devices are on thier own vlans. We setup Vlan 26 with an IP address of 192.168.27.1, and Vlan 27 with an IP address of 192.168.28.1. (should have matched up I know, too late when I thought of it)

My assumption was that by having the IP address on the vlan on the 8212, this was then the default gateway for that vlan. Working on Vlan 27, we have a device on a wired port attached to one of my 2910al units. The port on the 2910 directly connected to the device is untagged in vlan 27, and the fiber connection between my 2910 and 8212 is tagged for vlan 27. Note this vlan and ip addressing does not, or should not, touch my sonicwall.
We also have several wireless devices that have to connect to this vlan. So, I have vlan 27 in my msm765 controller, tagged. And also have vlan 27 tagged to the ports on the 2910 that have the msm422 access points attached. The vsc for the controller is set to egress to vlan 27, so that traffic on the wireless that attaches to the correct ssid just drops onto vlan 27.
Our problem is, that we can't seem to ping the devices almost at all. We can occasionally ping the device wired to the untagged port on the 2910, from the 8212. We usually cannot ping from the msm765, and cannot usually ping from another device on the same vlan in the wireless setup. All the devices are static ip addresses.
However, occsionally I can ping from the 8212 to one of the wireless devices, then loose it quickly. The tech with the medical equipment has his laptop here, also with a static ip address for the ssid on vlan 27, and occasionally I can ping him, again from the 8212. I cannot usually ping from the msm765, though it has been successful a couple of times. The laptop is windows, the devices are linux based I believe. I tried puting another address on the vlan in the 2910al (which it turns out can support some ip routing) but that didn't help.

Today I tried using my PCM to trace a route, and it can't seem to find my gateway of 192.168.28.1

Anyway, any ideas, or any other information you need so you can tell me what I am doing wrong?
Thanks.

>>> We setup Vlan 26 with an IP address of 192.168.27.1, and Vlan 27 with an IP address of 192.168.28.1. <<<
On the 8212 I assume?
If yes then the 8212 (with "ip routing") is sufficiently setup to route between the vlans.
The clients in vlan-26 (the 192.168.27.0 network) need to have the 192.168.27.1 configured as default gateway.
evenso for vlan27 -> gw:192.168.28.1

>>> The port on the 2910 directly connected to the device is untagged in vlan 27<<
sounds OK.
>>> and the fiber connection between my 2910 and 8212 is tagged for vlan 27. <<<
also good, but check if this is done both on the 8212 and on the 2910 side of the link?
Maybe you also want to configure the same link tagged for vlan 26 ?

>>>Note this vlan and ip addressing does not, or should not, touch my sonicwall.<<<
This will not be totally avoidable.
If a client adresses an adress outside your local subnets, the switch will redirect it using the default route (ip route 0.0.0.0 ....), wich leads to the sonicwall. As this doesn't know about these nets, it should drop this traffic, bot it DOES reach the sonicwall.


About the MSM785's
I've no experience with the MSM's but compare this to the Cisco WLC (wich i do know).

With cisco the AP's connect to the WLC over a management (v)lan
=> configure the switch-ports of the AP's as untagged in this vlan.
The access-point sets op a sort of "tunnel" over this management vlan to the WLC. The WLC then drops the packets on a data-vlan.
So your vlan between AP and controller is a different one than from controller to destination-hosts.

I need to look deeper into msm doc's to give better advice.

try these doc's
http://h10144.www1.hp.com/docs/myprocurve/MSMDesignGuide_May_09_WW_Eng_ltr.pdf
http://h10144.www1.hp.com/docs/myprocurve/MSMImplementationGuide_May_09_WW_Eng_ltr.pdf

Pieter, thank you. You confirmed that my setup of the vlans appeared correct, so I switched my focus to the VSC on the service controller. That was the ticket. I threw a few rocks at it, adjusting some settings I didn't think I needed, and then everything came to life. The two vlans are properly routing between each other, and now that I have the VSC correct, it routes across the wireless correctly too.

As a last test of the concepts, I put a printer on my older default_vlan of 199.199.x.x with the address of 199.199.199.3. However, I put the gateway of the printer as 199.199.190.100, the IP address of this range on the 8212. Then I connected wirelessly in this same subnet with that gateway and setup the printer on the laptop, and it worked correctly.

Then I switched the laptop to an ip address in the vlan 27 range, 192.168.28.35, with the gateway of 192.168.28.1.

With this setup on my laptop, and the setup on the printer, I was able to access and print to the printer, showing the routing enabled correctly from the 192.168.28.x subnet to the 199.199.x.x subnet.

So proof of the concept as you described. Excellent.

Now I need to ask if we can think outside the box.

As you've observed from previous notes, my original ip ranges were 199.199.x.x with a gateway of 199.199.199.254 (sonicwall) and 172.29.x.x with a gateway of 172.29.199.254 (sonicwall).

What I need to find a way to do, is to be able to route between the 8212 and the sonicwall in such a way that the devices on the network can still route across from one subnet to the other while I am in the process of trying to change them.

For example, let's say I have a printer on 172.29.196.25 with a gateway of 172.29.199.254. And I have a computer on 199.199.0.138 with a gateway of 199.199.199.254.
Currently, print commands from the computer get routed through the sonicwall.

What I need to find a way to do, is to let's say change the gateway of the printer (172.19.196.25) to 172.29.190.100(8212). Yet I can't change the gateway of the computer (199.199.0.138) from the current gateway of the sonicwall yet.

Is there any way that I can get the 8212 and/or sonicwall to see this traffic and route it back to the correct location? At this point, the traffic from the computer would travel to the sonicwall. The sonicwall might know the ip range of the printer, but it doesn't know how to get to the printers new gateway.

Hope that made sense.

Thank you very much for all your help. I think the ratings here are a bit skewed, you deserve a lot higher rank than wizard.