System very slow after replacing modem/router

[ILoveC4]

So, I have all my stuff set on static IP addresses.

DHCP range is .100 - .150.

All my static IP addresses are between .200 and .254.

Should I really spend the time to take all my devices and log the MAC address, reconfigure them to DHCP, and then create the IP reservation in the DHCP server based on MAC address? Using static IP's has worked great for YEARS...

[pstuart]

If its working, and it sounds like it is, no worries...

If I was setting up new, I would never assign static IPs and just reserve DHCP by MAC. That way I have one central spot managing all my IP devices.

The other problem with statics, is tracing down an IP conflict is hard and manual. (that's what she said)

It is way too easy to assign a new device a static IP already in place.

Also, if a device goes down, and it needs replacing you need to know what its IP was, and have to make sure the replacement gets same IP.

With DHCP reservations, you just change the MAC to the replacement device.

Plus, what happens when you run out of IPs and have to extend your subnet? Granted most homes don't have 254 IP devices, but eventually this will happen when every light switch, light bulb, outlet and appliance has an IP...

Here is a perfect example of why DHCP reservations beats statics...

My new work VPN is the same C class subnet as my house, so when I connected to the office, I couldn't route local traffic. I changed my DHCP server and network to serve up new IP range and a quick reboot of core devices and I was on a completely new network schema and now I can route both sets of traffic properly. With Static IPs, I would have to change all devices to new subnet.

Anyway, I'll get off my soapbox, but I stand by DHCP reservations beats Static IPs.

[ILoveC4]

If its working, and it sounds like it is, no worries...

If I was setting up new, I would never assign static IPs and just reserve DHCP by MAC. That way I have one central spot managing all my IP devices.

The other problem with statics, is tracing down an IP conflict is hard and manual. (that's what she said)

It is way too easy to assign a new device a static IP already in place.

Also, if a device goes down, and it needs replacing you need to know what its IP was, and have to make sure the replacement gets same IP.

With DHCP reservations, you just change the MAC to the replacement device.

Plus, what happens when you run out of IPs and have to extend your subnet? Granted most homes don't have 254 IP devices, but eventually this will happen when every light switch, light bulb, outlet and appliance has an IP...

Here is a perfect example of why DHCP reservations beats statics...

My new work VPN is the same C class subnet as my house, so when I connected to the office, I couldn't route local traffic. I changed my DHCP server and network to serve up new IP range and a quick reboot of core devices and I was on a completely new network schema and now I can route both sets of traffic properly. With Static IPs, I would have to change all devices to new subnet.

Anyway, I'll get off my soapbox, but I stand by DHCP reservations beats Static IPs.

It's good info, and I appreciate it.

[dentalben]

All good info. There's obviously a debate on DHCP vs. Static. My original install had the C4 components on a DHCP. When I upgraded the controller, I put everything on Static. Then I upgraded again (I've been with C4 since 2006), and went back to DHCP. I have a few items on my network that require a static IP (printer, security cam server, Dune player) and I've put those outside of the DHCP range of .2-.100. Everything seems to point to the new router/modem but there are a few other issues that make me wonder if my project is messed up. If I check performance on Detective Suite, my main controller is running ~96% CPU all the time, with almost a 3 on the load (very high). My director controller is minimal. See image below. What does everyone else's performance look like on their controller(s)?

For kicks, I'm attaching a Director log of about 40 seconds when I turned a TV on, played with the volume and then shut it off. What's odd to me, is:

Why does the system send/receive packets to my Onkyo receiver so frequently when it's not even involved in the above scene (it wasn't even turned on at the time)?

Is it normal for the system to check the status of so many other components when changing a source?

[pstuart]

A process overview would be best... Some process is stuck open, it is not normal for the cpu to be pegged at 99%

IP drivers have preset "check in" intervals to keep alive the connection, which would explain the freq check ins...

Ultimately, something is stuck, need to get a list of running processes to see what's stuck, probably restart that service, ultimately get at what's causing the issue is another matter.

[thecodeman]

Yeah my 300 idles at .60-.80 load, something is up in your project.

[pstuart]

Just out of curiousity... Do you have the appletv driver installed?

[dentalben]

Actually yes and no. I did not get the Apple TV driver from extravegetables but just created a new driver with simple connections, renamed it, and added it to my project. I'm not controlling it via my C4 remote. I just needed a way to select it as a video source. I could create a macro on the remote to have the TV change inputs, but haven't gotten to that point. I'm just trying to figure out how to avoid being roasted by my family when the system drags to a halt.

I've looked through my programming and project (Detective Suite was very helpful with that) and found a number of remnant code that was causing some loops and unnecessary stuff, but I still can't figure out why the load is still so high. I even went through the xml file for my project and cleaned out a few cobwebs, but still, it's running at full throttle for some reason.

My next step is to see if C4 support can go through my project to see what process(es) is causing this to happen.

[pstuart]

The "best" way is to get into the controller and get a "top" report of running processes. NOt sure if there is another other way via the software to do this.

The unofficial way is to SSH into the controller, log in via the root user and the standard password, unless your dealer changed it...

Then you can see what processes are causing the CPU to peg. Chances are its something simple. But as to what's causing it, that can be complex.

A reload of the software might solve the issue, if you feel your project is cleaned up... However, I suspect its probably a stuck process that is causing the issue. Just need to figure out which process.

Ultimately, I had a similar problem running my project off a 300. Turns out the 300 is just not powerful enough to run any multi-room project with 2 way / IP drivers, IMO. Upgrading to a 1000 solved all my issues. Now, I'd go with that 850 instead.

Here's a thought... If you have composer pro, try backing up your project, clearing the project on the 300 and rebooting the controller and seeing if the CPU is still pegged with a clean project. If it is, then restore your project and its not your project, but the software install that is causing the issue.

I've seen the OpenVPN process hang, especially if anything is blocking its connection to Control4. Usually restarting it solves it, but sometimes you have to reload it, and that means reloading the whole os. Of course, if something environmental, ie router config, firewall, etc. is causing the issue, you need to solve that first.

Good luck, it isn't easy... And you just might be fighting an underpowered system in the long run...

[thecodeman]

Something that could be locking it up is an incorrect DNS entry. Try changing it to 8.8.8.8

[dentalben]

I've checked and re-checked the DNS settings on multiple computers and it all "seems" to be correct; especially since each device should get that info from the DHCP controller. I've rebooted everything more times than I care to admit and it all starts out working fine, but slowly starts to drag...

I'm using PuTTY to SSH into my controller, but I haven't checked the processes, yet. Any idea what command line I can use to check the processes?

I've done a complete wipe and restore before and there are only a few things I like doing worse. Unless C4 corrected the restore process, it's not something I take lightly. I've considered jumping to an 800 controller, but fear that a jump in performance will only mask the true problem. I may try going to 2.2.1 just to see if that corrects any issues with the software. If not, then it might still be a project problem.

[xc420]

^yes, pstuart already mentioned it. Type in 'top' and hit [enter].

[dentalben]

I've attached a SS of the ^top command from last week when it all started (Putty SS) and one from today (Putty SS2). Does anyone see anything strange on those lists?

[pstuart]

Think it's what I suspected. The director process is busy doing something, but is not pegging the system at 99%.

Your Proc usage is 3-4 times a 1 proc system... This is major load for a linux system and is not ideal.

Something in your project is causing heavy load. I don't think replacing the modem / router is the cause, personally.

At this point, I'm not sure what else to recommend, ultimately, rebuilding the project, room by room and see which driver is causing the issue, if any would be my next step.

Since director is the process that interacts with drivers, its most likely a driver issue. But I did see in your first screenshot that Zserver was under pretty heavy load too, that's your mesh network for your zigbee devices.

Personally, I think you just have too little hardware for your project. Try to simplify it and see if the load goes down and responsiveness returns.

Start with any 2 way drivers, IP drivers, etc. Keep watching TOP and see if anything adds load. Through trial and error you will get to the source of the problem.

Also, just to rule out networking causing the issue, can you run a "ifconfig" and post a screen shot of that as well?

Also, another thing to run is "netstat -an" but the output will be long, so you may have to output that to a text files "> output.txt"

Ultimately, the controllers are just embedded linux type boxes... Basically you are now just troubleshooting a linux box and seeing what the issue might be...

[dentalben]

I've attached a SS of Ifconfig and the text from netstat, if any of that makes much sense to anyone. Just for kicks, here's a listing of everything tied into my project (I have one of those load calculator worsheets somewhere, but can't find it at the moment...):

HC300 primary

HC200B

IO Extender

C4 Amp

C4 XM Radio

Sony DVD Changer

Dune Media Player

Onkyo Receiver

DISH Receivers x 2

Apple TV

NAS

7" IE TS x 4

Shinybow 8x8 A/V matrix switch

PS3 x 2 (source, not controlled)

Wii (source, not controlled)

Televisions x 5

Projector

Motorized screen

SR250 Remotes x 6

Security system

Card Access mini remote

Motion sensors x 4 (not tied into security system)

Contact sensors x 4

Light switches, 2, 3 and 6 button keypads x 35

Wireless outlets x 3

Thermostats x 2

iPad controller

iPhone controller

Fireplace relay

Security camera (source, not controlled)

That about sums it up.

[ILoveC4]

I've attached a SS of Ifconfig and the text from netstat, if any of that makes much sense to anyone. Just for kicks, here's a listing of everything tied into my project (I have one of those load calculator worsheets somewhere, but can't find it at the moment...):

HC300 primary

HC200B

IO Extender

C4 Amp

C4 XM Radio

Sony DVD Changer

Dune Media Player

Onkyo Receiver

DISH Receivers x 2

Apple TV

NAS

7" IE TS x 4

Shinybow 8x8 A/V matrix switch

PS3 x 2 (source, not controlled)

Wii (source, not controlled)

Televisions x 5

Projector

Motorized screen

SR250 Remotes x 6

Security system

Card Access mini remote

Motion sensors x 4 (not tied into security system)

Contact sensors x 4

Light switches, 2, 3 and 6 button keypads x 35

Wireless outlets x 3

Thermostats x 2

iPad controller

iPhone controller

Fireplace relay

Security camera (source, not controlled)

That about sums it up.

I can't say whether it will certainly solve your problem or not, but at the very least...buy one of these dirt cheap HC1000's that are kicking around. If I were you I'd do that or buy an HC800 (more money, better option). You need more processing power than that HC300 can provide.

[thecodeman]

It wont help much, but I'd move director to the 200b since it's got a slightly spec-bumped processor. Also if services aren't split up (navigator, director, zigbee all on one box) that will kill your resources quick.

[pstuart]

Yeah, basically, there are a ton of active network connections, most likely issued from director...

Your system is under powered, IMO...

You have 64 active connections to local devices, 10 to external IPs.

Of the local devices, 12 connections are established to 192.168.0.28 and 10 connections are established with 192.168.0.9, 192.168.0.12 and 192.168.0.73, what ever those are, try pulling them out of the project and see if things improve.

There are also 48 local "loopback" 127.0.0.1 ports open and active which can be director drivers waiting for incoming data... This number "should" come down when you remove those 4 IP devices from your project.

Other then that, everything else looks normal, the load is most likely network traffic related and a proc that is 4x over taxed.

[dentalben]

Thanks to everyone for their input. This whole process has helped me clean out some cobwebs in my project, programming, etc. I ran the 2.2.1 update last night and things are working normally this morning and the cpu load is bouncing between 50-80%. The load is still high, so I'm going to give it a day or two to see if it starts slowing down again, but ultimately I'm going to add an 800 to the project as the main controller. I might keep the 300 as an additional ZAP server.

[CFUG]

^And as already mentioned, even the lowly 1000 will get you where you need to be at half the cost.

[ILoveC4]

Yeah...I just logged in and have been watching the Performance Tab under Detective Suite. I have what I feel is a fairly large project. It includes 8 video sources (some IR, some IP, some RS232), 8 audio sources, 7 displays, various touchpanels, IP cameras, 2 C4 AMPS, 2 C4 tuners, about 140 zigbee devices, etc...

I have HC1000V3 running director, HC800 running zigbee and navigator, and HC300C running ZAP and allowing IR control of the system (Navigator enabled for Slingbox). My family is home, and based the time of day I would say there is likely music playing in a few zones while my wife gets ready, and my kids are watching TV somewhere. I haven't seen the HC800 CPU exceed 1% CPU. The HC1000V3 has only gone over 3% a couple of times. What's interesting is the touchpanels bounce between 5% and 25%, but usually around 8% to 10%...they're all running photo screensaver so I'd imagine the photo transitions are what is taxing those.

I suspected mine would be higher because I have a lot of timers running, variables changing constantly, etc....

You just need a faster controller...that OG HC300 is pretty weak for 2.0+ in general, much less a sizeable project.

[dentalben]

The nice thing about an 800, is it only takes up 1U space in a rack, which is exactly how much free space I have.

[ILoveC4]

The nice thing about an 800, is it only takes up 1U space in a rack, which is exactly how much free space I have.

True. It isn't very deep either.

[dentalben]

*Update* I installed an HC800, made it director (migrated my project to the new controller) and added it to ZAP (not the Zigbee server though), and removed the 200 from my project. I have to say that I am EXTREMELY happy with the responsiveness of the system.

Once again, my family (read: wife) is happy with Control4.

[CFUG]

It's amazing what $1000 will do...