• Post Reply Bookmark Topic Watch Topic
  • New Topic

Wierd networking problems  RSS feed

 
Sheriff
Posts: 6920
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm currently tearing my hair out over a wierd networking problem with a Linux box. The symptoms are that after an indeterminate time (typically a few hours) it starts to ignore all network traffic (no ping, arp, dns ...). After a reboot (Ugh!) it starts listening again, but then "as soon as I stop watching" it stops listening.
As far as I am aware the machine is still running OK, although as it is colocated a few hundred miles away I haven't been able to go and check this out yet. I don't think it's a hardware problem, as the same box ran for weeks here when I was soak-testing it. There are, however, two main differences between the software configurations here and at the colo facility:
1. Here it was listening only to one IP (172.16.1.21) on eth0, there it is listening to 8 (213.131.169.104 on eth0, 23.131.169.105 on eth0:0 ... 213.131.169.111 on eth0:6)
2. Here it was being handled by my local DNS box, there it is running djbdns listening to 213.131.169.110 and 213.131.169.111 and providing DNS for several domains which it hosts.
When it's "working" it happily listens to all the addresses and provides DNS info correctly.
Oh, and it's also getting quite a lot of "code red" hits (typically 1-200 per day) and other internet traffic but it was nestled behind my firewall here.
Nothing seems to be gobbling up memory, processes or anything. I had a "top" running when it stopped last time, and there was nothing unusual in the process table, and only a loading of 0.1.
I will probably take a trip to the colo facility some time next week, but in the meanwhile I would really appreciate any help, suggestions or pointers to appropriate research material.
Many thanks.
 
Ranch Hand
Posts: 919
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
these are fairly obvious things but i thought i'd mention them anyway.
if you attach a modem to a serial line and put a tty on it you should be able to log in remotely, at least that might help avoid travelling so far to check it out. Although I'd understand if you viewed that as a potential security problem. if there is anyone on the colo site, you could ask them to plug in the modem to the phone line whenever you needed to use it, then unplug it afterwards.
the first thing that occurred to me is that it may be the target for some sort of dos attack. do you get a bugtraq feed, i used to find bugtraq quite helpful for problems. Try geek-girl for a starting point on that. Also is there anything untoward indicated in the system (or other) logs?
 
George Brown
Ranch Hand
Posts: 919
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
and back on the paranoid angle, do you have an IDS installed? I use 'snort' under linux, it seems to work ok (so far).
 
Frank Carver
Sheriff
Posts: 6920
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the reply,
I don't think adding a dial-up is a possibility, although I'll enquire. Sure would make things easier.
I'm still exploring to find if there is anything ususual in the logs. The reboot process is a bit cumbersome so I've only brought it back up twice. The first time I thought it was a glitch, and the second time I was still researching things when it turned up its toes. My priority at the time was to rescue live application data so I could put it onto a backup machine.
I want to be well prepared with things to look for/download in the short time I may have when I bring it up next time.
 
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Does it only quit working when your computer is idle? Or does it stop working purely at random?
 
George Brown
Ranch Hand
Posts: 919
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I was just wondering whether it may be some eth-driver problem.
man syslog.conf, try to set more debuging info... and you may catch the problem there.
It does still let you log in as normal via the console after the network activity has frozen doesn't it?
 
Frank Carver
Sheriff
Posts: 6920
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
All good suggestions, thanks.
I don't know if the machine is still running enough to access through the console or a serial port after this has happened. I'm a couple hundred miles from the box (it's in a commercial co-lo facility) and haven't had the chance to get to examine it yet!
I plan to go over there sometime this week, and I'll let you all know how I get on. Thanks again.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!